Python package with the goal to split datasets (e.g. words defined by several variables) into subsets that are as comparable as possible.
Discuit is a Dynamic item set clustering UI tool (with the UI part yet to come)
Discuit can split datasets (e.g. words defined by several variables) into subsets that are as comparable as possible.
The package takes a csv file as input and generates a defined number of matched sets for a given number of continuous and categorical variables. One of the categorical variables can be selected to be split absolutely even across sets. Discuit generates the following output:
In the future, this will be integrated in an GUI.
The project setup is documented in project_setup.md.
Discuit is available on PyPI!
pip install discuit
Or install the latest development version directly from GitHub:
git clone firstname.lastname@example.org:doerte/discuit-project.git
python3 -m pip install .
In the terminal, run Discuit with the following command:
python3 run_discuit.py "name of input file" [number of desired sets] --columns l/c/n/a/d --runs [desired number of runs]
Example: python3 discuit/run_discuit.py example/input.csv 2 --columns l a n c n d --runs 3
This will run Discuit with the provided testfile and create 2 subsets. The columns in the file are identified as "label", "categorical", "numerical", "absolute", "numerical" and "disregard" (in that order). The program will run 3 times (and create 3 output files).
The input file needs to be a .csv file with a first line containing headings followed by rows that represent the different items. Each column specifies one variable.
When launching the script, please specify per column what kind a data the script should expect:
The package will try maximally 20 times to come-up with a good split. If it doesn't it will give up and output it's last try. You can always run it again. Often it will succeed eventually. If not, consider dropping variables.
If you run the script without specifying --columns, you will be asked what you want per column. If you don't specify the desired number of runs, it will generate 1 output file.
If you choose an "absolute split variable", this variable cannot have missing data. The program will exit if it does.
For categorical variables, a dummy category is created that holds the items with missing data. For numerical variables,
missing data is replaced with the average of this variable.
If you prefer a different approach, please prepare your input file in a way that does not include missing data.
If you want to contribute to the development of Discuit,
have a look at the contribution guidelines.