Soil health is essential for infrastructure and plant growth. The UNSAT project aims to improve 3D soil image processing using machine learning and high-fidelity simulation data to study water transport and root-soil interactions during maize growth.
Welcome to the UNSAT documentation! This guide provides a comprehensive overview of the UNSAT project, an AI analysis tool designed for rooted soil analysis. It will walk you through installation, usage, and other key aspects, ensuring you can leverage UNSAT effectively.
UNSAT is a Python-based tool tailored for analyzing and modeling rooted soils using AI techniques. It utilizes advanced configurations and integrates with tools like Weights and Biases (wandb) for experiment tracking and model evaluation. This documentation is divided into several sections to help you navigate and use UNSAT with ease.
UNSAT is a specialized tool for analyzing soil structures, particularly focusing on rooted soils using machine learning and AI methodologies. The tool is designed to be flexible, allowing for a variety of configurations and setups to match different experimental needs. It integrates seamlessly with Weights and Biases for tracking experiments, and it is optimized to run efficiently on both local and remote environments, including HPC systems like Snellius.
Poetry is a dependency management tool that helps you manage your Python environments and dependencies efficiently. We highly recommend using Poetry to set up UNSAT. Follow these steps to get started:
Running on Snellius? Read first the subsection below
Clone the Repository:
First, clone the UNSAT repository to your local machine:
git clone https://github.com/UNSAT3D/unsat.git
Navigate to the Project Folder:
Move into the cloned repository directory:
cd unsat
Install Poetry:
If Poetry is not already installed on your system, you can install it using pip:
pip install poetry
Install Dependencies:
Run the following command to install all required dependencies:
poetry install
Activate the Environment:
To work within the UNSAT environment, activate the virtual environment created by Poetry:
poetry shell
Alternatively, you can run commands within this environment by prefixing them with poetry run
.
For more details on Poetry, visit the Poetry Documentation.
If you're working on the Snellius supercomputer, you'll need to load specific modules before installation. Follow these steps:
Load Required Modules:
module load 2023
module load Python/3.11.3-GCCcore-12.3.0
Install UNSAT: After loading the modules, follow the general installation steps as described in the Poetry Installation Guide.
For detailed instructions on submitting jobs on Snellius, refer to the Snellius Usage Section.
Once you've installed UNSAT, you're ready to start running experiments. This section will guide you through the process.
To run a basic experiment with UNSAT, navigate to the project directory and execute the following command:
poetry run python unsat/main.py fit -c configs/test_config.yaml --data.hdf5_path <path to data>
This command will start a short training session using the specified configuration file and data path. The results will be automatically uploaded to Weights and Biases. To understand and tailor your model configuration refer to this page of the manual of unsat.
Weights and Biases (wandb) is a powerful tool for tracking machine learning experiments. To set up wandb with UNSAT:
Login to wandb:
poetry run wandb login
Enter Your API Key:
You can find your API key here. Paste it when prompted.
For more detailed guidance, visit the Weights and Biases Quickstart Guide.
UNSAT is highly configurable, allowing you to tailor the system's behavior to your specific needs. All configurations are managed via YAML files, which can be easily edited or overridden. To understand and tailor your model configuration refer to this page of the manual of unsat.
You can override default configurations by passing additional config files or command-line arguments. For example, to specify a different profiler, you can run:
poetry run python unsat/main.py fit -c configs/test_config.yaml --trainer.profiler pytorch
Profiling your runs can help you optimize performance. UNSAT supports multiple profiling tools and configurations. To enable a predefined profiler, use the following command:
poetry run python unsat/main.py fit -c configs/profiler.yaml
You can mix and match configurations as needed to achieve the desired results.
To run experiments on Snellius, use the provided SLURM script. Here's how:
Submit a Job: From the top-level project directory, execute:
sbatch scripts/run.slurm configs/test_config.yaml
Check Outputs: After the job completes, various outputs will be generated:
For more advanced usage and troubleshooting on Snellius, consult the Snellius Documentation.
We welcome contributions from the community! To maintain code quality and consistency, please adhere to the following guidelines.
Our project uses a linter to enforce code quality. The linter runs automatically on each commit to GitHub. If you'd like to run the linter locally:
Activate Your Environment:
poetry shell
Install Pre-commit:
pre-commit install
The linter will automatically run after each commit, and it will suggest or apply fixes where necessary. Note that you may need to stage and commit the changes again after the linter has made adjustments.
Please follow our Contribution Guidelines when submitting pull requests. Ensure your code is well-documented, tested, and adheres to the project's coding standards.