kunefe

A Python package that helps users run containerized applications on HPC systems

1
contributor
217 commits | Last commit 7 months ago

Cite this software

What kunefe can do for you

Kunefe

Kunefe is a Python package that helps users run containerized applications (Docker) on HPC systems. It can:

  • convert an existing Docker image into an Apptainer image
  • connect to a remote host via SSH and run shell commands
  • generate scripts for batch jobs
  • submit generated job scripts to HPC batch queues (SLURM)
  • copy files and folders to a remote system
  • retrieve files and folders from a remote system
  • install Apptainer on a remote system
  • monitor the job queue (SLURM)

Requirements

In order to run your application with kunefe, you will need your application to be containerized. If your application is not containerized, you can follow this guide to learn how to do that.

The host system requirements (e.g. your laptop or desktop computer):

Remote system requirements (e.g. HPC cluster):

Note: If Apptainer is not available on the remote system, it can be installed using kunefe which will require the following packages to be installed on the remote system: curl rpm2cpio cpio

Badges

Tests Examples Documentation Status PyPI pyversions Checked with mypy pre-commit

fair-software.eu recommendations
howfairisfair-software badge
(1/5) code repositorygithub repo badge
(2/5) licensegithub license badge
(3/5) community registryRSD workflow pypi badge
(4/5) citationDOI
(5/5) checklistOpenSSF Best Practices
Code quality
Static analysisworkflow scq badge Code Smells Duplicated Lines (%) Reliability Rating Technical Debt Maintainability Rating Bugs Security Rating Vulnerabilities
Coverageworkflow scc badge
GitHub Actions 
Buildbuild
Howfairisbuild
Citation data consistencycffconvert
SonarCloudsonarcloud
MarkDown link checkermarkdown-link-check

Installation

To install the latest kunefe release from PyPI, run:

python -m pip install kunefe

To install the development version of kunefe from GitHub repository, do:

python -m pip install git+https://github.com/mess-nlesc/kunefe.git@main

Examples

See the examples in examples folder.

Documentation

See https://readthedocs.org/projects/kunefe/ for the code documentation.

The project setup

The project setup is documented in project_setup.md. Feel free to remove this document (and/or the link to this document) if you don't need it.

Contributing

If you want to contribute to the development of kunefe, have a look at the contribution guidelines.

Credits

This package was created with Cookiecutter and the NLeSC/python-template.

Keywords
Programming languages
  • Python 95%
  • Shell 4%
License
</>Source code
Packages
pypi.org

Participating organisations

Netherlands eScience Center

Contributors

Faruk Diblen
Faruk Diblen

Related projects

MESS

Modelling Emerging Societal Systems in Mesopotamia

Updated 2 months ago
Finished

Related software

Cerise

CE

Cerise is a service for running CWL workflows on HPC resources. You add software, tool descriptions, and connection information, then your user runs workflows whenever they want.

Updated 5 months ago
1

cerulean

CE

A Python 3 library for talking to HPC clusters and supercomputers.

Updated 28 months ago
1

PyXenon

PY

Python wrapper for the Xenon programming interface to various compute and storage resources.

Updated 28 months ago
3

Xenon

XE

If you are using remote machines to do your computations, and don’t feel like learning and implementing many different APIs, Xenon is the tool for you.

Updated 13 months ago
13 11