cerulean

• Python 3 library for talking to HPC clusters and supercomputers
• Copy files to and from remote machines
• Start processes locally or through SSH
• Manage jobs on a local or remote scheduler
• Python 3 / SSH / SFTP / Slurm / Torque

Cerulean is a Python 3 library for talking to HPC clusters and supercomputers. It lets you copy files between local and SFTP filesystems, it lets you start processes locally and remotely via SSH, and it lets you submit jobs to schedulers such as Slurm and Torque/PBS. The file access functions of Cerulean use a pathlib-like API, but unlike in pathlib, Cerulean supports remote file systems. That means that there is no longer just the local file system, but multiple file systems, and that Path objects have a particular file system that they are on. On High-Performance Computing machines, you don’t run commands directly. Instead, you submit batch jobs to a scheduler, which will place them in a queue, and run them when everyone else in line before you is done. With Cerulean, you can submit jobs to a scheduler and track their progress, using a simple Python API.