Job Generation

class slurmjobs.Singularity(command, overlay=None, sif=None, *a, **kw)

Generate jobs for sbatch + singularity. Use this for HPC Greene.

This is functionally equivalent to:

{# sbatch_args #}

# execute it inside the container
singularity exec --overlay ... ... /bin/bash << EOF

. /ext3/env.sh

{# shell_code_body #}

EOF

To setup a singularity overlay with a anaconda python environment, please feel free to use https://gist.github.com/beasteers/84cf1eb2e5cc7bc4cb2429ef2fe5209e I’m in the process of integrating it with slurmjobs.

Source Tutorial: https://sites.google.com/a/nyu.edu/nyu-hpc/services/Training-and-Workshops/tutorials/singularity-on-greene

class slurmjobs.Slurm(*a, sbatch=None, modules=None, n_gpus=None, n_cpus=None, nv=None, **kw)

Generate jobs for sbatch. This was used for HPC Prince. For HPC Greene, please use Singularity.

This is functionally equivalent to:

# sbatch_args
#SBATCH --nodes=...
#SBATCH --cpus-per-task=...
#SBATCH --gres=gpu:...
#SBATCH --mail-user=...
...

{# shell_code_body #}

class slurmjobs.Shell(command, name=None, cli=None, root_dir=None, backup=True, job_id=True, template=None, run_template=None, **options)

Generate batch jobs to be run without a task scheduler. This will create a grid of scripts that will be run using nohup, a program that will keep running your script even if the parent process (e.g. your ssh session) dies.

This is mainly just for doing small tests and whatnot.

import slurmjobs

jobs = Shell('python myscript.py')
jobs.generate([
    ('a', [1, 2]),
    ('b', [1, 2]),
])

class slurmjobs.Jobs(command, name=None, cli=None, root_dir=None, backup=True, job_id=True, template=None, run_template=None, **options)

The base class for Job generation. Sub-class this if you want to provide your own

import slurmjobs

batch = slurmjobs.SlurmBatch(
    'python train.py',
    email='mynetid@nyu.edu',
    conda_env='my_env')

# generate jobs across parameter grid
run_script, job_paths = batch.generate([
    ('kernel_size', [2, 3, 5]),
    ('nb_stacks', [1, 2]),
    ('lr', [1e-4, 1e-3]),
], receptive_field=6)

We use Jinja2 for our script templating. See https://jinja.palletsprojects.com/en/3.0.x/templates/ for documentation about its syntax and structure.

options

Template options. Anything here is made available to the template.

Type

dict

template

The Jinja2 template for the sbatch script.

Type

str

job_template

The Jinja2 template for the jobs script that launches the sbatch files.

Type

str

cli

The command line argument format your script uses. By default, this uses Fire. See args.FireArgument.

Type

str, slurmjobs.args.Argument

job_id_arg

The argument name to use for passing the job ID. Set to None to ignore omit the job ID.

Type

str

job_id_key_sep

The separator between key and value used in the job ID.

Type

str

job_id_item_sep

The separator between key-value items in the job ID.

Type

str

allowed_job_id_chars

Characters (other than alphanumeric) allowed in the job ID. Other characters are removed.

Type

str, list

special_character_replacement

A string that can be used in place of special characters not found in allowed_job_id_chars. Default is empty.

Type

str

key_abbreviations

A mapping from full name to abbreviation for keys in the job ID.

Type

dict

abbreviate_length

The length to abbreviate keys in the job ID.

Type

int

float_precision

The number of decimals to limit float values in the job ID.

Type

int

format_id_item(k, v)

Formats a key-value pair for the job ID.

You can override this to change how each key-value pair in a job ID is formatted.

Parameters

• k (str) – The argument key.

• value (any) – The argument value.

Returns

Return a tuple if you want it to be joined using job_id_key_sep. Return a string to be used as is. Return None to omit it from the job ID.

Be warned that omitting key/value pairs runs the risk of filename collisions which will mean multiple jobs overwritting each other in the same file.

format_job_id(args, keys=None, name=None, ignore_keys=None)

Convert a dictionary to a job ID.

Parameters

• args (GridItem) – The job dictionary.

• keys (list, tuple) – The keys that we want to include in the job ID.

• name (str) – The job name to include in the job ID. By default this will be the name associated with the Jobs object. Pass False to use no name.

generate(grid_=None, *a, ignore_job_id_keys=None, **kw)

Generate slurm jobs for every combination of parameters.

Parameters

• grid (Grid, list) – The parameter grid.

• *a – Positional arguments to add to the command.

• **kw – Additional keyword arguments to pass to the command.

Returns

str: The path to the run script.

list[str]: The list of paths for each job file.

Return type

tuple

generate_job(job_id, *a, _args=None, _grid=None, **params)

Generate a single slurm job file

generate_run_script(_job_paths, _grid=None, **kw)

Generate a job run script that will submit all jobs.