Slurm Queueing System Guide

Basic Slurm Terminology

• Basic Slurm Terminology
  • Partition (a.k.a. Queue)
  • Node
  • Job
  • Job Steps

Job Types and Job Submission

• Interactive Jobs
  • 1. Submitting interactive jobs with salloc
  • 2. Submitting interactive jobs with srun
  • Interactive Job Example
  • Keeping an interactive job alive
• Batch Jobs
  • Submitting batch jobs
  • Batch Job Example
• Job Arrays
  • How Job Arrays Work
  • Array Syntax Options
  • Example Job Array Script
• Job Dependencies
  • What Is a Job Dependency?
  • Basic Syntax
  • Common Dependency Types
  • Example: Simple Sequential Workflow
  • Example: Multiple Dependencies
  • Example: Using Job Arrays with Dependencies
  • Tips

Job Environment

• Slurm environmental variables

Strategies When Submitting Slurm Jobs

Due to LARCC’s resource policies (see Resource restrictions), users are limited to:

• 20 concurrent job submissions.

• 2 nodes maximum per user.

Depending on your resource requests, multiple jobs may run on the same node, but the 20-job submission limit must always be respected. To maximize resource utilization while complying with these restrictions, there are multiple strategies you can follow. This section explains the most common ones.

• Single Job with Multiple Job Steps
• Job Arrays
• Requesting CPU Nodes with Multiple Parallel Tasks
  • Case 1: Threaded Applications (e.g., OpenMP, TBB, MPI threads)
  • Case 2: Multi-Process Applications (Single Node)
  • Case 3: Hybrid Parallelism (Multiple Processes, Each with Threads)
  • Case 4: Multi-Node Jobs
  • Additional Options
• Single GPU node with one task per GPU and CPU cores evenly distributed across tasks