BlueStreak uses the Simple Linux Utility for Resource Management (SLURM) software for job queueing and scheduling system. Users must submit a walltime limit in their jobs. Otherwise, a default of 30 seconds will be used.

Partitions

BlueStreak users can submit their jobs to one of four partitions (queues) with different time and node limits.

Partition Name Time Limit Minimum # nodes Maximum # nodes Usage
debug 60 minutes 1 32 Testing and debugging programs
standard 24 hours 1 512 Normal execution queue
reserved 48 hours 4 1024 For jobs using a reservation on the system

Queue Information

To view all jobs running or queued on BlueStreak, type:

squeue -l

To view the jobs for a specific user:

squeue -l -u <NetID>

Type 'man squeue' for more information.

Job Submission

  • sample sbatch script

Jobs are submitted to the queue using an sbatch script, such as

#!/bin/sh
#SBATCH --job-name MyJob                 # Job name
#SBATCH --partition debug                # Partition [debug, preempt, standard, reserved]
#SBATCH --nodes 4                        # Number of nodes [must be power of 2 within queue limitations]
#SBATCH --ntasks-per-node 16             # Number of tasks per node (There are 16 cores per node)
#SBATCH --cpus-per-task 1                # Number of cores per task (If not 1, then you should use openmp or a compiler that can auto-parallelize)
#SBATCH --time 0-00:30:00                # wall time (30 minutes)
#SBATCH --mail-type ALL                  # Send e-mail when... [BEGIN, END, FAIL, REQUEUE, or ALL]
##SBATCH --mail-user username@domain.com  # e-mail address - (currently commented out)

srun yourprogram

Note that srun must appear before a command in order for it to run on a compute node

  • sbatch submits a job to the queue:
sbatch yourscript

Type 'man sbatch' for more information.

  • scancel cancels a queued or running job:
scancel <jobid>

Jobs that were previously running should now display a completed status.

Debugging

There is a tool called core processor that will parse core files for debugging. It is located at

 /bgsys/drivers/ppcfloor/coreprocessor/bin/coreprocessor.pl

You can also use the utility addr2line to parse core files.