Conditional execution of jobs using PBS or slurm

It is possible to start a job on the condition that another one completes beforehand; this may be necessary for instance if the input to one job is generated by another job. Job dependency is defined in PBS using the -W flag.

To illustrate with an example, suppose you need to start a job using the script second_job.sh after another job finished successfully. Assume the first job is started using script first_job.sh and the command to start the first job

qsub first_job.sh

returns the job ID 7777. Then, the command to start the second job is

qsub -W depend=afterok:7777 second_job.sh

This job dependency can be further automated (possibly to be included in a bash script) using environment variables:

JOB_ID_1=qsub first_job.sh

JOB_ID-2=qsub -W depend=afterok:$JOB_ID_1 second_job.sh

Ravindra Shinde
Ravindra Shinde
Research Scientist

I am a theoretical and computational condensed matter physicist and quantum chemist. I have around 10 years of research experience, with 6 years in scientific code development. I have authored 16 research articles, 1 book, and peer-reviewed 25 manuscripts.



Next
Previous