...
The output looks like this:
...
muscle -in O95_9990.fst -out O95_9990.aln
muscle -in O95_9991.fst -out O95_9991.aln
muscle -in O95_9992.fst -out O95_9992.aln
muscle -in O95_9993.fst -out O95_9993.aln
...
2. Then I submit these commands as a job array: cat commands.txt | ssub
How to configure it
Copy it to your ~/bin (or wherever). Then edit the top of the script:uname = your username tmpdir = directory where scripts are created max_size = number of nodes you want to use
Other things
It automatically creates random filenames for your scripts and job arrays. These files are created in the directory specified by "tmpdir" It can also submit individual scripts instead of a job array.
...
This will tell you the specifics of each job. There is also no priority allocation, so please be polite and choose the right queue for your job.
Submitting multiple files to process in the same job
Example from Spence -
I wanted to write bash files that would submit multiple files for the same analysis command on coyote. I used PBS_ARRAYID, which will take on values that you designate with the -t option of qsub.
I got access to qiime functions by adding the following line to the bottom of my .bashrc file:
export PATH="$PATH:/srv/pkg/python/python-2.7.6/pkg/qiime/qiime-1.8.0/qiime-1.8.0-release/bin"
Then I made my .bashrc file the source in my submission script (see below). The DATA variable just shortens a directory where I store my data.
To run all this, I created the file below then typed the following at the command line: qsub -q long -t 1-10 pickRepSet.sh
(the -t option will vary my PBS_ARRAYID variable from 1 to 10, iterating through my 10 experimental files).
#!/bin/sh
#filename: pickRepSet.sh
#
# PBS script to run a job on the myrinet-3 cluster.
# The lines beginning #PBS set various queuing parameters.
#
# -N Job Name
#PBS -N pickRepSet
#
# -l resource lists that control where job goes
#PBS -l nodes=1
#
# Where to write output
#PBS -e stderr
#PBS -o stdout
#
# Export all my environment variables to the job
#PBS -V
#
source /home/sjspence/.bashrc
DATA="/net/radiodurans/alm/sjspence/data/140509_ML1/"
pick_rep_set.py -i ${DATA}fwd_otu/uclust_picked_otus${PBS_ARRAYID}/ML1-${PBS_ARRAYID}_filt_otus.txt -f ${DATA}fwd_filt/dropletBC/ML1-${PBS_ARRAYID}_filt.fna -o ${DATA}fwd_otu/repSet/ML1-${PBS_ARRAYID}_rep.fna