This is all about how to compute on coyote.
Gaining access:
I'm pretty sure that [greg] still helps out with this, you would probably need to ask him for access. Also, if you are working off campus, you need to log into your on-campus account first then ssh onto coyote. If you have questions about this, just ask me.
What is available:
modules
module add python/2.7.3
module add atlas/3.8.3
module add suitesparse/20110224
module add numpy/1.5.1
module add scipy/0.11.0
module add biopython
module add matplotlib/1.1.0
module add matlab
Not QIIME as far as I can tell.
Interactive computing:
I've was able to get onto a node with:
qsub -I -l nodes=1
But when I tried to use matlab with (module add matlab) it didn't work (although it did work upon ssh'ing)
Submitting multiple jobs:
Before running the program below, make sure to load the following modules (I tried without and I got an error loading argparse):
module add python/2.7.3
module add atlas/3.8.3
module add suitesparse/20110224
module add biopython
module add java/1.6.0_21
Also, there are different timed queues, so make sure if you get this working that it submits to the right queue. From what I can tell there is short
From Mark and Chris (although I have yet to be able to make it work- I can get the script to write but it doesn't run)-
I've been using a script chris wrote which works pretty well:/home/csmillie/bin/ssub
What it does
It streamlines job submission. If you give it a list of commands, it will (1) create scripts for them, and (2) submit them as a job array. You can give it the list of commands as command line arguments or through a pipe.
Quick examples
1. Submit a single command to the cluster ssub "python /path/to/script.py > /path/to/output.txt"
2. Submit multiple commands to the cluster (use semicolon separator) ssub "python /path/to/script1.py; python /path/to/script2.py"
3. Submit a list of commands to the cluster (newline separator) cat /list/of/commands.txt | ssub
Detailed example /home/csmillie/alm/mammals/aln/95/
In this directory, I have 12,352 fasta files I want to align. I can do this on 100 nodes quite easily:
1. First, I create a list of commands: for x in `ls *fst`; do y=$
; echo muscle -in $x -out $y.aln; done > commands.txt
The output looks like this:
...
muscle -in O95_9990.fst -out O95_9990.aln
muscle -in O95_9991.fst -out O95_9991.aln
muscle -in O95_9992.fst -out O95_9992.aln
muscle -in O95_9993.fst -out O95_9993.aln
...
2. Then I submit these commands as a job array: cat commands.txt | ssub
How to configure it
Copy it to your ~/bin (or wherever). Then edit the top of the script:uname = your username tmpdir = directory where scripts are created max_size = number of nodes you want to use
Other thingsIt automatically creates random filenames for your scripts and job arrays. These files are created in the directory specified by "tmpdir" It can also submit individual scripts instead of a job array.
How to run matlab:
This is running interactively upon login. I'm not sure if there is a better way to do this.
ssh -X username@coyote.mit.edu
module add matlab
matlab
-A window should pop up and so you can use the matlab GUI.