Blog from May, 2014

Introduction- Scott's script to consolidate all of the OTU calling scripts and submit them to the cluster with one command

(smile_train: Get's you where you want to go and your happy)

Works on coyote and maybe the broad, but maybe not other clusters.

You should install in lib (bin get crowded)- each person should install their own.

Make a lib directory unless this already exists:

mkdir lib

Clone this into the lib folder you just make using git:

git clone https://github.com/almlab/SmileTrain 

To get some information, go to the wiki and follow the directions:

https://github.com/almlab/SmileTrain/wiki

Alter train.cfg (you either have to link to usearch etc) to have your new names for the following (for example I changed these):

[User]

username=spacocha

tmp_directory=/net/radiodurans/alm/spacocha/tmp

library=/net/radiodurans/alm/spacocha/lib/SmileTrain

bashrc=/net/radiodurans/alm/spacocha/lib/SmileTrain/bashrcs/coyote.sh

Then, make the tmp_directory:

mkdir  /net/radiodurans/alm/spacocha/tmp

You can have a detailed description of anything by using --help.

This will source the bashrc that script that Scott made.

Submit from the head node (when you log on), and if it's long, you;re going to have to use this:

ssh coyote

screen

Now you are inside screen (screen -ls will tell you which ones you are running):

You can name the screens

screen -S SPtest

you can detatch and keep running:

RUN COMMAND

contol^A D

(or type man screen to get information)

and screen reattach will give you

screen -R SPtest

but then you have to actually stop them by typing exit from within screen

exit

PCR, real-time PCR, primers, column and SPRI clean-up of reactions

Preparation of PCR primer stocks and working solutions

stocks

spin freezedried stocks, 1min full speed

add sterile H2O (molecular biology grade) to a final concentration of 100µM

working solutions

485µl sterile H2O (molecular biology grade)

+ 15µl primer stock

--> 3µM  

DNA column purification (Qiagen PCR clean-up / Qiaquick Gel Extraction)

Loading

mix reaction + 5Vol PBI buffer in Epi

place column in collection tube

load on column

spin 1min, full speed, RT

discard floughthrough

alternatively for gel extraction of DNA bands

cut bands on blue light table (DeLong lab), or on UV Transilluminator (Thompson lab) with clean     razor blade

transfer into Epi

weigh Epis (~1g) and gel slices

+3Vol/weight (300µl/100mg) GC buffer

incubate tubes at 50˚C, 10min, vortex gently every 2min

+ 1 Vol/weight (100µl/100mg) Isopropanol (improves yield especially for fragments below 500bp                             or above 4kb)

mix by inverting tube

place column in collection tube

load on column (if volume to big for column, then load, spin, discard flowthrough, and load rest)

spin 1min, full speed, RT

discard floughthrough

Washing

+750µl PE buffer (seal bottle tight to avoid EtOH evaporation)

spin 1min, full speed, RT

discard floughthrough

place column back in empty collection tube

Drying

spin (to dry) 30sec, full speed, RT

turn column in collection tube by 180˚

spin (to dry) 1min, full speed, RT

discard floughthrough and collection tube

place column in new Epi

dry open column under laminar air flow for 2min

Elution

+35-50µl EB or sterile H2O (molecular biology grade)

incubate at least for 5min

spin (to elute DNA) 30sec, full speed, RT

turn column in Epi by 180˚

spin (to elute DNA) 1min, full speed, RT

discard column and store Epi with DNA

SPRI clean up and primer/dimer removal (Agencourt AMPure XP beads)

Preparations:

adjust PCR reaction to 50µl with EB

vortex SPRI beads 1600rpm, 10sec

aliquot 45µl of beads into one 1.5ml tube per library

Binding to beads

add 50µl PCR reaction to beads

mix by pipetting/vortex 1600rpm

incubate for 5-7min

separate on magnet for 2min

remove and discard SN while tube stays on magnet

Wash - removal of salts, enzymes and low molecular weight DNA

wash beads carefully twice with 70% EtOH while tube stays on magnet (do not disturb bead      pellet)

incubate for 30sec

remove all SN

repeat

Dry

air dry on magnet for 15min

Elution

remove tube from magnet, add 20µl EB

vortex 1600rpm, 10sec

incubate at RT for 5min

separate on magnet for 2min

collect SN and transfer into new 1.5ml tube

Quantitative real-time PCR

Mastermix

    for 200µl (8x) reaction:

    10.525µl    H2O

    5µl           5x Phusion Pol buffer

    0.5µl        dNTP mix 10mM

    3.3+3.3µl  primers, 3µM

    2µl           template (try different 10 fold dilutions)

    0.125µl     SYBR Green I

    0.25µl        Pol (Phusion)

    prepare reaction in PCR tubes (or 96well plate) with optical covers that fit the utilized    

real-time PCR

cycle at         1) 98˚C, 20''

               2) 98˚C, 15''

               3) specific Annealing Temp˚C, 20''

               4) 72˚C, 20'' (for fragments shorther than 1kb)

               5) go back to step 2 45x

    always use at least 3 replicates per sample

    always include 3 replicates of a non-template (H2O) control

16S By Hand Library Preparation

Materials:

●      Agencourt Ampure XP, A63881 (60mL, $300)

●      2 Roche LichtCycler480 384-well plate

●      1:100 dilution of SYBR stock (Invitrogen S7563, 10,000x)

●      Step 1/ Initial QPCR Primers ( PE_16s_v4U515-F OR PE_16s_v4_Bar#,  PE_16s_v4_E786_R)

●      Step 2 primers ( PE-III-PCR-F, PE-IV-PCR-XXX)

●      Final QPCR primers (PE Seq F, PE Seq R)

●      HF Phusion (NEB, M0530L)

●      KAPA SYBR 2xMM for final QPCR

●      Invitrogen Super magnet (16 or 8 sample capacity)

Determination of  Step 1 Cycle Time and Sample Check:

Materials used:

○   Contents of MM

○   P200 multi-channel pipette

○    96 well QPCR plate (96 well for opticon stocked in lab)

○   Clear QPCR plate covers

Initial QPCR master mix (MM)

Run this step in duplicate or triplicate to best estimate proper cycling time

Initial QPCR Program (Opticon):

Heat:

98°C – 30 seconds

Amplify:

98°C – 30 seconds

52°C – 30 seconds

72°C – 30 seconds

Cool:

4°C - continuous

Use Ct (bottom of curve, not mid-log) of curves to determine dilutions for step 1 amplification (google docs, Illumina Library QPCR and Multiplexing)

Breakdown of QPCR amplification math (done to normalize each sample):

○   delta Ct = Sample Ct - lowest Ct in sample set

○   fold = 1.75^(delta Ct)

○   dilution needed = fold

○  note - that input is 2uL per RXN so sample with lowest Ct gets 2uL undiluted

 Please note – samples may fail due to too little, too much material, or a poor reaction. It is recommended that failed samples be re-run before moving forward

Library Preparation:

Step 1

Please Note: Samples are run as four 25uL reactions that are pooled at end of cycling

1st step Master Mix 25uL RXN (MM1)

16s Step 1 Program:

Heat:

98°C – 30 seconds

Amplify:

98°C – 30 seconds

52°C – 30 seconds

72°C – 30 seconds

Cool:

4°C - continuous

Run amplification cycle number determined via QPCR (no more than 20 cycles allowed)

After cycling pool duplicates, now have 1x 100uL reaction per sample

SPRI Clean Up

Materials used:

○   SPRI beads

○   70% EtOH

○   EB

○   Invitrogen super magnet

- Vortex cold AmpureXP beads, pool DNA from PCR tubes (~100uL)

- Aliquot 85.5uL beads into Epi’s – let equilibrate to RT

- Add DNA (take 95uL) + beads (85.5uL) = 180.5 uL

- Incubate 13’ @ RT

- Separate ON magnet 2’

- While ON magnet, remove/discard SN

- Wash beads 2x with 70% EtOH, 500uL each wash

- Air dry beads for 15-20’ on magnet

- Remove from magnet, elute in 40uL H2O, vortex to resuspend

- Incubate (at least 7’)

-  Separate on magnet 2’

- Collect 35-40 ul and save SN

Sample Re-Aliquoting and Step 2

Please Note: Samples are run as four 25uL reactions that are pooled at end of cycling

2nd step Master Mix  25uL RXNs (MM2)

16s Step 2 Program:

Heat:

98°C – 30 seconds

Amplify:

98°C – 30 seconds

83°C – 30 seconds

72°C – 30 seconds

Cool:

4°C - continuous

Run 9 cycles of amplification

- After cycling pool duplicates, now have 1x 100uL reaction per sample

SPRI Clean Up

Materials used:

○   SPRI beads

○   70% EtOH

○   EB

○   Invitrogen super magnet

- Vortex cold AmpureXP beads, pool DNA from PCR tubes (~100uL)

- Aliquot 85.5uL beads into Epi’s – let equilibrate to RT

- Add DNA (take 95uL) + beads (85.5uL) = 180.5 uL

- Incubate 13’ @ RT

- Separate ON magnet 2’

- While ON magnet, remove/discard SN

- Wash beads 2x with 70% EtOH, 500uL each wash

- Air dry beads for 15-20’ on magnet

- Remove from magnet, elute in 40uL H2O, vortex to resuspend

- Incubate (at least 7’)

-  Separate on magnet 2’

- Collect 35-40 ul and save SN

Final QPCR

Once you have a substantial or all of your samples prepared you can run a final QPCR to determine dilutions and volumes for multiplexing. This step also confirms that the library preparation was successful

QPCR Master Mix (QPCR MM, 20uL RXN)

Final QPCR Program (Opticon)

Heat:

95°C – 5 minutes

Amplify:

95°C – 10 seconds

60°C – 20 seconds

72°C – 30 seconds

Melting Curve:

95°C – 5 seconds

65°C – 1 minute

97°C - continuous

Cool:

40°C – 10 seconds

Run 35 cycles of amplification

Use mid-log phase of curves to determine volumes for multiplexing (google docs, Illumina Library QPCR and Multiplexing)

Please note – samples may fail due to too little, too much material, or a poor reaction. It is recommended that failed samples be re-run before moving forward

Breakdown of QPCR multiplexing math (done to normalize each sample):

○  delta Ct = Sample Ct - lowest Ct in sample set

○  fold = 1.75^(delta Ct)

○  ratio = 1/fold

○  volume to mix b/c of ratio = X*ratio (X = minimum desired volume per sample)

○  how to dilute = fold

○  note - sample with lowest Ct will get an undiluted Xuls added to final multiplex, X can be raised or lowered to accommodate the needed volume of other samples

Sample Multiplexing and Submission for Sequencing:

- Once samples have been multiplexed aliquot ~20uL of the final mix and submit it to the BioMicro Center for sequencing

Introduction

This is the general outline for running AdaptML. This guide was generated by Sarah Preheim, not Lawrence David, just to keep that in mind. It is meant to complement the user guide provided on the official web page, but also includes some additional information for non-experts.

Download

Download AdaptML from the Alm Lab website and install as directed:

http://almlab.mit.edu.ezproxyberklee.flo.org/adaptml.html

Input tree

Unrooted tree is used to run AdaptML. Usually I make them with phyml for 2,000 or fewer sequences:

phyml_v2.4.4/exe/phyml_linux all_u.phy 0 i 1 100 GTR e e 4 e BIONJ y y

Running AdaptML

Here is an example of how to run AdaptML:

python ../../latest_adaptml/habitats/trunk/AdaptML_Analyze.py tree=./All_simple_7_ur_particle.phy_phyml_tree.txt2 hab0=16 outgroup=ECK_1 write=./ thresh=0.025

Finding Stable Habitats

Although there was a set of habitats predicted, you want to determine how often those same habitats would be predicted from 100 different iterations of AdaptML.

For example:

Now try to standardize with 100 runs:

foreach f ( 0 1 2 3 4 5 6 7 8 9 )

foreach? foreach d (0 1 2 3 4 5 6 7 8 9)

foreach? mkdir ${f}${d}_dir

foreach? python ../../latest_adaptml/habitats/trunk/AdaptML_Analyze.py tree=./All_simple_7_ur_particle.phy_phyml_tree.txt2 hab0=16 outgroup=ECK_1 write=./${f}${d}_dir/ thresh=0.025

foreach? perl parse_migration3.pl ./${f}${d}_dir/habitat.matrix > ./${f}${d}_dir/habitat.matrix.tab

foreach? end

Make a list of all of the migration.tab files

ls *_dir/habitat.matrix.tab > migration_list.txt2

Then use temp_dist2.pl to get the habitat number distribution.

perl ~/bin/temp_dist2.pl migration_list.txt2

4    5

5    21

6    53

7    20

8    1

The original migration.tab file in ./040609_dir/ has 6 habitats, so just use that to see the percent matching.

Then use get_stable_habitats6.pl and you might have to change the percent matching

so it’s as high as it can go before it dies with double hits.

perl get_stable_habitats6.pl 0.015 ../040609_dir/migration.tab migration_list.txt2

Results with 0.024:

11    39

3    100

13    100

10    80

12    100

14    99

Make clusters

Once you have found a set of stable habitats you want to go with, then make the clusters. This example is for the tree ./example_noroot.phy_phyml_tree.txt3, outgroup ECK_1 using 9900.file to make the final clusters (using python 2.5.1 and you may have to change the paths to point to the AdaptML software):

python ~/AdaptML_dir/latest_AdaptML/habitats/trunk/AdaptML_Analyze.py tree=./example_noroot.phy_phyml_tree.txt3 hab0=16 outgroup=ECK_1 write=./example_dir thresh=0.05

perl ~/bin/migration2color_codes.pl ./example_dir/migration.matrix color_template.txt > ./example_dir/color.file

mkdir ./example_dir/rand_trials

python ~/AdaptML_dir/latest_AdaptML/clusters/getstats/rand_JointML.py habitats=./example_dir/migration.matrix mu=./example_dir/mu.val tree=./example_noroot.phy_phyml_tree.txt3 outgroup=ECK_1 write=./example_dir/rand_trials/

python ~/AdaptML_dir/latest_AdaptML/clusters/getstats/GetLikelihoods.py ./example_dir/rand_trials/ ./example_dir/

python ~/AdaptML_dir/latest_AdaptML/clusters/trunk/JointML.py habitats=./example_dir/migration.matrix mu=./example_dir/mu.val  color=./example_dir/color.file tree=./example_noroot.phy_phyml_tree.txt3 write=./example_dir/ outgroup=ECK_1  thresh=./example_dir/9900.file