Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Optimization of globus transfer, various suggestions

Code Block
guc is globus-url-copy - acronym invented by Levente
srm is a software layer over that - it makes sure transfers complete

There is a –r recursive copy option;

Or else you can do a loop in shell:
set daq   = (1380001.daq 1380003.daq 1380004.daq);
foreach f ($daq)
globus-url-copy -p 25 file:/star/bla/bla/$f 
gsiftp://pdsfgrid.nersc.gov/bla/bla/bla/$f
end 


 If you specify a source ending in "XXX/" , it will treat XXX as a
directory and transfer all files in XXX from the XXX directory

Use -p 25 option for having more streams, but isn't 25 overdoing it? You got 1 file
cut into 25 buffer chunks and all this has to be re-assembled on arrival.
I'd try +-2 around 8.

Use different end-point machine:  dtn01.nersc.gov  to avoid STAR,ATLAS conflicts

there is also carvergrid.nersc.gov gatekeeper I have access too.

 pull from carver, you need to load the osg module ,and then you have the globus commands available

bbbb