Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

Synchronize a disk with HPSS

Code Block
Upload dedicated ssh-key to http://www.globusonline.org/
Then from my laptop I did:

ssh -i .ssh/id_dsa-globus-balewski -t cli.globusonline.org endpoint-list
should show nothing.  But add the -p,
This should produce a public list that includes nersc#dtn & nersc#hpss. (always just use the -t to hide any passwords you might need ...)
...
nersc#dtn             
nersc#dtn_new         
nersc#hpss            
...

Activate both those endpoints,

ssh -t cli.globusonline.org endpoint-activate nersc#*
This will prompt for a pass phrase - use your nim password.

Now list again & this should show the endpoints with a time associated with it ( we can increase the time to 11 days as needed).
ssh -i .ssh/id_dsa-globus-balewski -t cli.globusonline.org endpoint-list 
Warning: No xauth data; using fake authentication data for X11 forwarding.
nersc#dtn     11:59:28
nersc#dtn_new 11:59:28
nersc#hpss    11:59:28
Connection to cli.globusonline.org closed.


Now copy 1 file from HPSS to disk.  Here's my example.

balewski-mac:~ balewski$ echo "nersc#hpss/home/pb/balewski/portertest1/aukaa.tar28MB nersc#dtn/project/projectdirs/star/porter/auk.tar"
| ssh/global/scratch/sd/balewski/w2011/starFebB/036/st_physics_12036062_raw_1010001.daq" | ssh -i .ssh/id_dsa-globus-balewski -t cli.globusonline.org transfer
Pseudo-terminal will not be allocated because stdin is not a terminal.
Warning: No xauth data; using fake authentication data for X11 forwarding.
Task ID: bd01974c-49dc-11e0-8b04-123139054450
Created transfer task with 1 file(s)



It will return a task id and you can query that specifically via

ssh -t cli.globusonline.org details your-task-id


If this all works, then try to sync a directory tree from a source to a destination
- here is my example from disk to hpss

echo "nersc#dtn/global/scratch/sd/porter/reco/2007ProductionMinBias/
nersc#hpss/home/p/porter/ -r -s 1 " | ssh -t cli.globusonline.org transfer

You need the slashes at the end of the directory names.  The 's 1' means use filesize to
determine success.

Let me know how that goes.   The same syntax could be used to copy files/directories between
 rcf and nersc once the rcf endpoint is configured.

...