bpcp with globbing
becker at scyld.com
Thu Nov 13 21:31:49 EST 2003
On Thu, 13 Nov 2003, Jean Paul Ampuero wrote:
> I am trying to gather output files from the slaves to the master node
> using the bpcp command (example: bpcp 2:/scratch/ampuero/SCEC1/S001*
> But globbing does not work the way I'd like: bpcp tries to expand the *
> in the master,
> instead of in the slave.
This is the behavior your would expect: globbing is done by the local
shell before the expanded args are passed to bpcp.
> Similar problem with "bpsh -a cp /scratch/ampuero/SCEC1/S0* ~ampuero".
> Is there a workaround ?
To get the exact semantics of shell globbing, you must use the shell
itself. Here is a broken-out way of doing what you wish:
FILES=`bpsh 2 /bin/sh -c 'echo /scratch/ampuero/SCEC1/S001*'
for file in $FILES; do
bpcp 2:$file ~ampuero
The clearest way to do this in a single line is to 'reverse' the
direction of the copy:
bpsh 2 bpcp `bpsh 2 /bin/sh -c 'echo /scratch/ampuero/SCEC1/S001*' master:~ampuero
Some detailed notes about this:
- By running the 'bpcp' on the "read-from" node we don't have to prepend
"2:" to each file name.
- 'bpcp' is modeled after 'rcp', which has a similar local globbing
- You use you might be able to simplify the specific commands to not
use shell globbing. But using '/bin/sh' prevents misinterpretations,
such confusing regexp and globbing: does ".*" mean all files or
just dot files?
- The command above relies on 'echo' being a shell built-in.
Donald Becker becker at scyld.com
Scyld Computing Corporation http://www.scyld.com
914 Bay Ridge Road, Suite 220 Scyld Beowulf cluster system
Annapolis MD 21403 410-990-9993
Beowulf mailing list, Beowulf at beowulf.org
To change your subscription (digest mode or unsubscribe) visit http://www.beowulf.org/mailman/listinfo/beowulf
More information about the Beowulf