FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 03-11-2009, 03:05 PM
Ole Tange
 
Default #518696 ITP: parallel -- build and execute command lines from standard input in parallel]

One of friends alerted me to your discussion of 'parallel' and whether
other tools can replace it. Here are some good examples to try to
reproduce without 'parallel':

$ ls | grep abc | parallel gzip -c >/tmp/file.gz

Try this in a dir with 1000 50 kB sized mixed files with file names
that include these characters: [-'` "+*<>&$!?|]. The output is a valid
.gz file as the output is grouped. In other solutions not using
parallel you will often have a race condition (such as running two
cat's in parallel).

$ ls | parallel diff {} foo ">"{}.diff
$ ls | parallel 'echo -n {}" "; ls {}|wc -l'

These should also be run on files with names containing interesting
characters [-'` "+*<>&$!?|].

$ seq 1 255 | parallel -j 50 'ping -c 1 10.0.0.{} && wget
http://status-server/status.cgi?ip=10.0.0.{}'

The 3 above would normally require making a script and calling it. I
have yet to see how to do that on the command line (including the
parallelization).

$ cat shellscript
ls foo
touch bar
host foo.bar
ping bar.foo
seq 1 10 | ssh foo.bar "cat >/tmp/count"
wget http://foo.bar/info
[...1000000 other one line shell commands...]
$ cat shellscript | time sh
$ cat shellscript | time parallel

This one of the ways I use 'parallel' most with the shellscript often
being generated.

There are more examples in the man-page.

<Disclosure>
I am the author of 'parallel'
</Disclosure>

/Ole


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 03:24 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org