FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 02-01-2012, 09:53 PM
Stephen Harris
 
Default Bash scripting - Remotely ran commands break while loop

On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
> echo " server2
> server2" |
> while read confLine; do
> echo "--> $confLine"
> ssh peter@$confLine ls
> echo "--> END $confLine"
> done

> The "for" loop in the script above will run twice but the "while" loop
> below it will run only once.

> Any idea what would cause the ssh command to break the while loop?

"ssh" is reading from stdin and passing the data over to the remote
machine. You can test this with
ssh peter@$confLine 'read x ; echo we got $x'

To stop it doing this, use the "-n" flag
ssh -n peter@$confLine ls

--

rgds
Stephen
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-02-2012, 02:03 AM
Peter Blajev
 
Default Bash scripting - Remotely ran commands break while loop

On Wed, Feb 1, 2012 at 2:53 PM, Stephen Harris <lists@spuddy.org> wrote:

> On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
> > echo " server2
> > server2" |
> > while read confLine; do
> > echo "--> $confLine"
> > ssh peter@$confLine ls
> > echo "--> END $confLine"
> > done
>
> > The "for" loop in the script above will run twice but the "while" loop
> > below it will run only once.
>
> > Any idea what would cause the ssh command to break the while loop?
>
> "ssh" is reading from stdin and passing the data over to the remote
> machine. You can test this with
> ssh peter@$confLine 'read x ; echo we got $x'
>
> To stop it doing this, use the "-n" flag
> ssh -n peter@$confLine ls


This is it. Right on Stephen. Thank you very much. I can't believe I've
gone so long without knowing it.

This works for me. I still don't have full understanding of it but I'll do
some more reading.

Unfortunately I can't always use the (-n) option. If I wan't to send data
through the pipe then the (-n) won't work. For example (on top of my head):
mysqldump dB | ssh peter@remoteServer "mysql dB"

In my script I ended up using "ssh -n" when I want to work on the output of
remotely ran command and "ssh" without (-n) when I want to send data over
ssh to a remote command.

This so far is not breaking the while loop and it seems to be working but
it makes me nervous.

Any note will be appreciated.

Thanks again.

--
Peter
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-02-2012, 02:13 AM
Stephen Harris
 
Default Bash scripting - Remotely ran commands break while loop

On Wed, Feb 01, 2012 at 07:03:33PM -0800, Peter Blajev wrote:
> On Wed, Feb 1, 2012 at 2:53 PM, Stephen Harris <lists@spuddy.org> wrote:
>
> > On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
> > > echo " server2
> > > server2" |
> > > while read confLine; do
> > > echo "--> $confLine"
> > > ssh peter@$confLine ls
> > > echo "--> END $confLine"
> > > done

> > > Any idea what would cause the ssh command to break the while loop?
> >
> > "ssh" is reading from stdin and passing the data over to the remote
> > machine. You can test this with
> > ssh peter@$confLine 'read x ; echo we got $x'
> >
> > To stop it doing this, use the "-n" flag
> > ssh -n peter@$confLine ls
>
> Unfortunately I can't always use the (-n) option. If I wan't to send data
> through the pipe then the (-n) won't work. For example (on top of my head):
> mysqldump dB | ssh peter@remoteServer "mysql dB"

In this situation, ssh will read from the pipe and not from the "echo"
statement and so it won't break your while loop.

The thing you need to understand is how redirection works.
echo foo | while read
do
...
done

Everything from the "while" to the "done" will have stdin configured to
read from the pipe (the output of "echo"). Here's an example:
% echo "a
b
c
d" | while read a
do
echo We have $a
read b
echo And $b
done
We have a
And b
We have c
And d

You can see that the "read" statement inside the loop is reading from
the pipe and so draining input.

An "ssh" on its own, in this situation, would drain _all_ the input.
The "-n" flag tells ssh to not do this. But, equally, when you do
sqldump | ssh
then you've told ssh to take input from the output of sqldump, and so it's
no longer reading the from outer loop.

Instead of "ssh -n" you could do "ssh < /dev/null" or "echo | ssh" or
other options, and get the same sort of effect; you're setting up the
stdin to ssh to be from somewhere _other_ than your main echo.

--

rgds
Stephen
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-02-2012, 02:38 AM
Peter Blajev
 
Default Bash scripting - Remotely ran commands break while loop

On Wed, Feb 1, 2012 at 7:13 PM, Stephen Harris <lists@spuddy.org> wrote:

> On Wed, Feb 01, 2012 at 07:03:33PM -0800, Peter Blajev wrote:
> > On Wed, Feb 1, 2012 at 2:53 PM, Stephen Harris <lists@spuddy.org> wrote:
> >
> > > On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
> > > > echo " server2
> > > > server2" |
> > > > while read confLine; do
> > > > echo "--> $confLine"
> > > > ssh peter@$confLine ls
> > > > echo "--> END $confLine"
> > > > done
>
> > > > Any idea what would cause the ssh command to break the while loop?
> > >
> > > "ssh" is reading from stdin and passing the data over to the remote
> > > machine. You can test this with
> > > ssh peter@$confLine 'read x ; echo we got $x'
> > >
> > > To stop it doing this, use the "-n" flag
> > > ssh -n peter@$confLine ls
> >
> > Unfortunately I can't always use the (-n) option. If I wan't to send data
> > through the pipe then the (-n) won't work. For example (on top of my
> head):
> > mysqldump dB | ssh peter@remoteServer "mysql dB"
>
> In this situation, ssh will read from the pipe and not from the "echo"
> statement and so it won't break your while loop.
>
> The thing you need to understand is how redirection works.
> echo foo | while read
> do
> ...
> done
>
> Everything from the "while" to the "done" will have stdin configured to
> read from the pipe (the output of "echo"). Here's an example:
> % echo "a
> b
> c
> d" | while read a
> do
> echo We have $a
> read b
> echo And $b
> done
> We have a
> And b
> We have c
> And d
>
> You can see that the "read" statement inside the loop is reading from
> the pipe and so draining input.
>
> An "ssh" on its own, in this situation, would drain _all_ the input.
> The "-n" flag tells ssh to not do this. But, equally, when you do
> sqldump | ssh
> then you've told ssh to take input from the output of sqldump, and so it's
> no longer reading the from outer loop.
>
> Instead of "ssh -n" you could do "ssh < /dev/null" or "echo | ssh" or
> other options, and get the same sort of effect; you're setting up the
> stdin to ssh to be from somewhere _other_ than your main echo.


Wow. Great lesson. Thank you Stephen.
It makes more sense now.

I'll save this email for sure.

--
Peter
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-03-2012, 10:45 AM
Marc Deop
 
Default Bash scripting - Remotely ran commands break while loop

On Wednesday 01 February 2012 19:03:33 Peter Blajev wrote:
> On Wed, Feb 1, 2012 at 2:53 PM, Stephen Harris <lists@spuddy.org> wrote:
>
> > On Wed, Feb 01, 2012 at 01:07:31PM -0800, Peter Blajev wrote:
> > > echo " server2
> > > server2" |
> > > while read confLine; do
> > > echo "--> $confLine"
> > > ssh peter@$confLine ls
> > > echo "--> END $confLine"
> > > done
> >
> > > The "for" loop in the script above will run twice but the "while" loop
> > > below it will run only once.
> >
> > > Any idea what would cause the ssh command to break the while loop?
> >
> > "ssh" is reading from stdin and passing the data over to the remote
> > machine. You can test this with
> > ssh peter@$confLine 'read x ; echo we got $x'
> >
> > To stop it doing this, use the "-n" flag
> > ssh -n peter@$confLine ls
>
>
> This is it. Right on Stephen. Thank you very much. I can't believe I've
> gone so long without knowing it.
>
> This works for me. I still don't have full understanding of it but I'll do
> some more reading.
>
> Unfortunately I can't always use the (-n) option. If I wan't to send data
> through the pipe then the (-n) won't work. For example (on top of my head):
> mysqldump dB | ssh peter@remoteServer "mysql dB"
>
> In my script I ended up using "ssh -n" when I want to work on the output of
> remotely ran command and "ssh" without (-n) when I want to send data over
> ssh to a remote command.
>
> This so far is not breaking the while loop and it seems to be working but
> it makes me nervous.
>

Curious I found this very same problem some days ago and wrote about it in my blog hehe

All the options you have described in the thread will usually work but I suggest you to think about another option. Make the while read from another file descriptor as in:

while read <&3 line; do
......
done 3< file

And why this? because not only ssh reads from standard input but others like ffmpeg do as well(and probably more...)

Regards,

Marc Deop
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 01:51 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org