Reading a variable line by line with while loop
 

2009/12/1 Ray Parrish <crp@cmc.net>:
> Hello,
>
> I have been using while loops to read files in line by line when I want
> to process things on a line oriented basis. Sometimes i have the set of
> lines I want to read through already in a variable, and have to write
> them to file before beginning to read line by line.
>
> I was wondering if it is possible to read a variable line by line
> somehow with a while loop? Would it possibly be faster than using a file
> to read from?
>
> I was thinking something along the lines of the following code might
> work, but have not tested it yet.
>
> while read ThisLine; do
> * * *# process each line
> * * *echo "$ThisLine"
> done < `echo "$Variable"`
>
> Is this something that might work, and would it be more efficient than
> writing to, and reading from a file? Is there some other way I've missed
> to make it work?

This will probably not work.

echo "$Variable" | while read
...
done

has better chances of working.

Loïc

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Reading a variable line by line with while loop
 

Loïc Grenié wrote:
> 2009/12/1 Ray Parrish <crp@cmc.net>:
>
>> Hello,
>>
>> I have been using while loops to read files in line by line when I want
>> to process things on a line oriented basis. Sometimes i have the set of
>> lines I want to read through already in a variable, and have to write
>> them to file before beginning to read line by line.
>>
>> I was wondering if it is possible to read a variable line by line
>> somehow with a while loop? Would it possibly be faster than using a file
>> to read from?
>>
>> I was thinking something along the lines of the following code might
>> work, but have not tested it yet.
>>
>> while read ThisLine; do
>> # process each line
>> echo "$ThisLine"
>> done < `echo "$Variable"`
>>
>> Is this something that might work, and would it be more efficient than
>> writing to, and reading from a file? Is there some other way I've missed
>> to make it work?
>>
>
> This will probably not work.
>
> echo "$Variable" | while read
> ...
> done
>
> has better chances of working.
>
> Loïc
>
You are right, your method works, whereas mine did not. Thank you for
the valuable pointer.

Here is what I got to work.

# read this script into the variable
Variable=`cat $0`
# Read each line in variable, and echo it to standard out.
echo "$Variable" | while read ThisLine; do
# process each line
echo "$ThisLine"
done

Thanks again, and later! Ray Parrish

--
The Future of Technology.
http://www.rayslinks.com/The%20Future%20of%20Technology.html
Ray's Links, a variety of links to usefull things, and articles by Ray.
http://www.rayslinks.com
Writings of "The" Schizophrenic, what it's like to be a schizo, and other
things, including my poetry.
http://www.writingsoftheschizophrenic.com



--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Reading a variable line by line with while loop
 

Loïc Grenié wrote:
> 2009/12/1 Ray Parrish <crp@cmc.net>:
>
>> Hello,
>>
>> I have been using while loops to read files in line by line when I want
>> to process things on a line oriented basis. Sometimes i have the set of
>> lines I want to read through already in a variable, and have to write
>> them to file before beginning to read line by line.
>>
>> I was wondering if it is possible to read a variable line by line
>> somehow with a while loop? Would it possibly be faster than using a file
>> to read from?
>>
>> I was thinking something along the lines of the following code might
>> work, but have not tested it yet.
>>
>> while read ThisLine; do
>> # process each line
>> echo "$ThisLine"
>> done < `echo "$Variable"`
>>
>> Is this something that might work, and would it be more efficient than
>> writing to, and reading from a file? Is there some other way I've missed
>> to make it work?
>>
>
> This will probably not work.
>
> echo "$Variable" | while read
> ...
> done
>
> has better chances of working.
>
> Loïc
>
Well, that one test worked, but I cannot get the following to work -

# Read bash history into a variable for use with combobox call
History=""
BashHistory=`cat ~/.bash_history`
echo "$BashHistory" | while read ThisCommand; do
# Replace spaces in each command line with __ [double underlines]
ThisCommand=${ThisCommand// /__}
# Concatenate the results to the Command variable
History="$History $ThisCommand"
done
echo "History - $History"

The last echo command returns nothing, but if I put an echo command in
the loop either before, or after the replace spaces command, it echoes
every line of the variable to the console.

Does anyone have any idea why it's not working because I am stumped...

Later, Ray Parrish

--
The Future of Technology.
http://www.rayslinks.com/The%20Future%20of%20Technology.html
Ray's Links, a variety of links to usefull things, and articles by Ray.
http://www.rayslinks.com
Writings of "The" Schizophrenic, what it's like to be a schizo, and other
things, including my poetry.
http://www.writingsoftheschizophrenic.com



--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Reading a variable line by line with while loop
 

Ray Parrish wrote:
> Loïc Grenié wrote:
>
>> 2009/12/1 Ray Parrish <crp@cmc.net>:
>>
>>
>>> Hello,
>>>
>>> I have been using while loops to read files in line by line when I want
>>> to process things on a line oriented basis. Sometimes i have the set of
>>> lines I want to read through already in a variable, and have to write
>>> them to file before beginning to read line by line.
>>>
>>> I was wondering if it is possible to read a variable line by line
>>> somehow with a while loop? Would it possibly be faster than using a file
>>> to read from?
>>>
>>> I was thinking something along the lines of the following code might
>>> work, but have not tested it yet.
>>>
>>> while read ThisLine; do
>>> # process each line
>>> echo "$ThisLine"
>>> done < `echo "$Variable"`
>>>
>>> Is this something that might work, and would it be more efficient than
>>> writing to, and reading from a file? Is there some other way I've missed
>>> to make it work?
>>>
>>>
>> This will probably not work.
>>
>> echo "$Variable" | while read
>> ...
>> done
>>
>> has better chances of working.
>>
>> Loïc
>>
>>
> Well, that one test worked, but I cannot get the following to work -
>
> # Read bash history into a variable for use with combobox call
> History=""
> BashHistory=`cat ~/.bash_history`
> echo "$BashHistory" | while read ThisCommand; do
> # Replace spaces in each command line with __ [double underlines]
> ThisCommand=${ThisCommand// /__}
> # Concatenate the results to the Command variable
> History="$History $ThisCommand"
> done
> echo "History - $History"
>
> The last echo command returns nothing, but if I put an echo command in
> the loop either before, or after the replace spaces command, it echoes
> every line of the variable to the console.
>
> Does anyone have any idea why it's not working because I am stumped...
>
> Later, Ray Parrish
>
Hello again,

OK, I understand that the | is causing a sub shell to execute for the
echo command, so the variable $History even 'though declared globally,
is local to the loop with the pipe, and therefore not available after
the loop exits.

I have tested this by putting an echo "History - $History" command right
after the concatenation line in the loop. This causes some pretty slow
execution as the extremely long string get echoed over and over again,
but showed that the variable was indeed getting updated each time
through the loop.

Does anyone know how to fix my code so it can access the value of the
$History variable after the loop exits?

Thanks again, Ray Parrish

--
The Future of Technology.
http://www.rayslinks.com/The%20Future%20of%20Technology.html
Ray's Links, a variety of links to usefull things, and articles by Ray.
http://www.rayslinks.com
Writings of "The" Schizophrenic, what it's like to be a schizo, and other
things, including my poetry.
http://www.writingsoftheschizophrenic.com



--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Reading a variable line by line with while loop
 

On Tue, 2009-12-01 at 06:57 -0800, Ray Parrish wrote:
> Ray Parrish wrote:
> >
> Hello again,
>
> OK, I understand that the | is causing a sub shell to execute for the
> echo command, so the variable $History even 'though declared globally,
> is local to the loop with the pipe, and therefore not available after
> the loop exits.
>

Right. subshells do not pass their environment back to their parent.
The read command is pretty inefficient. You might want to learn sed.
It is well suited to this particular task. Something like:

History=`sed 's/ /__/g' ~/.bash_history`

The substitution replaces all spaces in the Bash history with __.
Output is interpolated inline and assigned to the History variable.

--
Smoot Carl-Mitchell
Computer Systems and
Network Consultant
smoot@tic.com
+1 480 922 7313
cell: +1 602 421 9005

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Reading a variable line by line with while loop
 

Smoot Carl-Mitchell wrote:
> On Tue, 2009-12-01 at 06:57 -0800, Ray Parrish wrote:
>
>> Ray Parrish wrote:
>>
>> Hello again,
>>
>> OK, I understand that the | is causing a sub shell to execute for the
>> echo command, so the variable $History even 'though declared globally,
>> is local to the loop with the pipe, and therefore not available after
>> the loop exits.
>>
>>
>
> Right. subshells do not pass their environment back to their parent.
> The read command is pretty inefficient. You might want to learn sed.
> It is well suited to this particular task. Something like:
>
> History=`sed 's/ /__/g' ~/.bash_history`
>
> The substitution replaces all spaces in the Bash history with __.
> Output is interpolated inline and assigned to the History variable.
>
Thank you! That worked great! Now I wonder is there is another great sed
command that will remove redundancy in the bash history, so no command
line is repeated in the final results in the $History variable?

I don't mean to get you to do my programming for me, and I will begin
studying the sed manual right after I send this to see if I can figure
it out myself. I just thought I'd pick your brain a bit more. 8-)

Later, Ray Parrish

--
The Future of Technology.
http://www.rayslinks.com/The%20Future%20of%20Technology.html
Ray's Links, a variety of links to usefull things, and articles by Ray.
http://www.rayslinks.com
Writings of "The" Schizophrenic, what it's like to be a schizo, and other
things, including my poetry.
http://www.writingsoftheschizophrenic.com



--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Reading a variable line by line with while loop
 

Ray Parrish wrote:

> Smoot Carl-Mitchell wrote:
> Thank you! That worked great! Now I wonder is there is another great
> sed command that will remove redundancy in the bash history, so no
> command line is repeated in the final results in the $History
> variable?

man uniq

which requires you to sort it, too:

man sort

since it checks for repeated (consecutive) lines. If order matters,
you'll need something more complicated.

sed can probably be made to do it, but piping through sort and uniq is
probably nicer.


--
Avi Greenbury
http://aviswebsite.co.uk ;)
http://aviswebsite.co.uk/asking-questions

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Reading a variable line by line with while loop
 

2009/12/1 Avi Greenbury <avismailinglistaccount@googlemail.com>:
> Ray Parrish wrote:
>
>> Smoot Carl-Mitchell wrote:
>> Thank you! That worked great! Now I wonder is there is another great
>> sed command that will remove redundancy in the bash history, so no
>> command line is repeated in the final results in the $History
>> variable?
>
> man uniq
>
> which requires you to sort it, too:
>
> man sort

sort -u

my friend (discovered two weeks ago after years of sort|uniq).

Loïc

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Reading a variable line by line with while loop
 

2009/12/1 Ray Parrish <crp@cmc.net>:
> Ray Parrish wrote:
>> Loïc Grenié wrote:
>>
>>> 2009/12/1 Ray Parrish <crp@cmc.net>:
>>>
>>>
>>>> Hello,
>>>>
>>>> I have been using while loops to read files in line by line when I want
>>>> to process things on a line oriented basis. Sometimes i have the set of
>>>> lines I want to read through already in a variable, and have to write
>>>> them to file before beginning to read line by line.
>>>>
>>>> I was wondering if it is possible to read a variable line by line
>>>> somehow with a while loop? Would it possibly be faster than using a file
>>>> to read from?
>>>>
>>>> I was thinking something along the lines of the following code might
>>>> work, but have not tested it yet.
>>>>
>>>> while read ThisLine; do
>>>> * * *# process each line
>>>> * * *echo "$ThisLine"
>>>> done < `echo "$Variable"`
>>>>
>>>> Is this something that might work, and would it be more efficient than
>>>> writing to, and reading from a file? Is there some other way I've missed
>>>> to make it work?
>>>>
>>>>
>>> * * This will probably not work.
>>>
>>> echo "$Variable" | while read
>>> ...
>>> done
>>>
>>> * * has better chances of working.
>>>
>>> * * * * * *Loïc
>>>
>>>
>> Well, that one test worked, but I cannot get the following to work -
>>
>> # Read bash history into a variable for use with combobox call
>> History=""
>> BashHistory=`cat ~/.bash_history`
>> echo "$BashHistory" | while read ThisCommand; do
>> * * *# Replace spaces in each command line with __ [double underlines]
>> * * *ThisCommand=${ThisCommand// /__}
>> * * *# Concatenate the results to the Command variable
>> * * *History="$History $ThisCommand"
>> done
>> echo "History - $History"
>>
>> The last echo command returns nothing, but if I put an echo command in
>> the loop either before, or after the replace spaces command, it echoes
>> every line of the variable to the console.
>>
>> Does anyone have any idea why it's not working because I am stumped...
>>
>> Later, Ray Parrish
>>
> Hello again,
>
> OK, I understand that the | is causing a sub shell to execute for the
> echo command, so the variable $History even 'though declared globally,
> is local to the loop with the pipe, and therefore not available after
> the loop exits.
>
> I have tested this by putting an echo "History - $History" command right
> after the concatenation line in the loop. This causes some pretty slow
> execution as the extremely long string get echoed over and over again,
> but showed that the variable was indeed getting updated each time
> through the loop.
>
> Does anyone know how to fix my code so it can access the value of the
> $History variable after the loop exits?

You might need to adjust the ; but something along the
lines of:

echo "$BashHistory" | (while read ThisCommand; do
ThisCommand=${ThisCommand// /__};
History="$History $ThisCommand";
done;
echo "History - $History")

I don't know what you want to do but:

a=`cat file`
echo "$a" | whatever

is a strange way to do

cat file | whatever

or

whatever < file

In case of a loop

while read command
do
anything with $command
done < file

will do.

Loïc

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users

Reading a variable line by line with while loop
 

On Tue, 2009-12-01 at 07:39 -0800, Ray Parrish wrote:
> Smoot Carl-Mitchell wrote:
> >
> Thank you! That worked great! Now I wonder is there is another great sed
> command that will remove redundancy in the bash history, so no command
> line is repeated in the final results in the $History variable?

Check out the sort and uniq command. You need sort for uniq to work
correctly when deduping lines in a file. The original order is not
preserved. The simple idiom for deduping stdin and writing it to stdout
is:

sort | uniq

>
> I don't mean to get you to do my programming for me, and I will begin
> studying the sed manual right after I send this to see if I can figure
> it out myself. I just thought I'd pick your brain a bit more. 8-)

Happy to help. sed is extremely useful and it is always a good idea to
learn regular expression which sed uses. They are quite powerful.

--
Smoot Carl-Mitchell
Computer Systems and
Network Consultant
smoot@tic.com
+1 480 922 7313
cell: +1 602 421 9005

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users