FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu User

 
 
LinkBack Thread Tools
 
Old 12-02-2009, 12:13 AM
James Michael Fultz
 
Default Reading a variable line by line with while loop

* Ray Parrish <crp@cmc.net> [2009-12-01 15:29 -0800]:
> Well, I'm making some progress. I have the following code which works to
> a point.
>
> while read ThisCommand; do
> ThisCommand=${ThisCommand// /__};
> if `echo "${History}" | grep "${ThisCommand}" 1>/dev/null 2>&1`
> then
> echo "nothing" >/dev/null

The colon ( builtin which evaluates as a successful command execution
is useful where you to effectively do nothing in your script.

: does nothing

You could also perform a negative test in order to avoid the do-nothing
block of your if-statement. Also, you can evaluate grep's exit status
directly rather than the output of the command. You also probably want
to test for fixed strings rather than regexes, thus the '-F' option to
grep.

if ! echo "$History" | grep -F "$ThisCommand" >/dev/null

> else
> History="$History $ThisCommand";
> fi
> done < <(cat ~/.bash_history)

Here, you don't need cat (nor process substitution) at all and could
write it more efficiently.

done < ~/.bash_history

Also, if you're using Bash and want to read the contents of a file into
a variable, you can spare using cat as well.

FOO=`cat foobar.txt`

or

FOO=$(cat foobar.txt)

becomes

FOO=$(< foobar.txt)

> Command=`Xdialog --stdout --title "Ray's Bash Manager - Select
> Command to Copy" --cancel-label "Exit" --combobox "Select a Command to
> Copy to the Clipboard." 0 60 $History`
>
> The if statement is meant to catch duplicate lines, and it does, unless
> they contain quotes in them, and then they get duplicated in the output.
> Is there some way to trap even the lines with quotes in them?

Appears the conditional in your if-statement was faulty. if itself acts
only upon the exit status of the command list given to it -- not the
output of those commands.

if [ "$FOO" = "xyz" ]

It's actually the [ command performing the expression evaluation.

if [[ "$FOO" == "xyz" ]]

Again, the [[ command evaluates the expression.

if echo "$FOO" | grep xyz

Here, a pipeline using grep is used to test for the presence of
a string.

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 12-02-2009, 12:19 AM
James Michael Fultz
 
Default Reading a variable line by line with while loop

* Ray Parrish <crp@cmc.net> [2009-12-01 16:45 -0800]:
> I have found an if statement that reduces the duplicates to only two
> pairs for the entire file. That's pretty good considering the number of
> actual duplicates there were in the file to begin with. Here is the code
> for that one -
>
> function CompressHistory {
> BashHistory=`cat ~/.bash_history`
> while read ThisCommand; do
> ThisCommand=${ThisCommand// /__};
> if [[ "$History" =~ "$ThisCommand" ]]
> then
> echo "nothing" >/dev/null
> else
> History="$History $ThisCommand";
> fi
> done <<< "$BashHistory"
> echo "$History"
> }
>
> This next bit of code has me frazzled trying to figure out the proper
> regular expression to say "if this string exists within the larger
> string". Nothing I have tried so far has worked, so here is the code,
> and maybe someone can correct my regular expression so it works to weed
> out duplicates as well.
>
> BashHistory=`cat ~/.bash_history`
> while read ThisCommand; do
> ThisCommand=${ThisCommand// /__};
> if [[ "$History" == [.]*$ThisCommand[.]* ]]

I think you may want this.

if [[ "$History" =~ .*"$ThisCommand".* ]]

Also, placing '.' inside of brackets treats it as a literal character
when used in a regex.

> then
> echo "nothing" >/dev/null
> else
> History="$History $ThisCommand";
> fi
> done <<< "$BashHistory"
>
> From what I have read the dot is supposed to match any character, and
> then the * specifies that any number of characters can appear in that
> position. I'm just realizing that would specify perhaps one character
> only, repeated to the ends of the large string before it would match.

Your description of the dot and asterisk regex metacharacters is
correct, but I think that you are confused on regular expressions and
glob characters.

Bash uses regular expressions in a [[ ... =~ ... ]] expression.
Whereas, glob expansion could be used in a [[ ... == ... ]] expression.

> I do not know how to specify that the whole front, and the whole back of
> the variable should be the same with the matching string in the middle
> of them.

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 12-02-2009, 12:41 AM
Ray Parrish
 
Default Reading a variable line by line with while loop

James Michael Fultz wrote:
> * Ray Parrish <crp@cmc.net> [2009-12-01 16:45 -0800]:
>
>> I have found an if statement that reduces the duplicates to only two
>> pairs for the entire file. That's pretty good considering the number of
>> actual duplicates there were in the file to begin with. Here is the code
>> for that one -
>>
>> function CompressHistory {
>> BashHistory=`cat ~/.bash_history`
>> while read ThisCommand; do
>> ThisCommand=${ThisCommand// /__};
>> if [[ "$History" =~ "$ThisCommand" ]]
>> then
>> echo "nothing" >/dev/null
>> else
>> History="$History $ThisCommand";
>> fi
>> done <<< "$BashHistory"
>> echo "$History"
>> }
>>
>> This next bit of code has me frazzled trying to figure out the proper
>> regular expression to say "if this string exists within the larger
>> string". Nothing I have tried so far has worked, so here is the code,
>> and maybe someone can correct my regular expression so it works to weed
>> out duplicates as well.
>>
>> BashHistory=`cat ~/.bash_history`
>> while read ThisCommand; do
>> ThisCommand=${ThisCommand// /__};
>> if [[ "$History" == [.]*$ThisCommand[.]* ]]
>>
>
> I think you may want this.
>
> if [[ "$History" =~ .*"$ThisCommand".* ]]
>
> Also, placing '.' inside of brackets treats it as a literal character
> when used in a regex.
>
>
>> then
>> echo "nothing" >/dev/null
>> else
>> History="$History $ThisCommand";
>> fi
>> done <<< "$BashHistory"
>>
>> From what I have read the dot is supposed to match any character, and
>> then the * specifies that any number of characters can appear in that
>> position. I'm just realizing that would specify perhaps one character
>> only, repeated to the ends of the large string before it would match.
>>
>
> Your description of the dot and asterisk regex metacharacters is
> correct, but I think that you are confused on regular expressions and
> glob characters.
>
> Bash uses regular expressions in a [[ ... =~ ... ]] expression.
> Whereas, glob expansion could be used in a [[ ... == ... ]] expression.
>
>
>> I do not know how to specify that the whole front, and the whole back of
>> the variable should be the same with the matching string in the middle
>> of them.
>>
OK, thanks for the tips. I did find a way to make the glob expansion
work. [not that I know a glob expansion from anything else, just
identifying it by the == you mentioned] Here is working code that now
only misses one pair of duplicates in the entire file. I do not know
what it is about that particular line that makes it not match, but it's
kinda frustrating.

Note the curly braces around the ThisCommand variable in the if
statement. Without them this expression would not catch any matches at all.

function CompressBashHistory {
BashHistory=`cat ~/.bash_history`
while read ThisCommand; do
ThisCommand=${ThisCommand// /__};
if [[ "$History" == *${ThisCommand}* ]]
then
echo "nothing" >/dev/null
else
History="$History $ThisCommand";
fi
done <<< "$BashHistory"
Command=`Xdialog --stdout --title "Ray's Bash Manager - Select
Command to Copy" --cancel-label "Exit" --combobox "Select a Command to
Copy to the Clipboard." 0 60 $History`
echo "$Command"
}

Here is the line that always gets repeated just once in the results -

SearchTerm="<div><a__href="index.html">Ray's__Link s__Home</a>{A-Za-z0-9/<>]*</div>"

Is there something in that line that prevents it from matching itself
when tested?

Thanks again, Ray Parrish

--
The Future of Technology.
http://www.rayslinks.com/The%20Future%20of%20Technology.html
Ray's Links, a variety of links to usefull things, and articles by Ray.
http://www.rayslinks.com
Writings of "The" Schizophrenic, what it's like to be a schizo, and other
things, including my poetry.
http://www.writingsoftheschizophrenic.com



--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 12-02-2009, 12:55 AM
Ray Parrish
 
Default Reading a variable line by line with while loop

NoOp wrote:
> On 12/01/2009 01:40 AM, Ray Parrish wrote:
>
>> Hello,
>>
>> I have been using while loops to read files in line by line when I want
>> to process things on a line oriented basis. Sometimes i have the set of
>> lines I want to read through already in a variable, and have to write
>> them to file before beginning to read line by line.
>>
>> I was wondering if it is possible to read a variable line by line
>> somehow with a while loop? Would it possibly be faster than using a file
>> to read from?
>>
>> I was thinking something along the lines of the following code might
>> work, but have not tested it yet.
>>
>> while read ThisLine; do
>> # process each line
>> echo "$ThisLine"
>> done < `echo "$Variable"`
>>
>> Is this something that might work, and would it be more efficient than
>> writing to, and reading from a file? Is there some other way I've missed
>> to make it work?
>>
>> Thanks for any help you can be.
>>
>> Later, Ray Parrish
>>
>>
>
> I think that your continued learning experience with bash/command
> line/scripts et al is educational (to some), but what/how are all of
> these posts from you related to Ubuntu technical support, and more
> specifically to this list?
>
Damned if I know... I just figured this was a good forum to discuss bash
scripting since it is such an inherent part of Ubuntu that without
learning it you really cannot administer your machine as easily.

Currently I'm working on a bash history manager, and bash based all
around bash manager. When I'm done with it, it will be able to
interactively run bash commands either from the history list, or from
being typed in, and will output all the normal output any good or bad
command would put out.

It will also provide a GUI interface to the settings in ~/.bashrc, and
if I can figure out how to do it, will be able to run down through the
bash history, testing each command for success, or failure, and weeding
out the lines that are not syntactically correct by parsing the stderr
output for clues.

I do have a tech support question that I'll leave for another thread,
but it is a request for a bash command to find the script responsible
for attempting to start my firewall a second time during boot up causing
a "Fail" message because it is already started. 8-) Got a one liner that
covers discovering which scripts actually run the command on start up?

Later, Ray Parrish

--
The Future of Technology.
http://www.rayslinks.com/The%20Future%20of%20Technology.html
Ray's Links, a variety of links to usefull things, and articles by Ray.
http://www.rayslinks.com
Writings of "The" Schizophrenic, what it's like to be a schizo, and other
things, including my poetry.
http://www.writingsoftheschizophrenic.com



--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 12-02-2009, 03:03 AM
James Michael Fultz
 
Default Reading a variable line by line with while loop

* Ray Parrish <crp@cmc.net> [2009-12-01 17:41 -0800]:
> James Michael Fultz wrote:
> > * Ray Parrish <crp@cmc.net> [2009-12-01 16:45 -0800]:
> >
> >> I have found an if statement that reduces the duplicates to only two
> >> pairs for the entire file. That's pretty good considering the number of
> >> actual duplicates there were in the file to begin with. Here is the code
> >> for that one -
> >>
> >> function CompressHistory {
> >> BashHistory=`cat ~/.bash_history`
> >> while read ThisCommand; do
> >> ThisCommand=${ThisCommand// /__};
> >> if [[ "$History" =~ "$ThisCommand" ]]
> >> then
> >> echo "nothing" >/dev/null
> >> else
> >> History="$History $ThisCommand";
> >> fi
> >> done <<< "$BashHistory"
> >> echo "$History"
> >> }
> >>
> >> This next bit of code has me frazzled trying to figure out the proper
> >> regular expression to say "if this string exists within the larger
> >> string". Nothing I have tried so far has worked, so here is the code,
> >> and maybe someone can correct my regular expression so it works to weed
> >> out duplicates as well.
> >>
> >> BashHistory=`cat ~/.bash_history`
> >> while read ThisCommand; do
> >> ThisCommand=${ThisCommand// /__};
> >> if [[ "$History" == [.]*$ThisCommand[.]* ]]
> >>
> >
> > I think you may want this.
> >
> > if [[ "$History" =~ .*"$ThisCommand".* ]]
> >
> > Also, placing '.' inside of brackets treats it as a literal character
> > when used in a regex.
> >
> >
> >> then
> >> echo "nothing" >/dev/null
> >> else
> >> History="$History $ThisCommand";
> >> fi
> >> done <<< "$BashHistory"
> >>
> >> From what I have read the dot is supposed to match any character, and
> >> then the * specifies that any number of characters can appear in that
> >> position. I'm just realizing that would specify perhaps one character
> >> only, repeated to the ends of the large string before it would match.
> >>
> >
> > Your description of the dot and asterisk regex metacharacters is
> > correct, but I think that you are confused on regular expressions and
> > glob characters.
> >
> > Bash uses regular expressions in a [[ ... =~ ... ]] expression.
> > Whereas, glob expansion could be used in a [[ ... == ... ]] expression.
> >
> >
> >> I do not know how to specify that the whole front, and the whole back of
> >> the variable should be the same with the matching string in the middle
> >> of them.
> >>
> OK, thanks for the tips. I did find a way to make the glob expansion
> work. [not that I know a glob expansion from anything else, just
> identifying it by the == you mentioned]

$ man 7 glob

May help to explain glob patterns.

$ man 7 regex

Likewise for regular expressions.

> Here is working code that now only misses one pair of duplicates in
> the entire file. I do not know what it is about that particular line
> that makes it not match, but it's kinda frustrating.
>
> Note the curly braces around the ThisCommand variable in the if
> statement. Without them this expression would not catch any matches at all.
>
> function CompressBashHistory {
> BashHistory=`cat ~/.bash_history`
> while read ThisCommand; do
> ThisCommand=${ThisCommand// /__};
> if [[ "$History" == *${ThisCommand}* ]]

Try quoting variable which is the fixed part of your pattern.

if [[ "$History" == *"${ThisCommand}"* ]]

> then
> echo "nothing" >/dev/null
> else
> History="$History $ThisCommand";
> fi
> done <<< "$BashHistory"
> Command=`Xdialog --stdout --title "Ray's Bash Manager - Select
> Command to Copy" --cancel-label "Exit" --combobox "Select a Command to
> Copy to the Clipboard." 0 60 $History`
> echo "$Command"
> }
>
> Here is the line that always gets repeated just once in the results -
>
> SearchTerm="<div><a__href="index.html">Ray's__Link s__Home</a>{A-Za-z0-9/<>]*</div>"
>
> Is there something in that line that prevents it from matching itself
> when tested?

I think it's the embedded regex that's being interpolated and
misinterpreted as a glob, and there may be issues with embedded quote
characters as well. I think that quoting the variable embedded in the
pattern will help.

You may already peruse Usenet; I do not know. Anyway, if you are
inclined, the group comp.unix.shell is a well of knowledge for shell
scripting.

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 

Thread Tools




All times are GMT. The time now is 08:47 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org