Google Summer of Code: Bash AST and grammar
 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello portage developers

My name is Nathan Eloe, and I'm working on a Google Summer of Code
project in which I'm writing a library that will create an Abstract
Syntax Tree of the bash grammar. In essence I am recreating the bash
grammar and structuring it as an AST.

I am writing to you because you have shown interest in
this project. I have attached a bash script which demonstrates the
supported functionality of the grammar thus far and the resulting AST.
I would greatly appreciate input on the tree as well as insight into
missing functionality you might see.

Thanks,
Nathan Eloe
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkxG+tAACgkQFpoRlVgtqKYoRgCfUXxQBBY9J2 zixrthtaRK+Yej
NesAnR5/69t68J/nlswiw6LEvXebUENi
=MuDw
-----END PGP SIGNATURE-----
#!/bin/bash
# A "featureful" script that demonstrates all the functionality of the parser

#############################################
#Copyright 2010 Nathan Eloe
#
#This file is part of libbash.
#
#libbash is free software: you can redistribute it and/or modify
#it under the terms of the GNU General Public License as published by
#the Free Software Foundation, either version 2 of the License, or
#(at your option) any later version.
#
#libbash is distributed in the hope that it will be useful,but WITHOUT ANY WARRANTY; without even the implied warranty of
#MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#GNU General Public License for more details.
#
#You should have received a copy of the GNU General Public License
#along with libbash. If not, see <http://www.gnu.org/licenses/>.
##############################################

#comment
#demonstrates expansions in strings both double- and non-quoted
function lots_o_echo() {
echo "The number of tests that have failed: $failedtests"
echo '$failedtests'
echo $failedtests
}

do_some_arith() {
(( 5*4 ))
(( 5**4 ))
(( $failedtests+5/4 ))
(( $z+-3 ))
}

function arrays() (
asdf=(a b c d)
echo ${asdf[3]}
foo=(`echo 6` b c d)
arr[foo]=3
bar=(a b [5]=c);
)

echo {a,b}
echo {a..d}
echo {{a,b},c,d}
echo a{b,c}

$(echo foobar)
ls |grep gunit >> filelist

case `echo asdf` in
gz)
echo yay
;;
bzip)
echo three
;;
*) echo woo
;;
esac

for each in `ls |grep log`; do
echo $each
cat each
done

for ((5+3;6+2;3+1)); do echo yay; done

select each in `ls |grep output`; do
echo asdf 2> /dev/null
done

if echo yay2; then
echo yay
fi

until [[ -a this/is.afile ]]; do
touch this/is.afile
done

while [ -n foobar ]; do
echo "file found"
done

if test 5 -eq 6; then
echo "something's wrong"
fi

echo this command has multiple arguments

wc <(cat /usr/share/dict/linux.words)

cd build && ./configure && make && make_install || echo fail

cd /usr/bin; ls -al|grep more

asdf=parameters
${asdf:-foo}
${asdf:8}
${!asdf*}
${!asdf@}
${#foo}
${replaice/with/pattern}
${asdf#bar}
${asdf##bar}
${asdf%bar}
${asdf%bar}
$1 $@ $*
$?
${PV//./_}
${PV/#foo/bar}
${PV/%foo/bar}

MY_PN=${PN/asterisk-}

cat asdf |grep three 2>&1 > /dev/null
echo asdf >> APPEND
echo cat <<<word

Google Summer of Code: Bash AST and grammar
 

On 07/21/10 15:49, Nathan Eloe wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello portage developers

My name is Nathan Eloe, and I'm working on a Google Summer of Code
project in which I'm writing a library that will create an Abstract
Syntax Tree of the bash grammar. In essence I am recreating the bash
grammar and structuring it as an AST.

I am writing to you because you have shown interest in
this project. I have attached a bash script which demonstrates the
supported functionality of the grammar thus far and the resulting AST.


Nice, you have covered quite a few constructs alraedy.



I would greatly appreciate input on the tree


I'm unsure if such a big file and tree is good to handle.
I propose to split this test suite to many small cases.



as well as insight into
missing functionality you might see.


A few things I haven't seen, yet:

# Adjacent changes in quotation
echo "abc"'"'"def" # Makes abc"def

# Escape sequences like ..
echo "abc"def"

# Invocation from ibnside strings
echo "foo $(echo foo), bar `echo bar`"

# Heredocs (and their variations ..)
cat <<EOF >/dev/stdout
1
2
EOF

# Construct of $((..))
# Refs to variable withzout "$" upfront
z=3; echo $((z+3))

Best,



Sebastian

Google Summer of Code: Bash AST and grammar
 

On 07/21/10 16:56, Sebastian Pipping wrote:

A few things I haven't seen, yet:

# Adjacent changes in quotation
echo "abc"'"'"def" # Makes abc"def

# Escape sequences like ..
echo "abc"def"

# Invocation from ibnside strings
echo "foo $(echo foo), bar `echo bar`"

# Heredocs (and their variations ..)
cat <<EOF >/dev/stdout
1
2
EOF

# Construct of $((..))
# Refs to variable withzout "$" upfront
z=3; echo $((z+3))


An important one I forgot:

# Line continuation
echo one
two

Potentially hase quite a few sub cases ..

Best,



Sebastian

Google Summer of Code: Bash AST and grammar
 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 7/21/10 10:25 AM, Sebastian Pipping wrote:
> On 07/21/10 16:56, Sebastian Pipping wrote:
>> A few things I haven't seen, yet:
>>
>> # Adjacent changes in quotation
>> echo "abc"'"'"def" # Makes abc"def
>>
>> # Escape sequences like ..
>> echo "abc"def"
>>
>> # Invocation from ibnside strings
>> echo "foo $(echo foo), bar `echo bar`"

I'll test those, though I think the only one that I haven't even thought
about was the escape sequences

>> # Heredocs (and their variations ..)
>> cat <<EOF >/dev/stdout
>> 1
>> 2
>> EOF
>>

Heredocs/strings are... present but broken. The problem is they just
consume strings like crazy. Perhaps with some semantic predicates I can
get it to stop at the end. For now though it would just keep consuming
till the end of the file

>> # Construct of $((..))
>> # Refs to variable withzout "$" upfront
>> z=3; echo $((z+3))
>
> An important one I forgot:
>
> # Line continuation
> echo one
> two
>

Both of these have yet to be implemented, though doing so should be trivial.

Thanks for the input! Any and all input I can get is helpful.

Nathan
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.14 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAkxHH5MACgkQFpoRlVgtqKbnfwCbBdgYaqZfSL xa7GTgrtV1wZfg
e94An3HkOPUE6BVQ1YsyeogJ92ZsRK6z
=5ezP
-----END PGP SIGNATURE-----