FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > ArchLinux > ArchLinux Pacman Development

 
 
LinkBack Thread Tools
 
Old 11-14-2008, 10:45 AM
Jan Mette
 
Default Status of "Splitting packages with makepkg"

Am Freitag 14 November 2008 04:48:27 schrieb Allan McRae:
> Aaron Griffin wrote:
> > On Thu, Nov 13, 2008 at 11:33 AM, Allan McRae <allan@archlinux.org> wrote:
> >> Aaron Griffin wrote:
> >>> On Thu, Nov 13, 2008 at 11:18 AM, Allan McRae <allan@archlinux.org>
wrote:
> >>>> Allan McRae wrote:
> >>>>> Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:
> >>>>>> Hello Allen,
> >>>>>>
> >>>>>> the prototype has a little problem in case you give -L option to
> >>>>>> makepkg
> >>>>>> as it is now:
> >>>>>>
> >>>>>> As makepkg pipes the output of the build-function into tee, the
> >>>>>> build-function gets executed in a sub-shell. Any changes of
> >>>>>> variables in
> >>>>>> context of main-shell from inside build-function will not work in
> >>>>>> that case. This leads to that we have to set the values of all
> >>>>>> split-package-variables outside of the respective build-functions.
> >>>>>
> >>>>> As I said, the implementation is still a bit away... The KDEmod guys
> >>>>> override the PKGBUILD variables the same way. I will look into what
> >>>>> they
> >>>>> for logging.
> >>>>
> >>>> Testing with this script indicates to me that everything is fine...
> >>>>
> >>>> test.sh:
> >>>> #! /bin/bash
> >>>>
> >>>> output="FOO"
> >>>>
> >>>> build() {
> >>>> output="BAR"
> >>>> echo $output
> >>>> }
> >>>>
> >>>> echo $output
> >>>> build 2>&1
> >>>> build 2>&1 | tee buildlog
> >>>>
> >>>>> ./test.sh
> >>>>
> >>>> FOO
> >>>> BAR
> >>>> BAR
> >>>>
> >>>>> cat buildlog
> >>>>
> >>>> BAR
> >>>>
> >>>>
> >>>> So what exactly are you say is wrong?
> >>>
> >>> I think the case he means is this:
> >>> test.sh:
> >>> #! /bin/bash
> >>> output="FOO"
> >>> build() {
> >>> output="BAR"
> >>> echo $output
> >>> }
> >>>
> >>> echo $output
> >>> build 2>&1 | tee buildlog
> >>> echo $output
> >>> # end
> >>>
> >>> ./test.sh
> >>> FOO
> >>> BAR
> >>> FOO
> >>>
> >>> Which, after thinking a little about it, is the way we want this to
> >>> happen, yes?
> >>
> >> I think, yes it is. From the KDEmod implementation, in the case where
> >> we are not logging (and hence no tee) we need to revert the variables to
> >> what they were before calling the build function.
> >> Anyway, this will all become more apparent as I finish implementing it
> >> (with heavy borrowing for all the patches floating around). The
> >> implementation can be a bit more complex to accommodate a simpler
> >> PKGBUILD as far as I am concerned. Few people look at makepkg, many
> >> look at PKGBUILDs.
> >
> > Hell, you might be able to cheat.
> > LOGFILE=/dev/null
> > # if -L is specified, set LOGFILE to something else
> > and always run:
> > build() 2>&1 | tee $LOGFILE
> >
> > Bam, now variables are reset automatically. Just make sure you comment
> > that nicely 8)
>
> Very nice idea. That will clean up the whole split package patches nicely.
>
> Allan


Hi,

i just want to give a small update on the KDEmod split patches. I havent
updated them since some months, but the last round of hacking on them
brought in some nice little changes.

The current patches are here:

http://www.kdemod.ath.cx/svn/branches/kde41/_buildsystem/patches/

The split patch is: splitpkg/3_makepkg-kdemod-splitpkg.patch

I have combined our original approach with the improvements from
http://bugs.archlinux.org/task/7982 and fixed all bugs i could find so far.

There are also some more changes:
- the split stuff has been moved into its own function, which makes the
changes a lot more cleaner, imho...
- the split packages will be created in subdirs of /pkg, so we dont pollute
the builddir so much
- repackaging etc works

The only drawback i have experienced so far is that the -i switch does
not work with split packages, but i guess there is a easy solution...

You can also find the patches to create separate debug packages at
this place, but these need some more "integration work" and testing.
Currently we have 2 makepkg's (makepkg, splitpkg), because i havent had
the time yet to write a unified split-debug patch that works with both
"normal" and split pkgs.

Oh, and i just want to say that i love Allans PKGBUILD prototype. Putting
the split packages into the pkgname is an awesome idea

Greetings

Jan




_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 11-14-2008, 10:57 AM
"Marc - A. Dahlhaus"
 
Default Status of "Splitting packages with makepkg"

Allan McRae schrieb:

Aaron Griffin wrote:
On Thu, Nov 13, 2008 at 11:33 AM, Allan McRae <allan@archlinux.org>
wrote:


Aaron Griffin wrote:

On Thu, Nov 13, 2008 at 11:18 AM, Allan McRae <allan@archlinux.org>
wrote:




Allan McRae wrote:



Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:



Hello Allen,

the prototype has a little problem in case you give -L option to
makepkg
as it is now:

As makepkg pipes the output of the build-function into tee, the
build-function gets executed in a sub-shell. Any changes of
variables

in
context of main-shell from inside build-function will not work
in that

case. This leads to that we have to set the values of all
split-package-variables outside of the respective build-functions.




As I said, the implementation is still a bit away... The KDEmod
guys

override the PKGBUILD variables the same way. I will look into what
they
for logging.



Testing with this script indicates to me that everything is fine...

test.sh:
#! /bin/bash

output="FOO"

build() {
output="BAR"
echo $output
}

echo $output
build 2>&1
build 2>&1 | tee buildlog





./test.sh



FOO
BAR
BAR




cat buildlog



BAR


So what exactly are you say is wrong?



I think the case he means is this:
test.sh:
#! /bin/bash
output="FOO"
build() {
output="BAR"
echo $output
}

echo $output
build 2>&1 | tee buildlog
echo $output
# end

./test.sh
FOO
BAR
FOO

Which, after thinking a little about it, is the way we want this to
happen, yes?

I think, yes it is. From the KDEmod implementation, in the case
where we
are not logging (and hence no tee) we need to revert the variables
to what

they were before calling the build function.
Anyway, this will all become more apparent as I finish implementing
it (with
heavy borrowing for all the patches floating around). The
implementation can

be a bit more complex to accommodate a simpler PKGBUILD as far as I am
concerned. Few people look at makepkg, many look at PKGBUILDs.



Hell, you might be able to cheat.
LOGFILE=/dev/null
# if -L is specified, set LOGFILE to something else
and always run:
build() 2>&1 | tee $LOGFILE

Bam, now variables are reset automatically. Just make sure you comment
that nicely 8)


Very nice idea. That will clean up the whole split package patches
nicely.


Allan


_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev

Hello,

the problem is that we need the vars to be the same as inside the
build-function as we need them outside of build-function for package
creation. The function that creates and compresses the actual package
will be executed outside of the scope of the sub-shell so you get
allways the same package-vars (the global defined) while the main-shell
creates the package for every split-package without knowing about any
variable overloading from inside subshell-scope of the build-functions...


So i think as long as we stay with this loggin implentation we can
scratch the overloading feathure alltogether and have to declare an
array inside the scope of the main-shell for each variable...

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 11-17-2008, 10:21 PM
"Marc - A. Dahlhaus"
 
Default Status of "Splitting packages with makepkg"

Allan McRae schrieb:

Aaron Griffin wrote:
On Thu, Nov 13, 2008 at 11:33 AM, Allan McRae <allan@archlinux.org>
wrote:


Aaron Griffin wrote:

On Thu, Nov 13, 2008 at 11:18 AM, Allan McRae <allan@archlinux.org>
wrote:




Allan McRae wrote:



Marc - A. Dahlhaus [ Administration | Westermann GmbH ] wrote:



Hello Allen,

the prototype has a little problem in case you give -L option to
makepkg
as it is now:

As makepkg pipes the output of the build-function into tee, the
build-function gets executed in a sub-shell. Any changes of
variables

in
context of main-shell from inside build-function will not work
in that

case. This leads to that we have to set the values of all
split-package-variables outside of the respective build-functions.




As I said, the implementation is still a bit away... The KDEmod
guys

override the PKGBUILD variables the same way. I will look into what
they
for logging.



Testing with this script indicates to me that everything is fine...

test.sh:
#! /bin/bash

output="FOO"

build() {
output="BAR"
echo $output
}

echo $output
build 2>&1
build 2>&1 | tee buildlog





./test.sh



FOO
BAR
BAR




cat buildlog



BAR


So what exactly are you say is wrong?



I think the case he means is this:
test.sh:
#! /bin/bash
output="FOO"
build() {
output="BAR"
echo $output
}

echo $output
build 2>&1 | tee buildlog
echo $output
# end

./test.sh
FOO
BAR
FOO

Which, after thinking a little about it, is the way we want this to
happen, yes?

I think, yes it is. From the KDEmod implementation, in the case
where we
are not logging (and hence no tee) we need to revert the variables
to what

they were before calling the build function.
Anyway, this will all become more apparent as I finish implementing
it (with
heavy borrowing for all the patches floating around). The
implementation can

be a bit more complex to accommodate a simpler PKGBUILD as far as I am
concerned. Few people look at makepkg, many look at PKGBUILDs.



Hell, you might be able to cheat.
LOGFILE=/dev/null
# if -L is specified, set LOGFILE to something else
and always run:
build() 2>&1 | tee $LOGFILE

Bam, now variables are reset automatically. Just make sure you comment
that nicely 8)


Very nice idea. That will clean up the whole split package patches
nicely.


Allan


_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev


Hello,

i played around a bit with the split-package implementation and got some
ideas i'd like to share.


I think we can actualy fix the case up real easy and provide more
usefull information in the logfiles if we use a container function
around the actual build and the package creation and pipe it as a whole
into tee. An adaption of current makepkg would look like this:


...
if [ 1 -lt ${#pkgname[@]} ]
then
# splitpackage build
for package in ${pkgname[@]}
do
run_build ${package} 2>&1 | tee ${logfile}
done
else
run_build build 2>&1 | tee ${logfile}
fi
...


run_build() {
... old run_build logic ...
if [ 'function' == type -t ${1} ]
then
if eval ${1}
then
tidi_install
create_package
else
# build error
fi
else
# undefined error
fi
}

We would get consitent variable overloading in the functions that
actualy need it and get the global defined variables back after each
package was build but also get all errors that we faced durig the build,
tidy and package creation in our logfile...


Another thing that could be usefull would be a separate logfile for each
splitpackage.


What about a special "prepare" function in a splitpackage case where we
build everything and use the splitpackage-build functions to just
install the content we want for a specific split-package?


this could be usefull to e.g. separate development content (headers and
static libs) from dynamic librarys...


also a option to actualy overload the contents of pkgname from the
commandline would be usefull to repackage only a singe split-package and
not all split-package inside of PKGBUILD...


In case of e.g. php, one could use overloading of pkgname content from
the commandline to rebuild only the mysql and mysqli modules if mysql
dynamic library changed its version...



Marc

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 

Thread Tools




All times are GMT. The time now is 01:24 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org