quickpkg and PKG_INSTALL_MASK
 

I got the impression from docs that PKG_INSTALL_MASK would actually
mask files out so they never get into the binary package, this doesn't
seem to happen.
Did I misunderstand? If I did, I think an MASK to do the above
would be a worthy addition to quickpkg, very useful for embedded targets to keep the pkg size down.
Also PKG_INSTALL_KEEP which would list files keep(rest is dropped) would be nice.

Oh, something else I wonder about. How does pre/post install work with
quickpkgs? Is it possible to have such scripts and then have qmerge execute
them?

Jocke

quickpkg and PKG_INSTALL_MASK
 

On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:




I got the impression from docs that PKG_INSTALL_MASK would actually

mask files out so they never get into the binary package, this doesn't

seem to happen.

Did I misunderstand? If I did, I think an MASK to do the above

would be a worthy addition to quickpkg, very useful for embedded targets to keep the pkg size down.

Also PKG_INSTALL_KEEP which would list files keep(rest is dropped) would be nice.



Oh, something else I wonder about. How does pre/post install work with

quickpkgs? Is it possible to have such scripts and then have qmerge execute

them?



Jocke





Yes, PKG_INSTALL_MASK on time of installation will mask the files defined in it.
Binary package do contain all files, even the masked files.
Kfir

quickpkg and PKG_INSTALL_MASK
 

On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
>
>
> On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
>
> I got the impression from docs that PKG_INSTALL_MASK would
> actually
> mask files out so they never get into the binary package, this
> doesn't
> seem to happen.
> Did I misunderstand? If I did, I think an MASK to do the above
> would be a worthy addition to quickpkg, very useful for
> embedded targets to keep the pkg size down.
> Also PKG_INSTALL_KEEP which would list files keep(rest is
> dropped) would be nice.
>
> Oh, something else I wonder about. How does pre/post install
> work with
> quickpkgs? Is it possible to have such scripts and then have
> qmerge execute
> them?
>
> Jocke
>
>
> Yes, PKG_INSTALL_MASK on time of installation will mask the files
> defined in it.
> Binary package do contain all files, even the masked files.
> Kfir


$PKG_INSTALL_MASK is supposed to omit the files in it's list from making
it into the binary pkgs in the first place. The idea there was to make
smaller binary pkgs for embedded devices and such (for use only with
private repos).

$INSTALL_MASK is supposed to omit the files listed in it from being
installed on the file system.

--
solar <solar@gentoo.org>
Gentoo Linux

quickpkg and PKG_INSTALL_MASK
 

solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:
>
> On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
> >
> >
> > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> > <joakim.tjernlund@transmode.se> wrote:
> >
> > I got the impression from docs that PKG_INSTALL_MASK would
> > actually
> > mask files out so they never get into the binary package, this
> > doesn't
> > seem to happen.
> > Did I misunderstand? If I did, I think an MASK to do the above
> > would be a worthy addition to quickpkg, very useful for
> > embedded targets to keep the pkg size down.
> > Also PKG_INSTALL_KEEP which would list files keep(rest is
> > dropped) would be nice.
> >
> > Oh, something else I wonder about. How does pre/post install
> > work with
> > quickpkgs? Is it possible to have such scripts and then have
> > qmerge execute
> > them?
> >
> > Jocke
> >
> >
> > Yes, PKG_INSTALL_MASK on time of installation will mask the files
> > defined in it.
> > Binary package do contain all files, even the masked files.
> > Kfir
>
>
> $PKG_INSTALL_MASK is supposed to omit the files in it's list from making
> it into the binary pkgs in the first place. The idea there was to make
> smaller binary pkgs for embedded devices and such (for use only with
> private repos).
>
> $INSTALL_MASK is supposed to omit the files listed in it from being
> installed on the file system.

Hi Solar, long time no see :)

So the current behaviour is a bug, good to know.

I am contemplating a greater problem too. Our system require we can install
multiple versions of our SW and switch between them. This is easy to do when it
comes to our own app but not when one want to upgrade core parts of the system, like
libc etc.

So I am thinking one could use --bind mounts and switch_root to solve that. Basically
one has a skeleton root FS with /bin, /lib, /usr /opt etc.
Each upgrade goes into dirs like:
bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
and --bind mounts them under the corresponding skeleton dir.

To do that one needs an way to repackage a root fs created by a bunch of different
ebuilds, some of them spanning several of the above dirs, into a package per xx_1.x.y dir.
Not sure if this can be done with the current portage/portage-utils and how, any
ideas welcome :)

Perhaps there is a better way then the above?

Jocke

quickpkg and PKG_INSTALL_MASK
 

On Mon, 2011-12-26 at 19:03 +0100, Joakim Tjernlund wrote:
> solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:
> >
> > On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
> > >
> > >
> > > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> > > <joakim.tjernlund@transmode.se> wrote:
> > >
> > > I got the impression from docs that PKG_INSTALL_MASK would
> > > actually
> > > mask files out so they never get into the binary package, this
> > > doesn't
> > > seem to happen.
> > > Did I misunderstand? If I did, I think an MASK to do the above
> > > would be a worthy addition to quickpkg, very useful for
> > > embedded targets to keep the pkg size down.
> > > Also PKG_INSTALL_KEEP which would list files keep(rest is
> > > dropped) would be nice.
> > >
> > > Oh, something else I wonder about. How does pre/post install
> > > work with
> > > quickpkgs? Is it possible to have such scripts and then have
> > > qmerge execute
> > > them?
> > >
> > > Jocke
> > >
> > >
> > > Yes, PKG_INSTALL_MASK on time of installation will mask the files
> > > defined in it.
> > > Binary package do contain all files, even the masked files.
> > > Kfir
> >
> >
> > $PKG_INSTALL_MASK is supposed to omit the files in it's list from making
> > it into the binary pkgs in the first place. The idea there was to make
> > smaller binary pkgs for embedded devices and such (for use only with
> > private repos).
> >
> > $INSTALL_MASK is supposed to omit the files listed in it from being
> > installed on the file system.
>
> Hi Solar, long time no see :)
>
> So the current behaviour is a bug, good to know.

I would not really call it a bug. *INSTALL_MASK is a portage feature
itself. quickpkg does not support all the features of portage itself.
You could/should file a feature request bug for quickpkg to add such
support. Maybe file a bug for portage-utils@ as well because I just
checked and looks like we never added support for PKG_INSTALL_MASK in
qpkg.c (c version of quickpkg)


> I am contemplating a greater problem too. Our system require we can install
> multiple versions of our SW and switch between them. This is easy to do when it
> comes to our own app but not when one want to upgrade core parts of the system, like
> libc etc.
>
> So I am thinking one could use --bind mounts and switch_root to solve that. Basically
> one has a skeleton root FS with /bin, /lib, /usr /opt etc.
> Each upgrade goes into dirs like:
> bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
> Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
> and --bind mounts them under the corresponding skeleton dir.
>
> To do that one needs an way to repackage a root fs created by a bunch of different
> ebuilds, some of them spanning several of the above dirs, into a package per xx_1.x.y dir.
> Not sure if this can be done with the current portage/portage-utils and how, any
> ideas welcome :)
>
> Perhaps there is a better way then the above?
>
> Jocke

Not sure about all this.


--
solar <solar@gentoo.org>
Gentoo Linux

quickpkg and PKG_INSTALL_MASK
 

solar <solar@gentoo.org> wrote on 2011/12/26 21:38:52:
>
> On Mon, 2011-12-26 at 19:03 +0100, Joakim Tjernlund wrote:
> > solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:
> > >
> > > On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
> > > >
> > > >
> > > > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> > > > <joakim.tjernlund@transmode.se> wrote:
> > > >
> > > > I got the impression from docs that PKG_INSTALL_MASK would
> > > > actually
> > > > mask files out so they never get into the binary package, this
> > > > doesn't
> > > > seem to happen.
> > > > Did I misunderstand? If I did, I think an MASK to do the above
> > > > would be a worthy addition to quickpkg, very useful for
> > > > embedded targets to keep the pkg size down.
> > > > Also PKG_INSTALL_KEEP which would list files keep(rest is
> > > > dropped) would be nice.
> > > >
> > > > Oh, something else I wonder about. How does pre/post install
> > > > work with
> > > > quickpkgs? Is it possible to have such scripts and then have
> > > > qmerge execute
> > > > them?
> > > >
> > > > Jocke
> > > >
> > > >
> > > > Yes, PKG_INSTALL_MASK on time of installation will mask the files
> > > > defined in it.
> > > > Binary package do contain all files, even the masked files.
> > > > Kfir
> > >
> > >
> > > $PKG_INSTALL_MASK is supposed to omit the files in it's list from making
> > > it into the binary pkgs in the first place. The idea there was to make
> > > smaller binary pkgs for embedded devices and such (for use only with
> > > private repos).
> > >
> > > $INSTALL_MASK is supposed to omit the files listed in it from being
> > > installed on the file system.
> >
> > Hi Solar, long time no see :)
> >
> > So the current behaviour is a bug, good to know.
>
> I would not really call it a bug. *INSTALL_MASK is a portage feature
> itself. quickpkg does not support all the features of portage itself.
> You could/should file a feature request bug for quickpkg to add such
> support. Maybe file a bug for portage-utils@ as well because I just
> checked and looks like we never added support for PKG_INSTALL_MASK in
> qpkg.c (c version of quickpkg)

I don't really get the difference between the two as impl. today.
What can PKG_INSTALL_MASK do that INSTALL_MASK can't(or vice versa)?

>
>
> > I am contemplating a greater problem too. Our system require we can install
> > multiple versions of our SW and switch between them. This is easy to do when it
> > comes to our own app but not when one want to upgrade core parts of the system, like
> > libc etc.
> >
> > So I am thinking one could use --bind mounts and switch_root to solve that. Basically
> > one has a skeleton root FS with /bin, /lib, /usr /opt etc.
> > Each upgrade goes into dirs like:
> > bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
> > Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
> > and --bind mounts them under the corresponding skeleton dir.
> >
> > To do that one needs an way to repackage a root fs created by a bunch of different
> > ebuilds, some of them spanning several of the above dirs, into a package per xx_1.x.y dir.
> > Not sure if this can be done with the current portage/portage-utils and how, any
> > ideas welcome :)
> >
> > Perhaps there is a better way then the above?
> >
> > Jocke
>
> Not sure about all this.

Yeah, it is a bit much :) I either case it would be nice if one
could group several bianry pkgs into one that qmerge can install. Is that possible?

Jocke

quickpkg and PKG_INSTALL_MASK
 

On Tue, Dec 27, 2011 at 11:40 AM, Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:


solar <solar@gentoo.org> wrote on 2011/12/26 21:38:52:

>

> On Mon, 2011-12-26 at 19:03 +0100, Joakim Tjernlund wrote:

> > solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:

> > >

> > > On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:

> > > >

> > > >

> > > > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund

> > > > <joakim.tjernlund@transmode.se> wrote:

> > > >

> > > > * * * * I got the impression from docs that PKG_INSTALL_MASK would

> > > > * * * * actually

> > > > * * * * mask files out so they never get into the binary package, this

> > > > * * * * doesn't

> > > > * * * * seem to happen.

> > > > * * * * Did I misunderstand? If I did, I think an MASK to do the above

> > > > * * * * would be a worthy addition to quickpkg, very useful for

> > > > * * * * embedded targets to keep the pkg size down.

> > > > * * * * Also PKG_INSTALL_KEEP which would list files keep(rest is

> > > > * * * * dropped) would be nice.

> > > >

> > > > * * * * Oh, something else I wonder about. How does pre/post install

> > > > * * * * work with

> > > > * * * * quickpkgs? Is it possible to have such scripts and then have

> > > > * * * * qmerge execute

> > > > * * * * them?

> > > >

> > > > * * * * Jocke

> > > >

> > > >

> > > > Yes, PKG_INSTALL_MASK on time of installation will mask the files

> > > > defined in it.

> > > > Binary package do contain all files, even the masked files.

> > > > Kfir

> > >

> > >

> > > $PKG_INSTALL_MASK is supposed to omit the files in it's list from making

> > > it into the binary pkgs in the first place. The idea there was to make

> > > smaller binary pkgs for embedded devices and such (for use only with

> > > private repos).

> > >

> > > $INSTALL_MASK is supposed to omit the files listed in it from being

> > > installed on the file system.

> >

> > Hi Solar, long time no see :)

> >

> > So the current behaviour is a bug, good to know.

>

> I would not really call it a bug. *INSTALL_MASK is a portage feature

> itself. quickpkg does not support all the features of portage itself.

> You could/should file a feature request bug for quickpkg to add such

> support. Maybe file a bug for portage-utils@ as well because I just

> checked and looks like we never added support for PKG_INSTALL_MASK in

> qpkg.c (c version of quickpkg)



I don't really get the difference between the two as impl. today.

What can PKG_INSTALL_MASK do that INSTALL_MASK can't(or vice versa)?


What I undersood is that* PKG_INSTALL_MASK will make smaller binary tbz file, and
INSTALL_MASK will at install time, mask files, so the tbz will have all files in it.



Kfir

>

>

> > I am contemplating a greater problem too. Our system require we can install

> > multiple versions of our SW and switch between them. This is easy to do when it

> > comes to our own app but not when one want to upgrade core parts of the system, like

> > libc etc.

> >

> > So I am thinking one could use --bind mounts and switch_root to solve that. Basically

> > one has a skeleton root FS with /bin, /lib, /usr /opt etc.

> > Each upgrade goes into dirs like:

> > * bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/

> > Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use

> > and --bind mounts them under the corresponding skeleton dir.

> >

> > To do that one needs an way to repackage a root fs created by a bunch of different

> > ebuilds, some of them spanning several of the above dirs, into a package per xx_1.x.y dir.

> > Not sure if this can be done with the current portage/portage-utils and how, any

> > ideas welcome :)

> >

> > Perhaps there is a better way then the above?

> >

> > *Jocke

>

> Not sure about all this.



Yeah, it is a bit much :) I either case it would be nice if one

could group several bianry pkgs into one that qmerge can install. Is that possible?



*Jocke

quickpkg and PKG_INSTALL_MASK
 

solar <solar@gentoo.org> wrote on 2011/12/26 21:38:52:
>
> On Mon, 2011-12-26 at 19:03 +0100, Joakim Tjernlund wrote:
> > solar <solar@gentoo.org> wrote on 2011/12/25 18:52:52:
> > >
> > > On Sun, 2011-12-25 at 11:34 +0200, Kfir Lavi wrote:
> > > >
> > > >
> > > > On Fri, Dec 23, 2011 at 3:23 PM, Joakim Tjernlund
> > > > <joakim.tjernlund@transmode.se> wrote:
> > > >
> > > > I got the impression from docs that PKG_INSTALL_MASK would
> > > > actually
> > > > mask files out so they never get into the binary package, this
> > > > doesn't
> > > > seem to happen.
> > > > Did I misunderstand? If I did, I think an MASK to do the above
> > > > would be a worthy addition to quickpkg, very useful for
> > > > embedded targets to keep the pkg size down.
> > > > Also PKG_INSTALL_KEEP which would list files keep(rest is
> > > > dropped) would be nice.
> > > >
> > > > Oh, something else I wonder about. How does pre/post install
> > > > work with
> > > > quickpkgs? Is it possible to have such scripts and then have
> > > > qmerge execute
> > > > them?
> > > >
> > > > Jocke
> > > >
> > > >
> > > > Yes, PKG_INSTALL_MASK on time of installation will mask the files
> > > > defined in it.
> > > > Binary package do contain all files, even the masked files.
> > > > Kfir
> > >
> > >
> > > $PKG_INSTALL_MASK is supposed to omit the files in it's list from making
> > > it into the binary pkgs in the first place. The idea there was to make
> > > smaller binary pkgs for embedded devices and such (for use only with
> > > private repos).
> > >
> > > $INSTALL_MASK is supposed to omit the files listed in it from being
> > > installed on the file system.
> >
> > Hi Solar, long time no see :)
> >
> > So the current behaviour is a bug, good to know.
>
> I would not really call it a bug. *INSTALL_MASK is a portage feature
> itself. quickpkg does not support all the features of portage itself.
> You could/should file a feature request bug for quickpkg to add such
> support. Maybe file a bug for portage-utils@ as well because I just
> checked and looks like we never added support for PKG_INSTALL_MASK in
> qpkg.c (c version of quickpkg)

I did file a bug, got some answer too. See bug 396315

Jocke

quickpkg and PKG_INSTALL_MASK
 

On 26/12/2011 18:03, Joakim Tjernlund wrote:

I am contemplating a greater problem too. Our system require we can install
multiple versions of our SW and switch between them. This is easy to do when it
comes to our own app but not when one want to upgrade core parts of the system, like
libc etc.

So I am thinking one could use --bind mounts and switch_root to solve that. Basically
one has a skeleton root FS with /bin, /lib, /usr /opt etc.
Each upgrade goes into dirs like:
bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
and --bind mounts them under the corresponding skeleton dir.



I'm doing something like this using aufs. The performance seems "not
bad", but you get a couple MB or so memory hit. (I'm using squashfs as
well, so unsure which causes the main memory increase).


I think if the device is reasonably beefy then you should have no
problem with aufs or some similar overlay filesystem (I think some new
one got added to kernel recently?). Failing that, what about good old
"cp -a" to merge all the input dirs? Takes some time and disk space,
but nice and simple?


Good luck

Ed W

quickpkg and PKG_INSTALL_MASK
 

Ed W <lists@wildgooses.com> wrote on 2012/01/23 19:43:49:

> From: Ed W <lists@wildgooses.com>
> To: gentoo-embedded@lists.gentoo.org
> Date: 2012/01/23 19:44
> Subject: Re: [gentoo-embedded] quickpkg and PKG_INSTALL_MASK
>
> On 26/12/2011 18:03, Joakim Tjernlund wrote:
> > I am contemplating a greater problem too. Our system require we can install
> > multiple versions of our SW and switch between them. This is easy to do when it
> > comes to our own app but not when one want to upgrade core parts of the system, like
> > libc etc.
> >
> > So I am thinking one could use --bind mounts and switch_root to solve that. Basically
> > one has a skeleton root FS with /bin, /lib, /usr /opt etc.
> > Each upgrade goes into dirs like:
> > bin_1.x.y/ usr_1.x.y/ lib_1.x.y/ opt_1.x.y/ sbin_1.x.y/
> > Then, from an initramfs, one selects which of xxx_1.x.y dir one wants to use
> > and --bind mounts them under the corresponding skeleton dir.
> >
>
> I'm doing something like this using aufs. The performance seems "not
> bad", but you get a couple MB or so memory hit. (I'm using squashfs as
> well, so unsure which causes the main memory increase).

hmm, not sure how aufs would work out. One would like to permanently delete the
old sw at some point and I don't see how, perhaps I misunderstand something?

How does aufs do w.r.t power cuts? We need a reliable FS w.r.t power cuts. In the
past we have used JFFS2 on NOR flash and that has worked really well. Now
we need to switch to NAND due to size requirements.

>
> I think if the device is reasonably beefy then you should have no
> problem with aufs or some similar overlay filesystem (I think some new
> one got added to kernel recently?). Failing that, what about good old
> "cp -a" to merge all the input dirs? Takes some time and disk space,
> but nice and simple?

Not sure I get the "cp -a" idea. You are suggesting to create a new root FS by
copying the non changed dirs from the previous version and combine these with
the new SW?

Jocke