FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian dpkg

 
 
LinkBack Thread Tools
 
Old 03-02-2011, 03:35 PM
Neil Williams
 
Default Bootstrapping a foreign architecture with multiarch

On Wed, 2 Mar 2011 15:06:11 +0100
Raphael Hertzog <hertzog@debian.org> wrote:

> 1/ Anywhere where the user might specify a package name, he should be
> able to specifiy "package:arch" to cope with Multi-Arch: same
> packages that can be co-installed (e.g. libc6:i386 and libc6:amd64 on
> the same system). Those package names can also appear in outputs of
> dpkg-query commands (dpkg -l, dpkg -S, dpkg-query -W).

Multistrap can support that, it simply passes the package:arch to apt
and the package listing is set by the user, so this shouldn't be a
problem.

> 2/ Any program that parses /var/lib/dpkg/status with the assumption
> that there's only one entry with "Package: foo" is wrong. Uniqueness
> is now only guaranteed on the tuple (Package, Architecture).

I'll check other packages but multistrap is unaffected by the
non-uniqueness in /var/lib/dpkg/status and other tools which might do
this won't survive the implementation of multi-arch because they are
based around the current dpkg-cross method. There might need to be
temporary fixes. Most already use dpkg-query for this purpose.

> In general parsing the status file should not be done, instead you
> should use dpkg-query.

Multistrap cannot use dpkg --unpack because multistrap is primarily
working with a foreign architecture and therefore cannot allow the
preinst to run. Therefore, multistrap needs to *create* /var/lib/dpkg/*
from the various files exposed via dpkg -e. Multistrap needs to remain
architecture-neutral - it will run 'dpkg --configure -a' inside the
chroot if the architecture is native but the package processing and the
creation of the data in /var/lib/dpkg/ MUST be architecture-neutral.
Currently, dpkg does not support this, so multistrap has to do it
instead.

Multistrap cannot use dpkg-query because, at the point where this needs
to be done, dpkg hasn't installed anything (in the directory which will
become the root filesystem).

Multistrap doesn't *parse* the status file, it *creates* it.

Please don't assert that this is wrong until dpkg allows -unpack to
work for foreign architectures by not running the preinst scripts.
Preinst scripts need to be handled separately once the system can boot
(either into a rescue environment and using chroot or directly).

Note, this isn't actually affected by Multi-Arch - the foreign packages
are not being installed in a Multi-Arch way, they are not being
installed alongside native binaries. The foreign packages are
downloaded and unpacked into a new, clean, subdirectory. This
sub-directory *might* actually include packages from more than one
architecture (neither of which needs to be the same as the external
architecture), once Multi-Arch is in place, but the primary purpose is
to create a root filesystem of an arbitrary architecture in an arbitrary
directory with no direct relationship to the external system, other
than that the external apt executable is used to calculate the
dependencies and the external dpkg executable is used to extract the
package data (using dpkg -X and dpkg -e).

I'd love to be able to use dpkg --unpack with a directory but first,
dpkg must handle *not* running the preinst scripts. Not just when the
architecture differs but unconditionally so that multistrap can retain
architecture-neutrality which is very important when trying to debug
why a particular root filesystem fails to boot or fails to configure.

Running the maintainer scripts needs to be a separate process, callable
separately, from the creation of the dpkg metadata in /var/lib/dpkg/.
Unpack should simply put the files from the package into a nominated
directory, create the dpkg metadata in a path which starts with that
directory and NOT run the maintainer scripts, ever. Then, foreign
bootstrapping can be trivial.

What will happen with maintainer scripts of Multi-Arch packages when
installed alongside native packages? Why are these being retained if
they cannot be executed?

> 3/ Any program that assumes the current layout of control files
> (/var/lib/dpkg/info/<package>.<something>) will be broken (at
> least for some packages) since the layout will change to support
> Multi-Arch: same packages that can be co-installed.
>
> You should use "dpkg-query --control-path <package> <something>" to
> retrieve the path of the file. This has been introduced in dpkg
> 1.15.4 and is thus in squeeze already.

1. Which packages currently show this behaviour? dpkg doesn't show any
change in the --control-path setting for dpkg itself.

2. What are the changes in /var/lib/dpkg/info/* ?

For Multistrap, every file in /var/lib/dpkg/ has to be created by
multistrap, based on the data obtained from dpkg -e (basically the
DEBIAN/ content). This data needs only to be sufficient that dpkg can
correctly configure the packages once dpkg itself is able to be
executed inside the new filesystem.

As above, dpkg-query is useless in this situation - it has no data
in the relevant location unless multistrap creates it.

> Do you know packages that will be affected by the above problems?
> Please file a bug and usertag it with this command:
> $ bts user debian-dpkg@lists.debian.org . usertag $bug + multiarch

Done. #616111

Bug CC'd.

--


Neil Williams
=============
http://www.linux.codehelp.co.uk/
 
Old 03-03-2011, 06:31 AM
Raphael Hertzog
 
Default Bootstrapping a foreign architecture with multiarch

Hi,

(I'm not commenting on the approach used by multistrap in general, I just
reply to your questions)

On Wed, 02 Mar 2011, Neil Williams wrote:
> What will happen with maintainer scripts of Multi-Arch packages when
> installed alongside native packages? Why are these being retained if
> they cannot be executed?

The scripts are executed. That just means that Multi-Arch: same package
should avoid use of binary executables as maintainer scripts.

But a maintainer script implemented as a shell script can use any program
whose presence is guaranteed by the dependencies, just as usual, whatever
its arch.

> > 3/ Any program that assumes the current layout of control files
> > (/var/lib/dpkg/info/<package>.<something>) will be broken (at
> > least for some packages) since the layout will change to support
> > Multi-Arch: same packages that can be co-installed.
> >
> > You should use "dpkg-query --control-path <package> <something>" to
> > retrieve the path of the file. This has been introduced in dpkg
> > 1.15.4 and is thus in squeeze already.
>
> 1. Which packages currently show this behaviour? dpkg doesn't show any
> change in the --control-path setting for dpkg itself.

The change will only take place once we roll out multiarch... if you use
one of my snapshot you already have a different value here (it embeds the
architecture string). But that's not the definitive layout, see below.

> 2. What are the changes in /var/lib/dpkg/info/* ?

Multi-Arch: same packages will use
/var/lib/dpkg/info/<package>:<arch>.<something>

> For Multistrap, every file in /var/lib/dpkg/ has to be created by
> multistrap, based on the data obtained from dpkg -e (basically the
> DEBIAN/ content). This data needs only to be sufficient that dpkg can
> correctly configure the packages once dpkg itself is able to be
> executed inside the new filesystem.

It's sufficient, you just need to know the value of the Multi-Arch field.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110303073151.GF6036@rivendell.home.ouaza.com">ht tp://lists.debian.org/20110303073151.GF6036@rivendell.home.ouaza.com
 
Old 04-01-2011, 09:31 PM
Neil Williams
 
Default Bootstrapping a foreign architecture with multiarch

On Thu, 3 Mar 2011 08:31:51 +0100
Raphael Hertzog <hertzog@debian.org> wrote:

> > > 3/ Any program that assumes the current layout of control files
> > > (/var/lib/dpkg/info/<package>.<something>) will be broken (at
> > > least for some packages) since the layout will change to support
> > > Multi-Arch: same packages that can be co-installed.
> > >
> > > You should use "dpkg-query --control-path <package> <something>" to
> > > retrieve the path of the file. This has been introduced in dpkg
> > > 1.15.4 and is thus in squeeze already.

dpkg cannot be executed inside the chroot because it has not
necessarily been unpacked at this stage, it certainly has not been
configured. (dpkg is running from outside the chroot).

Yet dpkg-query won't work with --admindir to operate from outside:

$ dpkg-query --admindir /home/neil/code/chroots/natty-amd64/var/lib/dpkg --control-path libc6 postinst
$

Inside:
/home/test# dpkg-query --control-path libc6 postinst
/var/lib/dpkg/info/libc6:amd64.postinst

/home/test# dpkg-query --admindir /var/lib/dpkg --control-path libc6 control
/var/lib/dpkg/info/libc6:amd64.control

$ ls -l /home/neil/code/chroots/natty-amd64/var/lib/dpkg/status
-rw-r--r-- 1 root root 261652 Apr 1 21:46 /home/neil/code/chroots/natty-amd64/var/lib/dpkg/status

dpkg outside the chroot is 1.15.8.10

Do you want a bug report for that one?

Anyways:

> Multi-Arch: same packages will use
> /var/lib/dpkg/info/<package>:<arch>.<something>
>
> > For Multistrap, every file in /var/lib/dpkg/ has to be created by
> > multistrap, based on the data obtained from dpkg -e (basically the
> > DEBIAN/ content). This data needs only to be sufficient that dpkg can
> > correctly configure the packages once dpkg itself is able to be
> > executed inside the new filesystem.
>
> It's sufficient, you just need to know the value of the Multi-Arch field.

True. I've got a change now which uses dpkg -f to check the Multi-Arch
field of the actual downloaded .deb (independent of arch) and use that
to detect Multi-Arch: same, then lookup Architecture in the same way,
add that to the filename of the created maintainer script etc.
for /path/to/chroot/var/lib/dpkg/info/*

--


Neil Williams
=============
http://www.linux.codehelp.co.uk/
 
Old 04-01-2011, 09:43 PM
Neil Williams
 
Default Bootstrapping a foreign architecture with multiarch

On Fri, 1 Apr 2011 22:31:07 +0100
Neil Williams <codehelp@debian.org> wrote:

> Inside:
> /home/test# dpkg-query --control-path libc6 postinst
> /var/lib/dpkg/info/libc6:amd64.postinst
>
> /home/test# dpkg-query --admindir /var/lib/dpkg --control-path libc6 control
> /var/lib/dpkg/info/libc6:amd64.control
>
> $ ls -l /home/neil/code/chroots/natty-amd64/var/lib/dpkg/status
> -rw-r--r-- 1 root root 261652 Apr 1 21:46 /home/neil/code/chroots/natty-amd64/var/lib/dpkg/status
>
> dpkg outside the chroot is 1.15.8.10
>
> Do you want a bug report for that one?

Further to this, --control-path doesn't include 'list' but
$pkg:$arch.list files are created.

--


Neil Williams
=============
http://www.linux.codehelp.co.uk/
 
Old 04-02-2011, 03:47 AM
Guillem Jover
 
Default Bootstrapping a foreign architecture with multiarch

Hi!

On Fri, 2011-04-01 at 22:31:07 +0100, Neil Williams wrote:
> dpkg cannot be executed inside the chroot because it has not
> necessarily been unpacked at this stage, it certainly has not been
> configured. (dpkg is running from outside the chroot).

(An Essential package does not need to be configured to be functional.)

> Yet dpkg-query won't work with --admindir to operate from outside:
>
> $ dpkg-query --admindir /home/neil/code/chroots/natty-amd64/var/lib/dpkg --control-path libc6 postinst
> $
>
> Inside:
> /home/test# dpkg-query --control-path libc6 postinst
> /var/lib/dpkg/info/libc6:amd64.postinst
>
> /home/test# dpkg-query --admindir /var/lib/dpkg --control-path libc6 control
> /var/lib/dpkg/info/libc6:amd64.control
>
> $ ls -l /home/neil/code/chroots/natty-amd64/var/lib/dpkg/status
> -rw-r--r-- 1 root root 261652 Apr 1 21:46 /home/neil/code/chroots/natty-amd64/var/lib/dpkg/status
>
> dpkg outside the chroot is 1.15.8.10
>
> Do you want a bug report for that one?

No, because the external dpkg has no knowlegde of multiarch and the
new db layout, in addition you will trash the db in case there's more
than one “Multi-Arch: same” package instance installed when using an
older dpkg.

> Anyways:
>
> > Multi-Arch: same packages will use
> > /var/lib/dpkg/info/<package>:<arch>.<something>
> >
> > > For Multistrap, every file in /var/lib/dpkg/ has to be created by
> > > multistrap, based on the data obtained from dpkg -e (basically the
> > > DEBIAN/ content). This data needs only to be sufficient that dpkg can
> > > correctly configure the packages once dpkg itself is able to be
> > > executed inside the new filesystem.
> >
> > It's sufficient, you just need to know the value of the Multi-Arch field.
>
> True. I've got a change now which uses dpkg -f to check the Multi-Arch
> field of the actual downloaded .deb (independent of arch) and use that
> to detect Multi-Arch: same, then lookup Architecture in the same way,
> add that to the filename of the created maintainer script etc.
> for /path/to/chroot/var/lib/dpkg/info/*

If you are not going to install multiple instances of “Multi-Arch: same”
packages, then just keep writting the files as you are currently
doing, the next dpkg native run on that chroot image will
automatically upgrade the db layout. Which seems to be the less
onerous way to handle this. Otherwise you'll have to duplicate the
db layout handling, including format file, etc, and at that point I'd
say you are on your own.

On Fri, 2011-04-01 at 22:43:35 +0100, Neil Williams wrote:
> Further to this, --control-path doesn't include 'list' but
> $pkg:$arch.list files are created.

This was done on purpose, .list and .conffiles are internal files, the
correct interfaces to access the information is dpkg-query.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110402034747.GA29794@gaara.hadrons.org">http://lists.debian.org/20110402034747.GA29794@gaara.hadrons.org
 

Thread Tools




All times are GMT. The time now is 06:39 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org