Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian dpkg (http://www.linux-archive.org/debian-dpkg/)
-   -   Dpkg/Shlibs.pm: multiarch search paths (http://www.linux-archive.org/debian-dpkg/503550-dpkg-shlibs-pm-multiarch-search-paths.html)

Steve Langasek 03-21-2011 06:45 AM

Dpkg/Shlibs.pm: multiarch search paths
 
Hi guys,

Another patch for multiarch support. The need for this was discovered when
trying to bootstrap a cross-toolchain against a multiarchified
eglibc-source.



We should explicitly prepend the appropriate multiarch paths to our library
search path. These would be picked up later on anyway in the case of a
native build, but for, e.g., bootstrapping a cross-toolchain the needed
multiarch paths aren't going to be found in ld.so.conf.

---
debian/changelog | 5 +++++
scripts/Dpkg/Shlibs.pm | 13 +++++++++++--
2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 2d0883c..67fbae1 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -104,6 +104,11 @@ dpkg (1.16.0) UNRELEASED; urgency=low
DEB_BUILD_MULTIARCH, that return the "ideal" GNU triplet for each
architecture which should be used as the path component for library
installation.
+ * Dpkg/Shlibs.pm: we should explicitly prepend the appropriate multiarch
+ paths to our library search path. These would be picked up later on
+ anyway in the case of a native build, but for, e.g., bootstrapping a
+ cross-toolchain the needed multiarch paths aren't going to be found in
+ ld.so.conf.

[ Updated programs translations ]
* German (Sven Joachim).
diff --git a/scripts/Dpkg/Shlibs.pm b/scripts/Dpkg/Shlibs.pm
index 95ef4fe..95950e1 100644
--- a/scripts/Dpkg/Shlibs.pm
+++ b/scripts/Dpkg/Shlibs.pm
@@ -29,7 +29,8 @@ use Dpkg::Gettext;
use Dpkg::ErrorHandling;
use Dpkg::Shlibs::Objdump;
use Dpkg::Path qw(resolve_symlink canonpath);
-use Dpkg::Arch qw(debarch_to_gnutriplet get_build_arch get_host_arch);
+use Dpkg::Arch qw(debarch_to_gnutriplet debarch_to_multiarch
+ get_build_arch get_host_arch);

use constant DEFAULT_LIBRARY_PATH =>
qw(/lib /usr/lib /lib32 /usr/lib32 /lib64 /usr/lib64
@@ -39,6 +40,10 @@ use constant DEFAULT_LIBRARY_PATH =>
# cross-build or a build of a cross-compiler
my @crosslibrarypaths;
my $crossprefix;
+# And when we're not building a cross-compiler, be sure to pick up the
+# multiarch paths
+my @multiarchpaths;
+my $multiarch;
# Detect cross compiler builds
if ($ENV{GCC_TARGET}) {
$crossprefix = debarch_to_gnutriplet($ENV{GCC_TARGET});
@@ -51,6 +56,7 @@ if ($ENV{DEB_TARGET_GNU_TYPE} and
# host for normal cross builds.
if (get_build_arch() ne get_host_arch()) {
$crossprefix = debarch_to_gnutriplet(get_host_arch());
+ $multiarch = debarch_to_multiarch(get_host_arch());
}
# Define list of directories containing crossbuilt libraries
if ($crossprefix) {
@@ -58,8 +64,11 @@ if ($crossprefix) {
"/$crossprefix/lib32", "/usr/$crossprefix/lib32",
"/$crossprefix/lib64", "/usr/$crossprefix/lib64";
}
+if ($multiarch) {
+ push @multiarchpaths, "/lib/$multiarch", "/usr/lib/$multiarch";
+}

-our @librarypaths = (DEFAULT_LIBRARY_PATH, @crosslibrarypaths);
+our @librarypaths = (@multiarchpaths, DEFAULT_LIBRARY_PATH, @crosslibrarypaths);

# Update library paths with LD_LIBRARY_PATH
if ($ENV{LD_LIBRARY_PATH}) {
--
1.7.1


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1300693517-11497-1-git-send-email-steve.langasek@linaro.org">http://lists.debian.org/1300693517-11497-1-git-send-email-steve.langasek@linaro.org

Raphael Hertzog 03-21-2011 07:55 AM

Dpkg/Shlibs.pm: multiarch search paths
 
Hi,

On Mon, 21 Mar 2011, Steve Langasek wrote:
> Another patch for multiarch support. The need for this was discovered when
> trying to bootstrap a cross-toolchain against a multiarchified
> eglibc-source.

Ok, the support for cross-compilation in dpkg-shlibdeps was contributed
by Emdebian so I'll ask them for their opinion.

Not knowing much about cross-compilation I had started with a sligthly
different approach than you, mimicking exactly what was done for
cross-compilation support.

Neil (or anyone else knowledgable on the topic), can you review the
patches and tell us what's really needed and most logical according to
you?

Steve's patch:
- adds the multi-arch path at the start of the search paths
- adds them only for stantard cross-compilation (host != build) but not
for the special cases used to detect a cross-compiler build

My patch:
- adds the multi-arch path before the cross-arch patch but after the
standard search path
- add them in all cases where the cross-compilation paths are used

Putting the multi-arch paths after the default path should not matter as
dpkg-shlibdeps skips libraries of non-matching architecture.

For the case of building a cross-compiler, Steve's experience tends to
prove that adding the multi-arch path is not needed... but I don't see why
it should be treated differently than the cross-arch paths.

Can you enlighten us?

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)
commit 980b01597405110a1769d069b5708651de1f241a
Author: Raphaël Hertzog <hertzog@debian.org>
Date: Mon Mar 21 09:36:29 2011 +0100

dpkg-shlibdeps: look into multi-arch paths

dpkg-shlibdeps should look into multi-arch paths when cross-compiling or
building a cross-compiler.

Reported-by: Steve Langasek <steve.langasek@linaro.org>

diff --git a/debian/changelog b/debian/changelog
index 2d0883c..476b20c 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -92,6 +92,8 @@ dpkg (1.16.0) UNRELEASED; urgency=low
spotting it.
* Use the correct mtime when installing a file with statoverrides.
LP: #739179
+ * Improve dpkg-shlibdeps to look into multi-arch paths when
+ cross-compiling or building a cross-compiler.

[ Jonathan Nieder ]
* Remove support for use of synchronous sync(2), due to its pernicious
diff --git a/scripts/Dpkg/Shlibs.pm b/scripts/Dpkg/Shlibs.pm
index 95ef4fe..17354bb 100644
--- a/scripts/Dpkg/Shlibs.pm
+++ b/scripts/Dpkg/Shlibs.pm
@@ -29,7 +29,8 @@ use Dpkg::Gettext;
use Dpkg::ErrorHandling;
use Dpkg::Shlibs::Objdump;
use Dpkg::Path qw(resolve_symlink canonpath);
-use Dpkg::Arch qw(debarch_to_gnutriplet get_build_arch get_host_arch);
+use Dpkg::Arch qw(debarch_to_gnutriplet get_build_arch get_host_arch
+ gnutriplet_to_multiarch debarch_to_multiarch);

use constant DEFAULT_LIBRARY_PATH =>
qw(/lib /usr/lib /lib32 /usr/lib32 /lib64 /usr/lib64
@@ -38,23 +39,27 @@ use constant DEFAULT_LIBRARY_PATH =>
# Adjust set of directories to consider when we're in a situation of a
# cross-build or a build of a cross-compiler
my @crosslibrarypaths;
-my $crossprefix;
+my ($crossprefix, $multiarch);
# Detect cross compiler builds
if ($ENV{GCC_TARGET}) {
$crossprefix = debarch_to_gnutriplet($ENV{GCC_TARGET});
+ $multiarch = debarch_to_multiarch($ENG{GCC_TARGET});
}
if ($ENV{DEB_TARGET_GNU_TYPE} and
($ENV{DEB_TARGET_GNU_TYPE} ne $ENV{DEB_BUILD_GNU_TYPE}))
{
$crossprefix = $ENV{DEB_TARGET_GNU_TYPE};
+ $multiarch = gnutriplet_to_multiarch($ENV{DEB_TARGET_GNU_TYPE}) ;
}
# host for normal cross builds.
if (get_build_arch() ne get_host_arch()) {
$crossprefix = debarch_to_gnutriplet(get_host_arch());
+ $multiarch = debarch_to_multiarch(get_host_arch());
}
# Define list of directories containing crossbuilt libraries
if ($crossprefix) {
- push @crosslibrarypaths, "/$crossprefix/lib", "/usr/$crossprefix/lib",
+ push @crosslibrarypaths, "/lib/$multiarch", "/usr/lib/$multiarch",
+ "/$crossprefix/lib", "/usr/$crossprefix/lib",
"/$crossprefix/lib32", "/usr/$crossprefix/lib32",
"/$crossprefix/lib64", "/usr/$crossprefix/lib64";
}
Hi guys,

Another patch for multiarch support. The need for this was discovered when
trying to bootstrap a cross-toolchain against a multiarchified
eglibc-source.



We should explicitly prepend the appropriate multiarch paths to our library
search path. These would be picked up later on anyway in the case of a
native build, but for, e.g., bootstrapping a cross-toolchain the needed
multiarch paths aren't going to be found in ld.so.conf.

---
debian/changelog | 5 +++++
scripts/Dpkg/Shlibs.pm | 13 +++++++++++--
2 files changed, 16 insertions(+), 2 deletions(-)

diff --git a/debian/changelog b/debian/changelog
index 2d0883c..67fbae1 100644
--- a/debian/changelog
+++ b/debian/changelog
@@ -104,6 +104,11 @@ dpkg (1.16.0) UNRELEASED; urgency=low
DEB_BUILD_MULTIARCH, that return the "ideal" GNU triplet for each
architecture which should be used as the path component for library
installation.
+ * Dpkg/Shlibs.pm: we should explicitly prepend the appropriate multiarch
+ paths to our library search path. These would be picked up later on
+ anyway in the case of a native build, but for, e.g., bootstrapping a
+ cross-toolchain the needed multiarch paths aren't going to be found in
+ ld.so.conf.

[ Updated programs translations ]
* German (Sven Joachim).
diff --git a/scripts/Dpkg/Shlibs.pm b/scripts/Dpkg/Shlibs.pm
index 95ef4fe..95950e1 100644
--- a/scripts/Dpkg/Shlibs.pm
+++ b/scripts/Dpkg/Shlibs.pm
@@ -29,7 +29,8 @@ use Dpkg::Gettext;
use Dpkg::ErrorHandling;
use Dpkg::Shlibs::Objdump;
use Dpkg::Path qw(resolve_symlink canonpath);
-use Dpkg::Arch qw(debarch_to_gnutriplet get_build_arch get_host_arch);
+use Dpkg::Arch qw(debarch_to_gnutriplet debarch_to_multiarch
+ get_build_arch get_host_arch);

use constant DEFAULT_LIBRARY_PATH =>
qw(/lib /usr/lib /lib32 /usr/lib32 /lib64 /usr/lib64
@@ -39,6 +40,10 @@ use constant DEFAULT_LIBRARY_PATH =>
# cross-build or a build of a cross-compiler
my @crosslibrarypaths;
my $crossprefix;
+# And when we're not building a cross-compiler, be sure to pick up the
+# multiarch paths
+my @multiarchpaths;
+my $multiarch;
# Detect cross compiler builds
if ($ENV{GCC_TARGET}) {
$crossprefix = debarch_to_gnutriplet($ENV{GCC_TARGET});
@@ -51,6 +56,7 @@ if ($ENV{DEB_TARGET_GNU_TYPE} and
# host for normal cross builds.
if (get_build_arch() ne get_host_arch()) {
$crossprefix = debarch_to_gnutriplet(get_host_arch());
+ $multiarch = debarch_to_multiarch(get_host_arch());
}
# Define list of directories containing crossbuilt libraries
if ($crossprefix) {
@@ -58,8 +64,11 @@ if ($crossprefix) {
"/$crossprefix/lib32", "/usr/$crossprefix/lib32",
"/$crossprefix/lib64", "/usr/$crossprefix/lib64";
}
+if ($multiarch) {
+ push @multiarchpaths, "/lib/$multiarch", "/usr/lib/$multiarch";
+}

-our @librarypaths = (DEFAULT_LIBRARY_PATH, @crosslibrarypaths);
+our @librarypaths = (@multiarchpaths, DEFAULT_LIBRARY_PATH, @crosslibrarypaths);

# Update library paths with LD_LIBRARY_PATH
if ($ENV{LD_LIBRARY_PATH}) {
--
1.7.1


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/1300693517-11497-1-git-send-email-steve.langasek@linaro.org

Neil Williams 03-21-2011 09:25 AM

Dpkg/Shlibs.pm: multiarch search paths
 
On Mon, 21 Mar 2011 09:55:26 +0100
Raphael Hertzog <hertzog@debian.org> wrote:

(Trimming the CC: feel free to drop patches@l.o if linaro-dev@l.l.o is
more useful. Please keep debian-embedded in CC to reach me.)

> Not knowing much about cross-compilation I had started with a sligthly
> different approach than you, mimicking exactly what was done for
> cross-compilation support.

09:48 < codehelp> buxy: vorlon: my original query related to not so
much building a cross-compiler but building gcc-* to install and run on
cross, i.e. reverse-cross where build=amd64,host=armel,target=armel
09:48 < codehelp> cross-compiler build=amd64,host=amd64,target=armel
09:48 < codehelp> doesn't give you a libgcc1 which can be used to
satisfy dependencies of packages cross-built using the cross-compiler

Cross-building an entire distro, like Emdebian Crush, involved a long
winded process:
0: Build gcc-4.* as an arm cross-compiler (not armel in those days but
that's not important to the method)
1: Build build-dependencies of gcc using the cross-compiler
2: Build gcc-4.* as a normal package using the cross-compiler

The people who are most likely to be doing this now are Linaro.
Emdebian Crush stalled after Lenny (and used ARM not armel), so the
version of gcc-4.2 in Lenny should still build with the typical
"dpkg-buildpackage -aarm". Later versions of gcc probably have other
fixes which would cause the reverse cross build to fail, not due to the
failure of the original patches but due to changes in gcc itself.

To create a fully bootstrap-able system, gcc does need to support the
reverse cross and this will need testing in Multi-arch world.

Typically, gcc-4.* *should* be expected to build correctly just using:
$ dpkg-buildpackage -a$arch

No special target specification (except internally in debian/rules
etc.), no expectation of building a cross-compiler, just building it as
if it is any other package which needs to be cross-built.

Of course, if libgcc1 can be made available in any other way than
having to pretend that we actually want a native compiler on these
systems (99% of the time we really don't), then everyone wins.

> Neil (or anyone else knowledgable on the topic), can you review the
> patches and tell us what's really needed and most logical according to
> you?
>
> Steve's patch:
> - adds the multi-arch path at the start of the search paths
> - adds them only for stantard cross-compilation (host != build) but not
> for the special cases used to detect a cross-compiler build

Intuitively this does sound the correct way to do it, especially as
what I wanted from the original patch was a standard cross-build, not
the special case of building a cross-compiler, but:

> My patch:
> - adds the multi-arch path before the cross-arch patch but after the
> standard search path
>
> - add them in all cases where the cross-compilation paths are used
>
> Putting the multi-arch paths after the default path should not matter as
> dpkg-shlibdeps skips libraries of non-matching architecture.

This is more in line with how it actually works out when trying to
cross-build gcc itself.

I'm not sure of the benefit of putting the multiarch lib location in
front of the default lib location. I'm not sure if having some i386
libraries in a multi-arch path and the main system amd64 libraries in a
non-multi-arch path would actually matter (after all, the same libraries
*should* get updated together). That's a "traditional" Multi-Arch
question, so I'd defer to Steve on that one.

For cross-building, it's simply sufficient that the new
multi-arch-cross paths precede the old dpkg-cross paths and both
patches achieve this.

> For the case of building a cross-compiler, Steve's experience tends to
> prove that adding the multi-arch path is not needed... but I don't see why
> it should be treated differently than the cross-arch paths.
>
> Can you enlighten us?

For building a cross-compiler, Steve is correct - it doesn't matter.

For using that cross-compiler to cross-compile gcc itself, it will
matter. Multi-Arch paths will be necessary to build a version of
libgcc1 which can be installed on the target.

--


Neil Williams
=============
http://www.linux.codehelp.co.uk/

Matthias Klose 03-21-2011 12:03 PM

Dpkg/Shlibs.pm: multiarch search paths
 
On 21.03.2011 11:25, Neil Williams wrote:
> The people who are most likely to be doing this now are Linaro.
> Emdebian Crush stalled after Lenny (and used ARM not armel), so the
> version of gcc-4.2 in Lenny should still build with the typical
> "dpkg-buildpackage -aarm". Later versions of gcc probably have other
> fixes which would cause the reverse cross build to fail, not due to the
> failure of the original patches but due to changes in gcc itself.

the gcc-4.4 packaging is able to build a reverse cross. I didn't check gcc-4.5
and gcc-4.6.

Matthias


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4D874CA9.6040101@ubuntu.com">http://lists.debian.org/4D874CA9.6040101@ubuntu.com

Neil Williams 03-21-2011 12:08 PM

Dpkg/Shlibs.pm: multiarch search paths
 
On Mon, 21 Mar 2011 14:03:37 +0100
Matthias Klose <doko@ubuntu.com> wrote:

> On 21.03.2011 11:25, Neil Williams wrote:
> > The people who are most likely to be doing this now are Linaro.
> > Emdebian Crush stalled after Lenny (and used ARM not armel), so the
> > version of gcc-4.2 in Lenny should still build with the typical
> > "dpkg-buildpackage -aarm". Later versions of gcc probably have other
> > fixes which would cause the reverse cross build to fail, not due to the
> > failure of the original patches but due to changes in gcc itself.
>
> the gcc-4.4 packaging is able to build a reverse cross. I didn't check gcc-4.5
> and gcc-4.6.
>
> Matthias

Oops, sorry Matthias - that was meant to be "later versions of gcc
*might* have other (internal) changes which could cause..." I didn't
mean to infer that I'd tried and failed a reverse cross with newer
versions, I haven't had time to do that test. Thanks for the info on
gcc-4.4.

--


Neil Williams
=============
http://www.linux.codehelp.co.uk/

Steve Langasek 03-23-2011 07:14 PM

Dpkg/Shlibs.pm: multiarch search paths
 
Hi Hector,

On Mon, Mar 21, 2011 at 05:43:19PM +0000, Hector Oron wrote:
> (Apologies for big attachment on previous email)

> 2011/3/21 Hector Oron <hector.oron@gmail.com>:

> > 2011/3/21 Matthias Klose <doko@ubuntu.com>:

> >> the gcc-4.4 packaging is able to build a reverse cross. *I didn't check gcc-4.5
> >> and gcc-4.6.

> > A test build on gcc-4.6 shows reverse build fails. See attached
> > build-reverse-cross.log.txt
> > Build with: $ dpkg-buildpackage -aarmel -rfakeroot -us -uc

> > A non biarch test build on gcc-4.6 shows reverse build fails. See
> > attached build-reverse-cross.log.txt
> > Build with: *$ DEB_CROSS_NO_BIARCH=yes dpkg-buildpackage -aarmel
> > -rfakeroot -us -uc

> I tried the same builds on gcc-4.4 and gcc-4.5, compressed build logs attached.

> gcc-4.4 fails possibly related to cross gfortran support.
> gcc-4.5 seem to build fine.

Your mail doesn't seem to have made it through to the Debian mailing lists,
and it arrived at the linaro list with the attachments stripped (presumably
for size). :( Can you post these logs on a website somewhere?

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org

Wookey 03-24-2011 01:21 AM

Dpkg/Shlibs.pm: multiarch search paths
 
+++ Steve Langasek [2011-03-23 13:14 -0700]:
>
> Your mail doesn't seem to have made it through to the Debian mailing lists,
> and it arrived at the linaro list with the attachments stripped (presumably
> for size). :( Can you post these logs on a website somewhere?

Not suprised at 3.7Mb (what _were_ you thinking hector? :-)

I've put them here:
http://wookware.org/files/build-reverse-cross.log.txt
http://wookware.org/files/build-reverse-cross-no-biarch.log.txt

Wookey
--
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110324022103.GO17337@dream.aleph1.co.uk">http://lists.debian.org/20110324022103.GO17337@dream.aleph1.co.uk

Hector Oron 03-24-2011 09:56 AM

Dpkg/Shlibs.pm: multiarch search paths
 
Hi,

2011/3/23 Steve Langasek <steve.langasek@linaro.org>:

>> (Apologies for big attachment on previous email)

> Your mail doesn't seem to have made it through to the Debian mailing lists,
> and it arrived at the linaro list with the attachments stripped (presumably
> for size). :( *Can you post these logs on a website somewhere?

Logs available at:
http://emdebian.org/~zumbi/tmp/

Cheers
--
*Hctor Orn

"Our Sun unleashes tremendous flares expelling hot gas into the Solar
System, which one day will disconnect us."

-- Day DVB-T stop working nicely
Video flare: http://antwrp.gsfc.nasa.gov/apod/ap100510.html


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: AANLkTimBvr5eqPQCxb_j73EqZcaFZHTXhfg35W6Vg11o@mail .gmail.com">http://lists.debian.org/AANLkTimBvr5eqPQCxb_j73EqZcaFZHTXhfg35W6Vg11o@mail .gmail.com


All times are GMT. The time now is 02:23 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.