FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 09-24-2012, 12:48 PM
Cyril Brulebois
 
Default assumptions about the build environment.

Wouter Verhelst <wouter@debian.org> (24/09/2012):
> On Sat, Sep 22, 2012 at 07:06:57PM +0200, Adam Borowski wrote:
> > You'd need a separate chroot for every arch you want to be able to
> > compile for.
>
> Buildd machines typically have only one chroot for each distribution
> they build, as they don't build for multiple architectures.

I guess “typically” was indeed warranted, s390/s390x come to mind (ditto
on the porterbox side).

Mraw,
KiBi.
 
Old 09-27-2012, 12:14 PM
Wookey
 
Default assumptions about the build environment.

+++ Wouter Verhelst [2012-09-24 13:14 +0200]:
> On Sat, Sep 22, 2012 at 07:06:57PM +0200, Adam Borowski wrote:
> >
> > That it breaks multiarch builds.
>
> Multiarch builds are not done on our buildd machines.

Yet. They very likely will be soon, for partial architectures and
cross-toolchains.

And we still want them to work, in cross-compiling configurations,
too, at least for core stuff.

> [...]
> > As for complex "hard to patch" build systems, could you give me an example?

apache/apr :-)

But the case of uname _is_ easy to deal with - use dpkg-architecture.

And on the original point, anything checking /sys for HOST arch stuff
is likely to be getting it wrong too - fine for BUILD arch checks.

> > I can't quite think of a case where patching references to uname could be
> > tricky.
>
> I can't give you an example of *this* particular issue; but given the
> amount of complex build systems I've seen, I don't think it's too
> unreasonable to assume they exist.
>
> I'm not saying we should change policy to mandate that i386 builds in
> i386 chroots should be done with linux32 running or something similar.
> But I also do happen to think that tuning buildd machines to do a bit
> more than what policy requires them to do is usually a good thing. For I
> used to keep debhelper installed in my buildd chroots, even though it's
> not part of build-essential; but since almost all packages use it in
> some form or another, it was being installed and deinstalled enough that
> I could gain some time by having it installed by default. The result
> was of course that packages who mistakenly failed to add debhelper to
> their build-depends would successfully build on my buildd hosts, but I
> didn't think that was much of a problem.
>
> Similarly, if adding linux32 to the environment on amd64 hosts building
> inside a chroot means the success rate for packages built will go up, I
> don't think we should refrain from doing so -- unless, of course, doing
> so would cause more problems than it would solve, which was the point of
> my question.

Well, this is indeed exactly the point. If we specify what can be
assumed by packages, but then actually don't test that, we test
something slightly different (by putting extra stuff int he
environment by default) then bugs will not be found.

It's a simple tradeoff between rigour and build success/hassle. In
general I like the rigour in Debian and have spent the last year or so
being rigourous in cross-building/multiarch stuff which has found an
awful lot of (expected) breakage. (You an gloss over a lot of stuff by
installing qemu in chroots when cross-building, for example, and often
it's the best thing to do)

In general I'm in favour of agreeing what can be relied upon and
then only providing that in buildds. But you are quite right that for
the purposes of buildds doing native building this improves your
success rate by glossing over potentially-dubious checks, with little
cost.

> After all, the autobuilder network is not meant to do QA; we have
> resources for doing full-archive rebuilds for that purpose. Instead, the
> autobuilder network is meant to make sure we build as many packages as
> possible. If we can avoid some issues in building packages that isn't
> really worth spelling out in policy by just doing some minor
> configuration tweak on a buildd host, I think we should do so -- and
> this certainly qualifies, IMO.

I only run cross-buildds, not native ones, so this isn't my call. Just
bear in mind that every such choice of providing more than is strictly
mandated will be hiding bugs in other circumstances.

Wookey
--
Principal hats: Linaro, Emdebian, Wookware, Balloonboard, ARM
http://wookware.org/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120927121449.GP24695@stoneboat.aleph1.co.uk">htt p://lists.debian.org/20120927121449.GP24695@stoneboat.aleph1.co.uk
 
Old 09-27-2012, 01:55 PM
Guillem Jover
 
Default assumptions about the build environment.

On Thu, 2012-09-27 at 13:14:49 +0100, Wookey wrote:
> But the case of uname _is_ easy to deal with - use dpkg-architecture.

Well, not really if the patch is supposed to go upstream, given that
dpkg-architecture is not a distribution-neutral interface.

In most of the cases uname is wrong because the build system should
be checking for features instead of hardcoded lists of architectures
and because its output is not really normalized across different
operating systems, or as mentioned on this thread the detection should
really be delayed until run-time. But if the architecture is needed for
whatever reason then using something like config.guess/sub's output and
a way to distinguish between host and build architectures is probably
better.

> And on the original point, anything checking /sys for HOST arch stuff
> is likely to be getting it wrong too - fine for BUILD arch checks.

Right.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120927135557.GA508@gaara.hadrons.org">http://lists.debian.org/20120927135557.GA508@gaara.hadrons.org
 
Old 09-27-2012, 03:28 PM
Wouter Verhelst
 
Default assumptions about the build environment.

On Thu, Sep 27, 2012 at 01:14:49PM +0100, Wookey wrote:
> In general I'm in favour of agreeing what can be relied upon and
> then only providing that in buildds. But you are quite right that for
> the purposes of buildds doing native building this improves your
> success rate by glossing over potentially-dubious checks, with little
> cost.

I guess the question here is whether we write policy on how to build a
package in support of buildd machines, or whether things are the other
way around, that is, we use buildd machines to verify packages'
compliance to policy.

If we were doing the latter, we should probably also do things like run
lintian on a package after it built, or run it through piuparts, or do
any number of other time-consuming things that would improve the quality
of our distribution overall. We are, however, not doing that, because
this is not the primary focus of the buildd network. Instead, the
primary focus of the buildd network is to build our packages for their
architecture in a timely manner.

Now I'm not saying that I'm opposed, in principle, to such testing if it
provided a net benefit; and to some extent, we are in fact already doing
that (since maintainers are encouraged to enable test suites as part of
the regular build of their packages). However, when the buildd network
is backlogged because of broken or ageing hardware, or because of
other issues not directly under the control of the buildd admins, these
admins should have the ability to cut some corners (while still
producing correct packages) so that they can cut down on the time
required when things are getting tight. To be able to do this, the
amount of things the hosts in the buildd network absolutely must do
should be a list that is shorter rather than longer.

In essense, the buildd network is not a QA resource; and while it is
okay to use it as such if buildd hosts have time to spare, I believe we
should ensure that our QA team's primary resources for archive-wide
tests should be found elsewhere, so that buildd admins have the liberty
to do what they need to do: ensure that an architecture can still be
part of the next release, by building the packages that we need to
build.

--
The volume of a pizza of thickness a and radius z can be described by
the following formula:

pi zz a
 
Old 10-07-2012, 08:58 AM
Jakub Wilk
 
Default assumptions about the build environment.

More questions about build env assumptions:

Can you assume that /sbin and /usr/sbin are within PATH?

Can you assume that the SHELL environment variable (_not_ the makefile
variable) is set to something 10.4-compliant?


(My personal answers to these questions are: no and no.)

--
Jakub Wilk


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20121007085831.GA7899@jwilk.net">http://lists.debian.org/20121007085831.GA7899@jwilk.net
 
Old 10-07-2012, 09:54 AM
Eric Valette
 
Default assumptions about the build environment.

While working on debian one thing I have not managed to find is
documentation on what packages can and can't assume about the build
environment. Does such documentation exist and if not should it be
created.

Some specific cases i'm wondering about:

I just discovered that on my beagleboard XM (under armhf sid) nacl
(which previously build on a debian experimental armhf buildd but
not a debian unstable armhf buildd) will build if /sys is mounted
but will not build if it is not mounted. Can packages assume that
/sys will be mounted in the build environment or not?


I consider /sys almost as essential as /proc. However I wonder
what a build process would need it for.


I will hijack this thread a bit and I know there is another one running
on the subject, but assumption like this makes it impossible to cross
compile most packages.


I'm currently trying to compile armhf package for the rasberry pi on a
amd64 machine and naively though it would be easy to do with multiarch.
I screwed my machines(replaced the dynamic linkers, ftp and other tools
by arm binaries although I followed the scarce available documentation).


Natively compiling package as big as XBMC on the PI is a nightmare and
current tools fails really short because you:
1) need a root filesystem for the machines. You can use debootstrap but
will need many additionnal packages that are yet not build,
2) cannot install produced .deb in the root filesystem exept by
running them on qemu which is..


Any hint?



--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 50715160.5050300@free.fr">http://lists.debian.org/50715160.5050300@free.fr
 
Old 10-07-2012, 09:39 PM
Roger Leigh
 
Default assumptions about the build environment.

On Sun, Oct 07, 2012 at 10:58:31AM +0200, Jakub Wilk wrote:
> More questions about build env assumptions:
>
> Can you assume that /sbin and /usr/sbin are within PATH?

At which point(s) during the build?

For sbuild:

During any command run as the build user (*not* root), it will
default to

my %common_keys = (
'PATH' => {
DEFAULT => "/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games"
},

(from Sbuild::ConfBase). They should be present when running commands
as root, though I'd have to double-check that.

> Can you assume that the SHELL environment variable (_not_ the
> makefile variable) is set to something 10.4-compliant?

It looks like we currently pass SHELL through, unless you configure it
otherwise. Maybe this should be revisited?

> (My personal answers to these questions are: no and no.)

I think that this is correct (as the default behaviour; it could be
configured otherwise).


Regards,
Roger

--
.'`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools
`- GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20121007213942.GI18133@codelibre.net">http://lists.debian.org/20121007213942.GI18133@codelibre.net
 
Old 10-07-2012, 09:41 PM
Roger Leigh
 
Default assumptions about the build environment.

On Sun, Oct 07, 2012 at 11:54:40AM +0200, Eric Valette wrote:
> I'm currently trying to compile armhf package for the rasberry pi on
> a amd64 machine and naively though it would be easy to do with
> multiarch. I screwed my machines(replaced the dynamic linkers, ftp
> and other tools by arm binaries although I followed the scarce
> available documentation).
>
> Natively compiling package as big as XBMC on the PI is a nightmare
> and current tools fails really short because you:
> 1) need a root filesystem for the machines. You can use debootstrap
> but will need many additionnal packages that are yet not build,
> 2) cannot install produced .deb in the root filesystem exept
> by running them on qemu which is..
>
> Any hint?

schroot will let you run a non-native chroot with qemu, and you can
use this with sbuild for package building. sbuild now also has
initial support or multiarch cross building.


Regards,
Roger

--
.'`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' schroot and sbuild http://alioth.debian.org/projects/buildd-tools
`- GPG Public Key F33D 281D 470A B443 6756 147C 07B3 C8BC 4083 E800


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20121007214110.GJ18133@codelibre.net">http://lists.debian.org/20121007214110.GJ18133@codelibre.net
 
Old 10-08-2012, 03:46 PM
Ian Jackson
 
Default assumptions about the build environment.

Jakub Wilk writes ("Re: assumptions about the build environment."):
> More questions about build env assumptions:

My answers:

> Can you assume that /sbin and /usr/sbin are within PATH?

No. Package builds are supposed to be done as the user; the rootness
of the "install" target is just for file permissions and doesn't mean
it should be using admin tools.

> Can you assume that the SHELL environment variable (_not_ the makefile
> variable) is set to something 10.4-compliant?

No. SHELL might refer to ksh, csh or even emacs.

Ian.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20594.62821.442938.293069@chiark.greenend.org.uk"> http://lists.debian.org/20594.62821.442938.293069@chiark.greenend.org.uk
 

Thread Tools




All times are GMT. The time now is 04:27 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org