FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 01-20-2010, 08:37 AM
Lucas Nussbaum
 
Default GPL-licensed software linked against libssl on buildds!

On 20/01/10 at 00:48 -0800, Steve Langasek wrote:
> On Wed, Jan 20, 2010 at 02:22:33PM +1300, Lucas Nussbaum wrote:
>
> > Why spend a lot of time on tasks that provide little benefit, and also
> > some disadvantages (in some cases, the fixes might be non-obvious, and
> > requires changes to the packaging that tend to obscure it, for example
> > by using --disable-foo for each and every option we don't want)?
>
> I'm not asking anyone to spend time on this task, but I still consider
> missing build-conflicts a bug. Ignoring these bugs by insisting on clean
> chroot environments for all official package builds is no solution - what if
> one of your build-dependencies pulls in one of these other packages,
> resulting in an undistributable (license-incompatible) misbuild? If the
> build-conflicts had been declared, or if the --without-foo option had been
> passed, we would not have to worry about such a misbuild.

If the chroot env is clean, the build process is likely to be very
similar on your system and on the buildds. So even without
build-conflicts, it is likely that no additional build-deps will be
pulled. It's true that that isn't a full guarantee (differences between
archs, binNMUs done later in the package lifecycle), but clean chroot
environments offer much more guarantee than the current situation, which
is based only on the maintainer disabling all unused options or adding
all the proper build-conflict. That is hard and error-prone:: among the
packages you maintain, for example, sqsh picks up an additional dep on
tcl8.4 if tcl-dev is installed.
--
| Lucas Nussbaum
| lucas@lucas-nussbaum.net http://www.lucas-nussbaum.net/ |
| jabber: lucas@nussbaum.fr GPG: 1024D/023B3F4F |


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-20-2010, 09:11 AM
Stefano Zacchiroli
 
Default GPL-licensed software linked against libssl on buildds!

On Wed, Jan 20, 2010 at 10:13:46PM +1300, Lucas Nussbaum wrote:
> What's the problem with documentation such as
> https://wiki.ubuntu.com/PbuilderHowto (except it's an Ubuntu
> documentation)? I think that the process of building with pbuilder is
> reasonably well documented.

Let's be realistic. We still have Debian developers not using pbuilder
cowbuilder, are you really arguing that _users_ will use it invariably
to rebuild their packages? I don't have numbers, but I'm reasonably sure
"apt-get source -b" is still the most used tool to rebuild packages
together with the magic recipe "fakeroot debian/rules binary".
Additionally, if you have to debug build failures, rebuilding in a real
system is still handier.

> > On the same line, this whole issue is one of the reason why we have
> > relationships like Build-Conflicts. Why having a non-declared
> > Build-Conflicts shouldn't be a bug?
> Feel free to start filing bugs. A good start would be the list of source
> packages[1] from 2008 that probably have a missing build-conflict, since
> they produced different binary packages (according to debdiff) in an
> unclean chroot. (that list contains some false positives)

You're cheating now :-) I was just arguing that your aut-aut was not
warranted, that we can live with a mixed environment in which those bugs
are not RC (and hence should not be pursued as actively as we do for RC
ones), but are still bug that ought to be filed. Be assured that I'll
file the one that will cross my path.

Cheers.

--
Stefano Zacchiroli -o- PhD in Computer Science PostDoc @ Univ. Paris 7
zack@{upsilon.cc,pps.jussieu.fr,debian.org} -<>- http://upsilon.cc/zack/
Dietro un grande uomo c' ..| . |. Et ne m'en veux pas si je te tutoie
sempre uno zaino ...........| ..: |.... Je dis tu tous ceux que j'aime
 
Old 01-20-2010, 10:05 AM
Steve Langasek
 
Default GPL-licensed software linked against libssl on buildds!

On Wed, Jan 20, 2010 at 10:37:48PM +1300, Lucas Nussbaum wrote:
> > I'm not asking anyone to spend time on this task, but I still consider
> > missing build-conflicts a bug. Ignoring these bugs by insisting on clean
> > chroot environments for all official package builds is no solution - what if
> > one of your build-dependencies pulls in one of these other packages,
> > resulting in an undistributable (license-incompatible) misbuild? If the
> > build-conflicts had been declared, or if the --without-foo option had been
> > passed, we would not have to worry about such a misbuild.

> If the chroot env is clean, the build process is likely to be very
> similar on your system and on the buildds.

This "very likely" and $2 buys you a cup of coffee. I have seen cases of
build-dep skew across architectures resulting in binaries in the archive
with different dependencies.

> It's true that that isn't a full guarantee (differences between
> archs, binNMUs done later in the package lifecycle),

binNMUs are a non-negligible use case. I've seen packages end up with wrong
dependencies across binNMUs due to this issue as well.

> but clean chroot environments offer much more guarantee than the current
> situation, which is based only on the maintainer disabling all unused
> options or adding all the proper build-conflict.

No, *neither* is a guarantee, which is why it's warranted to use *both*
approaches to minimize the occurrence of bugs.

Suppose that libcups2-dev adds a dependency on libssl-dev, because it starts
to provide two versions of libcups - the default lib still using gnutls, and
an alternative lib linked against openssl. The majority of packages will
get no new dependencies, because libcups.so will still link against gnutls;
but netatalk will wind up with a dependency on openssl and will be RC buggy
as a result.

Where does the bug lie, and when did it appear? Is it a bug in libcups-dev
for depending on libssl-dev? No, no one would insist that the cups
maintainer revert this change (at least not for this reason). Is it a bug
in the maintainer for not having checked first that no reverse-deps build in
wrong ways as a result of this change? Well, that's at least as hard and
error-prone as doing more general unclean-chroot checks, if not moreso.

No, the bug is in the netatalk package for building differently when
different packages are installed. The severity of the bug may vary over
time, but it's already latent in the package before the cups maintainers
make their change.

The operative principle that's been in place since Build-Depends and
Build-Conflicts were first formulated was that these + build-essential
should provide *all* the information needed for package builds to be
reproducible. We may fall short of that standard from time to time, but
that's not a reason to abandon as unsupported anyone building packages in an
environment other than an empty chroot.

> That is hard and error-prone::

Yes. But I don't think this is a reason to abandon the principle,
especially given the class of bug described above.

Anyway, if you're going to insist that everyone do all their builds in clean
chroots, I could just as well insist the opposite - that everyone build all
their packages in cluttered chroots, to ensure no missing Build-Conflicts.


> among the packages you maintain, for example, sqsh picks up an additional
> dep on tcl8.4 if tcl-dev is installed.

Oh, thanks for the info. This seems to be due to a namespace collision
between tcl-dev and a commercial library that sqsh supports building
against; I'll add a build-conflicts.

And I don't think the existence of bugs in my packages disproves my
argument.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org
 
Old 01-20-2010, 10:07 AM
"Bernhard R. Link"
 
Default GPL-licensed software linked against libssl on buildds!

* Lucas Nussbaum <lucas@lucas-nussbaum.net> [100120 01:26]:
> There are two ways to attack that problem:
>
> (1) We decide that we want to provide the guarantee that packages
> build the correct way in unclean envs. That mean making such bugs RC,
> basically, and making efforts to find such bugs.

If you s/unclean/non-minimal/ then I think that is already the status
quo. A package building indeterministic in non-minimal builds was in
practise indeterministic and when catched got the appropiate bug
reports.

Hochachtungsvoll,
Bernhard R. Link


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-20-2010, 05:30 PM
Russ Allbery
 
Default GPL-licensed software linked against libssl on buildds!

Patrick Schoenfeld <schoenfeld@debian.org> writes:
> On Tue, Jan 19, 2010 at 04:04:07PM -0800, Russ Allbery wrote:

>> Uh, since as long as I've been part of the project. I think this is at
>> least the third time that I recall the same topic coming up on -devel.

> Wow. How often a topic comes up on -devel is an indicator how
> representative a given idea is in the whole developer body? It might be
> a sign that the people who want it tend to ask for it a lot.

Each time we had this discussion previously, my perception of the
consensus was what I stated earlier.

> Well, I can't tell if most packages use autoconf. Probably they do. But
> there are plenty of other build systems around, all with its own
> caveats. So my statement "... illussionary to support /every/ possible
> circumstance ..." is still true.

It's illusory to assume we can remove all bugs from our packages, for any
type of significant bug that you care to mention. That doesn't mean we
should stop fixing bugs.

> That does not mean that we shouldn't fix such bugs if they arise
> (obviously we should) but having priority on it is a different thing.

Then I'm not sure that you're disagreeing with me?

All I said was that it's a bug that should normally be fixed. If you also
agree that it's a bug that should normally be fixed, you may want to
consider whether you're fighting shadows here. I didn't say it was
RC, and I didn't say it needed to be a priority over other bugs,
necessarily.

>> sbuild has never provided this in the history of the project. I really
>> have to question the emphasis put on this given that we've lived for
>> all these years without having that and by fixing the packages to do
>> the right thing.

> Eh.. what? I'm using sbuild to build my packages before uploading them
> locally. It uses schroot and schroot supports LVM snapshots. Which is
> what I'm using. Probably it has never been deployed on our buildds but
> that has nothing to do with what sbuild can provide.

I didn't say anything about what sbuild *could* provide. I said that it
*has not* provided that, in the sense that the buildds have never
previously done what you describe above. The project has not collapsed.
Clearly it's possible to, in general, fix these problems when they arise
without using guaranteed-clean chroots.

Separately, I do agree that having better guarantees around the chroots is
a good idea. I'm just pointing out that there's no reason to make that
the only solution to the problem, and in fact the project has not
historically needed this and still has done an okay job with such bugs.

Steve made the same point with fewer words and more precision.

> Oh, well, after I read the link I even remember it. Yes, if we are aware
> of problems there is reason to fix them. That doesn't mean that we
> should always build in dirty chrooots, though.

I think we may not actually be disagreeing.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-20-2010, 06:09 PM
Patrick Schoenfeld
 
Default GPL-licensed software linked against libssl on buildds!

On Wed, Jan 20, 2010 at 10:30:13AM -0800, Russ Allbery wrote:
> > That does not mean that we shouldn't fix such bugs if they arise
> > (obviously we should) but having priority on it is a different thing.
>
> Then I'm not sure that you're disagreeing with me?

Oh I don't. However in one of your first mails you sounded quiet different.
Somewhere in the line of "we do not need buildds with clean
environments, because after all we want packages to be buildable
in dirty environments as well".

Best Regards,
Patrick


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-20-2010, 06:14 PM
Russ Allbery
 
Default GPL-licensed software linked against libssl on buildds!

Patrick Schoenfeld <schoenfeld@debian.org> writes:
> On Wed, Jan 20, 2010 at 10:30:13AM -0800, Russ Allbery wrote:

>>> That does not mean that we shouldn't fix such bugs if they arise
>>> (obviously we should) but having priority on it is a different thing.

>> Then I'm not sure that you're disagreeing with me?

> Oh I don't. However in one of your first mails you sounded quiet
> different. Somewhere in the line of "we do not need buildds with clean
> environments, because after all we want packages to be buildable in
> dirty environments as well".

Hm, well, I re-read all of my messages to this thread and I'm not seeing
it, but I certainly apologize for giving you that impression! That's not
what I was trying to communicate at all.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 01-20-2010, 09:21 PM
Michael Banck
 
Default GPL-licensed software linked against libssl on buildds!

On Tue, Jan 19, 2010 at 03:40:22PM -0800, Russ Allbery wrote:
> Lucas Nussbaum <lucas@lucas-nussbaum.net> writes:
> > On 19/01/10 at 14:36 -0800, Russ Allbery wrote:
>
> >> Well, I would argue that proper package builds in dirty environments is
> >> something we want in Debian anyway, and while this isn't the ideal
> >> method to find it, it would be a bug regardless of how the buildds
> >> worked.
>
> > Why would we want that?
>
> > I mean, it's very difficult to guarantee that packages build correctly
> > in dirty envs. I don't really see the point of enforcing that when we
> > have the technology (pbuilder, sbuild + lvm snapshots) there to ignore
> > that problem.
>
> Because we want our users to be able to patch and rebuild our software to
> suit their needs. Asking them to set up a chroot build environment is
> asking quite a lot.

That is certainly a good goal, but I think it should be done outside the
scope of autobuilding, where we want clean, reproducable builds.

Something like an occasional archive-wide rebuild using a specially
prepared, overly-tainted (with -dev libraries) chroot and comparing to a
second run with clean chroot would be more worthwhile I think (albeit more work
as well).


Michael


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 02-01-2010, 11:07 PM
Wouter Verhelst
 
Default GPL-licensed software linked against libssl on buildds!

On Wed, Jan 20, 2010 at 10:37:48PM +1300, Lucas Nussbaum wrote:
> On 20/01/10 at 00:48 -0800, Steve Langasek wrote:
> > On Wed, Jan 20, 2010 at 02:22:33PM +1300, Lucas Nussbaum wrote:
> >
> > > Why spend a lot of time on tasks that provide little benefit, and also
> > > some disadvantages (in some cases, the fixes might be non-obvious, and
> > > requires changes to the packaging that tend to obscure it, for example
> > > by using --disable-foo for each and every option we don't want)?
> >
> > I'm not asking anyone to spend time on this task, but I still consider
> > missing build-conflicts a bug. Ignoring these bugs by insisting on clean
> > chroot environments for all official package builds is no solution - what if
> > one of your build-dependencies pulls in one of these other packages,
> > resulting in an undistributable (license-incompatible) misbuild? If the
> > build-conflicts had been declared, or if the --without-foo option had been
> > passed, we would not have to worry about such a misbuild.
>
> If the chroot env is clean,

I hate to break the news, but no build environment (look, full word) is
ever clean. There are environment variables, other processes running on
the same system, and various other things that can influence it.
Granted, rogue environment variables are rarely going to be a problem on
a buildd host, but clock skew or rogue processes from previous builds
might not be. Okay, that's a stretch. Still.

At any rate, here are some facts:
- A package that builds differently because something is (or is not)
installed on the build system is buggy. Period. It has nothing to do
with the build system, it's the package.
- When a package has a buggy debian/rules and/or debian/control file,
and it gets built on 11 architectures, surely one of those
architectures is going to hit that bug.
- A clean chroot takes time and processing power. You need to drop and
recreate the chroot between builds, upgrade the same Build-Essential
packages every time you do an upgrade, copy the apt cache in and out
of the chroot (or keep downloading the same packages over and over),
and various other things. LVM snapshots fix some, though not all, of
those problems, and introduce a few of their own.
I don't know about you, but I'd rather have the buildd spend
processing power on building packages. Having it fail at producing a
good package because the maintainer didn't do a good enough job is
nothing new -- they do that all the time.

As such, I'm rather unconvinced of the merits of this LVM snapshot
thing...

--
The biometric identification system at the gates of the CIA headquarters
works because there's a guard with a large gun making sure no one is
trying to fool the system.
http://www.schneier.com/blog/archives/2009/01/biometrics.html


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 02-02-2010, 05:59 AM
Lucas Nussbaum
 
Default GPL-licensed software linked against libssl on buildds!

On 02/02/10 at 01:07 +0100, Wouter Verhelst wrote:
> At any rate, here are some facts:
> - A package that builds differently because something is (or is not)
> installed on the build system is buggy. Period. It has nothing to do
> with the build system, it's the package.

... but I question that it is a bug that we want to spend time fixing.

> - A clean chroot takes time and processing power. You need to drop and
> recreate the chroot between builds, upgrade the same Build-Essential
> packages every time you do an upgrade, copy the apt cache in and out
> of the chroot (or keep downloading the same packages over and over),
> and various other things. LVM snapshots fix some, though not all, of
> those problems, and introduce a few of their own.
> I don't know about you, but I'd rather have the buildd spend
> processing power on building packages. Having it fail at producing a
> good package because the maintainer didn't do a good enough job is
> nothing new -- they do that all the time.

I think that the question is whether we would rather have the maintainer
spend time fixing those issues, or the buildd spend time dealing with
the consequences of using LVM snapshots.

I personally think that if we have a way to use CPU time to solve a
problem that would require maintainer time otherwise, we should use it.

(I am fully aware that putting more load on the buildds might require
adding buildds. However, I have the impression that the time required to
maintain several identical buildds doesn't grow linearly, so it wouldn't
be a too big problem.)
--
| Lucas Nussbaum
| lucas@lucas-nussbaum.net http://www.lucas-nussbaum.net/ |
| jabber: lucas@nussbaum.fr GPG: 1024D/023B3F4F |


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 08:55 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org