FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 10-25-2011, 01:34 AM
Paul Wise
 
Default A debian/rules target to rebuild pre-built stuff?

Hi all,

Since Debian does not require everything to be built from source (such
as autotools build systems, firmware and so on), would it be a good
idea to codify in policy that every package shipping files not built
from source should have a way to completely rebuild everything from
source? In this way we can find out whether or not the tools used to
build such files are still working as expected.

I am thinking this particular bikeshed should have
Build-Depends-Rebuild and debian/rules rebuild inside it.

If there is consensus on this issue then I will file a request on
policy to get this added to the next version.

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAKTje6E5K53c0=yXu+c64n26AhLiiRd6A4O5NhhEe1ibuXmPk A@mail.gmail.com">http://lists.debian.org/CAKTje6E5K53c0=yXu+c64n26AhLiiRd6A4O5NhhEe1ibuXmPk A@mail.gmail.com
 
Old 10-25-2011, 03:06 AM
Russ Allbery
 
Default A debian/rules target to rebuild pre-built stuff?

Paul Wise <pabs@debian.org> writes:

> Since Debian does not require everything to be built from source (such
> as autotools build systems, firmware and so on), would it be a good idea
> to codify in policy that every package shipping files not built from
> source should have a way to completely rebuild everything from source?
> In this way we can find out whether or not the tools used to build such
> files are still working as expected.

The results of that build seem unlikely to ever be seriously tested
currently, which makes me a little dubious that it's worth making a rule
about it. Or, put another way, I'm not sure that this is substantially
less controversial than just requiring debian/rules build rebuild
everything from source.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87bot5db36.fsf@windlord.stanford.edu">http://lists.debian.org/87bot5db36.fsf@windlord.stanford.edu
 
Old 10-25-2011, 03:22 AM
Paul Wise
 
Default A debian/rules target to rebuild pre-built stuff?

On Tue, Oct 25, 2011 at 11:06 AM, Russ Allbery wrote:

> The results of that build seem unlikely to ever be seriously tested
> currently, which makes me a little dubious that it's worth making a rule
> about it.

I would wager that majority of such results would be tested during the
build process.

> Or, put another way, I'm not sure that this is substantially
> less controversial than just requiring debian/rules build rebuild
> everything from source.

That seems unlikely given that there are many many packages using
autotools-based build systems.

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAKTje6EX3Ltpc=pmTAS4DSdqjcqn+veyVrY0wKoRjMMAMrQnv Q@mail.gmail.com">http://lists.debian.org/CAKTje6EX3Ltpc=pmTAS4DSdqjcqn+veyVrY0wKoRjMMAMrQnv Q@mail.gmail.com
 
Old 10-25-2011, 03:54 AM
Russ Allbery
 
Default A debian/rules target to rebuild pre-built stuff?

Paul Wise <pabs@debian.org> writes:
> On Tue, Oct 25, 2011 at 11:06 AM, Russ Allbery wrote:

>> The results of that build seem unlikely to ever be seriously tested
>> currently, which makes me a little dubious that it's worth making a
>> rule about it.

> I would wager that majority of such results would be tested during the
> build process.

Wouldn't that require using this target by default?

Or do you mean that *if* someone uses this target, then a simple package
build will test it? Hm, maybe. There are a lot of failures that will be
caught that way, but some that won't (Autotools failures can just mean not
finding supporting libraries, for instance).

>> Or, put another way, I'm not sure that this is substantially less
>> controversial than just requiring debian/rules build rebuild everything
>> from source.

> That seems unlikely given that there are many many packages using
> autotools-based build systems.

Either way, you're requiring the package maintainer to do the work to
create a target that rebuilds all that stuff. It's actually somewhat more
work to implement an optional target than to just make build do that. And
either way, they're going to be responsible for making sure it works,
except that under this proposal the buildds wouldn't test it for them.

I suppose there's the different between "should" and "must" which takes
some of the pressure off (although I suspect that will just lead to people
ignoring it).

I think I'd rather push people to use the dh add-on that regenerates all
the autotools files, since that makes things relatively easy provided that
the upstream files actually work with the latest versions.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87wrbtbu9f.fsf@windlord.stanford.edu">http://lists.debian.org/87wrbtbu9f.fsf@windlord.stanford.edu
 
Old 10-25-2011, 04:10 AM
Paul Wise
 
Default A debian/rules target to rebuild pre-built stuff?

On Tue, Oct 25, 2011 at 11:54 AM, Russ Allbery wrote:

> Or do you mean that *if* someone uses this target, then a simple package
> build will test it? *Hm, maybe. *There are a lot of failures that will be
> caught that way, but some that won't (Autotools failures can just mean not
> finding supporting libraries, for instance).

Right. Hmm, I guess so.

> Either way, you're requiring the package maintainer to do the work to
> create a target that rebuilds all that stuff. *It's actually somewhat more
> work to implement an optional target than to just make build do that. *And
> either way, they're going to be responsible for making sure it works,
> except that under this proposal the buildds wouldn't test it for them.

The buildds would not, but hopefully those doing archive-wide rebuilds
would. Point taken anyways.

> I suppose there's the different between "should" and "must" which takes
> some of the pressure off (although I suspect that will just lead to people
> ignoring it).

Yeah I would expect it to be ignored a fair bit, "must" or not.

> I think I'd rather push people to use the dh add-on that regenerates all
> the autotools files, since that makes things relatively easy provided that
> the upstream files actually work with the latest versions.

While I also sometimes recommend, this isn't just about autotools.

One example from the archive: firmware-free. Source code for the
embedded software blobs is present but at least one of them no longer
builds or never did.

One example from outside the archive: MegaGlest is a game with images
rendered from 3D models, but those images are no longer possible to
render because the Blender API changed substantially in 2.50. In
addition the pre-rendered images are in the upstream VCS and the
models and blender stuff are in an external repository in an obscure
location.

The other thing is that this is likely to substantially increase build
times, which might not be a good idea.

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAKTje6GNy+WFrq9dV4XuSL-UiNg8rWJumPZRFG5Zq2gceP99RA@mail.gmail.com">http://lists.debian.org/CAKTje6GNy+WFrq9dV4XuSL-UiNg8rWJumPZRFG5Zq2gceP99RA@mail.gmail.com
 
Old 10-25-2011, 08:23 AM
Adam Borowski
 
Default A debian/rules target to rebuild pre-built stuff?

On Tue, Oct 25, 2011 at 11:22:05AM +0800, Paul Wise wrote:
> On Tue, Oct 25, 2011 at 11:06 AM, Russ Allbery wrote:
> > The results of that build seem unlikely to ever be seriously tested
> > currently, which makes me a little dubious that it's worth making a rule
> > about it.
>
> I would wager that majority of such results would be tested during the
> build process.

There is no way to test firmware, images, PDFs, etc.

> > Or, put another way, I'm not sure that this is substantially
> > less controversial than just requiring debian/rules build rebuild
> > everything from source.
>
> That seems unlikely given that there are many many packages using
> autotools-based build systems.

If they use AM_MAINTAINER_MODE and it's "disabled" [1], there's no way to
check if they aren't in DFSG and/or GPL violation by shipping sourceless
code. Forbidding it would at least deal with patching autotools output
rather than source.

I don't get why, when everything else has to be built from source,
AM_MAINTAINER_MODE gets an exemption.


Gnome's stance:
http://blogs.gnome.org/desrt/2011/09/08/am_maintainer_mode-is-not-cool/
GNU's and the inventor of AM_MAINTAINER_MODE's stance:
http://www.gnu.org/s/hello/manual/automake/maintainer_002dmode.html


[1]. The naming is pretty misleading, it should be called
AM_DISABLE_MAINTAINER_MODE. It goes:

* without: ok
* AM_MAINTAINER_MODE, --disable-maintainer-mode: sourceless code
* AM_MAINTAINER_MODE, --enable-maintainer-mode: ok

--
1KB // Yo momma uses IPv4!
 
Old 10-25-2011, 09:57 AM
Ben Hutchings
 
Default A debian/rules target to rebuild pre-built stuff?

On Tue, 2011-10-25 at 12:10 +0800, Paul Wise wrote:
[...]
> One example from the archive: firmware-free. Source code for the
> embedded software blobs is present but at least one of them no longer
> builds or never did.
[...]

I've been working on that specific case...

Ben.

--
Ben Hutchings
DNRC Motto: I can please only one person per day.
Today is not your day. Tomorrow isn't looking good either.
 
Old 10-25-2011, 10:02 AM
Paul Wise
 
Default A debian/rules target to rebuild pre-built stuff?

On Tue, Oct 25, 2011 at 5:57 PM, Ben Hutchings wrote:
> On Tue, 2011-10-25 at 12:10 +0800, Paul Wise wrote:
> [...]
>> One example from the archive: firmware-free. Source code for the
>> embedded software blobs is present but at least one of them no longer
>> builds or never did.
> [...]
>
> I've been working on that specific case...

Excellent news, thanks.

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAKTje6FuLZ4JELtCmGbpA=O-ZdFqOUbgNRhYBhrMQv06mNxkew@mail.gmail.com">http://lists.debian.org/CAKTje6FuLZ4JELtCmGbpA=O-ZdFqOUbgNRhYBhrMQv06mNxkew@mail.gmail.com
 
Old 10-25-2011, 12:28 PM
Henrique de Moraes Holschuh
 
Default A debian/rules target to rebuild pre-built stuff?

On Tue, 25 Oct 2011, Adam Borowski wrote:
> If they use AM_MAINTAINER_MODE and it's "disabled" [1], there's no way to
> check if they aren't in DFSG and/or GPL violation by shipping sourceless
> code. Forbidding it would at least deal with patching autotools output
> rather than source.

As a rule, you are supposed to get rid of all autogenerated files and
rebuild them from scratch when packaging for Debian. AM_MAINTAINER_MODE
changes nothing in that case, as you will readly notice any upstream
breakage when you try to build the package after importing a new upstream
release.

Granted, there are exceptions to all rules, but autotools has not been one
for more than a decade. In fact, we had a LOT of breakage over the years
because maintainers built packages with whatever buggy autotooling upstream
used, instead of retooling. libtool was the worst source of nastyness, but
even the simple stuff like GNU config (config.sub/.guess) caused some
trouble.

> I don't get why, when everything else has to be built from source,
> AM_MAINTAINER_MODE gets an exemption.

AFAIK, it doesn't get any exceptions, at least not in Debian.

What happens is that we do not pressure people nearly as much as we could at
the "rebuild everything autogenerated from source when not impossible" thing
in Debian, which means maintainers don't get pressured into retooling build
systems and rebuilding intermediate files.

And that happens because a large number of maintainers and upstreams really
don't care at all for the build system of whatever they're building. Not
that I know of any good buildsystems: autotools and cmake are both hideous.
Still, one has to strike a balance between distro quality, and taking out
the fun of maintaining packages for a lot of people... so we don't complain
of a half-done job until it breaks something.

After all, chances are good that you'll find that *you* have to fix
upstream's build-system and keep it in good order. Not everyone thinks this
should be part of the downstream maintainer's call of duty.

--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111025122824.GA6368@khazad-dum.debian.net">http://lists.debian.org/20111025122824.GA6368@khazad-dum.debian.net
 
Old 10-25-2011, 03:22 PM
"Bernhard R. Link"
 
Default A debian/rules target to rebuild pre-built stuff?

* Henrique de Moraes Holschuh <hmh@debian.org> [111025 14:28]:
> On Tue, 25 Oct 2011, Adam Borowski wrote:
> > If they use AM_MAINTAINER_MODE and it's "disabled" [1], there's no way to
> > check if they aren't in DFSG and/or GPL violation by shipping sourceless
> > code. Forbidding it would at least deal with patching autotools output
> > rather than source.
>
> As a rule, you are supposed to get rid of all autogenerated files and
> rebuild them from scratch when packaging for Debian. AM_MAINTAINER_MODE
> changes nothing in that case, as you will readly notice any upstream
> breakage when you try to build the package after importing a new upstream
> release.

As another rule, you are not supposed to derivate much from upstream
without a reason. Using build scripts generated with a different version
is such a derivation.

So while you are supposed to redo them, you are also supposed to keep
them, and it depends on a lot of other factors what is the best thing to
do.

> Granted, there are exceptions to all rules, but autotools has not been one
> for more than a decade. In fact, we had a LOT of breakage over the years
> because maintainers built packages with whatever buggy autotooling upstream
> used, instead of retooling. libtool was the worst source of nastyness, but
> even the simple stuff like GNU config (config.sub/.guess) caused some
> trouble.

Libtool is quite a beast, much different to autoconf and automake.
Upstreams needing config.sub/.guess usually means there is some ugliness
hidden in there (most of the time that ugliness it called libtool,
though).

When using libtool I guess it makes sense to retool it, but I think
without libtool it depends how much changes you want to do to the
build system.

Bernhard R. Link


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111025152231.GA26232@server.brlink.eu">http://lists.debian.org/20111025152231.GA26232@server.brlink.eu
 

Thread Tools




All times are GMT. The time now is 02:49 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org