FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian dpkg

 
 
LinkBack Thread Tools
 
Old 07-10-2008, 10:02 PM
Raphael Hertzog
 
Default New Build-Options field and build-arch option, please review

Hello,

in order to fix #229357 I decided to add a new Build-Options field.
I modified Dpkg::BuildOptions to parse this field and DEB_BUILD_OPTIONS.
And I added support for a build-arch option, that if present, will let
dpkg-buildpackage call debian/rules build-arch and build-indep.

It's not obvious that this was the right choice when you think of the
currently existing build options but once you start thinking of possible
additions (as requested in #489771), it becomes more evident that it makes
sense. Even if some build options should really only be used in
the field while others should only be used in the environment variable,
the possibility to override the former with the latter is nice.

The current patchset is available in my public repository but I'll attach
it as well so that you can easily review it. I intend to merge it this
week-end after some tests but feel free to test and comment in the mean
time.

http://git.debian.org/?p=users/hertzog/dpkg.git;a=shortlog;h=refs/heads/pu/bug229357-build-options

The patchset only applies on top of master.

Cheers,
--
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/
 
Old 07-10-2008, 11:19 PM
Felipe Sateler
 
Default New Build-Options field and build-arch option, please review

El 10/07/08 18:02 Raphael Hertzog escribió:
> Hello,
>
> in order to fix #229357 I decided to add a new Build-Options field.
> I modified Dpkg::BuildOptions to parse this field and DEB_BUILD_OPTIONS.
> And I added support for a build-arch option, that if present, will let
> dpkg-buildpackage call debian/rules build-arch and build-indep.
>
> It's not obvious that this was the right choice when you think of the
> currently existing build options but once you start thinking of possible
> additions (as requested in #489771), it becomes more evident that it makes
> sense. Even if some build options should really only be used in
> the field while others should only be used in the environment variable,
> the possibility to override the former with the latter is nice.

I'm not really sure this is right. There are two things that we want to do
here: declare that a package supports something, and asking the package to do
something. This difference is blurred now, and I think it is confusing.
OTOH, it gives the benefit of being able to ignore the package capabilities
via the environment (ie, unset a given option).
I fear it will give rise to abuses such as setting parallel=n in the control
file.


Saludos,
Felipe Sateler
 
Old 07-11-2008, 07:10 AM
Raphael Hertzog
 
Default New Build-Options field and build-arch option, please review

On Thu, 10 Jul 2008, Felipe Sateler wrote:
> El 10/07/08 18:02 Raphael Hertzog escribió:
> > Hello,
> >
> > in order to fix #229357 I decided to add a new Build-Options field.
> > I modified Dpkg::BuildOptions to parse this field and DEB_BUILD_OPTIONS.
> > And I added support for a build-arch option, that if present, will let
> > dpkg-buildpackage call debian/rules build-arch and build-indep.
> >
> > It's not obvious that this was the right choice when you think of the
> > currently existing build options but once you start thinking of possible
> > additions (as requested in #489771), it becomes more evident that it makes
> > sense. Even if some build options should really only be used in
> > the field while others should only be used in the environment variable,
> > the possibility to override the former with the latter is nice.
>
> I'm not really sure this is right. There are two things that we want to do
> here: declare that a package supports something, and asking the package to do
> something. This difference is blurred now, and I think it is confusing.

Even if there's only two things, the fact is that the package maintainer
wants not only to decide what is supported but he might also want to
enable some features... if you check the case that I listed above, we also
want to use Build-Options to _enable_ specific hardening measures. Because
the maintainer knows best which hardening measures should be enabled. But
we also want the builder to be able to override them for example to test
if the package now supports a previously disabled hardening measure.

The meaning of each build options is specific to each, there's no global
rule that works for all cases. That's why we have documentation of each
option in dpkg-buildpackage.

> I fear it will give rise to abuses such as setting parallel=n in the control
> file.

There are dozens of ways to "abuse" any interface if you choose to use
it in a way that contradicts the documentation. But that's not a reason
to limit the flexibility offered by an interface.

Cheers,
--
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 07-11-2008, 09:49 AM
Bill Allombert
 
Default New Build-Options field and build-arch option, please review

On Thu, Jul 10, 2008 at 07:19:16PM -0400, Felipe Sateler wrote:
> El 10/07/08 18:02 Raphael Hertzog escribió:
> > Hello,
> >
> > in order to fix #229357 I decided to add a new Build-Options field.
> > I modified Dpkg::BuildOptions to parse this field and DEB_BUILD_OPTIONS.
> > And I added support for a build-arch option, that if present, will let
> > dpkg-buildpackage call debian/rules build-arch and build-indep.
> >
> > It's not obvious that this was the right choice when you think of the

Maybe it is not obvious, but since noone proposed another working
solution in the ten years this issue exists, there is no alternative.

> > currently existing build options but once you start thinking of possible
> > additions (as requested in #489771), it becomes more evident that it makes
> > sense. Even if some build options should really only be used in
> > the field while others should only be used in the environment variable,
> > the possibility to override the former with the latter is nice.
>
> I'm not really sure this is right. There are two things that we want to do
> here: declare that a package supports something, and asking the package to do
> something. This difference is blurred now, and I think it is confusing.
> OTOH, it gives the benefit of being able to ignore the package capabilities
> via the environment (ie, unset a given option).
> I fear it will give rise to abuses such as setting parallel=n in the control
> file.

I concur. This also create a namespace problem by conflating the
'Build-Options' namespace with the DEB_BUILD_OPTIONS namespace.
Since a developer can put virtually anything in DEB_BUILD_OPTIONS
(and check for it in debian/rules) even if it is not mentionned
in policy, this is a real issue.

Cheers,
--
Bill. <ballombe@debian.org>

Imagine a large red swirl here.


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 07-11-2008, 11:07 AM
Joey Hess
 
Default New Build-Options field and build-arch option, please review

Raphael Hertzog wrote:
> Even if there's only two things, the fact is that the package maintainer
> wants not only to decide what is supported but he might also want to
> enable some features...

Did you think about having two fields, one to specify the set of
supported options, and one to allow setting defaults?

FWIW, Manoj, Steve, Yuri[1] and I had a good chat about this on the
train across Scotland last summer.

For some types of options, it makes sense to not just declare that
they're supported, but that some particular combinations of options is
supported, while declaring other combinations as unsupported. This would
be particularly useful when setting compile options (including librarary
link combinations).

Hmm, my notebook[2] from that trip suggests the following syntax:

Build-Options: strip, debug, bar, foo, !foo+bar

Indicating that foo and bar cannot be combined.

Also, I think it would be a good idea to explicitly make "x-foo" be
reserved for non-standard options.

--
see shy jo

[1] A gentoo guy. Who better to discuss build options. ;-)
[2] http://farm2.static.flickr.com/1121/639896787_dfe8d0f8c2.jpg
 
Old 07-11-2008, 04:47 PM
Russ Allbery
 
Default New Build-Options field and build-arch option, please review

Raphael Hertzog <hertzog@debian.org> writes:

> Even if there's only two things, the fact is that the package maintainer
> wants not only to decide what is supported but he might also want to
> enable some features... if you check the case that I listed above, we
> also want to use Build-Options to _enable_ specific hardening
> measures. Because the maintainer knows best which hardening measures
> should be enabled. But we also want the builder to be able to override
> them for example to test if the package now supports a previously
> disabled hardening measure.

This doesn't make sense to me. The maintainer writes debian/rules; why
would they need to change Build-Options in debian/control to enable
anything about the build?

I'd rather see Build-Options in debian/control be clearly defined as
capabilities that the package supports and not used as a substitute for
the existing DEB_BUILD_OPTIONS method of controlling what the build does
in practice. (And I'd prefer it to be called Build-Options-Supported or
something along those lines.) I think this still fits for #489771; the
presence of the hardening option in Build-Options-Supported indicates that
the package can usefully be built with hardening (it doesn't cause the
package build to break or the binaries to malfunction). If the package
maintainer wants the package to always be built with those options, they
should make that change directly in debian/rules, not via this method.
They're going to have to test each flag that goes into the hardening
options separately anyway to make sure that it works (the current proposed
hardening flags break many packages, and if you follow debian/changelog
files, you'll see that many maintainers have added them blindly and then
had to roll back when they break).

Using a debian/control field to set DEB_BUILD_OPTIONS in dpkg-buildpackage
is a solution looking for a problem, IMO, and I'd rather not see that
tangled up with the much-needed problem of specifying which options a
package supports and finally dealing with the whole build-arch/build-indep
mess.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 07-13-2008, 06:18 AM
Goswin von Brederlow
 
Default New Build-Options field and build-arch option, please review

Joey Hess <joeyh@debian.org> writes:

> Raphael Hertzog wrote:
>> Even if there's only two things, the fact is that the package maintainer
>> wants not only to decide what is supported but he might also want to
>> enable some features...
>
> Did you think about having two fields, one to specify the set of
> supported options, and one to allow setting defaults?
>
> FWIW, Manoj, Steve, Yuri[1] and I had a good chat about this on the
> train across Scotland last summer.
>
> For some types of options, it makes sense to not just declare that
> they're supported, but that some particular combinations of options is
> supported, while declaring other combinations as unsupported. This would
> be particularly useful when setting compile options (including librarary
> link combinations).

Similary the user might want to enable options that are supported,
force enable (override) options that are undeclared or forbidden or
disable options that are declared as default. It should be possible to
specify a delta or mask to the packages options and not just set the
options, e.g. to say "use default options + bar - foo".

> Hmm, my notebook[2] from that trip suggests the following syntax:
>
> Build-Options: strip, debug, bar, foo, !foo+bar
>
> Indicating that foo and bar cannot be combined.

Should that be more like

Build-Options: strip, debug
Build-Supported: bar, foo, !foo+bar, !baz

Indicating that strip and debug should be used by default. Further bar
and foo may be enable but not together and baz must not be used.

Not that !baz would be different from not listing baz. !baz would be a
strong indicator that the build will break.

> Also, I think it would be a good idea to explicitly make "x-foo" be
> reserved for non-standard options.


Further I think it would be good if one could say "use bar if
supported" in an environment variable or conffile. For example the
user might want to use parallel building with 2 cores if the package
supports it. But if the package does not then there should be no
error. Otherwise one would have to change the environment for every
build according to the package capabilities.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 07-13-2008, 09:18 AM
Raphael Hertzog
 
Default New Build-Options field and build-arch option, please review

Hi,

thanks for your answers.

On Fri, 11 Jul 2008, Joey Hess wrote:
> Raphael Hertzog wrote:
> > Even if there's only two things, the fact is that the package maintainer
> > wants not only to decide what is supported but he might also want to
> > enable some features...
>
> Did you think about having two fields, one to specify the set of
> supported options, and one to allow setting defaults?

This might be possible but the limit is not always very clear: in the case
of build-arch, the simple fact that it's supported means that it's enabled
by default.

It doesn't really make sense to require the maintainer to put it in
Build-Options-Supported and also in Build-Options.

> For some types of options, it makes sense to not just declare that
> they're supported, but that some particular combinations of options is
> supported, while declaring other combinations as unsupported. This would
> be particularly useful when setting compile options (including librarary
> link combinations).

You're thinking of options in terms of configure flags, is that right?
--with-mysql might be incompatible with --with-postgresql but both might
coexist with yet another feature.

I'm not sure I want to go that far in the logic of Build-Options. I
certainly would consider nice to have a sort of "flavor" mechanism
where the maintainer can propose various combinations of options.

Build-Options-Supported: flavor=mysql,postgresql,oracle,all
Build-Options-Default: flavor=all

But we could also express this with:
Build-Options: possible-flavor=mysql,postgresql,oracle,all
default-flavor=all

(Well the set of prefix can be discussed and set in stone, exactly like
I have used the "no-" prefix to disable an option previously set)

> Also, I think it would be a good idea to explicitly make "x-foo" be
> reserved for non-standard options.

Fine.

On Fri, 11 Jul 2008, Russ Allbery wrote:
> Raphael Hertzog <hertzog@debian.org> writes:
>
> > Even if there's only two things, the fact is that the package maintainer
> > wants not only to decide what is supported but he might also want to
> > enable some features... if you check the case that I listed above, we
> > also want to use Build-Options to _enable_ specific hardening
> > measures. Because the maintainer knows best which hardening measures
> > should be enabled. But we also want the builder to be able to override
> > them for example to test if the package now supports a previously
> > disabled hardening measure.
>
> This doesn't make sense to me. The maintainer writes debian/rules; why
> would they need to change Build-Options in debian/control to enable
> anything about the build?

Because they want that anyone can easily rebuild it with that option
disabled?

> I'd rather see Build-Options in debian/control be clearly defined as
> capabilities that the package supports and not used as a substitute for
> the existing DEB_BUILD_OPTIONS method of controlling what the build does
> in practice. (And I'd prefer it to be called Build-Options-Supported or
> something along those lines.) I think this still fits for #489771; the
> presence of the hardening option in Build-Options-Supported indicates that
> the package can usefully be built with hardening (it doesn't cause the
> package build to break or the binaries to malfunction).

Separating the two meanings is always possible, see above for a
discussion.

> If the package maintainer wants the package to always be built with
> those options, they should make that change directly in debian/rules,
> not via this method.

Why? (and it's not "always", it's by _default_)

I find it rather nice that we have a common way to enable this for all
packages: add a hardening-wrapper to Build-Depends, add the option
indicating which of the hardenings flags to enable, and you're done
and it works for all packages.

Of course, you can also set the right variables in debian/rules directly
but then you make it complex for anyone to disable those build options
(for example to verify if a failure can be attributed to one of these
hardening options).

Cheers,
--
Raphaël Hertzog

Le best-seller français mis à jour pour Debian Etch :
http://www.ouaza.com/livre/admin-debian/


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 07-13-2008, 06:31 PM
Russ Allbery
 
Default New Build-Options field and build-arch option, please review

Raphael Hertzog <hertzog@debian.org> writes:
> On Fri, 11 Jul 2008, Russ Allbery wrote:

>> This doesn't make sense to me. The maintainer writes debian/rules; why
>> would they need to change Build-Options in debian/control to enable
>> anything about the build?

> Because they want that anyone can easily rebuild it with that option
> disabled?

That is already supported using the existing DEB_BUILD_OPTIONS mechanism.

I may be confused about your mental model here, but it seems like you're
moving rules about how the package is built from the package itself into
dpkg-buildpackage. If that's really what's happening, I think that is a
truly dreadful idea and strongly object. It should be possible to build
the package using whatever flags and options are the default by running
debian/rules build without involving dpkg-buildpackage at all, which
implies that the package should not be relying on dpkg-buildpackage to
provide compiler and linker flags. Those defaults should be in
debian/rules, just as they always have been for Debian packages.

If some set of flags, such as hardening, should be possible to easily
disable, this is exactly the same case as we have right now with
optimization and with stripping. The way to support that is to specify
another DEB_BUILD_OPTIONS flag which, if set, instructs the package to
modify its behavior accordingly. Furthermore, that allows the package
maintainer to provide more useful defaults specific to that package, such
as exactly the hardening flags that *that* package supports, rather than
some default (and possibly changing) set from dpkg-buildpackage.

DEB_BUILD_OPTIONS then stays clearly semantically separate from the
Build-Options-Supported field; the latter specifies which interfaces the
package supports, and the former is the way to actually *use* those
interfaces, with some exceptions for interfaces that can be used other
ways (such as build-arch/build-indep).

>> If the package maintainer wants the package to always be built with
>> those options, they should make that change directly in debian/rules,
>> not via this method.

> Why? (and it's not "always", it's by _default_)

See above. By moving the logic from debian/rules into dpkg-buildpackage,
we would be breaking a common workflow when working with packages.
Running debian/rules build in an unpacked source package to test would no
longer be a reasonable development step since you may get a completely
different compile than dpkg-buildpackage would give you.

I think the way that optimization and stripping are handled right now
works fairly well in practice, and I think we should be building on that
as a model, not replacing it with some entirely different method that
relies on additional external programs to wrap debian/rules.

The choice between always and by default can be handled using the existing
DEB_BUILD_OPTIONS mechanism just as optimization and stripping are now.

> I find it rather nice that we have a common way to enable this for all
> packages: add a hardening-wrapper to Build-Depends, add the option
> indicating which of the hardenings flags to enable, and you're done and
> it works for all packages.

Instead of doing that, you add hardening-wrapper to Build-Depends and
modify debian/rules to invoke it. The process is just as simple.

> Of course, you can also set the right variables in debian/rules directly
> but then you make it complex for anyone to disable those build options
> (for example to verify if a failure can be attributed to one of these
> hardening options).

Not if you implement a DEB_BUILD_OPTIONS flag at the same time. You can
then make hardening-wrapper trigger off of the DEB_BUILD_OPTIONS flag and
the package maintainer doesn't even have to handle it directly (very
similar to how debhelper packages let dh_strip handle DEB_BUILD_OPTIONS
for that flag).

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 07-13-2008, 06:55 PM
Russ Allbery
 
Default New Build-Options field and build-arch option, please review

Goswin von Brederlow <goswin-v-b@web.de> writes:

> Further I think it would be good if one could say "use bar if supported"
> in an environment variable or conffile. For example the user might want
> to use parallel building with 2 cores if the package supports it. But if
> the package does not then there should be no error.

The existing DEB_BUILD_OPTIONS mechanism already supports exactly these
semantics for the parallel=N option, so as long as we don't break what we
already have working, this should continue to behave appropriately and we
can add additional options for similar cases.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 08:07 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org