FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian GCC

 
 
LinkBack Thread Tools
 
Old 10-29-2010, 05:22 PM
"brian m. carlson"
 
Default DSO linking changes for wheezy

On Fri, Oct 29, 2010 at 03:43:57PM +0200, Matthias Klose wrote:
> For wheezy I'm planning to change the linking behaviour for DSOs
> (turning on --as-needed and --no-copy-dt-needed-entries. The
> rationale is summarized in
> http://wiki.debian.org/ToolChain/DSOLinking. I would like to know
> about issues with these changes on some of the Debian ports, and if
> we need to disable one of these changes on some port.

I test my software on SPARC frequently to be sure that it works properly
(and doesn't have alignment problems). I've not seen any problems at
all, so I expect everything will be fine on SPARC.

--
brian m. carlson / brian with sandals: Houston, Texas, US
+1 832 623 2791 | http://www.crustytoothpaste.net/~bmc | My opinion only
OpenPGP: RSA v4 4096b: 88AC E9B2 9196 305B A994 7552 F1BA 225C 0223 B187
 
Old 11-14-2010, 02:06 PM
Roger Leigh
 
Default DSO linking changes for wheezy

On Sun, Nov 14, 2010 at 01:51:49PM +0100, Kurt Roeckx wrote:
> On Sun, Nov 07, 2010 at 04:19:10PM +0000, Roger Leigh wrote:
> > On Fri, Oct 29, 2010 at 03:43:57PM +0200, Matthias Klose wrote:
> > > For wheezy I'm planning to change the linking behaviour for DSOs (turning
> > > on --as-needed and --no-copy-dt-needed-entries. The rationale is
> > > summarized in http://wiki.debian.org/ToolChain/DSOLinking. I would like
> > > to know about issues with these changes on some of the Debian ports, and
> > > if we need to disable one of these changes on some port.
> >
> > While I understand the rationale for --no-copy-dt-needed-entries for
> > preventing encapsulation violations via indirect linking, I don't agree
> > with the use of --as-needed *at all*. If a library has been explicitly
> > linked in, it shouldn't be removed. This is an issue for fixing in
> > individual packages, not in the toolchain.
> >
> > I can understand on using it on a per-package basis, but not in the
> > actual toolchain defaults. The compiler and linker *should not be
> > second-guessing the user*. This can break perfectly legitimate code
> > making use of ELF constructors and other features which won't be
> > picked out just by looking at symbol usage.
>
> People have been claiming that constructors or init section are a
> possible problem. I have yet to see an example where it breaks.

It's not a very widely used feature. I'm sure it's trivial to make
such a test case. Portable software tends not to make use of ELF-
specific features like this, but that's not an excuse for breaking
perfectly legitimate code.

But whether or not there are real life examples, --as-needed is
*fundamentally wrong*. It's deliberately *not doing what the user
requested*, and to make that misfeature the system-wide default
would be entirely inappropriate. If a package wishes to make use
of such a feature after understanding the implications, then they
are free to do so. But to make it the default--I don't think that's
a technically sound decision.


Regards,
Roger

--
.'`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
`- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
 
Old 11-14-2010, 02:24 PM
Roger Leigh
 
Default DSO linking changes for wheezy

On Sun, Nov 14, 2010 at 01:19:08PM +0100, Julien Cristau wrote:
> On Fri, Oct 29, 2010 at 15:43:57 +0200, Matthias Klose wrote:
>
> > For wheezy I'm planning to change the linking behaviour for DSOs
> > (turning on --as-needed and --no-copy-dt-needed-entries. The
> > rationale is summarized in
> > http://wiki.debian.org/ToolChain/DSOLinking. I would like to know
> > about issues with these changes on some of the Debian ports, and if
> > we need to disable one of these changes on some port.
> >
> --no-add-needed sounds like it'll cause a *lot* of build failures for no
> particular gain. I don't think it's a good idea.

This change will definitely cause a lot of link failures; having some
concrete numbers to determine how many would be quite useful here, e.g.
from an archive-wide rebuild.

Example failure case:

#593876 libboost-filesystem-dev: Undeclared indirect dependency of boost_filesystem on boost_system causes link failure

While --no-copy-dt-needed-entries does "fix" programs depending upon
indirect linkage, this is something we've been relying on for over a
decade and has worked quite well in practice. While strict correctness
is nice to have, and I've already fixed my programs to work with strict
linking, I'm not entirely sure why indirect linking is that bad in
practice.

Note that in the above Boost example, you get caught out just due to
some inline functions in headers resulting an a completely unexpected
additional dependency, so the need for linking is there, but would have
otherwise been happily satisfied indirectly. Also, it means that the
user of a library needs to be intimately aware of its internals which
is not good. If the Boost filesystem library changes how it works but
without changing its public interface, I could be screwed again in six
months time. This is partly the fault of Boost for exposing its
internals in its headers, but disallowing indirect linking make it
worse.

Overall, it could be for the best, but it will be painful initially.


Regards,
Roger

--
.'`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
`- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
 
Old 11-15-2010, 01:14 AM
Matt Turner
 
Default DSO linking changes for wheezy

On Sun, Nov 14, 2010 at 8:40 PM, Adam Goode <adam@spicenitz.org> wrote:
> On 11/14/2010 12:42 PM, Matt Turner wrote:
>> Please ignore me if I've misunderstood the situation, firstly.
>>
>> Both Fedora and Gentoo are using --as-needed by default now. And from
>> what I've read (google: site:blog.flameeyes.eu as-needed) --as-needed
>> is certainly useful and prevents lots of unnecessary problems.
>>
>
> I don't believe Fedora (as of 14) is using --as-needed.

<mattst88> airlied_, does Fedora use --as-needed by default? Fedora 14 too?
<airlied_> mattst88: yes


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: AANLkTikfcXtSQS4cZ2OF7NY7xYueV3BbgPdJKO_SrC=g@mail .gmail.com">http://lists.debian.org/AANLkTikfcXtSQS4cZ2OF7NY7xYueV3BbgPdJKO_SrC=g@mail .gmail.com
 
Old 11-15-2010, 05:16 AM
Roland McGrath
 
Default DSO linking changes for wheezy

> <mattst88> airlied_, does Fedora use --as-needed by default? Fedora 14 too?
> <airlied_> mattst88: yes

The naming of the options makes people easily confused.
--no-add-needed is the only option Fedora's gcc passes.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20101115061626.8F8B22C4E5@topped-with-meat.com">http://lists.debian.org/20101115061626.8F8B22C4E5@topped-with-meat.com
 
Old 11-15-2010, 08:57 PM
Matthias Klose
 
Default DSO linking changes for wheezy

On 15.11.2010 07:16, Roland McGrath wrote:

<mattst88> airlied_, does Fedora use --as-needed by default? Fedora 14 too?
<airlied_> mattst88: yes


The naming of the options makes people easily confused.
--no-add-needed is the only option Fedora's gcc passes.


yes, OpenSuse is using --as-needed, but not --no-add-needed.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4CE1ACB3.1090508@debian.org">http://lists.debian.org/4CE1ACB3.1090508@debian.org
 
Old 11-15-2010, 09:22 PM
Roland McGrath
 
Default DSO linking changes for wheezy

> On 15.11.2010 07:16, Roland McGrath wrote:
> yes, OpenSuse is using --as-needed, but not --no-add-needed.

That is a pretty nutty choice.


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20101115222235.2B3672C075@topped-with-meat.com">http://lists.debian.org/20101115222235.2B3672C075@topped-with-meat.com
 
Old 11-15-2010, 11:24 PM
Roger Leigh
 
Default DSO linking changes for wheezy

On Mon, Nov 15, 2010 at 11:02:57PM +0100, Matthias Klose wrote:
> On 14.11.2010 16:06, Roger Leigh wrote:
>>>> While I understand the rationale for --no-copy-dt-needed-entries for
>>>> preventing encapsulation violations via indirect linking, I don't agree
>>>> with the use of --as-needed *at all*. If a library has been explicitly
>>>> linked in, it shouldn't be removed. This is an issue for fixing in
>>>> individual packages, not in the toolchain.
>>>>
>>>> I can understand on using it on a per-package basis, but not in the
>>>> actual toolchain defaults. The compiler and linker *should not be
>>>> second-guessing the user*. This can break perfectly legitimate code
>>>> making use of ELF constructors and other features which won't be
>>>> picked out just by looking at symbol usage.
>>>
>>> People have been claiming that constructors or init section are a
>>> possible problem. I have yet to see an example where it breaks.
>>
>> It's not a very widely used feature. I'm sure it's trivial to make
>> such a test case. Portable software tends not to make use of ELF-
>> specific features like this, but that's not an excuse for breaking
>> perfectly legitimate code.
>>
>> But whether or not there are real life examples, --as-needed is
>> *fundamentally wrong*. It's deliberately *not doing what the user
>> requested*, and to make that misfeature the system-wide default
>> would be entirely inappropriate. If a package wishes to make use
>> of such a feature after understanding the implications, then they
>> are free to do so. But to make it the default--I don't think that's
>> a technically sound decision.
>
> maybe, and fix it in N - ~100 packages? Or fix the ~100 packages? The
> point of injection is for discussion. I would prefer having this set in
> dpkg-buildflags, and then disabled by these ~100 packages. Note that
> this is probably the same like modifying the N - ~100 packages, as almost
> no package respects dpkg-buildflags yet.

What's the actual problem --as-needed is trying to solve?

The answer is mainly unwanted libraries being linked in as a result
of using pkg-config (and various other -config variants), though there
are other, lesser, culprits. The pkg-config .pc files for gtk, gnome
and other libraries add in many libraries, most of which aren't
typically needed.

The solution: fix the .pc files!

Using --as-needed is merely papering over the actual root problem.
It "fixes" the symptoms, but it's not addressing the actual cause.
The number of packages providing broken .pc files is not large, and
the number breaking due to relying on this brokenness is likely
just as small.

Other libraries being linked unnecessarily can be removed on a
per-package basis. lintian is warning about this, so most developers
should be aware of the problem.

Damaging our toolchain to work around buggy build scripts is wrong; we
should just fix the scripts!


Regards,
Roger

--
.'`. Roger Leigh
: :' : Debian GNU/Linux http://people.debian.org/~rleigh/
`. `' Printing on GNU/Linux? http://gutenprint.sourceforge.net/
`- GPG Public Key: 0x25BFB848 Please GPG sign your mail.
 
Old 11-15-2010, 11:37 PM
Matthias Klose
 
Default DSO linking changes for wheezy

On 16.11.2010 01:24, Roger Leigh wrote:

On Mon, Nov 15, 2010 at 11:02:57PM +0100, Matthias Klose wrote:

On 14.11.2010 16:06, Roger Leigh wrote:

While I understand the rationale for --no-copy-dt-needed-entries for
preventing encapsulation violations via indirect linking, I don't agree
with the use of --as-needed *at all*. If a library has been explicitly
linked in, it shouldn't be removed. This is an issue for fixing in
individual packages, not in the toolchain.

I can understand on using it on a per-package basis, but not in the
actual toolchain defaults. The compiler and linker *should not be
second-guessing the user*. This can break perfectly legitimate code
making use of ELF constructors and other features which won't be
picked out just by looking at symbol usage.


People have been claiming that constructors or init section are a
possible problem. I have yet to see an example where it breaks.


It's not a very widely used feature. I'm sure it's trivial to make
such a test case. Portable software tends not to make use of ELF-
specific features like this, but that's not an excuse for breaking
perfectly legitimate code.

But whether or not there are real life examples, --as-needed is
*fundamentally wrong*. It's deliberately *not doing what the user
requested*, and to make that misfeature the system-wide default
would be entirely inappropriate. If a package wishes to make use
of such a feature after understanding the implications, then they
are free to do so. But to make it the default--I don't think that's
a technically sound decision.


maybe, and fix it in N - ~100 packages? Or fix the ~100 packages? The
point of injection is for discussion. I would prefer having this set in
dpkg-buildflags, and then disabled by these ~100 packages. Note that
this is probably the same like modifying the N - ~100 packages, as almost
no package respects dpkg-buildflags yet.


What's the actual problem --as-needed is trying to solve?


why did I explain it in the wiki?


The answer is mainly unwanted libraries being linked in as a result
of using pkg-config (and various other -config variants), though there
are other, lesser, culprits. The pkg-config .pc files for gtk, gnome
and other libraries add in many libraries, most of which aren't
typically needed.

The solution: fix the .pc files!


and add more .pc files? Definitely not. I didn't see that many packages where
different binaries/libaries were linked with a different set of libraries.
Usually this is already introduced by upstreams.



Using --as-needed is merely papering over the actual root problem.
It "fixes" the symptoms, but it's not addressing the actual cause.
The number of packages providing broken .pc files is not large, and
the number breaking due to relying on this brokenness is likely
just as small.

Other libraries being linked unnecessarily can be removed on a
per-package basis. lintian is warning about this, so most developers
should be aware of the problem.

Damaging our toolchain to work around buggy build scripts is wrong; we
should just fix the scripts!


again, this is not a script/pkgconfig problem only.

Matthias


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4CE1D23F.7000506@debian.org">http://lists.debian.org/4CE1D23F.7000506@debian.org
 
Old 11-15-2010, 11:51 PM
Matt Turner
 
Default DSO linking changes for wheezy

On Mon, Nov 15, 2010 at 7:24 PM, Roger Leigh <rleigh@codelibre.net> wrote:
> What's the actual problem --as-needed is trying to solve?
>
> The answer is mainly unwanted libraries being linked in as a result
> of using pkg-config (and various other -config variants), though there
> are other, lesser, culprits. *The pkg-config .pc files for gtk, gnome
> and other libraries add in many libraries, most of which aren't
> typically needed.
>
> The solution: fix the .pc files!
>
> Using --as-needed is merely papering over the actual root problem.
> It "fixes" the symptoms, but it's not addressing the actual cause.
> The number of packages providing broken .pc files is not large, and
> the number breaking due to relying on this brokenness is likely
> just as small.

I can't see why you think --as-needed is fundamentally wrong or unnecessary.

Check out http://www.gentoo.org/proj/en/qa/asneeded.xml

--as-needed has saved tons of time for upgrades like Cairo in Gentoo,
where Cairo had been linked to glitz which is now useless and gone. So
when people upgraded Cairo, all the software that linked against it
(and also unnecessarily linked against glitz) broke because glitz was
missing. Maybe this isn't important for binary distributions. I don't
know.

I mean, I don't particularly care what Debian does. I just find the
fervor with which you fight against --as-needed strange.

Matt


--
To UNSUBSCRIBE, email to debian-gcc-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: AANLkTinSMTH+KrapikUPgrWW9niQqiY6OgmSUXDoxZNP@mail .gmail.com">http://lists.debian.org/AANLkTinSMTH+KrapikUPgrWW9niQqiY6OgmSUXDoxZNP@mail .gmail.com
 

Thread Tools




All times are GMT. The time now is 01:23 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org