FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 04-22-2012, 04:44 AM
Steven J Long
 
Default .LIBPATTERNS harmful?

Hi,

I've been working with GNU make quite a lot recently, and I came across the
.LIBPATTERNS variable. This variable means that make expands all -lname
prerequisites via a library path search of /lib and /usr/lib *before* any
command sees it. (It searches local paths set in the makefile first, which
is useful for linking to built libs, though imo the build-system is better
off using -L parameters in LDFLAGS for those.)

You can read about it in 'info make' Section 4.5.6 (just hit / and type
LIBPATTERN<Enter> to find it.)

The default setting is active in make as installed, as it should be, which
you can verify with: make -p -f /dev/null|grep -F LIBPATTERN

I can find nothing overriding it in portage, which makes sense, since in
general one cannot know if the package in question uses gmake .LIBPATTERNS
to link to locally-built libs. However I can't help thinking of it as
harmful for a package manager, since a command like ld would be given a
parameter of say, /usr/lib/libfoo.so, not -lfoo, meaning LDFLAGS would be
irrelevant for its lookup.

My feeling is that build-systems reliant on the default gmake behaviour for
locally-built libs (ie not setting any -L params and also having to link
locally) would be rare, but it's just that: a gut-feeling with no data.
Preferably they'd be marked as such so that the package manager could deal
with that corner-case, while patches to supply local -L params could be
worked on, in advance of submission upstream.

I'd hope upstream would accept them, since it makes cross-development
easier. (One definitely does not want make expanding -lname to a library in
/lib or /usr/lib in that case, and it's better to error out if the library
can't be found than link to host libs.)

The reason I bring it up is because we have been discussing library linkage
issues wrt initramfs. I also seem to recall quite a few blog posts and
discussions of arbitrary linkage to libs in /usr. .LIBPATTERNS not being
empty would certainly explain that.

Regards,
Steve.
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
 
Old 04-22-2012, 03:17 PM
Mike Frysinger
 
Default .LIBPATTERNS harmful?

On Sunday 22 April 2012 00:44:11 Steven J Long wrote:
> I can find nothing overriding it in portage, which makes sense, since in
> general one cannot know if the package in question uses gmake .LIBPATTERNS
> to link to locally-built libs. However I can't help thinking of it as
> harmful for a package manager, since a command like ld would be given a
> parameter of say, /usr/lib/libfoo.so, not -lfoo, meaning LDFLAGS would be
> irrelevant for its lookup.

.LIBPATTERNS only matters if you specify the -lfoo in the dependency, and then
link in via an automatic make variable.

e.g. this:
$ cat Makefile
all: test
test: -lm
$ echo 'main(){}' > test.c
$ make
cc test.c /usr/lib/libm.so -o test

so the easy answer is: don't add -lfoo flags as dependencies to make targets.
if you want to have something link in a library, do:
$ cat Makefile
all: test
test: LDLIBS += -lm
$ make
cc test.c -lm -o test

> I'd hope upstream would accept them, since it makes cross-development
> easier. (One definitely does not want make expanding -lname to a library in
> /lib or /usr/lib in that case, and it's better to error out if the library
> can't be found than link to host libs.)

i've seen this usage in only one or two packages before. and when i notified
the respective upstream, they weren't really doing it on purpose, so a simple
patch (like i showed above) they were fine with merging.
-mike
 
Old 04-22-2012, 10:13 PM
Steven J Long
 
Default .LIBPATTERNS harmful?

Mike Frysinger wrote:

> On Sunday 22 April 2012 00:44:11 Steven J Long wrote:
>> I can find nothing overriding it in portage, which makes sense, since in
>> general one cannot know if the package in question uses gmake
>> .LIBPATTERNS to link to locally-built libs. However I can't help thinking
>> of it as harmful for a package manager, since a command like ld would be
>> given a parameter of say, /usr/lib/libfoo.so, not -lfoo, meaning LDFLAGS
>> would be irrelevant for its lookup.
>
> .LIBPATTERNS only matters if you specify the -lfoo in the dependency, and
> then link in via an automatic make variable.
>
Indeed. But that is accepted, conventional usage of make: it's why $+
exists, for example.

> e.g. this:
> $ cat Makefile
> all: test
> test: -lm
> $ echo 'main(){}' > test.c
> $ make
> cc test.c /usr/lib/libm.so -o test
>
> so the easy answer is: don't add -lfoo flags as dependencies to make
> targets. if you want to have something link in a library, do:
> $ cat Makefile
> all: test
> test: LDLIBS += -lm
> $ make
> cc test.c -lm -o test
>
The problem with target-specific variables, is that they also apply to any
prerequisites (and their prerequisites..) that get updated. While LDLIBS is
only used for linking final executables, it's not a general solution, since
it's possible for an object file to depend on a built executable (eg if the
binary is used to build an input file, or is a test program, or happens to
be a prerequisite of another target in the chain):

$ echo 'main(){}' > bar.c
$ echo 'const char * foo(void) { return "fubar"; }' > foo.c
$ printf '%s
' '#include <stdio.h>' 'const char * foo(void);' 'int main(){'
' printf ("Result: %s
", foo()); }' > test.c

$ cat Makefile
test: foo.o -lm
foo.o: bar
$ make
cc bar.c -o bar
cc -c -o foo.o foo.c
cc test.c foo.o /usr/lib/libm.so -o test

vs:
$ cat Makefile
test: foo.o
test: LDLIBS += -lm
foo.o: bar
$ make
cc bar.c -lm -o bar
cc -c -o foo.o foo.c
cc test.c foo.o -lm -o test

Furthermore, if we apply the method iteratively (let's say bar uses
pthreads) and add:
bar: LDLIBS += -lpthread
..then we get:
cc bar.c -lm -lpthread -o bar
..meaning that test's lib/s will be searched before those specified by bar.
For the gallery, this is all a no-no in general: just because a lib should
be linked into one object, does not mean it should link into another, unless
it's explicitly been specified. Libs are searched in the order they appear
on the command-line, so test's libs would interpose symbols in bar's link.

All this makes the method package-specific, so it has to be done (and
tested) on a case-by-case basis, and imo liable to random breakage due to
interposition on prerequisite linkages. Worse, the value is unpredictable,
since it varies according to which targets are being updated during the make
run (and have thus added to LDLIBS for their prerequisites.) While this
might not matter so much for distro-builds, it can mess up development
builds.

In testing how to fix this, I found running:
make .LIBPATTERNS=
..gave me:
make: *** No rule to make target `-lpthread', needed by `bar'. Stop.
So I just added a .PHONY line:
$ cat Makefile
test: foo.o -lm
foo.o: bar
bar: -lpthread
.PHONY: -lm -lpthread

..and oddly enough, I found this alone, was enough to disable the expansion
of those libs:
$ make
cc bar.c -lpthread -o bar
cc -c -o foo.o foo.c
cc test.c foo.o -lm -o test

I got the same result with: make .LIBPATTERNS:= so using both seems like the
best general solution, since we're then guaranteed make will do no libname
substitution, and we can use conventional -lfoo deps for external libs.

>> I'd hope upstream would accept them, since it makes cross-development
>> easier. (One definitely does not want make expanding -lname to a library
>> in /lib or /usr/lib in that case, and it's better to error out if the
>> library can't be found than link to host libs.)
>
> i've seen this usage in only one or two packages before. and when i
> notified the respective upstream, they weren't really doing it on purpose,
> so a simple patch (like i showed above) they were fine with merging.

Thanks, that's exactly the kind of knowledge I don't have, and it's good to
know that no-one really wants -lfoo looked up and substituted by make.

Adding -lname specified prereqs to .PHONY is a simple fix, although if
they're internal there won't be any lookup done by make. Personally I think
that's a good thing, but I don't know how it'd affect things; for instance a
package building okay now with a dep on an internal -lfoo which is expanded
to a target which can be built.

But as you state, you've not seen it done on purpose (and it would mess up
cross-compiles) so I guess the fix there is for the internal lib to be
specified as a filename, libfoo.so or .a (and perhaps -L .)?

In any event, we seem to agree that we don't want .LIBPATTERNS expansions
happening.

Personally, I feel setting .LIBPATTERNS empty by default, either via make
command-line (or setenv unless the package is marked as needing the
expansions) would be 'correct', in that it would pick up potential problems
straightaway, but I don't have the knowledge to assess the consequences. It
wouldn't affect packages using the LDLIBS+= target-specific setting, at
least, but would break packages using -lfoo prerequisites via automatic
variables, which haven't been patched to use LDLIBS+= or add .PHONY deps.

Might be something to consider for ebuild-developer mode, so new ebuilds
don't come in with the potential for borked linkage.

A case-by-case fix would be to add .LIBPATTERNS:= to the makefile when
adding .PHONY deps.

Anyhow, thanks for discussion and sharing your know-how; it means I now know
how to handle it in our builds at least (.PHONY for external libs specified
as -lfoo, .LIBPATTERNS set empty, filenames for internal libs and -L params
for their directories.)

Regards,
Steve.
--
#friendly-coders -- Where everybody knows your nickname ;-)
 

Thread Tools




All times are GMT. The time now is 04:36 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org