FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 05-31-2010, 07:12 PM
Michał Górny
 
Default toolchain-funcs.eclass: functions to call compiler

Hello,

As suggested by jer@g.o in bug #321165, I would like to raise a proposal
on extending toolchain-funcs.eclass with helper functions to compile
applications directly.

There are many simple applications which come without neither a sophisticated
build system or even a tiny Makefile. In some cases, such applications aren't
even packages as tarball -- a single, compressed source file is published
instead.

In case of these applications, ebuilds inherit toolchain-funcs.eclass to
retrieve compiler information (tc-getCC/tc-getCXX) and call compiler manually,
passing appropriate flags.

This way, we get certain amount of code being repeated in many ebuilds,
slightly differing in the way of calling the compiler and order of arguments
being passed to it (see examples below). Although I haven't heard of any
problems related to that, it is certainly not fool-proof and hard to maintain.

As jer suggested, I'm providing below a listing of example ebuilds calling
the compiler directly and notes on differences between the way they call it.
Similar list could be obtained through calling:
$ grep tc-getCXX.*CXXFLAGS.*-o /usr/portage/*/*/*.ebuild
$ grep tc-getCC.*CFLAGS.*-o /usr/portage/*/*/*.ebuild

C++ examples include:
* app-misc/clockywock - single C++ file, ${LDFLAGS} before ${CXXFLAGS},
* app-pda/iripdb - two C++ files, with echoing the commands, ${LDFLAGS}
ignored,
* app-text/duconv - single C++ file with separate compilation and linking,
${LDFLAGS} after ${CXXFLAGS},
* dev-cpp/tree - single C++ file, ${LDFLAGS} ignored,
* games-arcade/gunocide2ex - multiple C++ files, ${LDFLAGS} after ${CXXFLAGS},
* mail-filter/normalizemime,
* media-sound/hearnet/hearnet,
* net-misc/l7-protocols,
* x11-apps/amlc.

C examples include:
* app-admin/hwreport - single C file, ${LDFLAGS} after ${CFLAGS},
* app-admin/sysrqd - single C file, ${LDFLAGS} ignored,
* app-arch/unlzx - single C file, ${LDFLAGS} after ${CFLAGS},
* app-arch/unmakeself - single C file, '-o' after libraries,
* app-cdr/bchunk - ${LDFLAGS} ignored,
* app-cdr/{cddetect,nrg2iso,uif2iso}
* app-crypt/scsign - '-pthread' at the beginning, '-o' after sources,
* app-emacs/mpg123-el - ${LDFLAGS} ignored,
* app-forensics/{cmospwd,galleta,pasco,rifiuti} - ${LDFLAGS} ignored,
'-o' after sources,
* app-misc/hilite - ${LDFLAGS} before ${CFLAGS},
* app-misc/irtrans-irclient - ${LDFLAGS} ignored,
* app-misc/sl - '-o' after libs, command echoed,
* app-text/unpaper - ${LDFLAGS} before ${CFLAGS}, '-o' after sources,
* dev-games/paklib - ${LDFLAGS} ignored, '-o' after sources,
* dev-java/rxtx - ${LDFLAGS} before ${CFLAGS}, '-o' after sources,
* dev-lang/bff,
* dev-libs/libhash - ${LDFLAGS} ignored,
* dev-libs/libmemcache - ${LDFLAGS} ignored,
* dev-libs/nss - (pseudo-configure check), seems ugly anyway,
* dev-tcltk/tclperl - ${LDFLAGS} ignored, '-o' in the middle of flags,
* dev-tex/tex4ht - ${CPPFLAGS} used (not many ebuilds care about them),
* dev-tinyos/{channelgen,listen,serial-forwarder} - ${LDFLAGS} ignored,
* dev-util/bsdiff,
* dev-util/lockrun - ${LDFLAGS} ignored, '-o' after sources,
* games-puzzle/trimines - creating pseudo-Makefile, ${LDFLAGS} ignored,
'sdl-config --cflags' after '-o', '-o' after sources,
* games-server/tetrix - ${LDFLAGS} ignored,
* games-util/biounzip,
* mail-filter/qmail-scanner - ${LDFLAGS} ignored,
* media-gfx/pngrewrite - ${LDFLAGS} before ${CFLAGS}, '-o' after libs,
* media-sound/cd-discid - echoing commands,
* media-sound/modplugplay,
* media-sound/mserv - '-o' after sources,
* media-sound/mup,
* media-sound/wavegain - ${LDFLAGS} before ${CFLAGS}, '-o' after sources,
defines after '-o',
* media-tv/mythtv - ${LDFLAGS} after sources,
* media-video/cpvts - echoing commands,
* net-analyzer/bigeye - ${LDFLAGS} ignored,
* net-analyzer/nagios-nrpe - ${LDFLAGS} ignored,
* net-analyzer/nipper - ${LDFLAGS} ignored, '-o' after sources,
* net-analyzer/ttcp - ${LDFLAGS} ignored,
* net-dns/bind-dns-keygen - ${LDFLAGS} ignored,
* net-dns/tinystats - ${LDFLAGS} after '-o',
* net-fs/mount-cifs - ${LDFLAGS} ignored,
* net-fs/samba-client,
* net-im/bitlbee - ${LDFLAGS} ignored,
* net-irc/ircd-hybrid - ${LDFLAGS} ignored,
* net-mail/qmailanalog - ${LDFLAGS} ignored, '-o' after sources,
* net-misc/connect,
* net-misc/{datapipe,getdate} - ${LDFLAGS} ignored,
* net-misc/ifenslave,
* net-print/cups-pdf - ${LDFLAGS} ignored,
* net-print/poster - ${LDFLAGS} ignored, '-o' after libs,
* sci-biology/consed - ${LDFLAGS} ignored, '-o' after sources,
* sci-chemistry/imosflm - separate compiling and linking, einfo printing,
* sci-libs/ufconfig - separate compiling and linking,
* sys-apps/netboot-base - ${LDFLAGS} ignored,
* sys-apps/usb_modeswitch,
* sys-apps/util-linux - (configure-like check),
* sys-block/lsiutil,
* sys-devel/gcc-config,
* x11-libs/gl2ps - separate compiling and linking,
* x11-misc/blockdpy,
* x11-misc/xidle - echoing, using eval.

And that's all for gentoo-x86 -- that's over 80 packages, with over 30
ignoring ${LDFLAGS}, not counting lesser issues. There's also 5 similar
packages in Sunrise.

I think that number is large enough to seriously consider adding such
functions into toolchain-funcs.eclass. I'm attaching an example implementation
(as a diff to toolchain-funcs.eclass) and a diff to a sample ebuild
(zpaq-1.10.ebuild from Sunrise, in particular).

Please notice that I'm not really attached to the calling syntax I used there,
and I'm open to hear better ideas. I've decided to pass all additional
arguments (flags) through environmental variables to allow user to intuitively
supply long input file lists.

--
Best regards,
Michał Górny

<http://mgorny.alt.pl>
<xmpp:mgorny@jabber.ru>
 
Old 06-01-2010, 02:11 AM
Ryan Hill
 
Default toolchain-funcs.eclass: functions to call compiler

On Mon, 31 May 2010 21:12:46 +0200
Michał Górny <gentoo@mgorny.alt.pl> wrote:

> * media-sound/wavegain - ${LDFLAGS} before ${CFLAGS}, '-o' after sources,
> defines after '-o',

Just curious why you're pointing out the ordering of options. As far as I
know it doesn't matter (except some LDFLAGS where you can go -Wl,--foo
someblah -Wl,--no-foo otherblah and doesn't really apply here).

But I like the idea of the patch itself.

--
fonts, there's a hole in my neighbourhood
gcc-porting, down which of late i cannot help but fall
wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662
 
Old 06-01-2010, 02:14 AM
Ryan Hill
 
Default toolchain-funcs.eclass: functions to call compiler

On Mon, 31 May 2010 20:11:22 -0600
Ryan Hill <dirtyepic@gentoo.org> wrote:

> On Mon, 31 May 2010 21:12:46 +0200
> Michał Górny <gentoo@mgorny.alt.pl> wrote:
>
> > * media-sound/wavegain - ${LDFLAGS} before ${CFLAGS}, '-o' after sources,
> > defines after '-o',
>
> Just curious why you're pointing out the ordering of options. As far as I
> know it doesn't matter (except some LDFLAGS where you can go -Wl,--foo
> someblah -Wl,--no-foo otherblah and doesn't really apply here).

Nevermind I missed you explaining this. Consistency == good.


--
fonts, there's a hole in my neighbourhood
gcc-porting, down which of late i cannot help but fall
wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662
 
Old 06-02-2010, 07:16 AM
Mike Frysinger
 
Default toolchain-funcs.eclass: functions to call compiler

On Monday, May 31, 2010 15:12:46 Michał Górny wrote:
> There are many simple applications which come without neither a
> sophisticated build system or even a tiny Makefile. In some cases, such
> applications aren't even packages as tarball -- a single, compressed
> source file is published instead.
>
> In case of these applications, ebuilds inherit toolchain-funcs.eclass to
> retrieve compiler information (tc-getCC/tc-getCXX) and call compiler
> manually, passing appropriate flags.

use emake then and leverage make's implicit rules. sys-apps/unscd is a pretty
straight forward example.
-mike
 
Old 06-02-2010, 01:17 PM
Michał Górny
 
Default toolchain-funcs.eclass: functions to call compiler

On Wed, 2 Jun 2010 03:16:12 -0400
Mike Frysinger <vapier@gentoo.org> wrote:

> On Monday, May 31, 2010 15:12:46 Michał Górny wrote:
> > There are many simple applications which come without neither a
> > sophisticated build system or even a tiny Makefile. In some cases,
> > such applications aren't even packages as tarball -- a single,
> > compressed source file is published instead.
> >
> > In case of these applications, ebuilds inherit
> > toolchain-funcs.eclass to retrieve compiler information
> > (tc-getCC/tc-getCXX) and call compiler manually, passing
> > appropriate flags.
>
> use emake then and leverage make's implicit rules.

The implicit make rules are less universal and -- in the fact -- pretty
poor. They're strictly make-dependant, which reduces the amount of
control over their behavior. In my opinion, if we should ever use such
behavior, it should be rather technical implementation of the functions
I'm proposing instead of inline use.

POSIX (man 1p make) defines only simple rule for C files:
$(CC) $(CFLAGS) $(LDFLAGS) -o $@ $<

I wasn't able to find a rule for C++ files. GNU Make seems a little
better but it's documentation is blurry. Although it supports both C
and C++, and pretty wide set of variables, it's still less than
solution proposed by me. And it's only 'de-facto standard', which isn't
guaranteed to be kept unchanged in the future.

Namely, advantages of my solution over directly calling emake is:
1) direct control over how compiler will be called == better
portability,
2) possibility of using any output filename (with emake we'd have to
rename),
3) possibility of clearly using multiple input files,
4) possibility of using any input file suffix,
5) clear separation between user-specified and ebuild-specified flags
(yes, I'm aware of '+=' GNU make extension).

And after all, starting a make jobserver for a single compiler process
doesn't seem really useful.

> sys-apps/unscd is a pretty straight forward example.

In my opinion it's rather a poor example. It's the simplest case
possible -- single file, no additional flags, no libs.

--
Best regards,
Michał Górny

<http://mgorny.alt.pl>
<xmpp:mgorny@jabber.ru>
 
Old 06-02-2010, 09:56 PM
Mike Frysinger
 
Default toolchain-funcs.eclass: functions to call compiler

On Wednesday, June 02, 2010 09:17:00 Michał Górny wrote:
> On Wed, 2 Jun 2010 03:16:12 -0400 Mike Frysinger wrote:
> > On Monday, May 31, 2010 15:12:46 Michał Górny wrote:
> > > There are many simple applications which come without neither a
> > > sophisticated build system or even a tiny Makefile. In some cases,
> > > such applications aren't even packages as tarball -- a single,
> > > compressed source file is published instead.
> > >
> > > In case of these applications, ebuilds inherit
> > > toolchain-funcs.eclass to retrieve compiler information
> > > (tc-getCC/tc-getCXX) and call compiler manually, passing
> > > appropriate flags.
> >
> > use emake then and leverage make's implicit rules.
>
> The implicit make rules are less universal and -- in the fact -- pretty
> poor. They're strictly make-dependant, which reduces the amount of
> control over their behavior. In my opinion, if we should ever use such
> behavior, it should be rather technical implementation of the functions
> I'm proposing instead of inline use.

ebuilds requires GNU make which means the implicit rules are "universal". not
that i really know what you're talking about. the rules are also clearly
documented and support all the proper flags we currently support.

> POSIX (man 1p make) defines only simple rule for C files:
> $(CC) $(CFLAGS) $(LDFLAGS) -o $@ $<

POSIX rules are irrelevant. read the GNU make documentation.

> Namely, advantages of my solution over directly calling emake is:
> 1) direct control over how compiler will be called == better
> portability,

irrelevant. this is like saying we should use POSIX shell code instead of
bash because it's "better portability". we use the tools that are required by
the ebuild system and that means GNU make.

> 2) possibility of using any output filename (with emake we'd have to
> rename),

the -o location is irrelevant in the compile line, and most packages name the
source file the same as the output, so not that big of a deal

> 3) possibility of clearly using multiple input files,

yep

> 4) possibility of using any input file suffix,

perhaps, but very few packages are stupid enough to name a file moocow.txt
when it's really a C file

> 5) clear separation between user-specified and ebuild-specified flags
> (yes, I'm aware of '+=' GNU make extension).

hard to sat it's irrelevant

> And after all, starting a make jobserver for a single compiler process
> doesn't seem really useful.

and serializing multiple files is worse

> > sys-apps/unscd is a pretty straight forward example.
>
> In my opinion it's rather a poor example. It's the simplest case
> possible -- single file, no additional flags, no libs.

and there are many like it

btw, toolchain-funcs is not really the best place to add this. it's an eclass
for querying, not compiling. better to start a new one.
-mike
 
Old 06-05-2010, 01:16 PM
Michał Górny
 
Default toolchain-funcs.eclass: functions to call compiler

On Wed, 2 Jun 2010 17:56:38 -0400
Mike Frysinger <vapier@gentoo.org> wrote:

> On Wednesday, June 02, 2010 09:17:00 Michał Górny wrote:
> > On Wed, 2 Jun 2010 03:16:12 -0400 Mike Frysinger wrote:
> > > use emake then and leverage make's implicit rules.
> >
> > The implicit make rules are less universal and -- in the fact --
> > pretty poor. They're strictly make-dependant, which reduces the
> > amount of control over their behavior. In my opinion, if we should
> > ever use such behavior, it should be rather technical
> > implementation of the functions I'm proposing instead of inline use.
>
> ebuilds requires GNU make which means the implicit rules are
> "universal". not that i really know what you're talking about. the
> rules are also clearly documented and support all the proper flags we
> currently support.

Ok, if we can surely assume 'emake' will always call GNU Make, then I
agree with your solution and withdraw my ideas.

--
Best regards,
Michał Górny

<http://mgorny.alt.pl>
<xmpp:mgorny@jabber.ru>
 
Old 06-06-2010, 04:17 AM
Mike Frysinger
 
Default toolchain-funcs.eclass: functions to call compiler

On Saturday, June 05, 2010 09:16:29 Michał Górny wrote:
> On Wed, 2 Jun 2010 17:56:38 -0400 Mike Frysinger wrote:
> > On Wednesday, June 02, 2010 09:17:00 Michał Górny wrote:
> > > On Wed, 2 Jun 2010 03:16:12 -0400 Mike Frysinger wrote:
> > > > use emake then and leverage make's implicit rules.
> > >
> > > The implicit make rules are less universal and -- in the fact --
> > > pretty poor. They're strictly make-dependant, which reduces the
> > > amount of control over their behavior. In my opinion, if we should
> > > ever use such behavior, it should be rather technical
> > > implementation of the functions I'm proposing instead of inline use.
> >
> > ebuilds requires GNU make which means the implicit rules are
> > "universal". not that i really know what you're talking about. the
> > rules are also clearly documented and support all the proper flags we
> > currently support.
>
> Ok, if we can surely assume 'emake' will always call GNU Make, then I
> agree with your solution and withdraw my ideas.

i'm not saying `emake` covers all solutions you proposed, just that the base
usage isnt nearly as bleak. there are some cases where having a wrapper
around emake would be useful (such as compiling multiple files or linking
things in). so if you wanted to slap something together, we can look at it
further.
-mike
 
Old 06-06-2010, 07:45 AM
Michał Górny
 
Default toolchain-funcs.eclass: functions to call compiler

On Sun, 6 Jun 2010 00:17:27 -0400
Mike Frysinger <vapier@gentoo.org> wrote:

> i'm not saying `emake` covers all solutions you proposed, just that
> the base usage isnt nearly as bleak. there are some cases where
> having a wrapper around emake would be useful (such as compiling
> multiple files or linking things in). so if you wanted to slap
> something together, we can look at it further.

Well, I've already covered one corner case myself, which is the zpaq
ebuild. I would approve a comment on my idea here:

http://overlays.gentoo.org/proj/sunrise/changeset/10641/sunrise/app-arch/zpaq/zpaq-1.10.ebuild

The hard part was that zpaq.cpp had to be compiled to both the execu-
table and the stub object file (with additional -DOPT flag).

I've decided that the simplest and most fail-proof idea was to create
an additional source file, defining 'OPT' and including the real code,
and compiling all of that within a single 'emake' call.

--
Best regards,
Michał Górny

<http://mgorny.alt.pl>
<xmpp:mgorny@jabber.ru>
 
Old 06-08-2010, 10:44 PM
Mike Frysinger
 
Default toolchain-funcs.eclass: functions to call compiler

On Sunday, June 06, 2010 03:45:51 Michał Górny wrote:
> On Sun, 6 Jun 2010 00:17:27 -0400 Mike Frysinger wrote:
> > i'm not saying `emake` covers all solutions you proposed, just that
> > the base usage isnt nearly as bleak. there are some cases where
> > having a wrapper around emake would be useful (such as compiling
> > multiple files or linking things in). so if you wanted to slap
> > something together, we can look at it further.
>
> Well, I've already covered one corner case myself, which is the zpaq
> ebuild. I would approve a comment on my idea here:
>
> http://overlays.gentoo.org/proj/sunrise/changeset/10641/sunrise/app-arch/zp
> aq/zpaq-1.10.ebuild
>
> The hard part was that zpaq.cpp had to be compiled to both the execu-
> table and the stub object file (with additional -DOPT flag).
>
> I've decided that the simplest and most fail-proof idea was to create
> an additional source file, defining 'OPT' and including the real code,
> and compiling all of that within a single 'emake' call.

C/C++ files should always have a newline at the end ... that stub file is
currently missing one.

i dont know anything about zpaq, but it looks like you're installing the bare
.o files ? that's a bit odd. usually things like this are installed as
static archives for people to link in ...

wrt the emake stuff, that looks fine
-mike
 

Thread Tools




All times are GMT. The time now is 09:05 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org