FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 06-17-2012, 12:02 PM
Thomas Sachau
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

Duncan schrieb:
> Thomas Sachau posted on Sat, 16 Jun 2012 12:31:40 +0200 as excerpted:
>
>> Since i am not that sure about my ability to write formal specs, i am
>> presenting my first draft for further review and suggestions for
>> improvement.
>
> Just a format suggestion. Call it nitpicky if you want, and yes, my
> client isn't perfect, but I'm sure people with a bit of experience
> writing such specs will tell you I'm not alone...
>
> Several of your points ended up as very long single lines. My client can
> wrap, but that wraps the points as well (so for example 2.1 starts in the
> middle of a line). So I was left with the choice to either massively
> horizontally scroll, or of trying to figure out where one point ended and
> another began, since wrapping it... /wrapped/ it, so points appeared in
> the middle of a line.
>
> Please:
>
> * If you use long lines, leave a vertical space (blank line) between
> points so when a client wraps them, they wrap as individual paragraphs.
>
> * Alternatively, wrap at something sensible. (The traditional wrap for
> posting is 72 chars or so, 80 minus a few to allow a few levels of
> quoting without rewrap. I wouldn't complain at 90, but if you're going
> to bother, you might as well go the standard route and avoid further
> issues.)
>
> Long lines as paragraphs would probably be easier especially early in the
> process when you're modifying a lot, but you still risk (even more)
> limited clients having issues with it. YMMV.
>

I suggest you look for a better client to handle the line wrapping
better. ;-) In the meantime, the same file attached with wrapped lines.

--

Thomas Sachau
Gentoo Linux Developer


For amd64 users, there is sometimes the issue, that they need 32bit libs for
certain packages (e.g. wine or many binary-only packages). Currently,
they only get them prepackaged in binary form with the emul-linux-x86-*
packages. But those packages have a few issues (list does not have to be
complete):

-they only contain a limited set of 32bit packages
-they are precompiled, so the user cannot define his own flags
-they have to be manually maintained and updated


So the idea was to add the ability to compile 32bit packages with support from
the package manager, so there is no need for additional packages to
maintain. After this was originally implemented, it was further extended
to allow cross-compilation for other targets, not only limited to 32bit
packages.


The basic way, how this should work:

1. Check for the current profile being multilib, this means, that the
MULTILIB_ABIS var exists and has more then 1 (space seperated) value.
Additionally, the DEFAULT_ABI var has to be defined and its content should
be part of the MULTILIB_ABIS var. Only continue with the following steps,
if this is true
2.1 Take the entries from MULTILIB_ABIS var and add a USE flag for each of them
in the form of multilib_abi_$value
2.2 Add "abiwrapper" as a USE flag
3. Check, if the package has USE=multilib enabled. Only continue with the
following steps, if this is false.
4. Add a use dependency for each USE flag added in step 2 for all dependencies
except those defined in a space seperated list of the NO_AUTO_FLAG var. The
dependency should then look like category/package[multilib_abi_$value?]
5. Find the first enabled USE flag from the list of step 2, start with
multilib_abi_$DEFAULT_ABI during the search. If none is enabled, use
multilib_abi_$DEFAULT_ABI. In the following, $ABI will reference the $value
part of the selected USE flag.
6. Before the pkg_setup phase is executed, setup the environment as following:
-export CHOST with $CHOST_$DEFAULT_ABI
-export $CC with $CHOST-gcc
-export $CXX with $CHOST-g++
-export $FC with $CHOST-gfortran
-export $CHOST with $CHOST_$ABI
-export $CBUILD with $CHOST_$ABI
-export $CDEFINE with $CDEFINE_$ABI
-export $CCASFLAGS with ${CCASFLAGS:-${CFLAGS}} and append $CFLAGS_$ABI
-export $CFLAGS with $CFLAGS and append $CFLAGS_$ABI
-export $CPPFLAGS with $CPPFLAGS and append $CPPFLAGS_$ABI
-export $CXXFLAGS with $CXXFLAGS and append $CFLAGS_$ABI
-export $FCFLAGS with $FCFLAGS and append $CFLAGS_$ABI
-export $FFLAGS with $FFLAGS and append $CFLAGS_$ABI
-export $ASFLAGS with $ASFLAGS and append $ASFLAGS_$ABI
-export $LDFLAGS with $LDFLAGS and append $CFLAGS_$ABI
-export $PKG_CONFIG_PATH with /usr/$LIBDIR_$ABI/pkgconfig

7. After src_install is finished:
7.1 Execute the following or equivalent code (prep_ml_binaries is a function
from multilib-portage from [1]):
prep_ml_binaries $(find "${D}"usr/bin "${D}"usr/sbin "${D}"bin "${D}"sbin -type f -name '*-config' 2>/dev/null)
7.2 If USE flag abiwrapper is enabled, execute:
local noabi=()
for i in ${MULTILIB_ABIS}; do
noabi+=( ! -name '*-'${i} )
done
if use abiwrapper ; then
for i in $(find "${D}"usr/bin/ "${D}"usr/sbin "${D}"bin "${D}"sbin -type f ${noabi[@]} 2>/dev/null); do
prep_ml_binaries "${i}"
done
fi
7.3. After src_install, do the following checks:
-is $Dusr/include not empty
-is $usr/$LIBDIR_$ABI not empty
-is abiwrapper USE flag enabled and any of /bin /usr/bin /sbin
/usr/sbin not empty
If any of those checks is true, save the content of $D to $D.$ABI. Check if
another USE flag from step 2 is enabled, if no, then continue with step 8,
otherwise reset everything to the state before pkg_setup. Find the next enabled
USE flag from step 5 and continue with that on step 6.
8. If there have been at least 2 USE flags from step 2 enabled, check if the
header files between the different target $ABIs differ. Move the files, that
dont differ, back to $D. If files differ, they are moved to
$Dusr/include/gentoo-multilib/$ABI and the following or equivalent code is
executed in $D ($ALL_ABIS is the content of $MULTILIB_ABIS, for which the USE
flags from step 2 have been enabled) (_create_abi_includes is a function from
multilib-portage from [1]):
set --
for diffabi in ${ALL_ABIS}; do
local define_var=CDEFINE_${diffabi}
set -- "$@" "${!define_var}:${dir}/gentoo-multilib/${diffabi}"
done
_create_abi_includes "$@"
Now move the rest of the content of the temporary dirs back into $D with the
content of $D.$DEFAULT_ABI being the last.
9. pkg_preinst, pkg_postinst, pkg_prerm and pkg_postrm are executed within a
loop of the following form
for each enabled USE flag from step 2 do
setup environment as described in step 6
execute phase
done

[1]: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=blob;f=bin/auto-multilib.sh;hb=refs/heads/multilib

Optional: The code from step 6, 7.1, 7.2 and 8 (for managing the headers) could
be moved into an eclass. Then the PM executes this or an equivalent internal
function with newer versions of that eclass overwrite the internal functions.
That way, those parts could be updated without the need to update change
the PMS or to wait for another EAPI for an update/change.
 
Old 06-17-2012, 05:47 PM
Duncan
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

Thomas Sachau posted on Sun, 17 Jun 2012 14:02:26 +0200 as excerpted:

> I suggest you look for a better client to handle the line wrapping
> better. ;-) In the meantime, the same file attached with wrapped lines.

Thanks. (And I'm actually involved upstream tho not as a coder, so
maybe...)

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
 
Old 06-19-2012, 05:56 PM
Luca Barbato
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/16/2012 12:31 PM, Thomas Sachau wrote:
> Since i am not that sure about my ability to write formal specs, i am
> presenting my first draft for further review and suggestions for
> improvement.

Currently I'm experimenting with evil hack with qemu-static (hopefully
that would make my student life happier later).

What would be nice on a full multilib/multiabi system is to have

/lib/ld-linux-foo-arch.so.2 for each active architecture

/usr/$CTUPLE/ as destdir for each of them

PATH and ld.so.conf pointing to them accordingly.

and possibly split RDEPEND/DEPEND to have HDEPEND to list build
dependencies that need to be run on host.

lu

- --

Luca Barbato
Gentoo/linux
http://dev.gentoo.org/~lu_zero

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.18 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk/gvVYACgkQ6Ex4woTpDjQlaQCg1W/PZxOo/2EDktU0XR6/48nf
9RcAoOg40vFoSIX7gNpD82qdym54MB4m
=hEyq
-----END PGP SIGNATURE-----
 
Old 06-19-2012, 06:14 PM
Thomas Sachau
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

Luca Barbato schrieb:
> On 06/16/2012 12:31 PM, Thomas Sachau wrote:
>> Since i am not that sure about my ability to write formal specs, i am
>> presenting my first draft for further review and suggestions for
>> improvement.
>
> Currently I'm experimenting with evil hack with qemu-static (hopefully
> that would make my student life happier later).
>
> What would be nice on a full multilib/multiabi system is to have
>
> /lib/ld-linux-foo-arch.so.2 for each active architecture

This should be under control by the libc ebuild/package, not controlled
by the package manager.

>
> /usr/$CTUPLE/ as destdir for each of them

if you mean $CTUPLE as an arch, so that every arch should have all files
for each package, then something like that does not exist. It may be
possible with an extended multilib-portage, but since i did not think
about it nor tested it, i cant say for sure.

> and possibly split RDEPEND/DEPEND to have HDEPEND to list build
> dependencies that need to be run on host.

What should the difference between DEPEND and HDEPEND be?

--

Thomas Sachau
Gentoo Linux Developer
 
Old 06-19-2012, 06:16 PM
Thomas Sachau
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

Thomas Sachau schrieb:
> Duncan schrieb:
>> Thomas Sachau posted on Sat, 16 Jun 2012 12:31:40 +0200 as excerpted:
>>
>>> Since i am not that sure about my ability to write formal specs, i am
>>> presenting my first draft for further review and suggestions for
>>> improvement.
>>
>> Just a format suggestion. Call it nitpicky if you want, and yes, my
>> client isn't perfect, but I'm sure people with a bit of experience
>> writing such specs will tell you I'm not alone...
>>
>> Several of your points ended up as very long single lines. My client can
>> wrap, but that wraps the points as well (so for example 2.1 starts in the
>> middle of a line). So I was left with the choice to either massively
>> horizontally scroll, or of trying to figure out where one point ended and
>> another began, since wrapping it... /wrapped/ it, so points appeared in
>> the middle of a line.
>>
>> Please:
>>
>> * If you use long lines, leave a vertical space (blank line) between
>> points so when a client wraps them, they wrap as individual paragraphs.
>>
>> * Alternatively, wrap at something sensible. (The traditional wrap for
>> posting is 72 chars or so, 80 minus a few to allow a few levels of
>> quoting without rewrap. I wouldn't complain at 90, but if you're going
>> to bother, you might as well go the standard route and avoid further
>> issues.)
>>
>> Long lines as paragraphs would probably be easier especially early in the
>> process when you're modifying a lot, but you still risk (even more)
>> limited clients having issues with it. YMMV.
>>
>
> I suggest you look for a better client to handle the line wrapping
> better. ;-) In the meantime, the same file attached with wrapped lines.
>

Since there is again no response at all, it seems like everyone is ok
with this, so i will propose to add this to the next council agenda for
EAPI-5 addition.

--

Thomas Sachau
Gentoo Linux Developer
 
Old 06-19-2012, 06:16 PM
Ciaran McCreesh
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

On Tue, 19 Jun 2012 20:16:39 +0200
Thomas Sachau <tommy@gentoo.org> wrote:
> Since there is again no response at all, it seems like everyone is ok
> with this, so i will propose to add this to the next council agenda
> for EAPI-5 addition.

Got a diff for PMS?

--
Ciaran McCreesh
 
Old 06-19-2012, 06:54 PM
Thomas Sachau
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

Ciaran McCreesh schrieb:
> On Tue, 19 Jun 2012 20:16:39 +0200
> Thomas Sachau <tommy@gentoo.org> wrote:
>> Since there is again no response at all, it seems like everyone is ok
>> with this, so i will propose to add this to the next council agenda
>> for EAPI-5 addition.
>
> Got a diff for PMS?
>

Last time you only requested enough details for implementation, you did
not require a PMS diff, so i wrote more details for the implementation.

If you, Brian (for pkgcore) and zmedico accept this for EAPI-5, i might
look into creating a diff against PMS but until then, i dont want to
waste my time, especially since noone commented on the implementation
details or the technical details and any change would require even more
work to rework/adjust the PMS diff.

--

Thomas Sachau
Gentoo Linux Developer
 
Old 06-19-2012, 07:02 PM
Ciaran McCreesh
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

On Tue, 19 Jun 2012 20:54:07 +0200
Thomas Sachau <tommy@gentoo.org> wrote:
> Ciaran McCreesh schrieb:
> > On Tue, 19 Jun 2012 20:16:39 +0200
> > Thomas Sachau <tommy@gentoo.org> wrote:
> >> Since there is again no response at all, it seems like everyone is
> >> ok with this, so i will propose to add this to the next council
> >> agenda for EAPI-5 addition.
> >
> > Got a diff for PMS?
>
> Last time you only requested enough details for implementation, you
> did not require a PMS diff, so i wrote more details for the
> implementation.
>
> If you, Brian (for pkgcore) and zmedico accept this for EAPI-5, i
> might look into creating a diff against PMS but until then, i dont
> want to waste my time, especially since noone commented on the
> implementation details or the technical details and any change would
> require even more work to rework/adjust the PMS diff.

Here's the thing: I doubt the lack of feedback is down to everyone
being happy with the proposal. I strongly suspect it's because people
look at it and go "huh?". You've provided exactly the kind of
information that no-one cares about (e.g. long lists of variable names,
which will probably just come from a var in profiles), and none of what
really matters.

I think you'll get a better response if you write this up in GLEP form
(i.e. motivation, examples etc) to describe to ebuild developers what's
in it for them, and as a diff against PMS so that package mangler
authors can work out exactly what you're saying.

--
Ciaran McCreesh
 
Old 06-19-2012, 08:36 PM
Brian Harring
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

On Tue, Jun 19, 2012 at 08:54:07PM +0200, Thomas Sachau wrote:
> Ciaran McCreesh schrieb:
> > On Tue, 19 Jun 2012 20:16:39 +0200
> > Thomas Sachau <tommy@gentoo.org> wrote:
> >> Since there is again no response at all, it seems like everyone is ok
> >> with this, so i will propose to add this to the next council agenda
> >> for EAPI-5 addition.
> >
> > Got a diff for PMS?
> >
>
> Last time you only requested enough details for implementation, you did
> not require a PMS diff, so i wrote more details for the implementation.
>
> If you, Brian (for pkgcore) and zmedico accept this for EAPI-5, i might
> look into creating a diff against PMS but until then, i dont want to
> waste my time, especially since noone commented on the implementation
> details or the technical details and any change would require even more
> work to rework/adjust the PMS diff.

You need a glep here frankly; per the norm, if you want things to move
faster, then put in time- aka, generate a patch against PMS, write a
patch for portage, etc, you get the idea. The bit re: a PMS patch is
mostly that in looking at your proposal... well, I personally don't
want to write that patch (nor do I suspect ulm/ciaran do either).

One thing to note; this has been posted for all of 2-3 days; that's
not exactly much time for 1) people to comment, 2) people to frankly
comprehend the quite dense description you wrote.

Please write a glep covering details of the implementation,
background, preferablly why this route over others. Bluntly... clue
everyone else in rather than hoping they'll just sign off on a fairly
opaque list of things. It'll be useful for dev education also-
which is a bit of a requirement for stuff of this sort considering
it's not going to be a magic deploy/shit works everywhere situation I
suspect.

Would also be useful getting commentary from crossdev folk considering
your solution is intended to be (best I can tell) full cross
compilation support, and they've been leading that front for many,
many years.

Cheers-
~brian
 
Old 06-19-2012, 09:07 PM
Thomas Sachau
 
Default spec draft for cross-compile support in future EAPI (EAPI-5)

Brian Harring schrieb:
> On Tue, Jun 19, 2012 at 08:54:07PM +0200, Thomas Sachau wrote:
>> Ciaran McCreesh schrieb:
>>> On Tue, 19 Jun 2012 20:16:39 +0200
>>> Thomas Sachau <tommy@gentoo.org> wrote:
>>>> Since there is again no response at all, it seems like everyone is ok
>>>> with this, so i will propose to add this to the next council agenda
>>>> for EAPI-5 addition.
>>>
>>> Got a diff for PMS?
>>>
>>
>> Last time you only requested enough details for implementation, you did
>> not require a PMS diff, so i wrote more details for the implementation.
>>
>> If you, Brian (for pkgcore) and zmedico accept this for EAPI-5, i might
>> look into creating a diff against PMS but until then, i dont want to
>> waste my time, especially since noone commented on the implementation
>> details or the technical details and any change would require even more
>> work to rework/adjust the PMS diff.
>
> You need a glep here frankly; per the norm, if you want things to move
> faster, then put in time- aka, generate a patch against PMS, write a
> patch for portage, etc, you get the idea.

Since multilib-portage code is just a git branch for the portage code, i
effectively have a patch for portage. ;-)


Before i start investing more time, a question to the PM developers:

Do you prefer having everything hardcoded in PMS or can you accept
outsourcing bigger code pieces into some sort of eclass (i am thinking
about some external code base, which can be duplicated by the package
manager with internal code, but has to be used, if the external eclass
has a newer version/revision then the duplicated internal code)?

I am especially thinking about the setup of the environment and the code
details for the wrappers for binaries and headers, hardcoding those
details into PMS makes it hard to change/fix issues later on.

--

Thomas Sachau
Gentoo Linux Developer
 

Thread Tools




All times are GMT. The time now is 01:00 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org