FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 12-01-2010, 05:57 PM
Thomas Sachau
 
Default MULTI_ABI support addition to main tree portage

Hi,

i have already written about this some months ago and updated the code in relation to the comments
especially from vapier.

Basicly, it does now first set abi-specific vars (like CC, CFLAGS and others (setup_abi_env function
in bin/auto-multilib.sh contains the full list), then does build the package as usual. If additional
ABIs are requested, it checks after src_install, if there are possible ABI-specific files (libs,
headers or, if requested for every ABI, also binaries). If those are found, the image dir is moved
away and a new run is started, where again at start abic-specific vars are set and then the complete
src* phases are run. Once all requested ABIs are done, the image dirs are merged into the final
image dir. The following pkg_* phases are each running for every ABI.
Currently, only different libs and headers are installed by default, binaries will be the ones from
the default ABI, unless you tell portage to install binaries for all requested ABIs, in which case a
wrapper will select the ABI-specific binary depending on the environment.
The current implementation uses a USE-dep like way internally to satisfy the needed dependencies, so
that e.g. 32bit libs on a 64bit platform get their required dependencies built with 32bit libs
installed. For the rare case, where the crosscompile does fail and there is only a need for the
binary and no linking against the libs, i have also a var, which disables this auto-dependency
calculation for specified packages.

For the user interface, portage shows a USE_EXPANDed var, which contains the avaidable ABIs, as an
example for "emerge -pv media-libs/jpeg":

[ebuild R ] media-libs/jpeg-8b USE="-static-libs" MULTILIB_ABI="amd64 x86"

Those ABIs can be handled like USE flags, in this case, they are "multilib_abi_amd64" and
"multilib_abi_x86", so you can use those USE flags to enable/disable specific possible ABIs either
globally or per package.

The basic implementation can be used without changing main tree ebuilds or eclasses, but e.g. for
the replacement of emul-* libs, this will require EAPI-support for ABI-specific USE-deps for
binary-only packages or packages like wine.

I would first like to see, if there are any bigger concerns especially with the implementation and
how it is supposed to work.

If there are no such concerns or if they have been resolved, i would like to request some help for
the documentation and PMS-patch related work.

For install instructions, please have a look at [1], the code can be found in the multilib branch of
portage git repo at [2].

While i did not yet get to the implementation of it, i would also like to propose something similiar
for languages (like python, ruby or others), so that the basic parts are inside the PM and we can
drop the different ways of implementation and allow users a much more fine-grained control on a per
package base (in relation to the current way python eclass based very complex implementation works).

[1]: http://github.com/sjnewbury/multilib-overlay/tree/portage-multilib/doc
[2]: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=shortlog;h=refs/heads/multilib

--
Thomas Sachau

Gentoo Linux Developer
 
Old 12-01-2010, 06:05 PM
Fabian Groffen
 
Default MULTI_ABI support addition to main tree portage

Hi Thomas,

On 01-12-2010 19:57:46 +0100, Thomas Sachau wrote:
> The current implementation uses a USE-dep like way internally to
> satisfy the needed dependencies, so that e.g. 32bit libs on a 64bit
> platform get their required dependencies built with 32bit libs
> installed.

This just means that if I install package X, which depends on Y and
MULTILIB_ABI="amd64 x86", Portage will complain (or reinstall) Y if it
doesn't have both amd64 and x86, right?

> While i did not yet get to the implementation of it, i would also like
> to propose something similiar for languages (like python, ruby or
> others), so that the basic parts are inside the PM and we can drop the
> different ways of implementation and allow users a much more
> fine-grained control on a per package base (in relation to the current
> way python eclass based very complex implementation works).

This would be absolutely great!


--
Fabian Groffen
Gentoo on a different level
 
Old 12-01-2010, 06:11 PM
Alexey Shvetsov
 
Default MULTI_ABI support addition to main tree portage

Well =)

This will be killer feature in gentoo =P
Also what about more complex arhes than ia32? like mips of ppc?

PS also with this feature seems amd64 and x86 can be merged in one
arch (like it was done in kernel) since its only abis of ia32

2010/12/1 Thomas Sachau <tommy@gentoo.org>:
> Hi,
>
> i have already written about this some months ago and updated the code in relation to the comments
> especially from vapier.
>
> Basicly, it does now first set abi-specific vars (like CC, CFLAGS and others (setup_abi_env function
> in bin/auto-multilib.sh contains the full list), then does build the package as usual. If additional
> ABIs are requested, it checks after src_install, if there are possible ABI-specific files (libs,
> headers or, if requested for every ABI, also binaries). If those are found, the image dir is moved
> away and a new run is started, where again at start abic-specific vars are set and then the complete
> src* phases are run. Once all requested ABIs are done, the image dirs are merged into the final
> image dir. The following pkg_* phases are each running for every ABI.
> Currently, only different libs and headers are installed by default, binaries will be the ones from
> the default ABI, unless you tell portage to install binaries for all requested ABIs, in which case a
> wrapper will select the ABI-specific binary depending on the environment.
> The current implementation uses a USE-dep like way internally to satisfy the needed dependencies, so
> that e.g. 32bit libs on a 64bit platform get their required dependencies built with 32bit libs
> installed. For the rare case, where the crosscompile does fail and there is only a need for the
> binary and no linking against the libs, i have also a var, which disables this auto-dependency
> calculation for specified packages.
>
> For the user interface, portage shows a USE_EXPANDed var, which contains the avaidable ABIs, as an
> example for "emerge -pv media-libs/jpeg":
>
> [ebuild * R * ] media-libs/jpeg-8b *USE="-static-libs" MULTILIB_ABI="amd64 x86"
>
> Those ABIs can be handled like USE flags, in this case, they are "multilib_abi_amd64" and
> "multilib_abi_x86", so you can use those USE flags to enable/disable specific possible ABIs either
> globally or per package.
>
> The basic implementation can be used without changing main tree ebuilds or eclasses, but e.g. for
> the replacement of emul-* libs, this will require EAPI-support for ABI-specific USE-deps for
> binary-only packages or packages like wine.
>
> I would first like to see, if there are any bigger concerns especially with the implementation and
> how it is supposed to work.
>
> If there are no such concerns or if they have been resolved, i would like to request some help for
> the documentation and PMS-patch related work.
>
> For install instructions, please have a look at [1], the code can be found in the multilib branch of
> portage git repo at [2].
>
> While i did not yet get to the implementation of it, i would also like to propose something similiar
> for languages (like python, ruby or others), so that the basic parts are inside the PM and we can
> drop the different ways of implementation and allow users a much more fine-grained control on a per
> package base (in relation to the current way python eclass based very complex implementation works).
>
> [1]: http://github.com/sjnewbury/multilib-overlay/tree/portage-multilib/doc
> [2]: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=shortlog;h=refs/heads/multilib
>
> --
> Thomas Sachau
>
> Gentoo Linux Developer
>
>



--
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, Russia
Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexxyum@gmail.com
mailto:alexxy@gentoo.org
mailto:alexxy@omrb.pnpi.spb.ru
 
Old 12-01-2010, 07:19 PM
Thomas Sachau
 
Default MULTI_ABI support addition to main tree portage

Am 01.12.2010 20:05, schrieb Fabian Groffen:
> Hi Thomas,
>
> On 01-12-2010 19:57:46 +0100, Thomas Sachau wrote:
>> The current implementation uses a USE-dep like way internally to
>> satisfy the needed dependencies, so that e.g. 32bit libs on a 64bit
>> platform get their required dependencies built with 32bit libs
>> installed.
>
> This just means that if I install package X, which depends on Y and
> MULTILIB_ABI="amd64 x86", Portage will complain (or reinstall) Y if it
> doesn't have both amd64 and x86, right?

Right.

--
Thomas Sachau

Gentoo Linux Developer
 
Old 12-01-2010, 07:27 PM
Thomas Sachau
 
Default MULTI_ABI support addition to main tree portage

Am 01.12.2010 20:11, schrieb Alexey Shvetsov:
> Well =)
>
> This will be killer feature in gentoo =P
> Also what about more complex arhes than ia32? like mips of ppc?

I cant speak in detail about the other arches, since i dont know them that much. Basicly, if you can
crosscompile for a different arch/abi (so either the target arch/abi binaries run on your arch or
none are executed/needed for that), then this should be possible for them too. The code itself is
not limiting it (unless i missed some location), so you should be able to test this with an adjusted
custom or additionally sourced profile.

--
Thomas Sachau

Gentoo Linux Developer
 
Old 12-01-2010, 07:30 PM
Diego Elio Pettenò
 
Default MULTI_ABI support addition to main tree portage

Il giorno mer, 01/12/2010 alle 22.11 +0300, Alexey Shvetsov ha scritto:
>
> PS also with this feature seems amd64 and x86 can be merged in one
> arch (like it was done in kernel) since its only abis of ia32
>
I would suggest against that.

For the kernel it's somewhat easier, but for userland, x86 and amd64 are
definitely far enough that I wouldn't be surprised if it'll take a few
more years before we can easily consider the two keywords a single one.

Just think of a relatively-common situation.

void *bar = foo();

with foo implicitly declared. On 32-bit userland it'll be "all fine",
but will crash badly on 64-bit userland.

And this is without adding to the necessity of PIC, and the rest of
little details that this brings with it.

For the sake of safety, let's _not_ merge this, as we have said too many
times for me to dig up.


And finally, let's not call it ia32. No matter what Intel wants it to be
called, if you were to call it like that, you'd just have a number of
people asking why their ia64 stage don't work.

--
Diego Elio Pettenò — “Flameeyes”
http://blog.flameeyes.eu/

If you found a .asc file in this mail and know not what it is,
it's a GnuPG digital signature: http://www.gnupg.org/
 
Old 12-01-2010, 08:44 PM
Alexey Shvetsov
 
Default MULTI_ABI support addition to main tree portage

Well ok =)

i call it ia32 since its original name of this arch however it can be
better called x86 (x86_32 and x86_64)

PS seems many users were confused with ia64 since they associate it
with core2 and nahalem

2010/12/1 Diego Elio Pettenò <flameeyes@gmail.com>:
> Il giorno mer, 01/12/2010 alle 22.11 +0300, Alexey Shvetsov ha scritto:
>>
>> PS also with this feature seems amd64 and x86 can be merged in one
>> arch (like it was done in kernel) since its only abis of ia32
>>
> I would suggest against that.
>
> For the kernel it's somewhat easier, but for userland, x86 and amd64 are
> definitely far enough that I wouldn't be surprised if it'll take a few
> more years before we can easily consider the two keywords a single one.
>
> Just think of a relatively-common situation.
>
> void *bar = foo();
>
> with foo implicitly declared. On 32-bit userland it'll be "all fine",
> but will crash badly on 64-bit userland.
>
> And this is without adding to the necessity of PIC, and the rest of
> little details that this brings with it.
>
> For the sake of safety, let's _not_ merge this, as we have said too many
> times for me to dig up.
>
>
> And finally, let's not call it ia32. No matter what Intel wants it to be
> called, if you were to call it like that, you'd just have a number of
> people asking why their ia64 stage don't work.
>
> --
> Diego Elio Pettenò — “Flameeyes”
> http://blog.flameeyes.eu/
>
> If you found a .asc file in this mail and know not what it is,
> it's a GnuPG digital signature: http://www.gnupg.org/
>
>
>



--
Best Regards,
Alexey 'Alexxy' Shvetsov
Petersburg Nuclear Physics Institute, Russia
Department of Molecular and Radiation Biophysics
Gentoo Team Ru
Gentoo Linux Dev
mailto:alexxyum@gmail.com
mailto:alexxy@gentoo.org
mailto:alexxy@omrb.pnpi.spb.ru
 
Old 12-01-2010, 08:47 PM
Diego Elio Pettenò
 
Default MULTI_ABI support addition to main tree portage

Il giorno gio, 02/12/2010 alle 00.44 +0300, Alexey Shvetsov ha scritto:
>
> i call it ia32 since its original name of this arch however it can be
> better called x86 (x86_32 and x86_64)

It is by far not the original name of the architecture… it was retconned
after IA-64 was released. And it only covers x86, not x86-64/amd64
(which Intel called IA32e or EM64T iirc).

FWIW the kernel calls it i386.

> PS seems many users were confused with ia64 since they associate it
> with core2 and nahalem
>
Both of which have nothing to do with IA64. Given that users don't seem
to grasp the difference between architectures, why considering merging
two architectures that are, simply speaking, different (in userland,
that is)?

--
Diego Elio Pettenò — “Flameeyes”
http://blog.flameeyes.eu/

If you found a .asc file in this mail and know not what it is,
it's a GnuPG digital signature: http://www.gnupg.org/
 
Old 12-01-2010, 08:57 PM
Michał Górny
 
Default MULTI_ABI support addition to main tree portage

On Wed, 01 Dec 2010 22:47:45 +0100
Diego Elio Pettenò <flameeyes@gmail.com> wrote:

> x86-64/amd64 (which Intel called IA32e or EM64T iirc).

I think I've seen them calling it 'Intel 64' too which introduces even
more confusion.

--
Best regards,
Michał Górny
 
Old 12-02-2010, 11:53 AM
Pacho Ramos
 
Default MULTI_ABI support addition to main tree portage

El mi, 01-12-2010 a las 19:57 +0100, Thomas Sachau escribi:
> Hi,
>
> i have already written about this some months ago and updated the code in relation to the comments
> especially from vapier.
>
> Basicly, it does now first set abi-specific vars (like CC, CFLAGS and others (setup_abi_env function
> in bin/auto-multilib.sh contains the full list), then does build the package as usual. If additional
> ABIs are requested, it checks after src_install, if there are possible ABI-specific files (libs,
> headers or, if requested for every ABI, also binaries). If those are found, the image dir is moved
> away and a new run is started, where again at start abic-specific vars are set and then the complete
> src* phases are run. Once all requested ABIs are done, the image dirs are merged into the final
> image dir. The following pkg_* phases are each running for every ABI.
> Currently, only different libs and headers are installed by default, binaries will be the ones from
> the default ABI, unless you tell portage to install binaries for all requested ABIs, in which case a
> wrapper will select the ABI-specific binary depending on the environment.
> The current implementation uses a USE-dep like way internally to satisfy the needed dependencies, so
> that e.g. 32bit libs on a 64bit platform get their required dependencies built with 32bit libs
> installed. For the rare case, where the crosscompile does fail and there is only a need for the
> binary and no linking against the libs, i have also a var, which disables this auto-dependency
> calculation for specified packages.
>
> For the user interface, portage shows a USE_EXPANDed var, which contains the avaidable ABIs, as an
> example for "emerge -pv media-libs/jpeg":
>
> [ebuild R ] media-libs/jpeg-8b USE="-static-libs" MULTILIB_ABI="amd64 x86"
>
> Those ABIs can be handled like USE flags, in this case, they are "multilib_abi_amd64" and
> "multilib_abi_x86", so you can use those USE flags to enable/disable specific possible ABIs either
> globally or per package.
>
> The basic implementation can be used without changing main tree ebuilds or eclasses, but e.g. for
> the replacement of emul-* libs, this will require EAPI-support for ABI-specific USE-deps for
> binary-only packages or packages like wine.
>
> I would first like to see, if there are any bigger concerns especially with the implementation and
> how it is supposed to work.
>
> If there are no such concerns or if they have been resolved, i would like to request some help for
> the documentation and PMS-patch related work.
>
> For install instructions, please have a look at [1], the code can be found in the multilib branch of
> portage git repo at [2].
>
> While i did not yet get to the implementation of it, i would also like to propose something similiar
> for languages (like python, ruby or others), so that the basic parts are inside the PM and we can
> drop the different ways of implementation and allow users a much more fine-grained control on a per
> package base (in relation to the current way python eclass based very complex implementation works).
>
> [1]: http://github.com/sjnewbury/multilib-overlay/tree/portage-multilib/doc
> [2]: http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=shortlog;h=refs/heads/multilib
>


Nice to see progress on this, it's a headache to have to make emul
packages grow forever when any application (usually wine) requires
it :-S, and this would allow much more flexibility

Thanks a lot :-D
 

Thread Tools




All times are GMT. The time now is 05:35 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org