FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 06-17-2012, 10:51 AM
Marien Zwart
 
Default Dynamic SLOTs

On zo, 2012-06-17 at 09:26 +0200, Michał Górny wrote:
> I'm attaching a reStructuredText version of the spec. You can view it
> rendered as a gist[1]. But please keep the replies on the list, rather
> than forking the gist.

I've only thought about this a little, but some thoughts so far:

A common operation in an ebuild will be (say) "get the selected version
of python". That's easy enough if there's only one kind of dynamic slot
being used by the ebuild (just use CURRENT_SLOTS directly), but if the
ebuild supports more than one kind of dynamic slot you need a helper
function that picks the selected value of that one kind of slot out of
CURRENT_SLOTS, either relying on the naming scheme or on a list of
supported values of that kind of slot hardcoded in the helper function
(in an eclass). That seems a little fragile. Your "dynamic slot groups"
idea could help with this, by having that list sit in a more sensible
place than in an eclass, and perhaps by having the package mangler
provide a variable per slot group (a little like how USE_EXPAND works).

I really dislike the implicit dependencies. First of all: it is entirely
possible for ebuild A supporting dynamic slots for (say) python to have
a build-time dependency on ebuild B that supports dynamic slots (also
for python), but only to get at a utility to run, not to use B as a
library. In that case we don't want to force slots to match, and (as you
point out) that may not even be possible if the two packages don't
support the same slots. Also, it just breaks down completely if you
don't use your "slot groups" idea: if A has DYNAMIC_SLOTS="|| ( py2.6
py2.7 )" and B has DYNAMIC_SLOTS="|| ( ruby1.8 ruby1.9 )" it makes no
sense to require the dynamic slot setting to match, but if A has
DYNAMIC_SLOTS="|| ( py2.5 py2.6 )" and B has "|| ( py2.7 py3.2 )" then
we do need them to match, and the user has requested an impossible
situation (he needs to use versions of A and B that have at least one
supported python version in common). Without "slot groups" the package
manager cannot tell these two cases apart.

I think it'd make more sense to have those slot groups, per-slot group
variables like DYNAMIC_SLOTS_PYTHON="py2.7 py3.2" resulting in
CURRENT_SLOT_PYTHON getting set, and an addition to the dependency
syntax that lets you depend on a matching named dynamic slot. That makes
your DYNAMIC_SLOTS="|| ( py2.6 py2.7 ruby1.8 ruby1.9 )" example
impossible, but I think it'd be uncommon for that approach to make
sense: I think it'd usually make more sense to split that into two
packages, one per language.


--
Marien Zwart
 
Old 06-17-2012, 12:09 PM
Thomas Sachau
 
Default Dynamic SLOTs

Michał Górny schrieb:
> Hello,
>
> I have prepared a first draft of 'dynamic SLOT' specification. This is
> my proposal in attempt to solve the problem of building packages for
> multiple Python and Ruby versions. It could also be reused for multilib.
>
> The spec tries to explain the broad idea, and all problems relevant to
> it. It also lists a few problems which are still unsolved and I think
> they will cause the spec to change after hearing your ideas.
>
> To be honest, I tried to keep it as simple as possible. Please don't
> say it doesn't solve all the world problems because it simply won't.
>
> I'm attaching a reStructuredText version of the spec. You can view it
> rendered as a gist[1]. But please keep the replies on the list, rather
> than forking the gist.
>
> [1]:https://gist.github.com/2943774
>

Since you have not responded to my lines in IRC, i will repeat them here:

First: How does the user see, which slots are possible and which ones
are currently active and which are currently not selected?

Beside that, it seems to solve things pretty similar to the proposed way
in multilib-portage for cross-compiling (which could also be adapted for
multi-slot languages) with different wording and with additional work
for ebuild maintainers. And since my proposal already uses USE flags,
things would not change visually for users of e.g. ruby or php.

--

Thomas Sachau
Gentoo Linux Developer
 
Old 06-17-2012, 01:21 PM
Ciaran McCreesh
 
Default Dynamic SLOTs

On Sun, 17 Jun 2012 09:26:55 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> I have prepared a first draft of 'dynamic SLOT' specification. This is
> my proposal in attempt to solve the problem of building packages for
> multiple Python and Ruby versions. It could also be reused for
> multilib.

I'm pretty sure we can't go anywhere with this until all the things
that manually access VDB are gone. Can you work on that first?

--
Ciaran McCreesh
 
Old 06-17-2012, 03:29 PM
Michał Górny
 
Default Dynamic SLOTs

On Sun, 17 Jun 2012 14:09:14 +0200
Thomas Sachau <tommy@gentoo.org> wrote:

> Michał Górny schrieb:
> > Hello,
> >
> > I have prepared a first draft of 'dynamic SLOT' specification. This
> > is my proposal in attempt to solve the problem of building packages
> > for multiple Python and Ruby versions. It could also be reused for
> > multilib.
> >
> > The spec tries to explain the broad idea, and all problems relevant
> > to it. It also lists a few problems which are still unsolved and I
> > think they will cause the spec to change after hearing your ideas.
> >
> > To be honest, I tried to keep it as simple as possible. Please don't
> > say it doesn't solve all the world problems because it simply won't.
> >
> > I'm attaching a reStructuredText version of the spec. You can view
> > it rendered as a gist[1]. But please keep the replies on the list,
> > rather than forking the gist.
> >
> > [1]:https://gist.github.com/2943774
> >
>
> Since you have not responded to my lines in IRC, i will repeat them
> here:
>
> First: How does the user see, which slots are possible and which ones
> are currently active and which are currently not selected?

Implementation is left to be package-manager specific. I guess colorful
output (similar to USE flags) would be enough.

> Beside that, it seems to solve things pretty similar to the proposed
> way in multilib-portage for cross-compiling (which could also be
> adapted for multi-slot languages) with different wording and with
> additional work for ebuild maintainers. And since my proposal already
> uses USE flags, things would not change visually for users of e.g.
> ruby or php.

I'm sad you aren't even trying to listen. Your attempt implies that
every single change in targets requires rebuilding all of them. If I
weren't using 32-bit libs, and now I want to compile 32-bit wine, I
have to recompile most of my libraries for both ABIs. That is
a no go for me.

And adjusting that for other multi-slot languages is pointless. Because
they do the same already.

--
Best regards,
Michał Górny
 
Old 06-17-2012, 03:39 PM
Michał Górny
 
Default Dynamic SLOTs

On Sun, 17 Jun 2012 11:43:30 +0200
Hans de Graaff <graaff@gentoo.org> wrote:

> On Sun, 2012-06-17 at 09:26 +0200, Michał Górny wrote:
>
> > I'm attaching a reStructuredText version of the spec. You can view
> > it rendered as a gist[1]. But please keep the replies on the list,
> > rather than forking the gist.
>
> I don't like the approach taken in 6. I'd rather state that there
> should not be file collisions between the dynamic slots. We already
> handle things this way in ruby (with a common 'all' and specific
> version builds).

That is impossible for most of the build systems. If Ruby has a nice
ability to split between building common and ABI-specific stuff, that's
great. But Python doesn't have one. Bindings built using other
languages don't have that either.

The usual way of handling that is through letting all the ABIs install
to the same image, and then installing whatever got merged as a result.
The spec practically suggests the same yet split in time.

Shortly saying, we can't do that or we will be stuck installing
everything by hand, which will practically make this idea useless.

> For 9c I can't see us limiting users to a single ruby implementation
> by default (the only current exception is www-apache/passenger), so a
> combined ||() block makes no sense to me. I think it is better to be
> explicit here and express the real situation with multiple ||() blocks
> if needed.

I think you misunderstand the concept of combined block. It was mostly
supposed to be useful when a single package provides bindings for
multiple languages, so that a single build would involve building
bindings for one ABI of one language.

Multiple blocks would involve building bindings for both one Python ABI
and one Ruby ABI at a time which means scary results. That probably
even won't work, so splitting it is the only alternative to one big
block.

> Finally, I don't expect ruby to use this unless we can ensure that
> this works with our current ebuilds without changes. I'm fine with
> supporting some code in the eclass to determine which mechanism to
> use in which way, but we won't be spending huge amounts of time
> switching to yet another system. To me the perceived benefits aren't
> big enough.

I'm trying hard to make this as painless as possible but I can't
guarantee single ebuilds (uncommon cases) wouldn't need changes.
However, I'm trying hard to ensure that majority would be clearly
handled by eclasses.

--
Best regards,
Michał Górny
 
Old 06-17-2012, 03:46 PM
Thomas Sachau
 
Default Dynamic SLOTs

Michał Górny schrieb:
> On Sun, 17 Jun 2012 14:09:14 +0200
> Thomas Sachau <tommy@gentoo.org> wrote:
>
>> Michał Górny schrieb:
>>> Hello,
>>>
>>> I have prepared a first draft of 'dynamic SLOT' specification. This
>>> is my proposal in attempt to solve the problem of building packages
>>> for multiple Python and Ruby versions. It could also be reused for
>>> multilib.
>>>
>>> The spec tries to explain the broad idea, and all problems relevant
>>> to it. It also lists a few problems which are still unsolved and I
>>> think they will cause the spec to change after hearing your ideas.
>>>
>>> To be honest, I tried to keep it as simple as possible. Please don't
>>> say it doesn't solve all the world problems because it simply won't.
>>>
>>> I'm attaching a reStructuredText version of the spec. You can view
>>> it rendered as a gist[1]. But please keep the replies on the list,
>>> rather than forking the gist.
>>>
>>> [1]:https://gist.github.com/2943774
>>>
>>
>> Since you have not responded to my lines in IRC, i will repeat them
>> here:
>>
>> First: How does the user see, which slots are possible and which ones
>> are currently active and which are currently not selected?
>
> Implementation is left to be package-manager specific. I guess colorful
> output (similar to USE flags) would be enough.

So you dont like my solution with USE flags and then suggest some USE
flag like output for your own solution? If you want to use something USE
flag like, then simply use USE flags, they already exist. ;-)

>
>> Beside that, it seems to solve things pretty similar to the proposed
>> way in multilib-portage for cross-compiling (which could also be
>> adapted for multi-slot languages) with different wording and with
>> additional work for ebuild maintainers. And since my proposal already
>> uses USE flags, things would not change visually for users of e.g.
>> ruby or php.
>
> I'm sad you aren't even trying to listen. Your attempt implies that
> every single change in targets requires rebuilding all of them. If I
> weren't using 32-bit libs, and now I want to compile 32-bit wine, I
> have to recompile most of my libraries for both ABIs. That is
> a no go for me.

So you want to build a 32bit package, which is depending on 32bit libs,
but want to do that without the needed dependencies? Please tell me, how
that works.

>
> And adjusting that for other multi-slot languages is pointless. Because
> they do the same already.
>

So you dont like one framework for all multi-slot languages and prefer
having each one using their own solution? Or do you just dislike my idea
for them and want to use your own suggestion for them?

--

Thomas Sachau
Gentoo Linux Developer
 
Old 06-17-2012, 03:46 PM
Michał Górny
 
Default Dynamic SLOTs

On Sun, 17 Jun 2012 12:51:50 +0200
Marien Zwart <marienz@gentoo.org> wrote:

> A common operation in an ebuild will be (say) "get the selected
> version of python". That's easy enough if there's only one kind of
> dynamic slot being used by the ebuild (just use CURRENT_SLOTS
> directly), but if the ebuild supports more than one kind of dynamic
> slot you need a helper function that picks the selected value of that
> one kind of slot out of CURRENT_SLOTS, either relying on the naming
> scheme or on a list of supported values of that kind of slot
> hardcoded in the helper function (in an eclass). That seems a little
> fragile. Your "dynamic slot groups" idea could help with this, by
> having that list sit in a more sensible place than in an eclass, and
> perhaps by having the package mangler provide a variable per slot
> group (a little like how USE_EXPAND works).

Yes, you are right.

> I really dislike the implicit dependencies. First of all: it is
> entirely possible for ebuild A supporting dynamic slots for (say)
> python to have a build-time dependency on ebuild B that supports
> dynamic slots (also for python), but only to get at a utility to run,
> not to use B as a library. In that case we don't want to force slots
> to match, and (as you point out) that may not even be possible if the
> two packages don't support the same slots.

Hmm, that is true. However, it all depends on how the utility would be
called. As I see it, there are two possibilities here:

a) utility is called using the currently selected Python version,
b) utility is called using currently built Python version.

In case (b) the complete dependency graph for that version of Python is
probably required anyway. In case (a) indeed that may even cause
the necessary version of the utility to be missing.

First thought on handling this is to invent additional dependency
symbol which would disable dynamic SLOTs for that particular dependency
(opt-out seems to be simpler to handle globally than opt-in).

> Also, it just breaks down
> completely if you don't use your "slot groups" idea: if A has
> DYNAMIC_SLOTS="|| ( py2.6 py2.7 )" and B has DYNAMIC_SLOTS="||
> ( ruby1.8 ruby1.9 )" it makes no sense to require the dynamic slot
> setting to match, but if A has DYNAMIC_SLOTS="|| ( py2.5 py2.6 )" and
> B has "|| ( py2.7 py3.2 )" then we do need them to match, and the
> user has requested an impossible situation (he needs to use versions
> of A and B that have at least one supported python version in
> common). Without "slot groups" the package manager cannot tell these
> two cases apart.
>
> I think it'd make more sense to have those slot groups, per-slot group
> variables like DYNAMIC_SLOTS_PYTHON="py2.7 py3.2" resulting in
> CURRENT_SLOT_PYTHON getting set, and an addition to the dependency
> syntax that lets you depend on a matching named dynamic slot. That
> makes your DYNAMIC_SLOTS="|| ( py2.6 py2.7 ruby1.8 ruby1.9 )" example
> impossible, but I think it'd be uncommon for that approach to make
> sense: I think it'd usually make more sense to split that into two
> packages, one per language.

Agreed. The next version of the spec (if the goal seemed still possible
to achieve) will certainly have those.

--
Best regards,
Michał Górny
 
Old 06-17-2012, 03:53 PM
Michał Górny
 
Default Dynamic SLOTs

On Sun, 17 Jun 2012 17:46:00 +0200
Thomas Sachau <tommy@gentoo.org> wrote:

> >> Beside that, it seems to solve things pretty similar to the
> >> proposed way in multilib-portage for cross-compiling (which could
> >> also be adapted for multi-slot languages) with different wording
> >> and with additional work for ebuild maintainers. And since my
> >> proposal already uses USE flags, things would not change visually
> >> for users of e.g. ruby or php.
> >
> > I'm sad you aren't even trying to listen. Your attempt implies that
> > every single change in targets requires rebuilding all of them. If I
> > weren't using 32-bit libs, and now I want to compile 32-bit wine, I
> > have to recompile most of my libraries for both ABIs. That is
> > a no go for me.
>
> So you want to build a 32bit package, which is depending on 32bit
> libs, but want to do that without the needed dependencies? Please
> tell me, how that works.

I'm trying to build a 32bit package and its 32bit dependencies. Your
solution involves building a 32bit package and rebuilding all 64bit
packages which happen to be its dependencies for no reason.

> > And adjusting that for other multi-slot languages is pointless.
> > Because they do the same already.
>
> So you dont like one framework for all multi-slot languages and prefer
> having each one using their own solution? Or do you just dislike my
> idea for them and want to use your own suggestion for them?

I'm just saying that your framework doesn't change anything for user;
it just involves some work to change the label.

--
Best regards,
Michał Górny
 
Old 06-17-2012, 04:27 PM
Thomas Sachau
 
Default Dynamic SLOTs

Michał Górny schrieb:
> On Sun, 17 Jun 2012 17:46:00 +0200
> Thomas Sachau <tommy@gentoo.org> wrote:
>
>>>> Beside that, it seems to solve things pretty similar to the
>>>> proposed way in multilib-portage for cross-compiling (which could
>>>> also be adapted for multi-slot languages) with different wording
>>>> and with additional work for ebuild maintainers. And since my
>>>> proposal already uses USE flags, things would not change visually
>>>> for users of e.g. ruby or php.
>>>
>>> I'm sad you aren't even trying to listen. Your attempt implies that
>>> every single change in targets requires rebuilding all of them. If I
>>> weren't using 32-bit libs, and now I want to compile 32-bit wine, I
>>> have to recompile most of my libraries for both ABIs. That is
>>> a no go for me.
>>
>> So you want to build a 32bit package, which is depending on 32bit
>> libs, but want to do that without the needed dependencies? Please
>> tell me, how that works.
>
> I'm trying to build a 32bit package and its 32bit dependencies. Your
> solution involves building a 32bit package and rebuilding all 64bit
> packages which happen to be its dependencies for no reason.

You should already know, that "for no reason" is plain wrong. Beside the
point, that you can build just the 32bit libs, if you dont want 64bit
ones. And you did not answer my questions when i asked you about
maintainence of your suggestion back then to keep every target as a
seperate package (which itself has its own disadvantages like common
files and UI questions).

Either way, my working solution might have some overhead, but is easy to
control and maintain for both devs and users.
If you get to the point, that you can show me your suggestion working
with the main tree, it might be much more clear, what you actually want
to do and how you want to implement it.

--

Thomas Sachau
Gentoo Linux Developer
 
Old 06-18-2012, 08:34 AM
Michał Górny
 
Default Dynamic SLOTs

On Sun, 17 Jun 2012 09:26:55 +0200
Michał Górny <mgorny@gentoo.org> wrote:

> I'm attaching a reStructuredText version of the spec. You can view it
> rendered as a gist[1]. But please keep the replies on the list, rather
> than forking the gist.
>
> [1]:https://gist.github.com/2943774

Updated version. I've introduced dynamic SLOT groups and updated all
relevant information.

I didn't break DYNAMIC_SLOTS in ebuild to multiple variables yet but it
will be easy to change that if necessary.

This fixes problem previously described as 9a (dependencies). However,
I've added a new issue, 10c which marienz pointed out.

--
Best regards,
Michał Górny
 

Thread Tools




All times are GMT. The time now is 11:11 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org