sub-slots (for EAPI 5)
 

On Fri, 7 Sep 2012 20:17:17 +0200
Fabian Groffen <grobian@gentoo.org> wrote:
> Eh, no. Now it just always breaks when you perform a downgrade, and
> revdev-rebuild or @preserved-libs won't help you. I prefer that you
> give best practices how to use sub-slots to make Portage also able to
> do a recompile of bar when libfnord in the same SLOT gets downgraded.
> (Because minors are used for compatible changes -- additions -- to the
> ABI.)

Downgrades aren't covered by sub-slots, slots, regular dependencies,
libtool, or anything else.

--
Ciaran McCreesh

sub-slots (for EAPI 5)
 

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

On 07/09/12 02:17 PM, Fabian Groffen wrote:
>
> No, not a GLEP, per se. I'm trying to understand what sub-slot
> does and is. I think I'm starting to understand now. However, for
> this feature to be added to an EAPI, IMO it would be nice if there
> are resources that make it for most developers very clear how this
> feature should be used (and how not), and what kind of problems it
> can solve.
>
> I guess real-life examples, more extensively described than you
> did before, with exactly where it goes wrong, and how the situation
> is improved would help.
>

I agree; I expect devmanual.gentoo.org would need a nice big page (or
section in the SLOTs page) describing it, if not at the very least a
nice entry on wiki.g.o


>>>> [1] No advantage as sub-slots wouldn't relate to this,
>>>> UNLESS the masking would then remove -all- SLOT="0/5"
>>>> versions from the tree. In that case, the old SLOT="0/3"
>>>> provider would be the 'upgrade' path and so 'foo' and 'bar'
>>>> would be triggered for rebuild (since the lib they were
>>>> linked to would be disappearing during emerge -uDN)
>>>
>>> So your example SLOTs are wrong, and should have included the
>>> minor (always!?!) since downgrading a lib that goes back to an
>>> older minor means functions no longer exist, thus a consumer
>>> can potentially break.
>>
>> If those (missing) functions are necessary then the either the
>> full slot, or the particular minimum package version, of libfnord
>> would need to be specified in the consumer. This isn't any
>> different from how things work now.
>
> Eh, no. Now it just always breaks when you perform a downgrade,
> and revdev-rebuild or @preserved-libs won't help you. I prefer
> that you give best practices how to use sub-slots to make Portage
> also able to do a recompile of bar when libfnord in the same SLOT
> gets downgraded. (Because minors are used for compatible changes --
> additions -- to the ABI.) (And the recompile is preferably done
> against the headers of the downgraded version, but with the newer
> version's lib still around, such that if this is a vital binary
> such as Python, it will not break down -- however, this is most
> likely too much to ask.)
>

I guess maybe i'm not following your example. To spell it out better,
here's what I'm understanding:

bar-1.0 has (prior to slot-operators) an RDEPEND="app-cat/libfnord".
No version specified. As such, it'll build successfully against
either libfnord-1 or libfnord-2. Right now (as i understand it) a
downgrade from libfnord-2 to libfnord-1 would "break" bar-1.0 bot
revdep-rebuild would detect and fix it.

sub-slots + slot-operators would alleviate what I just described,
auto-rebuilding bar (at least that's the theory; i've had some issues
getting portage to downgrade things even with regular slots so some
work on the implementation might be needed with this, dunno)

*IF* on the other hand, you're referring to the case of libfnord-2.1
downgrading to libfnord-2 , then yes I can see how bar-1.0 would be
broken and revdep-rebuild wouldn't fix it. However, as far as I
understand it, proper LTVERSIONing should mean that bar wouldn't break
as anything which would break bar on a libfnord-2 downgrade shouldn't
be used by bar in libfnord-2.1 -- ie, the differences would be
internal to libfnord and not directly used by bar.

If a package is -not- properly LTVERSIONed though, sub-slot bumps
could help alleviate this issue at the ebuild level.


>> This is why, for the rebuild of bar-2.4 to be triggered on
>> upgrade to libfnord-3.5 the SLOT= var within libfnord-3.5.ebuild
>> would need to have something other than SLOT="0/5", ie,
>> SLOT="0/5.12"
>
> Yeah, but can I also avoid bar-2.4 being recompiled when I install
> libfnord-3.5? It's not necessary, because the 5-ABI of libfnord
> is supposed to be backwards compatible. (At least that's the
> idea.) Like mentioned before, I DO want bar-2.4 to be recompiled if
> I have to downgrade libfnord to a version before the one I had
> installed when I compiled bar-2.4, though.
>

Sure -- since bar-2.4 does support a codepath for <libfnord-3.5,
there's no reason to rebuild bar-2.4 to enforce it afaict.

As for the downgrade, bar-2.4 should link against libfnord.so.5.12
rather than libfnord.so.5 due to the LTVERSION-specific codepath I
think; and I also think that sub-slots couldn't be used to trigger the
rebuild on downgrade if there shouldn't be a rebuild on upgrade;
@preserve-libs would be the only way to handle this one.


>>> Do you allow sub-slot to depend on e.g. USE-flags in use?
>>> Suppose libfnord has a USE-flag cow that adds special cow
>>> interfaces to the ABI/API. Would SLOT="X/${PV}$(use cow &&
>>> echo -- -cow)" work? (I think SLOT is supposed to be
>>> metadata-static, but does the sub-slot part count to that?)
>>
>> No, afaik slots (including sub-slots) can't be dynamic. Plus we
>> already have comprehensive use deps solutions so why would we
>> need that?
>
> Because the ABI changes. But I guess you're right that we can
> safely ignore packages that screw it up badly enough here.

I think that the best way to resolve something like this would be more
comprehensive use dep usage -- ie, if bar uses the bits of libfnord
available through USE="cow" then bar is effectively dependant on those
bits and so a libfnord:=[cow?] dep (and IUSE="cow" addition if not
already there) might be appropriate.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iF4EAREIAAYFAlBKP2oACgkQ2ugaI38ACPA3lAD/blxSdo1onKom/rFESPbQVWU4
bXNDbxlE28YNWTjBipkBAIxacbdMUcp8t7drd+Ldh1LULp3tEP QwIhdFPtdylGi7
=0/rQ
-----END PGP SIGNATURE-----

sub-slots (for EAPI 5)
 

On 07-09-2012 19:21:57 +0100, Ciaran McCreesh wrote:
> On Fri, 7 Sep 2012 20:17:17 +0200
> Fabian Groffen <grobian@gentoo.org> wrote:
> > Eh, no. Now it just always breaks when you perform a downgrade, and
> > revdev-rebuild or @preserved-libs won't help you. I prefer that you
> > give best practices how to use sub-slots to make Portage also able to
> > do a recompile of bar when libfnord in the same SLOT gets downgraded.
> > (Because minors are used for compatible changes -- additions -- to the
> > ABI.)
>
> Downgrades aren't covered by sub-slots, slots, regular dependencies,
> libtool, or anything else.

It seems I mistakenly took slot-operator-deps and sub-slots as something
that can be mapped onto ABIs. Doing so, however has proven to be wrong.

It appears slot-operator-deps do have some resemblance with ABI here
(especially if :* would be written in PMS such that it only allows
upgrades, no downgrades), but sub-slots are completely unrelated.

I don't like the mixing of the two in a single var, at all. I think I'd
much more prefer Portage to understand ABIs and potentially versions, to
make it explicit why it is doing what.


--
Fabian Groffen
Gentoo on a different level

sub-slots (for EAPI 5)
 

On Fri, 7 Sep 2012 20:49:35 +0200
Fabian Groffen <grobian@gentoo.org> wrote:
> On 07-09-2012 19:21:57 +0100, Ciaran McCreesh wrote:
> > On Fri, 7 Sep 2012 20:17:17 +0200
> > Fabian Groffen <grobian@gentoo.org> wrote:
> > > Eh, no. Now it just always breaks when you perform a downgrade,
> > > and revdev-rebuild or @preserved-libs won't help you. I prefer
> > > that you give best practices how to use sub-slots to make Portage
> > > also able to do a recompile of bar when libfnord in the same SLOT
> > > gets downgraded. (Because minors are used for compatible changes
> > > -- additions -- to the ABI.)
> >
> > Downgrades aren't covered by sub-slots, slots, regular dependencies,
> > libtool, or anything else.
>
> It seems I mistakenly took slot-operator-deps and sub-slots as
> something that can be mapped onto ABIs. Doing so, however has proven
> to be wrong.

It's not entirely wrong. There's a reason we stopped using the word
"ABI", though: it's a meaningless term with a lot of misleading
connotations.

> It appears slot-operator-deps do have some resemblance with ABI here
> (especially if :* would be written in PMS such that it only allows
> upgrades, no downgrades), but sub-slots are completely unrelated.

Downgrades are a different, unrelated problem. If you're trying to
solve that, you'll need a different, orthogonal solution. Note, though,
that downgrade breakages are typically not covered by whatever you think
an ABI is.

--
Ciaran McCreesh

sub-slots (for EAPI 5)
 

On 07-09-2012 14:39:38 -0400, Ian Stakenvicius wrote:
> I guess maybe i'm not following your example. To spell it out better,
> here's what I'm understanding:
>
> bar-1.0 has (prior to slot-operators) an RDEPEND="app-cat/libfnord".
> No version specified. As such, it'll build successfully against
> either libfnord-1 or libfnord-2. Right now (as i understand it) a
> downgrade from libfnord-2 to libfnord-1 would "break" bar-1.0 bot
> revdep-rebuild would detect and fix it.

right or portage would preserve libfnord-2's libs

> sub-slots + slot-operators would alleviate what I just described,
> auto-rebuilding bar (at least that's the theory; i've had some issues
> getting portage to downgrade things even with regular slots so some
> work on the implementation might be needed with this, dunno)
>
> *IF* on the other hand, you're referring to the case of libfnord-2.1
> downgrading to libfnord-2 , then yes I can see how bar-1.0 would be
> broken and revdep-rebuild wouldn't fix it. However, as far as I
> understand it, proper LTVERSIONing should mean that bar wouldn't break
> as anything which would break bar on a libfnord-2 downgrade shouldn't
> be used by bar in libfnord-2.1 -- ie, the differences would be
> internal to libfnord and not directly used by bar.

Well, yes and no. The no applies to the case where bar works around a
missing function by e.g. implementing it itself, conditionally. A
package maintainer might not have been aware of this, hence not
expressed this in the dependencies (e.g. requiring the version that
contains the missing function).
Anyway, we should avoid downgrades of libraries, so no need to discuss
this any further. It doesn't break more than it does already.


--
Fabian Groffen
Gentoo on a different level

sub-slots (for EAPI 5)
 

On 09/07/2012 11:17 AM, Fabian Groffen wrote:
> No, not a GLEP, per se. I'm trying to understand what sub-slot does and
> is. I think I'm starting to understand now. However, for this feature
> to be added to an EAPI, IMO it would be nice if there are resources that
> make it for most developers very clear how this feature should be used
> (and how not), and what kind of problems it can solve.
>
> I guess real-life examples, more extensively described than you did
> before, with exactly where it goes wrong, and how the situation is
> improved would help.

Perhaps some of the greatest frustrations for Gentoo users stem from the
lack of support for automatic rebuild of packages when necessary.
Imagine how nice it would be if necessary rebuilds would automatically
occur when appropriate, so that you wouldn't experience build failures
that require you to manually intervene by running revdep-rebuild,
perl-cleaner, or something like that. And there are other kinds of
necessary rebuilds that don't trigger build failures, but lead to
runtime failures that are noticed much later (like xorg driver failures
after a major xorg-server update). Sub-slots can be used to solve the
bulk of problems like these that our users have had to deal with manually.

> Eh, no. Now it just always breaks when you perform a downgrade, and
> revdev-rebuild or @preserved-libs won't help you. I prefer that you
> give best practices how to use sub-slots to make Portage also able to do
> a recompile of bar when libfnord in the same SLOT gets downgraded.
> (Because minors are used for compatible changes -- additions -- to the
> ABI.)
> (And the recompile is preferably done against the headers of the
> downgraded version, but with the newer version's lib still around, such
> that if this is a vital binary such as Python, it will not break down --
> however, this is most likely too much to ask.)

It might be worthwhile to try come up with some way to handle minor
downgrades in some later EAPI, but it adds complexity. Meanwhile,
sub-slots are a relatively simple extension to slot-operator deps, and
they are poised to greatly improve user experience (via automatic
rebuilds) if they are included in EAPI 5.
--
Thanks,
Zac

sub-slots (for EAPI 5)
 

On 07-09-2012 19:55:53 +0100, Ciaran McCreesh wrote:
> > It appears slot-operator-deps do have some resemblance with ABI here
> > (especially if :* would be written in PMS such that it only allows
> > upgrades, no downgrades), but sub-slots are completely unrelated.
>
> Downgrades are a different, unrelated problem. If you're trying to
> solve that, you'll need a different, orthogonal solution. Note, though,
> that downgrade breakages are typically not covered by whatever you think
> an ABI is.

I don't really want to solve downgrades. I just want to fully
understand what slot-deps is supposed to do, and what not.
It seems I fail in doing so.


--
Fabian Groffen
Gentoo on a different level

sub-slots (for EAPI 5)
 

On 07-09-2012 12:03:16 -0700, Zac Medico wrote:
> On 09/07/2012 11:17 AM, Fabian Groffen wrote:
> > I guess real-life examples, more extensively described than you did
> > before, with exactly where it goes wrong, and how the situation is
> > improved would help.
>
> Perhaps some of the greatest frustrations for Gentoo users stem from the
> lack of support for automatic rebuild of packages when necessary.
> Imagine how nice it would be if necessary rebuilds would automatically
> occur when appropriate, so that you wouldn't experience build failures
> that require you to manually intervene by running revdep-rebuild,
> perl-cleaner, or something like that. And there are other kinds of
> necessary rebuilds that don't trigger build failures, but lead to
> runtime failures that are noticed much later (like xorg driver failures
> after a major xorg-server update). Sub-slots can be used to solve the
> bulk of problems like these that our users have had to deal with manually.

I like that! Kudos for making it work!

I just wonder what the heck that has to do with SLOT.
This discussion has been done before in this thread, and it somehow
settled.

> ... sub-slots are a relatively simple extension to slot-operator deps,
> and they are poised to greatly improve user experience (via automatic
> rebuilds) if they are included in EAPI 5.

And we want it. But is it a good idea to add some feature that feels
like just a hack?


--
Fabian Groffen
Gentoo on a different level

sub-slots (for EAPI 5)
 

On Fri, 7 Sep 2012 21:25:22 +0200
Fabian Groffen <grobian@gentoo.org> wrote:
> I like that! Kudos for making it work!
>
> I just wonder what the heck that has to do with SLOT.

The correct fix for "not needing to rebuild" stuff is to SLOT libraries
like crazy, and have a SLOT per thing-we-don't-call-ABI. This then
needs := dependencies, so that packages can say "and remember which
SLOT I was built against".

However, there are some packages that cannot easily be SLOTted to the
degree that we'd like. This is where sub-SLOTs come in. Given
sub-SLOTted packages dep:1/a and dep:1/b, this says "I'd like to have
slots 1a and 1b, but it's too difficult to allow 1a and 1b to be
installed at the same time".

So suppose the user has pkg with a dependency upon dep, with slot 1 and
a := operator. They install pkg when dep:1/a is installed. The user
then installs dep:1/b. In an ideal world, dep:1/a would remain
installed in parallel with dep:1/b, but your friendly Gentoo developers
have decided it's not worth their time to allow this. Thus, dep:1/a has
to be uninstalled to allow dep:1/b to be installed. But this would
break pkg, since pkg needs dep:1/a. However, a clever dependency
resolver can note that reinstalling pkg would fix it, since dep:1/b
also satisfies pkg's slot 1 := dependency (but not the locked 1/a
dependency that the installed version of pkg has picked up).

--
Ciaran McCreesh

sub-slots (for EAPI 5)
 

On 07-09-2012 20:36:02 +0100, Ciaran McCreesh wrote:
> The correct fix for "not needing to rebuild" stuff is to SLOT libraries
> like crazy, and have a SLOT per thing-we-don't-call-ABI. This then
> needs := dependencies, so that packages can say "and remember which
> SLOT I was built against".
>
> However, there are some packages that cannot easily be SLOTted to the
> degree that we'd like. This is where sub-SLOTs come in. Given
> sub-SLOTted packages dep:1/a and dep:1/b, this says "I'd like to have
> slots 1a and 1b, but it's too difficult to allow 1a and 1b to be
> installed at the same time".
>
> So suppose the user has pkg with a dependency upon dep, with slot 1 and
> a := operator. They install pkg when dep:1/a is installed. The user
> then installs dep:1/b. In an ideal world, dep:1/a would remain
> installed in parallel with dep:1/b, but your friendly Gentoo developers
> have decided it's not worth their time to allow this. Thus, dep:1/a has
> to be uninstalled to allow dep:1/b to be installed. But this would
> break pkg, since pkg needs dep:1/a. However, a clever dependency
> resolver can note that reinstalling pkg would fix it, since dep:1/b
> also satisfies pkg's slot 1 := dependency (but not the locked 1/a
> dependency that the installed version of pkg has picked up).

Thanks. It seems we're there. At last.


--
Fabian Groffen
Gentoo on a different level