EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, Sep 2, 2012 at 5:23 PM, Michał Górny <mgorny@gentoo.org> wrote:
> On Sun, 2 Sep 2012 17:09:22 +0200
> Fabio Erculiani <lxnay@gentoo.org> wrote:
>
>> On Sun, Sep 2, 2012 at 4:57 PM, hasufell <hasufell@gentoo.org> wrote:
>> >
>> >
>> > Why not introduce a global useflag such as "suggested-deps" which
>> > complies with GLEP 62 meaning it will be in IUSE_RUNTIME.
>> >
>>
>> How do you manage to fix the PDEPEND "identity disorder" problem then?
>> Teaching devs to move to GLEP 62 is much harder than just telling them
>> to move dep strings to more appropriate locations.
>
> Much harder? So, devs today don't know how USE flags work? Or do you
> implying that devs should know bare technical details of package
> manager implementation? Or is it just an-ass argument to support
> an ass-thesis?

For instance, the amount of devs still improperly using pkg_setup for
build-time stuff, after years that they've been told to not do that,
is a good example of how, while we're great, we tend to be resilient
to change. This goes beyond software engineering, this is about
psychology. The more one thing is simple, the faster is recognized and
properly used by people.

Not to mention that I am one of the PMS writers here (Entropy cough),
and I know how much harder implementing runtime-switchable USE flags
is, compared to a simple SDEPEND solution.

>
>> Moreover, your solution just makes the USE flags abuse situation
>> worse: there are packages that use USE flags just to include extra,
>> optional packages in the dependencies... See USE=bluetooth in
>> net-misc/networkmanager for example (this is what I mean with USE
>> flags abuse, actually).
>
> No, it fixes it. It enables those packages to use the same solution,
> fixing its downsides.

It looks like it just tries to workaround the issue, not fix it to its
roots (PDEPEND is ill!).

>
>> I'm not saying that SDEPEND is the best one-size-fits-all solution but
>> you may agree that's the simplest and most effective one.
>
> pkg_postinst() is simpler. USE flags are simple and very effective, yet
> incorrect.

Could you tell me how would you plan to implement so API to read
"suggested" dependencies from pkg_postinst() output?
I understand that being API-friendly is not really the best feature of
Portage and ebuilds in general (one reason why our PackageKit backend
is a bit sucky) but this is not a good reason to keep doing things
like they've been always done.
(Information printed at pkg_postinst() is another interesting problem,
wrt API consumers, and this includes PackageKit, but I don't want to
drift away too much from the topic).

>
> An effective SDEPEND implementation is definitely nowhere close
> to simple. Nor is presenting those dependencies to users.

As I mentioned above, I know what it simple and what not, from a
Software Engineering point of view. And SDEPENDs are much simpler than
GLEP 62, and GLEP 62 does not fix the PDEPEND issue because
individuals will keep using it because at the end of the day, it is
order of magniture simpler than obscure new variables and USE flags.
As a rule of thumb, "simple" always shines over the complex and
convoluted in the long run.

>
> --
> Best regards,
> Michał Górny

Peace & love!

--
Fabio Erculiani

EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, 2 Sep 2012 17:51:00 +0200
Fabio Erculiani <lxnay@gentoo.org> wrote:

> On Sun, Sep 2, 2012 at 5:23 PM, Michał Górny <mgorny@gentoo.org>
> wrote:
> > On Sun, 2 Sep 2012 17:09:22 +0200
> > Fabio Erculiani <lxnay@gentoo.org> wrote:
> >
> >> On Sun, Sep 2, 2012 at 4:57 PM, hasufell <hasufell@gentoo.org>
> >> wrote:
> >> >
> >> >
> >> > Why not introduce a global useflag such as "suggested-deps" which
> >> > complies with GLEP 62 meaning it will be in IUSE_RUNTIME.
> >> >
> >>
> >> How do you manage to fix the PDEPEND "identity disorder" problem
> >> then? Teaching devs to move to GLEP 62 is much harder than just
> >> telling them to move dep strings to more appropriate locations.
> >
> > Much harder? So, devs today don't know how USE flags work? Or do you
> > implying that devs should know bare technical details of package
> > manager implementation? Or is it just an-ass argument to support
> > an ass-thesis?
>
> For instance, the amount of devs still improperly using pkg_setup for
> build-time stuff, after years that they've been told to not do that,
> is a good example of how, while we're great, we tend to be resilient
> to change. This goes beyond software engineering, this is about
> psychology. The more one thing is simple, the faster is recognized and
> properly used by people.
>
> Not to mention that I am one of the PMS writers here (Entropy cough),
> and I know how much harder implementing runtime-switchable USE flags
> is, compared to a simple SDEPEND solution.

Yes, I'm aware of that. You're trying to force a worse solution because
it is easier for you to implement in your partial package manager.

> >> Moreover, your solution just makes the USE flags abuse situation
> >> worse: there are packages that use USE flags just to include extra,
> >> optional packages in the dependencies... See USE=bluetooth in
> >> net-misc/networkmanager for example (this is what I mean with USE
> >> flags abuse, actually).
> >
> > No, it fixes it. It enables those packages to use the same solution,
> > fixing its downsides.
>
> It looks like it just tries to workaround the issue, not fix it to its
> roots (PDEPEND is ill!).

It is invalid to use those packages in PDEPEND; even Ciaran will tell
you that.

> >> I'm not saying that SDEPEND is the best one-size-fits-all solution
> >> but you may agree that's the simplest and most effective one.
> >
> > pkg_postinst() is simpler. USE flags are simple and very effective,
> > yet incorrect.
>
> Could you tell me how would you plan to implement so API to read
> "suggested" dependencies from pkg_postinst() output?
> I understand that being API-friendly is not really the best feature of
> Portage and ebuilds in general (one reason why our PackageKit backend
> is a bit sucky) but this is not a good reason to keep doing things
> like they've been always done.
> (Information printed at pkg_postinst() is another interesting problem,
> wrt API consumers, and this includes PackageKit, but I don't want to
> drift away too much from the topic).
>
> >
> > An effective SDEPEND implementation is definitely nowhere close
> > to simple. Nor is presenting those dependencies to users.
>
> As I mentioned above, I know what it simple and what not, from a
> Software Engineering point of view. And SDEPENDs are much simpler than
> GLEP 62, and GLEP 62 does not fix the PDEPEND issue because
> individuals will keep using it because at the end of the day, it is
> order of magniture simpler than obscure new variables and USE flags.
> As a rule of thumb, "simple" always shines over the complex and
> convoluted in the long run.

As long as it's complete. A simple incomplete solution is yet another
hack which will have to be worked-around in the future again. And
you'll end up with 5 different solutions which will have to somehow
work together to solve a single problem (see: exheres).

--
Best regards,
Michał Górny

EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, 2 Sep 2012 17:23:40 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> An effective SDEPEND implementation is definitely nowhere close
> to simple. Nor is presenting those dependencies to users.

Indeed it's not, but we *do* have a reference implementation and lots
of practical experience with it, and we have worked out all the
difficulties.

Having said that, if we're going with suggested dependencies for EAPI 5
(which I strongly suspect won't happen, since we seem to be wanting
EAPI 5 now rather than in several years time...) then there's a lot
more to getting it right than is mentioned in the original post, and it
needs to be written up properly.

--
Ciaran McCreesh

EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, Sep 2, 2012 at 8:08 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> and we have worked out all the difficulties.

Please elaborate. What difficulties? What did you implement other than
plain SDEPEND? With what features? Lots of detail missing.

>
> Having said that, if we're going with suggested dependencies for EAPI 5
> (which I strongly suspect won't happen, since we seem to be wanting
> EAPI 5 now rather than in several years time...) then there's a lot
> more to getting it right than is mentioned in the original post, and it
> needs to be written up properly.

Well, this depends on the quality of the PMS architecture, I am not
familiar with Paludis tho.
I don't remember to have listed anything about what needs to be done
at the implementation level actually, nor I really wanted to.
I always use the 5 minutes "rule of thumb" strategy to understand the
complexity of proposals: if it takes me more than 5 minutes to
understand it, then users (!= devs) will have to go through the same
or more "wtf-period". And the probability of them "giving up / getting
sick / ignoring it" is linear with the wtf-period.

>
> --
> Ciaran McCreesh



--
Fabio Erculiani

EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, 2 Sep 2012 20:46:19 +0200
Fabio Erculiani <lxnay@gentoo.org> wrote:
> On Sun, Sep 2, 2012 at 8:08 PM, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
> > and we have worked out all the difficulties.
>
> Please elaborate. What difficulties? What did you implement other than
> plain SDEPEND? With what features? Lots of detail missing.

The big issues you're ignoring:

* What to do if a package has an SDEPEND upon cat/pkg[x] and the user
has cat/pkg[-x] installed, or if another package in the resolution
depends upon cat/pkg and the user hasn't specified USE=x. Similarly,
an SDEPEND upon >=cat/pkg-2.1 and the user has cat/pkg-2.0 (which is
in the same slot as 2.1) installed. The right answer is to force a
reinstall with [x] / force the upgrade, and for the spec to
explicitly require this from implementations, but *why* this is the
case is fairly subtle.

* How to handle groups of dependencies in suggestions.

* How to display suggestions to the user, and allow the user to
confirm or reject suggestions. From a spec perspective, we don't
mandate any particular behaviour, but we do need to make sure that a
good quality implementation is possible. With SDEPEND as you're
proposing, we're going to have exactly the same issues we currently
have with REQUIRED_USE and blockers: ebuilds won't provide enough
information to provide a good user interface.

* What use? blocks in SDEPEND actually mean. Again, there's a right
answer here: it's for when a particular suggestion requires the base
package to be built in a particular way.

> > Having said that, if we're going with suggested dependencies for
> > EAPI 5 (which I strongly suspect won't happen, since we seem to be
> > wanting EAPI 5 now rather than in several years time...) then
> > there's a lot more to getting it right than is mentioned in the
> > original post, and it needs to be written up properly.
>
> Well, this depends on the quality of the PMS architecture, I am not
> familiar with Paludis tho.

Paludis already has it. The question is whether it can be implemented
for Portage.

> I don't remember to have listed anything about what needs to be done
> at the implementation level actually, nor I really wanted to.

That's a problem. You can't just magically describe a vague feature and
expect it to be implementable.

> I always use the 5 minutes "rule of thumb" strategy to understand the
> complexity of proposals: if it takes me more than 5 minutes to
> understand it, then users (!= devs) will have to go through the same
> or more "wtf-period". And the probability of them "giving up / getting
> sick / ignoring it" is linear with the wtf-period.

There's a big difference between understanding things for developers,
and getting it right for implementation. Where possible, users and
developers should only need a shallow understanding of a feature to be
able to use it, but for the purposes of the spec and the
implementation, we need a proper deep understanding of the topic, and
you aren't giving that.

--
Ciaran McCreesh

EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, 2 Sep 2012 19:08:33 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sun, 2 Sep 2012 17:23:40 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > An effective SDEPEND implementation is definitely nowhere close
> > to simple. Nor is presenting those dependencies to users.
>
> Indeed it's not, but we *do* have a reference implementation and lots
> of practical experience with it, and we have worked out all the
> difficulties.

Yes, I am pretty sure you tested it with *many different* packages.
Maybe even all packages you have in Exherbo. Could you give some actual
numbers, and details?

> Having said that, if we're going with suggested dependencies for EAPI
> 5 (which I strongly suspect won't happen, since we seem to be wanting
> EAPI 5 now rather than in several years time...) then there's a lot
> more to getting it right than is mentioned in the original post, and
> it needs to be written up properly.

Good to hear that. I always appreciate constructive critics with
an explanation what needs to be fixed.

--
Best regards,
Michał Górny

EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, 2 Sep 2012 21:07:30 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> On Sun, 2 Sep 2012 19:08:33 +0100
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > On Sun, 2 Sep 2012 17:23:40 +0200
> > Michał Górny <mgorny@gentoo.org> wrote:
> > > An effective SDEPEND implementation is definitely nowhere close
> > > to simple. Nor is presenting those dependencies to users.
> >
> > Indeed it's not, but we *do* have a reference implementation and
> > lots of practical experience with it, and we have worked out all the
> > difficulties.
>
> Yes, I am pretty sure you tested it with *many different* packages.
> Maybe even all packages you have in Exherbo. Could you give some
> actual numbers, and details?

That depends. Are we seriously considering this for implementation on
Gentoo at any time in the foreseeable future? Writing this all up
properly is a lot of work, and I don't see much point in putting in the
effort if it isn't going to go anywhere.

--
Ciaran McCreesh

EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, Sep 2, 2012 at 9:04 PM, Ciaran McCreesh
<ciaran.mccreesh@googlemail.com> wrote:
> On Sun, 2 Sep 2012 20:46:19 +0200
> Fabio Erculiani <lxnay@gentoo.org> wrote:
>> On Sun, Sep 2, 2012 at 8:08 PM, Ciaran McCreesh
>> <ciaran.mccreesh@googlemail.com> wrote:
>> > and we have worked out all the difficulties.
>>
>> Please elaborate. What difficulties? What did you implement other than
>> plain SDEPEND? With what features? Lots of detail missing.
>
> The big issues you're ignoring:

And you call them "big issues"?
Ah, the "you're ignoring" part looks a bit arrogant, are you short
with arguments ;-) ?

>
> --
> Ciaran McCreesh

--
Fabio Erculiani

EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, 2 Sep 2012 20:10:38 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Sun, 2 Sep 2012 21:07:30 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > On Sun, 2 Sep 2012 19:08:33 +0100
> > Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > > On Sun, 2 Sep 2012 17:23:40 +0200
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > An effective SDEPEND implementation is definitely nowhere close
> > > > to simple. Nor is presenting those dependencies to users.
> > >
> > > Indeed it's not, but we *do* have a reference implementation and
> > > lots of practical experience with it, and we have worked out all
> > > the difficulties.
> >
> > Yes, I am pretty sure you tested it with *many different* packages.
> > Maybe even all packages you have in Exherbo. Could you give some
> > actual numbers, and details?
>
> That depends. Are we seriously considering this for implementation on
> Gentoo at any time in the foreseeable future? Writing this all up
> properly is a lot of work, and I don't see much point in putting in
> the effort if it isn't going to go anywhere.

Yes, we are. Zac was very positive about implementing it.

--
Best regards,
Michał Górny

EAPI 5+: split PDEPEND introducing SDEPEND
 

On Sun, 2 Sep 2012 21:38:26 +0200
Fabio Erculiani <lxnay@gentoo.org> wrote:
> On Sun, Sep 2, 2012 at 9:04 PM, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
> > On Sun, 2 Sep 2012 20:46:19 +0200
> > Fabio Erculiani <lxnay@gentoo.org> wrote:
> >> On Sun, Sep 2, 2012 at 8:08 PM, Ciaran McCreesh
> >> <ciaran.mccreesh@googlemail.com> wrote:
> >> > and we have worked out all the difficulties.
> >>
> >> Please elaborate. What difficulties? What did you implement other
> >> than plain SDEPEND? With what features? Lots of detail missing.
> >
> > The big issues you're ignoring:
>
> And you call them "big issues"?

Based upon the amount of work it took to get them right for kdebuilds
and then Exherbo, yes.

--
Ciaran McCreesh