Richard Yao posted on Wed, 20 Jun 2012 16:50:33 -0400 as excerpted:

> On 06/20/2012 04:35 PM, Ciaran McCreesh wrote:
>> On Wed, 20 Jun 2012 16:25:30 -0400 Richard Yao <ryao@gentoo.org> wrote:

>>> POSIX Shell compliance
>>
>> So far as I know, every PM relies heavily upon bash anyway (and can't
>> easily be made not to), so even if developers would accept having to
>> rewrite all their eclasses, it still wouldn't remove the dep.
>>
> Lets address POSIX compliance in the ebuilds first. Then we can deal
> with the package managers.

Additionally, this is extremely unlikely because a number of developers
insist on bash, to the extent that it would likely split gentoo in half
if this were to be forced. It wouldn't pass council. It's unlikely to
even /get/ to council.

Openrc could move to POSIX shell because its primary dev at the time
wanted it that way and it's only a single package. However, even then,
doing it was controversial enough that said developer ended up leaving
gentoo in-part over that, tho he did continue to develop openrc as a
gentoo hosted project for quite some years. Now you're talking trying to
do it for /every/ (well, almost every) package, thus touching every
single gentoo dev. It's just not going to happen in even the medium term
(say for argument APIs 5-7ish), let alone be something practical enough
to implement, soon enough (even if everyone agreed on the general idea,
they don't), to be anything like conceivable for EAPI5.

So just let that one be. It's simply not worth tilting at that windmill.

(Arguably, multi-arch, while practical and actually working at least with
portage in an overlay, fails that last bit as well. If it was pushed,
perhaps for EAPI6 or 7, but it's just not practical to consider it for
EAPI5... unless you want to wait 3-5 years for EAPI5!)

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On 06/21/2012 04:29 AM, Duncan wrote:
> Richard Yao posted on Wed, 20 Jun 2012 16:50:33 -0400 as excerpted:
>
>> On 06/20/2012 04:35 PM, Ciaran McCreesh wrote:
>>> On Wed, 20 Jun 2012 16:25:30 -0400 Richard Yao <ryao@gentoo.org> wrote:
>>>> POSIX Shell compliance
>>> So far as I know, every PM relies heavily upon bash anyway (and can't
>>> easily be made not to), so even if developers would accept having to
>>> rewrite all their eclasses, it still wouldn't remove the dep.
>>>
>> Lets address POSIX compliance in the ebuilds first. Then we can deal
>> with the package managers.
> Additionally, this is extremely unlikely because a number of developers
> insist on bash, to the extent that it would likely split gentoo in half
> if this were to be forced. It wouldn't pass council. It's unlikely to
> even /get/ to council.
>
> Openrc could move to POSIX shell because its primary dev at the time
> wanted it that way and it's only a single package. However, even then,
> doing it was controversial enough that said developer ended up leaving
> gentoo in-part over that, tho he did continue to develop openrc as a
> gentoo hosted project for quite some years. Now you're talking trying to
> do it for /every/ (well, almost every) package, thus touching every
> single gentoo dev. It's just not going to happen in even the medium term
> (say for argument APIs 5-7ish), let alone be something practical enough
> to implement, soon enough (even if everyone agreed on the general idea,
> they don't), to be anything like conceivable for EAPI5.
>
> So just let that one be. It's simply not worth tilting at that windmill.
>
> (Arguably, multi-arch, while practical and actually working at least with
> portage in an overlay, fails that last bit as well. If it was pushed,
> perhaps for EAPI6 or 7, but it's just not practical to consider it for
> EAPI5... unless you want to wait 3-5 years for EAPI5!)
>
It is just a wish list.

Anyway, people need to decide on what they want from a new EAPI before
one is made. Once they decide, it should be possible to work out the
details.

On Thu, Jun 21, 2012 at 9:25 AM, Pacho Ramos <pacho@gentoo.org> wrote:
> El jue, 21-06-2012 a las 08:00 +0100, Ciaran McCreesh escribió:
>> On Thu, 21 Jun 2012 08:08:55 +0200
>> Pacho Ramos <pacho@gentoo.org> wrote:
>> > Also, if I remember correctly, Tommy asked for this some months ago,
>> > you asked for what he sent some days ago and now you require more and
>> > more work to delay things to be implemented.
>>
>> I still haven't seen a clear description of exactly what should be
>> changed and why. I've also not seen a description of exactly what
>> problem is being solved, nor a discussion of the relative merits of
>> implementing a solution to whatever that problem is. All I've seen is a
>> mess of code that "gets it working" in Portage (which isn't the same as
>> "implements it for Portage") and a big wall of text that contains lots
>> that no-one needs to know and little of what's important. This needs to
>> go through the GLEP process, and it needs a PMS diff.
>>
>> In case you're not aware, the first time Gentoo did multilib, it was
>> done as a series of random changes to Portage that no-one really
>> thought through or understood. As you can see, that didn't work...
>>
>
> Then, looks clear to me that the way to get things approved in newer
> EAPIs is not clear enough as looks like a lot of devs (like me) don't
> know them (for example, when things to be added to EAPI need also a GLEP
> and a PMS diff, also the needing to get an implementation for any
> package manager). Is this documented in some place? If not, I think it
> should be documented and, of course, it should be done by PMS team if
> possible as they clearly know what they expect to get for approval if
> needed since, I discussed some days ago, council will simply accept some
> specific features to be included in next eapi once they are accepted by
> PMS team. That way, we could save a lot of time, know what steps are
> pending, try to ask for help for some specific steps and, finally, get
> it discussed in PMS people providing all what is needed.
>
>
>> > I also don't understand why Gentoo is forced to stick with old ways
>> > of doing things until new EAPI is approved
>>
>> That's not what's going on here. The issue is that there might be one
>> person who understands what "the new way of doing things", but he
>> hasn't told us what he thinks that is. Once we get a proper
>> explanation, getting an EAPI out doesn't take long.
>>
>
> But you must confess that old problems like multilib support, force
> package rebuilding or optional dep support are still pending while still
> needing and, the problem with the way things are discussed now is that
> some day anybody arises the problem again, other one demands more things
> to be provided, a discussion starts, the problem gets stalled... one
> year later the same problem arises again. There is clearly a lack of
> information to the rest of developers about how to propose anything to
> get accepted for next EAPI.

I'm not following you here.

1) Usually features need a reference implementation.
2) For portage, there are like 3-5 people who know portage well enough
to write that for you.
3) You generally have to convince them to do it before you can move forward.
4) Most features never even get a reference implementation.

There is this vague idea that you can just propose something; get
consensus on the ML, everyone goes to implement it, and then it works
just as designed. That is usually called the 'waterfall' model and its
really hard to do correctly.

What I imagine the process is:

1) Submit an idea to the ML; you just need feedback (not consensus,
which is likely impossible for non-trivial ideas.)
1.1) Your idea could be a GLEP, but I don't think a GLEP is strictly required.
2) Take feedback from step one and make an initial implementation. You
will likely find some assumptions in your 'design' from step one were
wrong, as well as other implementation issues (UI, performance, etc.)
3) Modify your idea to address the problems in 2.
4) Modify your implementation to address the problems in 2.
5) Repeat steps 2-4 a few times until you have solved all the 'big' problems.
6) Submit a diff to PMS outlining how the package manager behavior is
changed by your feature. This generally needs to be specific enough so
that other package manager authors can implement the feature.
7) Submit a devmanual patch telling users how to use the feature.

Most people fail at step two; usually because they try to get
'consensus' at step one, which is stupid and a waste of time. There
are a few hundred people on this list; we will never all agree, on
basically anything. So take the feedback people give you and do
something with it.

>
>> The main problem here isn't even EAPI related. Ebuild developers don't
>> even know what they'll be expected, required or able to do for multilib.
>>
>> > while Exherbo is free to implement and use things like that special
>> > way of handling DEPENDENCIES without waiting for any EAPI to be
>> > accepted...
>>
>> The DEPENDENCIES proposal predates Exherbo. Gentoo originally didn't
>> have it because Portage couldn't implement it. Now it doesn't have it
>> because it's too controversial to get it approved.
>
> It was only a example, but thanks for the info
>
>> Exherbo does have it
>> because it was carefully discussed, compared to alternatives, planned
>> out, tested, accepted by the expendable figurehead, and then rolled out.
>>
>> > or maybe I am wrong and people is able to use any PM manager
>> > compliant with EAPI on exherbo without issues?
>>
>> If anyone ever manages to come up with another package mangler that can
>> get close to implementing what Exherbo needs, then sure.
>>
>
> Then, you accept exherbo is not forced to *only* follow EAPI while you
> force Gentoo and portage to only support features approved in an EAPI?
>

Portage can use whatever EAPIs portage wishes to publish (Zac has
published portage specific EAPIs in the past.) Generally in gentoo-x86
you can only use council approved EAPIs; so if portage was to publish
something like 'portage-1' you would have to get council approval to
use it in Gentoo-x86. It seems like a reasonable request to me, why
not talk to them about it.

The reason exherbo can 'do whatever they want' is because the project
is marketed that way. Seriously, go to Exherbo.org and look at the
'Why' section. Reason 2 is 'we need to break stuff whenever we feel it
is beneficial.' Gentoo is not marketed that way to our users. We
'generally promise' backwards compat for 6-12 months.

If we randomly added stuff to portage without being bound by EAPI then
we end up breaking stuff unintentionally all the time when rolling out
features.

-A

On 21 June 2012 05:33, Alec Warner <antarus@gentoo.org> wrote:
> On Wed, Jun 20, 2012 at 10:25 PM, Richard Yao <ryao@gentoo.org> wrote:
>> Here is my wishlist for EAPI 5:
[...]
>> POSIX Shell compliance
>> * * * *There has been a great deal of work done to give the user full control
>> of what is on his system and there is more that we can do there. In
>> particular, I think a lean Gentoo Linux system should be able to use
>> busybox sh and nothing else. That requires POSIX shell compliance.
>> OpenRC init scripts support this and the configure scripts support this.
>> The few exceptions are bugs that are addressed by the Gentoo BSD developers.
>> * * * *As such, I think we should make EAPI=5 use POSIX shell by default. If
>> an ebuild requires bash, we can allow the ebuild to declare that (e.g.
>> WANT_SH=bash), but that should be the exception and not the rule.
>>
>
> Our ebuilds are written in bash. [...] Screw
> trying to get the PM to stop using bash; developers are not interested
> in writing ebuilds in posix shell; bar none.
>
> Why would I as an ebuild author waste a bunch of time writing my
> ebuild in posix compatible sh when I can use bash (and if I had a
> better language than bash to write ebuilds in; I'd use that too.) You
> are free to write your ebuilds in posix sh; good luck to you.

Ebuilds are written in bash. And the convenience of using bash
far outweighs any benefits of using posix sh instead. One needs
to make a very strong case to convince enough developers to
change this...

--
Cheers,

Ben | yngwin
Gentoo developer
Gentoo Qt project lead

On 06/21/12 15:25, Pacho Ramos wrote:
> El jue, 21-06-2012 a las 08:00 +0100, Ciaran McCreesh escribió:
>> On Thu, 21 Jun 2012 08:08:55 +0200
>> Pacho Ramos <pacho@gentoo.org> wrote:
>>> Also, if I remember correctly, Tommy asked for this some months ago,
>>> you asked for what he sent some days ago and now you require more and
>>> more work to delay things to be implemented.
>> I still haven't seen a clear description of exactly what should be
>> changed and why. I've also not seen a description of exactly what
>> problem is being solved, nor a discussion of the relative merits of
>> implementing a solution to whatever that problem is. All I've seen is a
>> mess of code that "gets it working" in Portage (which isn't the same as
>> "implements it for Portage") and a big wall of text that contains lots
>> that no-one needs to know and little of what's important. This needs to
>> go through the GLEP process, and it needs a PMS diff.
>>
>> In case you're not aware, the first time Gentoo did multilib, it was
>> done as a series of random changes to Portage that no-one really
>> thought through or understood. As you can see, that didn't work...
>>
> Then, looks clear to me that the way to get things approved in newer
> EAPIs is not clear enough as looks like a lot of devs (like me) don't
> know them (for example, when things to be added to EAPI need also a GLEP
> and a PMS diff, also the needing to get an implementation for any
> package manager). Is this documented in some place?
No, and this is a rather random ad-hoc requirement that hasn't been
specified before.

All previous EAPI processes went through some debate with dev-portage@
and the other involved people (mostly pkgcore/ferringb and
paludis/ciaranm), then the proposal got handed to council to vote on,
and if council was happy that proposal was hammered into PMS and the new
EAPI published. Most of the time new features had a crude approximation
of a patch for PMS available so that the documentation updates were done
in a timely manner.

I have no idea why Ciaran is trying to redefine the process now suddenly
and why people are taking this nonsense seriously.

> If not, I think it
> should be documented and, of course, it should be done by PMS team if
> possible as they clearly know what they expect to get for approval if
> needed since, I discussed some days ago, council will simply accept some
> specific features to be included in next eapi once they are accepted by
> PMS team. That way, we could save a lot of time, know what steps are
> pending, try to ask for help for some specific steps and, finally, get
> it discussed in PMS people providing all what is needed.
I would say PMS team accepts what council signs off ... or am I seeing
the order wrong here?


So, the normal way to get a feature into the next EAPI is ... write a
specification to the best of your capabilities, present it here, when
people are done bashing it ask PMS people to help you prepare a PMS
patch so that you can suggest it to Council, and then it's mostly a
matter of waiting until the next EAPI is finalized (which currently runs
at a glacial pace of about one EAPI a year as far as I remember)


Take care,

Patrick

On Thu, 21 Jun 2012 19:15:02 +0800
Patrick Lauer <patrick@gentoo.org> wrote:
> > Then, looks clear to me that the way to get things approved in newer
> > EAPIs is not clear enough as looks like a lot of devs (like me)
> > don't know them (for example, when things to be added to EAPI need
> > also a GLEP and a PMS diff, also the needing to get an
> > implementation for any package manager). Is this documented in some
> > place?
> No, and this is a rather random ad-hoc requirement that hasn't been
> specified before.

It's dependent upon the level of complexity of the work, and whether or
not anyone on the PMS team understands it enough to be able to do the
work themselves. As far as I know, none of us do on this one, so it's
down to whoever wants it to educate us enough that we can get it done...

> All previous EAPI processes went through some debate with dev-portage@
> and the other involved people (mostly pkgcore/ferringb and
> paludis/ciaranm), then the proposal got handed to council to vote on,
> and if council was happy that proposal was hammered into PMS and the
> new EAPI published. Most of the time new features had a crude
> approximation of a patch for PMS available so that the documentation
> updates were done in a timely manner.

Actually, we've been passing pretty much final PMS patches to the
Council to vote on.

> I have no idea why Ciaran is trying to redefine the process now
> suddenly and why people are taking this nonsense seriously.

I'm not.

> I would say PMS team accepts what council signs off ... or am I seeing
> the order wrong here?

The PMS team makes suggestions to the Council, and the Council accepts
a subset of those. The Council can also suggest that the PMS team looks
at a particular feature, but as Gentoo has no mechanism in place for
forcing work to get done, that only works if there's someone with both
the time and the knowledge to figure it out.

> So, the normal way to get a feature into the next EAPI is ... write a
> specification to the best of your capabilities, present it here, when
> people are done bashing it ask PMS people to help you prepare a PMS
> patch so that you can suggest it to Council

Yup, and for difficult cases like the ones under discussion, a part of
presenting it is to demo a working implementation on real packages so
that we gain real world experience of the feature.

It's also important to note that "the best of your capabilities" may
not be anywhere near enough for the PMS people or the package mangler
people to do the remainder of the work.

> and then it's mostly a
> matter of waiting until the next EAPI is finalized (which currently
> runs at a glacial pace of about one EAPI a year as far as I remember)

I like how you simultaneously troll both sides of that issue. Weren't
you previously claiming there were too many EAPIs and that we shouldn't
have lots of new ones?

--
Ciaran McCreesh

On Thu, Jun 21, 2012 at 5:27 AM, Alec Warner <antarus@gentoo.org> wrote:
> There is this vague idea that you can just propose something; get
> consensus on the ML, everyone goes to implement it, and then it works
> just as designed. That is usually called the 'waterfall' model and its
> really hard to do correctly.
>

++

Waterfall has problems even in places where people are paid to do
work. It has little chance of working here.

Devs aren't paid to do work. They do what interests them. So, the
most effective way to make something happen is to do it yourself, or
better still make it something interesting, do it yourself, and
inspire others to help you out. A working program will always gather
more interest than a discussion on a list.

Plus Gentoo is all about choice - even if it never becomes the
standard lots of people will do it anyway. Look at paludis, systemd,
and so on. The autobuilt releases started out as drobbins side
project on funtoo before they became the mainstream Gentoo way.
Showing people that something works is always better than telling
them.

Rich

On Thu, 2012-06-21 at 08:00 +0100, Ciaran McCreesh wrote:
>
> In case you're not aware, the first time Gentoo did multilib, it was
> done as a series of random changes to Portage that no-one really
> thought through or understood. As you can see, that didn't work...

No, but paved the way for other distros as they had nothing even close.
I'm sure you remember back then. Don't be an ass.

--
Homer Parker <hparker@gentoo.org>

On Thu, 2012-06-21 at 09:24 +0200, justin wrote:
> Won't it be a good thing, if you instead of showing all of us, that
> you
> can tell where people fail to present something in the right way, help
> and guide them to write the necessary things like PMS patches, GLEPs
> etc., so that we can proceed in an efficient way?

That's not his style. Never has been, and I don't see that changing.

--
Homer Parker <hparker@gentoo.org>

On Thu, 21 Jun 2012 07:11:27 -0500
Homer Parker <hparker@gentoo.org> wrote:
> On Thu, 2012-06-21 at 08:00 +0100, Ciaran McCreesh wrote:
> > In case you're not aware, the first time Gentoo did multilib, it was
> > done as a series of random changes to Portage that no-one really
> > thought through or understood. As you can see, that didn't work...
>
> No, but paved the way for other distros as they had nothing
> even close. I'm sure you remember back then. Don't be an ass.

And what did Gentoo get out of it?

What I remember is Gentoo putting in lots of work randomly changing
things until things worked, and ending up not knowing what most of
those changes were or why they were done. The end result is that
there's still a random smattering of multilib-related mess cluttering
up ebuild internals that doesn't actually do anything except cause
intermittent breakages. Doing experiments is great as a way of
understanding the problem, but it isn't how you deliver a solution.
That takes a lot more work, and someone has to be prepared to do it.

--
Ciaran McCreesh