FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 03-12-2012, 04:30 PM
Rich Freeman
 
Default RFD: EAPI specification in ebuilds

On Mon, Mar 12, 2012 at 1:01 PM, Zac Medico <zmedico@gentoo.org> wrote:
> It would be very fragile without the sanity check / feedback mechanism
> that's already been suggested.

Another obvious check is to have repoman run a grep with the regexp
and give an error if there is not exactly one match.

Rich
 
Old 03-12-2012, 04:46 PM
Zac Medico
 
Default RFD: EAPI specification in ebuilds

On 03/12/2012 10:30 AM, Rich Freeman wrote:
> On Mon, Mar 12, 2012 at 1:01 PM, Zac Medico <zmedico@gentoo.org> wrote:
>> It would be very fragile without the sanity check / feedback mechanism
>> that's already been suggested.
>
> Another obvious check is to have repoman run a grep with the regexp
> and give an error if there is not exactly one match.

If we want to handle every possible screwup, including stray EAPI
assignments inside inherited eclasses, we still need to compare the
probed value to the value that's obtained from bash.

I guess you're hinting at using a non-bash ebuild format? Even if in
that case, the the package manager should simply use whatever
interpreter is appropriate for the probed EAPI. So, if EAPI 7 is a
posix-shell format, and the probed EAPI is 7, then the package manager
should source the ebuild with it's posix-shell instead of bash. If the
probed EAPI is not a supported EAPI, then it should skip the sourcing
entirely, and report the ebuild as having an unsupported EAPI.
--
Thanks,
Zac
 
Old 03-12-2012, 06:20 PM
Rich Freeman
 
Default RFD: EAPI specification in ebuilds

On Mon, Mar 12, 2012 at 1:46 PM, Zac Medico <zmedico@gentoo.org> wrote:
> If we want to handle every possible screwup, including stray EAPI
> assignments inside inherited eclasses, we still need to compare the
> probed value to the value that's obtained from bash.

Well, I wasn't intending to suggest that the repoman check need be the
only one. However, preventing problems is at least as useful as
detecting them.

That said, it would probably best to have exactly one way to determine
the official EAPI. If that is to parse the filename, then parse the
filename. If it is to grep for a regexp and expect exactly one hit
and parse it in a certain way, then do that.

It might be that the "one" official way is to grep for a regexp and if
you find it, use it, otherwise assume 0, 1, 2, 3, or 4 and source the
ebuild for it. That still gives you only a single answer (well,
except in situations where the current way is already broken).

If people want to abuse the EAPI syntax I suppose we can generate an
error, but ignoring it might be just as valid behavior. I'm not sure
what happens if you define PN/etc in ebuild besides things breaking in
a horrible manner. I'd put changing EAPI in the same category.

Rich
 
Old 03-14-2012, 01:01 AM
Brian Harring
 
Default RFD: EAPI specification in ebuilds

On Mon, Mar 12, 2012 at 09:05:26AM -0700, Zac Medico wrote:
> On 03/12/2012 01:36 AM, Brian Harring wrote:
> > On Sun, Mar 11, 2012 at 09:08:24PM -0700, Zac Medico wrote:
> >> 1) User downloads an overlay that doesn't provide cache. We want the
> >> package manager to give a pretty "EAPI unsupported" message, rather than
> >> spit out some bash syntax errors.
> >
> > This criticsm pretty much applies *strictly to the existing
> > implementation*. It's disenguous busting it in this fashion.
> >
> > EAPI as a function explicitly gives it an out before hitting any of
> > that, eliminating your entire critique. Same goes for parsing it out
> > of the ebuild, or renaming the extension.
>
> You're assuming that the ebuild calls your eapi() function before it
> uses any syntax that's unsupported by the user's installed version of bash.

A bit, although that's a pretty valid assumption frankly. For current
bash syntax, the only thing they could do that would cause issues is
bash regex's for example- which if they have regex's prior to the eapi
invocation, they're doing something stupid anyways.

Regardless, detecting and suppressing isn't too hard- start sourcing
w/ `set -e`, disabling that once `eapi` has been invoked for example.
Pretty sure people will scream "that's horrible", but it's
surprisingly effective.


> > 1) PM still doesn't support that EAPI, looks at the cache/ebuild:
> > checksums are the same, thus the stored EAPI is trustable, leading to
> > the PM knowing it still can't process that ebuild and masking it
> > appropriately.
>
> You're assuming that cache is provided by the repo,

Sigh. I'm making no such assumptions, nor would I; it's a stupid line
of thought.

All of this has to function in the absence of a cache, and that's a
core usage scenario.


> which is not
> guaranteed, depending on the source. Even if the cache does exist, then
> you're assuming it's in a format that the package manager can reliably
> parse the EAPI from, even though that EAPI may not be supported. That
> may or may not reliable assumption, and having a pre-defined protocol to
> directly obtain the EAPI without using the cache is much more reliable.

This is a nonstatement. To deal w/ the cache (validate it) you have
to be able to reliably pull EAPI out of the ebuild.

That's what this whole fucking discussion is about; really not sure
why you're trying to argue this point against eapi as a function. As
I already laid out, it can deal w/it, same as the rest. Importantly,
the approach can also work across the transition period preventing
current-day PMs (using current EAPI mechanisms) from breaking when
used against later EAPIs that were released via `eapi as a function`.


> > What I'd like to see, is accuracy in this discussion. Skip the
> > handwavey "complexity! complexity! complexity!" crap, same for
> > selective robustness definitions. Past attempts at this discussion
> > mostly failed due to people pulling crap like this and frankly it just
> > pisses people off.
>
> It's just a symptom of people not abiding by the KISS principle. When
> you start talking about an approach such as the "eapi() function"
> approach which introduces lots of unnecessary complexity, it naturally
> makes the whole discussion more complex and hand-wavey.

With respect; you're proposing we go gum up version parsing via
shoving EAPI directly into it. Literally, make what is already a
complex mess, worse. Apply some KISS to your proposal please.

Just hammering the point home; compatibility *is* complex. Claiming
otherwise is naive. Case in point: your proposal breaks the shit out
of any current-day package manager that saw such a filename.

I really have a hard time reading your posts when basic issues like
that aren't paid attention to, but you've no problems claiming
complexity/brokenness in other proposals.

As I said, I'd like to see some accuracy; not hand wavy buzzwords.

I'd much more like to see prototypes of peoples proposals in addition
since at least that way they would flush out the breakages in their
proposals (potentially dropping it in the process since some of these
are pretty half baked).

~brian
 
Old 03-14-2012, 01:16 AM
Zac Medico
 
Default RFD: EAPI specification in ebuilds

On 03/13/2012 07:01 PM, Brian Harring wrote:
> With respect; you're proposing we go gum up version parsing via
> shoving EAPI directly into it. Literally, make what is already a
> complex mess, worse. Apply some KISS to your proposal please.
>
> Just hammering the point home; compatibility *is* complex. Claiming
> otherwise is naive. Case in point: your proposal breaks the shit out
> of any current-day package manager that saw such a filename.

I'm not really interested in GLEP 55 or variants of it. I was just
saying that if we do go with a GLEP 55 variant, then I'd prefer one
that's similar to the "EAPI in the filename with one-time extension
change" option [1]. There are plenty of ways to do that without making
it difficult to separate the EAPI from the version part.

[1]
http://www.gentoo.org/proj/en/glep/glep-0055.html#eapi-in-the-filename-with-one-time-extension-change
--
Thanks,
Zac
 
Old 03-18-2012, 06:23 AM
Ralph Sennhauser
 
Default RFD: EAPI specification in ebuilds

On Wed, 7 Mar 2012 21:41:02 +0100
Ulrich Mueller <ulm@gentoo.org> wrote:

> Hi all,
>
> The way how we currently specify the EAPI in ebuilds has some
> problems. For example, there is no sane way to allow usage of features
> of a new bash version in a new EAPI. So we are currently stuck with
> bash 3.2. Also changes of global scope behaviour, like addition of new
> global scope functions (similar to "inherit") are not possible.
>
> These flaws are outlined in GLEP 55 [1]:
> | In order to get the EAPI the package manager needs to source the
> | ebuild, which itself needs the EAPI in the first place. Otherwise it
> | imposes a serious limitation, namely every ebuild, using any of the
> | future EAPIs, will have to be source'able by old package managers
> | [...]
>
> The council has voted down GLEP 55 more than a year ago, but at the
> same time requested that a solution for the mentioned issues should be
> found. [2] However, there was no progress since then.
>
> The issue arose again in bug 402167 [3] where several solutions have
> been discussed. Below, I try to summarise the possible options
> resulting from that discussion.
>
>
> *** Proposal 1: "Parse the EAPI assignment statement" ***
>
> This first proposal would require that the syntax of the EAPI
> assignment statement in ebuilds matches a well defined regular
> expression. A scan of the Portage tree shows that the statement only
> occurs in the following variations (using EAPI 4 as example):
>
> EAPI=4
> EAPI="4"
> EAPI='4'
>
> Sometimes this is followed by whitespace or a comment (starting with
> a # sign). Also, with very few exceptions the EAPI assignment occurs
> within the first few lines of the ebuild. For the vast majority of
> ebuilds it is in line 5.
>
> Written in a more formal way, appropriate for a specification:
> - Ebuilds must contain at most one EAPI assignment statement.
> - It must occur within the first N lines of the ebuild (N=10 and N=30
> have been suggested).
> - The statement must match the following regular expression (extended
> regexp syntax):
> ^[ ]*EAPI=(['"]?)([A-Za-z0-9._+-]*)1[ ]*(#.*)?$
>
> Note: The first and the third point are already fulfilled by all
> ebuilds in the Portage tree. The second point will require very few
> ebuilds to be changed (9 packages for N=10, or 2 packages for N=30).
>
> The package manager would determine the EAPI by parsing the assignment
> with above regular expression. A sanity check would be added. Citing
> Zac Medico in [3]: "The fact that we can compare the probed EAPI to
> the actual EAPI variable after the ebuild is sourced seems like a
> perfect sanity check. We could easily detect inconsistencies and flag
> such ebuilds as invalid, providing a reliable feedback mechanism to
> ebuild developers."
>
> This proposal comes in two variants:
> 1a) The change is applied retroactively for all EAPIs.
> 1b) It is only applied for EAPI 5 and later (which means that the
> result of the EAPI parsing would be discarded for earlier EAPIs).
>
>
> *** Proposal 2: "EAPI in header comment" ***
>
> A different approach would be to specify the EAPI in a specially
> formatted comment in the ebuild's header. No syntax has been suggested
> yet, but I believe that the following would work as a specification:
> - The EAPI must be declared in a special comment in the first line of
> the ebuild's header, as follows:
> - The first line of the ebuild must contain the word "ebuild",
> followed by whitespace, followed by the EAPI, followed by
> end-of-line or whitespace.
>
> Again, the proposal comes in two variants:
> 2a) It is combined with a one time change of the file extension, like
> .ebuild -> .eb.
> 2b) The usual EAPI assignment statement in the ebuild is still
> required, at least for a transition period.
>
> In the 2a case, the EAPI variable could be made read-only in bash
> before sourcing the ebuild. In the 2b case, a sanity check similar to
> the one mentioned above would be added.
>
>
> What do you think?
>
> (I really hope for a constructive discussion here. So, if you want
> to comment that all of the above proposals suck and GLEP 55 is much
> superior, then please open a new thread for it.)
>
> Ulrich
>

Currently 5 proposals are listed on the wiki. [4]

While all of them have some temptations the actual goal is to make
obtaining the EAPI the very first step so everything else can be
defined in terms of EAPI and so immediately deployable in future. This
are changes in atom syntax like needed for GLEP 54 or those bash
feature often mentioned besides many other things one can think of.

GLEP 55 requires changing ebuild extensions on a regular basis but
doesn't impose any limit on the ebuild format or atom syntax, only the
file extensions would be imposed. The ebuild extensions for GLEP 55
would likely always be ebuild-<integer> as integers are reserved for
future use by Gentoo. While for example .ebuild-5 is still recognised
as an ebuild; .eb .ebld .ebd .bld .dliube .dlbe .be are not. This
brings me to the point if not GLEP 55 then only if we can keep .ebuild.
That one-time extension change thingy smells like being stubborn.

If we aren't going to to change the extension the most sensible thing
is to pack the eapi into the ebuild as done now but get rid of as many
limitations as possible. Looking at the proposals for eapi specification
in ebuilds the header comment is the clear winner here. The eapi can
also instruct the package manager to remove the first line for further
use of the ebuild if required for obscure formats / languages.

I already ruled out proposal "3 EAPI in header comment and one-time
change of file extension" for me as an option, see above. And "2 EAPI
in header comment" comes with compatibility issues for an undefined
amount of time.
If we want to keep .ebuild but avoid the compat issue another variant
would be "EAPI in header comment and one-time change of ebuild
location" or more formal:

6 EAPI in header comment and one-time change of ebuild location:

- add a directory $CATEGORY/$PN/ebuilds to ebuild repositories.
- all files in $CATEGORY/$PN/ebuilds are ebuilds and are using a well
defined first line to denote the EAPI.
- For practical reasons the header should be a bash comment. PMs
shouldn't have to remove or skip first line from file for further
processing of ebuilds supporting bash comments.
- the .ebuild extension can be kept but could be changed if ever
desired. This due to the filename only having meaning if the
EAPI of the file is known.

Comparing this with GLEP 55 then this allows us to keep .ebuild in
return of some overhead with roughly the same pros and cons otherwise,
right?

Ralph


>
> [1] <http://www.gentoo.org/proj/en/glep/glep-0055.html#problem>
> [2]
> <http://www.gentoo.org/proj/en/council/meeting-logs/20100823-summary.txt>
> [3] <https://bugs.gentoo.org/show_bug.cgi?id=402167>
>
[4] <http://wiki.gentoo.org/wiki/Alternate_EAPI_mechanisms>
 
Old 03-18-2012, 10:18 AM
Rich Freeman
 
Default RFD: EAPI specification in ebuilds

On Sun, Mar 18, 2012 at 3:23 AM, Ralph Sennhauser <sera@gentoo.org> wrote:
> The ebuild extensions for GLEP 55
> would likely always be ebuild-<integer> as integers are reserved for
> future use by Gentoo.

Looking at GLEP 55 the proposal was ebuild-<EAPI> - not
ebuild-<integer>. I don't believe there is any restriction that EAPIs
be integers or incremental in nature.

Don't mean to nitpick, but perhaps not all realize this...

Rich
 
Old 03-18-2012, 10:27 AM
Ulrich Mueller
 
Default RFD: EAPI specification in ebuilds

>>>>> On Sun, 18 Mar 2012, Ralph Sennhauser wrote:

> If we want to keep .ebuild but avoid the compat issue another
> variant would be "EAPI in header comment and one-time change of
> ebuild location" or more formal:

> 6 EAPI in header comment and one-time change of ebuild location:

> - add a directory $CATEGORY/$PN/ebuilds to ebuild repositories.
> - all files in $CATEGORY/$PN/ebuilds are ebuilds and are using a
> well defined first line to denote the EAPI.
> - For practical reasons the header should be a bash comment. PMs
> shouldn't have to remove or skip first line from file for further
> processing of ebuilds supporting bash comments.
> - the .ebuild extension can be kept but could be changed if ever
> desired. This due to the filename only having meaning if the
> EAPI of the file is known.

Similar ideas have been discussed before, in 2007 and 2009:
<http://archives.gentoo.org/gentoo-dev/msg_fe3f0b9d050ead86aed9b42ce7ec93b0.xml>
<http://archives.gentoo.org/gentoo-dev/msg_2e41942be33d8595cf7152aa91417fbe.xml>

> Comparing this with GLEP 55 then this allows us to keep .ebuild in
> return of some overhead with roughly the same pros and cons
> otherwise, right?

From a technical point of view, it's the same pros and cons. There are
however non-technical aspects for all these propositions. For example,
we would have to live with ebuilds in different directories for a long
transition period (as we would have to live with two file extensions
for a long time if we changed that).

Ulrich
 
Old 03-19-2012, 12:12 AM
Steven J Long
 
Default RFD: EAPI specification in ebuilds

Firstly, wrt probing the ebuild for EAPI=.. I'd just like to point out that
a regex is not required during the scan, and nor is restricting it to the
first N lines, though the latter may be desirable and could trivially
exclude comment and whitespace-only or empty lines.

Ciaran McCreesh wrote:
>Michael Orlitzky wrote:
>> Fair enough, but aren't you arguing the opposite point with Zac? If
>> ebuilds are data, fine, we write EAPI=4 somewhere and be done with
>> it. Anything not having that format is out-of-spec.
>
> The problem is that right now there's no way to determine the format of
> the data until you already know the format of the data.
Well, we know it's bash..

> We hack around
> this by not allowing "drastic" format changes, where "drastic" includes
> "using things in newer versions of bash" and "not adding new global
> scope commands".
>
> The question under discussion is whether we a) keep "what format the
> data is in" as being part of the data, but impose some strange and
> arbitrary conditions on it
Stipulating an allowed set of characters is in no way arbitrary, nor
strange- we already have similar restrictions on category and package names,
versions, keywords and USE flags, for example. Requiring that the EAPI
assignment for a bash .ebuild must be a literal (ie EAPI="foo" or EAPI=foo
or EAPI='foo') at the start of a line, is not hard to understand; as you
said ebuild authors already have to deal with lots of other subtle
restrictions. As Marc Schiffbauer said, EAPI "might be the most important
constraint in an ebuild at all" (from this and earlier discussion, it's
clear that it definitely is) -- ebuild developers have to know about it, and
this is a simple, clear restriction. Michał Górny stated: "The most
important ebuild variables like EAPI should be readable on sight, without
having to lookup random variables, functions etc" and in fact, all ebuilds
in the tree already use a string literal- it just makes more sense from a
code readability pov, quite apart from anything else.

You mentioned indentation in another mail; afaics there is no problem with
whitespace at the start of the line.

Ensuring EAPI declaration and sourced value match, has value beyond use in
EAPI extraction: it's a sanity check that should be performed in any case,
way before an ebuild hits the tree.

> , b) make a one-time change to have some kind
> of 'header' inside the file describing its format that isn't really part
> of the data itself, or c) admit that GLEP 55 already solved the problem
> and we might as well just fix the issue properly once and for all, even
> if GLEP 55's author is considered by some to be one of Satan's little
> minions.
>
Regardless of your past behaviour, my objections to GLEP 55 are, and always
have been, technical: it breaks encapsulation, which once implemented cannot
be taken back. It results in a mess of filename extensions, which are
confusing and irrelevant to end-users, as well as making other tools and
scripts trickier to implement; a simple example: a highlighting editor would
need to pattern-match the file extension. It's not needed: the simplest,
least-invasive alternative already works, and should have been adopted years
ago, when the Council asked for alternatives to be tried. The tree is
clearly in shape to do so now, though.

Package versions have to be in the filename to avoid collisions, and indeed
the information is relevant to both end-users and developers. EAPI, while
vital to the mangler and of immediate concern to developers, matches neither
of those. Since it is of immediate concern, restricting it to a string
literal makes sense from both maintenance (which is why it matches tree-
usage) and implementation perspectives. And specifying what characters are
allowed is a no-brainer; it's odd that that still has not been done, despite
it also being a requirement for embedding EAPI in filenames.

Your motivation for GLEP-55 states: "In order to get the EAPI the package
manager needs to source the ebuild." Given a suitable specification, that
isn't the case. repoman checks and explicit documentation are all that's
needed beyond that.

As for non-bash ebuilds, I have always agreed with antarus that they should
simply use a different extension. Adding a new extension per source language
is a *lot* cleaner than one per EAPI.

Regards,
Steve.
--
#friendly-coders -- We're friendly, but we're not /that/ friendly ;-)
 
Old 03-19-2012, 12:36 AM
Kent Fredric
 
Default RFD: EAPI specification in ebuilds

On 19 March 2012 14:12, Steven J Long <slong@rathaus.eclipse.co.uk> wrote:
>
> As for non-bash ebuilds, I have always agreed with antarus that they should
> simply use a different extension. Adding a new extension per source language
> is a *lot* cleaner than one per EAPI.

Ok: If we take this notion and enshrine it in stone:

If we assume Bash 4 is a seperate language from Bash 3, as its
syntax-backwards-incompatible, is it fair to suggest that for some
future EAPI which require Bash 4, that the extension change to suit?

ie: move from .ebuild to .ebuild4 , where '.ebuild' conveys the
format is bash, and that '.ebuild4' is bash4 only?

That way you have a forwards declaration of the syntax/file format
required to parse the file, but no declaration of the EAPI, so you're
not breaking encapsulation.

This is breaking the direct file==eapi connection, but still
maintaining a loose file<->eapi connection.

Its /sort/ of like the "one time extension change" proposal, except
its less 'arbitrary' than something like .eb , and it gives us the
future option of changing the suffix again if bash 5 comes out with
different syntax.

Then we can do

.ebuild = EAPI 0 - 4 & bash >= 3
.ebuild4 = EAPI5 - 9 & bash >= 4
.ebuild5 = EAPI10 - 15 & bash >= 5

Thoughts?

--
Kent

perl -e* "print substr( "edrgmaM* SPA NOcomil.ic@tfrken", $_ * 3,
3 ) for ( 9,8,0,7,1,6,5,4,3,2 );"
 

Thread Tools




All times are GMT. The time now is 04:03 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org