FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 09-05-2012, 03:24 PM
Alec Warner
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Wed, Sep 5, 2012 at 2:46 PM, Rich Freeman <rich0@gentoo.org> wrote:
> On Wed, Sep 5, 2012 at 7:27 AM, Ciaran McCreesh
> <ciaran.mccreesh@googlemail.com> wrote:
>> Uhm. O(n) == O(n/2). Anything assuming they're different is just wrong.
>
> We're basically debating definitions. O notation is used to indicate
> how algorithms scale and nobody uses O(n/2) and such as a result.
>
> An algorithm that is twice as slow is still twice as slow. That might
> or might not matter. However, this isn't the domain where O notation
> is used. You use O notation when your algorithm takes 30 seconds to
> run and you want to know what happens with the dataset doubles 3000
> times. It generally doesn't matter if the result is that it will take
> 1 trillion years to operate or 2 trillion years. You care more about
> whether it will take minutes, hours, weeks, years, or whatever.
>
> I can't really think of any practical examples where multiplying the
> time to parse a list of maybe 50 items vs 5 lists of 10 items is going
> to make that big of a difference. They're just lines in a text file -
> your CPU can compare a few billions characters per second. Sure, if
> you add 75 layers of abstraction you might be able to find just the
> right point where a factor of 5 is going to make it intolerable but a
> factor of 1 is almost acceptable, but go ahead and add/remove a few
> layers and suddenly it is all fine or all horrible anyway. That is a
> bit contrived. That's why everybody ignores constant factors in O
> notation anyway.

so tl;dr, this doesn't matter because string comparison is very fast.

-A

>
> Rich
>
 
Old 09-05-2012, 04:15 PM
Michał Górny
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Wed, 05 Sep 2012 00:06:45 +0000
"Jorge Manuel B. S. Vicetto" <jmbsvicetto@gentoo.org> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 31-08-2012 20:46, Ciaran McCreesh wrote:
>
> <snip>
>
> > Also, we're getting rather a lot of *DEPEND variables here... If
> > we're making people make major changes to their deps, which for
> > HDEPEND we definitely would be, then the "it's expensive since
> > people would have to redo their deps" argument against a combined
> > DEPENDENCIES variable goes out of the window, so we should rethink
> > that too.
>
> I have to agree with Ciaran, instead of multiplying DEPEND variables,
> it's probably time we move to a single DEPENDENCIES variable.

Well, if you really insist that Gentoo is the right place to reinvent
the wheel of bash variables...

If we really want to go this route, then please at least require
explicit label at start of DEPENDENCIES. And the same when appending to
DEPENDENCIES -- just so 'unlikely' mistakes will leave us with hours of
debugging.

Furthermore, please think of eclasses. They *all* will have to append
dependencies in two ways, in an EAPI-conditional way. Exherbo doesn't
have that problem because they don't care about backwards
compatibility. We will support old EAPIs for at least few years if not
until the end of Gentoo. Not that appending dependencies in eclasses is
really that good idea.

Remember that this requirement will actually cause migration to EAPI 5
to be even harder than to any previous EAPIs. Migrating a single ebuild
will require rewriting the dependencies, and migrating an eclass will
require adding a lot of dirty code. Especially if it is python.eclass.

And we will have to convert them back to old-style dependencies anyway.
For the sake of compatibility with external tools.

And as a final note, on a request from ulm, I have created a wiki page
where you could list all proposed new dependency types so we can have
a broader look when deciding:

http://wiki.gentoo.org/wiki/Future_EAPI/New_dependency_types

--
Best regards,
Michał Górny
 
Old 09-06-2012, 05:58 AM
Ciaran McCreesh
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Wed, 5 Sep 2012 18:15:43 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> If we really want to go this route, then please at least require
> explicit label at start of DEPENDENCIES. And the same when appending
> to DEPENDENCIES -- just so 'unlikely' mistakes will leave us with
> hours of debugging.

We should take the exheres-0 rules for labels and eclasses, which limit
labels' scopes to blocks, and which introduce an extra ( ) block around
the outside when doing eclass variable merging.

> Not that appending dependencies in eclasses is really that good idea.

Dependencies aren't appended over eclasses, they're merged.

(And I have a sneaking recollection of PMS not documenting this
properly...)

> Remember that this requirement will actually cause migration to EAPI 5
> to be even harder than to any previous EAPIs. Migrating a single
> ebuild will require rewriting the dependencies, and migrating an
> eclass will require adding a lot of dirty code.

Migrating to EAPI 5 requires rewriting dependencies anyway if we're
adding in HDEPEND. Also, earlier EAPIs have introduced new phase
functions, which is a far ickier change for ebuilds than this.

> Especially if it is python.eclass.

You know what the solution there is...

> And we will have to convert them back to old-style dependencies
> anyway. For the sake of compatibility with external tools.

No, external tools are required to be EAPI aware. If they're not, then
the external tools need fixing.

--
Ciaran McCreesh
 
Old 09-06-2012, 07:39 AM
Michał Górny
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Thu, 6 Sep 2012 06:58:51 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Wed, 5 Sep 2012 18:15:43 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > If we really want to go this route, then please at least require
> > explicit label at start of DEPENDENCIES. And the same when appending
> > to DEPENDENCIES -- just so 'unlikely' mistakes will leave us with
> > hours of debugging.
>
> We should take the exheres-0 rules for labels and eclasses, which
> limit labels' scopes to blocks, and which introduce an extra ( )
> block around the outside when doing eclass variable merging.

Because? I believe we should take 'Gentoo rules', including required
explicit build+run at the start.

> > Not that appending dependencies in eclasses is really that good
> > idea.
>
> Dependencies aren't appended over eclasses, they're merged.

Thanks for correcting my wording, like the naming was really relevant
to the topic.

> (And I have a sneaking recollection of PMS not documenting this
> properly...)

Yes, I think PMS is pretty silent about this. I think it doesn't even
say that in phase functions the contents are implementation-defined.

> > Remember that this requirement will actually cause migration to
> > EAPI 5 to be even harder than to any previous EAPIs. Migrating a
> > single ebuild will require rewriting the dependencies, and
> > migrating an eclass will require adding a lot of dirty code.
>
> Migrating to EAPI 5 requires rewriting dependencies anyway if we're
> adding in HDEPEND. Also, earlier EAPIs have introduced new phase
> functions, which is a far ickier change for ebuilds than this.

Do you really believe in HDEPEND in EAPI 5? I've already postponed this
in my mind. Also, not every single ebuild will actually need it.

> > And we will have to convert them back to old-style dependencies
> > anyway. For the sake of compatibility with external tools.
>
> No, external tools are required to be EAPI aware. If they're not, then
> the external tools need fixing.

Changing package manager API like that between EAPI is just bad. You
know that, don't you?

--
Best regards,
Michał Górny
 
Old 09-06-2012, 08:00 AM
Ciaran McCreesh
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Thu, 6 Sep 2012 09:39:25 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> On Thu, 6 Sep 2012 06:58:51 +0100
> Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > On Wed, 5 Sep 2012 18:15:43 +0200
> > Michał Górny <mgorny@gentoo.org> wrote:
> > > If we really want to go this route, then please at least require
> > > explicit label at start of DEPENDENCIES. And the same when
> > > appending to DEPENDENCIES -- just so 'unlikely' mistakes will
> > > leave us with hours of debugging.
> >
> > We should take the exheres-0 rules for labels and eclasses, which
> > limit labels' scopes to blocks, and which introduce an extra ( )
> > block around the outside when doing eclass variable merging.
>
> Because? I believe we should take 'Gentoo rules', including required
> explicit build+run at the start.

You mean, you want to invent some new rules that don't quite work,
rather than using the ones that do... The whole "initial labels" thing
isn't an issue for exheres-0, since rather than appending, the
resulting metadata variable ends up with extra ( ) blocks like this:

(
stuff/from-the-exheres
)
(
stuff/from-exlib-1
)
(
stuff/from-exlib-2
)

so there's no possibility of labels ending up applied to the wrong
thing.

If you just append, you'd have to have some way of validating that
eclasses all individually specify an initial label. That's not
something that can easily be done.

> > (And I have a sneaking recollection of PMS not documenting this
> > properly...)
>
> Yes, I think PMS is pretty silent about this. I think it doesn't even
> say that in phase functions the contents are implementation-defined.

That's covered under the general rules for environment variables. The
issue is that PMS doesn't make a good distinction between the metadata
variable's value and the environment variable value. The wording that's
in there currently dates back to how things worked a very long time ago.

> > > Remember that this requirement will actually cause migration to
> > > EAPI 5 to be even harder than to any previous EAPIs. Migrating a
> > > single ebuild will require rewriting the dependencies, and
> > > migrating an eclass will require adding a lot of dirty code.
> >
> > Migrating to EAPI 5 requires rewriting dependencies anyway if we're
> > adding in HDEPEND. Also, earlier EAPIs have introduced new phase
> > functions, which is a far ickier change for ebuilds than this.
>
> Do you really believe in HDEPEND in EAPI 5? I've already postponed
> this in my mind. Also, not every single ebuild will actually need it.

*shrug* if all the new *DEPEND stuff ends up in EAPI 6, then there's no
point in DEPENDENCIES for EAPI 5. But we'll have to sort this out
sooner or later...

> > > And we will have to convert them back to old-style dependencies
> > > anyway. For the sake of compatibility with external tools.
> >
> > No, external tools are required to be EAPI aware. If they're not,
> > then the external tools need fixing.
>
> Changing package manager API like that between EAPI is just bad. You
> know that, don't you?

Your choices are to make the package manager API abstract out this sort
of thing, or to require client updates for a new EAPI. And as it's
fairly hard to predict what's going to be in a new EAPI, being too
abstract is pretty much a lost cause.

You can't simply convert new concepts to old concepts, since there's no
pre-EAPI-5 concept of what any of these new dependency types are.
Treating an HDEPEND or an IDEPEND as a DEPEND is just wrong.

--
Ciaran McCreesh
 
Old 09-06-2012, 08:11 AM
Brian Harring
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Wed, Sep 05, 2012 at 12:06:45AM +0000, Jorge Manuel B. S. Vicetto wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 31-08-2012 20:46, Ciaran McCreesh wrote:
>
> <snip>
>
> > Also, we're getting rather a lot of *DEPEND variables here... If
> > we're making people make major changes to their deps, which for
> > HDEPEND we definitely would be, then the "it's expensive since
> > people would have to redo their deps" argument against a combined
> > DEPENDENCIES variable goes out of the window, so we should rethink
> > that too.
>
> I have to agree with Ciaran, instead of multiplying DEPEND variables,
> it's probably time we move to a single DEPENDENCIES variable.

Personally, my complaints re: it are that 1) while minor, the
labels in some cases are overly verbose; recommendations instead of
recommends, suggestions instead of suggests, etc. 2) An actual
flaw in their design (imo): it tries to intermix two different
forms of parsing, without any real justification for *why* beyond
*hey look kids, I can!*; The two can intersect in slightly fucked
up ways, case in point:

DEPENDENCIES="
run+build:
cat/the
x? ( cat/cow
test:
y? ( cat/says
z? ( cat/moo
)))

Now, there may be some unstated rules that disallow that, but if that
*is* allowed, that's frankly dumb. As to if it's disallowed, it's
kind of a design flaw that the situation can occur requiring an
explicit suppression.


Rather than invent and try intermixing a secondary form, just using
the existing strikes me as saner; either we can have a specific
use_expand group like thus:

DEPENDENCIES="
dep_run? ( cat/monkeys )
dep_run+build? ( cat/foo )"

Or, preferable imo, do away w/ the +, use a more natural ',' for phase
separation, and use ':';

DEPENDENCIES="
dep:run? ( cat/monkeys )
dep:run,build? ( cat/foo )"

Doing it that way reuses the existing parsing infrastructure (good)
via just requiring a change to the use validation machinery (easy if
the PM is implemented sanely).

It also is able to express things that exheres variation can't do as
cleanly; considering build/fetch/post/run/test as the viable dep
targets:

DEPENDENCIES="
build+fetch+post+test:
some-dep"
vs
DEPENDENCIES="!dep:run? ( some-dep )"

I don't much expect that to occur, but the potential exists, thus
mentioning it.


One unstated fault re: DEPENDENCIES btw, is it will not play nice w/
exactly one of blocks. Treating '^^' as "exactly one of", consider:

DEPENDENCIES="
^^ (
run:
cat/blah
build:
cat/dar
cat/foon
)"

Is that a stupid dep? You bet your ass it is.. But it would have to
be explicitly suppressed by the parser for any such construct- moreso,
repoman would have to spot it which is slightly unfun.

Finally, one note; while certain folk have been making lots of noise
about DEPENDENCIES being the best thing since sliced bread, their
isn't much comment about how one actually transitions to it without
making eclass authors who have to support both pre-DEPENDENCIES, and
post-DEPENDENCIES eapis happy; kind of swiss cheeses the hell out of
the code frankly.

A compatibility hack that stacks them is strongly advisable; something
akin to the following:

Literally, we do the following:
inherit() {
if eapi blah; then
local DEPEND PDEPEND RDEPEND
<usual saving/protection of DEPENDENCIES var>
else
<usual saving/protection of DEPEND/PDEPEND/RDEPEND vars>
fi
<normal sourcing machinery>
if eapi blah; then
local _deps=( ) _x
for _x in DEPENDENCIES DEPEND RDEPEND PDEPEND; do
[ -n "${!_x}" ] && deps+=( "${!_x}" )
done
[ ${#deps} -ne 0 ] && DEPENDENCIES="${deps[*]}"
unset DEPEND RDEPEND PDEPEND _x _deps
<normal stacking/restoration of DEPENDENCIES rules>
else
<normal stacking/restoration of RDEPEND/PDEPEND/DEPEND>
fi
}

Via that, we eclasses that are pure DEPENDENCIES eapi wise, just set
the DEPENDENCIES directly; those that have to support multiple eapi's
(aka, every fricking eclass that exists right now) can just use the
old form, shifting into the new form as things progress.


Either way, the dual parsing required for exheres version I'm -1 on;
I'm generally wary of NIH modifications, but the form I've mentioned
that reuses our existing machinery is +.5...ish... from my standpoint
(+1 on the form, just kind of 'meh' on the single var angle despite
mostly agreeing w/ the reasoning).

Either way, on w/ the flaming/insinuations of
idiocy/counter-insinuations of being a wee bit too friendly w/
sheep...
~harring
 
Old 09-06-2012, 08:27 AM
Michał Górny
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Thu, 6 Sep 2012 09:00:40 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Thu, 6 Sep 2012 09:39:25 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > On Thu, 6 Sep 2012 06:58:51 +0100
> > Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:
> > > On Wed, 5 Sep 2012 18:15:43 +0200
> > > Michał Górny <mgorny@gentoo.org> wrote:
> > > > If we really want to go this route, then please at least require
> > > > explicit label at start of DEPENDENCIES. And the same when
> > > > appending to DEPENDENCIES -- just so 'unlikely' mistakes will
> > > > leave us with hours of debugging.
> > >
> > > We should take the exheres-0 rules for labels and eclasses, which
> > > limit labels' scopes to blocks, and which introduce an extra ( )
> > > block around the outside when doing eclass variable merging.
> >
> > Because? I believe we should take 'Gentoo rules', including required
> > explicit build+run at the start.
>
> You mean, you want to invent some new rules that don't quite work,
> rather than using the ones that do... The whole "initial labels" thing
> isn't an issue for exheres-0, since rather than appending, the
> resulting metadata variable ends up with extra ( ) blocks like this:
>
> (
> stuff/from-the-exheres
> )
> (
> stuff/from-exlib-1
> )
> (
> stuff/from-exlib-2
> )
>
> so there's no possibility of labels ending up applied to the wrong
> thing.
>
> If you just append, you'd have to have some way of validating that
> eclasses all individually specify an initial label. That's not
> something that can easily be done.

In that format there is not a single thing which *can be done easily*.

But what I was saying is that I dislike the implicit 'no label ==
build+run'. It's unclear, very unclear.

Why the heck:

( foo/bar )

introduces another label than:

use? ( foo/bar )

?

> > > > Remember that this requirement will actually cause migration to
> > > > EAPI 5 to be even harder than to any previous EAPIs. Migrating a
> > > > single ebuild will require rewriting the dependencies, and
> > > > migrating an eclass will require adding a lot of dirty code.
> > >
> > > Migrating to EAPI 5 requires rewriting dependencies anyway if
> > > we're adding in HDEPEND. Also, earlier EAPIs have introduced new
> > > phase functions, which is a far ickier change for ebuilds than
> > > this.
> >
> > Do you really believe in HDEPEND in EAPI 5? I've already postponed
> > this in my mind. Also, not every single ebuild will actually need
> > it.
>
> *shrug* if all the new *DEPEND stuff ends up in EAPI 6, then there's
> no point in DEPENDENCIES for EAPI 5. But we'll have to sort this out
> sooner or later...

Yes, there's more time for a meaningful discussion without switching
everything upside-down just because you did it.

--
Best regards,
Michał Górny
 
Old 09-06-2012, 08:31 AM
Ciaran McCreesh
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Thu, 6 Sep 2012 10:27:55 +0200
Michał Górny <mgorny@gentoo.org> wrote:
> But what I was saying is that I dislike the implicit 'no label ==
> build+run'. It's unclear, very unclear.
>
> Why the heck:
>
> ( foo/bar )
>
> introduces another label than:
>
> use? ( foo/bar )
>
> ?

Labels are propagated into child blocks, so it doesn't introduce a new
label. I think you've misunderstood something here...

--
Ciaran McCreesh
 
Old 09-06-2012, 08:42 AM
Michał Górny
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Thu, 6 Sep 2012 09:31:29 +0100
Ciaran McCreesh <ciaran.mccreesh@googlemail.com> wrote:

> On Thu, 6 Sep 2012 10:27:55 +0200
> Michał Górny <mgorny@gentoo.org> wrote:
> > But what I was saying is that I dislike the implicit 'no label ==
> > build+run'. It's unclear, very unclear.
> >
> > Why the heck:
> >
> > ( foo/bar )
> >
> > introduces another label than:
> >
> > use? ( foo/bar )
> >
> > ?
>
> Labels are propagated into child blocks, so it doesn't introduce a new
> label. I think you've misunderstood something here...

Right, too many ()s for me.

Then my argument that I want us to require explicit label at start of
global () still holds.

--
Best regards,
Michał Górny
 
Old 09-11-2012, 02:13 PM
Michał Górny
 
Default HDEPEND (host dependencies for cross-compilation) for EAPI 5?

On Thu, 6 Sep 2012 01:11:45 -0700
Brian Harring <ferringb@gmail.com> wrote:

> A compatibility hack that stacks them is strongly advisable;
> something akin to the following:
>
> Literally, we do the following:
> inherit() {
> if eapi blah; then
> local DEPEND PDEPEND RDEPEND
> <usual saving/protection of DEPENDENCIES var>
> else
> <usual saving/protection of DEPEND/PDEPEND/RDEPEND vars>
> fi
> <normal sourcing machinery>
> if eapi blah; then
> local _deps=( ) _x
> for _x in DEPENDENCIES DEPEND RDEPEND PDEPEND; do
> [ -n "${!_x}" ] && deps+=( "${!_x}" )
> done
> [ ${#deps} -ne 0 ] && DEPENDENCIES="${deps[*]}"
> unset DEPEND RDEPEND PDEPEND _x _deps
> <normal stacking/restoration of DEPENDENCIES rules>
> else
> <normal stacking/restoration of RDEPEND/PDEPEND/DEPEND>
> fi
> }
>
> Via that, we eclasses that are pure DEPENDENCIES eapi wise, just set
> the DEPENDENCIES directly; those that have to support multiple eapi's
> (aka, every fricking eclass that exists right now) can just use the
> old form, shifting into the new form as things progress.

If we decide to go with a such a hack, then we either have to support
it indefinitely, or to decide to drop the support in some further EAPI.

If we go for the latter, then it's just delaying the ugly conditional
eclasses will have to suffer at some random point in the future. Well,
maybe two eclasses less if we wait with it for an EAPI which will
provide 'killer features' which will render the eclasses unusable with
older EAPIs. And way, it will be a bit confusing to remember two switch
points...

If we go for the former... then some developers will ask: why eclasses
and not ebuilds? Why?

--
Best regards,
Michał Górny
 

Thread Tools




All times are GMT. The time now is 11:37 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org