FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 03-31-2010, 09:20 AM
Brian Harring
 
Default pkg_pretend USE validation and VALID_USE alternative

Hola all-

For those who aren't familiar, pkg_pretend is in EAPI4- the main usage
of it is will be use dep checking- this email is specifically
regarding an alternative to it that *should* be superior for that use
case, but I'm looking for feedback.

Basically, we use the original VALID_USE proposal from way back in
'05- if you're familiar w/ MYOPTIONS, they're reasonably similar.

Roughly, VALID_USE is a list of constraints stating what the allowed
use flag combinations are for this pkg. If you think of normal
depdencies (I must have openssl and python merged prior), it's the
same machinery.

Examples:

# if build is set, python and openssl must be unset
VALID_USE="build? ( !python !openssl )"

# if mysql is set, sqlite must not be, and vice versa.
# note mysql/sqlite do *not* have to be set also.
VALID_USE="mysql? ( !sqlite ) !sqlite? ( mysql )"

# mysql or sqlite must be set; exclusive or.
VALID_USE="mysql? ( !sqlite ) !mysql ( sqlite )"

# alternative syntax, adding an xor group operator
# note xor isn't required- you can do the same thing
# via spelling it out, it's just a convenient thing to have.
VALID_USE="^^ ( mysql sqlite )"

# if gui is enabled, a widget set must be specified- can build
# multiple widget bindings
VALID_USE="X? ( || ( gtk qt motif ) )"


Note that like dependencies, these are assertions. More importantly,
they're also data rather then executable code. Via it being data, up
front GUI's can tell you exactly what conflicts arise, and what needs
to be adjusted.

Doing it as executable, can, but it's iterative and reliant on the dev
writing a clear message every time (rather then the UI tool getting
it right once). Clarifying iterative, consider

USE="build X"
w/ valid use states being-
VALID_USE="build? ( !X !python ) X? ( ^^ ( gtk qt ) ) gtk? ( ssl )"

The user first flips off build. They rerun emerge- next they're told
"you must choose gtk or qt, not both". They change to USE="X gtk".
Next they're told "you need ssl turned on if you want gtk".

At this point, the user goes and gets a beer because aparently Murphy
hates them and it's going to be a long, long night.


There also is one major issue with relying on pkg_pretend
(executable) for use state validation vs doing it as VALID_USE
(data)- the package manager cannot know what states are valid thus
limiting the things it can do. Literally the pkg manager is screwed
when it comes to use cycle breaking. If the pkg manager doesn't know
what states are valid, when it encounters a use cycle it doesn't know
what intermediate builds it can do to break that cycle- literally if
USE state validation is in pkg_pretend, the *user* will have to walk
the PM through breaking the cycle instead of the PM figuring out the
proper steps to break it.

Executive summary: if use validation is implemented via pkg_pretend,
1) it still has the iterative issue
2) it's harder for configuration tools to deal with (iterative issues
above, plus the fact they get a blob of text they're stuck trying to
parse)
3) it pretty much eliminates the possibility of doing use cycle
breaking properly (which is already an issue, only going to get worse-
hence why this was proposed in '04/'05 when we started playing w/ use
deps in portage 3 at the time).


Comments desired; assuming no significant blowback, I'll be pushing
this to the council level since eapi4 is annoying feature locked right
now.

Thanks-
~harring
 
Old 03-31-2010, 09:48 AM
Ulrich Mueller
 
Default pkg_pretend USE validation and VALID_USE alternative

>>>>> On Wed, 31 Mar 2010, Brian Harring wrote:

> Roughly, VALID_USE is a list of constraints stating what the allowed
> use flag combinations are for this pkg. If you think of normal
> depdencies (I must have openssl and python merged prior), it's the
> same machinery.

Maybe we should first discuss if we want to drop the following
rule [1] which your proposal seems to contradict:

| Occasionally, ebuilds will have conflicting USE flags for
| functionality. Checking for them and returning an error is not a
| viable solution. Instead, you must pick one of the USE flags in
| conflict to favour.

Ulrich

[1] <http://devmanual.gentoo.org/general-concepts/use-flags/>
 
Old 03-31-2010, 10:46 AM
Brian Harring
 
Default pkg_pretend USE validation and VALID_USE alternative

Note that while I inadvertantly cross posted (I was intending on
cc'ing council@gentoo.org, not the ml), doubt they need to be cc'd
further- my original attention was to effectively ensure they were
paying aware of the details of this so that when I took it to them
folk were informed.

CC'ing gentoo-council so folk following it there know it moved
over to -dev. Your discussion of devmanual relevance needs some -dev
consensus anyways before the council should be deciding on it.

Also the cross posting is making betelgeuse cry anyways (and pissing
off my procmail setup)


On Wed, Mar 31, 2010 at 11:48:37AM +0200, Ulrich Mueller wrote:
> >>>>> On Wed, 31 Mar 2010, Brian Harring wrote:
>
> > Roughly, VALID_USE is a list of constraints stating what the allowed
> > use flag combinations are for this pkg. If you think of normal
> > depdencies (I must have openssl and python merged prior), it's the
> > same machinery.
>
> Maybe we should first discuss if we want to drop the following
> rule [1] which your proposal seems to contradict:

Not just my proposal- council contradicted it via even letting
pkg_pretend into EAPI3 (now EAPI4):

http://www.mail-archive.com/gentoo-council@lists.gentoo.org/msg00493.html


> | Occasionally, ebuilds will have conflicting USE flags for
> | functionality. Checking for them and returning an error is not a
> | viable solution. Instead, you must pick one of the USE flags in
> | conflict to favour.
>
> [1] <http://devmanual.gentoo.org/general-concepts/use-flags/>

I honestly consider the ebuild silently making decisions on the user's
behalf *worse*. Consider if openoffice silently made decisions like
that- 4 hours later it'll wind up choosing the option you didn't
really want and you'll be in a foul mood.

Frankly is the devmanual even relevant on at this point beyond good
practices btw? Last I looked through it, there was a rather unhealthy
mix of good policy that we follow, and policy that isn't relevant
anymore- in need of some cleanup at the very least.


~harring
 
Old 03-31-2010, 10:57 AM
Brian Harring
 
Default pkg_pretend USE validation and VALID_USE alternative

Note I inadvertantly cross posted, I was intending on cc'ing
council@gentoo.org.

As such one final cc to that ml to end this subthread while pulling
this back to -dev.


On Wed, Mar 31, 2010 at 11:16:22PM +1300, Alistair Bush wrote:
> > Hola all-
> >
> > Comments desired; assuming no significant blowback, I'll be pushing
> > this to the council level since eapi4 is annoying feature locked right
>
> I think this solution is far better, until someone smarter than me tells me
> otherwise.
>
> Don't know whether I like the VALID_USE var name, so please at least think
> about something a little better if you can . ( I like green, but not too
> green if you know what I mean )
>
> Will we still have to define the use flags in IUSE?

Yes, although if folks have a better proposal that incorporates
VALID_USE into IUSE I'm definitely open to it- the original proposal
for VALID_USE tried to inline it into IUSE, but it got ugly (hence it
mutating it this form).

The problem w/ trying to reuse IUSE is the following (sorry, I like
lists of assertions)-

*) IUSE currently serves as a list of valid USE flags, just that. No
repeat specification of a flag (which means the dev in question is
unlikely to typo a flag).

*) having a single specified list of valid use flags is the basis for
doing validation of use flags used in all other metadata. In other
words, that list of valid use flags *really* needs to go out of it's
way to make human error hard.

*) VALID_USE is a set of assertions on the allowed state of USE; IUSE
is just a list of flags. Intermixing the two in a way that is still
readable is really ugly

*) given a library that has optional perl and python bindings
(which can be toggled freely, they're standalone flags) I've nfc how
one would sanely specify that w/in IUSE while adding VALID_USE
semantics. Possibly, you could include use conditionals into IUSE,
and treat the () contents as assertions- but that makes adding xor in
hard, is rather ugly/hard on the eyes, and violates the DRY (Don't
Repeat Yourself) principle from above.

Definitely open to counterproposals that address those
concerns however...


> I'm guessing we can't use IUSE to store this information because of the whole
> glep-55 thing.

glep-55 is unrelated to this as far as I can tell- if you think
otherwise please clarify.

Thanks
~harring
 
Old 03-31-2010, 11:04 AM
Ulrich Mueller
 
Default pkg_pretend USE validation and VALID_USE alternative

>>>>> On Wed, 31 Mar 2010, Brian Harring wrote:

> Not just my proposal- council contradicted it via even letting
> pkg_pretend into EAPI3 (now EAPI4):

> http://www.mail-archive.com/gentoo-council@lists.gentoo.org/msg00493.html

It says "displaying conflicting USE flags" which doesn't necessarily
imply that the ebuild should fail.

> I honestly consider the ebuild silently making decisions on the
> user's behalf *worse*.

Right, this is exactly what we should decide on, before talking about
possible implementations.

We already have enough issues with circular dependencies, and I'm
sceptical about adding additional failures on USE flag conflicts.
Display a warning, but don't error out.

Ulrich
 
Old 03-31-2010, 11:11 AM
Brian Harring
 
Default pkg_pretend USE validation and VALID_USE alternative

On Wed, Mar 31, 2010 at 01:04:39PM +0200, Ulrich Mueller wrote:
> We already have enough issues with circular dependencies, and I'm
> sceptical about adding additional failures on USE flag conflicts.
> Display a warning, but don't error out.

Solve the cyclical dependency via breaking the use cycle and doing two
builds of the target instead.

It *does* work. Now I'm reasonably sure I've since *broken* that
support in pkgcore since stubbs/myself worked it out (resolver still
supports it though), but that doesn't mean restoring it is hard.

Frankly I'd rather see us support these things then have the PM bitch
at the user or make guesses on the users behalf.

My 2 cents either way- mainly responding since you seem to be ignoring
that as long as pkg_pretend isn't fricking used, we've got a way to
break use induced cyclical deps automatically via the PM.

~harring
 
Old 03-31-2010, 03:38 PM
"Paweł Hajdan, Jr."
 
Default pkg_pretend USE validation and VALID_USE alternative

On 3/31/10 1:04 PM, Ulrich Mueller wrote:
> We already have enough issues with circular dependencies, and I'm
> sceptical about adding additional failures on USE flag conflicts.
> Display a warning, but don't error out.

How about only allowing local USE flags to conflict? This also seems to
be the most common use case.

Anyway, the earlier the user can react to USE flags conflict, the better.

Paweł Hajdan jr
 
Old 03-31-2010, 05:49 PM
Alex Alexander
 
Default pkg_pretend USE validation and VALID_USE alternative

On Wed, Mar 31, 2010 at 02:20:35AM -0700, Brian Harring wrote:
> Hola all-
>
> For those who aren't familiar, pkg_pretend is in EAPI4- the main usage
> of it is will be use dep checking- this email is specifically
> regarding an alternative to it that *should* be superior for that use
> case, but I'm looking for feedback.
>
> Basically, we use the original VALID_USE proposal from way back in
> '05- if you're familiar w/ MYOPTIONS, they're reasonably similar.
>
> Roughly, VALID_USE is a list of constraints stating what the allowed
> use flag combinations are for this pkg. If you think of normal
> depdencies (I must have openssl and python merged prior), it's the
> same machinery.
>
> [snip]
>
> Comments desired; assuming no significant blowback, I'll be pushing
> this to the council level since eapi4 is annoying feature locked right
> now.

I like the whole concept, it is clean and straightforward.

Also, notifying the user of any possible issues and breaking so he can
make the actual decision sounds far better than making the choice for him.

VALID_USE does look a bit strange.

how about
IUSE_RULES
or
IUSE_RESTRICTIOMS
or
RUSE
?

--
Alex Alexander :: wired
Gentoo Developer
www.linuxized.com
 
Old 03-31-2010, 07:46 PM
Brian Harring
 
Default pkg_pretend USE validation and VALID_USE alternative

On Wed, Mar 31, 2010 at 08:49:26PM +0300, Alex Alexander wrote:
> VALID_USE does look a bit strange.
>
> how about
> IUSE_RULES
> or
> IUSE_RESTRICTIOMS
> or
> RUSE
> ?
It's not really IUSE; the constraints it specifies applies to USE
only.

USE_STATES, VALID_USES, VALID_USE_STATES, ALLOWED_USE, etc.

Actual name I don't hugely care about, I'm more interested in ensuring
we don't rule out doing use cycle breaking via a bad design decision.

~harring
 
Old 03-31-2010, 07:56 PM
Ciaran McCreesh
 
Default pkg_pretend USE validation and VALID_USE alternative

On Wed, 31 Mar 2010 12:46:26 -0700
Brian Harring <ferringb@gmail.com> wrote:
> Actual name I don't hugely care about, I'm more interested in
> ensuring we don't rule out doing use cycle breaking via a bad design
> decision.

Cycle breaking requires explicit instructions from the ebuilds in
question (many of which are system things, which further complicates it)
along with support from Portage, so it's a distant future, lot of work
thing.

Since we need pkg_pretend to cover all the things that aren't use flag
related anyway, it makes sense to just go with that rather than
delaying things even further. When in the distant future Portage
becomes able to deal with cycle breaking, ebuilds can be converted to
use something like VALID_USE when they're also updated to export
information on which of their flags can safely be toggled.

--
Ciaran McCreesh
 

Thread Tools




All times are GMT. The time now is 05:15 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org