FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 02-27-2009, 01:41 AM
Mart Raudsepp
 
Default Repository stacking and complementary overlays

A. On K, 2009-02-25 at 04:56 -0800, Brian Harring wrote:
> On Wed, Feb 25, 2009 at 01:42:38PM +0100, Gilles Dartiguelongue wrote:
> > Le mardi 24 février 2009 * 09:47 -0800, Brian Harring a écrit :
> > > On Sun, Feb 22, 2009 at 11:26:48PM -0800, Donnie Berkholz wrote:
> > > > This is your friendly reminder! Same bat time (typically the 2nd & 4th
> > > > Thursdays at 2000 UTC / 1600 EST), same bat channel (#gentoo-council @
> > > > irc.freenode.net) !
> > >
> > > Informal request, but it would be useful to get an idea of the
> > > councils views on portage overlay compatibility issues.
> > >
> > > Specifically, when it comes to gentoo repositories, there is one, and
> > > only one definition of what that is- pms's repo spec. The problem
> > > here is that the only repository truly conformant to that spec is
> > > gentoo-x86, for the rest of the repositories (overlays realistically)
> > > whatever portage supports seems to be the eventual standard they grow
> > > towards.
> > >
> > > Problem with this is that there is *zero* way to spot these non-pms
> > > repositories as it stands. Simplest example, under portage overlays
> > > can unmask pkgs globally (gnome overlay reverting masks in
> > > gentoo-x86),
> >
> > I reply here as part of the gnome herd and partly responsible for the
> > mask reverting in the overlay. I didn't know something used in
> > gentoo-x86 couldn't be used in an overlay.
>
> Suspect I wasn't clear; you *can* use things from the parent (although
> that whole relationship is outside of PMS); the problem here is that
> y'all are reverting something in the *master*.
>
> Literally, bug-buddy was masked in gentoo-x86; enabling your overlay
> reverts that masking in *gentoo-x86*. Only reason this even works is
> due to portage internals being limited (everything is stacked
> together, no true standalones possible).

So here the reverting of a masking in gentoo-x86 is quite intentional
and currently desired. The problem is that
a) most importantly - no way to explicitly express a complementary
overlay to a given repository
b) less importantly - gentoo-x86 has no true base profile where to mask
things, as base/ isn't really the base of everything, and the global
package.mask is for some reason defined as something different from a
really base profile (global mask negation disallowed even in arch
profiles of the same repository, etc)

This mail thread is about a)

> > Could you point me to the PMS section that treat this ?
> Flip through the package.mask section (snagging from profiles.tex
> directly)-
>
> """
> Note that the {-spec} syntax can be used to remove a mask in a
> parent profile, but not necessarily a global mask (from
> {profiles/package.mask}, section~
ef{profiles-package.mask}).
>
>
ote Portage currently treats {profiles/package.mask} as being on
> the leftmost branch of the inherit tree when it comes to {-lines}.
> This behaviour may not be relied upon.
> """

By this snippet we could simply move the current relevant maskings from
profiles/package.mask to profiles/base/package.mask and call it a day
(and screw over the few profiles that don't end up parenting base/), as
QA forced us to do in case of per-arch mask negations in gentoo-x86 a
while back.
But it doesn't seem to be as simple as that.

The wording "but not _necessarily_ a global mask" is quite inconclusive
as well.

> Note the 'parent profile'. Why they're claiming repo level masking
> can't be reversed for that repo, not sure (reasonably sure several
> profiles rely on it). Either way, your overlay is trying to revert
> entries it doesn't have in that stack.

We'd like the stack to be the same as gentoo-x86. It is a complementary
overlay to gentoo-x86 and definitely not a separate stand-alone
repository.

> Only reason it flies for portage is because it collapses it all into
> one stack; for managers designed to support multiple standalone repos
> that assumption no longer applies, thus that behaviour (outside of
> PMS) breaks.

Last I knew the official council approved PMS was meant to describe
portage behaviour at the time, which appears to have been the same along
the way - treating all overlays in the same "stack" as PORTDIR, perhaps
as there is no means to declare a different "stack".
There are claims that this behaviour (everything in the same stack) is
simply a bug - I dare to disagree and claim instead that this behaviour
is simply about defaulting PORTDIR_OVERLAYs to be complementary to
PORTDIR (as the relation in the variable names quite clearly and
logically states!) and therefore in the same "stack" and that we need a
different way to declare a different stack and standalone repository.
I also claim other PMs than portage should default to extending the
PORTDIR repo_name "stack" as well with overlay entries in
PORTDIR_OVERLAY instead of making a new "stack" as they do now. An
PORTDIR_OVERLAY overlay is an overlay on top of PORTDIR and therefore in
the same "stack", not a new repository that gets a new "stack".



With a very quick thought I see some possible ways to make this more
clear and formal:

a) Declare in a reposity/overlay what its parent repository is, if any -
gnome overlay profiles/parent_repo (or some such) would contain
"gentoo". Could also perhaps extend repo_name, e.g "gentoo+gnome"
b) Declaring what "stack" a repository/overlay belongs to - e.g, both
gentoo main repository and gnome overlay (and quite all other existing
_overlays_) both saying "gentoo" in their profiles/stack_name (with a
better name) file.
c) Maintain the behaviour of portage as is, and assume PORTDIR_OVERLAY
is _overlaying_ PORTDIR as the name suggests, but have some kind of
sanity checks that can be performed where appropriate either by means of
options a) or b) or some different way
d) Make use of sets for solving the current use case in question, as has
been suggested - this seems to involve portage work and actually
understanding why this is a good solution -- concrete explanation on
that is most welcome for consideration and evaluation.

Keep in mind that this (especially options presented as a), b) and c))
covers more than just package.mask negation, as the triggering reason
for this discussion. Knowing that an overlay is complementary to a given
repository or repositories, and knowing that a repository is standalone
from gentoo-x86 could be beneficial in more cases than this specific
one.


For package.mask negation as done in GNOME overlay it's also about
profiles/package.mask vs profiles/base/package.mask behaviour (in
addition to negation of masks in a different repository/overlay) - the
discussion of that is not a subject to this thread, so please make a new
thread when wanting to discuss about that.


Note that I'm on devaway for up to a week from this moment, just getting
an actual discussion going on this topic meanwhile as promised at
council meeting.


--
Mart Raudsepp
Gentoo Developer
Mail: leio@gentoo.org
Weblog: http://planet.gentoo.org/developers/leio
 
Old 03-02-2009, 03:48 PM
Ciaran McCreesh
 
Default Repository stacking and complementary overlays

On Fri, 27 Feb 2009 04:41:23 +0200
Mart Raudsepp <leio@gentoo.org> wrote:
> So here the reverting of a masking in gentoo-x86 is quite intentional
> and currently desired.

This is fundamentally broken as a concept.

Adding an overlay should not have any impact upon other repositories.
It should be possible for a user to add an overlay, and make limited
use of that repository, without having to worry that the mere act of
adding that overlay will make massive changes to what's visible in
other repositories.

Overlays shouldn't be altering the visibility of things outside of that
overlay without explicit user action.

> By this snippet we could simply move the current relevant maskings
> from profiles/package.mask to profiles/base/package.mask and call it
> a day (and screw over the few profiles that don't end up parenting
> base/), as QA forced us to do in case of per-arch mask negations in
> gentoo-x86 a while back.
> But it doesn't seem to be as simple as that.

Well no, because profiles/base/ in your overlay is entirely unrelated
to profiles/base/ in the master.

> > Only reason it flies for portage is because it collapses it all
> > into one stack; for managers designed to support multiple
> > standalone repos that assumption no longer applies, thus that
> > behaviour (outside of PMS) breaks.
>
> Last I knew the official council approved PMS was meant to describe
> portage behaviour at the time, which appears to have been the same
> along the way - treating all overlays in the same "stack" as PORTDIR,
> perhaps as there is no means to declare a different "stack".

PMS does not attempt to document Portage behaviour in the cases where
Portage behaviour is dumb. That's the reason there's as little as
possible mentioned regarding overlays there -- Portage's overlay model
is a horrible hack, and forcing package managers to implement it rather
than offering a true multiple repository model would be a serious hit
on usability.

The way forward here is to identify what you're trying to achieve,
whilst ignoring how things are currently defined or what is or is not
possible. Then we can look at that and work out whether it can be
mapped to an existing solution or some easily-implementable new
solution. Starting with implementation is the wrong approach.

--
Ciaran McCreesh
 
Old 03-02-2009, 10:55 PM
Gilles Dartiguelongue
 
Default Repository stacking and complementary overlays

Le lundi 02 mars 2009 16:48 +0000, Ciaran McCreesh a crit :
> On Fri, 27 Feb 2009 04:41:23 +0200
> Mart Raudsepp <leio@gentoo.org> wrote:
> > So here the reverting of a masking in gentoo-x86 is quite
> intentional and currently desired.
>
> This is fundamentally broken as a concept.
>
> Adding an overlay should not have any impact upon other repositories.
> It should be possible for a user to add an overlay, and make limited
> use of that repository, without having to worry that the mere act of
> adding that overlay will make massive changes to what's visible in
> other repositories.
>
> Overlays shouldn't be altering the visibility of things outside of
> that overlay without explicit user action.

well that's unfortunate that it doesn't fit that view but that's still
what was desired. See next block for details.

> The way forward here is to identify what you're trying to achieve,
> whilst ignoring how things are currently defined or what is or is not
> possible. Then we can look at that and work out whether it can be
> mapped to an existing solution or some easily-implementable new
> solution. Starting with implementation is the wrong approach.

We didn't implement anything but let's just talk about what we wanted to
see. We simply wanted overlay users to keep testing gnome 2.24
components that were masked or using masked packages in
base/package.mask so we just made sure those packages had the proper
keyword visibility.

--
Gilles Dartiguelongue <eva@gentoo.org>
Gentoo
 
Old 03-02-2009, 10:59 PM
Ciaran McCreesh
 
Default Repository stacking and complementary overlays

On Tue, 03 Mar 2009 00:55:38 +0100
Gilles Dartiguelongue <eva@gentoo.org> wrote:
> We didn't implement anything but let's just talk about what we wanted
> to see. We simply wanted overlay users to keep testing gnome 2.24
> components that were masked or using masked packages in
> base/package.mask so we just made sure those packages had the proper
> keyword visibility.

So if you could mask 'testing'ish things that're in the overlay
(already possible), and provide an easy, consistent way for users who
choose to to unmask and keyword a particular group of packages, would
that solve the problem?

--
Ciaran McCreesh
 
Old 03-04-2009, 08:17 AM
Gilles Dartiguelongue
 
Default Repository stacking and complementary overlays

Le lundi 02 mars 2009 23:59 +0000, Ciaran McCreesh a crit :
> On Tue, 03 Mar 2009 00:55:38 +0100
> Gilles Dartiguelongue <eva@gentoo.org> wrote:
> > We didn't implement anything but let's just talk about what we wanted
> > to see. We simply wanted overlay users to keep testing gnome 2.24
> > components that were masked or using masked packages in
> > base/package.mask so we just made sure those packages had the proper
> > keyword visibility.
>
> So if you could mask 'testing'ish things that're in the overlay
> (already possible),

well there was nothing masked in the overlay really, for 2.24, because
the overlay is for testing stuff already. Let me try to rephrase and see
if I can make it clearer. Although we moved some packages to the tree,
some packages didn't ended up moving because they needed more testing or
huge regression fixing patches but they were masked by gentoo-x86 masks
because of their dependencies. Hence the mask reverting in overlay.

> and provide an easy, consistent way for users who
> choose to to unmask and keyword a particular group of packages, would
> that solve the problem?

wrt to previous point, it would probably help a great deal even though
I'm not sure it would completely solve this case.

--
Gilles Dartiguelongue <eva@gentoo.org>
Gentoo
 
Old 03-05-2009, 12:37 AM
"Jorge Manuel B. S. Vicetto"
 
Default Repository stacking and complementary overlays

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ciaran McCreesh wrote:
> On Fri, 27 Feb 2009 04:41:23 +0200
> Mart Raudsepp <leio@gentoo.org> wrote:
>> So here the reverting of a masking in gentoo-x86 is quite intentional
>> and currently desired.
>
> This is fundamentally broken as a concept.
>
> Adding an overlay should not have any impact upon other repositories.
> It should be possible for a user to add an overlay, and make limited
> use of that repository, without having to worry that the mere act of
> adding that overlay will make massive changes to what's visible in
> other repositories.

This seems desirable and reasonable.
As I replied to this subject earlier regarding KDE, let me complement
that information. In the case of the KDE team, we keep work on a release
all in the same place, so we don't need to unmask some KDE packages in
tree for those using the overlay. However, we some times have deps on
packages from other teams and or in other overlays, so I hope the repo
deps would help here (not to unmask those packages, if they're masked,
but to add a dep on a particular repo and allowing the PM explain to the
user that he/she needs to unmask a particular version in the tree /
overlay).

- --
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / SPARC / KDE
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkmvLOgACgkQcAWygvVEyAJkCQCfVjp/ADyOyCs+HPpV9cVrUAs6
2XkAnj7n2jAcocZp/9Qc/RtxIXmCAiHQ
=2K3f
-----END PGP SIGNATURE-----
 
Old 03-05-2009, 12:58 AM
Zac Medico
 
Default Repository stacking and complementary overlays

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Jorge Manuel B. S. Vicetto wrote:
> This seems desirable and reasonable.
> As I replied to this subject earlier regarding KDE, let me complement
> that information. In the case of the KDE team, we keep work on a release
> all in the same place, so we don't need to unmask some KDE packages in
> tree for those using the overlay. However, we some times have deps on
> packages from other teams and or in other overlays, so I hope the repo
> deps would help here (not to unmask those packages, if they're masked,
> but to add a dep on a particular repo and allowing the PM explain to the
> user that he/she needs to unmask a particular version in the tree /
> overlay).

The problem with repo deps is that they're too restrictive since
they assume that only a specific repo can satisfy the dep. Suppose
that you migrate some of the packages from the overlay to the main
tree? Now you've got installed packages that are trying to pull in
deps from the wrong repo. Or suppose that somebody else has an
overlay with a compatible package?

I think a better way to reference another repo is with the
layout.conf approach suggested in the "QA Overlay Layout support"
thread [1]. For example, if packages from the java-experimental repo
depend on some ebuilds or eclasses from the java-overlay repo, it's
specified via a "masters" entry in layout.conf. If any of those
ebuilds/eclasses happen to migrate to the main tree then the
migration is seamless.

[1]
http://archives.gentoo.org/gentoo-dev/msg_33c61550b4ed2b7b25dd5a4110e1ec81.xml
- --
Thanks,
Zac
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAkmvMcIACgkQ/ejvha5XGaNmkwCePw/XjWCqZtIXq5yXQ4gpHALL
fXUAoMqkmJ30Go2SJaqS2lzs+8axyLwn
=ju66
-----END PGP SIGNATURE-----
 
Old 03-05-2009, 01:16 AM
"Jorge Manuel B. S. Vicetto"
 
Default Repository stacking and complementary overlays

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Zac Medico wrote:
> Jorge Manuel B. S. Vicetto wrote:
>> This seems desirable and reasonable.
>> As I replied to this subject earlier regarding KDE, let me complement
>> that information. In the case of the KDE team, we keep work on a release
>> all in the same place, so we don't need to unmask some KDE packages in
>> tree for those using the overlay. However, we some times have deps on
>> packages from other teams and or in other overlays, so I hope the repo
>> deps would help here (not to unmask those packages, if they're masked,
>> but to add a dep on a particular repo and allowing the PM explain to the
>> user that he/she needs to unmask a particular version in the tree /
>> overlay).
>
> The problem with repo deps is that they're too restrictive since
> they assume that only a specific repo can satisfy the dep. Suppose
> that you migrate some of the packages from the overlay to the main
> tree? Now you've got installed packages that are trying to pull in
> deps from the wrong repo. Or suppose that somebody else has an
> overlay with a compatible package?

I agree repo deps might be restrictive, but in some cases we might
really want to be restrictive. For instance, I might want to have a dep
on a package in the GNOME official overlay - I might not want to get the
version in some other overlay listed in layman.
When the package moves to the tree or if work moves to another overlay,
then sure, it will mean more work. But sometimes that might be preferred.

> I think a better way to reference another repo is with the
> layout.conf approach suggested in the "QA Overlay Layout support"
> thread [1]. For example, if packages from the java-experimental repo
> depend on some ebuilds or eclasses from the java-overlay repo, it's
> specified via a "masters" entry in layout.conf. If any of those
> ebuilds/eclasses happen to migrate to the main tree then the
> migration is seamless.

I like this proposal as well. I think repo deps and "master layouts" are
complementary and not alternatives. Getting back to KDE (sorry for being
so self-centered), we would make the kde-experimental overlay have
kde-testing as the master overlay. So we could do some work in eclasses
in kde-testing without needing to copy them to kde-experimental.

> [1]
> http://archives.gentoo.org/gentoo-dev/msg_33c61550b4ed2b7b25dd5a4110e1ec81.xml

- --
Regards,

Jorge Vicetto (jmbsvicetto) - jmbsvicetto at gentoo dot org
Gentoo- forums / Userrel / Devrel / SPARC / KDE
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkmvNhUACgkQcAWygvVEyAJTDwCfeFwqVMtUY4 r+BR5mX6kyGhuk
tTAAoJG0kQksJnVM6Fd/WwPHxJmBJZoi
=eSaw
-----END PGP SIGNATURE-----
 
Old 03-05-2009, 09:20 AM
"Marijn Schouten (hkBst)"
 
Default Repository stacking and complementary overlays

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

The problem of ebuilds in one overlay not seeing ebuilds in another overlay,
would also be solved by the package manager NOT failing to see/notice/use/allow
ebuilds from all installed overlays. Then there would be no need for a hierarchy
among overlays.

Marijn

- --
Sarcasm puts the iron in irony, cynicism the steel.

Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
<http://www.gentoo.org/proj/en/lisp/>, #gentoo-{lisp,ml} on FreeNode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkmvp1IACgkQp/VmCx0OL2wFqACfSTjfdtY7abG/yc06gW4Q+YlT
Nd0AoIYIwFBD4BEKoA/PVg35K4Sia+C4
=IWat
-----END PGP SIGNATURE-----
 
Old 03-05-2009, 11:24 AM
"Marijn Schouten (hkBst)"
 
Default Repository stacking and complementary overlays

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Marijn Schouten (hkBst) wrote:
> The problem of ebuilds in one overlay not seeing ebuilds in another overlay,
> would also be solved by the package manager NOT failing to see/notice/use/allow
> ebuilds from all installed overlays. Then there would be no need for a hierarchy
> among overlays.
>
> Marijn

It was pointed out to me by zlin that this does not help with installing
`missing' `master' overlays. This is a good point. However it is possible that
even though an overlay is missing, ebuilds from another installed overlay or
main tree or a local overlay can satisfy dependencies. Further it is possible
that two overlays have a mutual dependency on eachother.

Maybe these possibilities are only theoretical.

Marijn

- --
Sarcasm puts the iron in irony, cynicism the steel.

Marijn Schouten (hkBst), Gentoo Lisp project, Gentoo ML
<http://www.gentoo.org/proj/en/lisp/>, #gentoo-{lisp,ml} on FreeNode
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkmvxGMACgkQp/VmCx0OL2x2+ACgh6ESoDDWhW1DCFnc2bz1VItw
xFIAoLuSQt1G42MR5umWrCoZwGD3thbG
=XnsF
-----END PGP SIGNATURE-----
 

Thread Tools




All times are GMT. The time now is 07:03 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org