FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 02-07-2011, 03:43 PM
Samuli Suominen
 
Default avoiding urgent stabilizations

On 02/07/2011 06:19 PM, "Paweł Hajdan, Jr." wrote:
> From time to time there are stabilization bugs where the current stable
> is broken. For example, https://bugs.gentoo.org/show_bug.cgi?id=353487
>
> However, in theory that should not happen, because presumably the
> current stable has been tested in the past and considered not broken.
>

In this case, xdm has been broken for more than a year because it has
been ignoring pambase by not using system-local-login.

Only the problem came to light after ConsoleKit really started to
require this functionality from pambase.

Tricky...

But it was still tested, unfortunately the test environment that was
used leaked some environment variables that made XDM appear to work, so
this was catched too late.

Sorry.

- Samuli
 
Old 02-07-2011, 04:35 PM
Pacho Ramos
 
Default avoiding urgent stabilizations

El lun, 07-02-2011 a las 18:43 +0200, Samuli Suominen escribió:
> On 02/07/2011 06:19 PM, "Paweł Hajdan, Jr." wrote:
> > From time to time there are stabilization bugs where the current stable
> > is broken. For example, https://bugs.gentoo.org/show_bug.cgi?id=353487
> >
> > However, in theory that should not happen, because presumably the
> > current stable has been tested in the past and considered not broken.
> >
>
> In this case, xdm has been broken for more than a year because it has
> been ignoring pambase by not using system-local-login.
>
> Only the problem came to light after ConsoleKit really started to
> require this functionality from pambase.
>
> Tricky...
>
> But it was still tested, unfortunately the test environment that was
> used leaked some environment variables that made XDM appear to work, so
> this was catched too late.
>
> Sorry.
>
> - Samuli
>
>

I think that maybe we could have an even higher "priority" field called,
for example, "Broken Stable" that should be *only* used for that cases
and that arch teams should try fix sooner. What do you think?
 
Old 02-07-2011, 04:45 PM
"Andreas K. Huettel"
 
Default avoiding urgent stabilizations

We've been discussing this @FOSDEM too. My suggestion was that any bug that
visibly hurts stable users should always be considered at least MAJOR in
bugzilla.

To expand on this a bit more
* a stable update that makes the computer nonfunctional is definitely BLOCKER
(and should be reverted in CVS immediately when it becomes known, at latest
when it is understood, by anyone who is around at the time and can do so)
* a non-functional stable package in the system set should be CRITICAL.

Just my 2ct, but it is really important not to hurt stable users. This is how
we lose most people.


> I think that maybe we could have an even higher "priority" field called,
> for example, "Broken Stable" that should be *only* used for that cases
> and that arch teams should try fix sooner. What do you think?

--

Andreas K. Huettel
Gentoo Linux developer
dilfridge@gentoo.org
http://www.akhuettel.de/
 
Old 02-07-2011, 07:50 PM
Markos Chandras
 
Default avoiding urgent stabilizations

On Mon, Feb 07, 2011 at 06:45:10PM +0100, Andreas K. Huettel wrote:
>
> We've been discussing this @FOSDEM too. My suggestion was that any bug that
> visibly hurts stable users should always be considered at least MAJOR in
> bugzilla.
>
> To expand on this a bit more
> * a stable update that makes the computer nonfunctional is definitely BLOCKER
> (and should be reverted in CVS immediately when it becomes known, at latest
> when it is understood, by anyone who is around at the time and can do so)
> * a non-functional stable package in the system set should be CRITICAL.
>
> Just my 2ct, but it is really important not to hurt stable users. This is how
> we lose most people.
>
The rolling way we stabilize the packages makes the stable tree pretty
much fragile to breakages and stuff. This is because you cannot predict
what is going to happen to the rest of the tree if you stabilize a newer
package. It may have unpredictable consequences to the rest of the
packages. My suggestion, as I said to fosdem, is to freeze, or take a
snapshot if you like, of the current tree, stabilize what you need to
stabilize, test the whole tree ( at least compile wise ) for a couple of
weeks and then replace the existing stable tree. Of course this requires
automated script testing, hardware facilities etc etc that we don't have
so claiming that stable tree is "stable" is quite wrong.

Regards,
--
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
 
Old 02-07-2011, 08:02 PM
"Paweł Hajdan, Jr."
 
Default avoiding urgent stabilizations

On 2/7/11 9:50 PM, Markos Chandras wrote:
> My suggestion, as I said to fosdem, is to freeze, or take a
> snapshot if you like, of the current tree, stabilize what you need to
> stabilize, test the whole tree ( at least compile wise ) for a couple of
> weeks and then replace the existing stable tree. Of course this requires
> automated script testing, hardware facilities etc etc that we don't have
> so claiming that stable tree is "stable" is quite wrong.

This more thorough testing sounds really interesting. But do we really
lack hardware resources?

There are machines available for various arches at
<http://www.gentoo.org/proj/en/infrastructure/dev-machines.xml>. I have
at least a few chromium-related chroots on miranda, and I've never heard
complaints, so it seems a few more chroots for arch testing wouldn't hurt.

Of course for testing bootability and whether X11 starts up correctly,
etc we'd probably have to host some virtual machines, but better compile
testing (for example for toolchain updates) would be a good start.

Paweł
 
Old 02-08-2011, 02:36 AM
Duncan
 
Default avoiding urgent stabilizations

Paweł Hajdan, Jr. posted on Mon, 07 Feb 2011 22:02:36 +0100 as excerpted:

> On 2/7/11 9:50 PM, Markos Chandras wrote:
>> My suggestion, as I said to fosdem, is to freeze, or take a
>> snapshot if you like, of the current tree, stabilize what you need to
>> stabilize, test the whole tree ( at least compile wise ) for a couple
>> of weeks and then replace the existing stable tree. Of course this
>> requires automated script testing, hardware facilities etc etc that we
>> don't have so claiming that stable tree is "stable" is quite wrong.
>
> This more thorough testing sounds really interesting. But do we really
> lack hardware resources?

Disclaimer: I run ~arch (plus choice unmasks/overlays where ~arch is
already unsuitably outdated for my tastes), so this doesn't affect me
regardless. That said...

The above suggestion sounds to me like increasing the bureaucracy and
hassle of stabilizing packages even more. We already have trouble with
outdated stable, especially on some archs. Do we /really/ want the
reputation of competing with Debian-stal^hble for staleness?

Every few years someone comes up with the idea of creating a /truly/
stable, aka "enterprise" keyword/branch/whatever. Every few years, it
doesn't happen, for both resource and (arguably) philosophical reasons.

IMO, that's simply not suitable for or compatible with "mainline" Gentoo
and its rolling updates, etc. Yes, it's possible to do it. A lot of
things are "possible", but that doesn't mean they're practical. "It's
Gentoo, Jim, but not as we know it!"

As such, if someone/somegroup does decide to go that route, IMO the best
approach would be a separate Gentoo-based distribution, where freezing and
testing an entire tree for self-consistency and compatibility makes a bit
more sense. There are already a number of Gentoo-based distributions out
there. Certainly, talking to them about the problems they face and the
solutions they use, if not using one of them directly, could be a good
place to start.

Similarly, just as Gentoo has never made any bones about not being a hand-
holding distribution, in many cases, "you break, you get to keep the
pieces" (tho users and devs do try to help and devs do try to prevent,
where it's "sane" to do so), and it's not uncommon to see people saying
that if the install speed of a binary distribution or the centrally
controlled decisions of an Ubuntu are what one is after, Gentoo isn't
really where you should be looking, I'd say the same applies, to some
degree, here. Yes, we can try to keep stable breakage to a minimum, but
on a rolling release system, it's /going/ to happen, and I'd argue that
the sort of wholesale freeze-and-test discussed above really /would/ be
"Gentoo, Jim, but not as we know it", were it to be implemented as such in
Gentoo. That's not what Gentoo is /about/. The rolling updates are so
much a part of what makes Gentoo /Gentoo/, that take them out, as
wholesale freeze and test would effectively do, and you no longer have
Gentoo, at least as historically known.

That being the case, why confuse people about both the new product and
Gentoo as it currently is, by calling the new product Gentoo at all? If
it's effectively a Gentoo-based distribution that isn't itself Gentoo, at
least as historically considered, call it a Gentoo based distribution.
Don't call it Gentoo, thus avoiding the confusion on both sides.

JMHO, but I've been around long enough to have seen this discussion cycle
at least twice before... Unfortunately, the result is often the loss of a
few devs. OTOH, if their goal is to make Gentoo a wholesale freeze and
test distribution, perhaps it's better for both them and Gentoo that such
efforts get applied somewhere more appropriate to that goal.

OTOH, if some big name comes up with suitable big resources to devote to
such a project and they like the Gentoo name, perhaps a Gentoo-Enterprise
might indeed be practical. But the resource scaling that's going to
require is enough beyond what we're doing today that I'd think seeing
someone step up with an offer of such resources before we start seriously
discussing it, is a worthwhile prerequisite. And even then, making Gentoo
that dependent on that sort of major sponsorship, regardless of who or
what form, would change the historically "community distribution" aspect
substantially enough that arguably, that would be "Gentoo, Jim, but not as
we know it", as well. But at that point I guess it'd be a question for
the Gentoo foundation to decide, since they own the name and decide
permissions for it in that regard.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
 
Old 02-08-2011, 07:24 AM
Markos Chandras
 
Default avoiding urgent stabilizations

On Mon, Feb 07, 2011 at 10:02:36PM +0100, "Paweł Hajdan, Jr." wrote:
> On 2/7/11 9:50 PM, Markos Chandras wrote:
> > My suggestion, as I said to fosdem, is to freeze, or take a
> > snapshot if you like, of the current tree, stabilize what you need to
> > stabilize, test the whole tree ( at least compile wise ) for a couple of
> > weeks and then replace the existing stable tree. Of course this requires
> > automated script testing, hardware facilities etc etc that we don't have
> > so claiming that stable tree is "stable" is quite wrong.
>
> This more thorough testing sounds really interesting. But do we really
> lack hardware resources?
>
Yes!
> There are machines available for various arches at
> <http://www.gentoo.org/proj/en/infrastructure/dev-machines.xml>. I have
> at least a few chromium-related chroots on miranda, and I've never heard
> complaints, so it seems a few more chroots for arch testing wouldn't hurt.
>
No. Miranda, in particular, can go down anytime soon. This is what infra
told me twice.

> Of course for testing bootability and whether X11 starts up correctly,
> etc we'd probably have to host some virtual machines, but better compile
> testing (for example for toolchain updates) would be a good start.
>
> Paweł
>
Lets move this conversation on the thread that Tim started earlier
today. Seems like there is some hardware available for us on OSUOSL

Regards,
--
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
 
Old 02-08-2011, 07:38 AM
"Paweł Hajdan, Jr."
 
Default avoiding urgent stabilizations

On 2/8/11 9:24 AM, Markos Chandras wrote:
> On Mon, Feb 07, 2011 at 10:02:36PM +0100, "Paweł Hajdan, Jr." wrote:
>> There are machines available for various arches at
>> <http://www.gentoo.org/proj/en/infrastructure/dev-machines.xml>. I have
>> at least a few chromium-related chroots on miranda, and I've never heard
>> complaints, so it seems a few more chroots for arch testing wouldn't hurt.
>>
> No. Miranda, in particular, can go down anytime soon. This is what infra
> told me twice.

Oh, they didn't tell me that. What's the problem with miranda?
 
Old 02-08-2011, 10:43 AM
Roy Bamford
 
Default avoiding urgent stabilizations

On 2011.02.07 20:50, Markos Chandras wrote:
[snip]

> My suggestion, as I said to fosdem, is to freeze, or take a
> snapshot if you like, of the current tree, stabilize what you need to
> stabilize, test the whole tree ( at least compile wise ) for a couple
> of weeks and then replace the existing stable tree. Of course this
> requires automated script testing, hardware facilities etc etc that
> we don't have so claiming that stable tree is "stable" is quite
> wrong.
>
> Regards,
> --
> Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
>

Markos,

This is exactly what releng used to do for installer CDs. This was
last used for 2008.0 CD/DVD. A snapshot of the stable tree was taken in
February 200.8 and the release hit the mirrors in September.

The seven month test/fix/retest that it took meant that the CD would
not boot on new hardware as the kernel lacked drivers.

You will find similar lags when releng used this approach for other
earlier releases.

We would need to move to a release cycle like the kernel. Calling a
feature freeze at the start of the test cycle. As we can't everything,
we might as well distribute binaries of what was tested - just as
releng used to do.

To me thats not Gentoo as we would loose the rolling updates.

There are degrees of stable. I believe most Gentoo users
realise this fairly early on and stick with Gentoo because they like
the balance it strikes between Debian stable and bleeding edge.
There is a price to pay for being more up to date and it a trade off
Gentoo users are aware off. Of course, that does not prevent them
bringing breakages to our attention.

--
Regards,

Roy Bamford
(Neddyseagoon) a member of
gentoo-ops
forum-mods
trustees
 
Old 02-08-2011, 11:03 AM
Markos Chandras
 
Default avoiding urgent stabilizations

On Tue, Feb 08, 2011 at 11:43:33AM +0000, Roy Bamford wrote:
> On 2011.02.07 20:50, Markos Chandras wrote:
> [snip]
>
> > My suggestion, as I said to fosdem, is to freeze, or take a
> > snapshot if you like, of the current tree, stabilize what you need to
> > stabilize, test the whole tree ( at least compile wise ) for a couple
> > of weeks and then replace the existing stable tree. Of course this
> > requires automated script testing, hardware facilities etc etc that
> > we don't have so claiming that stable tree is "stable" is quite
> > wrong.
> >
> > Regards,
> > --
> > Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
> >
>
> Markos,
>
> This is exactly what releng used to do for installer CDs. This was
> last used for 2008.0 CD/DVD. A snapshot of the stable tree was taken in
> February 200.8 and the release hit the mirrors in September.
>
> The seven month test/fix/retest that it took meant that the CD would
> not boot on new hardware as the kernel lacked drivers.
>
> You will find similar lags when releng used this approach for other
> earlier releases.
>
> We would need to move to a release cycle like the kernel. Calling a
> feature freeze at the start of the test cycle. As we can't everything,
> we might as well distribute binaries of what was tested - just as
> releng used to do.
>
> To me thats not Gentoo as we would loose the rolling updates.
>
> There are degrees of stable. I believe most Gentoo users
> realise this fairly early on and stick with Gentoo because they like
> the balance it strikes between Debian stable and bleeding edge.
> There is a price to pay for being more up to date and it a trade off
> Gentoo users are aware off. Of course, that does not prevent them
> bringing breakages to our attention.
>
> --
> Regards,
>
> Roy Bamford
> (Neddyseagoon) a member of
> gentoo-ops
> forum-mods
> trustees

Roy,

I see what you are saying. However, the 6 months testing is far from
what I have in mind. My only intention is to bring a more stable
experience to our users. Or, stop claiming that our stable tree rocks
and Gentoo is perfect for servers because it is not. Ye ye ye I know
that many many of you have Gentoo on servers but do not forget that you
are developers and you know your way around during breakages. Yes,
stable tree breaks FAR TOO often. I blame myself for my arch testing of
course however I can't do much about that. Arch teams cannot afford any
slacking (at least the two major arches) because we have 100 new bugs
every three days. This is extremely dangers to demotivate people who
will burn out sooner or later. You may think that this is an extreme
case but I can guarantee you that amd64 stable tree can become really
obsolete within one month. .

Our stable tree is definitely not suitable for server usage unless
you have plenty of free time to
deal with stupid upgrades because nobody, for example, cared to write a
proper elog or news item. You are probably not aware of that, since 99%
of you run testing tree however if you visit forums and stuff you will
see many many users complaining about stable tree. If we keep going down
this road we will end up sooner or later to be similar to Exherbo where
only really power users and developers will know their way around.

Either you like it or not, arch teams are understaffed. All of them.
Therefore we cannot afford a updated stable tree with high QA around
it. We need to find a more efficient way to test packages on testing
tree so we can mark them stable with minimal time and cpu cost. We need
dedicated build boxes, like Diego's tinderbox, to test the testing tree
over and over against critical/common/trivial QA problems. If we manage
that, moving packages from testing->stable will be much more time
efficient and we can guarantee a high quality stable tree.

ps1: Personally I have stopped suggesting gentoo stable for server usage
and I always suggest testing to new users.

ps2: Roy, this is not a personal attack. Do not misinterpret my tone

Regards,
--
Markos Chandras / Gentoo Linux Developer / Key ID: B4AFF2C2
 

Thread Tools




All times are GMT. The time now is 10:07 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org