Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian Development (http://www.linux-archive.org/debian-development/)
-   -   A concrete proposal for rolling implementation (http://www.linux-archive.org/debian-development/522038-concrete-proposal-rolling-implementation.html)

Josselin Mouette 05-04-2011 12:24 PM

A concrete proposal for rolling implementation
 
Hi,

during the recent discussions about rolling, a proposal was made in a
blog comment, and after giving it some quick thoughts, most people I’ve
talked with seem to think it is a good idea, so it’s time for it to be
discussed at large.

It starts from the following fact: if you want a testing system that
works correctly, you usually have to add APT lines for unstable, while
pinning them to only install specific packages you need to unbreak
what’s broken in unstable.

The idea is to make this process automatic. Let me elaborate.

The new “rolling” suite
-----------------------
This would be a pseudo-suite, like experimental. Except that while
experimental is built on top of unstable and filled manually by
maintainers, rolling would be built on top of testing and filled
semi-automatically. A rolling system would have typically 2 APT lines:
one for testing and one for rolling.

The rolling suite would only exist for a subset of architectures (we
could start with powerpc, i386 and amd64), generated by picking up
packages from unstable. Typically it would be generated from an override
file that looks like:
source-package version
xserver-xorg-video-ati 1.2.3-4
...
The rolling suite would try to have a package that has *at least* this
version. If it is found in testing, the package is removed from rolling.
If otherwise it is found in unstable, the package is picked from
unstable.

This way, when something is broken in testing and cannot be unbroken
quickly, a maintainer who notices it could add (or make the people in
charge add) the necessary packages to the override file. If, for a
reason or another, an important bug fix or a security update doesn’t
propagate to testing quickly enough, you can now just add it and the
necessary dependencies to rolling, and people using it aren’t affected.
Whenever the affected packages finally migrate to testing, the
discrepancy between rolling and testing automatically disappears.

The reason for the “at least” version rule is that new uploads to
unstable are supposed to fix the situation in testing anyway. I don’t
think we should keep in rolling packages that are no longer in unstable.

A concrete example
------------------
Let’s imagine something that might happen soon (although of course we
will try hard for it not to happen): a new version of nautilus migrates
to testing, but it was missing a Breaks - it doesn’t work with the
version of gnome-settings-daemon in testing. The new
gnome-settings-daemon in unstable works, but it won’t migrate because
there is a libgnomekbd transition in progress, and gnome-screensaver
which is part of the transition doesn’t build on s390.

In this case, we can just add libgnomekbd and gnome-settings-daemon to
the override file. Users of the rolling suite will have the two versions
of libgnomekbd available, and they can update their systems to a working
state.

Why I like it
-------------
First of all, this idea doesn’t affect *at all* the current release
process. It just takes people willing to maintain the override file -
and we could even choose to let any DD modify it. And it’s much faster
to modify such a file than telling every user from testing that they
have to upgrade to the unstable version.

And just as importantly, I think it should just work. There’s very
little chance that people get completely hosed systems like it happens
sometimes for unstable. There are all chances that something broken in
testing can be fixed by pulling a handful of packages from unstable.

What to do during freezes
-------------------------
I’m not sure we really need to do something different in times of
freeze. Our time would be better spent by reducing the freeze time and
making it more predictable; squeeze has been an awesome step in this
direction.

If we want to do something different though, there is a simple recipe:
allow packages to be picked up from unstable, but also from
experimental. Again, no disruption: people can keep on breaking some
pieces of experimental, but if they want some other pieces to be useful
for rolling users, they just need to be committed to more carefulness
and to add them to the override file.


What do you think?

--
Joss


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1304511852.9090.85.camel@pi0307572">http://lists.debian.org/1304511852.9090.85.camel@pi0307572

Piotr Ożarowski 05-04-2011 01:22 PM

A concrete proposal for rolling implementation
 
[Josselin Mouette, 2011-05-04]
> This would be a pseudo-suite, like experimental. Except that while
> experimental is built on top of unstable and filled manually by
> maintainers, rolling would be built on top of testing and filled
> semi-automatically. A rolling system would have typically 2 APT lines:
> one for testing and one for rolling.

+1

[...]
> What to do during freezes
> -------------------------
> I’m not sure we really need to do something different in times of
> freeze. Our time would be better spent by reducing the freeze time and
> making it more predictable; squeeze has been an awesome step in this
> direction.

I'm not interested in helping f.e. with Perl bugs so once the ones
I care about are fixed, I just want to focus on Sid (i.e. upload new
upstream versions, prepare new transitions etc.) - I can wait a month or
two, but these long freezes demotivate me a lot.
--
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl www.griffith.cc www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110504132207.GM17596@piotro.eu">http://lists.debian.org/20110504132207.GM17596@piotro.eu

Didier Raboud 05-04-2011 01:26 PM

A concrete proposal for rolling implementation
 
Piotr Ożarowski wrote:

> [Josselin Mouette, 2011-05-04]
>> This would be a pseudo-suite, like experimental. Except that while
>> experimental is built on top of unstable and filled manually by
>> maintainers, rolling would be built on top of testing and filled
>> semi-automatically. A rolling system would have typically 2 APT lines:
>> one for testing and one for rolling.
>
> +1
>
> [...]
>> What to do during freezes
>> -------------------------
>> I’m not sure we really need to do something different in times of
>> freeze. Our time would be better spent by reducing the freeze time and
>> making it more predictable; squeeze has been an awesome step in this
>> direction.
>
> I'm not interested in helping f.e. with Perl bugs so once the ones
> I care about are fixed, I just want to focus on Sid (i.e. upload new
> upstream versions, prepare new transitions etc.) - I can wait a month or
> two, but these long freezes demotivate me a lot.

While I agree with the demotivation stance, why can't those packages be
uploaded to experimental, fwiw ?



--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: iprk5f$hgt$1@dough.gmane.org">http://lists.debian.org/iprk5f$hgt$1@dough.gmane.org

Raphael Hertzog 05-04-2011 01:30 PM

A concrete proposal for rolling implementation
 
Hi,

I came to the same conclusion than you after the discussion we had in
the comments of your article. I think it's the right approach. I still
have a few comments though.

On Wed, 04 May 2011, Josselin Mouette wrote:
> It starts from the following fact: if you want a testing system that
> works correctly, you usually have to add APT lines for unstable, while
> pinning them to only install specific packages you need to unbreak
> what’s broken in unstable.

I think you meant "what's broken in testing" (i.e. you take a few selected
packages from unstable to fix bugs present in testing).

> This would be a pseudo-suite, like experimental. Except that while
> experimental is built on top of unstable and filled manually by
> maintainers, rolling would be built on top of testing and filled
> semi-automatically. A rolling system would have typically 2 APT lines:
> one for testing and one for rolling.

It doesn't need to be a pseudo-suite. It's a collection of packages taken
in testing or unstable, it's not more complicated to make it a full suite.

And PR-wise, it's way better to avoid "testing" appearing in the
sources.list.

Also it means that the day where we have improved our processes in
unstable/testing so that rolling is no longer necessary, we can simply merge
testing and rolling in a single suite.

> The rolling suite would only exist for a subset of architectures (we
> could start with powerpc, i386 and amd64), generated by picking up

Why powerpc? If we really take more than i386/amd64 for rolling, there are
more users of armel than of powerpc.

> packages from unstable. Typically it would be generated from an override
> file that looks like:
> source-package version
> xserver-xorg-video-ati 1.2.3-4
> ...
> The rolling suite would try to have a package that has *at least* this
> version. If it is found in testing, the package is removed from rolling.
> If otherwise it is found in unstable, the package is picked from
> unstable.

You still need some sort of britney to verify that the package is effectively
installable in rolling, and to verify you're not breaking installability
of other packages available in rolling.

But yes the basic principle is to stay as close to testing as possible, to
get as much as possible fixed via testing (in particular when it's
possible to fix via an updated version pushed through t-p-u) and for the
rest to cherry-pick some updates from unstable.

Once we diverged from testing, there's the question of what to do when the
package gets updated in unstable. By default the answer is "nothing"
unless the updated package fix a RC bug that is not present in testing
but that was introduced since then (and is now present in the rolling
version).

> Why I like it
> -------------
> First of all, this idea doesn’t affect *at all* the current release
> process. It just takes people willing to maintain the override file -
> and we could even choose to let any DD modify it. And it’s much faster
> to modify such a file than telling every user from testing that they
> have to upgrade to the unstable version.

I don't believe in the "let any DD modify it" but there should be a simple
way for DD to evaluate and submit such requests of integration into
rolling.

> And just as importantly, I think it should just work. There’s very
> little chance that people get completely hosed systems like it happens
> sometimes for unstable. There are all chances that something broken in
> testing can be fixed by pulling a handful of packages from unstable.

I agree with this. There might some corner-cases where we might require
direct uploads into rolling but if we stick to i386/amd64, it's easy
enough to do even without specific support on the buildd side.

> What to do during freezes
> -------------------------

I leave that aside for now. Your proposal covers the need to push newer
upstream versions to users but doesn't respond to the desire of developers
to continue development. But it's really another discussion so I prefer to
not discuss it in that thread.

> What do you think?

+1 from me. Thank you for the proposal!

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110504133040.GB24264@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110504133040.GB24264@rivendell.home.ouaza.com

Piotr Ożarowski 05-04-2011 01:32 PM

A concrete proposal for rolling implementation
 
[Didier Raboud, 2011-05-04]
> While I agree with the demotivation stance, why can't those packages be
> uploaded to experimental, fwiw ?

because that's not what experimental is for and it's harder to use it
(did you notice that python3.2 is in experimental or did someone gave
you proper apt-pinning rule when Squeeze was frozen?)
--
Piotr Ożarowski Debian GNU/Linux Developer
www.ozarowski.pl www.griffith.cc www.debian.org
GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110504133210.GN17596@piotro.eu">http://lists.debian.org/20110504133210.GN17596@piotro.eu

Josselin Mouette 05-04-2011 01:43 PM

A concrete proposal for rolling implementation
 
Le mercredi 04 mai 2011 * 15:30 +0200, Raphael Hertzog a écrit :
> On Wed, 04 May 2011, Josselin Mouette wrote:
> > It starts from the following fact: if you want a testing system that
> > works correctly, you usually have to add APT lines for unstable, while
> > pinning them to only install specific packages you need to unbreak
> > what’s broken in unstable.
>
> I think you meant "what's broken in testing" (i.e. you take a few selected
> packages from unstable to fix bugs present in testing).

Yes, of course.

> It doesn't need to be a pseudo-suite. It's a collection of packages taken
> in testing or unstable, it's not more complicated to make it a full suite.

It cannot be “just” a full suite. When you add a package coming from
unstable, you must also keep the version that is already in testing. To
follow on my example, if you allow libgnomekbd and gnome-settings-daemon
from unstable, you still need libgnomekbd from testing, otherwise other
packages will become uninstallable.

> And PR-wise, it's way better to avoid "testing" appearing in the
> sources.list.

That’s really an implementation detail. If you really want to hide it,
you just need to ensure rolling can have two versions of the same
sources at the same time.

> > The rolling suite would only exist for a subset of architectures (we
> > could start with powerpc, i386 and amd64), generated by picking up
>
> Why powerpc? If we really take more than i386/amd64 for rolling, there are
> more users of armel than of powerpc.

Yes, sure. It was just an example.

> You still need some sort of britney to verify that the package is effectively
> installable in rolling, and to verify you're not breaking installability
> of other packages available in rolling.

If rolling is just an overlay on testing, I don’t think that’s
necessary. The worst that could happen is that the version in rolling is
uninstallable, but the version in testing would still be.

What the britney-like thing could do is bring automatically all
dependencies that are actually necessary for the package to be
installable. But this could also make things more complicated, since
britney tests source packages, not binaries. You might bring a source in
rolling only for a runtime that is needed, but the development header
being uninstallable is not a problem. Britney would prevent that, while
it would still be a good move.

> Once we diverged from testing, there's the question of what to do when the
> package gets updated in unstable. By default the answer is "nothing"
> unless the updated package fix a RC bug that is not present in testing
> but that was introduced since then (and is now present in the rolling
> version).

I’m not entirely sure, but I think it’s better to pick the update from
unstable instead of letting in rolling a package that is no longer
available somewhere else. It should make upgrades smoother, and it
should also be less work for maintainers, since it avoids having another
version from which problems can arise.

--
Joss


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1304516627.9090.154.camel@pi0307572">http://lists.debian.org/1304516627.9090.154.camel@pi0307572

Raphael Hertzog 05-04-2011 02:20 PM

A concrete proposal for rolling implementation
 
On Wed, 04 May 2011, Josselin Mouette wrote:
> > It doesn't need to be a pseudo-suite. It's a collection of packages taken
> > in testing or unstable, it's not more complicated to make it a full suite.
>
> It cannot be “just” a full suite. When you add a package coming from
> unstable, you must also keep the version that is already in testing. To
> follow on my example, if you allow libgnomekbd and gnome-settings-daemon
> from unstable, you still need libgnomekbd from testing, otherwise other
> packages will become uninstallable.

A full suite can have 2 versions of the same source package and
can contain both libgnomekbd4 and libgnomekbd7. It's not a problem.

> > And PR-wise, it's way better to avoid "testing" appearing in the
> > sources.list.
>
> That’s really an implementation detail. If you really want to hide it,
> you just need to ensure rolling can have two versions of the same
> sources at the same time.

OK. Testing already can, so it should be ok for rolling too.

> > You still need some sort of britney to verify that the package is effectively
> > installable in rolling, and to verify you're not breaking installability
> > of other packages available in rolling.
>
> If rolling is just an overlay on testing, I don’t think that’s
> necessary. The worst that could happen is that the version in rolling is
> uninstallable, but the version in testing would still be.

Yes but as long as it's uninstallable, the bug is not fixed for the user.
So while we did not break anything, we did not fix anything either.
:-)

> What the britney-like thing could do is bring automatically all
> dependencies that are actually necessary for the package to be
> installable. But this could also make things more complicated, since
> britney tests source packages, not binaries. You might bring a source in
> rolling only for a runtime that is needed, but the development header
> being uninstallable is not a problem. Britney would prevent that, while
> it would still be a good move.

Yes, we could try to improve britney towards this.

But I'm not sure it's a good idea to pick only some binary packages from a
source package. It happens often enough that the package lack a strict
dependency that might be required and picking all the binaries from a
source package limits the risk of upgrading them separately (and thus
experiencing the bug).

> > Once we diverged from testing, there's the question of what to do when the
> > package gets updated in unstable. By default the answer is "nothing"
> > unless the updated package fix a RC bug that is not present in testing
> > but that was introduced since then (and is now present in the rolling
> > version).
>
> I’m not entirely sure, but I think it’s better to pick the update from
> unstable instead of letting in rolling a package that is no longer
> available somewhere else. It should make upgrades smoother, and it
> should also be less work for maintainers, since it avoids having another
> version from which problems can arise.

In that case, those updates should follow the same rules than for testing
itself. It would be unreasonable to accept the new unstable upload
immediately.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110504142011.GE24264@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110504142011.GE24264@rivendell.home.ouaza.com

Goswin von Brederlow 05-04-2011 02:20 PM

A concrete proposal for rolling implementation
 
Josselin Mouette <joss@debian.org> writes:

> This way, when something is broken in testing and cannot be unbroken
> quickly, a maintainer who notices it could add (or make the people in
> charge add) the necessary packages to the override file. If, for a
> reason or another, an important bug fix or a security update doesn??t
> propagate to testing quickly enough, you can now just add it and the
> necessary dependencies to rolling, and people using it aren??t affected.
> Whenever the affected packages finally migrate to testing, the
> discrepancy between rolling and testing automatically disappears.

That sounds like a nice idea. Maybe call it hot-fix instead of rolling. :)

I would suggest one more thing though. Sometimes it is know that a
package breaks on upgrade and maybe even causes data loss. But the fix
might not be aparent or quick to implement. Maybe it would be nice if
one could then also remove or block a package so people won't upgrade to
the known bad version while the maintainer works on a fix.

Note: this would prbably require a full Packages file and people to only
add rolling to sources.list without also having testing.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87fwoua6oo.fsf@frosties.localnet">http://lists.debian.org/87fwoua6oo.fsf@frosties.localnet

Gunnar Wolf 05-04-2011 03:30 PM

A concrete proposal for rolling implementation
 
Piotr Ożarowski dijo [Wed, May 04, 2011 at 03:22:07PM +0200]:
> [Josselin Mouette, 2011-05-04]
> > This would be a pseudo-suite, like experimental. Except that while
> > experimental is built on top of unstable and filled manually by
> > maintainers, rolling would be built on top of testing and filled
> > semi-automatically. A rolling system would have typically 2 APT lines:
> > one for testing and one for rolling.
>
> +1

I'll also state it: Josselin's proposal looks very interesting, simple
and compatible with what we have now!

> [...]
> > What to do during freezes
> > -------------------------
> > I’m not sure we really need to do something different in times of
> > freeze. Our time would be better spent by reducing the freeze time and
> > making it more predictable; squeeze has been an awesome step in this
> > direction.
>
> I'm not interested in helping f.e. with Perl bugs so once the ones
> I care about are fixed, I just want to focus on Sid (i.e. upload new
> upstream versions, prepare new transitions etc.) - I can wait a month or
> two, but these long freezes demotivate me a lot.

For many bugs, it's not only that I'm not interested, but I'm also
disqualified. Of course, a long freeze can be demotivating, and it can
also cause a spike of work when we reopen unstable for "anything goes"
uploads.

In my case, I used this last freeze to redo the packaging for one of
my complex packages, and kept up-to-date with upstream via
experimental - So I was breaking stuff and keeping up to date at the
same time. It cannot always work like this, of course, I'm just
mentioning a way to keep the fun flowing while in freeze.

Now, long freezes are complicated, true. But I don't think keeping
unstable disconnected from (frozen|testing) will really help us. If
all uploads during the freeze have to go through t-p-u, we will lose a
bit of what gives coherence to the whole system.

I feel, as many others have stated, that the Squeeze release cycle was
quite a successful one, even with some erupting flames here and
there... Probably we should focus more on keeping the bug count lower
during the non-freezing period? That should surely lead to a shorter
freeze period.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110504153025.GC15466@gwolf.org">http://lists.debian.org/20110504153025.GC15466@gwolf.org

Cyril Brulebois 05-04-2011 03:43 PM

A concrete proposal for rolling implementation
 
Hi,

(you already know, but let's state that on dd@ too)

Josselin Mouette <joss@debian.org> (04/05/2011):
> during the recent discussions about rolling, a proposal was made in
> a blog comment, and after giving it some quick thoughts, most people
> I’ve talked with seem to think it is a good idea, so it’s time for
> it to be discussed at large.
>
> It starts from the following fact: if you want a testing system that
> works correctly, you usually have to add APT lines for unstable,
> while pinning them to only install specific packages you need to
> unbreak what’s broken in unstable.

your proposal certainly makes a lot of sense. Having to quick-fetch
packages from unstable to avoid long-term breakages seems to be the
major issue for prospective testing users, and “automating” (whatever
the details) that pinning seems like an easy and non-disruptive (as
far as the testing process is concerned -- AFAICT, IMVHO, etc.) way
to fix that major usability issue.

Thank you for coming with that concrete proposal.

Mraw,
KiBi.


All times are GMT. The time now is 07:52 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.