A concrete proposal for rolling implementation
Hi,
during the recent discussions about rolling, a proposal was made in a blog comment, and after giving it some quick thoughts, most people I’ve talked with seem to think it is a good idea, so it’s time for it to be discussed at large. It starts from the following fact: if you want a testing system that works correctly, you usually have to add APT lines for unstable, while pinning them to only install specific packages you need to unbreak what’s broken in unstable. The idea is to make this process automatic. Let me elaborate. The new “rolling” suite ----------------------- This would be a pseudo-suite, like experimental. Except that while experimental is built on top of unstable and filled manually by maintainers, rolling would be built on top of testing and filled semi-automatically. A rolling system would have typically 2 APT lines: one for testing and one for rolling. The rolling suite would only exist for a subset of architectures (we could start with powerpc, i386 and amd64), generated by picking up packages from unstable. Typically it would be generated from an override file that looks like: source-package version xserver-xorg-video-ati 1.2.3-4 ... The rolling suite would try to have a package that has *at least* this version. If it is found in testing, the package is removed from rolling. If otherwise it is found in unstable, the package is picked from unstable. This way, when something is broken in testing and cannot be unbroken quickly, a maintainer who notices it could add (or make the people in charge add) the necessary packages to the override file. If, for a reason or another, an important bug fix or a security update doesn’t propagate to testing quickly enough, you can now just add it and the necessary dependencies to rolling, and people using it aren’t affected. Whenever the affected packages finally migrate to testing, the discrepancy between rolling and testing automatically disappears. The reason for the “at least” version rule is that new uploads to unstable are supposed to fix the situation in testing anyway. I don’t think we should keep in rolling packages that are no longer in unstable. A concrete example ------------------ Let’s imagine something that might happen soon (although of course we will try hard for it not to happen): a new version of nautilus migrates to testing, but it was missing a Breaks - it doesn’t work with the version of gnome-settings-daemon in testing. The new gnome-settings-daemon in unstable works, but it won’t migrate because there is a libgnomekbd transition in progress, and gnome-screensaver which is part of the transition doesn’t build on s390. In this case, we can just add libgnomekbd and gnome-settings-daemon to the override file. Users of the rolling suite will have the two versions of libgnomekbd available, and they can update their systems to a working state. Why I like it ------------- First of all, this idea doesn’t affect *at all* the current release process. It just takes people willing to maintain the override file - and we could even choose to let any DD modify it. And it’s much faster to modify such a file than telling every user from testing that they have to upgrade to the unstable version. And just as importantly, I think it should just work. There’s very little chance that people get completely hosed systems like it happens sometimes for unstable. There are all chances that something broken in testing can be fixed by pulling a handful of packages from unstable. What to do during freezes ------------------------- I’m not sure we really need to do something different in times of freeze. Our time would be better spent by reducing the freeze time and making it more predictable; squeeze has been an awesome step in this direction. If we want to do something different though, there is a simple recipe: allow packages to be picked up from unstable, but also from experimental. Again, no disruption: people can keep on breaking some pieces of experimental, but if they want some other pieces to be useful for rolling users, they just need to be committed to more carefulness and to add them to the override file. What do you think? -- Joss -- To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org Archive: 1304511852.9090.85.camel@pi0307572">http://lists.debian.org/1304511852.9090.85.camel@pi0307572 |
A concrete proposal for rolling implementation
[Josselin Mouette, 2011-05-04]
> This would be a pseudo-suite, like experimental. Except that while > experimental is built on top of unstable and filled manually by > maintainers, rolling would be built on top of testing and filled > semi-automatically. A rolling system would have typically 2 APT lines: > one for testing and one for rolling. +1 [...] > What to do during freezes > ------------------------- > I’m not sure we really need to do something different in times of > freeze. Our time would be better spent by reducing the freeze time and > making it more predictable; squeeze has been an awesome step in this > direction. I'm not interested in helping f.e. with Perl bugs so once the ones I care about are fixed, I just want to focus on Sid (i.e. upload new upstream versions, prepare new transitions etc.) - I can wait a month or two, but these long freezes demotivate me a lot. -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org Archive: 20110504132207.GM17596@piotro.eu">http://lists.debian.org/20110504132207.GM17596@piotro.eu |
A concrete proposal for rolling implementation
Piotr Ożarowski wrote:
> [Josselin Mouette, 2011-05-04] >> This would be a pseudo-suite, like experimental. Except that while >> experimental is built on top of unstable and filled manually by >> maintainers, rolling would be built on top of testing and filled >> semi-automatically. A rolling system would have typically 2 APT lines: >> one for testing and one for rolling. > > +1 > > [...] >> What to do during freezes >> ------------------------- >> I’m not sure we really need to do something different in times of >> freeze. Our time would be better spent by reducing the freeze time and >> making it more predictable; squeeze has been an awesome step in this >> direction. > > I'm not interested in helping f.e. with Perl bugs so once the ones > I care about are fixed, I just want to focus on Sid (i.e. upload new > upstream versions, prepare new transitions etc.) - I can wait a month or > two, but these long freezes demotivate me a lot. While I agree with the demotivation stance, why can't those packages be uploaded to experimental, fwiw ? -- To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org Archive: iprk5f$hgt$1@dough.gmane.org">http://lists.debian.org/iprk5f$hgt$1@dough.gmane.org |
A concrete proposal for rolling implementation
Hi,
I came to the same conclusion than you after the discussion we had in the comments of your article. I think it's the right approach. I still have a few comments though. On Wed, 04 May 2011, Josselin Mouette wrote: > It starts from the following fact: if you want a testing system that > works correctly, you usually have to add APT lines for unstable, while > pinning them to only install specific packages you need to unbreak > what’s broken in unstable. I think you meant "what's broken in testing" (i.e. you take a few selected packages from unstable to fix bugs present in testing). > This would be a pseudo-suite, like experimental. Except that while > experimental is built on top of unstable and filled manually by > maintainers, rolling would be built on top of testing and filled > semi-automatically. A rolling system would have typically 2 APT lines: > one for testing and one for rolling. It doesn't need to be a pseudo-suite. It's a collection of packages taken in testing or unstable, it's not more complicated to make it a full suite. And PR-wise, it's way better to avoid "testing" appearing in the sources.list. Also it means that the day where we have improved our processes in unstable/testing so that rolling is no longer necessary, we can simply merge testing and rolling in a single suite. > The rolling suite would only exist for a subset of architectures (we > could start with powerpc, i386 and amd64), generated by picking up Why powerpc? If we really take more than i386/amd64 for rolling, there are more users of armel than of powerpc. > packages from unstable. Typically it would be generated from an override > file that looks like: > source-package version > xserver-xorg-video-ati 1.2.3-4 > ... > The rolling suite would try to have a package that has *at least* this > version. If it is found in testing, the package is removed from rolling. > If otherwise it is found in unstable, the package is picked from > unstable. You still need some sort of britney to verify that the package is effectively installable in rolling, and to verify you're not breaking installability of other packages available in rolling. But yes the basic principle is to stay as close to testing as possible, to get as much as possible fixed via testing (in particular when it's possible to fix via an updated version pushed through t-p-u) and for the rest to cherry-pick some updates from unstable. Once we diverged from testing, there's the question of what to do when the package gets updated in unstable. By default the answer is "nothing" unless the updated package fix a RC bug that is not present in testing but that was introduced since then (and is now present in the rolling version). > Why I like it > ------------- > First of all, this idea doesn’t affect *at all* the current release > process. It just takes people willing to maintain the override file - > and we could even choose to let any DD modify it. And it’s much faster > to modify such a file than telling every user from testing that they > have to upgrade to the unstable version. I don't believe in the "let any DD modify it" but there should be a simple way for DD to evaluate and submit such requests of integration into rolling. > And just as importantly, I think it should just work. There’s very > little chance that people get completely hosed systems like it happens > sometimes for unstable. There are all chances that something broken in > testing can be fixed by pulling a handful of packages from unstable. I agree with this. There might some corner-cases where we might require direct uploads into rolling but if we stick to i386/amd64, it's easy enough to do even without specific support on the buildd side. > What to do during freezes > ------------------------- I leave that aside for now. Your proposal covers the need to push newer upstream versions to users but doesn't respond to the desire of developers to continue development. But it's really another discussion so I prefer to not discuss it in that thread. > What do you think? +1 from me. Thank you for the proposal! Cheers, -- Raphaël Hertzog ◈ Debian Developer Follow my Debian News ▶ http://RaphaelHertzog.com (English) ▶ http://RaphaelHertzog.fr (Français) -- To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org Archive: 20110504133040.GB24264@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110504133040.GB24264@rivendell.home.ouaza.com |
A concrete proposal for rolling implementation
[Didier Raboud, 2011-05-04]
> While I agree with the demotivation stance, why can't those packages be > uploaded to experimental, fwiw ? because that's not what experimental is for and it's harder to use it (did you notice that python3.2 is in experimental or did someone gave you proper apt-pinning rule when Squeeze was frozen?) -- Piotr Ożarowski Debian GNU/Linux Developer www.ozarowski.pl www.griffith.cc www.debian.org GPG Fingerprint: 1D2F A898 58DA AF62 1786 2DF7 AEF6 F1A2 A745 7645 -- To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org Archive: 20110504133210.GN17596@piotro.eu">http://lists.debian.org/20110504133210.GN17596@piotro.eu |
A concrete proposal for rolling implementation
Le mercredi 04 mai 2011 * 15:30 +0200, Raphael Hertzog a écrit :
> On Wed, 04 May 2011, Josselin Mouette wrote: > > It starts from the following fact: if you want a testing system that > > works correctly, you usually have to add APT lines for unstable, while > > pinning them to only install specific packages you need to unbreak > > what’s broken in unstable. > > I think you meant "what's broken in testing" (i.e. you take a few selected > packages from unstable to fix bugs present in testing). Yes, of course. > It doesn't need to be a pseudo-suite. It's a collection of packages taken > in testing or unstable, it's not more complicated to make it a full suite. It cannot be “just” a full suite. When you add a package coming from unstable, you must also keep the version that is already in testing. To follow on my example, if you allow libgnomekbd and gnome-settings-daemon from unstable, you still need libgnomekbd from testing, otherwise other packages will become uninstallable. > And PR-wise, it's way better to avoid "testing" appearing in the > sources.list. That’s really an implementation detail. If you really want to hide it, you just need to ensure rolling can have two versions of the same sources at the same time. > > The rolling suite would only exist for a subset of architectures (we > > could start with powerpc, i386 and amd64), generated by picking up > > Why powerpc? If we really take more than i386/amd64 for rolling, there are > more users of armel than of powerpc. Yes, sure. It was just an example. > You still need some sort of britney to verify that the package is effectively > installable in rolling, and to verify you're not breaking installability > of other packages available in rolling. If rolling is just an overlay on testing, I don’t think that’s necessary. The worst that could happen is that the version in rolling is uninstallable, but the version in testing would still be. What the britney-like thing could do is bring automatically all dependencies that are actually necessary for the package to be installable. But this could also make things more complicated, since britney tests source packages, not binaries. You might bring a source in rolling only for a runtime that is needed, but the development header being uninstallable is not a problem. Britney would prevent that, while it would still be a good move. > Once we diverged from testing, there's the question of what to do when the > package gets updated in unstable. By default the answer is "nothing" > unless the updated package fix a RC bug that is not present in testing > but that was introduced since then (and is now present in the rolling > version). I’m not entirely sure, but I think it’s better to pick the update from unstable instead of letting in rolling a package that is no longer available somewhere else. It should make upgrades smoother, and it should also be less work for maintainers, since it avoids having another version from which problems can arise. -- Joss -- To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org Archive: 1304516627.9090.154.camel@pi0307572">http://lists.debian.org/1304516627.9090.154.camel@pi0307572 |
A concrete proposal for rolling implementation
On Wed, 04 May 2011, Josselin Mouette wrote:
> > It doesn't need to be a pseudo-suite. It's a collection of packages taken > > in testing or unstable, it's not more complicated to make it a full suite. > > It cannot be “just” a full suite. When you add a package coming from > unstable, you must also keep the version that is already in testing. To > follow on my example, if you allow libgnomekbd and gnome-settings-daemon > from unstable, you still need libgnomekbd from testing, otherwise other > packages will become uninstallable. A full suite can have 2 versions of the same source package and can contain both libgnomekbd4 and libgnomekbd7. It's not a problem. > > And PR-wise, it's way better to avoid "testing" appearing in the > > sources.list. > > That’s really an implementation detail. If you really want to hide it, > you just need to ensure rolling can have two versions of the same > sources at the same time. OK. Testing already can, so it should be ok for rolling too. > > You still need some sort of britney to verify that the package is effectively > > installable in rolling, and to verify you're not breaking installability > > of other packages available in rolling. > > If rolling is just an overlay on testing, I don’t think that’s > necessary. The worst that could happen is that the version in rolling is > uninstallable, but the version in testing would still be. Yes but as long as it's uninstallable, the bug is not fixed for the user. So while we did not break anything, we did not fix anything either. :-) > What the britney-like thing could do is bring automatically all > dependencies that are actually necessary for the package to be > installable. But this could also make things more complicated, since > britney tests source packages, not binaries. You might bring a source in > rolling only for a runtime that is needed, but the development header > being uninstallable is not a problem. Britney would prevent that, while > it would still be a good move. Yes, we could try to improve britney towards this. But I'm not sure it's a good idea to pick only some binary packages from a source package. It happens often enough that the package lack a strict dependency that might be required and picking all the binaries from a source package limits the risk of upgrading them separately (and thus experiencing the bug). > > Once we diverged from testing, there's the question of what to do when the > > package gets updated in unstable. By default the answer is "nothing" > > unless the updated package fix a RC bug that is not present in testing > > but that was introduced since then (and is now present in the rolling > > version). > > I’m not entirely sure, but I think it’s better to pick the update from > unstable instead of letting in rolling a package that is no longer > available somewhere else. It should make upgrades smoother, and it > should also be less work for maintainers, since it avoids having another > version from which problems can arise. In that case, those updates should follow the same rules than for testing itself. It would be unreasonable to accept the new unstable upload immediately. Cheers, -- Raphaël Hertzog ◈ Debian Developer Follow my Debian News ▶ http://RaphaelHertzog.com (English) ▶ http://RaphaelHertzog.fr (Français) -- To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org Archive: 20110504142011.GE24264@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110504142011.GE24264@rivendell.home.ouaza.com |
A concrete proposal for rolling implementation
Josselin Mouette <joss@debian.org> writes:
> This way, when something is broken in testing and cannot be unbroken > quickly, a maintainer who notices it could add (or make the people in > charge add) the necessary packages to the override file. If, for a > reason or another, an important bug fix or a security update doesn??t > propagate to testing quickly enough, you can now just add it and the > necessary dependencies to rolling, and people using it aren??t affected. > Whenever the affected packages finally migrate to testing, the > discrepancy between rolling and testing automatically disappears. That sounds like a nice idea. Maybe call it hot-fix instead of rolling. :) I would suggest one more thing though. Sometimes it is know that a package breaks on upgrade and maybe even causes data loss. But the fix might not be aparent or quick to implement. Maybe it would be nice if one could then also remove or block a package so people won't upgrade to the known bad version while the maintainer works on a fix. Note: this would prbably require a full Packages file and people to only add rolling to sources.list without also having testing. MfG Goswin -- To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org Archive: 87fwoua6oo.fsf@frosties.localnet">http://lists.debian.org/87fwoua6oo.fsf@frosties.localnet |
A concrete proposal for rolling implementation
Piotr Ożarowski dijo [Wed, May 04, 2011 at 03:22:07PM +0200]:
> [Josselin Mouette, 2011-05-04] > > This would be a pseudo-suite, like experimental. Except that while > > experimental is built on top of unstable and filled manually by > > maintainers, rolling would be built on top of testing and filled > > semi-automatically. A rolling system would have typically 2 APT lines: > > one for testing and one for rolling. > > +1 I'll also state it: Josselin's proposal looks very interesting, simple and compatible with what we have now! > [...] > > What to do during freezes > > ------------------------- > > I’m not sure we really need to do something different in times of > > freeze. Our time would be better spent by reducing the freeze time and > > making it more predictable; squeeze has been an awesome step in this > > direction. > > I'm not interested in helping f.e. with Perl bugs so once the ones > I care about are fixed, I just want to focus on Sid (i.e. upload new > upstream versions, prepare new transitions etc.) - I can wait a month or > two, but these long freezes demotivate me a lot. For many bugs, it's not only that I'm not interested, but I'm also disqualified. Of course, a long freeze can be demotivating, and it can also cause a spike of work when we reopen unstable for "anything goes" uploads. In my case, I used this last freeze to redo the packaging for one of my complex packages, and kept up-to-date with upstream via experimental - So I was breaking stuff and keeping up to date at the same time. It cannot always work like this, of course, I'm just mentioning a way to keep the fun flowing while in freeze. Now, long freezes are complicated, true. But I don't think keeping unstable disconnected from (frozen|testing) will really help us. If all uploads during the freeze have to go through t-p-u, we will lose a bit of what gives coherence to the whole system. I feel, as many others have stated, that the Squeeze release cycle was quite a successful one, even with some erupting flames here and there... Probably we should focus more on keeping the bug count lower during the non-freezing period? That should surely lead to a shorter freeze period. -- To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org Archive: 20110504153025.GC15466@gwolf.org">http://lists.debian.org/20110504153025.GC15466@gwolf.org |
A concrete proposal for rolling implementation
Hi,
(you already know, but let's state that on dd@ too) Josselin Mouette <joss@debian.org> (04/05/2011): > during the recent discussions about rolling, a proposal was made in > a blog comment, and after giving it some quick thoughts, most people > I’ve talked with seem to think it is a good idea, so it’s time for > it to be discussed at large. > > It starts from the following fact: if you want a testing system that > works correctly, you usually have to add APT lines for unstable, > while pinning them to only install specific packages you need to > unbreak what’s broken in unstable. your proposal certainly makes a lot of sense. Having to quick-fetch packages from unstable to avoid long-term breakages seems to be the major issue for prospective testing users, and “automating” (whatever the details) that pinning seems like an easy and non-disruptive (as far as the testing process is concerned -- AFAICT, IMVHO, etc.) way to fix that major usability issue. Thank you for coming with that concrete proposal. Mraw, KiBi. |
| All times are GMT. The time now is 03:38 AM. |
VBulletin, Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.