Proposed udpates policy change
 

Matthew Garrett wrote:
> The ability for maintainers to flag an update directly into the updates
> repository will be disabled. Before being added to updates, the package
> must receive a net karma of +3 in Bodhi.

Even if it already went to testing and sit there for ages? This will lead to
MANY updates being stalled in testing forever! Hardly any update is getting
+3 karma right now. Karma is completely useless in practice. People "gaming
the system" are going to be the only way updates can go out at all under
such a proposal.

Sure, we'd all love that kind of test coverage. It's just not practicable at
all. We need to be realistic.

In addition, it has been explained very clearly in the discussions on this
mailing list why the current karma system is a poor indicator of update
stability. Relying on it as the only way an update can go stable is just
insane.

> It is the expectation of Fesco that the majority of updates should
> easily be able to garner the necessary karma in a minimal space of time.

You gotta be kidding! Just look at the current update landscape to see how
far from the truth this is.

(And I also wonder with what authority you speak in the name of FESCo as a
whole. The fact that what you're saying is so clearly wrong makes this all
the more an issue.)

Kevin Kofler

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Proposed udpates policy change
 

Matthew Garrett wrote:
> What guards you against changes in the buildroot, non-deterministic
> compiler bugs, cosmic rays and the like? The point is to test the
> binaries that are being pushed into the hands of the users.

There's a point at which the probability of breakage is so low that it's a
waste of time and effort to even consider it. Software will never be perfect
anyway.

Kevin Kofler

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Proposed udpates policy change
 

Michael Schwendt wrote:

> On Mon, 8 Mar 2010 21:59:29 +0000, Matthew wrote:
>> It is the expectation of Fesco that the majority of updates should
>> easily be able to garner the necessary karma in a minimal space of time.
>
> Your wording or FESCo's?

Definitely not mine, I can assure you! To me, that statement sounds more
like something out of The Onion than like something one can actually claim
with a straight face!

Kevin Kofler

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Proposed udpates policy change
 

On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
> We assume the following axioms:
>
> 1) Updates to stable that result in any reduction of functionality to
> the user are unacceptable.
>
> 2) It is impossible to ensure that functionality will not be reduced
> without sufficient testing.
>
> 3) Sufficient testing of software inherently requires manual
> intervention by more than one individual.

I agree with these.

> Proposal
> --------
>
> The ability for maintainers to flag an update directly into the updates
> repository will be disabled. Before being added to updates, the package
> must receive a net karma of +3 in Bodhi.

I have some difficulty agreeing that +3 is "sufficient testing" for all
packages. I know a *lot* of people who used Gwibber when I maintained
it. It's probably the most used package I have ever maintained, and I
couldn't get anybody to test it without lobbying people in #fedora-docs
and on the Planet. +3 is too much for packages like these.

It's not the job of a package maintainer to campaign and say "hey, pay
attention to my package that only provides one Python function for a few
people who need it," let alone a nicely-done end-user application that a
lot of people use.

Maybe +3 karma is OK if we allow for a push straight to updates after
the old_testing period comes along. (Oh look, another new one of those
in my INBOX now.)

> At present, this policy will not apply to updates that are flagged as
> security updates.

Why? Don't those need testing too?

--
Ian Weller <ian@ianweller.org>
() ascii ribbon campaign - against html e-mail
/ www.asciiribbon.org - against proprietary attachments
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Proposed udpates policy change
 

On Mon, Mar 08, 2010 at 06:28:45PM -0500, Martin Langhoff wrote:
> On Mon, Mar 8, 2010 at 4:59 PM, Matthew Garrett <mjg59@srcf.ucam.org> wrote:
> > The future
> > ----------
> >
> > Defining the purpose of Fedora updates is outside the scope of Fesco.
> > However, we note that updates intended to add new functionality are more
> > likely to result in user-visible regressions, and updates that alter ABI
> > or API are likely to break local customisations even if all Fedora
> > packages are updated to match.
>
> Thanks to you, Fesco and all developers working on this. There is
> something that everyone seems to be missing on this track, and I'd
> like to bring some attention to it.
>
> Distros do integration work, and the main thrust of the QA work around
> a release is that all the packages work nicely _together_, that all
> the subtle interactions interact the right way.
>
> Updates are always risky. There is no doubt that the updated package
> worked fine for the maintainer, and yet we see updates bombing out
> spectacularly relatively often. This is because pushing out a single
> package update is what a maintainer does, but this "package focus"
> undermines what the distro does -- _integration_.
>
> Small, low-risk bugfix updates respect that distro-wide goal. Major
> risky updates disregard that goal -- yes, a specific package is new!
> improved! but the implicit social contrct with your end users ("a
> nicely integrated OS") is broken because the time and effort to shake
> out the subtle integration issues were skipped.
>
There are seveal things to note, though.
* The dnssec update which prompted this policy was not actually a problem of
integration with the rest of the OS.
* The policy doesn't place any value on the types of updates that are made,
therefore it is only making an indirect change to integration. For
instance, KDE updates that have been talked about elsewhere would likely
still go throguh as they enjoy widespread testing as part of the KDE SIG.



> >From an OLPC PoV, major updates are rather troubling. We put together
> two OSs based on Fedora (XO and XS OSs). We test the integrated OS
> quite a bit ("we" being an outstanding team of volunteers around the
> world), and we override a few packages, some of them very tightly
> coupled to the platform (that is -- likely victims of subtle
> interactions that may go haywire on a major update).
>
> Given that, wearing my "XS lead dev" hat, I hope that Fedora focuses
> its "update" policy on the safe, sane, low-risk bugfixes. Otherwise,
> downstream projects that do careful QA are between a rock and a hard
> place -- between taking potentially exploding updates or ignoring
> updates altogether... and missing out on important bugfixes.
>
This is an incomplete look at the picture. If you stop large and complex
bugfix updates for fear of what they do to integration at the Fedora level,
then OLPC will still miss out on the important bugfixes in their derived
distros. If they pull in the bugfixes becausee they deem them worthy
enough, then why shouldn't Fedora have shipped them in the first place and
let OLPC exclude them?

The Fedora maintainers need to judge whether a large and complex update
makes the user's experience enough better (for instance, by fixing a large
number of bugs or enough important bugs) that they should perform the
update. OLPC maintainers can decide whether they have the same values as
the Fedora maintainer. In no case can either Fedora or the OLPC maintainer
abandon their evaluation of the risks and rewards of consuming updates --
it's just a matter of whether the downstream has to package the update
themselves or exclude the update that was provided.

-Toshio
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Proposed udpates policy change
 

On Mon, Mar 08, 2010 at 06:45:49PM -0500, Martin Langhoff wrote:
> On Mon, Mar 8, 2010 at 6:31 PM, Toshio Kuratomi <a.badger@gmail.com> wrote:
> >> 3) Sufficient testing of software inherently requires manual
> >> intervention by more than one individual.
> >>
> > This isn't entirely true either.
>
> #3 is so true that is central to what distros are about. Upstream
> probably released a good updated version, but the distro role is to do
> the integration, and shake out all the bugs in those subtle
> interactions between components.
>
> Otherwise, distros could provide the installer + @base and for the
> rest we could all grab RPMs from upstreams' websites.
>
> The QA of the integrated set of packages at particular release is hard
> and complex. That's what Fedora does, as a distro, and is central to
> what Fedora is, and the implicit social contract -- these components
> perform together in tune like a well-rehearsed orchestra.
>
> If the trombonist buys a new and shiny trombone, great, but for the
> piece he's playing tonight he'll have to play with the old trombone.
> The new trombone may be slightly out of tune, or louder, or tinnier;
> none of those things are bad per se, but it will ruin the combined
> effort.
>
> > One person can manually evaluate
>
> That's the "it works for me" attitude. Works for small software
> projects with a couple users. Not for a hugely complex OS, one that
> others use as the base for their own work.
>

If you had bothered to quote my complete proposal you would find that you
didn't have to bother writing your message. I'll pull a Jef and quote it:

> > This isn't entirely true either. One person can manually evaluate the
> > impact of certain changes to certain pieces of software. But this is less
> > of an issue than the first axiom as the number of packages that fit this
> > category is likely to be small.

You're willing to say that if I update one of my packages that has a script
of 30 lines, is a leaf node, and the update is to give the script an
optional argument to print output to stdout instead of writing to a file
that I'm incapable of building that package and then QA'ing the package from
the update-testing repository?

I'm specific that this is not a major problem because the number of packages
that can fall into this category is small. But #3 is not a sterling example
of an axiom as there are packages in the repository where small changes can
be applied and tested for regressions by a single person.

-Toshio
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Proposed udpates policy change
 

On Mon, Mar 8, 2010 at 7:39 PM, Toshio Kuratomi <a.badger@gmail.com> wrote:
> You're willing to say that if I update one of my packages that has a script
> of 30 lines, is a leaf node, and the update is to give the script an
> optional argument to print output to stdout instead of writing to a file
> that I'm incapable of building that package and then QA'ing the package from
> the update-testing repository?

Yes. And that is 0.001% of the package updates. In fact, skip the
noise of pushing that as an update, wait until something interesting
happens to roll it out.

There is no gain in rolling an rpm everytime. And there is a cost --
in many "cheap" resources (bandwidth, cpu burn in builders), and in
costly resources, like review time of your downstreams if they care
about their QA.

> But #3 is not a sterling example
> of an axiom

#3 is correct for 99.9% of the worthwhile package updates. Don't call
it an axiom if it bothers you to be unfair to a tiny portion of
updates.

But is a damn important point that is central to what a distro is.

cheers,


m
--
martin.langhoff@gmail.com
martin@laptop.org -- School Server Architect
- ask interesting questions
- don't get distracted with shiny stuff - working code first
- http://wiki.laptop.org/go/User:Martinlanghoff
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Proposed udpates policy change
 

On Mon, Mar 08, 2010 at 07:55:03PM -0500, Martin Langhoff wrote:
> On Mon, Mar 8, 2010 at 7:39 PM, Toshio Kuratomi <a.badger@gmail.com> wrote:
> > You're willing to say that if I update one of my packages that has a script
> > of 30 lines, is a leaf node, and the update is to give the script an
> > optional argument to print output to stdout instead of writing to a file
> > that I'm incapable of building that package and then QA'ing the package from
> > the update-testing repository?
>
> Yes. And that is 0.001% of the package updates. In fact, skip the
> noise of pushing that as an update, wait until something interesting
> happens to roll it out.
>
> There is no gain in rolling an rpm everytime. And there is a cost --
> in many "cheap" resources (bandwidth, cpu burn in builders), and in
> costly resources, like review time of your downstreams if they care
> about their QA.
>
But if someone requests this feature because they need to use it in their
environment, then the risk::reward ratio is low enough to justify it.

> > But #3 is not a sterling example
> > of an axiom
>
> #3 is correct for 99.9% of the worthwhile package updates. Don't call
> it an axiom if it bothers you to be unfair to a tiny portion of
> updates.
>
> But is a damn important point that is central to what a distro is.
>
And once again you've failed to quote the part of my message where I say
that this affects a small number of packages. Thanks.

-Toshio
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Proposed udpates policy change
 

On Mon, Mar 8, 2010 at 1:59 PM, Matthew Garrett <mjg59@srcf.ucam.org> wrote:
> This is the policy that I expect to be discussed during the Fesco
> meeting tomorrow. This is entirely orthogonal to the ongoing discussions
> regarding whether updates in stable releases should be expected to
> provide features or purely bugfixes, and I don't see any conflict in
> introducing it before those discussions have concluded.
>
> Introduction
> ------------
>
> We assume the following axioms:
>
> 1) Updates to stable that result in any reduction of functionality to
> the user are unacceptable.
>
> 2) It is impossible to ensure that functionality will not be reduced
> without sufficient testing.
>
> 3) Sufficient testing of software inherently requires manual
> intervention by more than one individual.

Hmm. So. I have a package, perl-Moose, that has 4,667 tests run at
build time. It depends on perl-Class-MOP, which has 2,225 tests, and
it in turn depends on perl, which has 234,776 tests run at build. On
a future note, we're working on setting up smoke testing, so when we,
say, rebuild perl-Class-MOP we also run perl-Moose's tests.

If I rebuild perl-Moose, or really, any of these packages, what sort
of manual testing would you suggest we require before pushing the
update?

-Chris
--
Chris Weyl
Ex astris, scientia
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel

Proposed udpates policy change
 

On 03/08/2010 11:45 PM, Michael Schwendt wrote:
> On Mon, 8 Mar 2010 23:21:45 +0100, Sven wrote:
>
>> On Mon, Mar 08, 2010 at 09:59:29PM +0000, Matthew Garrett wrote:
>>
>>> Before being added to updates, the package must receive a net karma of
>>> +3 in Bodhi.
>>
>> [...]
>>
>>> It is the expectation of Fesco that the majority of updates should
>>> easily be able to garner the necessary karma in a minimal space of time.
>>
>> I don't know what to say.
>
> Right now (and after sending my early reply), I feel dizzy. I hope to
> not take another look at the fedora devel list folder until tomorrow
> when it will contain hundreds of message. After those monster threads
> last week, now this.
>
>> If Fesco is aiming at getting rid of all the pesky packagers maintaining low
>> profile packages: You're well on your way.
>
> Well said.

Yes.

As others already said, the axioms this proposal is based on are wrong,
the conclusions are wrong, the technology being applied is flawed (karma
votes), ... this whole proposal is insane.

> Also important, Matthew even fills a FESCo seat. Proposals like that are not
> what I expect from FESCo members.

This begs for a question: How many people participating to Fedora does
this FESCO represent? IIRC, the last election had a very low
participation, which in many democratic elections would have meant the
elections to have missed some quorums and the elections to be invalid.

> I'm severely disappointing.

Ditto.

Ralf




Ralf

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel