FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 10-31-2010, 06:18 AM
Michael Schwendt
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

On Sun, 31 Oct 2010 04:37:38 +0100, Kevin wrote:

> Martin Stransky wrote:
> > there's a new Firefox update waiting in Bodhi and we can't push it to
> > stable because of new rules. We recommend you to update to it ASAP as it
> > fixes a public critical 0day vulnerability
> > (https://bugzilla.mozilla.org/show_bug.cgi?id=607222).
>
> Looks like the F13 build got karma quickly enough to land directly in stable
> after all, the F12 build, on the other hand, was stuck in testing for 2 days
> before finally making it out to stable. Yet another blatant example of
> failure of the Update Acceptance Criteria, needlessly exposing our users to
> critical vulnerabilities.
>
> (And no, by giving yet another special exception to Firefox wouldn't be a
> solution. ;-) This problem can hit any other app as well.)
>
> Kevin Kofler

Okay, feedback time.

Lately, there have been several attempts at urging proventesters (and not
just testers in general) to give positive karma for aging critpath updates.
It also has been decided by someone (or maybe even a comittee) to spam
proventesters daily with "[old_testing_critpath]" messages for all three
dist releases, with no day to unsubscribe from that (other than leaving
proventesters group, which is what at least one person has threatened with,
or filtering those messages).

Dunno about other testers (and there aren't many yet), but I have abandoned
F-12 long ago due to lack of time when F-13 became the one to use on a daily
basis. And some time before F-14 Beta, my desktop has been switched to boot
F-14 by default. That's the only opportunity to evaluate F-14 early and
possibly find issues prior to its release. On the contrary, most of Fedora's
users will wait for the final release, and many users will wait even longer.
It's highly likely that bugzilla can confirm that.

F-14 is the the only way forward, and don't like to spend time on F-13 and
older anymore. That also applies to packagers I maintain or monitor. I simply
don't see the user base [target group] anymore.

About positive karma in bodhi, I don't feel comfortable signing off
arbitrary updates just because they didn't crash for me after five
minutes. With some updates, regression has slipped through already.
And the more bugs an update addresses with either patches or a version
upgrade, the more careful I would like to be when testing something.
Also, in my book, an update working on F-14 may still malfunction on an
older dist release due to differences in dependences and the core setup. I
still don't understand why some non-security updates are rushed out with
sometimes not even the package maintainer(s) having tested them at all.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 11-01-2010, 12:06 AM
Adam Williamson
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

On Sun, 2010-10-31 at 04:37 +0100, Kevin Kofler wrote:
> Martin Stransky wrote:
> > there's a new Firefox update waiting in Bodhi and we can't push it to
> > stable because of new rules. We recommend you to update to it ASAP as it
> > fixes a public critical 0day vulnerability
> > (https://bugzilla.mozilla.org/show_bug.cgi?id=607222).
>
> Looks like the F13 build got karma quickly enough to land directly in stable
> after all, the F12 build, on the other hand, was stuck in testing for 2 days
> before finally making it out to stable. Yet another blatant example of
> failure of the Update Acceptance Criteria, needlessly exposing our users to
> critical vulnerabilities.

Kevin, could you *please* not word things like that? There's just no
need for it.

I already wrote this to -test a couple of days ago:

http://lists.fedoraproject.org/pipermail/test/2010-October/095135.html

and we're discussing it there. I think the thread demonstrates things
tend to go much more constructively if you avoid throwing words like
'blatant' and 'failure' and 'needlessly' around. We designed a policy,
put it into effect, now we're observing how well it works and we can
modify its implementation on the fly. It doesn't need to be done in an
adversarial spirit.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 11-01-2010, 12:18 AM
Miloslav Trmač
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

Adam Williamson p*še v Ne 31. 10. 2010 v 18:06 -0700:
> On Sun, 2010-10-31 at 04:37 +0100, Kevin Kofler wrote:
> Yet another blatant example of
> > failure of the Update Acceptance Criteria, needlessly exposing our users to
> > critical vulnerabilities.
>
> Kevin, could you *please* not word things like that? There's just no
> need for it.
>
> I already wrote this to -test a couple of days ago:
>
> http://lists.fedoraproject.org/pipermail/test/2010-October/095135.html
>
> and we're discussing it there. I think the thread demonstrates things
> tend to go much more constructively if you avoid throwing words like
> 'blatant' and 'failure' and 'needlessly' around.
Did we not fail our users? Was there a real need to fail them? (As a
non-native speaker, I won't judge the relative merits of "blatant" vs.
"sucks".)

> We designed a policy,
> put it into effect, now we're observing how well it works and we can
> modify its implementation on the fly. It doesn't need to be done in an
> adversarial spirit.
Given that _this exact scenario_ was repeatedly brought up since the
very start of the update acceptance criteria proposals, I think some
frustration is quite justified. This situation is not really a
surprise, and Fedora did not have to unnecessarily expose users to a
vulnerability in order to relearn this lesson.

In addition to being constructive about remedying the situation,
shouldn't we try to be more constructive about _not introducing such
situations_ in the first place?
Mirek

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 11-01-2010, 04:08 PM
Adam Williamson
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

On Mon, 2010-11-01 at 02:18 +0100, Miloslav Trmač wrote:

> > Kevin, could you *please* not word things like that? There's just no
> > need for it.
> >
> > I already wrote this to -test a couple of days ago:
> >
> > http://lists.fedoraproject.org/pipermail/test/2010-October/095135.html
> >
> > and we're discussing it there. I think the thread demonstrates things
> > tend to go much more constructively if you avoid throwing words like
> > 'blatant' and 'failure' and 'needlessly' around.
> Did we not fail our users? Was there a real need to fail them? (As a
> non-native speaker, I won't judge the relative merits of "blatant" vs.
> "sucks".)

I didn't say that what Kevin said was *wrong*, I said it wasn't the best
way to word it.

> > We designed a policy,
> > put it into effect, now we're observing how well it works and we can
> > modify its implementation on the fly. It doesn't need to be done in an
> > adversarial spirit.
> Given that _this exact scenario_ was repeatedly brought up since the
> very start of the update acceptance criteria proposals, I think some
> frustration is quite justified. This situation is not really a
> surprise, and Fedora did not have to unnecessarily expose users to a
> vulnerability in order to relearn this lesson.

On the other hand, other scenarios were also brought up, which have not
come to pass - for instance, the same thing happening to Fedora 13 or
Fedora 14. If we had simply accepted the predictions of doom and not
implemented the policy at all, we would be without its benefits for the
development of F13 and F14.

> In addition to being constructive about remedying the situation,
> shouldn't we try to be more constructive about _not introducing such
> situations_ in the first place?
> Mirek

Saying 'oh dear, this might not work, we'd better not try' is rarely a
good approach, IMHO. It's better to try things, with the proviso that
you accept when they aren't working and withdraw or modify them.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 11-01-2010, 04:09 PM
Adam Williamson
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

On Mon, 2010-11-01 at 03:54 +0100, Kevin Kofler wrote:

> There's exactly one constructive thing to do, it's repealing this set of
> policies (Critical Path and Update Acceptance Criteria) in its entirety.
>
> An update should go stable when the maintainer says so, karma should be
> purely informational feedback for the maintainer.

I disagree. The evidence you cite does not support this conclusion. We
implemented the policies for three releases. There are significant
problems with one release. This does not justify the conclusion that the
policies should be entirely repealed.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 11-01-2010, 04:29 PM
Miloslav Trmač
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

Adam Williamson p*še v Po 01. 11. 2010 v 10:08 -0700:
> > > We designed a policy,
> > > put it into effect, now we're observing how well it works and we can
> > > modify its implementation on the fly. It doesn't need to be done in an
> > > adversarial spirit.
> > Given that _this exact scenario_ was repeatedly brought up since the
> > very start of the update acceptance criteria proposals, I think some
> > frustration is quite justified. This situation is not really a
> > surprise, and Fedora did not have to unnecessarily expose users to a
> > vulnerability in order to relearn this lesson.
>
> On the other hand, other scenarios were also brought up, which have not
> come to pass - for instance, the same thing happening to Fedora 13 or
> Fedora 14. If we had simply accepted the predictions of doom and not
> implemented the policy at all, we would be without its benefits for the
> development of F13 and F14.
A problem with this line of argument is that the benefits are not quite
apparent to me.

> > In addition to being constructive about remedying the situation,
> > shouldn't we try to be more constructive about _not introducing such
> > situations_ in the first place?

> Saying 'oh dear, this might not work, we'd better not try' is rarely a
> good approach, IMHO.
That is a cost-benefit comparison. "New" does not imply "improved".

> It's better to try things, with the proviso that
> you accept when they aren't working and withdraw or modify them.
It's even better not to dismiss known problems with the policy, and to
make sure the policy can handle them properly from the start. This was
not a surprise, this was an "unforced error".
Mirek

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 11-01-2010, 04:39 PM
Adam Williamson
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

On Mon, 2010-11-01 at 18:29 +0100, Miloslav Trmač wrote:

> > On the other hand, other scenarios were also brought up, which have not
> > come to pass - for instance, the same thing happening to Fedora 13 or
> > Fedora 14. If we had simply accepted the predictions of doom and not
> > implemented the policy at all, we would be without its benefits for the
> > development of F13 and F14.

> A problem with this line of argument is that the benefits are not quite
> apparent to me.

The policies prevented us from shipping a number of completely broken
updates, which is exactly what they were intended to do. I don't have a
command handy to do a search for rejected proposed critpath updates for
F14, but if you figure it out, you can see the precise results of the
policy there.

> > > In addition to being constructive about remedying the situation,
> > > shouldn't we try to be more constructive about _not introducing such
> > > situations_ in the first place?
>
> > Saying 'oh dear, this might not work, we'd better not try' is rarely a
> > good approach, IMHO.
> That is a cost-benefit comparison. "New" does not imply "improved".

We had an extensive discussion about the benefits of testing important
updates at the time the policy went into effect. I don't think it's
really necessary to re-hash the entire thing. For the record, I did not
say nor do I believe that "new" inevitably implies "improved".

> > It's better to try things, with the proviso that
> > you accept when they aren't working and withdraw or modify them.
> It's even better not to dismiss known problems with the policy, and to
> make sure the policy can handle them properly from the start. This was
> not a surprise, this was an "unforced error".

Sorry, but characterizing it as a 'known problem' is misleading. It's
easy to forecast failure, and you'll likely always be correct in *some*
cases if you forecast enough failures. Only if you precisely forecast
only the failures that actually happen, and do not forecast any failures
that don't happen, can your forecast be considered truly reliable. If
this had truly been a 'known problem' then those who predicted it would
also have correctly chosen *not* to predict failure in the case of
Fedora 13 and Fedora 14. The fact is that they did predict a failure
which has not, in fact, come to pass (neither F13 nor F14 have long
queues of old critpath updates).
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 11-01-2010, 04:51 PM
Miloslav Trmač
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

Adam Williamson p*še v Po 01. 11. 2010 v 10:39 -0700:
> On Mon, 2010-11-01 at 18:29 +0100, Miloslav Trmač wrote:
> > > It's better to try things, with the proviso that
> > > you accept when they aren't working and withdraw or modify them.
> > It's even better not to dismiss known problems with the policy, and to
> > make sure the policy can handle them properly from the start. This was
> > not a surprise, this was an "unforced error".
>
> Sorry, but characterizing it as a 'known problem' is misleading. It's
> easy to forecast failure, and you'll likely always be correct in *some*
> cases if you forecast enough failures. Only if you precisely forecast
> only the failures that actually happen, and do not forecast any failures
> that don't happen, can your forecast be considered truly reliable.
The accuracy of prediction, and especially accuracy of the timing, is
not at all relevant. In fact, it is _preciselly_ the unknown nature of
risks that requires thinking about them in advance.

People don't wear helmets because they know when something will hit
their head, but because they don't know when, or even if, it will.
Mirek

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 11-01-2010, 04:55 PM
Adam Williamson
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

On Mon, 2010-11-01 at 18:51 +0100, Miloslav Trmač wrote:

> > Sorry, but characterizing it as a 'known problem' is misleading. It's
> > easy to forecast failure, and you'll likely always be correct in *some*
> > cases if you forecast enough failures. Only if you precisely forecast
> > only the failures that actually happen, and do not forecast any failures
> > that don't happen, can your forecast be considered truly reliable.

> The accuracy of prediction, and especially accuracy of the timing, is
> not at all relevant. In fact, it is _preciselly_ the unknown nature of
> risks that requires thinking about them in advance.

Which rather contradicts your description of it as a 'known problem',
yes?

> People don't wear helmets because they know when something will hit
> their head, but because they don't know when, or even if, it will.
> Mirek

That's not really a relevant analogy. You can't 'wear a helmet' in this
case. There's no way we could have implemented the policy and 'worn a
helmet' by allowing updates to happen without the conditions of the
policy being fulfilled; that would effectively negate the policy.

If you want to continue with the analogy, what you seem to be saying is
that we should never have implemented the policy in the first place,
which is not analogous to wearing a helmet; it's analogous to never
leaving the house just in case something hits you on the head.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 11-01-2010, 05:01 PM
Miloslav Trmač
 
Default The new Update Acceptance Criteria are broken (was: Heads Up - New Firefox update)

Adam Williamson p*še v Po 01. 11. 2010 v 10:55 -0700:
> On Mon, 2010-11-01 at 18:51 +0100, Miloslav Trmač wrote:
> > > Sorry, but characterizing it as a 'known problem' is misleading. It's
> > > easy to forecast failure, and you'll likely always be correct in *some*
> > > cases if you forecast enough failures. Only if you precisely forecast
> > > only the failures that actually happen, and do not forecast any failures
> > > that don't happen, can your forecast be considered truly reliable.
>
> > The accuracy of prediction, and especially accuracy of the timing, is
> > not at all relevant. In fact, it is _preciselly_ the unknown nature of
> > risks that requires thinking about them in advance.
>
> Which rather contradicts your description of it as a 'known problem',
> yes?
No; the existence of the problem was known, only the timing and precise
extent was not.


> If you want to continue with the analogy, what you seem to be saying is
> that we should never have implemented the policy in the first place,
That is one option; another would be adding a "I'm the maintainer and I
really mean it" checkbox for security updates (with FESCo/Fedora
QA/somebody else reviewing the cases retrospectively, if they feel like
it); yeat another is not enforcing the policy on security updates at
all, as I seem to remember was proposed (or even implemented?) at one
time.
Mirek

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 11:45 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org