FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 12-01-2010, 08:22 PM
Doug Ledford
 
Default old_testing_critpath notifications

On 12/01/2010 01:41 PM, Adam Williamson wrote:
> On Wed, 2010-12-01 at 13:23 -0500, Doug Ledford wrote:
>
>> That being said, F14 went out with a broken mdadm *purely* because of
>> this policy.
>
>> Evidently my update was approved somewhere along the way, but because of
>> the volume of bodhi spam I get, I missed it.
>
> ...so what you're saying is that F14 did not in fact go out with a
> broken mdadm *purely* because of the policy, but in a small part because
> of the policy and in a large part because you don't read / filter your
> emails carefully enough.

Um, no. It's because of the policy, period.

>> So I'm not sure if it
>> could have made F14 final or not, but I know it didn't because I was
>> working on other things at the time.
>
> bodhi - 2010-10-14 22:36:08
> Critical path update approved
>
> The final change deadline was 10-18; you had four days to push the
> update.

Oh my, 4 whole days you say? Let's see, the initial bodhi ticket was
filed on 2010-08-05, it was initially pushed to testing on 2010-08-10
(so five days just to push), an automated bug telling me mdadm-3.1.4 had
been released upstream was filed on 2010-09-13, this package was
approved for stable on 2010-10-14, and final drop dead push date was
2010-10-18. If the initial push was any indication, it would have
missed the final release by a day even if I pushed it the second it
became approved! But I would have been pushing an outdated package to boot!

I'm sorry, but no, you don't get to say ONE DAMN WORD about me not
getting it pushed in a 4 day window when the ticket itself took 60+ days
to get approved! And that was after I put out a general request for
testing of the update on devel@ in order to try and drum up testing.

If the ticket can be allowed to languish that long, then I don't feel in
the least bit guilty that I didn't drop my other Red Hat
responsibilities on the floor when the ticket was finally approved. By
the time it was approved, I had already moved on. Fortunately, I wasn't
under the same restrictions on Red Hat Enterprise Linux 6, so it went GA
with a much better, fixed mdadm relative to Fedora.

--
Doug Ledford <dledford@redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford

Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-01-2010, 08:34 PM
Luke Macken
 
Default old_testing_critpath notifications

On Mon, Nov 29, 2010 at 01:36:18PM +0000, Petr Pisar wrote:
> On 2010-11-29, Peter Robinson <pbrobinson@gmail.com> wrote:
> > On Mon, Nov 29, 2010 at 9:56 AM, Petr Pisar <ppisar@redhat.com> wrote:
> >
> > Proven testers do get copies of these emails (dozens of them) and its
> > also summarised in the updates-testing report for all to see.
> >
> Oh, I thought <test@l.f.o.> described as `For testers of Fedora
> development releases' concerns alpha and similar (pre-)releases only.
>
> In that case, only one issue remains: to stop spamming package
> maintainer.

The next release of bodhi that I'm going to push out to our releng
servers this week will stop spamming proventesters and maintainers about
these, since this data is now in the updates-testing digests that get
sent out to the test-list with each push.

luke
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-01-2010, 08:35 PM
Adam Williamson
 
Default old_testing_critpath notifications

On Wed, 2010-12-01 at 16:22 -0500, Doug Ledford wrote:

> If the ticket can be allowed to languish that long, then I don't feel in
> the least bit guilty that I didn't drop my other Red Hat
> responsibilities on the floor when the ticket was finally approved. By
> the time it was approved, I had already moved on.

I didn't say you should. I said your assertion that F14 went out with an
older mdadm '*purely* because of this policy' was not supported by
evidence. The package could have been included in F14 had it been pushed
soon after critpath testing approval was granted. Therefore, it's simply
not correct that the policy alone prevented it being included. (If you'd
submitted the update to stable close to the final date it wouldn't have
taken five days to be pushed; rel-eng do pushes far more often when it's
close to a deadline).

> Fortunately, I wasn't
> under the same restrictions on Red Hat Enterprise Linux 6, so it went GA
> with a much better, fixed mdadm relative to Fedora.

The implied comparison here is unfair and hence not helpful; RHEL has a
large group of paid test staff. Fedora does not.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-01-2010, 08:40 PM
Luke Macken
 
Default old_testing_critpath notifications

On Wed, Dec 01, 2010 at 10:41:20AM -0800, Adam Williamson wrote:
> On Wed, 2010-12-01 at 13:23 -0500, Doug Ledford wrote:
>
> > That being said, F14 went out with a broken mdadm *purely* because of
> > this policy.
>
> > Evidently my update was approved somewhere along the way, but because of
> > the volume of bodhi spam I get, I missed it.
>
> ...so what you're saying is that F14 did not in fact go out with a
> broken mdadm *purely* because of the policy, but in a small part because
> of the policy and in a large part because you don't read / filter your
> emails carefully enough.
>
> > So I'm not sure if it
> > could have made F14 final or not, but I know it didn't because I was
> > working on other things at the time.
>
> bodhi - 2010-10-14 22:36:08
> Critical path update approved
>
> The final change deadline was 10-18; you had four days to push the
> update.

Also, if karma automatism was enabled for that update, it would have been
queued for pushing right when it was approved.

luke
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-01-2010, 08:49 PM
Doug Ledford
 
Default old_testing_critpath notifications

On 12/01/2010 04:40 PM, Luke Macken wrote:
> On Wed, Dec 01, 2010 at 10:41:20AM -0800, Adam Williamson wrote:
>> On Wed, 2010-12-01 at 13:23 -0500, Doug Ledford wrote:
>>
>>> That being said, F14 went out with a broken mdadm *purely* because of
>>> this policy.
>>
>>> Evidently my update was approved somewhere along the way, but because of
>>> the volume of bodhi spam I get, I missed it.
>>
>> ...so what you're saying is that F14 did not in fact go out with a
>> broken mdadm *purely* because of the policy, but in a small part because
>> of the policy and in a large part because you don't read / filter your
>> emails carefully enough.
>>
>>> So I'm not sure if it
>>> could have made F14 final or not, but I know it didn't because I was
>>> working on other things at the time.
>>
>> bodhi - 2010-10-14 22:36:08
>> Critical path update approved
>>
>> The final change deadline was 10-18; you had four days to push the
>> update.
>
> Also, if karma automatism was enabled for that update, it would have been
> queued for pushing right when it was approved.
>
> luke

I don't enable karma automatism because in the past I've seen people
report testing karma +1 when they did not, in fact, doing anything
useful in terms of testing (aka, they had no software raid devices, yet
they said the system still works...well, duh, it's only used on software
raid devices so if you don't have any, then it doesn't make any
difference to you).

--
Doug Ledford <dledford@redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford

Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-01-2010, 08:54 PM
Luke Macken
 
Default old_testing_critpath notifications

On Wed, Dec 01, 2010 at 04:49:07PM -0500, Doug Ledford wrote:
> On 12/01/2010 04:40 PM, Luke Macken wrote:
> > On Wed, Dec 01, 2010 at 10:41:20AM -0800, Adam Williamson wrote:
> >> On Wed, 2010-12-01 at 13:23 -0500, Doug Ledford wrote:
> >>
> >>> That being said, F14 went out with a broken mdadm *purely* because of
> >>> this policy.
> >>
> >>> Evidently my update was approved somewhere along the way, but because of
> >>> the volume of bodhi spam I get, I missed it.
> >>
> >> ...so what you're saying is that F14 did not in fact go out with a
> >> broken mdadm *purely* because of the policy, but in a small part because
> >> of the policy and in a large part because you don't read / filter your
> >> emails carefully enough.
> >>
> >>> So I'm not sure if it
> >>> could have made F14 final or not, but I know it didn't because I was
> >>> working on other things at the time.
> >>
> >> bodhi - 2010-10-14 22:36:08
> >> Critical path update approved
> >>
> >> The final change deadline was 10-18; you had four days to push the
> >> update.
> >
> > Also, if karma automatism was enabled for that update, it would have been
> > queued for pushing right when it was approved.
> >
> > luke
>
> I don't enable karma automatism because in the past I've seen people
> report testing karma +1 when they did not, in fact, doing anything
> useful in terms of testing (aka, they had no software raid devices, yet
> they said the system still works...well, duh, it's only used on software
> raid devices so if you don't have any, then it doesn't make any
> difference to you).

Yep, that happens. There are also people that add +0 comments to
updates saying "Untested". There is an obvious need for more
fine-grained karma types.

luke
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-01-2010, 08:55 PM
Doug Ledford
 
Default old_testing_critpath notifications

On 12/01/2010 04:35 PM, Adam Williamson wrote:
> On Wed, 2010-12-01 at 16:22 -0500, Doug Ledford wrote:
>
>> If the ticket can be allowed to languish that long, then I don't feel in
>> the least bit guilty that I didn't drop my other Red Hat
>> responsibilities on the floor when the ticket was finally approved. By
>> the time it was approved, I had already moved on.
>
> I didn't say you should. I said your assertion that F14 went out with an
> older mdadm '*purely* because of this policy' was not supported by
> evidence.

It *was* purely because of this policy. 60 days after I file a bodhi
ticket, I'm done working on whatever I was working on at the time. Even
if I had seen the approval notification, I wouldn't have pushed it with
only 4 days until freeze. Not when I've completely forgotten what was
in the package.

> The package could have been included in F14 had it been pushed
> soon after critpath testing approval was granted. Therefore, it's simply
> not correct that the policy alone prevented it being included. (If you'd
> submitted the update to stable close to the final date it wouldn't have
> taken five days to be pushed; rel-eng do pushes far more often when it's
> close to a deadline).
>
>> Fortunately, I wasn't
>> under the same restrictions on Red Hat Enterprise Linux 6, so it went GA
>> with a much better, fixed mdadm relative to Fedora.
>
> The implied comparison here is unfair and hence not helpful; RHEL has a
> large group of paid test staff. Fedora does not.

The comparison is 100% fair because it points out the fundamental
problem with the current policy: if you don't have a paid staff of
testers to make sure testing is done in a timely fashion, then you have
absolutely no business gating updates on a testing staff that doesn't
exist. It's nice in theory to think we can force testing of updates
prior to their release, but if the testing staff simply isn't there,
then you aren't improving the product, you're just stopping progress.

--
Doug Ledford <dledford@redhat.com>
GPG KeyID: CFBFF194
http://people.redhat.com/dledford

Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-01-2010, 09:02 PM
Adam Williamson
 
Default old_testing_critpath notifications

On Wed, 2010-12-01 at 16:54 -0500, Luke Macken wrote:

> Yep, that happens. There are also people that add +0 comments to
> updates saying "Untested". There is an obvious need for more
> fine-grained karma types.

I've sent out notes to the test list to ask people not to do either of
those things in future, I think the occurrence of them has gone down
significantly since those emails went out. (I also updated the proven
tester instructions page).

More fine-grained karma is still coming in Bodhi 2.0, right? My
understanding was that we'd already agreed on the framework for it (in
those threads on this list a few months back) and we were just waiting
for the implementation now.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-01-2010, 09:17 PM
Adam Williamson
 
Default old_testing_critpath notifications

On Wed, 2010-12-01 at 16:55 -0500, Doug Ledford wrote:

> The comparison is 100% fair because it points out the fundamental
> problem with the current policy: if you don't have a paid staff of
> testers to make sure testing is done in a timely fashion, then you have
> absolutely no business gating updates on a testing staff that doesn't
> exist. It's nice in theory to think we can force testing of updates
> prior to their release, but if the testing staff simply isn't there,
> then you aren't improving the product, you're just stopping progress.

The gating is not on 'a testing staff'. The gating is on *testing*.

I want to say again that I'm not particularly wedded to the current
policy and I don't mind at all if it changes, but I think we need to be
careful of the mindset that says 'we can't enforce any standards in
Fedora because it's a volunteer project so we must just accept what
people are willing to give us'.

Even though packaging in Fedora is a volunteer process, we still have
fairly rigorous packaging guidelines and a review process. We don't just
accept any package someone turns up and submits. i.e., we're enforcing
standards of quality, despite this being an entirely volunteer effort
and no-one being compelled to show up and provide packages of a
particular quality.

The concept of having a policy requiring updates to be tested before
they're issued is really no different. I think one point where we've
fallen over is that it wasn't sufficiently well discussed / communicated
in advance that this testing wasn't just going to 'get done' by some
independent group and no-one else would have to worry about it, but
would require a lot of people to chip in. In the same way that there
isn't some separate independent group that does package reviews, it's
just all maintainers chipping in when they can. I think perhaps those
who supported and voted for the policy kind of assumed this would
happen, and many others weren't actually aware of it.

I do think that for update testing to work well going forward we need to
engage more groups with it and make it clear it's not something that
some separate QA group is just going to do for everyone and no-one has
to worry about it. We can get, and already have got, some enthusiastic
people to sign up to run updates-testing and provide testing feedback
for the packages they use anyway, but the concept of there being a
hardcore group of dedicated testers who will go out of their way to
install, configure and test software they wouldn't usually use is not
one that's likely to fly, I don't think.

When software is packaged it's reasonable to expect that someone,
somewhere, uses it; if they don't, it probably shouldn't be packaged. We
need to find those people and engage them in the testing process, and it
seems to me that the maintainers of packages are as well placed as
anyone to help find and engage their users in this process.

In many cases it's easier than that; a lot of packages are maintained by
more than one person. It's not only perfectly okay but more or less
*what we want to happen* for co-maintainers to sign up as proven testers
and test each others' updates. There's a bunch of people in the anaconda
group, for instance; it's perfectly fine for you all to sign up as
proven testers and test each other's code. The testing doesn't have to
come from some impartial outside body, all we need is a sanity check.

I don't really see any reason why *everyone* who's a packager shouldn't
also have signed up to be a proven tester by now. I'd like to ask if
anyone has a perception that it's a hard process to get involved in, or
if they got the impression that they *shouldn't* get engaged in it, or
something like that. Maybe we can improve the presentation to make it
clear that this really ought to be a very wide-based process.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 12-01-2010, 09:53 PM
"Nathanael D. Noblet"
 
Default old_testing_critpath notifications

On 12/01/2010 03:17 PM, Adam Williamson wrote:
> I don't really see any reason why *everyone* who's a packager shouldn't
> also have signed up to be a proven tester by now. I'd like to ask if
> anyone has a perception that it's a hard process to get involved in, or
> if they got the impression that they *shouldn't* get engaged in it, or
> something like that. Maybe we can improve the presentation to make it
> clear that this really ought to be a very wide-based process.

Well I never read anything specifically about the requirements, however
based of the name alone, 'proven tester' and relating it to 'proven
packager' I assumed I'd need to be more experienced before I signed up.

Also, I don't find the tools for updates-testing particularly friendly
enough yet. I wrote in a thread awhile ago what I thought could be very
useful to entice people to use updates-testing. I know there are some
tools which together would allow me to do this, however I'm looking for
a very simple, comprehensive tool that shows me what is in updates
testing, what I've installed from there, or create a list of packages
I'm interested in in updates-testing and never show me otherwise etc...

I'm not saying I won't test without it - however I wouldn't be able to
give dedicated time. I test updates I push out, I test updates for bugs
I've filed. Otherwise I don't have time to look at anything else. I
include the thread as reference only. It seemed someone in that thread
did say they were looking at building just such a tool... Anyway. I want
to help out more, but I'm so busy with $realjob that I need a very easy
way to see updates I'm willing to test, pull them back if something goes
wrong, and submit karma quickly from one simple place...

http://lists.fedoraproject.org/pipermail/devel/2010-June/138077.html
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 04:10 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org