FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Kernel Team

 
 
LinkBack Thread Tools
 
Old 10-09-2012, 01:19 PM
Tim Flink
 
Default Review of Fedora 18 Release Criteria

As we're getting closer to the scheduled time for beta freeze, we'd
like to find out now if any of the current criteria or proposed criteria
changes are unreasonable to expect for beta. There may be more changes
for final as we get closer to that but I think that we're pretty close
to being done with the release requirements for beta.

The current (as of this writing) release criteria are available at:
- http://fedoraproject.org/wiki/Fedora_18_Alpha_Release_Criteria
- http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria
- http://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria

There have been several changes to the release criterion lately (mostly
due to changes in anaconda's functionality) and there are still a
couple of modifications pending (identified at the end of this email
with links to the most recent versions of the proposal)

The questions we're interested in are:
- Is there anything in the current criteria or proposed revisions that
seem unreasonable?
- Are there any areas where the criteria are lacking?

Thanks,

Tim


Final
=====

### mediacheck (change)
" All release media must include a standalone memory test utility. A
boot menu option to launch this utility must be present and must work
correctly "

to:

"If there is an embedded checksum in the image, it must match. If there
is a related UI element displayed after booting the image, it must work
and display the correct result."

http://lists.fedoraproject.org/pipermail/test/2012-September/110415.html

### network attached storage (pass for F18)
Remove for F18 (will be back in F19)

'The installer must be able to complete an installation using any
network-attached storage devices (e.g. iSCSI, FCoE, Fibre Channel)'

http://lists.fedoraproject.org/pipermail/test/2012-July/109122.html
_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 10-09-2012, 08:48 PM
David Cantrell
 
Default Review of Fedora 18 Release Criteria

On Tue, Oct 09, 2012 at 07:19:25AM -0600, Tim Flink wrote:
> As we're getting closer to the scheduled time for beta freeze, we'd
> like to find out now if any of the current criteria or proposed criteria
> changes are unreasonable to expect for beta. There may be more changes
> for final as we get closer to that but I think that we're pretty close
> to being done with the release requirements for beta.
>
> The current (as of this writing) release criteria are available at:
> - http://fedoraproject.org/wiki/Fedora_18_Alpha_Release_Criteria
> - http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria
> - http://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria

I would like to see changes to the blocker criteria for each release. The
first item on each release criteria is that all blockers must be CLOSED.
Blockers are determined by criteria defined below which always group
anaconda in because we cannot address those problems in a later update
release. This gets us on the bug fixing treadmill as we edge closer to each
release because every anaconda bug more or less becomes a blocker. What I
would like to see:

1) Installer blocker criteria needs to be more and more restrictive the
closer we get to a final release.

2) Installer blockers should only be granted when there is no other way to
accomplish the same task during installation. For example, if FCoE
configuration does not always work in the UI but does work when passed boot
parameters or via kickstart, we shouldn't consider it a blocker. It's an
unfortunate bug, but as described there is an install path for those users.

3) Ultimately we want the number of granted blockers to be lower and lower
from alpha to beta to rc.

--
David Cantrell <dcantrell@redhat.com>
Manager, Installer Engineering Team
Red Hat, Inc. | Westford, MA | EST5EDT

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 10-09-2012, 11:07 PM
Adam Williamson
 
Default Review of Fedora 18 Release Criteria

On Tue, 2012-10-09 at 16:48 -0400, David Cantrell wrote:
> On Tue, Oct 09, 2012 at 07:19:25AM -0600, Tim Flink wrote:
> > As we're getting closer to the scheduled time for beta freeze, we'd
> > like to find out now if any of the current criteria or proposed criteria
> > changes are unreasonable to expect for beta. There may be more changes
> > for final as we get closer to that but I think that we're pretty close
> > to being done with the release requirements for beta.
> >
> > The current (as of this writing) release criteria are available at:
> > - http://fedoraproject.org/wiki/Fedora_18_Alpha_Release_Criteria
> > - http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria
> > - http://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria

Thanks David! Some thoughts follow.

> I would like to see changes to the blocker criteria for each release. The
> first item on each release criteria is that all blockers must be CLOSED.
> Blockers are determined by criteria defined below which always group
> anaconda in because we cannot address those problems in a later update
> release. This gets us on the bug fixing treadmill as we edge closer to each
> release because every anaconda bug more or less becomes a blocker.

This paragraph was a bit tricky to read, but now I've given it a few
tries, it seems to be more or less a preamble, yes? I'm not sure if
you're suggesting that "Blockers are determined by criteria defined
below which always group anaconda in because we cannot address those
problems in a later update release. This gets us on the bug fixing
treadmill as we edge closer to each release because every anaconda bug
more or less becomes a blocker." is a problem, or just mentioning it as
background. It's perfectly true as background, but I don't see it as a
problem: it's just an innate characteristic of the software you write.
The installer is something that cannot be updated (for practical
purposes), it must work to a high standard as shipped, because if it
doesn't, that's a much bigger problem than a component which _can_ be
updated not working. I agree with your assessment, but I see it as an
inherent characteristic of an operating system installer, not any kind
of problem in the process.

> What I
> would like to see:
>
> 1) Installer blocker criteria needs to be more and more restrictive the
> closer we get to a final release.

This wasn't entirely clear to me, but I'm going to take a guess at what
I think you mean and reply to that. I think you're looking at the
situation where we get late into the validation process - say, we just
built RC2 and it's two days to go/no-go - and we find five bugs and mark
them as blockers. I'm guessing you're saying it'd be preferable to
identify blockers early and we should only add issues to the blocker
list late if they're _really bad_, because otherwise you just keep
fixing blockers. On that basis...

I appreciate that the 'blocker treadmill', as you describe it, can be
frustrating. But I don't think 'let's just not count bad issues as
blockers late in the process to give the developers a break' is the
answer to anything (except possibly 'how can we stop Will depleting the
U.S. strategic gin reserve?', but that's not the question this post was
trying to answer :>). What we're trying to do with the release
validation process as a whole is provide a clear framework for defining
the standards our releases should meet and a clear process for building
releases that meet those standards and verifying that they meet those
standards. I don't see that adding an element of time sensitivity to the
blocker evaluation process - 'issues of the type X are blockers if we
find them four weeks before release, but not if we find them one week
before release' - is a good way to achieve this.

'Blocker bugs' are just the 'release quality' question inverted: they
are the ways in which our releases must not be broken in order to meet
the minimum quality standards we've decided on. An issue which causes us
to fall below our minimum quality standards is a problem no matter when
it's discovered. I absolutely understand that it makes things easier for
the developers if we catch blocker bugs early, and we agree this is an
important goal and we have made and will continue to make efforts to
improve our ability to catch blockers as early as possible. I know it
sucks when we're on RC3 and we suddenly discover a major bug. But it's
still a major bug, and 'say it's not a blocker because we're late in the
process' doesn't sound like a good response to that suckage, to me. I
don't want to do that.

I believe we should set realistic minimum standards - those that are
achievable with the level of development resources we have in place, on
the release schedules Fedora is committed to. What this thread is about,
essentially, is checking that we are not currently setting that bar too
high, and demanding from you more than you have the resources to
possibly provide in the time available. We certainly believe that we
need input from the development teams to know where the bar should be
set. But I do believe the bar should be a bar, not a fuzzy field that
can be adjusted with excessive pragmatism. We should set realistic
standards, but they have to be solid ones that we don't compromise just
because time is short or the developers are getting tired of fixing
bugs.

What we (QA) as a team do try and do in those cases is look at the
situation and think what we could do in future to ensure the blocker
would get caught earlier. For instance, in the last few releases we've
been making a more concerted effort to complete testing even on TC/RC
builds that have obvious showstoppers - to catch the other bugs 'behind'
the showstoppers, rather than just catching the showstoppers and then
focusing work on getting them fixed, then continuing on with testing of
other functionality.

I don't mean to start a finger-pointing match, but I do think it's worth
bearing in mind that the 'blocker treadmill' is much more likely to
happen when there are major changes to anaconda, because these massively
increase the surface area of code that's prone to causing blocker bugs.
When we do a release, we can say with a reasonable degree of certainty
that the code in that release probably contains very few blocker bugs -
only ones we didn't catch in the validation process it just went
through.

If we then do another release in which that code isn't changed very
much, well, we aren't likely to have two hundred new blocker bugs.

But if we (Fedora) do, oh, let's just say as _entirely theoretical
examples_, rewrite the entire storage backend, or replace the entire
first stage of the whole installer, or rewrite the entire user
interface...we've just thrown out all the code that's relatively well
known to be 'blocker free', and replaced it with an entirely new chunk
of code about which we know just about nothing from a quality
perspective. Statistically speaking, no matter how awesome the person or
people writing it, that new chunk of code is very likely to contain more
blockers than the code it replaced. Major changes to the code inevitably
result in more blockers being present, and thus more blocker treadmill,
than light-touch maintenance of a mature codebase does. We (QA) are
always going to be able to find ten blockers in a well-known codebase
much faster than we can find two hundred blockers in a heavily revised
codebase.

Certainly QA has some responsibility for the 'blocker treadmill', as I
noted above, it's our responsibility to try and identify blocker bugs as
early and as quickly as possible, and this is something we can and
should always look to improve. But developers also have responsibility
for it. If you're stuck on a 'blocker treadmill' it could be an
indication that QA could and should have discovered the blocker bugs
faster, but it could also be an indication that you have been too
ambitious in your planning in terms of what amount of new or revised
code of acceptable quality you expected to be able to implement in what
time frame, and consequently you have delivered code that is heavily
bugged, at a late enough point in the development cycle that you
immediately wind up on a 'blocker treadmill' just fixing all the bugs in
the code you just delivered. I don't think it's controversial to say
this has been known to be a problem in the world of software development
before

> 2) Installer blockers should only be granted when there is no other way to
> accomplish the same task during installation. For example, if FCoE
> configuration does not always work in the UI but does work when passed boot
> parameters or via kickstart, we shouldn't consider it a blocker. It's an
> unfortunate bug, but as described there is an install path for those users.

In practice we do and always have considered workarounds in evaluating
blocker status for bugs. This isn't brilliantly called out in the
criteria pages, I admit, and we should improve that. The section
'(release) Blocker Bugs', right below the criteria, could really do with
some adjustment.

It's hard to be more precise than this because workaround evaluation is
one part of the blocker review process that more or less inevitably
continues to involve subjectivity, and it's very much a bug-by-bug
thing. But obviously, the more severe and more commonly-encountered the
issue, the less likely we usually are to accept 'there's a workaround'
as a reason not to take it as a blocker. The ease of the workaround and
the likelihood of a user thinking of it themselves - or at least
figuring that there _might_ be a workaround, and they should go and look
for one - are also taken into consideration.

So...we do consider workarounds. And yes, this should be explained more
clearly in the process documentation, we'll address that. I don't think
we should accept your principle - "Installer blockers should only be
granted when there is no other way to accomplish the same task during
installation." - as solidly as it's stated, though, as it removes too
much flexibility in the evaluation process.

To give a competing example, in anaconda 18.13 there is a bug in the new
partitioning process - I call it 'guided partitioning', the dialog which
attempts to help you free up space on a full disk, by deleting or
shrinking partitions - which causes it to crash when trying to delete
partitions. But if you go into the 'custom partitioning' interface you
can successfully delete partitions. So by your principle, we would not
take that bug as a release blocker. I don't think that would be a good
decision: we should not release an installer which crashes when you try
to follow the path you're guided to, for freeing up space to install the
operating system. 'Don't do what the installer recommends you to do,
instead go into this advanced process that's supposed to be for experts'
is a workaround, and hence satisfies your requirement for not-a-blocker,
but I really don't think it's a good story to tell people in the case of
such a critical bit of functionality.

It also has a clear negative effect on the very problem we're discussing
here: the broken code cannot get any testing. All we can know about a
codepath that's broken, but for which we accepted a workaround that
dodges the broken codepath, is 'it's completely broken'. If the broken
codepath is not treated as a blocker and fixed rapidly, we cannot test
it 'beyond' the blocker bug. There might be five further blockers behind
that bug, or just _regular_ bugs ('the UI sucks', 'it doesn't offer to
let me resize a partition it should have done', 'it prints a bogus error
message when I delete a partition'...all those kinds of perfectly normal
bugs), but if we take a workaround and called it 'not a blocker' it gets
dropped in priority, likely doesn't get fixed for weeks, and when it
turns out there's five other bugs 'behind' the showstopper...well, they
go on your treadmill. =) Accepting workarounds too readily actually
_impedes_ our ability to find other bugs swiftly.

> 3) Ultimately we want the number of granted blockers to be lower and lower
> from alpha to beta to rc.

I understand the motivation behind this, and I think it's a goal we can
attempt to address by ensuring comprehensive testing is done early (and
a goal you can help to address by ensuring major code changes land early
enough to be tested, and budgeting time and resources for fixing the
bugs that will _inevitably be present_ in any large chunk of new code).
But I don't think 'make it harder for a bug to qualify as a blocker the
later we get in the release process' is a good thing to do, even though
it would help to achieve this goal. To me it looks like a process hack
which would ultimately damage the quality of our releases. It's actually
something that we've specifically tried *not* to do in the blocker
review process, since it was implemented. We have very intentionally
attempted to review bugs 'impartially', treating blocker status as
something a bug either should have or should not have on its own merits,
and attempting not to take into account things that strictly should not
be taken into account, like 'is there a fix already?' or 'how close are
we to release?'

Thanks again for your thoughts!
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 10-09-2012, 11:48 PM
Adam Williamson
 
Default Review of Fedora 18 Release Criteria

On Tue, 2012-10-09 at 16:07 -0700, Adam Williamson wrote:
> On Tue, 2012-10-09 at 16:48 -0400, David Cantrell wrote:
> > On Tue, Oct 09, 2012 at 07:19:25AM -0600, Tim Flink wrote:
> > > As we're getting closer to the scheduled time for beta freeze, we'd
> > > like to find out now if any of the current criteria or proposed criteria
> > > changes are unreasonable to expect for beta. There may be more changes
> > > for final as we get closer to that but I think that we're pretty close
> > > to being done with the release requirements for beta.
> > >
> > > The current (as of this writing) release criteria are available at:
> > > - http://fedoraproject.org/wiki/Fedora_18_Alpha_Release_Criteria
> > > - http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria
> > > - http://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria
>
> Thanks David! Some thoughts follow.
>
> > I would like to see changes to the blocker criteria for each release. The
> > first item on each release criteria is that all blockers must be CLOSED.
> > Blockers are determined by criteria defined below which always group
> > anaconda in because we cannot address those problems in a later update
> > release. This gets us on the bug fixing treadmill as we edge closer to each
> > release because every anaconda bug more or less becomes a blocker.
>
> This paragraph was a bit tricky to read, but now I've given it a few
> tries, it seems to be more or less a preamble, yes? I'm not sure if
> you're suggesting that "Blockers are determined by criteria defined
> below which always group anaconda in because we cannot address those
> problems in a later update release. This gets us on the bug fixing
> treadmill as we edge closer to each release because every anaconda bug
> more or less becomes a blocker." is a problem, or just mentioning it as
> background. It's perfectly true as background, but I don't see it as a
> problem: it's just an innate characteristic of the software you write.
> The installer is something that cannot be updated (for practical
> purposes), it must work to a high standard as shipped, because if it
> doesn't, that's a much bigger problem than a component which _can_ be
> updated not working. I agree with your assessment, but I see it as an
> inherent characteristic of an operating system installer, not any kind
> of problem in the process.

Oh, to address one point I missed - it's certainly not the case that
'every anaconda bug more or less becomes a blocker'. There have been 372
bugs reported against the 'anaconda' component with '18' as the version:
32 of them have been accepted as blockers ('AcceptedBlocker' in the
whiteboard field), that's less than 10%. 365 bugs were reported against
'anaconda' version '17', and 25 of them were accepted as blockers.

Overall these numbers seem a bit low because installation-related bugs
can wind up being in livecd-tools or pungi or lorax or dracut or several
other components, but they're sufficient to demonstrate the basic point,
nowhere near all anaconda bugs are considered blockers. I don't think
that assertion can be used to support any argument, as it seems clearly
not to be the case.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 10-10-2012, 02:43 AM
Akira TAGOH
 
Default Review of Fedora 18 Release Criteria

----- 元のメッセージ -----
| As we're getting closer to the scheduled time for beta freeze, we'd
| like to find out now if any of the current criteria or proposed
| criteria
| changes are unreasonable to expect for beta. There may be more
| changes
| for final as we get closer to that but I think that we're pretty
| close
| to being done with the release requirements for beta.
|
| The current (as of this writing) release criteria are available at:
| - http://fedoraproject.org/wiki/Fedora_18_Alpha_Release_Criteria
| - http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria
| - http://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria
|
| There have been several changes to the release criterion lately
| (mostly
| due to changes in anaconda's functionality) and there are still a
| couple of modifications pending (identified at the end of this email
| with links to the most recent versions of the proposal)
|
| The questions we're interested in are:
| - Is there anything in the current criteria or proposed revisions
| that
| seem unreasonable?
| - Are there any areas where the criteria are lacking?

I was expecting not to see the sort of locale issues anymore since it's essential technology to support i18n/l10n and didn't doubt we haven't ever had any criteria for that and in fact it wasn't broken in a long time. however it happened and affected a lot of applications and testings. the sort of this issue should be caught up by alpha because of that. in fact it messed up a lot. so I'd like to propose adding something for locales/langauges in alpha criteria.

Though it may be not happened forever. but that would be good to make sure if we won't break it every releases.

Any thoughts?

--
Akira TAGOH

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 10-10-2012, 03:10 AM
Adam Williamson
 
Default Review of Fedora 18 Release Criteria

On Tue, 2012-10-09 at 22:43 -0400, Akira TAGOH wrote:
> ----- 元のメッセージ -----
> | As we're getting closer to the scheduled time for beta freeze, we'd
> | like to find out now if any of the current criteria or proposed
> | criteria
> | changes are unreasonable to expect for beta. There may be more
> | changes
> | for final as we get closer to that but I think that we're pretty
> | close
> | to being done with the release requirements for beta.
> |
> | The current (as of this writing) release criteria are available at:
> | - http://fedoraproject.org/wiki/Fedora_18_Alpha_Release_Criteria
> | - http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria
> | - http://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria
> |
> | There have been several changes to the release criterion lately
> | (mostly
> | due to changes in anaconda's functionality) and there are still a
> | couple of modifications pending (identified at the end of this email
> | with links to the most recent versions of the proposal)
> |
> | The questions we're interested in are:
> | - Is there anything in the current criteria or proposed revisions
> | that
> | seem unreasonable?
> | - Are there any areas where the criteria are lacking?
>
> I was expecting not to see the sort of locale issues anymore since
> it's essential technology to support i18n/l10n and didn't doubt we
> haven't ever had any criteria for that and in fact it wasn't broken in
> a long time. however it happened and affected a lot of applications
> and testings. the sort of this issue should be caught up by alpha
> because of that. in fact it messed up a lot. so I'd like to propose
> adding something for locales/langauges in alpha criteria.
>
> Though it may be not happened forever. but that would be good to make
> sure if we won't break it every releases.
>
> Any thoughts?

So this is an area we've been over a few times...

Last time we went over it, what we did was add an explicit translation
criterion for Final - "All critical path actions on release-blocking
desktop environments should correctly display all sufficiently complete
translations available for use" - and agree that other i18n/l10n issues
should be considered conditional infringements of the other criteria and
handled that way. e.g. if you can't pick a keyboard map properly, that's
a conditional infringement of "In most cases (see Blocker_Bug_FAQ), a
system installed according to any of the above criteria (or the
appropriate Beta or Final criteria, when applying this criterion to
those releases) must boot to the 'firstboot' utility on the first boot
after installation, without unintended user intervention, unless the
user explicitly chooses to boot in non-graphical mode. This includes
correctly accessing any encrypted partitions when the correct passphrase
is supplied. The firstboot utility must be able to create a working user
account", because the passphrase you picked for an encrypted partition
at install time would not be typed the same way when you came to unlock
it.

This more or less _works_, but on reflection (it came up in the 18
blocker review process), it seems pretty baroque and non-raptor-proof -
I don't think someone just reading the process documentation is
necessarily going to figure out that dodge. So it fails that test. (As
neatly shown by the fact that you posted this mail - obviously you
didn't figure it out :>) So I'm now rather of the opinion we should have
explicit criteria for the major i18n/l10n stuff, indeed.

We do actually cover this in the test cases, but again, we did it in an
'indirect' way: we added 'test it with a non-English keymap' to
https://fedoraproject.org/wiki/QA:Testcase_desktop_login and
https://fedoraproject.org/wiki/QA:Testcase_Anaconda_autopart_%
28encrypted%29_install . But again this isn't particularly obvious if
you just browse the test list. And now I look at it, it covers only
keymap, not locale, so we might need to add that somehow.

We have/had a trac ticket which was closed after the last go-round; I
re-opened it after having the thoughts described above.
https://fedorahosted.org/fedora-qa/ticket/81 - let's follow up on it
there.

I _think_ this actually broke between Alpha and Beta, didn't it? I might
be wrong, but I thought it worked at Alpha but then had got broken for
Beta.

Thanks for the thoughts!
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 10-10-2012, 06:38 AM
Akira TAGOH
 
Default Review of Fedora 18 Release Criteria

----- 元のメッセージ -----
| Last time we went over it, what we did was add an explicit
| translation
| criterion for Final - "All critical path actions on release-blocking
| desktop environments should correctly display all sufficiently
| complete
| translations available for use" - and agree that other i18n/l10n
| issues
| should be considered conditional infringements of the other criteria
| and
| handled that way. e.g. if you can't pick a keyboard map properly,
| that's
| a conditional infringement of "In most cases (see Blocker_Bug_FAQ), a
| system installed according to any of the above criteria (or the
| appropriate Beta or Final criteria, when applying this criterion to
| those releases) must boot to the 'firstboot' utility on the first
| boot
| after installation, without unintended user intervention, unless the
| user explicitly chooses to boot in non-graphical mode. This includes
| correctly accessing any encrypted partitions when the correct
| passphrase
| is supplied. The firstboot utility must be able to create a working
| user
| account", because the passphrase you picked for an encrypted
| partition
| at install time would not be typed the same way when you came to
| unlock
| it.

That's right. I thought it should works enough that way at that time.

| I don't think someone just reading the process documentation is
| necessarily going to figure out that dodge. So it fails that test.
| (As
| neatly shown by the fact that you posted this mail - obviously you
| didn't figure it out :>)

Well, not really. I think this criteria also affects to QA testing. though we could help for testing but that would be a good idea to have more testing widely, continually, and systematically, I just thought. so I'm here now

| We have/had a trac ticket which was closed after the last go-round; I
| re-opened it after having the thoughts described above.
| https://fedorahosted.org/fedora-qa/ticket/81 - let's follow up on it
| there.

Okay.

| I _think_ this actually broke between Alpha and Beta, didn't it? I
| might
| be wrong, but I thought it worked at Alpha but then had got broken
| for
| Beta.

No. as in the original report, it was there in even Alpha.

--
Akira TAGOH

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 10-10-2012, 04:17 PM
David Cantrell
 
Default Review of Fedora 18 Release Criteria

On Tue, Oct 09, 2012 at 04:07:54PM -0700, Adam Williamson wrote:
> On Tue, 2012-10-09 at 16:48 -0400, David Cantrell wrote:
> > On Tue, Oct 09, 2012 at 07:19:25AM -0600, Tim Flink wrote:
> > > As we're getting closer to the scheduled time for beta freeze, we'd
> > > like to find out now if any of the current criteria or proposed criteria
> > > changes are unreasonable to expect for beta. There may be more changes
> > > for final as we get closer to that but I think that we're pretty close
> > > to being done with the release requirements for beta.
> > >
> > > The current (as of this writing) release criteria are available at:
> > > - http://fedoraproject.org/wiki/Fedora_18_Alpha_Release_Criteria
> > > - http://fedoraproject.org/wiki/Fedora_18_Beta_Release_Criteria
> > > - http://fedoraproject.org/wiki/Fedora_18_Final_Release_Criteria
>
> Thanks David! Some thoughts follow.
>
> > I would like to see changes to the blocker criteria for each release. The
> > first item on each release criteria is that all blockers must be CLOSED.
> > Blockers are determined by criteria defined below which always group
> > anaconda in because we cannot address those problems in a later update
> > release. This gets us on the bug fixing treadmill as we edge closer to each
> > release because every anaconda bug more or less becomes a blocker.
>
> This paragraph was a bit tricky to read, but now I've given it a few
> tries, it seems to be more or less a preamble, yes? I'm not sure if
> you're suggesting that "Blockers are determined by criteria defined
> below which always group anaconda in because we cannot address those
> problems in a later update release. This gets us on the bug fixing
> treadmill as we edge closer to each release because every anaconda bug
> more or less becomes a blocker." is a problem, or just mentioning it as
> background. It's perfectly true as background, but I don't see it as a
> problem: it's just an innate characteristic of the software you write.

It was meant as background explanation for the points that followed.

> The installer is something that cannot be updated (for practical
> purposes), it must work to a high standard as shipped, because if it
> doesn't, that's a much bigger problem than a component which _can_ be
> updated not working. I agree with your assessment, but I see it as an
> inherent characteristic of an operating system installer, not any kind
> of problem in the process.

This is something I am aware of, having been working on anaconda for a long
time.

> > What I
> > would like to see:
> >
> > 1) Installer blocker criteria needs to be more and more restrictive the
> > closer we get to a final release.
>
> This wasn't entirely clear to me, but I'm going to take a guess at what
> I think you mean and reply to that. I think you're looking at the
> situation where we get late into the validation process - say, we just
> built RC2 and it's two days to go/no-go - and we find five bugs and mark
> them as blockers. I'm guessing you're saying it'd be preferable to
> identify blockers early and we should only add issues to the blocker
> list late if they're _really bad_, because otherwise you just keep
> fixing blockers. On that basis...

Early is better, but as you stated above it's just the nature of the
installer software. I get that and have no problem with it, what I'm
wanting evaluated for change is the process by which we agree something is a
blocker. Right now we run full out fixing pretty much everything we find
headed up to RC. I would like to see something similar to what we do for
RHEL where we start to tighten blocker acceptance criteria the closer we get
to a release.

> I appreciate that the 'blocker treadmill', as you describe it, can be
> frustrating. But I don't think 'let's just not count bad issues as
> blockers late in the process to give the developers a break' is the
> answer to anything (except possibly 'how can we stop Will depleting the
> U.S. strategic gin reserve?', but that's not the question this post was
> trying to answer :>). What we're trying to do with the release
> validation process as a whole is provide a clear framework for defining
> the standards our releases should meet and a clear process for building
> releases that meet those standards and verifying that they meet those
> standards. I don't see that adding an element of time sensitivity to the
> blocker evaluation process - 'issues of the type X are blockers if we
> find them four weeks before release, but not if we find them one week
> before release' - is a good way to achieve this.

You're not understanding what I was pointing out. The blocker criteria
between alpha and beta should be more open than the blocker criteria between
beta and rc. The idea is that we start accepting fewer bugs as blockers as
we get closer to RC. Every problem encountered can be evaluated along the
lines of:

1) Who is impacted?
2) Is there a workaround?
3) Is the workaround documented?
4) Is the problem in the standard install path?

And so on. I'm not saying we should compromise on release quality or
anything like that, but just start to ask more and more questions when
proposed blockers show up late. Is it really a blocker or not.

> 'Blocker bugs' are just the 'release quality' question inverted: they
> are the ways in which our releases must not be broken in order to meet
> the minimum quality standards we've decided on. An issue which causes us
> to fall below our minimum quality standards is a problem no matter when
> it's discovered. I absolutely understand that it makes things easier for
> the developers if we catch blocker bugs early, and we agree this is an
> important goal and we have made and will continue to make efforts to
> improve our ability to catch blockers as early as possible. I know it
> sucks when we're on RC3 and we suddenly discover a major bug. But it's
> still a major bug, and 'say it's not a blocker because we're late in the
> process' doesn't sound like a good response to that suckage, to me. I
> don't want to do that.

Again, that's not what I'm saying we should do. But if there's a traceback
that appears when someone is doing a LUKS installation to USB attached
storage that also has a pre-existing NTFS volume for "music"....that doesn't
sound like a really suport important use case late in the cycle. Maybe we
should consider documenting that as a limitation with or without a
workaround for that release and slating it for the next release.

We do want to make a good release and keep the users happy, but we need to
also remember that developers are people too and not just meat grinders for
bug reports.

> I believe we should set realistic minimum standards - those that are
> achievable with the level of development resources we have in place, on
> the release schedules Fedora is committed to. What this thread is about,
> essentially, is checking that we are not currently setting that bar too
> high, and demanding from you more than you have the resources to
> possibly provide in the time available. We certainly believe that we
> need input from the development teams to know where the bar should be
> set. But I do believe the bar should be a bar, not a fuzzy field that
> can be adjusted with excessive pragmatism. We should set realistic
> standards, but they have to be solid ones that we don't compromise just
> because time is short or the developers are getting tired of fixing
> bugs.

>From the installer team's perspective, the useful things for release
criteria to consider would be:

1) Adjustments to the blocker acceptance criteria that I explained.
2) Acknowledgement that concurrent releases are always in play for the
installer team. Generally two RHEL releases and one Fedora release.
RHEL gets priority.

> What we (QA) as a team do try and do in those cases is look at the
> situation and think what we could do in future to ensure the blocker
> would get caught earlier. For instance, in the last few releases we've
> been making a more concerted effort to complete testing even on TC/RC
> builds that have obvious showstoppers - to catch the other bugs 'behind'
> the showstoppers, rather than just catching the showstoppers and then
> focusing work on getting them fixed, then continuing on with testing of
> other functionality.

You have improved bug reporting earlier over the past few Fedora releases.

> I don't mean to start a finger-pointing match, but I do think it's worth
> bearing in mind that the 'blocker treadmill' is much more likely to
> happen when there are major changes to anaconda, because these massively
> increase the surface area of code that's prone to causing blocker bugs.
> When we do a release, we can say with a reasonable degree of certainty
> that the code in that release probably contains very few blocker bugs -
> only ones we didn't catch in the validation process it just went
> through.

This is nothing new to us, we've been through it many times before. But it
doesn't mean we enjoy it. Unless we want to put anaconda in a permanent
maintenance mode, it means we will always be playing this game.

> If we then do another release in which that code isn't changed very
> much, well, we aren't likely to have two hundred new blocker bugs.
>
> But if we (Fedora) do, oh, let's just say as _entirely theoretical
> examples_, rewrite the entire storage backend, or replace the entire
> first stage of the whole installer, or rewrite the entire user
> interface...we've just thrown out all the code that's relatively well
> known to be 'blocker free', and replaced it with an entirely new chunk
> of code about which we know just about nothing from a quality
> perspective. Statistically speaking, no matter how awesome the person or
> people writing it, that new chunk of code is very likely to contain more
> blockers than the code it replaced. Major changes to the code inevitably
> result in more blockers being present, and thus more blocker treadmill,
> than light-touch maintenance of a mature codebase does. We (QA) are
> always going to be able to find ten blockers in a well-known codebase
> much faster than we can find two hundred blockers in a heavily revised
> codebase.
>
> Certainly QA has some responsibility for the 'blocker treadmill', as I
> noted above, it's our responsibility to try and identify blocker bugs as
> early and as quickly as possible, and this is something we can and
> should always look to improve. But developers also have responsibility
> for it. If you're stuck on a 'blocker treadmill' it could be an
> indication that QA could and should have discovered the blocker bugs
> faster, but it could also be an indication that you have been too
> ambitious in your planning in terms of what amount of new or revised
> code of acceptable quality you expected to be able to implement in what
> time frame, and consequently you have delivered code that is heavily
> bugged, at a late enough point in the development cycle that you
> immediately wind up on a 'blocker treadmill' just fixing all the bugs in
> the code you just delivered. I don't think it's controversial to say
> this has been known to be a problem in the world of software development
> before

Let me be clear here: I am not complaining about the QA team performing
tests and reporting bugs. That's what you're supposed to do. The blocker
treadmill has been in place for many many many releases and we have always
been involved with iti (but we know this, you don't have to explain that
it's because of the nature of our software...in fact, that's been our
explanation for years now so the fact that you're telling us that seems to
indicate that people are finally starting to "get it"). Anaconda depends
on many components and many times last minute bugs are due to changes in
dependent components that caught us by surprise.

What I'm asking for is blocker acceptance criteria changes that are more
realistic given remaining time and other commitments. Or dump the
time/schedule requirement and just say we'll call it done when it's done. I
really don't care here, but we need to find a way to start denying blockers
as we get closer to a release.

> > 2) Installer blockers should only be granted when there is no other way to
> > accomplish the same task during installation. For example, if FCoE
> > configuration does not always work in the UI but does work when passed boot
> > parameters or via kickstart, we shouldn't consider it a blocker. It's an
> > unfortunate bug, but as described there is an install path for those users.
>
> In practice we do and always have considered workarounds in evaluating
> blocker status for bugs. This isn't brilliantly called out in the
> criteria pages, I admit, and we should improve that. The section
> '(release) Blocker Bugs', right below the criteria, could really do with
> some adjustment.
>
> It's hard to be more precise than this because workaround evaluation is
> one part of the blocker review process that more or less inevitably
> continues to involve subjectivity, and it's very much a bug-by-bug
> thing. But obviously, the more severe and more commonly-encountered the
> issue, the less likely we usually are to accept 'there's a workaround'
> as a reason not to take it as a blocker. The ease of the workaround and
> the likelihood of a user thinking of it themselves - or at least
> figuring that there _might_ be a workaround, and they should go and look
> for one - are also taken into consideration.
>
> So...we do consider workarounds. And yes, this should be explained more
> clearly in the process documentation, we'll address that. I don't think
> we should accept your principle - "Installer blockers should only be
> granted when there is no other way to accomplish the same task during
> installation." - as solidly as it's stated, though, as it removes too
> much flexibility in the evaluation process.
>
> To give a competing example, in anaconda 18.13 there is a bug in the new
> partitioning process - I call it 'guided partitioning', the dialog which
> attempts to help you free up space on a full disk, by deleting or
> shrinking partitions - which causes it to crash when trying to delete
> partitions. But if you go into the 'custom partitioning' interface you
> can successfully delete partitions. So by your principle, we would not
> take that bug as a release blocker. I don't think that would be a good
> decision: we should not release an installer which crashes when you try
> to follow the path you're guided to, for freeing up space to install the
> operating system. 'Don't do what the installer recommends you to do,
> instead go into this advanced process that's supposed to be for experts'
> is a workaround, and hence satisfies your requirement for not-a-blocker,
> but I really don't think it's a good story to tell people in the case of
> such a critical bit of functionality.

Crashes through the standard install path would of course be a blocker. Why
would you think I would want otherwise? You're reading my proposal way too
literally.

> It also has a clear negative effect on the very problem we're discussing
> here: the broken code cannot get any testing. All we can know about a
> codepath that's broken, but for which we accepted a workaround that
> dodges the broken codepath, is 'it's completely broken'. If the broken
> codepath is not treated as a blocker and fixed rapidly, we cannot test
> it 'beyond' the blocker bug. There might be five further blockers behind
> that bug, or just _regular_ bugs ('the UI sucks', 'it doesn't offer to
> let me resize a partition it should have done', 'it prints a bogus error
> message when I delete a partition'...all those kinds of perfectly normal
> bugs), but if we take a workaround and called it 'not a blocker' it gets
> dropped in priority, likely doesn't get fixed for weeks, and when it
> turns out there's five other bugs 'behind' the showstopper...well, they
> go on your treadmill. =) Accepting workarounds too readily actually
> _impedes_ our ability to find other bugs swiftly.

I didn't read this paragraph. This is a lot of text, but I've skimmed it.
What I think we should consider is whether Fedora should have a separate
entity or group that accepts blockers. Right now, that responsibility is
largely falling on your team.

> > 3) Ultimately we want the number of granted blockers to be lower and lower
> > from alpha to beta to rc.
>
> I understand the motivation behind this, and I think it's a goal we can
> attempt to address by ensuring comprehensive testing is done early (and
> a goal you can help to address by ensuring major code changes land early
> enough to be tested, and budgeting time and resources for fixing the
> bugs that will _inevitably be present_ in any large chunk of new code).
> But I don't think 'make it harder for a bug to qualify as a blocker the
> later we get in the release process' is a good thing to do, even though
> it would help to achieve this goal. To me it looks like a process hack
> which would ultimately damage the quality of our releases. It's actually
> something that we've specifically tried *not* to do in the blocker
> review process, since it was implemented. We have very intentionally
> attempted to review bugs 'impartially', treating blocker status as
> something a bug either should have or should not have on its own merits,
> and attempting not to take into account things that strictly should not
> be taken into account, like 'is there a fix already?' or 'how close are
> we to release?'

I'd like to apply my previous point here too.

--
David Cantrell <dcantrell@redhat.com>
Manager, Installer Engineering Team
Red Hat, Inc. | Westford, MA | EST5EDT

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 10-10-2012, 06:25 PM
Bruno Wolff III
 
Default Review of Fedora 18 Release Criteria

On Wed, Oct 10, 2012 at 12:17:41 -0400,
David Cantrell <dcantrell@redhat.com> wrote:


You're not understanding what I was pointing out. The blocker criteria
between alpha and beta should be more open than the blocker criteria between
beta and rc. The idea is that we start accepting fewer bugs as blockers as
we get closer to RC. Every problem encountered can be evaluated along the
lines of:


The point of accepting fewer things as blockers for alpha and beta is to
enable those to be released to help find bugs. And we don't want to hold
them up for things that are not things we want final to have, but that
don't interfere with testing too much.



1) Who is impacted?
2) Is there a workaround?
3) Is the workaround documented?
4) Is the problem in the standard install path?

And so on. I'm not saying we should compromise on release quality or
anything like that, but just start to ask more and more questions when
proposed blockers show up late. Is it really a blocker or not.


I disagree that we want to look at lateness as a critera for blockers. (It
does get some consideration for the nice to have designation.) I think
the kinds of bugs that lateness would be a consideration for, are not the
kinds of bugs we want to classify as blockers.


_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 
Old 10-10-2012, 08:18 PM
Adam Williamson
 
Default Review of Fedora 18 Release Criteria

On Wed, 2012-10-10 at 12:17 -0400, David Cantrell wrote:

> You're not understanding what I was pointing out. The blocker criteria
> between alpha and beta should be more open than the blocker criteria between
> beta and rc. The idea is that we start accepting fewer bugs as blockers as
> we get closer to RC. Every problem encountered can be evaluated along the
> lines of:
>
> 1) Who is impacted?
> 2) Is there a workaround?
> 3) Is the workaround documented?
> 4) Is the problem in the standard install path?
>
> And so on. I'm not saying we should compromise on release quality or
> anything like that, but just start to ask more and more questions when
> proposed blockers show up late. Is it really a blocker or not.

> Again, that's not what I'm saying we should do. But if there's a traceback
> that appears when someone is doing a LUKS installation to USB attached
> storage that also has a pre-existing NTFS volume for "music"....that doesn't
> sound like a really suport important use case late in the cycle. Maybe we
> should consider documenting that as a limitation with or without a
> workaround for that release and slating it for the next release.

I have a better understanding of where you're coming from now, and
particularly that you're basing this on a RHEL model (I didn't know this
was how RHEL did blocker tracking).

But I still disagree.

I don't think my disagreement is necessarily a problem for you,
though...let me see if I can put it a different way:

Take the example you just gave. My position isn't 'we should take that
bug as a blocker both four weeks out and one week out'. My position is
'we shouldn't take it as a blocker four weeks out _or_ one week out'.

I'd phrase the problem more as 'there is a tendency in blocker review to
accept things too casually early on'. We should only accept things as
blockers that we really believe we should hold the release up for - even
if it's six weeks until release and we know the bug is going to be fixed
in two days, we should not accept it as a blocker unless it passes the
'if today was go/no-go and this was the last bug, would we block the
release' smell test.

Rather than throw almost everything on the blocker list when it's early
and get more rigorous later on, I'd rather we just be rigorous the whole
time. The blocker list isn't a work list (using the blocker list as a
'todo list' / reminder list is a tendency I've noticed in both RHEL and
Fedora, and something I think we should fight), it is a list of bugs
that must be fixed for the release to go out. No more, no less. If we
wouldn't really block the release for a bug, it doesn't belong on the
list at any point in time. I think we've been making a good effort to
improve this in recent releases; it's not just an aspiration.

So if there are concrete cases where you think bugs have been accepted
as blockers that you wouldn't be comfortable taking as blockers the day
before release, that's _always a problem_, and please highlight those
bugs to us. I don't think it's 'not a problem' if we accepted them as
blockers a long time before release and they got fixed before any chance
of causing a slip, it's still a problem. We should stick firmly to our
tight, minimal definition of what constitutes a blocker, regardless of
the point in the release schedule.

> We do want to make a good release and keep the users happy, but we need to
> also remember that developers are people too and not just meat grinders for
> bug reports.

Sure, hence the need to keep the criteria realistic.

> 1) Adjustments to the blocker acceptance criteria that I explained.
> 2) Acknowledgement that concurrent releases are always in play for the
> installer team. Generally two RHEL releases and one Fedora release.
> RHEL gets priority.

I would prefer to phrase 2) the way I already did: Fedora should ensure
its release requirements are realistic in the context of the resources
available. If the 'resources available' are to an extent defined by
other commitments on the part of the anaconda team, then so be it, but
I'd rather consider it in that context, rather than haul RHEL
specifically into a Fedora context.

> What I'm asking for is blocker acceptance criteria changes that are more
> realistic given remaining time and other commitments. Or dump the
> time/schedule requirement and just say we'll call it done when it's done.

The schedule / blocker process balance is a bit of a delicate one, yeah.
In practice we start getting a lot of pressure if the blocker process
results in delays of several weeks. I think I'd like to look at this
way, though: any time that following the release validation process
properly results in a long slip, _that means something is wrong_. It
usually means either that the release criteria are excessive, or it
means that something went wrong in the feature process. I believe the
feature process as it stands is not adequate in ensuring proposed
features are really viable on the Fedora release schedule, and FESCo has
also not always been assertive enough in dropping or delaying features
that are clearly not meeting the schedule; I think sometimes problems
are due to that, rather than to over-ambitious release requirements.

> > To give a competing example, in anaconda 18.13 there is a bug in the new
> > partitioning process - I call it 'guided partitioning', the dialog which
> > attempts to help you free up space on a full disk, by deleting or
> > shrinking partitions - which causes it to crash when trying to delete
> > partitions. But if you go into the 'custom partitioning' interface you
> > can successfully delete partitions. So by your principle, we would not
> > take that bug as a release blocker. I don't think that would be a good
> > decision: we should not release an installer which crashes when you try
> > to follow the path you're guided to, for freeing up space to install the
> > operating system. 'Don't do what the installer recommends you to do,
> > instead go into this advanced process that's supposed to be for experts'
> > is a workaround, and hence satisfies your requirement for not-a-blocker,
> > but I really don't think it's a good story to tell people in the case of
> > such a critical bit of functionality.
>
> Crashes through the standard install path would of course be a blocker. Why
> would you think I would want otherwise? You're reading my proposal way too
> literally.

I'm afraid I have a tendency to read things literally, because processes
and policies are written down, and if the person who wrote them is eaten
by a raptor the only way you can reliably read them is 'literally' It
makes drafting things a pain in the ass, but I think it makes what you
wind up with a stronger process/policy. Your proposal didn't have escape
clauses or exceptions, it simply read "Installer blockers should only be
granted when there is no other way to accomplish the same task during
installation." That's a very clear and unequivocal statement. We could
not adopt it into the blocker evaluation process as it's written.

As I said in my initial mail, I think in practice we mostly already do
what you really want here, but we do need to document it more clearly.
As the documentation stands, it really doesn't cover the question of
workarounds much at all.

(in reference to my points about development planning)

> This is nothing new to us, we've been through it many times before.
> But it doesn't mean we enjoy it. Unless we want to put anaconda in a
> permanent maintenance mode, it means we will always be playing this
> game.

I don't think this is entirely correct. I don't want to get into
specifics because I'm not plugged into the development planning process
enough to make accurate criticisms, and I don't want to teach anyone's
grandma to suck eggs, but to talk generally:

Of course any project which involves major development will be more
susceptible to bugs than any project which is solely in maintenance
mode. But the _magnitude_ of the effect is certainly something that can
be addressed by good planning. I don't want to harp on this too much as
I'm sure it's something you're aware of, but we can certainly aim to be
realistic and rigorous in our planning and timing of major new features.
For instance, trying to ensure major developments are feature complete
at Alpha time, so we're not still writing stuff during the Beta cycle.
FESCo requiring much more detailed planning for feature submissions and
being more decisive about rejecting/delaying features that are clearly
lagging at Alpha stage. Stuff like that.

> I didn't read this paragraph. This is a lot of text, but I've skimmed it.
> What I think we should consider is whether Fedora should have a separate
> entity or group that accepts blockers. Right now, that responsibility is
> largely falling on your team.

This isn't really the intent. The blocker meeting SOP -
https://fedoraproject.org/wiki/QA:SOP_Blocker_Bug_Meeting - reads:

"Blocker Bug Meetings are not owned by any one team in Fedora. They are
a collaborative effort between Release Engineering, Quality Assurance,
Development, and Project Management."

and the intent is that, ideally, some or all of those groups should be
represented at blocker review meetings. Lately, it seems that this
hasn't been happening as much as it used to. We used to have fairly
solid attendance from releng, FPL / FPM, and interested development
parties. Recently it seems like these groups just aren't terribly
interested in showing up and the meetings tend to be all or mostly QA
people. And we go ahead and do the evaluations, because hell, someone
has to. But as the procedure clearly states, it's not a QA-owned
process, it's meant to be collaborative, and the onus is really on other
interested parties to _show up and take part_. We do usually ping
#anaconda when blocker review starts up and ask if anyone wants to take
part; often no-one does. I think it's seen as taking away precious
development time. It would be really nice if we starting getting more
developers and Robyn and Jaroslav showing up for more blocker reviews.
Admittedly, the meetings do get very long sometimes, but we've found it
pretty hard to resolve that (reviewing 30 blockers is going to take a
while, however you slice it). I do think it helps to have multiple
perspectives, and that QA folks often tend to be more 'liberal' about
accepting blockers than other groups; it would definitely be useful to
have other groups present to act as a check on this.

It might be an idea to start CCing the blocker meeting announcements out
a bit more widely, to help with this. Or heck, we could also have some
non-QA person run the meetings for a while, give it to FPL or FPM or
someone. That would liven things up.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Twitter: AdamW_Fedora | identi.ca: adamwfedora
http://www.happyassassin.net

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list
 

Thread Tools




All times are GMT. The time now is 01:40 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org