FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 03-09-2010, 05:12 PM
Adam Williamson
 
Default QA's Package update policy proposal

On Tue, 2010-03-09 at 18:56 +0100, Michael Schwendt wrote:

> Why not? The end is near, it seems to me. It's like fear of castration.
> These mad proposals deserve to be flamed. Thanks to the "Fedora School
> Of Developing Bad Attitudes", I see myself gaining experience on how to
> become a 2nd level troll. :-( I hate that, but these proposals pop up
> like mushrooms and spread like a disease.

Please provide details on what's mad about Kamil's proposal.

> If you - and the QA team - want to expand your testing activities, focus
> on the CRITPATH packages first. Do a good job there. Nobody from QA has
> ever given feedback to any of my updates, and it won't happen in the
> future either. There won't be real testing of those packages unless you
> could dedicate resources like some great users do it. But you can't.

The proposal isn't really about expanding testing activities, it's about
formally codifying how the updates process is actually supposed to work.
Right now, we don't in fact define how the Fedora update process is
supposed to work: how updates get submitted, how they get promoted and
how they get released, who has what responsibilities at what point, none
of these is written down. That's what the proposal submitted by Kamil is
about. It's not really about trying to impose additional testing or
restrictions on updates. The most important thing is to have a framework
explaining who does what at what point in the process; exactly what
criteria are used to accept or reject updates is not the main thrust of
this proposal. As Kamil wrote, the numbers in there at the moment are
essentially placeholders, they could be changed entirely, and that's not
the important thing about this proposal.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-09-2010, 05:38 PM
Michael Schwendt
 
Default QA's Package update policy proposal

On Tue, 09 Mar 2010 10:12:41 -0800, Adam wrote:

> Please provide details on what's mad about Kamil's proposal.

Here:

| The package updates must spend at least 14 days [1] in the
| 'updates-testing' repository, or at least 7 days [1] provided they have
| karma of at least 3 [1] and no negative feedback. Only after that the
| package maintainer may submit them to the 'updates' repository.

Even under consideration of [1], it's the same system that doesn't work
for me and won't work for me.

Under some circumstances it would be justified to overrule negative
feedback even. E.g. if it's a user who just wants to block a good update
because of his pet peeve.

> The proposal isn't really about expanding testing activities, it's about
> formally codifying how the updates process is actually supposed to work.

It isn't a snapshot of how it works currently, though. It is an attempt
at introducing and hardcoding a process, and I'm unhappy with the
proposals so far as long as they are supposed to cover my packages and
confine me into something that would be much more plausible for the
critpath package set.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-09-2010, 05:41 PM
Seth Vidal
 
Default QA's Package update policy proposal

On Tue, 9 Mar 2010, Michael Schwendt wrote:

> If you - and the QA team - want to expand your testing activities, focus
> on the CRITPATH packages first. Do a good job there. Nobody from QA has
> ever given feedback to any of my updates, and it won't happen in the
> future either.


I would not be opposed to the above.
Make the system work on critpath then mandate it from everyone else.

-sv



--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-09-2010, 05:50 PM
Kevin Kofler
 
Default QA's Package update policy proposal

Adam Williamson wrote:
> The proposal isn't really about expanding testing activities, it's about
> formally codifying how the updates process is actually supposed to work.
> Right now, we don't in fact define how the Fedora update process is
> supposed to work: how updates get submitted, how they get promoted and
> how they get released, who has what responsibilities at what point, none
> of these is written down. That's what the proposal submitted by Kamil is
> about. It's not really about trying to impose additional testing or
> restrictions on updates.

Then why does it codify things which are not current practice, and in fact
require Bodhi changes to be implementable at all?

And why is "updates get pushed to stable when the maintainer feels they are
stable" (which is basically the current practice) not sufficient? We should
trust maintainers. That's what we have maintainers for. Stuff like karma
should only be a tool to help the maintainer decide.

> The most important thing is to have a framework explaining who does what
> at what point in the process; exactly what criteria are used to accept or
> reject updates is not the main thrust of this proposal. As Kamil wrote,
> the numbers in there at the moment are essentially placeholders, they
> could be changed entirely, and that's not the important thing about this
> proposal.

The numbers aren't the only controversial part of that proposal. Even
changing all the numbers to 0 would still create a policy which is much more
restrictive than what we have now (e.g. direct stable pushes aren't in the
workflow at all; a direct stable push is different from requesting a push to
stable after 0 days of testing, which assumes that we allowed it to hit
testing in the first place, so it can only go to stable in the next push).

Kevin Kofler

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-09-2010, 06:00 PM
Adam Williamson
 
Default QA's Package update policy proposal

On Tue, 2010-03-09 at 19:38 +0100, Michael Schwendt wrote:

> > Please provide details on what's mad about Kamil's proposal.
>
> Here:
>
> | The package updates must spend at least 14 days [1] in the
> | 'updates-testing' repository, or at least 7 days [1] provided they have
> | karma of at least 3 [1] and no negative feedback. Only after that the
> | package maintainer may submit them to the 'updates' repository.
>
> Even under consideration of [1], it's the same system that doesn't work
> for me and won't work for me.

Again, the actual criteria used are completely up for discussion. They
don't even have to be to do with Bodhi karma, necessarily. It's just a
placeholder. The key point is just that we provide the option for there
to be a filter point between updates-testing and updates. The nature of
the filtering is not decided.

> Under some circumstances it would be justified to overrule negative
> feedback even. E.g. if it's a user who just wants to block a good update
> because of his pet peeve.

That's why Kamil quickly added the 'Exception process' section before
sending the proposal to this list, just to make it clear that we
envisage there would be an exception process for cases where the policy
should be circumvented. We didn't really get time to fill in the
details, but we did want it to be clear that there should be a process
for that.

> > The proposal isn't really about expanding testing activities, it's about
> > formally codifying how the updates process is actually supposed to work.
>
> It isn't a snapshot of how it works currently, though.

It's fairly close. As far as I can see, all it intentionally changes
from the current process is that it provides a review point prior to
acceptance into -testing. There actually already *is* a review point
prior to moving to -updates, but it's currently owned by rel-eng and is
not highly publicized, and very little gets rejected. It is there,
though: rel-eng explained this earlier in the threads, and explained
that they are unhappy with it being informal. I'm trying to find the
post, but there's just too much to wade through, sorry

I haven't seen much fundamental disagreement with this in principle,
during the discussion; the argument has been about what kind of criteria
should be used to review changes, and the proposal expressly doesn't
take a hard line on what they should be.

In case it's not entirely clear, the review point prior to acceptance
into -testing is intended to be used solely for automated testing for
really obvious and uncontroversial breakages, like dependency errors.
For instance, a fedora-packager update went into F-13 updates-testing
today with a dependency on a package that's not in the repositories.
What we aim to do in future is have AutoQA check for that kind of
obvious breakage and require it be fixed before the package gets into
updates-testing.

The review point between updates-testing and updates is for more
advanced and potentially manual testing, but again, that's entirely up
for debate. I don't *think* anyone would say the underlying principle
that we should allow for some kind of gatekeeping between
updates-testing and updates is a bad idea, but that's what the
discussion is for Do you think it's a bad idea to allow for this in
the policy? Or are you worried more about the actual criteria used for
the gatekeeping?

> It is an attempt
> at introducing and hardcoding a process, and I'm unhappy with the
> proposals so far as long as they are supposed to cover my packages and
> confine me into something that would be much more plausible for the
> critpath package set.

It would certainly be possible to have different gatekeeping criteria
for critpath and non-critpath updates, under the policy. That may well
be a good idea, especially at first. I'm sure it'll come up at the FESCo
meeting.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-09-2010, 06:18 PM
Adam Williamson
 
Default QA's Package update policy proposal

On Tue, 2010-03-09 at 11:00 -0800, Adam Williamson wrote:

> > It isn't a snapshot of how it works currently, though.
>
> It's fairly close. As far as I can see, all it intentionally changes
> from the current process is that it provides a review point prior to
> acceptance into -testing.

Additional - as Kevin points out, it also seems to require that all
updates be pushed to updates-testing, whereas currently you can push
directly to updates. I'm not sure if this is intentional or accidental,
Kamil's away at present. Whether it's *desired* is something else to
discuss, indeed.
--
Adam Williamson
Fedora QA Community Monkey
IRC: adamw | Fedora Talk: adamwill AT fedoraproject DOT org
http://www.happyassassin.net

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-09-2010, 06:25 PM
Toshio Kuratomi
 
Default QA's Package update policy proposal

On Tue, Mar 09, 2010 at 07:38:44PM +0100, Michael Schwendt wrote:
> On Tue, 09 Mar 2010 10:12:41 -0800, Adam wrote:
>
> > Please provide details on what's mad about Kamil's proposal.
>
> Here:
>
> | The package updates must spend at least 14 days [1] in the
> | 'updates-testing' repository, or at least 7 days [1] provided they have
> | karma of at least 3 [1] and no negative feedback. Only after that the
> | package maintainer may submit them to the 'updates' repository.
>
> Even under consideration of [1], it's the same system that doesn't work
> for me and won't work for me.
>
> Under some circumstances it would be justified to overrule negative
> feedback even. E.g. if it's a user who just wants to block a good update
> because of his pet peeve.
>
Mentioned in the Talk: page here:
https://fedoraproject.org/wiki/User_talk:Kparal/Proposal:Package_update_policy#Responsibilities

kparal acknowledges that it's something like he envisions although I didn't
confirm that that means he agrees with the text in the box or just the UI
design.

I'd rather see all accepted criticism moved to the main page so that we know
what ideas are currently in play in the current proposal but I'm not the
page author so everyone has a different style of writing up drafts.

-Toshio
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-09-2010, 06:53 PM
James Laska
 
Default QA's Package update policy proposal

On Tue, 2010-03-09 at 13:41 -0500, Seth Vidal wrote:
>
> On Tue, 9 Mar 2010, Michael Schwendt wrote:
>
> > If you - and the QA team - want to expand your testing activities, focus
> > on the CRITPATH packages first. Do a good job there. Nobody from QA has
> > ever given feedback to any of my updates, and it won't happen in the
> > future either.
>
>
> I would not be opposed to the above.
> Make the system work on critpath then mandate it from everyone else.

That's a sensible approach, I don't see any harm in using critpath as
the proving ground.

Thanks,
James

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-09-2010, 07:23 PM
Al Dunsmuir
 
Default QA's Package update policy proposal

Hello James,

Tuesday, March 9, 2010, 2:53:22 PM, you wrote:

> On Tue, 2010-03-09 at 13:41 -0500, Seth Vidal wrote:
>>
>> On Tue, 9 Mar 2010, Michael Schwendt wrote:
>>
>> > If you - and the QA team - want to expand your testing activities, focus
>> > on the CRITPATH packages first. Do a good job there. Nobody from QA has
>> > ever given feedback to any of my updates, and it won't happen in the
>> > future either.
>>
>>
>> I would not be opposed to the above.
>> Make the system work on critpath then mandate it from everyone else.

> That's a sensible approach, I don't see any harm in using critpath as
> the proving ground.

> Thanks,
> James

I think rather that if it is not important enough to start with
critpath, one would wonder how critcal the packages in critpath really
are to Fedora.

The second thing that I noticed was that common servers and clients
(such as mock, ssl, dns, ftp, samba, nfs, http were not considered
critical. Why not? Perhaps Fedora needs more layers of critical-ness -
critical, high impact, low impact?

Al

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-09-2010, 07:43 PM
James Laska
 
Default QA's Package update policy proposal

On Tue, 2010-03-09 at 19:50 +0100, Kevin Kofler wrote:
> Adam Williamson wrote:
> > The proposal isn't really about expanding testing activities, it's about
> > formally codifying how the updates process is actually supposed to work.
> > Right now, we don't in fact define how the Fedora update process is
> > supposed to work: how updates get submitted, how they get promoted and
> > how they get released, who has what responsibilities at what point, none
> > of these is written down. That's what the proposal submitted by Kamil is
> > about. It's not really about trying to impose additional testing or
> > restrictions on updates.
>
> Then why does it codify things which are not current practice, and in fact
> require Bodhi changes to be implementable at all?
>
> And why is "updates get pushed to stable when the maintainer feels they are
> stable" (which is basically the current practice) not sufficient? We should
> trust maintainers. That's what we have maintainers for. Stuff like karma
> should only be a tool to help the maintainer decide.

We'll make adjustments based on feedback so far. But I want to point
out that one goal for this policy is to reach consensus on a set of
criteria that all [1] packages must adhere to in order to be accepted as
Fedora updates. The important word for me is "accepted". This comes
long before any functional or bug verification. This is intended to
support the fundamentals of packaging software for Fedora that have
already been established and are used to evaluate all software upon
entry into Fedora [2].

Some basics I'd propose as a starting point for defining acceptance
criteria include:

1. repoclosure/conflicts - no package update can introduce broken
deps or conflicts. I'd recommend we apply this to both
'updates-testing' and 'updates' (but that's detailed below)
2. Package sanity
* No rpmlint failures
* Is the Source properly defined
* License review/examination (if possible)
* Upstream Source match tarball
* Package scriptlet syntax checks
3. Package must be newer than previously released versions - can't
ship newer package in N-1.
4. Any additional MUST requirements folks would like to see covered
from the package review requirements?

Perhaps we can adjust the proposed workflow to detail there are multiple
phases of testing package updates, the first of which is acceptance
testing packages. Once packages have landed in 'updates-testing',
that's when defect and functional verification can start.

With regards to defining acceptance criteria for package updates, does
the following correctly capture the next steps?

1. Determine the appropriate subset of acceptance criteria
2. Determine the appropriate package subset to apply the criteria -
@critpath or all
3. Agree on when the criteria will be applied
* Upon entry into 'updates-testing'
* And upon entry into 'updates' - perhaps same set, or a
more complete set of criteria

Am I missing content, or way off base there?

> > The most important thing is to have a framework explaining who does what
> > at what point in the process; exactly what criteria are used to accept or
> > reject updates is not the main thrust of this proposal. As Kamil wrote,
> > the numbers in there at the moment are essentially placeholders, they
> > could be changed entirely, and that's not the important thing about this
> > proposal.
>
> The numbers aren't the only controversial part of that proposal. Even
> changing all the numbers to 0 would still create a policy which is much more
> restrictive than what we have now (e.g. direct stable pushes aren't in the
> workflow at all; a direct stable push is different from requesting a push to
> stable after 0 days of testing, which assumes that we allowed it to hit
> testing in the first place, so it can only go to stable in the next push).

Good point.

Thanks,
James

[1] In light of earlier feedback, it seems reasonable to restrict this
to critpath packages at first.
[2]
https://fedoraproject.org/wiki/User:Kparal/Proposal:Package_update_policy

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 09:39 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org