FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 03-10-2010, 04:13 PM
Bruno Wolff III
 
Default QA's Package update policy proposal

On Tue, Mar 09, 2010 at 15:43:04 -0500,
James Laska <jlaska@redhat.com> wrote:
>
> 1. repoclosure/conflicts - no package update can introduce broken
> deps or conflicts. I'd recommend we apply this to both
> 'updates-testing' and 'updates' (but that's detailed below)
> 2. Package sanity
> * No rpmlint failures
> * Is the Source properly defined
> * License review/examination (if possible)
> * Upstream Source match tarball
> * Package scriptlet syntax checks
> 3. Package must be newer than previously released versions - can't
> ship newer package in N-1.
> 4. Any additional MUST requirements folks would like to see covered
> from the package review requirements?

File conflicts (assuming that "conflicts" above referred to just conflicts
dependencies).
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-10-2010, 07:57 PM
James Laska
 
Default QA's Package update policy proposal

On Tue, 2010-03-09 at 17:18 -0800, Adam Williamson wrote:
> On Tue, 2010-03-09 at 17:13 -0700, Kevin Fenzi wrote:
>
> > > Some basics I'd propose as a starting point for defining acceptance
> > > criteria include:
> > >
> > > 1. repoclosure/conflicts - no package update can introduce broken
> > > deps or conflicts. I'd recommend we apply this to both
> > > 'updates-testing' and 'updates' (but that's detailed below)
> > > 2. Package sanity
> > > * No rpmlint failures
> >
> > I think this one should not be there. Or should be heavily filtered.
> > rpmlint sometimes marks things as errors that are not.
>
> +1 here. The current upstream rpmlint errors/warnings list is not what
> we'd want to use to automatically approve/deny updates, I'm fairly sure.
> =) We'd need to either get quite drastic changes upstream, or overlay
> our own error/warning split on the tests (which is what Mandriva does;
> there's an automated rpmlint run on every package submission and it
> rejects packages if certain tests fail, but the definition of which
> tests are critical is *not* the upstream one).
>
> Over time this would work fine, but at the start it may possibly
> introduce some absurdity due to 'grandfathered in' situations: an update
> may be rejected due to some lint failure which was actually present in
> the version of the package that's being updated anyway (it's not a
> _change_ in the update). I'm not sure if that's really a problem - we
> could just argue that the problems should be fixed and when you're
> pushing an update is as good a time as any to fix them - but I thought
> it'd be worth mentioning in advance just in case. Anyway, we should be
> very careful and conservative to start with, in terms of what automated
> tests we introduce. I'd recommend we do a week or two where we 'dry run'
> the system - generate lists of updates which would have been blocked,
> but don't actually block them - to make sure the results are sane,
> before we start actually blocking things.

All fair points. Given that rpmlint is only run when packages are first
accepted into Fedora ... I suspect many packages would not meet this
criteria.

Another idea was to not allow any *new* rpmlint failures. Any failures
that exist in previous builds would be permitted.

Thanks,
James
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-10-2010, 08:00 PM
James Laska
 
Default QA's Package update policy proposal

On Wed, 2010-03-10 at 13:18 +0000, Paul Howarth wrote:
> On 09/03/10 20:43, James Laska wrote:
> > Some basics I'd propose as a starting point for defining acceptance
> > criteria include:
> >
> > 1. repoclosure/conflicts - no package update can introduce broken
> > deps or conflicts. I'd recommend we apply this to both
> > 'updates-testing' and 'updates' (but that's detailed below)
> > 2. Package sanity
> > * No rpmlint failures
>
> rpmlint, in common with many other "lint" tools, reports things that it
> thinks *may* be errors that actually are intended. To regard "no rpmlint
> failures" as a package sanity check is way over the top I think.
>
> Comparing the rpmlint output for an updated package with the rpmlint
> output for the currently in-repo package would be more useful as that
> could identify any new issues, but there should still be a means to
> override rpmlint if the maintainer can explain why it's not a problem.

Agreed. Comparing the output between the previous and the proposed
build seems to be a fair compromise.

Thanks,
James
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-10-2010, 08:00 PM
James Laska
 
Default QA's Package update policy proposal

On Wed, 2010-03-10 at 11:13 -0600, Bruno Wolff III wrote:
> On Tue, Mar 09, 2010 at 15:43:04 -0500,
> James Laska <jlaska@redhat.com> wrote:
> >
> > 1. repoclosure/conflicts - no package update can introduce broken
> > deps or conflicts. I'd recommend we apply this to both
> > 'updates-testing' and 'updates' (but that's detailed below)
> > 2. Package sanity
> > * No rpmlint failures
> > * Is the Source properly defined
> > * License review/examination (if possible)
> > * Upstream Source match tarball
> > * Package scriptlet syntax checks
> > 3. Package must be newer than previously released versions - can't
> > ship newer package in N-1.
> > 4. Any additional MUST requirements folks would like to see covered
> > from the package review requirements?
>
> File conflicts (assuming that "conflicts" above referred to just conflicts
> dependencies).

Ah yes. I wasn't specific enough about, but file conflicts is what was
meant.

Thanks,
James
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-10-2010, 09:50 PM
Al Dunsmuir
 
Default QA's Package update policy proposal

On Tuesday, March 9, 2010, 8:09:40 PM, Kevin Kofler wrote:

>> Jesse Keating wrote:
>> On Tue, 2010-03-09 at 16:08 -0800, Josh Stone wrote:
>>> It seems to cast doubt on the value of karma -- just because something
>>> gets lots of positive karma on N doesn't mean that N-1 is ok. Then
>>> again, the same concern is present in any grouped update if the voters
>>> haven't tried *all* of the packages mentioned.
>>
>> Even if you put an update for N and N-1 in the same form, once you
>> submit the request it splits it into two requests, one per Fedora
>> release. This means you'd have one set of karma per Fedora release.

> Indeed, and I'd argue that this is a problem, not a feature. If an update is
> confirmed to fix an issue in the current stable release and the previous
> stable release is affected by the exact same issue, I don't see a good
> reason not to push the update with identical changes to the previous stable
> release as well. Not doing it would result in the previous stable release
> not getting bugfixes in a timely manner, if at all, anymore, as it has a lot
> fewer testers.
> Kevin Kofler

Just to be clear, I am in the "adventurous user" camp that would
prefer to have the bug fix retrofitted to the older release. I would,
however, qualify that as follows:

The update to an older stable release should be made widely available
in that release's updates-testing after the equivalent (not
necessarily identical) fix has been widely tested in the latest stable
release. To me, this means separately developer unit tested, tested by
users in updates-testing and then for several weeks of real user use
in stable.

In parallel with that, separate developer unit testing of the older
release should be done, with entry to the older release
updates-testing only after the original fix has been proven in the
latest stable release.

This minimizes the risk that due to a different execution environment,
build environment, configuration or whatever the seemingly equivalent
fix does not work but causes a regression. You may start at the same
place in the older stable release, but may end up down and entirely
different rabbit hole.

Firing off the "same" fix for multiple targets other than rawhide and
the latest release makes me nervous.

So to summarize, fixes in multiple releases yes, but with separate
schedules for build and testing.
Al

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-10-2010, 11:11 PM
Kevin Kofler
 
Default QA's Package update policy proposal

Al Dunsmuir wrote:
> The update to an older stable release should be made widely available
> in that release's updates-testing after the equivalent (not
> necessarily identical) fix has been widely tested in the latest stable
> release.

Uh, no, just no.

They should go to updates-testing for both releases at the same time.
Anything else:
1. makes things harder for the maintainer, as he/she has to go through all
the Bodhi procedures twice,
2. just delays the fix for users for no good reason.

I can somewhat understand the argument that they should get separate testing
(even though I disagree with it), or even that the stable pushes should be
staged (even though I also disagree with that), but I really don't see what
it hurts to have the update available in updates-testing right away. Testing
is what updates-testing is for.

> This minimizes the risk that due to a different execution environment,
> build environment, configuration or whatever the seemingly equivalent
> fix does not work but causes a regression. You may start at the same
> place in the older stable release, but may end up down and entirely
> different rabbit hole.

Is this really such a common issue that it makes it worth delaying all
updates, including bugfixes, while waiting for testing that may never arrive
(because those folks who like testing things tend to run the current stuff)?

Kevin Kofler

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 10:49 AM
Al Dunsmuir
 
Default QA's Package update policy proposal

On Wednesday, March 10, 2010, 7:11:31 PM, Kevin Kofler wrote:

> Al Dunsmuir wrote:
>> The update to an older stable release should be made widely available
>> in that release's updates-testing after the equivalent (not
>> necessarily identical) fix has been widely tested in the latest stable
>> release.

> Uh, no, just no.

> They should go to updates-testing for both releases at the same time.
> Anything else:
> 1. makes things harder for the maintainer, as he/she has to go through all
> the Bodhi procedures twice,
> 2. just delays the fix for users for no good reason.

> I can somewhat understand the argument that they should get separate testing
> (even though I disagree with it), or even that the stable pushes should be
> staged (even though I also disagree with that), but I really don't see what
> it hurts to have the update available in updates-testing right away. Testing
> is what updates-testing is for.

A lot of the conflict that I am seeing is based on two different basic
assumptions: 1) the update will be good and 2) the update will fail.

It is OK after sufficient unit testing to go with approach #1 on the
latest release. If it turns out there are problems, they can be fixed
in one or more additional iterations.

For older releases, the presumption/requirement for stability is
higher. To meet that, I believe more people would prefer going with
approach #2 for these releases. Once the update works out OK with
no regressions in the latest release, you have a much higher
probability that your the equivalent update to the older release will
also be stable and meet that release's stability requirement.

Throwing both into their respective updates-testing simultaneously
does allow for wider user testing... but is limited by the actual
amount of testing. Allowing both releases to be promoted to stable at
the same time is the real problem. If you do not (or can not) hold
back the older (and in user expectations, more stable) release update
than any problem will have a much higher perceived and actual impact.
If you don't have the resources to ensure that older releases remain
more stable than newer releases, perhaps you do need to revisit
whether updates to both releases are a good idea.

>> This minimizes the risk that due to a different execution environment,
>> build environment, configuration or whatever the seemingly equivalent
>> fix does not work but causes a regression. You may start at the same
>> place in the older stable release, but may end up down and entirely
>> different rabbit hole.

> Is this really such a common issue that it makes it worth delaying all
> updates, including bugfixes, while waiting for testing that may never arrive
> (because those folks who like testing things tend to run the current stuff)?
> Kevin Kofler

Problems will happen, likely in the worst possible way at the worst
possible time for your users. If you work on that assumption, it just
makes sense to take more care and time to roll out your fixes the more
stable the release is expected to be.

If you and the users have a mismatch regarding expectations of
stability, it doesn't matter whether your changes are fixes or
retrofitted enhancement - there will be highly visible problem if you
try to cut corners in either time or effort.

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 12:09 PM
Kevin Kofler
 
Default QA's Package update policy proposal

Al Dunsmuir wrote:
> For older releases, the presumption/requirement for stability is
> higher.

Nonsense. The previous and current stable releases are both equally
supported, there isn't one which is "more stable" than the other.

> If you don't have the resources to ensure that older releases remain
> more stable than newer releases, perhaps you do need to revisit
> whether updates to both releases are a good idea.

The goal of continuing to maintain the previous stable release is NOT to
have a more conservative release available, but simply to allow users to
pick their own time for upgrading to the new release due to the disruptive
changes made between the old and the new release (i.e. those changes which
are intentionally NOT being pushed as updates, e.g. because they remove
features, require manual configuration changes or whatever reason). In fact,
the EOL time is chosen such that users can opt to skip a release entirely.
This doesn't mean that those users do not expect to get the same kind of
updates the current stable release gets (i.e. non-disruptive, but not
particularly conservative updates). In fact it's quite the opposite, as a
user I expect the release to be supported equally throughout its lifetime.

Kevin Kofler

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 12:28 PM
Al Dunsmuir
 
Default QA's Package update policy proposal

Hello Kevin,

Thursday, March 11, 2010, 8:09:02 AM, you wrote:

> Al Dunsmuir wrote:
>> For older releases, the presumption/requirement for stability is
>> higher.

> Nonsense. The previous and current stable releases are both equally
> supported, there isn't one which is "more stable" than the other.

Often the reason that folks are using an older stable release because
they *can not* update to the new stable release because it doesn't
work for them, or they *choose* to wait until the new release is
*proven* stable.

If you ignore that and assume you are free to do as you will to any
active release, I would submit you are putting your own wants ahead of
your users. Everyone loses in that case, and it is quite natural that
conflict arises.

Simultaneously updating all active releases is like climbing a
mountain without safety ropes. It works fine while everything goes
well, but the first slip means you are in for a big fall. Maintaining
the older releases with a heightened emphasis on stability is
Fedora's safety rope.

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 03-11-2010, 11:41 PM
Kevin Kofler
 
Default QA's Package update policy proposal

Al Dunsmuir wrote:
> Often the reason that folks are using an older stable release because
> they *can not* update to the new stable release because it doesn't
> work for them,

But that's exactly why we want to continue pushing those updates that do
work for them. :-)

> or they *choose* to wait until the new release is *proven* stable.

I don't see non-conservative, but non-disruptive updates, nor the practice
of pushing updates simultaneously to all supported releases as conflicting
with that.

Kevin Kofler

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 06:34 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org