On Sat, Nov 20, 2010 at 03:35:43PM -0700, Kevin Fenzi wrote:
> ok, I dug through the devel list for the last month or two and wrote
> down all the various ideas folks have come up with to change/improve
> Here (in no particular order) are the ideas and some notes from me on
> how we could enable them. Please feel free to add new (actual/concrete
> ideas or notes):
> * Just drop all the requirements/go back to before we had any updates
> * Change FN-1 to just security and major bugfix
> This may be hard to enforce or figure out if something is a major bugfix.
> * allow packages with a %check section to go direct to stable
> Bodhi would have to have a list or some way to note these packages,
> it would also need to change as they were added/removed. Perhaps it could
> just be an AutoQA +1 for having a check section?
> On the con side, some checks may be simple and may not note things
> that are fedora only issues.
> * setup a remote test env that people could use to test things.
> This would be lovely with some kind of cloud setup. Have a set of base kickstart
> files and a package/tester could request an instance, install the update, test, and then
> the instance would be removed. We would need a timelimit of some kind,
> and not sure there is any HW in infrastructure for this, but it would sure be cool.
> Perhaps working with the cloud sig we could use EC2 instances for this?
> * require testing only for packages where people have signed up to be testers
> Packages without 'official' testers could bypass testing or have some lower karma
> requirement. We would need for this a list of packages that have had people sign
> up to test.
> * Ask maintainers to provide test cases / test cases in wiki for each package?
> Test cases are not easy to make, many maintainers won't can't do so, but
> it would be lovely to have even a base checklist of things that should work
> in the package everytime.
> * have a way to get interested testers notified on bodhi updates for packages
> they care about.
> We would need to add some kind of tester list to pkgdb, and bodhi would need to be
> able to get this to mail them when a update changed state.
> We may not get many people signing up for some packages, but this might be a
> good way to know what packages we have testers for and get them more involved
> in testing. Ideally it could mail them on update submission at least.
> * reduced karma requirement on other releases when one has gone stable
> Bodhi would need to note when a update went to stable if the exact same
> version (with dist tag differences) was in testing for other releases.
> It could then allow less karma to go stable, or add +1 from the other
> update going stable.
> Other concrete ideas?
Security update ideas:
* Security updates may push directly to stable -- this would depend on
our weighing of costs: a security update may be broken. OTOH, if we
don't get security updates out fast, is that a worse danger to our users?
I think I'd like to try one of the other options below first but those
require a bit more work/coordination so we need to think of the cost to
implement as well.
* Ask the QA team to commit to testing security updates. The idea is that
updates flagged security have a dedicated pool of testers that will check
them and get them out ASAP.
* Security updates have a small timeout period -- there's two questions
here: Is a week (as is currently the case for non-critpath) too long?
Can wehave a timeout even for critpath packages so a security update
doesn't get held up forever?
* critpath package timeout -- let's have a timeout period even for critpath
packages. Perhaps this could be longer than for non-critpath. The big
issue that people have observed with depending on timeouts, though, is
that pushing new updates in the meantime resets the time that a package
needs to wait to get to stable. Could bodhi be changed to let multiple
packages be in the testing repository at one time and only obsolete them
when a newer package enters the stable repo? That would allow longer
timeout periods to make more sense.
Lack of manpower ideas:
* Allow anonymous karma to count. Anonymous karma would allow more people
who report bugs in bugzilla to add karma in bodhi without having to get
a second account in the Fedora Account System. For critpath packages,
we're already mandating that a proventester must give karma so we're
making sure that someone with an account is giving feedback there.
* Do we believe that there's value in silent testing (knowing that people
have a package installed from testing but have not submitted karma either
way?) If so, create a tool that ties into yumdb and if the user opts in,
submits that data to us. Then add karma based upon that data.
Variation on an option you recorded:
* Drop testing requirement until AutoQA can give karma. Rework based on
what tests auto qacan perform.
devel mailing list