Provide more testing feedback (was: Refining the update queues/process)
On Wed, Mar 03, 2010 at 02:06:33PM -0800, Adam Williamson wrote:
> On Wed, 2010-03-03 at 17:04 +0100, Till Maas wrote:
> > I mind have misunderstood it, but afaics it only says that it will be
> > tested, because it spent time in updates-testing, but this is not even
> > true nowadays, even if packages stay long in updates-testing.
> as we've explained several times, most packages that go to
> updates-testing for a few days *are* being tested, even if they get no
> apparent Bodhi feedback. Several QA group members run with
> updates-testing enabled and so get all packages (that they have
> installed) which go through updates-testing. They do not file positive
If it is limited on the packages the user is installed, a lot of
packages are not tested.
> feedback for every single package because there's just too many, but if
> they notice breakage, they file negative feedback.
So here is a first ugly script to easily give feedback for all installed
testing updates that were created after a certain date (I did not find
an easy way to get all testing updates, one did not yet comment on):
It kind of works like this: Show the update, ask for karma to create a
comment, otherwise it is skipped. Then a comment needs to be entered.
Sometimes it will ask for the FAS password using the bodhi client.
Comment? -1/0/1 ->karma, other -> skip> 1
Comment> "rpmdev-newspec foo" still works
The prompt has readline support, so if you just want to use the same
comment again and again, you can use cursor-up to get the comment again.
Btw. the script fails in case only subpackages of a package are
installed. But if it is used, I will create a saner version.
Nevertheless, there is also now the Fedora Engineering Service, so if
you want to improve your testing tools, you might want to ask there.
> So - for the third time - a package being in updates-testing for a few
> days and getting no negative feedback is a moderate strength indicator
> that it's not egregiously broken. Not a super-strong indicator, but
> better than a kick in the teeth.
It probably only means that the meta-data of the installed package is
not broken, but if they do not use all packages installed daily, then
there is not much test coverage.
devel mailing list