FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Server Development

 
 
LinkBack Thread Tools
 
Old 05-03-2012, 03:23 PM
James Page
 
Default UDS-Q Topic: Server Bug Triage Process

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Hi All

At UDS-Q we will be discussing the relatives merits and deficiencies
of the bug triage process that we have been using in the server team
for the last 12 months or so (see [0]) as well as decided on how to
approach the current backlog of bugs.

FYI people on the weekly triage rota normally use the triage report
[1] to identify bugs to work on for their allocated slots.

I'd be interested in hearing peoples opinions on the current process
in advance of UDS - specifically what you think works and what does
not and how you think things might be improved.

Regards

James


[0] https://wiki.ubuntu.com/ServerTeam/KnowledgeBase#triager
[1] http://status.qa.ubuntu.com/reports/ubuntu-server/triage-report.html
- --
James Page
Ubuntu Core Developer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJPoqL9AAoJEL/srsug59jD4wQP/AwDMbVNoSf3GYNFVnOwvwrf
IClUAyStGc9vwTXjtsCiORZSPuanQt3QGvYmYv9ex6DGOz+XiT V/IhhFdrz/kTA/
G6AAkdTN9I0w8uMmbtwTMOQlVd+qe2FYNTqzD4aw6GgMjT6Tup zw1R9il68UGHw+
GrWAWvYyg3uV2YqRG2iLL/DJshFa+rlZmF9H+HdcUD5sAKC+ZtvgQ5JtN2KGmUTO
Cftrq3nrwCebX9ogsPp+h6VKpVcYYoB6pFHaBbcLKwOViWLBvr JzfCPYnwlLn01o
qafqTd6+4XCT9zNTbKzWXFXhhRUZ6QcAEjQG4QIgVEd4T3L6g3 5l3Boxinc8vU5f
hA8A2xKja4e6ovO9DEdCdqr3ejxsXPpz2+MA6bkNnQJIUBoKLZ zwSiCysa7Q+IMm
SO4xLos0Ied+WxqleN7PAazy7kAHlYhdc23ovR0MJX/3FPPG46W5y3Y4q8jnrKhw
/ISekGVq5kuZhiJ/FjhRe8O3eQeKN1OP/0SU11WvAp+tDy8M74bp4uhfcKfrGq4s
UszdVf2JJg5HvxhMnOeOWbBpkkXbuBxwYpsUXqhH5P5QfnHY6B tOu87gCU1CIwBd
yjrapuFh3Fvwuuap/wU4c2y3X+FbuzXfkFQUJoSr3v0qr3NvllYGNhn2mMTqYv/A
FEOc68t1DkMpcRvdpqHg
=F+3T
-----END PGP SIGNATURE-----

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam
 
Old 05-03-2012, 04:14 PM
James Page
 
Default UDS-Q Topic: Server Bug Triage Process

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

Forgot to link the actual blueprint:

https://blueprints.launchpad.net/ubuntu/+spec/servercloud-q-bug-triage-review



On 03/05/12 16:23, James Page wrote:
> Hi All
>
> At UDS-Q we will be discussing the relatives merits and
> deficiencies of the bug triage process that we have been using in
> the server team for the last 12 months or so (see [0]) as well as
> decided on how to approach the current backlog of bugs.
>
> FYI people on the weekly triage rota normally use the triage
> report [1] to identify bugs to work on for their allocated slots.
>
> I'd be interested in hearing peoples opinions on the current
> process in advance of UDS - specifically what you think works and
> what does not and how you think things might be improved.
>
> Regards
>
> James
>
>
> [0] https://wiki.ubuntu.com/ServerTeam/KnowledgeBase#triager [1]
> http://status.qa.ubuntu.com/reports/ubuntu-server/triage-report.html


- --
>
James Page
Ubuntu Core Developer
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQIcBAEBCAAGBQJPoq7kAAoJEL/srsug59jDu0QP/3iZD9Ev2B+8Yw77U++9FbzS
oQfQDi7C9pB8BRru3MZdH4AqoO3G6Wouxy+1xDLxFb0ItMyKBJ Xh1xymH/g02g/t
vg3olGBKhNgymNnrPZNbmwEBOcyzIH7+TEu58eF+UZfNu8hv/HjRww+t+TgOo9P8
xxqeEe0TC5lFfW+1Kec2MWtztT5da+qk0zQAtLEdaoYq6uatFF 6ROmf36f2rNPk/
tqiLb0XASzlVmGuoWryuJASLkR1guulOpD1w0+UrWyLqR5Id1s gtzpYePN/1E5wD
m1MgxR42yFmWz9EaO1r9MRf/0eVIRcuo6peqeBZ/1EgHNRUCH0SuELrZFCqOzXLh
HVqPV4Hxr76nUysppMvrEZwJVDAGf2VPRavLgxOIeFCGnIBPZH meHuayAuztlpiU
hPsMwLVMaCLb12S1pzJm2E4sXcvcqND9WccQIUKZk4q85AD26S 1BWffGhWiMZYeM
Ptfo233MsFlO5sASkCRwR3kAMnYP5y1MSdpFCx010vhFiPnIbb yXqjV0mZ1t432P
m3damEUXkTDgYsFdUzzp81HAYzFVZxeFvz7QmiK+2rszAOWJEn N8Soq4X8vbXlLc
5Y0a45qSmzwowLAR2ir2kgmBmNx+0VqP02eiKtb9aoYGXsh8BP hqf+ru1JVN5jhb
/LI9Drt6aMbS3uvVMCpi
=3qfG
-----END PGP SIGNATURE-----

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam
 
Old 05-07-2012, 06:41 PM
Robie Basak
 
Default UDS-Q Topic: Server Bug Triage Process

There were two issues I wanted to bring up. I apologise for not managing
to send this email before UDS - I was away last week. So this is a bit
of a brain dump.

1. Triage list does not reduce after triaging is complete.

Bugs that can be worked on out of a triage list will inevitably be
progressed. Bugs that cannot be worked on out of a triage list will stay
there. So any list which permits bugs that cannot be worked on will tend
to fill up with bugs that cannot be worked on. This can make such a list
useless. So bugs that cannot be progressed must somehow be prevented
from appearing in any triage list.

So with this requirement in mind, I think that the current issues are:

a) Many bugs currently stay on the triage list because we can't progress
them. In particular, this includes bugs that are reported adequately but
not confirmed and with no instructions to reproduce. This is a
reasonable bug state, but unfortunately triagers can do nothing with
them. I think that these bugs are what cause most of the triaging
difficulty at the moment. For the purposes of this discussion I'm going
to call these bugs "awkward" bugs.

b) Bug importance is very difficult to select for awkward bugs. This is
because the real importance depends on the impact of the bug, and at
this stage this is often unknown. Importance can of course be changed
later, but setting a bug to low importance currently makes it unlikely
that it will be looked at again soon. If a bug is reported once,
initially appears not to be severe, but consequently turns out to be
very important, it could be missed.

c) If we set the Importance to remove awkward bugs from our first triage
list, all we are really doing is shifting the problem to a future triage
list. Really the awkward bugs should not appear on a triage list at all
until they become non-awkward (eg. get confirmed, or somebody posts
further information or steps to reproduce).

What I'm looking for is a way to say "this bug looks fine but cannot be
worked on until something happens that is out of the control of a
triager". Currently we keep state only in bugs, which is perhaps a good
idea. But it seems that we're unwilling to mark this state as such in
the bug, perhaps in order not to discourage the community from
submitting them. What would be really nice would be some kind of tag or
marker that automatically disappears as soon as a bug is changed in any
way so that it gets the attention of a triager again.

So our options are: 1) maintain state outside of the bugs themselves, 2)
mark the "cannot be worked on state" inside the bug using a tag, or 3)
be forever plagued with bugs in our triage lists


2. Automatic submission of package maintainer script failures

2a. Inadequate bug description often makes these very difficult to
triage, and they stay on the list. Often the description is so vague
that it's difficult to even prompt for further information without
turning the bug into a full one-on-one support channel which we don't
really have the bandwidth to do. I propose that these bugs are managed
separately from regular triage, and/or have a standard "inadequate
report" type standard response, but written specifically for maintainer
script failures (since the usual one doesn't seem very appropriate for
these cases).

2b. The reporters of most of these bugs with inadequate descriptions
appear not to be interested in taking them further.

2c. Most are support requests and not bugs, reducing the time that
triagers get to triage actual bugs.

2d. Desktop package maintainer scripts are expected rarely to fail, and
if they do it probably is a bug. Server package maintainer scripts often
try to set up daemons, but users customise daemons beyond what
maintainer scripts can be expected to understand, causing them to fail
and encourage reporting as bugs. And desktop users often install server
packages which then confuses the matter when they try and upgrade.
Linked with 2c, this causes a big problem for bug triage.

Of course, there is no real distinction between server and desktop
packages expect in the nature of what they are expected to do, and thus
what their maintainer scripts try to do.

I think it would be appropriate to see these bugs dealt with
automatically (boilerplate response) unless or until:

1) It's a "proper" bug report where the submitter has actually spent
some effort in reporting it (ie. with a bug description on par with
what we'd expect for a manually submitted bug).

2) Aggregrate analysis suggests that the bug is real and thus we
should look into it.

3) We've caught up with all the other bugs.

At this point, I suggest always changing the subject of the bug from the
automatic subject line to one that describes the real problem. This
would really help to make the real bugs stand out from the automatic
ones.

The boilerplate would of course explain the situation with instructions
on how to get the bug looked at if the reporter wants to take it further
(eg. a variant of
https://wiki.ubuntu.com/Bugs/Responses#Not_described_well)

Robie

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam
 
Old 05-07-2012, 07:28 PM
Serge Hallyn
 
Default UDS-Q Topic: Server Bug Triage Process

Thanks, Robie, all well thought out and stated. I agree with each point
here.

-serge

Quoting Robie Basak (robie.basak@canonical.com):
> There were two issues I wanted to bring up. I apologise for not managing
> to send this email before UDS - I was away last week. So this is a bit
> of a brain dump.
>
> 1. Triage list does not reduce after triaging is complete.
>
> Bugs that can be worked on out of a triage list will inevitably be
> progressed. Bugs that cannot be worked on out of a triage list will stay
> there. So any list which permits bugs that cannot be worked on will tend
> to fill up with bugs that cannot be worked on. This can make such a list
> useless. So bugs that cannot be progressed must somehow be prevented
> from appearing in any triage list.
>
> So with this requirement in mind, I think that the current issues are:
>
> a) Many bugs currently stay on the triage list because we can't progress
> them. In particular, this includes bugs that are reported adequately but
> not confirmed and with no instructions to reproduce. This is a
> reasonable bug state, but unfortunately triagers can do nothing with
> them. I think that these bugs are what cause most of the triaging
> difficulty at the moment. For the purposes of this discussion I'm going
> to call these bugs "awkward" bugs.
>
> b) Bug importance is very difficult to select for awkward bugs. This is
> because the real importance depends on the impact of the bug, and at
> this stage this is often unknown. Importance can of course be changed
> later, but setting a bug to low importance currently makes it unlikely
> that it will be looked at again soon. If a bug is reported once,
> initially appears not to be severe, but consequently turns out to be
> very important, it could be missed.
>
> c) If we set the Importance to remove awkward bugs from our first triage
> list, all we are really doing is shifting the problem to a future triage
> list. Really the awkward bugs should not appear on a triage list at all
> until they become non-awkward (eg. get confirmed, or somebody posts
> further information or steps to reproduce).
>
> What I'm looking for is a way to say "this bug looks fine but cannot be
> worked on until something happens that is out of the control of a
> triager". Currently we keep state only in bugs, which is perhaps a good
> idea. But it seems that we're unwilling to mark this state as such in
> the bug, perhaps in order not to discourage the community from
> submitting them. What would be really nice would be some kind of tag or
> marker that automatically disappears as soon as a bug is changed in any
> way so that it gets the attention of a triager again.
>
> So our options are: 1) maintain state outside of the bugs themselves, 2)
> mark the "cannot be worked on state" inside the bug using a tag, or 3)
> be forever plagued with bugs in our triage lists
>
>
> 2. Automatic submission of package maintainer script failures
>
> 2a. Inadequate bug description often makes these very difficult to
> triage, and they stay on the list. Often the description is so vague
> that it's difficult to even prompt for further information without
> turning the bug into a full one-on-one support channel which we don't
> really have the bandwidth to do. I propose that these bugs are managed
> separately from regular triage, and/or have a standard "inadequate
> report" type standard response, but written specifically for maintainer
> script failures (since the usual one doesn't seem very appropriate for
> these cases).
>
> 2b. The reporters of most of these bugs with inadequate descriptions
> appear not to be interested in taking them further.
>
> 2c. Most are support requests and not bugs, reducing the time that
> triagers get to triage actual bugs.
>
> 2d. Desktop package maintainer scripts are expected rarely to fail, and
> if they do it probably is a bug. Server package maintainer scripts often
> try to set up daemons, but users customise daemons beyond what
> maintainer scripts can be expected to understand, causing them to fail
> and encourage reporting as bugs. And desktop users often install server
> packages which then confuses the matter when they try and upgrade.
> Linked with 2c, this causes a big problem for bug triage.
>
> Of course, there is no real distinction between server and desktop
> packages expect in the nature of what they are expected to do, and thus
> what their maintainer scripts try to do.
>
> I think it would be appropriate to see these bugs dealt with
> automatically (boilerplate response) unless or until:
>
> 1) It's a "proper" bug report where the submitter has actually spent
> some effort in reporting it (ie. with a bug description on par with
> what we'd expect for a manually submitted bug).
>
> 2) Aggregrate analysis suggests that the bug is real and thus we
> should look into it.
>
> 3) We've caught up with all the other bugs.
>
> At this point, I suggest always changing the subject of the bug from the
> automatic subject line to one that describes the real problem. This
> would really help to make the real bugs stand out from the automatic
> ones.
>
> The boilerplate would of course explain the situation with instructions
> on how to get the bug looked at if the reporter wants to take it further
> (eg. a variant of
> https://wiki.ubuntu.com/Bugs/Responses#Not_described_well)
>
> Robie
>
> --
> ubuntu-server mailing list
> ubuntu-server@lists.ubuntu.com
> https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
> More info: https://wiki.ubuntu.com/ServerTeam

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam
 
Old 05-07-2012, 10:44 PM
Scott Kitterman
 
Default UDS-Q Topic: Server Bug Triage Process

On Monday, May 07, 2012 11:41:46 AM Robie Basak wrote:
> So our options are: 1) maintain state outside of the bugs themselves, 2)
> mark the "cannot be worked on state" inside the bug using a tag, or 3)
> be forever plagued with bugs in our triage lists

4) Go back to the way it used to be (no auto expiring of incomplete) and mark
such bugs as incomplete. Currently incomplete mostly means "I'm marking your
bug invalid, but I'm not willing to tell you - I'll let LP do it after it
times out".

Scott K

--
ubuntu-server mailing list
ubuntu-server@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-server
More info: https://wiki.ubuntu.com/ServerTeam
 

Thread Tools




All times are GMT. The time now is 04:39 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org