Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   ArchLinux Development (http://www.linux-archive.org/archlinux-development/)
-   -   Possible signoff addendum (http://www.linux-archive.org/archlinux-development/151544-possible-signoff-addendum.html)

"Aaron Griffin" 08-31-2008 05:12 AM

Possible signoff addendum
 
One of the things I was thinking about, because sometimes signoffs can
stagnate....

Should we have a time-based limit?

That is, if a package has been in testing for X days, with no
complaints, but no signoffs, yet, can we consider it fully functional,
assuming the packager is comfortable with that.

I'd like opinions on this. I'm trying to solve the issue of the big
ol' backlog of pending-signoff packages.

Cheers,
Aaron
:wq

Allan McRae 08-31-2008 08:24 AM

Possible signoff addendum
 
Aaron Griffin wrote:

One of the things I was thinking about, because sometimes signoffs can
stagnate....

Should we have a time-based limit?

That is, if a package has been in testing for X days, with no
complaints, but no signoffs, yet, can we consider it fully functional,
assuming the packager is comfortable with that.

I'd like opinions on this. I'm trying to solve the issue of the big
ol' backlog of pending-signoff packages.



I quite like this idea. A "no bug report after given period of time =
no problems" strategy should work as the community can do a lot more
testing in total than we can.


Allan

Eduardo Romero 08-31-2008 03:35 PM

Possible signoff addendum
 
On Sun, 2008-08-31 at 18:24 +1000, Allan McRae wrote:
> Aaron Griffin wrote:
> > One of the things I was thinking about, because sometimes signoffs can
> > stagnate....
> >
> > Should we have a time-based limit?
> >
> > That is, if a package has been in testing for X days, with no
> > complaints, but no signoffs, yet, can we consider it fully functional,
> > assuming the packager is comfortable with that.
> >
> > I'd like opinions on this. I'm trying to solve the issue of the big
> > ol' backlog of pending-signoff packages.
> >
>
> I quite like this idea. A "no bug report after given period of time =
> no problems" strategy should work as the community can do a lot more
> testing in total than we can.
>
> Allan
>
>
+1, probably by X days enough enough users have tested the package, and
if there are no bugs it can be considered stable.

"Dusty Phillips" 08-31-2008 04:11 PM

Possible signoff addendum
 
2008/8/31 Aaron Griffin <aaronmgriffin@gmail.com>:
> One of the things I was thinking about, because sometimes signoffs can
> stagnate....
>
> Should we have a time-based limit?
>
> That is, if a package has been in testing for X days, with no
> complaints, but no signoffs, yet, can we consider it fully functional,
> assuming the packager is comfortable with that.
>
> I'd like opinions on this. I'm trying to solve the issue of the big
> ol' backlog of pending-signoff packages.

I can imagine this causing people to stop signing off altogether
("meh, it'll be signed off automatically in a couple weeks"). This
could lead to people not testing altogether or to a "packages must
spend X days in testing" philosophy in the long run. This could knock
the 'bleeding edge' off the Arch.

I think its better to find a way to encourage people to do more
signing off. Some kind of incentive or motivation. Some ideas:

- policy to do at least two signoffs for every package submitted to
testing (bump it to three or four in times where testing is
backlogged)

- count the number of signoffs people do on the web based signoff and
reward the big signoffers with recognition and the non-signoffers with
negative recognition (possibly just a list of top three and bottom
three signoffers on the dashboard or signoffs page). This sounds a bit
big brotherish...

- somehow flag packages that have been in testing for an unreasonable
amount of time. Possibly sorting the signoffs list by length in
testing, listing exceptionally long-time testing packages in the
dashboard, e-mailing the list periodically with a list of such
packages, etc.

Another alternative:

- Create a web based interface to allow community users to signoff on
packages. We could have a policy of X number of user signoffs is equal
to one developer signoff. Developers can be trusted to test it better
(or not? ;-), but arch members can contribute too. In that case we
still need some incentive to the end users to bother doing this. There
aren't a lot of people like dolby and cactus who seem to actually like
working really hard with as little recognition as possible.

Dusty

"Aaron Griffin" 09-02-2008 04:59 AM

Possible signoff addendum
 
On Sun, Aug 31, 2008 at 11:11 AM, Dusty Phillips <buchuki@gmail.com> wrote:
> Another alternative:
>
> - Create a web based interface to allow community users to signoff on
> packages. We could have a policy of X number of user signoffs is equal
> to one developer signoff. Developers can be trusted to test it better
> (or not? ;-), but arch members can contribute too. In that case we
> still need some incentive to the end users to bother doing this. There
> aren't a lot of people like dolby and cactus who seem to actually like
> working really hard with as little recognition as possible.

I actually thought about this a while back. It was in terms of "each
package needs 10 points, developer signoffs are 5 points, TU signoffs
are 3 points, user signoffs are one point"

It might be a useful way to find active community members too. If you
could think of a way to creatively implement something like this, I'd
be all for it, assuming other people are.

But yeah, I think you may be right about the "encouraging people not
to sign off at all" part. It would make people lazy.

Paul Mattal 09-02-2008 12:02 PM

Possible signoff addendum
 
Aaron Griffin wrote:

On Sun, Aug 31, 2008 at 11:11 AM, Dusty Phillips <buchuki@gmail.com> wrote:

Another alternative:

- Create a web based interface to allow community users to signoff on
packages. We could have a policy of X number of user signoffs is equal
to one developer signoff. Developers can be trusted to test it better
(or not? ;-), but arch members can contribute too. In that case we
still need some incentive to the end users to bother doing this. There
aren't a lot of people like dolby and cactus who seem to actually like
working really hard with as little recognition as possible.


I actually thought about this a while back. It was in terms of "each
package needs 10 points, developer signoffs are 5 points, TU signoffs
are 3 points, user signoffs are one point"

It might be a useful way to find active community members too. If you
could think of a way to creatively implement something like this, I'd
be all for it, assuming other people are.

But yeah, I think you may be right about the "encouraging people not
to sign off at all" part. It would make people lazy.


It's not necessarily a bad thing to have the signoffs come from testers
who are not developers-- it saves time for developers to be developing
if others are testing.


I know I test when I can, but usually get around to it less when I have
a backlog of developing to do already.


So it might be a better division of labor to bring actual testers on
board who like testing and who help us out by doing that. With the web
interface, we should be able to give them just access to signoffs.


Just a thought.

- P

Eduardo Romero 09-02-2008 01:52 PM

Possible signoff addendum
 
On Tue, 2008-09-02 at 08:02 -0400, Paul Mattal wrote:
>
> It's not necessarily a bad thing to have the signoffs come from testers
> who are not developers-- it saves time for developers to be developing
> if others are testing.
>
> I know I test when I can, but usually get around to it less when I have
> a backlog of developing to do already.
>
> So it might be a better division of labor to bring actual testers on
> board who like testing and who help us out by doing that. With the web
> interface, we should be able to give them just access to signoffs.
>
> Just a thought.
>
> - P
+1, Agree.

"Dusty Phillips" 09-04-2008 06:52 PM

Possible signoff addendum
 
2008/9/2 Aaron Griffin <aaronmgriffin@gmail.com>:
> On Sun, Aug 31, 2008 at 11:11 AM, Dusty Phillips <buchuki@gmail.com> wrote:
>> Another alternative:
>>
>> - Create a web based interface to allow community users to signoff on
>> packages. We could have a policy of X number of user signoffs is equal
>> to one developer signoff. Developers can be trusted to test it better
>> (or not? ;-), but arch members can contribute too. In that case we
>> still need some incentive to the end users to bother doing this. There
>> aren't a lot of people like dolby and cactus who seem to actually like
>> working really hard with as little recognition as possible.
>
> I actually thought about this a while back. It was in terms of "each
> package needs 10 points, developer signoffs are 5 points, TU signoffs
> are 3 points, user signoffs are one point"
>
> It might be a useful way to find active community members too. If you
> could think of a way to creatively implement something like this, I'd
> be all for it, assuming other people are.

The obvious implementation would be to give people the option to sign
up for an account and to give them signoff permission (as Paul
suggested). This is easy to implement but it means non-devs have to
have access to dev.archlinux.org or we have to break some of the
caching on the public site (which would make cactus weep). It also
means yet another fricking user account that people have to sign up
for.

A less invasive option would be to add a 'signoff this package' link
to testing packages just like there is currently a 'flag out of date'
link. But it would be hard to police if some asshat wanted to come
through and click the signoff button 50 times even though the package
was thoroughly busted.

There have been some bug reports suggesting linking packages to
flyspray tickets (or vice versa), and I have a personal vendetta
against flyspray so another option would be to create a new
package-based bug tracker and port existing bugs and user accounts
from flyspray. Then anyone with a 'Arch Linux KISS Bugtracker' account
would also have the ability to signoff on packages. So people could
actively log in and either say "yo this package works so good I want
it to go live (ie: signoff)" or "wtf this package is broken, fix it
fix it whine whine whine (ie: attach a bug to the package)".

The question isn't so much which of these to do as how much time do I
have to commit to it (within epsilon of 0), how many other awesome
people want to submit patches (approximately 2), and how soon do we
need it (two months back?)

Dusty

PS: sorry for not replying sooner, that little bit of time I have for
Arch is mostly being eaten up by schwag lately.

Eric Belanger 09-09-2008 04:56 PM

Possible signoff addendum
 
On Thu, 4 Sep 2008, Dusty Phillips wrote:


2008/9/2 Aaron Griffin <aaronmgriffin@gmail.com>:

On Sun, Aug 31, 2008 at 11:11 AM, Dusty Phillips <buchuki@gmail.com> wrote:

Another alternative:

- Create a web based interface to allow community users to signoff on
packages. We could have a policy of X number of user signoffs is equal
to one developer signoff. Developers can be trusted to test it better
(or not? ;-), but arch members can contribute too. In that case we
still need some incentive to the end users to bother doing this. There
aren't a lot of people like dolby and cactus who seem to actually like
working really hard with as little recognition as possible.


I actually thought about this a while back. It was in terms of "each
package needs 10 points, developer signoffs are 5 points, TU signoffs
are 3 points, user signoffs are one point"

It might be a useful way to find active community members too. If you
could think of a way to creatively implement something like this, I'd
be all for it, assuming other people are.


The obvious implementation would be to give people the option to sign
up for an account and to give them signoff permission (as Paul
suggested). This is easy to implement but it means non-devs have to
have access to dev.archlinux.org or we have to break some of the
caching on the public site (which would make cactus weep). It also
means yet another fricking user account that people have to sign up
for.

A less invasive option would be to add a 'signoff this package' link
to testing packages just like there is currently a 'flag out of date'
link. But it would be hard to police if some asshat wanted to come
through and click the signoff button 50 times even though the package
was thoroughly busted.

There have been some bug reports suggesting linking packages to
flyspray tickets (or vice versa), and I have a personal vendetta
against flyspray so another option would be to create a new
package-based bug tracker and port existing bugs and user accounts
from flyspray. Then anyone with a 'Arch Linux KISS Bugtracker' account
would also have the ability to signoff on packages. So people could
actively log in and either say "yo this package works so good I want
it to go live (ie: signoff)" or "wtf this package is broken, fix it
fix it whine whine whine (ie: attach a bug to the package)".

The question isn't so much which of these to do as how much time do I
have to commit to it (within epsilon of 0), how many other awesome
people want to submit patches (approximately 2), and how soon do we
need it (two months back?)

Dusty

PS: sorry for not replying sooner, that little bit of time I have for
Arch is mostly being eaten up by schwag lately.




This sounds complicated.

I think we could improve signoff by being more organized. One of the
current problem is that we have packages that very few devs uses (wireless
drivers, file syatem tools, raid, lvm, etc). As we don't know how many dev
uses these, at every signoff thread we are probably waiting for signoffs
that will never come. And the users probably assume that eventually a dev will
signoff. So Aaron has to come and give his "untested" signoff just to get
things moving.


To improve this, we should make a list of these low-usage packages with
the name of the devs that actually uses them (and for which architecture).
This might even encourage signoff if a dev knows he's the only one who can
signoff a package. If there are packages that not enough dev use to do the
signoff, we could do one of two options:


1) Make an exceptions for these packages. We move them to core once the
devs who use them have signed off. We could also force them to remain in
testing for a minimal time (few days) to be more safe. This is equivalent
to what we do now when Aaron gives his "untested" signoff except we don't
wait for a few weeks for signoffs that will never come.


2) We involve the community at large. In the signoff thread, we
explicitely state that we need community signoffs and for which
architecture. Users signoff by replying directly to the dev who started
the signoff thread or to the arch-general ML. If a few users signoff, then
we move the package to core.


The points system is not pratical. If no dev use a package, then we would
need signoff from 7 TU or 20 users or any combination. This is a lot of
signoff. If very few devs use a package, then it's probably the case for
the community as well so it might take a long time to get all these
signoffs. The signoff threads are on the arch-dev-public ML which is
intented for Arch developpement. I assume that users subscribed to this
list are more technical oriented and have Arch at heart. So if they
signoff a package, then they probably know how to test it, have tested it
and verified that it work as intended. Anyway, we would have their email
address in case there are problems.


Eric

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

Thayer Williams 09-12-2008 06:39 PM

Possible signoff addendum
 
> On Tue, 2008-09-02 at 08:02 -0400, Paul Mattal wrote:
> >
> > It's not necessarily a bad thing to have the signoffs come from testers
> > who are not developers-- it saves time for developers to be developing
> > if others are testing.
> >
> > I know I test when I can, but usually get around to it less when I have
> > a backlog of developing to do already.

I would be happy to test more packages, but I am hesitant to open up the
testing repo on my system after having some bad encounters in the past.

I've also got an arch in virtualbox, but that doesn't really help for
hardware-related stuff like the recent 3945 wifi update.

How do you other devs manage your test packages? Do you use separate test
machines or are ya just livin' on the edge? Any best practises you'd like
to share?


All times are GMT. The time now is 07:05 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.