I'm calling for participation in a new team working on things around
reports, bug analysis, heartbeat tracking in Gentoo:


Goals
=====
- help track the heartbeat/breath of Gentoo
(bugs fixed per day, bug distribution per herd, ..)

- point to problems we may be overlooking in Gentoo
(letting numbers speak to us and listening to them)

- help to prioritize when distributing available resources
(specific calls for help, people switching teams, ..)


Data we will be working with
============================
- The Bug database (XML from Bugzilla, ..)
- Package tree history (VCS logs, ..)
- User setup details (gentoo-smolt, ..)
- ..


Concrete tasks
==============
- bugday.g.o re-write (work in progress)
- work on association between bugs and packages
(for all bugs, not just bugday ones)
- get real numbers on how much active manpower we have


People
======
These poeple are in de facto:
- deathwing00
- sping
- <you here?>


If this is stuff which
- you have time for
- you are good at

please join us!



Sebastian

On Sat, Apr 3, 2010 at 5:54 PM, Sebastian Pipping <sping@gentoo.org> wrote:
> I'm calling for participation in a new team working on things around
> reports, bug analysis, heartbeat tracking in Gentoo:
>
>
> Goals
> =====
> - help track the heartbeat/breath of Gentoo
> * *(bugs fixed per day, bug distribution per herd, ..)
>
> - point to problems we may be overlooking in Gentoo
> * *(letting numbers speak to us and listening to them)
>
> - help to prioritize when distributing available resources
> * *(specific calls for help, people switching teams, ..)
>
>
> Data we will be working with
> ============================
> - The Bug database *(XML from Bugzilla, ..)
> - Package tree history *(VCS logs, ..)
> - User setup details *(gentoo-smolt, ..)
> - ..
>
>
> Concrete tasks
> ==============
> - bugday.g.o re-write *(work in progress)
> - work on association between bugs and packages
> *(for all bugs, not just bugday ones)
> - get real numbers on how much active manpower we have
>
>
> People
> ======
> These poeple are in de facto:
> - deathwing00
> - sping
> - <you here?>

Sebastian, count me in. I believe this sort of stuff, or small
parts of what you are mentioning, was done in the newsletter. I
believe this is something real nice we should put somewhere public.

>
>
> If this is stuff which
> - you have time for
> - you are good at
>
> please join us!
>
>
>
> Sebastian
>

Best regards,

--
Jesus Rivero (Neurogeek)
Gentoo Python Team

On Sat, Apr 3, 2010 at 3:24 PM, Sebastian Pipping <sping@gentoo.org> wrote:
> I'm calling for participation in a new team working on things around
> reports, bug analysis, heartbeat tracking in Gentoo:

I will help with scripting; but probably not much else.

-A

>
>
> Goals
> =====
> - help track the heartbeat/breath of Gentoo
> * *(bugs fixed per day, bug distribution per herd, ..)
>
> - point to problems we may be overlooking in Gentoo
> * *(letting numbers speak to us and listening to them)
>
> - help to prioritize when distributing available resources
> * *(specific calls for help, people switching teams, ..)
>
>
> Data we will be working with
> ============================
> - The Bug database *(XML from Bugzilla, ..)
> - Package tree history *(VCS logs, ..)
> - User setup details *(gentoo-smolt, ..)
> - ..
>
>
> Concrete tasks
> ==============
> - bugday.g.o re-write *(work in progress)
> - work on association between bugs and packages
> *(for all bugs, not just bugday ones)
> - get real numbers on how much active manpower we have
>
>
> People
> ======
> These poeple are in de facto:
> - deathwing00
> - sping
> - <you here?>
>
>
> If this is stuff which
> - you have time for
> - you are good at
>
> please join us!
>
>
>
> Sebastian
>
>
>

On 04/04/10 00:30, Jesus Rivero (Neurogeek) wrote:
> Sebastian, count me in.

Cool!


> I believe this sort of stuff, or small
> parts of what you are mentioning, was done in the newsletter. I
> believe this is something real nice we should put somewhere public.

Can you elaborate on that? I'm not sure what what you refer to.



Sebastian

On 04/04/10 00:42, Alec Warner wrote:
> I will help with scripting; but probably not much else.

Just right, there will be lots of scripting :-)

The current script dumping groud is hosted here:
http://git.goodpoint.de/?p=gentoo-bug-heartbeat.git;a=summary



Sebastian

On Sat, Apr 3, 2010 at 6:27 PM, Sebastian Pipping <sping@gentoo.org> wrote:
> On 04/04/10 00:30, Jesus Rivero (Neurogeek) wrote:
>> * *Sebastian, count me in.
>
> Cool!
>
>
>> I believe this sort of stuff, or small
>> parts of what you are mentioning, was done in the newsletter. I
>> believe this is something real nice we should put somewhere public.
>
> Can you elaborate on that? *I'm not sure what what you refer to.
>

Yep. Take for instance this issue of GMN [1], here you can find
some bugzie statistics like bugs assigned to teams/devs, bugs
resolutions and other cool things.

>
>
> Sebastian
>
>
[1] http://www.gentoo.org/news/en/gmn/20080121-newsletter.xml#bugs-stats

--
Jesus Rivero (Neurogeek)
Gentoo Python Team

* Sebastian Pipping <sping@gentoo.org>:
> - Package tree history (VCS logs, ..)

> - get real numbers on how much active manpower we have

I am generating monthly stats for gentoo-x86 for a year or so:
http://dev.gentoo.org/~tove/stats/gentoo-x86/

It lists the number of commits per month (cvs-log-20...) for all
"active" devs. Monthly commits per dev over time (dev/commits-$dev...).

cvs-log-sum.txt total number of commits per dev.

slacker.txt combines the last two month and adds further information for
devs without commits (date of last commit, date of last bugzilla
interaction, dev bug information, away status).

Since 2010 I also add cvs-log-all-.. which adds all not-retired devs
with gentoo-x86 access.

grep for Sum, Mean,...:
http://dev.gentoo.org/~tove/files/activedevs.txt
http://dev.gentoo.org/~tove/files/sum.txt
http://dev.gentoo.org/~tove/files/mean.txt
http://dev.gentoo.org/~tove/files/median.txt

On 04/04/10 00:24, Sebastian Pipping wrote:
> Concrete tasks
> ==============
> - [..]
> - work on association between bugs and packages
> (for all bugs, not just bugday ones)

I've been playing with code answering the questions:
- How do bugs and packages relate?
- How is bugload distributed across developers?

I came to a point where
- I have something that starts to be useful
- I am facing troubles that need further
consideration first, see below.


Current results
===============
Bug load per developer
----------------------
http://dev.gentoo.org/~sping/bug-heartbeat/report--bug-count-by-person.html

Bug load per herd
-----------------
http://dev.gentoo.org/~sping/bug-heartbeat/report--bug-count-by-herd.html

Bugs in a tree of category and package
--------------------------------------
http://dev.gentoo.org/~sping/bug-heartbeat/report--bugs-by-package.html

Bugs that may lack mention of package name
------------------------------------------
http://dev.gentoo.org/~sping/bug-heartbeat/report--bugs-without-package.html


Question to be answered
=======================
- herds.xml does not hold membership lists for
all projects, not even herds. The load
evaluator needs access to complete mappings
to generate output close to reality.
How can such a mapping be made without
duplicating data?

- Pulling XML out of Bugzilla does no longer
feel right with this amount of data:
it's 15,000 open bugs that need to be
refreshed periodically in chunks of 100 bugs
(Bugzilla's limit) each. That's 150 request
for a full sync. How can that be improved?


Generating reports yourself
===========================
In case you want to play with the code yourself:
http://git.goodpoint.de/?p=gentoo-bug-heartbeat.git;a=summary

Run the report generator as following:

# bzip2 -d xmlbugs.xml.bz2
# python modules/heartbeat/reporter.py

The .bz2 file needed is up at
<http://dev.gentoo.org/~sping/bug-heartbeat/>, too.



Sebastian

On 7 April 2010 13:20, Sebastian Pipping <sping@gentoo.org> wrote:
> *- herds.xml does not hold membership lists for
> * all projects, not even herds. *The load
> * evaluator needs access to complete mappings
> * to generate output close to reality.
> * How can such a mapping be made without
> * duplicating data?

herds.xml should be the definitive resource for herds, so any herds
that are not on there should be added, and outdated information should
be brought up to date.

Other projects should have membership listed on their project pages.
Again, where these are outdated, they should be brought up to date.

Cheers,
--
Ben de Groot
Gentoo Qt project lead developer
Gentoo Wiki project lead

On Tuesday 06 April 2010 10:03:26 Torsten Veller wrote:
> * Sebastian Pipping <sping@gentoo.org>:
> > - Package tree history (VCS logs, ..)
> >
> > - get real numbers on how much active manpower we have
>
> I am generating monthly stats for gentoo-x86 for a year or so:
> http://dev.gentoo.org/~tove/stats/gentoo-x86/
>
> It lists the number of commits per month (cvs-log-20...) for all
> "active" devs. Monthly commits per dev over time (dev/commits-$dev...).
>
> cvs-log-sum.txt total number of commits per dev.
>
> slacker.txt combines the last two month and adds further information for
> devs without commits (date of last commit, date of last bugzilla
> interaction, dev bug information, away status).
>
> Since 2010 I also add cvs-log-all-.. which adds all not-retired devs
> with gentoo-x86 access.
>
> grep for Sum, Mean,...:
> http://dev.gentoo.org/~tove/files/activedevs.txt
> http://dev.gentoo.org/~tove/files/sum.txt
> http://dev.gentoo.org/~tove/files/mean.txt
> http://dev.gentoo.org/~tove/files/median.txt
Very useful. Thank you very much
--
Markos Chandras (hwoarang)
Gentoo Linux Developer
Web: http://hwoarang.silverarrow.org