1 of 400 dpkg databases corrupt?
 

Looking at
<URL: http://qa.debian.org/popcon.php?package=popularity-contest >, I
see that only 99.75% of the machines reporting to popcon.debian.org
have the package popularity-contest installed. As this is impossible,
I suspect this mean that 0.25% (1 of 400) of the machines reporting to
popcon.debian.org got a corrupt/inconsitent dpkg database. Is this an
acceptable ratio? Anything we can do to reduce the number of machines
with corrupt dpkg database?

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

1 of 400 dpkg databases corrupt?
 

Petter Reinholdtsen <pere@hungry.com> writes:

> Looking at
> <URL: http://qa.debian.org/popcon.php?package=popularity-contest >, I
> see that only 99.75% of the machines reporting to popcon.debian.org
> have the package popularity-contest installed. As this is impossible,
> I suspect this mean that 0.25% (1 of 400) of the machines reporting to
> popcon.debian.org got a corrupt/inconsitent dpkg database. Is this an
> acceptable ratio? Anything we can do to reduce the number of machines
> with corrupt dpkg database?

It's not *impossible*... someone could be running the scripts from the
package without having the package installed. I don't know why they'd do
that, though, or whether that's a more plausible explanation than a
corrupt database.

--
Russ Allbery (rra@debian.org) <http://www.eyrie.org/~eagle/>


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

1 of 400 dpkg databases corrupt?
 

[Russ Allbery]
> It's not *impossible*... someone could be running the scripts from the
> package without having the package installed. I don't know why they'd do
> that, though, or whether that's a more plausible explanation than a
> corrupt database.

Sure, it is possible for people to run the script manually without the
package, but I believe it is unlikely. The ratio of hosts submitting
to popcon without having popcon installed has been fairly stable, and
I kind of doubt that such activity is done regularly by several
people.

Happy hacking,
--
Petter Reinholdtsen


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

1 of 400 dpkg databases corrupt?
 

On Sat, Aug 23, 2008 at 09:13:42AM +0200, Petter Reinholdtsen wrote:
> [Russ Allbery]
> > It's not *impossible*... someone could be running the scripts from the
> > package without having the package installed. I don't know why they'd do
> > that, though, or whether that's a more plausible explanation than a
> > corrupt database.
>
> Sure, it is possible for people to run the script manually without the
> package, but I believe it is unlikely.

I believe .25% of systems having a corrupt dpkg database is also rather
unlikely; it would mean we're getting a far lower amount of bugreports
than we deserve.

> The ratio of hosts submitting to popcon without having popcon
> installed has been fairly stable, and I kind of doubt that such
> activity is done regularly by several people.

Indeed. This would also rule out a temporary bug in popcon (in that
case, it would have been a peak which would subside over time). Instead,
my guess is that there are corner-case situations in which popcon tries
to read the dpkg database at a time when it is in a state of flux; and
that because of that, popcon doesn't get all the existing data, only
part of it.

Of the top of my head, I can think of two possible examples where this
might be the case:
- If popcon tries to read the dpkg database while it's being installed
or immediately after (no clue whether it does), that could be the
problem. This would suggest that the ratio of the number of people
newly installing popcon versus the number of people having popcon
installed already is stable over time; this in turn would suggest an
increasing rate of growth.
- Another possibility might be a group of people having popcon and
something like cron-apt installed at the same time; if both cronjobs
trigger at approximately the same time, that would greatly increase
the chance that popcon is indeed trying to read the dpkg database at
the time when cron-apt is rewriting it.

I think the latter of the above two is the more likely. Of course, all
of the above is guesswork...

--
<Lo-lan-do> Home is where you have to wash the dishes.
-- #debian-devel, Freenode, 2004-09-22


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

1 of 400 dpkg databases corrupt?
 

I suspect this mean that 0.25% (1 of 400) of the machines reporting to
popcon.debian.org got a corrupt/inconsitent dpkg database.
Afaict dpkg has no mechanism in place for detecting or recovering from database curruption. The format of the datbase
means that curruption tends to lead to dpkg forgetting a number of packages exist and thinking one package has a very

long description.

Next time the user uses apt it will complain about broken packages and advise the user to use apt-get -f install. Apt
will then reinstall forgotten packages that other packages depend on but if nothing depends on the package it will be
left present on the system but not in the dpkg database.






--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

1 of 400 dpkg databases corrupt?
 

On Sun, 24 Aug 2008 02:07:48 +0200
Wouter Verhelst <wouter@debian.org> wrote:

>
> Indeed. This would also rule out a temporary bug in popcon (in that
> case, it would have been a peak which would subside over time).
> Instead, my guess is that there are corner-case situations in which
> popcon tries to read the dpkg database at a time when it is in a
> state of flux; and that because of that, popcon doesn't get all the
> existing data, only part of it.
>
> Of the top of my head, I can think of two possible examples where this
> might be the case:

[snip]

> - Another possibility might be a group of people having popcon and
> something like cron-apt installed at the same time; if both cronjobs

That would include me (only apticron and popcon) :-)

> trigger at approximately the same time, that would greatly increase
> the chance that popcon is indeed trying to read the dpkg database at
> the time when cron-apt is rewriting it.

I just use the defaults for apticron and popcon. If that is a problem
or popcon frequently happens during system updates, or while checking
for updates (e.g for those who use update-notifier, which doesn't
include me) then this could happen fairly often. Perhaps popcon needs
to check if the database is in use, just like any other apt consumer?
A wishlist or minor bug to this effect, perhaps?

>
> I think the latter of the above two is the more likely. Of course, all
> of the above is guesswork...
>


--
And that's my crabbing done for the day. Got it out of the way early,
now I have the rest of the afternoon to sniff fragrant tea-roses or
strangle cute bunnies or something. -- Michael Devore
GnuPG Key Fingerprint 86 F5 81 A5 D4 2E 1F 1C http://gnupg.org
The C Shore: http://www.wightman.ca/~cshore

1 of 400 dpkg databases corrupt?
 

On Sun, 24 Aug 2008 02:07:48 +0200
Wouter Verhelst <wouter@debian.org> wrote:

> - Another possibility might be a group of people having popcon and
> something like cron-apt installed at the same time; if both cronjobs
> trigger at approximately the same time, that would greatly increase
> the chance that popcon is indeed trying to read the dpkg database at
> the time when cron-apt is rewriting it.

Oh yes, another cron job for dpkg/apt: apt-cacher or apt-cacher-ng
which does a daily download of package lists and checks for expired
packages in the cache.

So I think Wouter is right and there are more packages that do
automatic package list updates than one might think at first.

Regards,

Daniel

--
And that's my crabbing done for the day. Got it out of the way early,
now I have the rest of the afternoon to sniff fragrant tea-roses or
strangle cute bunnies or something. -- Michael Devore
GnuPG Key Fingerprint 86 F5 81 A5 D4 2E 1F 1C http://gnupg.org
The C Shore: http://www.wightman.ca/~cshore

1 of 400 dpkg databases corrupt?
 

On Sat, Aug 23, 2008 at 04:32:46AM +0200, Petter Reinholdtsen wrote:
> Looking at
> <URL: http://qa.debian.org/popcon.php?package=popularity-contest >, I
> see that only 99.75% of the machines reporting to popcon.debian.org
> have the package popularity-contest installed. As this is impossible,
> I suspect this mean that 0.25% (1 of 400) of the machines reporting to
> popcon.debian.org got a corrupt/inconsitent dpkg database. Is this an
> acceptable ratio? Anything we can do to reduce the number of machines
> with corrupt dpkg database?

Broken harddisk - I have ~450 Machines under my umbrella spreaded around
germany - And the most occuring bug is a broken disk.

So when pocon is installed but basically most of the disk accesses fail
you get the above error i'd guess. So yes - the dpkg database might be
broken - but because of hardware failure.

Flo
--
Florian Lohoff flo@rfc822.org +49-171-2280134
Those who would give up a little freedom to get a little
security shall soon have neither - Benjamin Franklin

1 of 400 dpkg databases corrupt?
 

On Sat, Aug 23, 2008 at 04:32:46AM +0200, Petter Reinholdtsen wrote:

> Looking at
> <URL: http://qa.debian.org/popcon.php?package=popularity-contest >, I
> see that only 99.75% of the machines reporting to popcon.debian.org
> have the package popularity-contest installed. As this is impossible,
> I suspect this mean that 0.25% (1 of 400) of the machines reporting to
> popcon.debian.org got a corrupt/inconsitent dpkg database. Is this an
> acceptable ratio? Anything we can do to reduce the number of machines
> with corrupt dpkg database?

Could it be a case of machines that nfs-mount /etc and /usr, but not
/var, or machines updated by rsyncing /etc and /usr?


Ciao,

Enrico

--
GPG key: 1024D/797EBFAB 2000-12-05 Enrico Zini <enrico@debian.org>