Hello All,

Last few days my net connection drops randomly or slows down to a crawl. As a
result, there are several timeouts during package downloads. When a download
timeout occurs, pacman aborts with "invalid or corrupted package".

Is it possible to detect which package is corrupt and invalid, so that it can
be downloaded?

When it happened with one package, I deleted it manually and next time it
worked but when several packages get timeout, its hard to detect the failing
one.

Does pacman has built-in retry functionality?

Here is the complete log.

-------------

:: Starting full system upgrade...
resolving dependencies...
looking for inter-conflicts...

Targets (33): ati-dri-8.0.4-2 boost-libs-1.50.0-2 clang-3.1-3 cups-1.5.3-6
cups-filters-1.0.20-1 dbus-1.6.4-1 dbus-core-1.6.4-1 dovecot-2.1.8-2
gcc-4.7.1-5 gcc-libs-4.7.1-5
git-1.7.11.3-1 gnutls-3.0.21-2 icu-49.1.2-2
imagemagick-6.7.8.6-1 intel-dri-8.0.4-2 khrplatform-devel-8.0.4-2
libcups-1.5.3-6 libegl-8.0.4-2 libgl-8.0.4-2
libglapi-8.0.4-2 libgles-8.0.4-2 libmp4v2-2.0.0-2 llvm-3.1-3
mesa-8.0.4-2 nginx-1.2.2-2 python2-distribute-0.6.28-1 qt-4.8.2-3 qt-
gstreamer-0.10.2-2 svga-dri-8.0.4-2
taglib-1.7.2-2 ttf-liberation-2.00.0-1 tzdata-2012d-1 xf86-
video-intel-2.20.1-1

Total Installed Size: 515.40 MiB
Net Upgrade Size: 1.88 MiB

Proceed with installation? [Y/n] y
(33/33) checking package integrity
[################################################## ###################] 100%
error: failed to commit transaction (invalid or corrupted package)
Errors occurred, no packages were upgraded.

-------------

With pacman -sy --debug I get following error snippet

-------------
debug: found cached pkg: /var/cache/pacman/pkg/ati-dri-8.0.4-2-
x86_64.pkg.tar.xz
debug: sig data:
iQEcBAABAgAGBQJQC9QHAAoJEJRlerIPKgkrd/EH/jhoIY6szxkh3AFGHogBLGe33tIcGhMchGI9jtxEht/Df+IB9X9tYC5nG62pOL9+v0Sl4yVLBW4Ni0mYdwQ0UfJr3Mt1s 7C3LbTI1ucU8WZMZicXc8tDKurRnPBDkAqnhK4+H7
4qa6ALNfIDQK7PVQTIedAQDgO9WsPd3DsKcOfNbEXZrDZSBU3a woaQuezKaoif/JnSt7iRXn3hd8piuUgQuLQuksAe+Oa6MtEgXoCHQ22uzXa6nEV 1EkO0q1sf8+0w/9+4mD3ejUutfLAQbxqmpfmw3kEuQ1eaICRLRgRu0ugKuwqi9cv dDBiw/27Z1QTkY
aUNtvdT66Wb5OuAJGI=
debug: checking signature for /var/cache/pacman/pkg/ati-dri-8.0.4-2-
x86_64.pkg.tar.xz
debug: 1 signatures returned
debug: fingerprint: ADC8A1FCC15E01D45310419E94657AB20F2A092B
debug: summary: valid
debug: summary: green
debug: status: Success
debug: timestamp: 1342952455
debug: exp_timestamp: 0
debug: validity: full; reason: Success
debug: key: ADC8A1FCC15E01D45310419E94657AB20F2A092B, Andreas Radke
<andyrtr@archlinux.org>, owner_trust unknown, disabled 0
debug: signature is valid
debug: signature is fully trusted
debug: returning error 6 from _alpm_pkg_validate_internal : wrong or NULL
argument passed
-------------

--
Regards
Shridhar

Followup,

On Wednesday 25 Jul 2012 9:03:42 AM Shridhar Daithankar wrote:

> With pacman -sy --debug I get following error snippet
>
> -------------
> debug: found cached pkg: /var/cache/pacman/pkg/ati-dri-8.0.4-2-
> x86_64.pkg.tar.xz
> debug: sig data:
> iQEcBAABAgAGBQJQC9QHAAoJEJRlerIPKgkrd/EH/jhoIY6szxkh3AFGHogBLGe33tIcGhMchGI9
> jtxEht/Df+IB9X9tYC5nG62pOL9+v0Sl4yVLBW4Ni0mYdwQ0UfJr3Mt1s 7C3LbTI1ucU8WZMZicX
> c8tDKurRnPBDkAqnhK4+H7
> 4qa6ALNfIDQK7PVQTIedAQDgO9WsPd3DsKcOfNbEXZrDZSBU3a woaQuezKaoif/JnSt7iRXn3hd
> 8piuUgQuLQuksAe+Oa6MtEgXoCHQ22uzXa6nEV1EkO0q1sf8+0 w/9+4mD3ejUutfLAQbxqmpfmw3
> kEuQ1eaICRLRgRu0ugKuwqi9cvdDBiw/27Z1QTkY aUNtvdT66Wb5OuAJGI=
> debug: checking signature for /var/cache/pacman/pkg/ati-dri-8.0.4-2-
> x86_64.pkg.tar.xz
> debug: 1 signatures returned
> debug: fingerprint: ADC8A1FCC15E01D45310419E94657AB20F2A092B
> debug: summary: valid
> debug: summary: green
> debug: status: Success
> debug: timestamp: 1342952455
> debug: exp_timestamp: 0
> debug: validity: full; reason: Success
> debug: key: ADC8A1FCC15E01D45310419E94657AB20F2A092B, Andreas Radke
> <andyrtr@archlinux.org>, owner_trust unknown, disabled 0
> debug: signature is valid
> debug: signature is fully trusted
> debug: returning error 6 from _alpm_pkg_validate_internal : wrong or NULL
> argument passed
> -------------

I deleted ati-dri package and redownloaded it. This time, without any timeouts
and it still failed the upgrade with same error for the same package.

So I uninstalled ati-dri and xf86-video-ati. I have intel card and got all
those extra drivers because I installed the whole group for simplicity.

Try again and this time it failed for gcc-libs. Delete, redownload and the
error still persists.

and I don't think uninstalling gcc-libs is an option.

I also found https://bugs.archlinux.org/task/28014 but I don't think that can
solely fix the issue. --debug is printing enough information or so I would
like to think.

Pl. Help.
--
Regards
Shridhar

Shridhar Daithankar <ghodechhap@ghodechhap.net> writes:

> I deleted ati-dri package and redownloaded it. This time, without any timeouts
> and it still failed the upgrade with same error for the same package.
>
> So I uninstalled ati-dri and xf86-video-ati. I have intel card and got all
> those extra drivers because I installed the whole group for simplicity.
>
> Try again and this time it failed for gcc-libs. Delete, redownload and the
> error still persists.
>
> and I don't think uninstalling gcc-libs is an option.
>
> I also found https://bugs.archlinux.org/task/28014 but I don't think that can
> solely fix the issue. --debug is printing enough information or so I would
> like to think.
>
> Pl. Help.

Does it continue after a "pacman -Scc"?

---------------------------------
Jeremiah Dodds

github: https://github.com/jdodds
irc : exhortatory
---------------------------------

On Wed, Jul 25, 2012 at 11:51 AM, Jeremiah Dodds
<jeremiah.dodds@gmail.com> wrote:
> Shridhar Daithankar <ghodechhap@ghodechhap.net> writes:
>
>> I deleted ati-dri package and redownloaded it. This time, without any timeouts
>> and it still failed the upgrade with same error for the same package.
>>
>> So I uninstalled ati-dri and xf86-video-ati. I have intel card and got all
>> those extra drivers because I installed the whole group for simplicity.
>>
>> Try again and this time it failed for gcc-libs. Delete, redownload and the
>> error still persists.
>>
>> and I don't think uninstalling gcc-libs is an option.
>>
>> I also found https://bugs.archlinux.org/task/28014 but I don't think that can
>> solely fix the issue. --debug is printing enough information or so I would
>> like to think.
>>
>> Pl. Help.
>
> Does it continue after a "pacman -Scc"?
>
Why are you recommending clearing his entire cache (which can be quite
useful) when he's already specifically cleared the offending package
and still has problems?

Oon-Ee Ng <ngoonee.talk@gmail.com> writes:

> On Wed, Jul 25, 2012 at 11:51 AM, Jeremiah Dodds
> <jeremiah.dodds@gmail.com> wrote:
>> Shridhar Daithankar <ghodechhap@ghodechhap.net> writes:
>>
>>> I deleted ati-dri package and redownloaded it. This time, without any
>>> timeouts
>>> and it still failed the upgrade with same error for the same package.
>>>
>>> So I uninstalled ati-dri and xf86-video-ati. I have intel card and got all
>>> those extra drivers because I installed the whole group for simplicity.
>>>
>>> Try again and this time it failed for gcc-libs. Delete, redownload and the
>>> error still persists.
>>>
>>> and I don't think uninstalling gcc-libs is an option.
>>>
>>> I also found https://bugs.archlinux.org/task/28014 but I don't think that can
>>> solely fix the issue. --debug is printing enough information or so I would
>>> like to think.
>>>
>>> Pl. Help.
>>
>> Does it continue after a "pacman -Scc"?
>>
> Why are you recommending clearing his entire cache (which can be quite
> useful) when he's already specifically cleared the offending package
> and still has problems?

I've run into situations where I thought I had cleared a corrupted
package but didn't actually do so, or where I missed one. Considering
he's tried multiple times, it might be worth clearing the entire thing
to make sure he's not missing something.


---------------------------------
Jeremiah Dodds

github: https://github.com/jdodds
irc : exhortatory
---------------------------------

On Tuesday 24 Jul 2012 11:58:14 PM Jeremiah Dodds wrote:
> Oon-Ee Ng <ngoonee.talk@gmail.com> writes:
> > On Wed, Jul 25, 2012 at 11:51 AM, Jeremiah Dodds
> >> Does it continue after a "pacman -Scc"?
> >
> > Why are you recommending clearing his entire cache (which can be quite
> > useful) when he's already specifically cleared the offending package
> > and still has problems?
>
> I've run into situations where I thought I had cleared a corrupted
> package but didn't actually do so, or where I missed one. Considering
> he's tried multiple times, it might be worth clearing the entire thing
> to make sure he's not missing something.

That is an option I have not yet tried but I just want to preserve the
reproduction and debug the problem if there is any.


--
Regards
Shridhar

Op woensdag 25 juli 2012 10:30:21 schreef Shridhar Daithankar:
> On Tuesday 24 Jul 2012 11:58:14 PM Jeremiah Dodds wrote:
> > Oon-Ee Ng <ngoonee.talk@gmail.com> writes:
> > > On Wed, Jul 25, 2012 at 11:51 AM, Jeremiah Dodds
> > >
> > >> Does it continue after a "pacman -Scc"?
> > >
> > > Why are you recommending clearing his entire cache (which can be quite
> > > useful) when he's already specifically cleared the offending package
> > > and still has problems?
> >
> > I've run into situations where I thought I had cleared a corrupted
> > package but didn't actually do so, or where I missed one. Considering
> > he's tried multiple times, it might be worth clearing the entire thing
> > to make sure he's not missing something.
>
> That is an option I have not yet tried but I just want to preserve the
> reproduction and debug the problem if there is any.
>
>
> --
> Regards
> Shridhar

You could try to switch mirrors, use a mirror that is not dropping all the
time, or is this problem related to your own internet ?

My guess is we will see more of this in the future since the package checking
is much more tight now than in the past.

--Ike

2012/7/25 Ike Devolder <ike.devolder@gmail.com>:
>> That is an option I have not yet tried but I just want to preserve the
>> reproduction and debug the problem if there is any.

Maybe this will help:

cd /var/lib/pacman/pkg
for pkg in *; do bsdtar -tf $pkg > /dev/null || echo "$pkg is broken"; done

This is strange, for me, pacman always showed which package is broken
(and asked to delete it). Can you disable any ftp mirrors from your
mirrorlist ([1])? Could you post your pacman.conf?

[1] https://bbs.archlinux.org/viewtopic.php?pid=1050214#p1050214

--
Krzysztof Warzecha

On Wednesday 25 Jul 2012 11:15:48 AM Krzysztof Warzecha wrote:
> 2012/7/25 Ike Devolder <ike.devolder@gmail.com>:
> >> That is an option I have not yet tried but I just want to preserve the
> >> reproduction and debug the problem if there is any.
>
> Maybe this will help:
>
> cd /var/lib/pacman/pkg
> for pkg in *; do bsdtar -tf $pkg > /dev/null || echo "$pkg is broken"; done
>
> This is strange, for me, pacman always showed which package is broken
> (and asked to delete it). Can you disable any ftp mirrors from your
> mirrorlist ([1])? Could you post your pacman.conf?
>
> [1] https://bbs.archlinux.org/viewtopic.php?pid=1050214#p1050214

Ok, that did the trick.

Last package I was getting error for was gcc-libs. So I removed it from cache.

Then I searched the cache for broken packages, as suggested above and found
icu package which wasn't completely download i.e. only a .xz.part file, not a
.xz file.

Removed that and pacman -Syu. It worked.

I also have another i686 VM for $DAYJOB(I am not letting some closed source
vpn solution take over my desktop network :P ) and it had the same problem.

So I checked up the part files there and found qt-4.8.x...part. Removed it and
it worked there as well.

I am going to reproduce this problem next time by forcefully interrupting a
download(if my ISP does not beat me to it already) and file a bug.

Thanks for all the help

--
Regards
Shridhar