FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > ArchLinux > ArchLinux Pacman Development

 
 
LinkBack Thread Tools
 
Old 05-12-2008, 11:54 AM
"Dan McGee"
 
Default Weird bug in sync/upgrade behavior

dmcgee@dublin ~
$ pacSyu
:: Synchronizing package databases...
pacman-git 0.5K 3.1M/s 00:00:00 [---------------------] 100%
testing 15.1K 130.1K/s 00:00:00 [---------------------] 100%
core is up to date
extra 312.3K 214.5K/s 00:00:01 [---------------------] 100%
community 335.9K 148.4K/s 00:00:02 [---------------------] 100%
unstable 4.7K 92.6K/s 00:00:00 [---------------------] 100%
:: Starting full system upgrade...
warning: bzip2: local (1.0.5-2) is newer than core (1.0.4-3)
warning: kernel26: local (2.6.25-1) is newer than core (2.6.24.4-1)
warning: libldap: local (2.3.40-1) is newer than core (2.3.39-2)
warning: libtool: local (2.2.4-1) is newer than core (2.2-2)
warning: licenses: local (2.4-1) is newer than core (2.3-1)
warning: links: local (2.1pre35-1) is newer than core (2.1pre33-1)
warning: ntfs-3g: local (1.2412-1) is newer than core (1.2310-1)
warning: openssh: local (5.0p1-1) is newer than core (4.7p1-6)
warning: pcre: local (7.7-1) is newer than core (7.6-3)
warning: sudo: local (1.6.9p15-1) is newer than core (1.6.9p12-1)
local database is up to date

dmcgee@dublin ~
$ pacSyu
:: Synchronizing package databases...
pacman-git is up to date
testing is up to date
core is up to date
extra is up to date
community is up to date
unstable is up to date
:: Starting full system upgrade...
warning: pacman-git: local (20080511-1) is newer than pacman-git (20080427-1)
local database is up to date

Not sure when this got fuggered up (although It was probably me), but
as you can see, we have a problem above. For some reason, when all
databases have been updated except for core, it prefers packages in
core over those in testing? This could be a super-old bug, thinking
about it- We have an alpm_list_t of databases that are stored in conf
file order, and since all the other ones got updated (which means
removed from the list and readded?), core ends up getting bumped to
the top and testing ends up below it, meaning the core packages are
preferred. Note that when I first ran this, a libtool upgrade was
available, and it did not prompt me for that. However, the second run
did prompt me.

Can anyone else try to reproduce this? Try deleting all .lastupdate
files except the one for core, if you have testing enabled, and seeing
what happens on the first and second runs of -Syu.

This could be a prime case for git-bisect if we need to track this
down. I'm currently running a pacman-git I built yesterday (I think).

-Dan

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 05-12-2008, 12:19 PM
Allan McRae
 
Default Weird bug in sync/upgrade behavior

Dan McGee wrote:
> dmcgee@dublin ~
> $ pacSyu
> :: Synchronizing package databases...
> pacman-git 0.5K 3.1M/s 00:00:00 [---------------------] 100%
> testing 15.1K 130.1K/s 00:00:00 [---------------------] 100%
> core is up to date
> extra 312.3K 214.5K/s 00:00:01 [---------------------] 100%
> community 335.9K 148.4K/s 00:00:02 [---------------------] 100%
> unstable 4.7K 92.6K/s 00:00:00 [---------------------] 100%
> :: Starting full system upgrade...
> warning: bzip2: local (1.0.5-2) is newer than core (1.0.4-3)
> warning: kernel26: local (2.6.25-1) is newer than core (2.6.24.4-1)
> warning: libldap: local (2.3.40-1) is newer than core (2.3.39-2)
> warning: libtool: local (2.2.4-1) is newer than core (2.2-2)
> warning: licenses: local (2.4-1) is newer than core (2.3-1)
> warning: links: local (2.1pre35-1) is newer than core (2.1pre33-1)
> warning: ntfs-3g: local (1.2412-1) is newer than core (1.2310-1)
> warning: openssh: local (5.0p1-1) is newer than core (4.7p1-6)
> warning: pcre: local (7.7-1) is newer than core (7.6-3)
> warning: sudo: local (1.6.9p15-1) is newer than core (1.6.9p12-1)
> local database is up to date
>
> dmcgee@dublin ~
> $ pacSyu
> :: Synchronizing package databases...
> pacman-git is up to date
> testing is up to date
> core is up to date
> extra is up to date
> community is up to date
> unstable is up to date
> :: Starting full system upgrade...
> warning: pacman-git: local (20080511-1) is newer than pacman-git (20080427-1)
> local database is up to date
>
> Not sure when this got fuggered up (although It was probably me), but
> as you can see, we have a problem above. For some reason, when all
> databases have been updated except for core, it prefers packages in
> core over those in testing? This could be a super-old bug, thinking
> about it- We have an alpm_list_t of databases that are stored in conf
> file order, and since all the other ones got updated (which means
> removed from the list and readded?), core ends up getting bumped to
> the top and testing ends up below it, meaning the core packages are
> preferred. Note that when I first ran this, a libtool upgrade was
> available, and it did not prompt me for that. However, the second run
> did prompt me.
>
> Can anyone else try to reproduce this? Try deleting all .lastupdate
> files except the one for core, if you have testing enabled, and seeing
> what happens on the first and second runs of -Syu.
>
> This could be a prime case for git-bisect if we need to track this
> down. I'm currently running a pacman-git I built yesterday (I think).
>
> -Dan
>
>

I can not replicate this at all...

pacman-3.1.4
:: Synchronizing package databases...
testing 15.1K 58.9K/s 00:00:00
[#####################] 100%
core is up to date
extra 312.3K 58.7K/s 00:00:05
[#####################] 100%
community 335.9K 58.7K/s 00:00:06
[#####################] 100%
:: Starting full system upgrade...
local database is up to date

pacman built from git (in last 10 minutes)
:: Synchronizing package databases...
testing 15.1K 58.0K/s 00:00:00
[#####################] 100%
core is up to date
extra 312.3K 58.7K/s 00:00:05
[#####################] 100%
community 335.9K 58.8K/s 00:00:06
[#####################] 100%
:: Starting full system upgrade...
local database is up to date

Allan




_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 05-12-2008, 12:38 PM
Allan McRae
 
Default Weird bug in sync/upgrade behavior

Allan McRae wrote:
> Dan McGee wrote:
>
>> Can anyone else try to reproduce this? Try deleting all .lastupdate
>> files except the one for core, if you have testing enabled, and seeing
>> what happens on the first and second runs of -Syu.
>>
>>
>
> I can not replicate this at all...
>
>

I have even tried adding the extra repos you have enabled with no
success in replicating.

Allan



_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 05-12-2008, 12:54 PM
"Dan McGee"
 
Default Weird bug in sync/upgrade behavior

On Mon, May 12, 2008 at 7:38 AM, Allan McRae <mcrae_allan@hotmail.com> wrote:
> Allan McRae wrote:
>
> > Dan McGee wrote:
> >
> >> Can anyone else try to reproduce this? Try deleting all .lastupdate
> >> files except the one for core, if you have testing enabled, and seeing
> >> what happens on the first and second runs of -Syu.
> >>
> >>
> >
>
> > I can not replicate this at all...
> >
> >
>
> I have even tried adding the extra repos you have enabled with no
> success in replicating.

I was able to bisect it at least- and it was me. I haven't looked into
why this broke it yet, but if anyone else wants to poke at it, feel
free.

046003844739416ff6d168dd2dec76490adb0727 is first bad commit
commit 046003844739416ff6d168dd2dec76490adb0727
Author: Dan McGee <dan@archlinux.org>
Date: Sun May 11 19:29:23 2008 -0500

Remove some useless abstraction and start db cleanup

We have some useless abstractions like an alpm_db_rewind function. I've read
somewhere that readdir() was the worst filesystem function call invented,
and what do we do? Add a wrapper around it. Kill this abstraction and move
some other things into be_files that should be there anyway because they
are so tied to how a files backend works.

Signed-off-by: Dan McGee <dan@archlinux.org>

:040000 040000 534812c4e1341825ff0ef0446bba8a7e865f6d43
24fd6a8b87d064809de09a295e7b8fd4ede198b9 M lib
bisect run success


$ git bisect log
# bad: [663408532ae852e7123da6b9658df1cacc0c642d] Merge branch 'maint'
# good: [e3d35b3274f13cc9ed8d79fa873d6348e424259e] Add detailed
description to alpm_pkg_load
git-bisect start 'master' 'maint'
# good: [35135c0a0cbac592e72296c0ca64e9def0308942] Add -Rss option
git-bisect good 35135c0a0cbac592e72296c0ca64e9def0308942
# good: [db4258c1fdf27708968baf9c4e730a014af40fd8] Some comments for
_alpm_unpack.
git-bisect good db4258c1fdf27708968baf9c4e730a014af40fd8
# good: [4e6361642e632b8631e3d12ee33b1cf5293edb83] Rework
extract_single_file() temp file creation
git-bisect good 4e6361642e632b8631e3d12ee33b1cf5293edb83
# good: [502645c0e304a8ee062d8da7f162ff4c195e6be8] makepkg: Unify
start and end messages
git-bisect good 502645c0e304a8ee062d8da7f162ff4c195e6be8
# good: [0bfc8adf377e7c0d4870fd79999b359a00bc96e2] contrib/paclist:
list packages installed from given repo.
git-bisect good 0bfc8adf377e7c0d4870fd79999b359a00bc96e2
# good: [3c3cb001a441656c2afba62f0361b83d4987339c] Make all error
messages use pm_fprintf
git-bisect good 3c3cb001a441656c2afba62f0361b83d4987339c
# bad: [13f24a5bdabf9c2c7bfa07aff7176495bb775a0d] Refactor
pkg_load/parse_descfile into a new backend file
git-bisect bad 13f24a5bdabf9c2c7bfa07aff7176495bb775a0d
# bad: [046003844739416ff6d168dd2dec76490adb0727] Remove some useless
abstraction and start db cleanup
git-bisect bad 046003844739416ff6d168dd2dec76490adb0727

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 05-12-2008, 12:58 PM
"Dan McGee"
 
Default Weird bug in sync/upgrade behavior

On Mon, May 12, 2008 at 7:54 AM, Dan McGee <dpmcgee@gmail.com> wrote:
> On Mon, May 12, 2008 at 7:38 AM, Allan McRae <mcrae_allan@hotmail.com> wrote:
> > Allan McRae wrote:
> >
> > > Dan McGee wrote:
> > >
> > >> Can anyone else try to reproduce this? Try deleting all .lastupdate
> > >> files except the one for core, if you have testing enabled, and seeing
> > >> what happens on the first and second runs of -Syu.
> > >>
> > >>
> > >
> >
> > > I can not replicate this at all...
> > >
> > >
> >
> > I have even tried adding the extra repos you have enabled with no
> > success in replicating.
>
> I was able to bisect it at least- and it was me. I haven't looked into
> why this broke it yet, but if anyone else wants to poke at it, feel
> free.
>
> 046003844739416ff6d168dd2dec76490adb0727 is first bad commit
> commit 046003844739416ff6d168dd2dec76490adb0727
> Author: Dan McGee <dan@archlinux.org>
> Date: Sun May 11 19:29:23 2008 -0500
>
> Remove some useless abstraction and start db cleanup
>
> We have some useless abstractions like an alpm_db_rewind function. I've read
> somewhere that readdir() was the worst filesystem function call invented,
> and what do we do? Add a wrapper around it. Kill this abstraction and move
> some other things into be_files that should be there anyway because they
> are so tied to how a files backend works.
>
> Signed-off-by: Dan McGee <dan@archlinux.org>
>
> :040000 040000 534812c4e1341825ff0ef0446bba8a7e865f6d43
> 24fd6a8b87d064809de09a295e7b8fd4ede198b9 M lib
> bisect run success
>
>
> $ git bisect log
> # bad: [663408532ae852e7123da6b9658df1cacc0c642d] Merge branch 'maint'
> # good: [e3d35b3274f13cc9ed8d79fa873d6348e424259e] Add detailed
> description to alpm_pkg_load
> git-bisect start 'master' 'maint'
> # good: [35135c0a0cbac592e72296c0ca64e9def0308942] Add -Rss option
> git-bisect good 35135c0a0cbac592e72296c0ca64e9def0308942
> # good: [db4258c1fdf27708968baf9c4e730a014af40fd8] Some comments for
> _alpm_unpack.
> git-bisect good db4258c1fdf27708968baf9c4e730a014af40fd8
> # good: [4e6361642e632b8631e3d12ee33b1cf5293edb83] Rework
> extract_single_file() temp file creation
> git-bisect good 4e6361642e632b8631e3d12ee33b1cf5293edb83
> # good: [502645c0e304a8ee062d8da7f162ff4c195e6be8] makepkg: Unify
> start and end messages
> git-bisect good 502645c0e304a8ee062d8da7f162ff4c195e6be8
> # good: [0bfc8adf377e7c0d4870fd79999b359a00bc96e2] contrib/paclist:
> list packages installed from given repo.
> git-bisect good 0bfc8adf377e7c0d4870fd79999b359a00bc96e2
> # good: [3c3cb001a441656c2afba62f0361b83d4987339c] Make all error
> messages use pm_fprintf
> git-bisect good 3c3cb001a441656c2afba62f0361b83d4987339c
> # bad: [13f24a5bdabf9c2c7bfa07aff7176495bb775a0d] Refactor
> pkg_load/parse_descfile into a new backend file
> git-bisect bad 13f24a5bdabf9c2c7bfa07aff7176495bb775a0d
> # bad: [046003844739416ff6d168dd2dec76490adb0727] Remove some useless
> abstraction and start db cleanup
> git-bisect bad 046003844739416ff6d168dd2dec76490adb0727
>

diff --git a/lib/libalpm/cache.c b/lib/libalpm/cache.c
index fcd555e..bfbf995 100644
--- a/lib/libalpm/cache.c
+++ b/lib/libalpm/cache.c
@@ -24,6 +24,7 @@
#include <stdlib.h>
#include <errno.h>
#include <string.h>
+#include <dirent.h>

/* libalpm */
#include "cache.h"
@@ -54,6 +55,7 @@ int _alpm_db_load_pkgcache(pmdb_t *db)
_alpm_log(PM_LOG_DEBUG, "loading package cache for repository '%s'
",
db->treename);

+ rewinddir(db->handle);
while((info = _alpm_db_scan(db, NULL)) != NULL) {
_alpm_log(PM_LOG_FUNCTION, "adding '%s' to package
cache for db '%s'
",

alpm_pkg_get_name(info), db->treename);

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 05-12-2008, 01:04 PM
Xavier
 
Default Weird bug in sync/upgrade behavior

On Mon, May 12, 2008 at 2:58 PM, Dan McGee <dpmcgee@gmail.com> wrote:
> diff --git a/lib/libalpm/cache.c b/lib/libalpm/cache.c
> index fcd555e..bfbf995 100644
> --- a/lib/libalpm/cache.c
> +++ b/lib/libalpm/cache.c
> @@ -24,6 +24,7 @@
> #include <stdlib.h>
> #include <errno.h>
> #include <string.h>
> +#include <dirent.h>
>
> /* libalpm */
> #include "cache.h"
> @@ -54,6 +55,7 @@ int _alpm_db_load_pkgcache(pmdb_t *db)
> _alpm_log(PM_LOG_DEBUG, "loading package cache for repository '%s'
",
> db->treename);
>
> + rewinddir(db->handle);
> while((info = _alpm_db_scan(db, NULL)) != NULL) {
> _alpm_log(PM_LOG_FUNCTION, "adding '%s' to package
> cache for db '%s'
",
>
> alpm_pkg_get_name(info), db->treename);
>
>

Oh no shit, I noticed this and wanted to ask you about it, why you
removed that rewinddir call...
But then I looked at db_scan code, and it seems like it did a
rewinddir itself, so I thought it was fine ...
And when you showed your weird bug, I thought about that rewinddir
stuff again.. But I didn't say anything.. I suck.
I was still building your last git branch to try reproducing it first.

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 05-12-2008, 02:00 PM
"Dan McGee"
 
Default Weird bug in sync/upgrade behavior

On Mon, May 12, 2008 at 8:04 AM, Xavier <shiningxc@gmail.com> wrote:
> On Mon, May 12, 2008 at 2:58 PM, Dan McGee <dpmcgee@gmail.com> wrote:
> > diff --git a/lib/libalpm/cache.c b/lib/libalpm/cache.c
> > index fcd555e..bfbf995 100644
> > --- a/lib/libalpm/cache.c
> > +++ b/lib/libalpm/cache.c
> > @@ -24,6 +24,7 @@
> > #include <stdlib.h>
> > #include <errno.h>
> > #include <string.h>
> > +#include <dirent.h>
> >
> > /* libalpm */
> > #include "cache.h"
> > @@ -54,6 +55,7 @@ int _alpm_db_load_pkgcache(pmdb_t *db)
> > _alpm_log(PM_LOG_DEBUG, "loading package cache for repository '%s'
",
> > db->treename);
> >
> > + rewinddir(db->handle);
> > while((info = _alpm_db_scan(db, NULL)) != NULL) {
> > _alpm_log(PM_LOG_FUNCTION, "adding '%s' to package
> > cache for db '%s'
",
> >
> > alpm_pkg_get_name(info), db->treename);
> >
> >
>
> Oh no shit, I noticed this and wanted to ask you about it, why you
> removed that rewinddir call...
> But then I looked at db_scan code, and it seems like it did a
> rewinddir itself, so I thought it was fine ...
> And when you showed your weird bug, I thought about that rewinddir
> stuff again.. But I didn't say anything.. I suck.
> I was still building your last git branch to try reproducing it first.

I thought the same thing- I saw a rewinddir call in db_scan, so I
removed what I thought was an extra one here. Either way, I don't plan
on applying the above patch, as we should just do the rewinddir() call
inside db_scan() or do something more sane than the above hackjob. I
just wanted to point out this was the issue.

Allan- I don't know what filesystem you are using, but I wonder if it
has slightly different *dir() semantics so the bug did not manifest
itself there...

-Dan

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 05-12-2008, 02:10 PM
Xavier
 
Default Weird bug in sync/upgrade behavior

On Mon, May 12, 2008 at 4:00 PM, Dan McGee <dpmcgee@gmail.com> wrote:
>
> I thought the same thing- I saw a rewinddir call in db_scan, so I
> removed what I thought was an extra one here. Either way, I don't plan
> on applying the above patch, as we should just do the rewinddir() call
> inside db_scan() or do something more sane than the above hackjob. I
> just wanted to point out this was the issue.
>

Ok, I just understood how that worked now... I first thought we could
do rewinddir in db_scan but it's not easy. When we do a full scan, we
want do init the scan first with db_rewind, then scan each entry one
by one using one db_scan call each time.

I preferred the old way rather than the above hack, even if they do
exactly the same thing. It was a bit less obscure and more consistent:
_alpm_db_rewind(db);
while((info = _alpm_db_scan(db, NULL)) != NULL) {

Unless you find a better way of course

> Allan- I don't know what filesystem you are using, but I wonder if it
> has slightly different *dir() semantics so the bug did not manifest
> itself there...
>

I was able to reproduce it without problems, with latest git and ext3
filesystem.

_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 05-12-2008, 02:50 PM
Allan McRae
 
Default Weird bug in sync/upgrade behavior

Xavier wrote:
> On Mon, May 12, 2008 at 4:00 PM, Dan McGee <dpmcgee@gmail.com> wrote:
>
>> Allan- I don't know what filesystem you are using, but I wonder if it
>> has slightly different *dir() semantics so the bug did not manifest
>> itself there...
>>
>>
>
> I was able to reproduce it without problems, with latest git and ext3
> filesystem.
>

I am using latest git and ext3 too... I must be immune to all bugs!



_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 
Old 05-13-2008, 08:35 AM
Nagy Gabor
 
Default Weird bug in sync/upgrade behavior

Idézés Xavier <shiningxc@gmail.com>:

> On Mon, May 12, 2008 at 4:00 PM, Dan McGee <dpmcgee@gmail.com> wrote:
> >
> > I thought the same thing- I saw a rewinddir call in db_scan, so I
> > removed what I thought was an extra one here. Either way, I don't plan
> > on applying the above patch, as we should just do the rewinddir() call
> > inside db_scan() or do something more sane than the above hackjob. I
> > just wanted to point out this was the issue.
> >
>
> Ok, I just understood how that worked now... I first thought we could
> do rewinddir in db_scan but it's not easy. When we do a full scan, we
> want do init the scan first with db_rewind, then scan each entry one
> by one using one db_scan call each time.
>
> I preferred the old way rather than the above hack, even if they do
> exactly the same thing. It was a bit less obscure and more consistent:
> _alpm_db_rewind(db);
> while((info = _alpm_db_scan(db, NULL)) != NULL) {
>

As I see, we always call _alpm_db_scan with NULL param. I think we can remove
this param and modify alpm_db_scan to return with alpm_list_t of packages, which
can be thought as TOC, and the packages can be accessed via alpm_db_read. Thus
the rewinddir stuff can be moved there. On the other side we would lose some
flexibility (however, we can read whole db, and do an ~alpm_pkg_find +
alpm_db_read; and if this is needed more than once it worth loading pkgcache).

Bye


----------------------------------------------------
SZTE Egyetemi Könyvtár - http://www.bibl.u-szeged.hu
This mail sent through IMP: http://horde.org/imp/


_______________________________________________
pacman-dev mailing list
pacman-dev@archlinux.org
http://archlinux.org/mailman/listinfo/pacman-dev
 

Thread Tools




All times are GMT. The time now is 04:42 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org