FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 09-18-2008, 01:01 PM
BRM
 
Default Emerge takes a long time calculating dependencies...

I've noticed recently that Emerge is taking a long time (e.g. >20 minutes) to calculate the dependencies when I run "emerge world -vuDNp".
I'm running on a PII-233 w/ 190MB RAM and using the 2008.0 server profile. (I've thought about moving to the hardened profile, but want to at least get things finished configured first, which I'm taking my slow time doing.)
It use to run pretty fast (e.g. < 4 minutes). This started probably back in July some time, I haven't really added much of anything

The system originally ran the 2007.0 desktop profile (late 2007), then I moved it to the 2008.0 desktop profile (June 2008 - before the final release by a week or so I think...it was when a lot of questions started cycling on this list about the 2008.0 profile but before the final release was made), and later to the 2008.0 server profile (within a couple weeks). In case it matter, I was an late-early adopter of the 2008.0 profile set. (In moving to the server profile, I first moved to the 2008.0 profile, let it sit a week or so, and then moved to the server profile.)

The software is pretty up-to-date. It does have a couple ~x86 programs - but very very few. (Can't remember for sure off the top of my head though.)

I've gone through a few kernels on it, and have been simply deleting the kernel sources in /usr/src for the kernels I didn't use.

Any ideas on what might be making it go so slow?
I've thought about doing some clean-ups with e-merge (e.g. emerge --deep-clean), but haven't risked it yet.

In all other respects, the systems seems to be performing fine. It runs 24-7, hosts my internet connection and network services for my other systems.

TIA,

Ben
 
Old 09-18-2008, 01:08 PM
Nikos Chantziaras
 
Default Emerge takes a long time calculating dependencies...

BRM wrote:

I've noticed recently that Emerge is taking a long time (e.g. >20 minutes) to calculate the dependencies when I run "emerge world -vuDNp".
I'm running on a PII-233 w/ 190MB RAM and using the 2008.0 server profile. (I've thought about moving to the hardened profile, but want to at least get things finished configured first, which I'm taking my slow time doing.)
It use to run pretty fast (e.g. < 4 minutes). This started probably back in July some time, I haven't really added much of anything


The system originally ran the 2007.0 desktop profile (late 2007), then I moved it to the 2008.0 desktop profile (June 2008 - before the final release by a week or so I think...it was when a lot of questions started cycling on this list about the 2008.0 profile but before the final release was made), and later to the 2008.0 server profile (within a couple weeks). In case it matter, I was an late-early adopter of the 2008.0 profile set. (In moving to the server profile, I first moved to the 2008.0 profile, let it sit a week or so, and then moved to the server profile.)

The software is pretty up-to-date. It does have a couple ~x86 programs - but very very few. (Can't remember for sure off the top of my head though.)

I've gone through a few kernels on it, and have been simply deleting the kernel sources in /usr/src for the kernels I didn't use.

Any ideas on what might be making it go so slow?
I've thought about doing some clean-ups with e-merge (e.g. emerge --deep-clean), but haven't risked it yet.

In all other respects, the systems seems to be performing fine. It runs 24-7, hosts my internet connection and network services for my other systems.


Try to "defragment" the portage directories. That is, move them to
another disk/partition and then back again. This happens here too after
some time. After I do this, it's fast again. I'm on ext3.
 
Old 09-18-2008, 01:17 PM
"Daniel Pielmeier"
 
Default Emerge takes a long time calculating dependencies...

2008/9/18 BRM <bm_witness@yahoo.com>:
> Any ideas on what might be making it go so slow?
> I've thought about doing some clean-ups with e-merge (e.g. emerge --deep-clean), but haven't risked it yet.

Did you mean "emerge --depclean"?

> In all other respects, the systems seems to be performing fine. It runs 24-7, hosts my internet connection and network services for my other systems.

How do you sync your portage tree with this machine? How often? Do you
use some kind of script?

Regards,

Daniel
 
Old 09-18-2008, 01:24 PM
BRM
 
Default Emerge takes a long time calculating dependencies...

Yes - emerge --depclean.

And syncing portage is done nightly using a cron job to do "emerge --sync"; however, I run "emerge world -vuDNp" manually.

(The system also hosts portage via rsync for my other gentoo system on my network too, which I manually sync against it; I might be adding another system too in the same manner, not sure yet.)

Ben



----- Original Message ----
From: Daniel Pielmeier <daniel.pielmeier@googlemail.com>
To: gentoo-user@lists.gentoo.org
Sent: Thursday, September 18, 2008 9:17:35 AM
Subject: Re: [gentoo-user] Emerge takes a long time calculating dependencies...

2008/9/18 BRM <bm_witness@yahoo.com>:
> Any ideas on what might be making it go so slow?
> I've thought about doing some clean-ups with e-merge (e.g. emerge --deep-clean), but haven't risked it yet.

Did you mean "emerge --depclean"?

> In all other respects, the systems seems to be performing fine. It runs 24-7, hosts my internet connection and network services for my other systems.

How do you sync your portage tree with this machine? How often? Do you
use some kind of script?

Regards,

Daniel
 
Old 09-18-2008, 01:28 PM
Robert Bridge
 
Default Emerge takes a long time calculating dependencies...

On Thu, 18 Sep 2008 06:24:34 -0700 (PDT)
BRM <bm_witness@yahoo.com> wrote:

> (The system also hosts portage via rsync for my other gentoo system
> on my network too, which I manually sync against it; I might be
> adding another system too in the same manner, not sure yet.)

Not particularly relevant to the topic at hand, but had you considered
and NFS share for the portage tree? Simplifies tree management within
the network.

Rob.
 
Old 09-18-2008, 01:36 PM
"Daniel Pielmeier"
 
Default Emerge takes a long time calculating dependencies...

2008/9/18 BRM <bm_witness@yahoo.com>:
> And syncing portage is done nightly using a cron job to do "emerge --sync"; however, I run "emerge world -vuDNp" manually.

How often does this cron job run and what are the contents of your
cron script. Do you use "emerge --sync" or probably "eix-sync". I am
asking this because if you sync to often and you use eix-sync the
portage cache is deleted in advantage of the sync and is not recreated
if it fails because of identical timestamps between the local machine
and the rsync server.

So when calling emerge after this the cache has to be created before
and thus it seems dependency calculation takes very long.

> (The system also hosts portage via rsync for my other gentoo system on my network too, which I manually sync against it; I might be adding another system too in the same manner, not sure yet.)
>
> Ben
 
Old 09-18-2008, 01:43 PM
Ricardo Saffi Marques
 
Default Emerge takes a long time calculating dependencies...

Daniel Pielmeier wrote:
> How often does this cron job run and what are the contents of your
> cron script. Do you use "emerge --sync" or probably "eix-sync". I am
> asking this because if you sync to often and you use eix-sync the
> portage cache is deleted in advantage of the sync and is not recreated
> if it fails because of identical timestamps between the local machine
> and the rsync server.

In my case, my cron runs "eix-sync" every night at 00h and I just did an "time emerge -DNuvp world"
here, getting:

Total: 129 packages (103 upgrades, 18 new, 8 reinstalls), Size of downloads: 187,761 kB

real 0m21.074s
user 0m18.913s
sys 0m0.724s

So I assume once a day (let's say, every night) is a good choice for syncing, right? :-)
Never ever had a problem with slow dependency calculation on emerge.

--
Ricardo Saffi Marques
http://www.las.ic.unicamp.br/~saffi/
================================================== ====
Laboratory of System Administration and Security - LAS
Institute of Computing - IC
P.O. Box: 6176
University of Campinas - UNICAMP
13083-852, Campinas, SP, Brazil
================================================== ====
 
Old 09-18-2008, 02:01 PM
BRM
 
Default Emerge takes a long time calculating dependencies...

I only use the standard emerge utility. I can't access the system at the moment since I'm at work. I was hoping to have some ideas to try out tonight.

I believe my cron job just does a straight call to "emerge --sync" (e.g. no wrapping script, etc.); and it runs sometime between midnight and 6 AM. (Can't recall off hand.)
But - per gentoo portage protocol - it only runs once a day, no more.

And I am only referring to the time specified in the "calculating dependencies" stage when one first runs "emerge world -vuDNp".

I'll see if I have the space available to move portage on/off the partition it is on and see if that improves things; it is an ext3 partition that it is on.

Ben



----- Original Message ----
From: Daniel Pielmeier <daniel.pielmeier@googlemail.com>
To: gentoo-user@lists.gentoo.org
Sent: Thursday, September 18, 2008 9:36:23 AM
Subject: Re: [gentoo-user] Emerge takes a long time calculating dependencies...

2008/9/18 BRM <bm_witness@yahoo.com>:
> And syncing portage is done nightly using a cron job to do "emerge --sync"; however, I run "emerge world -vuDNp" manually.

How often does this cron job run and what are the contents of your
cron script. Do you use "emerge --sync" or probably "eix-sync". I am
asking this because if you sync to often and you use eix-sync the
portage cache is deleted in advantage of the sync and is not recreated
if it fails because of identical timestamps between the local machine
and the rsync server.

So when calling emerge after this the cache has to be created before
and thus it seems dependency calculation takes very long.

> (The system also hosts portage via rsync for my other gentoo system on my network too, which I manually sync against it; I might be adding another system too in the same manner, not sure yet.)
>
> Ben
 
Old 09-18-2008, 02:01 PM
Ricardo Saffi Marques
 
Default Emerge takes a long time calculating dependencies...

Ricardo Saffi Marques wrote:
> In my case, my cron runs "eix-sync" every night at 00h and I just did an "time emerge -DNuvp world"
> here, getting:
>
> Total: 129 packages (103 upgrades, 18 new, 8 reinstalls), Size of downloads: 187,761 kB
>
> real 0m21.074s
> user 0m18.913s
> sys 0m0.724s
>
> So I assume once a day (let's say, every night) is a good choice for syncing, right? :-)
> Never ever had a problem with slow dependency calculation on emerge.

Just out of curiosity, the sync entry on my cron:

00 00 * * * /usr/bin/eix-sync -q > /dev/null 2>&1 || echo -e "$(date)


$(hostname -f) was NOT
sync'ed.

" | mail -s "harley.las sync fail" saffi@las.ic.unicamp.br

And you can use "genlop -r" (from "app-portage/genlop") to check your sync history.

--
Ricardo Saffi Marques
http://www.las.ic.unicamp.br/~saffi/
================================================== ====
Laboratory of System Administration and Security - LAS
Institute of Computing - IC
P.O. Box: 6176
University of Campinas - UNICAMP
13083-852, Campinas, SP, Brazil
================================================== ====
 
Old 09-18-2008, 02:02 PM
"Daniel Pielmeier"
 
Default Emerge takes a long time calculating dependencies...

2008/9/18 Ricardo Saffi Marques <saffi@las.ic.unicamp.br>:
>
> In my case, my cron runs "eix-sync" every night at 00h and I just did an "time emerge -DNuvp world"
> here, getting:
>
> Total: 129 packages (103 upgrades, 18 new, 8 reinstalls), Size of downloads: 187,761 kB
>
> real 0m21.074s
> user 0m18.913s
> sys 0m0.724s
>
> So I assume once a day (let's say, every night) is a good choice for syncing, right? :-)
> Never ever had a problem with slow dependency calculation on emerge.

As far as I know the rsync servers are syncing against the master
server(s) about every half an hour which obviously also updates the
timestamp. So what I described above only happens if you sync within a
shorter period. Running eix-sync every day shouldn't be a problem.

Syncing twice an hour is also considered bad practise because of the
heavy traffic involved. It just happens to me sometimes if I test
stuff and had to sync within a shorter period.

I just asked this because it could be possible that the script calls
eix-sync twice or after an emerge --sync which is not necessary or for
some other reason it is called twice within a too short period.

Regarding the fragmentation. I use ext3 and reiserfs for the portage
tree and did not recognize a significant slowdown. There is a slowdown
but not from 4 to 20 minutes. At least in my case.

Regards,

Daniel
 

Thread Tools




All times are GMT. The time now is 08:27 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org