FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 01-26-2011, 11:12 AM
PK
 
Default Simultaneously emerging multiple packages with same dependencies

Hi,
Is there any way to simultaneously emerge multiple packages (multiple instances of 'emerge') that share common dependencies ?
I'm aware that portage uses locking mechanism before modifying 'world' file, but what about the actual building process ? I'd expect emerge to check if dependency package is already build/installed (or currently being build by another instance)*and just skip it in this case, however I haven't tried it yet.. Can anybody shred some light on this ?

Thanks,P.
 
Old 01-26-2011, 12:07 PM
Petri Rosenström
 
Default Simultaneously emerging multiple packages with same dependencies

On Wed, Jan 26, 2011 at 2:12 PM, PK <pkugrinas@gmail.com> wrote:
>
> Hi,
> Is there any way to simultaneously emerge multiple packages (multiple instances of 'emerge') that share common dependencies ?
> I'm aware that portage uses locking mechanism before modifying 'world' file, but what about the actual building process ? I'd expect emerge to check if dependency package is already build/installed (or currently being build by another instance)*and just skip it in this case, however I haven't tried it yet.. Can anybody shred some light on this ?
> Thanks,
> P.

Hi,

--jobs isn't enough? example emerge -j kde-meta

Best regards
Petri Rosenström
 
Old 01-26-2011, 12:49 PM
Neil Bothwick
 
Default Simultaneously emerging multiple packages with same dependencies

On Wed, 26 Jan 2011 12:12:57 +0000, PK wrote:

> Is there any way to simultaneously emerge multiple packages (multiple
> instances of 'emerge') that share common dependencies ?

Don't use multiple instances of emerge, they'll probably all try to merge
the same dependencies. Do it the way the portage developers intended, use
the --jobs option with a single emerge instance.


--
Neil Bothwick

The severity of the itch is inversely proportional to the reach.
 
Old 01-26-2011, 05:28 PM
PK
 
Default Simultaneously emerging multiple packages with same dependencies

Don't use multiple instances of emerge, they'll probably all try to merge

the same dependencies. Do it the way the portage developers intended, use

the --jobs option with a single emerge instance.





Sure, but sometimes while emerge is building a bunch of packages I remember that I've forgotten to specify a few more, my only option then is to wait till building process finishes. I think it should be possible to be more efficient than that.

Cheers,P.
 
Old 01-26-2011, 06:43 PM
Jacob Todd
 
Default Simultaneously emerging multiple packages with same dependencies

Then ^c and add/remove what you need.

On Jan 26, 2011 1:30 PM, "PK" <pkugrinas@gmail.com> wrote:>>
>> Don't use multiple instances of emerge, they'll probably all try to merge

>> the same dependencies. Do it the way the portage developers intended, use
>> the --jobs option with a single emerge instance.
>>
>>
> Sure, but sometimes while emerge is building a bunch of packages I remember

> that I've forgotten to specify a few more, my only option then is to wait
> till building process finishes. I think it should be possible to be more
> efficient than that.
>
> Cheers,

> P.
 
Old 01-27-2011, 04:12 AM
Nikos Chantziaras
 
Default Simultaneously emerging multiple packages with same dependencies

On 01/26/2011 02:12 PM, PK wrote:

Hi,

Is there any way to simultaneously emerge multiple packages (multiple
instances of 'emerge') that share common dependencies ?

I'm aware that portage uses locking mechanism before modifying 'world'
file, but what about the actual building process ? I'd expect emerge to
check if dependency package is already build/installed (or currently
being build by another instance) and just skip it in this case, however
I haven't tried it yet.. Can anybody shred some light on this ?


You can try, but the second instance with simply block until the lock
has been removed.


I'm not aware of any package system that supports this. I don't think
adding support for this justifies the added complexity.
 
Old 01-27-2011, 09:01 AM
Neil Bothwick
 
Default Simultaneously emerging multiple packages with same dependencies

On Thu, 27 Jan 2011 07:12:24 +0200, Nikos Chantziaras wrote:

> > I'm aware that portage uses locking mechanism before modifying 'world'
> > file, but what about the actual building process ? I'd expect emerge
> > to check if dependency package is already build/installed (or
> > currently being build by another instance) and just skip it in this
> > case, however I haven't tried it yet.. Can anybody shred some light
> > on this ?
>
> You can try, but the second instance with simply block until the lock
> has been removed.

The lock is not there for the entire emerge, I have run two emerges at
the same time, such as when I needed to install something while a world
update is in progress. It is possible, but not recommended as a general
strategy. That's what --jobs is for.


--
Neil Bothwick

Top Oxymorons Number 21: "Now, then ..."
 
Old 01-27-2011, 12:11 PM
Dale
 
Default Simultaneously emerging multiple packages with same dependencies

Neil Bothwick wrote:

On Thu, 27 Jan 2011 07:12:24 +0200, Nikos Chantziaras wrote:



I'm aware that portage uses locking mechanism before modifying 'world'
file, but what about the actual building process ? I'd expect emerge
to check if dependency package is already build/installed (or
currently being build by another instance) and just skip it in this
case, however I haven't tried it yet.. Can anybody shred some light
on this ?


You can try, but the second instance with simply block until the lock
has been removed.


The lock is not there for the entire emerge, I have run two emerges at
the same time, such as when I needed to install something while a world
update is in progress. It is possible, but not recommended as a general
strategy. That's what --jobs is for.





I have done the same thing and as long as the dependencies don't clash,
it works fine. However, if you start one emerge with a set of
dependencies, then start another and they clash somewhere in the middle,
portage has issues. That is where the locks would kick in I guess. I
would also imagine that portage could emerge the same package twice
too. If one instance of emerge doesn't know what the other instance has
already done, then the second one could emerge it again. Doesn't emerge
do all the calculating at the beginning and runs with that until the end?


I am using the -j option for the first time now. I'm updating KDE. It
seems to work fine. It doesn't scroll all the stuff like with a regular
emerges but this new rig is so fast, I can't read it anyway. I did have
a package to fail and it spit out the error for me to read.


I agree, using --jobs is the best way to do this. It works really well
if you have a fast multi-core CPU. I wish I had got me a 6 core one
now. ;-)


Dale

:-) :-)
 
Old 01-27-2011, 12:33 PM
Nikos Chantziaras
 
Default Simultaneously emerging multiple packages with same dependencies

On 01/27/2011 03:11 PM, Dale wrote:

[...]
I am using the -j option for the first time now. I'm updating KDE. It
seems to work fine. It doesn't scroll all the stuff like with a regular
emerges but this new rig is so fast, I can't read it anyway. I did have
a package to fail and it spit out the error for me to read.


You don't need that if you have MAKEOPTS set in your make.conf, which is
preferred. The -j option of emerge emerges multiple packages, while
with MAKEOPTS set to "-j4" or whatever does a parallel build in the same
package (meaning compiling multiple source files at the same time).


It's preferred because with "emerge -jN" the last package will only use
one CPU, while with "-jN" in MAKEOPTS even the last package will use N
CPUs. Furthermore, emerge can't always build N packages at the same
time because one can depend on the other, so it will have to wait until
the dependency is built.
 
Old 01-27-2011, 01:05 PM
YoYo Siska
 
Default Simultaneously emerging multiple packages with same dependencies

On Thu, Jan 27, 2011 at 03:33:21PM +0200, Nikos Chantziaras wrote:
> On 01/27/2011 03:11 PM, Dale wrote:
> >[...]
> >I am using the -j option for the first time now. I'm updating KDE. It
> >seems to work fine. It doesn't scroll all the stuff like with a regular
> >emerges but this new rig is so fast, I can't read it anyway. I did have
> >a package to fail and it spit out the error for me to read.
>
> You don't need that if you have MAKEOPTS set in your make.conf,
> which is preferred. The -j option of emerge emerges multiple
> packages, while with MAKEOPTS set to "-j4" or whatever does a
> parallel build in the same package (meaning compiling multiple
> source files at the same time).
>
> It's preferred because with "emerge -jN" the last package will only
> use one CPU, while with "-jN" in MAKEOPTS even the last package will
> use N CPUs. Furthermore, emerge can't always build N packages at
> the same time because one can depend on the other, so it will have
> to wait until the dependency is built.

On the other hand, unpacking, configure and install stages are not
parallel and emerge can do those in parallel for different packages...
The best would be somewhere in the middle


There are also the load-average options to -j, i.e.:
MAKEOPTS="-j -l5" emerge -j --load-average=5 ....

which makes make spawn parallel processes while load average is below 5
and the same for emerge spawning parallel ebuilds (when make isn't
parallel enough)

yoyo
 

Thread Tools




All times are GMT. The time now is 06:55 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org