On Monday 14 May 2012 15:08:32 Zac Medico wrote:
> Actually, the inode_var_name thing will not work unless it's all in
> one process.

hmm, true, but that's the level we currently parallelize at, so it's fine. we
do one subprocess per ELF and that includes the strip/splitdebug/splitsources.

parallelizing more than on a per-ELF basis will require much finer grained
queues which, while possible, would make the file much harder to hack on and
extend. and i'm not sure we'd see that much of a gain.
-mike

On 05/14/2012 01:10 PM, Mike Frysinger wrote:
> On Monday 14 May 2012 15:08:32 Zac Medico wrote:
>> Actually, the inode_var_name thing will not work unless it's all in
>> one process.
>
> hmm, true, but that's the level we currently parallelize at, so it's fine. we
> do one subprocess per ELF and that includes the strip/splitdebug/splitsources.
>
> parallelizing more than on a per-ELF basis will require much finer grained
> queues which, while possible, would make the file much harder to hack on and
> extend. and i'm not sure we'd see that much of a gain.
> -mike

The thing is, in the case of hardlinks, we're parallelizing multiple
times on the *same* elf. Anyway, I've fixed it by using a directory full
of hardlinks, in these commits:

http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=ad944275b88a50d2a1f694826b1 27cceb9221e78
http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=9ed00a9e70a3705164a5349145f f467e5c40ddfd
--
Thanks,
Zac

On Monday 14 May 2012 18:42:07 Zac Medico wrote:
> On 05/14/2012 01:10 PM, Mike Frysinger wrote:
> > On Monday 14 May 2012 15:08:32 Zac Medico wrote:
> >> Actually, the inode_var_name thing will not work unless it's all in
> >> one process.
> >
> > hmm, true, but that's the level we currently parallelize at, so it's
> > fine. we do one subprocess per ELF and that includes the
> > strip/splitdebug/splitsources.
> >
> > parallelizing more than on a per-ELF basis will require much finer
> > grained queues which, while possible, would make the file much harder to
> > hack on and extend. and i'm not sure we'd see that much of a gain.
>
> The thing is, in the case of hardlinks, we're parallelizing multiple
> times on the *same* elf. Anyway, I've fixed it by using a directory full
> of hardlinks, in these commits:

well, realistically speaking, hardlinking has been broken before the
parallelization work.
https://bugs.gentoo.org/400767

> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=ad9442
> 75b88a50d2a1f694826b127cceb9221e78
> http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commit;h=9ed00
> a9e70a3705164a5349145ff467e5c40ddfd

i'll go through it and see what's what
-mike