FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 08-21-2011, 12:35 AM
Peter Humphrey
 
Default Hoping someone can help explain distcc to me

On Saturday 20 August 2011 23:56:08 Paul Hartman wrote:

> I have a fast desktop computer and a slow laptop. Both are ~amd64
> Gentoo. After some of the recent discussions about Gentoo on slow
> devices, I thought I'd dust off the laptop and try to bring it up to
> date.
>
> I'd like to use distcc to make the desktop do all the compiling during
> emerges. I've never been able to get distcc working properly, or, at
> least, I've never been able to get it working to the point where using
> distcc is any faster than not using it at all.

Your laptop isn't sending its compilation jobs to your desktop. Have you
used gkrellm or similar to check for network traffic?

I think I'd take a different approach, one that I dare say you can
anticipate. By all means take the advice you're about to be offered to get
distcc working, but why not create a chroot on your desktop, NFS export the
packages directory from the laptop to it and do the whole job there? Then
return to the laptop and emerge -k. It does need more keystrokes but it uses
far less CPU. You just need to copy /var/lib/portage/world, make.conf and
the /etc/portage tree to the chroot before you start.

I did once get distcc working, but as Neil has observed re Atoms, a lot of
work is still done before compilation is farmed out to the distcc server,
rather diluting the benefit of distributed compilation.

--
Rgds
Peter Linux Counter 5290, 1994-04-23
 
Old 08-21-2011, 01:08 AM
Paul Hartman
 
Default Hoping someone can help explain distcc to me

On Sat, Aug 20, 2011 at 7:35 PM, Peter Humphrey
<peter@humphrey.ukfsn.org> wrote:
> On Saturday 20 August 2011 23:56:08 Paul Hartman wrote:
>
>> I have a fast desktop computer and a slow laptop. Both are ~amd64
>> Gentoo. After some of the recent discussions about Gentoo on slow
>> devices, I thought I'd dust off the laptop and try to bring it up to
>> date.
>>
>> I'd like to use distcc to make the desktop do all the compiling during
>> emerges. I've never been able to get distcc working properly, or, at
>> least, I've never been able to get it working to the point where using
>> distcc is any faster than not using it at all.
>
> Your laptop isn't sending its compilation jobs to your desktop. Have you
> used gkrellm or similar to check for network traffic?

Well, there are definitely some distcc jobs going to the desktop, I
see them, it's just that they finish so fast and there's a "long" time
between them. The logs indicate it is successfully compiling things.
But, it is in no way saturating the resources of the desktop, which is
what I was (perhaps naively) hoping to see.

> I think I'd take a different approach, one that I dare say you can
> anticipate. By all means take the advice you're about to be offered to get
> distcc working, but why not create a chroot on your desktop, NFS export the
> packages directory from the laptop to it and do the whole job there? Then
> return to the laptop and emerge -k. It does need more keystrokes but it uses
> far less CPU. You just need to copy /var/lib/portage/world, make.conf and
> the /etc/portage tree to the chroot before you start.
>
> I did once get distcc working, but as Neil has observed re Atoms, a lot of
> work is still done before compilation is farmed out to the distcc server,
> rather diluting the benefit of distributed compilation.

That was going to be my next approach if distcc just doesn't do enough.

Could I just export the entire laptop - everything from the root
directory and below - and chroot into that over the network? Then I
wouldn't even need to emerge -k...
 
Old 08-21-2011, 01:34 AM
victor romanchuk
 
Default Hoping someone can help explain distcc to me

> Both machines contain "distcc" in FEATURES. It's not using
> -march=native. I've tried various -jN values with no real difference
> in performance.

only client (your laptop) machine should be distcc featured. for server
(desktop) that feature is useless

> On the desktop, /etc/conf.d/distcc contains (among other things):
> DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/24"
> DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.100"

this is server distcc daemon configuration, one just instructs daemon on what
network interface to listen for incoming connections (interface with ip
192.168.0.100 in your case) and filter incoming distcc connections by source
address: allow only those coming from local network machines with ip addresses
192.168.0.1 to 192.168.0.254

then distccd have to be started: /etc/init.d/distccd start

> And /etc/distccd/hosts contains:
> 192.168.0.0/24

this file configures distcc client behavior (actually the configuration can be
complex, see distcc man page for details), but in trivial case (for home
computing) it might look like:

192.168.0.100/6 127.0.0.1/1

e.g the client is able to send up to 6 distcc jobs to 192.168.0.100 and limit to
one job at local machine. client's /etc/make.conf has to have distcc feature
enabled (FEATURES="distcc"). surely you can play with job distribution rules
around the network. `distcc --show-hosts` command displays what you configured.
the number of cuncurrently running jobs (-j flag) has to be not less than sum of
local and remote jobs

i had noticed that distcc is peevish about CFLAGS: these should be compatible on
both client and server. in my case i made these similar on both machines (laptop
is core2duo and desktop is core2quad; both are running amd64 arch)

yet another way to install packages on weak notebook running it on the same arch
as desktop runs, - is to create binaries at powerful machine (while emerging or
with quickpkg utility) and share $PKGDIR with laptop

hth
 
Old 08-21-2011, 01:58 AM
Peter Humphrey
 
Default Hoping someone can help explain distcc to me

On Sunday 21 August 2011 02:08:51 Paul Hartman wrote:

> Could I just export the entire laptop - everything from the root
> directory and below - and chroot into that over the network? Then I
> wouldn't even need to emerge -k...

No, I tried that and got myself tied in knots - well, actually it was the
whole portage tree that I exported, not the entire system. I forget what
went wrong now, but it's definitely cleaner to tell the server to build the
packages and the client to install from them. The emerge -k step is quick
too, and you have the advantage that you can see whether the packages are
actually there, unless you've switched colours off or not specified -v. (I
once found that they weren't there, which prompted me to go looking for the
config problem. Like Dale, I'm quite a good tester!)

You just have to make sure that the chroot is identical to the client.

--
Rgds
Peter Linux Counter 5290, 1994-04-23
 
Old 08-21-2011, 02:46 AM
Dale
 
Default Hoping someone can help explain distcc to me

Peter Humphrey wrote:

On Sunday 21 August 2011 02:08:51 Paul Hartman wrote:



Could I just export the entire laptop - everything from the root
directory and below - and chroot into that over the network? Then I
wouldn't even need to emerge -k...


No, I tried that and got myself tied in knots - well, actually it was the
whole portage tree that I exported, not the entire system. I forget what
went wrong now, but it's definitely cleaner to tell the server to build the
packages and the client to install from them. The emerge -k step is quick
too, and you have the advantage that you can see whether the packages are
actually there, unless you've switched colours off or not specified -v. (I
once found that they weren't there, which prompted me to go looking for the
config problem. Like Dale, I'm quite a good tester!)

You just have to make sure that the chroot is identical to the client.




Since you mentioned me. I wish I could set up a quicky from my 4 core
64 bit machine to compile 32 bit packages for a older 2GHz machine that
belongs to a friend. I was going to put Mandriva on it but the CD won;t
boot up properly. It stops at starting udev. Grrrrr.


How hard is it to set up a 64 bit machine to compile programs for a 32
bit system?


Dale

:-) :-)
 
Old 08-21-2011, 03:04 AM
Matthew Finkel
 
Default Hoping someone can help explain distcc to me

On Sat, Aug 20, 2011 at 10:46 PM, Dale <rdalek1967@gmail.com> wrote:

Peter Humphrey wrote:


On Sunday 21 August 2011 02:08:51 Paul Hartman wrote:



*


Could I just export the entire laptop - everything from the root

directory and below - and chroot into that over the network? Then I

wouldn't even need to emerge -k...

* *


No, I tried that and got myself tied in knots - well, actually it was the

whole portage tree that I exported, not the entire system. I forget what

went wrong now, but it's definitely cleaner to tell the server to build the

packages and the client to install from them. The emerge -k step is quick

too, and you have the advantage that you can see whether the packages are

actually there, unless you've switched colours off or not specified -v. (I

once found that they weren't there, which prompted me to go looking for the

config problem. Like Dale, I'm quite a good tester!)



You just have to make sure that the chroot is identical to the client.



*




Since you mentioned me. *I wish I could set up a quicky from my 4 core 64 bit machine to compile 32 bit packages for a older 2GHz machine that belongs to a friend. *I was going to put Mandriva on it but the CD won;t boot up properly. *It stops at starting udev. *Grrrrr.




How hard is it to set up a 64 bit machine to compile programs for a 32 bit system?



Dale



:-) *:-)




It's actually quite easy. IIRC, when I did it last, the only difference is that when you chroot into the subsystem you need prefix the command with linux32, e.g. linux32 chroot /path/to/chroot /bin/bash
 
Old 08-21-2011, 09:41 AM
Peter Humphrey
 
Default Hoping someone can help explain distcc to me

On Sunday 21 August 2011 04:04:05 Matthew Finkel wrote:

> On Sat, Aug 20, 2011 at 10:46 PM, Dale <rdalek1967@gmail.com> wrote:
> > How hard is it to set up a 64 bit machine to compile programs for a 32
> > bit system?
> >
> > Dale
> >
> > :-) :-)
>
> It's actually quite easy. IIRC, when I did it last, the only difference
> is that when you chroot into the subsystem you need prefix the command
> with linux32, e.g. linux32 chroot /path/to/chroot /bin/bash

Yes, just follow this guide:
http://www.gentoo.org/proj/en/base/amd64/howtos/chroot.xml

I did that and it was straightforward as far as I remember. I did spend some
time thinking at a few stages, to get an understanding of what I was doing
rather than just blindly following somebody else's prescription.

Then it's a matter of writing some simple scripts to mount the packages
directory on the big host. Here's mine, most of which I scrounged from
somewhere:

$ cat /etc/init.d/atom
#!/sbin/runscript

depend() {
need localmount
need bootmisc
}

start() {
ebegin "Mounting 32-bit chroot dirs"
mount -o bind /dev /mnt/atom/dev >/dev/null
mount -o bind /dev/pts /mnt/atom/dev/pts >/dev/null
mount -o bind /dev/shm /mnt/atom/dev/shm >/dev/null
mount -t proc /proc /mnt/atom/proc >/dev/null
mount -o bind /sys /mnt/atom/sys >/dev/null
mount -o bind /tmp /mnt/atom/tmp >/dev/null
mount -t nfs -o vers=3 192.168.2.2:/usr/portage/packages
/mnt/atom/usr/portage/packages
eend $? "An error occurred while attempting to mount 32-bit chroot
directories"
ebegin "Copying 32-bit chroot files"
cp -pf /etc/resolv.conf /mnt/atom/etc/ >/dev/null
cp -pf /etc/passwd /mnt/atom/etc/ >/dev/null
cp -pf /etc/shadow /mnt/atom/etc/ >/dev/null
cp -pf /etc/group /mnt/atom/etc/ >/dev/null
cp -pf /etc/hosts /mnt/atom/etc/ > /dev/null
cp -Ppf /etc/localtime /mnt/atom/etc/ >/dev/null
eend $? "An error occurred while attempting to copy 32-bit chroot files."
}

stop() {
ebegin "Unmounting 32-bit chroot dirs"
umount -f /mnt/atom/dev/pts >/dev/null
umount -f /mnt/atom/dev/shm >/dev/null
umount -f /mnt/atom/dev >/dev/null
umount -f /mnt/atom/proc >/dev/null
umount -f /mnt/atom/sys >/dev/null
umount -f /mnt/atom/tmp >/dev/null
umount -f /mnt/atom/usr/portage/packages >/dev/null
eend $? "An error occurred while attempting to unmount 32-bit chroot
directories"
}

I could list the steps of my daily routine to upgrade both the client and
the chroot if that would help.

--
Rgds
Peter Linux Counter 5290, 1994-04-23
 
Old 08-21-2011, 10:38 AM
Alex Schuster
 
Default Hoping someone can help explain distcc to me

victor romanchuk writes:

> > Both machines contain "distcc" in FEATURES. It's not using
> > -march=native. I've tried various -jN values with no real difference
> > in performance.

-jN in make.conf's MAPEOPTS variable I assume, not as argument to emerge,
which does something different. It also hides the normal compile messages,
in case of problems you should see something like 'failed to distribute to
<host>'.
And syslog on the desktop should list every distributed job.

Distcc helps a lot here when I use Gentoo on slow systems. Although they are
still slow, because of time spent for configuring, or linking.

And maybe you are also using ccache, and your tests were recompiles of stuff
already cached, so the slow machine is fast at this?


> i had noticed that distcc is peevish about CFLAGS: these should be
> compatible on both client and server. in my case i made these similar on
> both machines (laptop is core2duo and desktop is core2quad; both are
> running amd64 arch)

I don't think this is true - as long as the CHOST is identical, there should
be no problem.

> yet another way to install packages on weak notebook running it on the
> same arch as desktop runs, - is to create binaries at powerful machine
> (while emerging or with quickpkg utility) and share $PKGDIR with laptop

This means some extra work, and also use flags need to be compatible, but
the speedup would be much bigger than with just distcc.

What about exporting the whole root file system and mounting it on the fast
desktop, chrooting and emerging?

And then there's Sabayon, based on Gentoo, but with binary packages. And
when you want to customize things (use different use flags), you can still
compile stuff manually. I will try this for a slow desktop PC now, all this
compiling is so annoying.

Wonko
 
Old 08-21-2011, 01:53 PM
Joost Roeleveld
 
Default Hoping someone can help explain distcc to me

On Sunday, August 21, 2011 10:41:56 AM Peter Humphrey wrote:
> On Sunday 21 August 2011 04:04:05 Matthew Finkel wrote:
> > On Sat, Aug 20, 2011 at 10:46 PM, Dale <rdalek1967@gmail.com> wrote:
> > > How hard is it to set up a 64 bit machine to compile programs for a
> > > 32
> > > bit system?
> > >
> > > Dale
> > >
> > > :-) :-)
> >
> > It's actually quite easy. IIRC, when I did it last, the only difference
> > is that when you chroot into the subsystem you need prefix the command
> > with linux32, e.g. linux32 chroot /path/to/chroot /bin/bash
>
> Yes, just follow this guide:
> http://www.gentoo.org/proj/en/base/amd64/howtos/chroot.xml
>
> I did that and it was straightforward as far as I remember. I did spend some
> time thinking at a few stages, to get an understanding of what I was doing
> rather than just blindly following somebody else's prescription.
>
> Then it's a matter of writing some simple scripts to mount the packages
> directory on the big host. Here's mine, most of which I scrounged from
> somewhere:
>
> $ cat /etc/init.d/atom
> #!/sbin/runscript
>
> depend() {
> need localmount
> need bootmisc
> }
>
> start() {
> ebegin "Mounting 32-bit chroot dirs"
> mount -o bind /dev /mnt/atom/dev >/dev/null
> mount -o bind /dev/pts /mnt/atom/dev/pts >/dev/null
> mount -o bind /dev/shm /mnt/atom/dev/shm >/dev/null
> mount -t proc /proc /mnt/atom/proc >/dev/null
> mount -o bind /sys /mnt/atom/sys >/dev/null
> mount -o bind /tmp /mnt/atom/tmp >/dev/null
> mount -t nfs -o vers=3 192.168.2.2:/usr/portage/packages
> /mnt/atom/usr/portage/packages
> eend $? "An error occurred while attempting to mount 32-bit chroot
> directories"
> ebegin "Copying 32-bit chroot files"
> cp -pf /etc/resolv.conf /mnt/atom/etc/ >/dev/null
> cp -pf /etc/passwd /mnt/atom/etc/ >/dev/null
> cp -pf /etc/shadow /mnt/atom/etc/ >/dev/null
> cp -pf /etc/group /mnt/atom/etc/ >/dev/null
> cp -pf /etc/hosts /mnt/atom/etc/ > /dev/null
> cp -Ppf /etc/localtime /mnt/atom/etc/ >/dev/null
> eend $? "An error occurred while attempting to copy 32-bit chroot
> files." }
>
> stop() {
> ebegin "Unmounting 32-bit chroot dirs"
> umount -f /mnt/atom/dev/pts >/dev/null
> umount -f /mnt/atom/dev/shm >/dev/null
> umount -f /mnt/atom/dev >/dev/null
> umount -f /mnt/atom/proc >/dev/null
> umount -f /mnt/atom/sys >/dev/null
> umount -f /mnt/atom/tmp >/dev/null
> umount -f /mnt/atom/usr/portage/packages >/dev/null
> eend $? "An error occurred while attempting to unmount 32-bit chroot
> directories"
> }
>
> I could list the steps of my daily routine to upgrade both the client and
> the chroot if that would help.

That would help as I'm planning on setting this up myself as well for my
netbook.

Is there a way to automate the steps inside the chroot without having to have
a script inside the chroot?

--
Joost
 
Old 08-21-2011, 02:39 PM
victor romanchuk
 
Default Hoping someone can help explain distcc to me

>
>> i had noticed that distcc is peevish about CFLAGS: these should be
>> compatible on both client and server. in my case i made these similar on
>> both machines (laptop is core2duo and desktop is core2quad; both are
>> running amd64 arch)
> I don't think this is true - as long as the CHOST is identical, there should
> be no problem.

CHOST defines the arch (i686, amd64, arm ..) whilst CFLAGS control gcc behavior
and the binary code generation produced by compiler. in my case core2quad
(q8300) i'm using in the desktop supports sse4.1 instruction set and notebook
powered with core2duo (t7600) does not have that cpu feature. having option
`-msse4.1' set in CFLAGS at desktop side will causes frequent compilation
failures (initiated by distcc) or, in worst case - arbitrary crashes at notebook
when running binaries compiled in distributed distcc environment

>> yet another way to install packages on weak notebook running it on the
>> same arch as desktop runs, - is to create binaries at powerful machine
>> (while emerging or with quickpkg utility) and share $PKGDIR with laptop
> This means some extra work, and also use flags need to be compatible, but
> the speedup would be much bigger than with just distcc.
>
> What about exporting the whole root file system and mounting it on the fast
> desktop, chrooting and emerging?
>

i do not insist the distcc is the only or most efficient way to maintain gentoo
installation on a slow machine (having something more powerful nearby). just
tried to explain how does distcc work

victor
 

Thread Tools




All times are GMT. The time now is 08:25 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org