FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 05-17-2010, 11:37 AM
Iain Buchanan
 
Default identical drives, different free space!

On Mon, 2010-05-17 at 09:07 +0100, Neil Bothwick wrote:
> On Mon, 17 May 2010 11:21:50 +0930, Iain Buchanan wrote:
>
> > Well, it turns out I have the distfiles mounted with --bind to my
> > ftp/pub directory. And looking in the rsync man page:
>
> Why not set $DISTDIR to the true location of distfiles instead of using
> bind mounts?

because /usr/portage/distfiles IS the real location,
and /home/ftp/pub/gentoo/distfiles is the ftp shared location. vsftpd
doesn't handle symlinks, so I have to bind it.

Now that you mention it though, I could move it for real
into /home/ftp/pub/gentoo/distfiles and change DISTDIR... hm.

--
Iain Buchanan <iaindb at netspace dot net dot au>

Real programmers don't bring brown-bag lunches. If the vending machine
doesn't sell it, they don't eat it. Vending machines don't sell quiche.
 
Old 05-17-2010, 11:39 AM
Neil Bothwick
 
Default identical drives, different free space!

On Mon, 17 May 2010 12:31:17 +0100, David W Noon wrote:

> >> Well, it turns out I have the distfiles mounted with --bind to my
> >> ftp/pub directory. And looking in the rsync man page:
> >
> >Why not set $DISTDIR to the true location of distfiles instead of using
> >bind mounts?
>
> Because binding the directory to /home/ftp/pub makes the distfiles
> available to the rest of one's network via anonymous ftp. I do the
> same thing here, without the "pub" subdirectory, and exclude /home/ftp/
> from my backups.

So the distfiles are actually in /usr/portage/distfiles?

I share my distfiles but I don't use FTP as that means storing copies of
the same file on each computer. Instead, I use NFS. /mnt/portage is
shared across all machines on the network and DISTDIR is set
to /mnt/portage/distfiles in each make.conf.

Sharing /mnt/portage like this means I can also share my overlay across
the network at /mnt/portage/local.


--
Neil Bothwick

Top Oxymorons Number 18: Taped live
 
Old 05-17-2010, 12:20 PM
Iain Buchanan
 
Default identical drives, different free space!

On Mon, 2010-05-17 at 12:39 +0100, Neil Bothwick wrote:
> On Mon, 17 May 2010 12:31:17 +0100, David W Noon wrote:

...

> So the distfiles are actually in /usr/portage/distfiles?

for me yes, it looks the same for David.

> I share my distfiles but I don't use FTP as that means storing copies of
> the same file on each computer. Instead, I use NFS. /mnt/portage is
> shared across all machines on the network and DISTDIR is set
> to /mnt/portage/distfiles in each make.conf.
>
> Sharing /mnt/portage like this means I can also share my overlay across
> the network at /mnt/portage/local.

Until I pick up my laptop and drive to work, where network speeds to my
server drop from 100Mbit to 50kbit and I need that local copy!

Which is why I'm glad there are multiple ways to do it
--
Iain Buchanan <iaindb at netspace dot net dot au>

Old robot: I choose to believe what I was programmed to believe.
 
Old 05-17-2010, 01:40 PM
Neil Bothwick
 
Default identical drives, different free space!

On Mon, 17 May 2010 21:50:28 +0930, Iain Buchanan wrote:

> > I share my distfiles but I don't use FTP as that means storing copies
> > of the same file on each computer. Instead, I use NFS. /mnt/portage is
> > shared across all machines on the network and DISTDIR is set
> > to /mnt/portage/distfiles in each make.conf.


> Until I pick up my laptop and drive to work, where network speeds to my
> server drop from 100Mbit to 50kbit and I need that local copy!

I tend not to run emerges when away from home, although the lack of a
local copy does prove awkward after a kernel upgrade that requires a
rebuild of the wireless drivers. Not a situation I have to deal with any
more, thankfully.

> Which is why I'm glad there are multiple ways to do it

Indeed


--
Neil Bothwick

A chicken is an egg's way of producing more eggs.
 
Old 05-17-2010, 06:33 PM
David W Noon
 
Default identical drives, different free space!

On Mon, 17 May 2010 13:50:02 +0200, Neil Bothwick wrote about Re:
[gentoo-user] [SOLVED] identical drives, different free space!:

>On Mon, 17 May 2010 12:31:17 +0100, David W Noon wrote:
>
>> >> Well, it turns out I have the distfiles mounted with --bind to my
>> >> ftp/pub directory. And looking in the rsync man page:
>> >
>> >Why not set $DISTDIR to the true location of distfiles instead of
>> >using bind mounts?
>>
>> Because binding the directory to /home/ftp/pub makes the distfiles
>> available to the rest of one's network via anonymous ftp. I do the
>> same thing here, without the "pub" subdirectory, and
>> exclude /home/ftp/ from my backups.
>
>So the distfiles are actually in /usr/portage/distfiles?

Correct.

>I share my distfiles but I don't use FTP as that means storing copies
>of the same file on each computer. Instead, I use NFS. /mnt/portage is
>shared across all machines on the network and DISTDIR is set
>to /mnt/portage/distfiles in each make.conf.

I used to do that, but it meant my NFS server had to be running to
perform any software maintenance on any box, so it became a single point
of failure. The FTP approach allows each box to be self-reliant.

>Sharing /mnt/portage like this means I can also share my overlay across
>the network at /mnt/portage/local.

My boxes have different stuff in their overlays, and one uses no
overlay packages at all. Sharing overlays doesn't make much sense for
my set-up.
--
Regards,

Dave [RLU #314465]
================================================== ====================
dwnoon@ntlworld.com (David W Noon)
================================================== ====================
 
Old 05-17-2010, 08:53 PM
Neil Bothwick
 
Default identical drives, different free space!

On Mon, 17 May 2010 19:33:18 +0100, David W Noon wrote:

> >I share my distfiles but I don't use FTP as that means storing copies
> >of the same file on each computer. Instead, I use NFS. /mnt/portage is
> >shared across all machines on the network and DISTDIR is set
> >to /mnt/portage/distfiles in each make.conf.
>
> I used to do that, but it meant my NFS server had to be running to
> perform any software maintenance on any box, so it became a single point
> of failure. The FTP approach allows each box to be self-reliant.

Fair comment. I have DISTDIR on my mail server, so if that goes down,
I've more to worry about that a few tarballs. Even if it is inaccessible,
the other computers would simply download the files to the local
directory.

> >Sharing /mnt/portage like this means I can also share my overlay across
> >the network at /mnt/portage/local.
>
> My boxes have different stuff in their overlays, and one uses no
> overlay packages at all. Sharing overlays doesn't make much sense for
> my set-up.

It makes sense for me because everything is in one place, making
maintenance and backups simpler. Even if a package is only used on one
computer, for now, a central location still makes sense.


--
Neil Bothwick

The computer revolution is over. The computers won.
 
Old 05-18-2010, 03:13 AM
Bill Kenworthy
 
Default identical drives, different free space!

On Mon, 2010-05-17 at 21:53 +0100, Neil Bothwick wrote:
> On Mon, 17 May 2010 19:33:18 +0100, David W Noon wrote:
>
> > >I share my distfiles but I don't use FTP as that means storing copies
> > >of the same file on each computer. Instead, I use NFS. /mnt/portage is
> > >shared across all machines on the network and DISTDIR is set
> > >to /mnt/portage/distfiles in each make.conf.
> >
> > I used to do that, but it meant my NFS server had to be running to
> > perform any software maintenance on any box, so it became a single point
> > of failure. The FTP approach allows each box to be self-reliant.
>
> Fair comment. I have DISTDIR on my mail server, so if that goes down,
> I've more to worry about that a few tarballs. Even if it is inaccessible,
> the other computers would simply download the files to the local
> directory.
>
> > >Sharing /mnt/portage like this means I can also share my overlay across
> > >the network at /mnt/portage/local.
> >
> > My boxes have different stuff in their overlays, and one uses no
> > overlay packages at all. Sharing overlays doesn't make much sense for
> > my set-up.
>
> It makes sense for me because everything is in one place, making
> maintenance and backups simpler. Even if a package is only used on one
> computer, for now, a central location still makes sense.
>


As an alternative check out http-replicator - yes the clients do
download to a local directory but that can be cleaned afterwards. It
also allows download locally when you know you are taking the machine
(laptop?) elsewhere. An advantage over NFS is it seems to handle
parallel downloads of the same file so you can transparently build all
machines in parallel without the downloads stepping on each other over a
common NFS mount.

I also use a tmfs store for distfiles on one machine with plenty of ram
so thats a self-cleaning (on reboot alternative.

I have used NFS as well and its ok for data stores http-replicator is
much better. Beware - NFS can be slow and flakey if used for building
over (/var/tmp/portage).

The great thing about gentoo's build system is its so flexible!

BillK
 
Old 05-18-2010, 09:12 AM
Peter Humphrey
 
Default identical drives, different free space!

On Tuesday 18 May 2010 04:13:07 Bill Kenworthy wrote:

> As an alternative check out http-replicator - yes the clients do
> download to a local directory but that can be cleaned afterwards. It
> also allows download locally when you know you are taking the machine
> (laptop?) elsewhere.

Yet another approach is to have an rsync server on your LAN. In my case
it's the local server box (print, squid, mail etc). It's simple to set
up and to use, and all the boxes on the LAN can operate in their own
devious ways.

--
Rgds
Peter.
 
Old 05-18-2010, 10:19 AM
William Kenworthy
 
Default identical drives, different free space!

On Tue, 2010-05-18 at 10:12 +0100, Peter Humphrey wrote:
> On Tuesday 18 May 2010 04:13:07 Bill Kenworthy wrote:
>
> > As an alternative check out http-replicator - yes the clients do
> > download to a local directory but that can be cleaned afterwards. It
> > also allows download locally when you know you are taking the machine
> > (laptop?) elsewhere.
>
> Yet another approach is to have an rsync server on your LAN. In my case
> it's the local server box (print, squid, mail etc). It's simple to set
> up and to use, and all the boxes on the LAN can operate in their own
> devious ways.
>

The advantage of http-replicator is that it is a caching proxy - if it
isnt in the cache, it downloads it and then serves it out to one or more
clients - rsync/FTP/wget/... can just share whats already there, not go
get the file in the first place.

BillK

--
William Kenworthy <billk@iinet.net.au>
Home in Perth!
 
Old 05-18-2010, 10:26 AM
Neil Bothwick
 
Default identical drives, different free space!

On Tue, 18 May 2010 18:19:06 +0800, William Kenworthy wrote:

> The advantage of http-replicator is that it is a caching proxy - if it
> isnt in the cache, it downloads it and then serves it out to one or more
> clients - rsync/FTP/wget/... can just share whats already there, not go
> get the file in the first place.

What happens if the proxy is not available, such as when a laptop is away
from home? With NFS, the DISTDIR share simply isn't mounted and files are
downloaded to the local directory, there's no configuration switch needed
when away from home.

Caching the files locally does have its advantages, but for me the only
computer that would benefit from it, my netbook, is the one with the
least storage space to spare.


--
Neil Bothwick

I'm not opinionated, I'm just always right!
 

Thread Tools




All times are GMT. The time now is 11:08 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org