On Tuesday 18 May 2010 11:19:06 William Kenworthy wrote:

> The advantage of http-replicator is that it is a caching proxy - if
> it isnt in the cache, it downloads it and then serves it out to one
> or more clients - rsync/FTP/wget/... can just share whats already
> there, not go get the file in the first place.

My setup does exactly the same, since squid is running on the same box.

--
Rgds
Peter.

On Tue, 2010-05-18 at 11:30 +0100, Peter Humphrey wrote:
> On Tuesday 18 May 2010 11:19:06 William Kenworthy wrote:
>
> > The advantage of http-replicator is that it is a caching proxy - if
> > it isnt in the cache, it downloads it and then serves it out to one
> > or more clients - rsync/FTP/wget/... can just share whats already
> > there, not go get the file in the first place.
>
> My setup does exactly the same, since squid is running on the same box.
>

How have you configured it? - I wouldn't have though squid suitable
considering its designed for a different purpose and so regularly
expires items in its cache (i.e., they will be available for a limited
time before being cleaned.) If you extend max_age, then it becomes
unsuitable as a regular web proxy/cache unless you are running multiple
instances. There are posts saying that squid doesnt work well with
portage but other than a high miss rate (possibly because the files
expired?), no details are given.

Squid also seems to store its files named something
like /var/cache/squid/00/00/000000B9 so its hard to get at them directly
without having squid to serve them up while the http-replicator cache is
just the raw files - same as "distfiles" in fact.

see http://forums.gentoo.org/viewtopic.php?p=1138287#1138287 for details
on http-replicator.

BillK

--
William Kenworthy <billk@iinet.net.au>
Home in Perth!

On Tuesday 18 May 2010 12:59:28 William Kenworthy wrote:

> On Tue, 2010-05-18 at 11:30 +0100, Peter Humphrey wrote:
> > My setup does exactly the same, since squid is running on the same
> > box.
>
> How have you configured it? - I wouldn't have though squid suitable
> considering its designed for a different purpose and so regularly
> expires items in its cache (i.e., they will be available for a
> limited time before being cleaned.) If you extend max_age, then it
> becomes unsuitable as a regular web proxy/cache unless you are
> running multiple instances. There are posts saying that squid
> doesnt work well with portage but other than a high miss rate
> (possibly because the files expired?), no details are given.

In view of what you say, maybe I ought to look into http-replicator. I
have noticed some quite large files being fetched when I thought they
ought already to be in squid's cache.

--
Rgds
Peter.