reverse proxy with directory replication capability
 

I've a web server (server1) with 30G data footprint - i used to rsync
the directory to a rather frail (but expensive) VM, but the sheer no.
of files is pushing rsync out of memory on the VM.

ever since rsync failed on me, i run squid as reverse proxy on the VM
against server1 - it works "ok", but let me ask this anyway:

i am looking for a reverse proxy or a similar method, which would
replicate the directory structure *exactly as* the backend as it
serves clients, populate the structure as it fetches objects from the
backend. if the object requested is in its cache, serves it locally
(no IMS queries to server1), if not, it fetches from the backend and
also populates the directory structure. if the backend is dead, it
serves what it can from its own directory structure, and returns 404s
for stuff that it does not have and failed to ask backend for.

i am hoping this ask makes sense.

many thanks in advance for your advice..!


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: AANLkTin-LFJtxYMShEd_KR3618SFozxf9KQQqqOyNmkh@mail.gmail.co m">http://lists.debian.org/AANLkTin-LFJtxYMShEd_KR3618SFozxf9KQQqqOyNmkh@mail.gmail.co m

reverse proxy with directory replication capability
 

nevermind. i figured this out, proxy_store in nginx does exactly this.
go nginx!

location /images/ {
root /data/www;
error_page 404 = /fetch$uri;
}

location /fetch {
internal;
proxy_pass http://backend;
proxy_store on;
proxy_store_access user:rw group:rw all:r;
proxy_temp_path /data/temp;
alias /data/www;
}

http://wiki.nginx.org/NginxHttpProxyModule#proxy_store

2010/9/21 Evuraan <evuraan@gmail.com>:
> I've a web server (server1) with 30G data footprint - i used to rsync
> the directory to a rather frail (but expensive) VM, but the sheer no.
> of files is pushing rsync out of memory on the VM.
>
> ever since rsync failed on me, i *run squid as reverse proxy on the VM
> against server1 - it works "ok", but let me ask this anyway:
>
> i am looking for a reverse proxy or a similar method, which would
> replicate the directory structure *exactly as* the backend as it
> serves clients, populate the structure as it fetches objects from the
> backend. if the object requested is in its cache, serves it locally
> (no IMS queries to server1), if not, it fetches from the backend and
> also populates the directory structure. if the backend is dead, it
> serves what it can from its own directory structure, and returns 404s
> for stuff that it does not have and failed to ask backend for.
>
> i am hoping this ask makes sense.
>
> many thanks in advance for your advice..!
>


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: http://lists.debian.org/AANLkTinSrk0JpHhuitZHQNVLtAUqvqEVbOmSdSr5k@mail.g mail.com