2008/12/31 stan <stanb@panix.com>
Given a freshly restored machine that has been out of service for a good
while, containing a large collection of data files "A", and "B" a in
service machine thta has a subset of these data files, but has some newer
files that are ont on "A". How can I invoke rsync such that only the new
filee are trnasfered from "B" to "A"?
I am confused by the large number of options to this program.Type man rsync and look for the -u option. It should make sure only older files are replaced with newer files.In my own rsync script, with which I backup my internal drive to an external one, I use do something like:
rsync -auv "${INTERNAL_DRIVE}/MyFolder" "${EXTERNAL_DRIVE}/" >> "${MY_LOGFILE}"The -v is for "verbose" and the text that is produced is, in this example, redirected to "${MY_LOGFILE}", which I set to whatever file I want to add the text to.
J.R.*
--
One of the main causes of the fall of the roman empire was that, lacking
zero, they had no way to indicate successful termination of their C
programs.
--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
11-10-2010, 10:06 AM
Helmut Jarausch
rsync question
Hi,
the following behaviour of rsync puzzles me.
I have the following situation
<Source>/A this is a symlink !
<Dest>/A this is a real directory
now
cd <Dest>/A
rsync -auHz -ni --rsh=ssh --delete --exclude='/A/' <source
machine>:<Source>/ .
shows that it's going to delete the folder <Dest>/A
Can anybody please explain me why?
Thanks a lot,
Helmut.
(P.S. : I did google for it but couldn't find an explanation)
11-11-2010, 12:06 PM
Neal Hogan
rsync question
On Wed, Nov 10, 2010 at 5:06 AM, Helmut Jarausch
<jarausch@igpm.rwth-aachen.de> wrote:
> Hi,
>
> the following behaviour of rsync puzzles me.
>
> I have the following situation
>
> <Source>/A * * * * * * * *this is a symlink !
>
> <Dest>/A * * * * * * * * *this is a real directory
>
>
> now
> cd <Dest>/A
> rsync -auHz -ni --rsh=ssh --delete --exclude='/A/' <source
> machine>:<Source>/ .
>
> shows that it's going to delete the folder <Dest>/A
>
> Can anybody please explain me why?
I'm not clear on some of your details, but perhaps you could try
putting full paths to the dirs/files in your script, including the
destination dir. Also, remember that '/A' and '/A/' are not the same
wrt the source.
g'luck
07-31-2012, 05:05 AM
"Smithies, Russell"
rsync question
I'm trying to rsync a 8TB data folder containing squillions of small files and it's taking forever (i.e. weeks) to get anywhere.
I'm assuming the slow bit is check-summing everything with a single CPU (even though it's on a 12-core server ;-( )
Is it possible to do something simple like scp the whole dir in one go so they're duplicates in the first instance, then get rsync to just keep them in sync without an initial transfer?
Or is there a better way?
Thanx,
Russell Smithies
Infrastructure Technician
T 03 489 9085
M 027 4734 600
E russell.smithies@agresearch.co.nz
Invermay Agricultural Centre
Puddle Alley, Private Bag 50034, Mosgiel 9053, New Zealand
T +64 3 489 3809 F +64 3 489 3739 www.agresearch.co.nz<http://www.agresearch.co.nz/>
================================================== =====================
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
================================================== =====================
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
07-31-2012, 05:16 AM
John R Pierce
rsync question
On 07/30/12 10:05 PM, Smithies, Russell wrote:
> I'm trying to rsync a 8TB data folder containing squillions of small files and it's taking forever (i.e. weeks) to get anywhere.
> I'm assuming the slow bit is check-summing everything with a single CPU (even though it's on a 12-core server ;-( )
> Is it possible to do something simple like scp the whole dir in one go so they're duplicates in the first instance, then get rsync to just keep them in sync without an initial transfer?
use the rsync mode that goes off file timestamp and size. the
checksuming block algorithm is only useful on large files that get small
random block changes.
--
john r pierce N 37, W 122
santa cruz ca mid-left coast
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
07-31-2012, 06:44 AM
Mogens Kjaer
rsync question
On 07/31/2012 07:05 AM, Smithies, Russell wrote:
> Is it possible to do something simple like scp the whole dir in one go so they're duplicates in the first instance, then get rsync to just keep them in sync without an initial transfer?
>
> Or is there a better way?
I use tar and ttcp for an initial transfer:
On the receiving end:
ttcp -l5120 -r | tar xf -
On the transmitter:
tar cf - . | ttcp -l5120 -t name-of-receiver
Note: The files are transmitted without encryption.
I easily get 110 Mbytes/sec. on a gigabit connection.
If you need encryption, and your transfer is CPU limited, you should
investigate which cipher to use. In my case arcfour128 is the fastest,
so I use:
As far as I can see timestamp and size is the default.
I've turned off compression and I think I'm getting better throughput.
Running 4 rsync tasks and getting sustained transfers for several hours of just over 800Mb/sec :- )
--Russell
-----Original Message-----
From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of John R Pierce
Sent: Tuesday, 31 July 2012 5:16 p.m.
To: centos@centos.org
Subject: Re: [CentOS] rsync question
On 07/30/12 10:05 PM, Smithies, Russell wrote:
> I'm trying to rsync a 8TB data folder containing squillions of small files and it's taking forever (i.e. weeks) to get anywhere.
> I'm assuming the slow bit is check-summing everything with a single
> CPU (even though it's on a 12-core server ;-( ) Is it possible to do something simple like scp the whole dir in one go so they're duplicates in the first instance, then get rsync to just keep them in sync without an initial transfer?
use the rsync mode that goes off file timestamp and size. the checksuming block algorithm is only useful on large files that get small random block changes.
--
john r pierce N 37, W 122
santa cruz ca mid-left coast
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
================================================== =====================
Attention: The information contained in this message and/or attachments
from AgResearch Limited is intended only for the persons or entities
to which it is addressed and may contain confidential and/or privileged
material. Any review, retransmission, dissemination or other use of, or
taking of any action in reliance upon, this information by persons or
entities other than the intended recipients is prohibited by AgResearch
Limited. If you have received this message in error, please notify the
sender immediately.
================================================== =====================
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
rsync question
