Linux Archive

Linux Archive (
-   CentOS Development (
-   -   rsync speed (was CentOS 6 status update) (

Charles Polisher 06-02-2011 02:15 PM

rsync speed (was CentOS 6 status update)
Karanbir Singh wrote:
> Hi,
> Andrzej SzymaƄski wrote:
> > I don't know whether you use (and can use) the following hack to speed
> > up rsyncing the iso files:
> >
> > - you need to have the full (or mostly complete) package tree on the target
> We do some sanity tests that span the buildsys, the intermediary host
> and the final rsync targets. So its important that the first set of
> contents is sync'd out from the buildsys as-is. Perhaps extra caution,
> but we have had problems in the past ( even with rsync saying two files
> are identical when they were clearly not ).
> On the other hand, the ISOS can be fairly easily re-created from the
> tree, so if the rpms tree are is already in place that would be easier
> than going down the tar route.

I found out the hard way in a large production setting
about rsync parameters that were "new to me":

(Don't throttle transfers that run "too quickly")

(don't use the delta-xfer algorithm,
the whole file needs replacing)

(_might_ be useful, you'd have to test
if it helps on your system)

and apropos of well-defined behavior, you might have


This option puts the temporary file from each
updated file into a holding directory until the
end of the transfer, at which time all the files
are renamed into place in rapid succession. This
attempts to make the updating of the files a
little more atomic.
Charles Polisher

CentOS-devel mailing list

Karanbir Singh 06-02-2011 02:27 PM

rsync speed (was CentOS 6 status update)
On 06/02/2011 03:15 PM, Charles Polisher wrote:
> and apropos of well-defined behavior, you might have
> overlooked:
> --delay-updates

the msync machines all run with this option set, since it can take a
while to get 20 odd Gigs per release into place across 90 odd machines :D

Thanks for the tips and helpful feedback.

- KB
CentOS-devel mailing list

All times are GMT. The time now is 10:17 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.