FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu User

 
 
LinkBack Thread Tools
 
Old 02-18-2009, 06:32 AM
Robert Holtzman
 
Default rsync backup size

I run rsync with the command:

sudo rsync -vaHz --exclude '/proc' --exclude '/sys' --exclude '/media' /
/media/disk

to back up to a usb hard drive that already had a full backup on it.

According to the man page, rsync uses a "delta-transfer algorithm, which
reduces the amount of data sent over the network by sending only the
differences between the source files and the existing files in the
destination." This appears to happen with no special command option. The
summary at the end of the process is:

sent 134629383 bytes received 30327 bytes 2225780.33 bytes/sec
total size is 5635940395 speedup is 41.85

The total size agrees with the output from "du -ach /". The questions
that arise are

1. Does the total size in the summary relate to the source or the
target? If it's the target, it means there was no compression.

2. Running "du -ach /media/disk" showed ~8 Gb (the backup is the only
thing on the disk). Running "du -ach /" (with the usb disk removed)
showed ~6 Gb. Why is the backup larger than the system, especially
since I invoked the z option and was careful to exclude /media?

--
Bob Holtzman
"If you think you're getting free lunch,
check the price of the beer"

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-18-2009, 06:49 AM
Rashkae
 
Default rsync backup size

Robert Holtzman wrote:
> I run rsync with the command:
>
> sudo rsync -vaHz --exclude '/proc' --exclude '/sys' --exclude '/media' /
> /media/disk
>
> to back up to a usb hard drive that already had a full backup on it.
>
> According to the man page, rsync uses a "delta-transfer algorithm, which
> reduces the amount of data sent over the network by sending only the
> differences between the source files and the existing files in the
> destination." This appears to happen with no special command option. The
> summary at the end of the process is:
>
> sent 134629383 bytes received 30327 bytes 2225780.33 bytes/sec
> total size is 5635940395 speedup is 41.85
>

> The total size agrees with the output from "du -ach /". The questions
> that arise are
>
> 1. Does the total size in the summary relate to the source or the
> target? If it's the target, it means there was no compression.


The total size is of the source. That's your 6GB (after you add file
system overhead and files in the excluded dirs)


>
> 2. Running "du -ach /media/disk" showed ~8 Gb (the backup is the only
> thing on the disk). Running "du -ach /" (with the usb disk removed)
> showed ~6 Gb. Why is the backup larger than the system, especially
> since I invoked the z option and was careful to exclude /media?
>

You said the target already contained a full backup. You need to use
the --delete option to prune files from the existing backup that are no
longer on the source

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-18-2009, 09:20 AM
Owen Townend
 
Default rsync backup size

2009/2/18 Rashkae <ubuntu@tigershaunt.com>:
> Robert Holtzman wrote:
>> I run rsync with the command:
>>
>> sudo rsync -vaHz --exclude '/proc' --exclude '/sys' --exclude '/media' /
>> /media/disk
>>
>> to back up to a usb hard drive that already had a full backup on it.
>>
>> According to the man page, rsync uses a "delta-transfer algorithm, which
>> reduces the amount of data sent over the network by sending only the
>> differences between the source files and the existing files in the
>> destination." This appears to happen with no special command option. The
>> summary at the end of the process is:
>>
>> sent 134629383 bytes received 30327 bytes 2225780.33 bytes/sec
>> total size is 5635940395 speedup is 41.85
>>
>
>> The total size agrees with the output from "du -ach /". The questions
>> that arise are
>>
>> 1. Does the total size in the summary relate to the source or the
>> target? If it's the target, it means there was no compression.
[snip]

The man page may not be entirely unambiguous about this. The
compression is only for the transmission itself, the data doesn't stay
compressed.
This is for when cpu time is less limited than bandwidth which is not
the case here as the transfer is internal. Rsync is smart enough to
realise this and so there was no compression done - it would only
waste cpu for no benefit.
If you're trying to have the backup compressed you'll need to use a
compression tool such as gzip or bzip2. Gzip for example has an
--rsyncable flag to help with rsync's delta algorithm.

cheers,
Owen.

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-18-2009, 10:20 AM
Hal Burgiss
 
Default rsync backup size

On Wed, Feb 18, 2009 at 12:32:01AM -0700, Robert Holtzman wrote:
>
> 2. Running "du -ach /media/disk" showed ~8 Gb (the backup is the only
> thing on the disk). Running "du -ach /" (with the usb disk removed)
> showed ~6 Gb. Why is the backup larger than the system, especially
> since I invoked the z option and was careful to exclude /media?

If the filesystems are not the same on both end, there may be
different block sizes which can cause a "waste of space", especially
if there are a lot of small files. This can cause discrepancies.

--
Hal


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-18-2009, 12:46 PM
Smoot Carl-Mitchell
 
Default rsync backup size

On Wed, 2009-02-18 at 00:32 -0700, Robert Holtzman wrote:
> I run rsync with the command:
>
> sudo rsync -vaHz --exclude '/proc' --exclude '/sys' --exclude '/media' /
> /media/disk
>

> 2. Running "du -ach /media/disk" showed ~8 Gb (the backup is the only
> thing on the disk). Running "du -ach /" (with the usb disk removed)
> showed ~6 Gb. Why is the backup larger than the system, especially
> since I invoked the z option and was careful to exclude /media?

You may have some "sparse" (e.g. files with some blocks which are all
zero bytes) files on the source filesystem. You might try the -S flag
which will handle sparse files more efficiently. This should save space
unless the destination filesystem does not support sparse files.
--
Smoot Carl-Mitchell
Computer Systems and
Network Consultant
smoot@tic.com
+1 480 922 7313
cell: +1 602 421 9005

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-18-2009, 05:08 PM
"H.S."
 
Default rsync backup size

Robert Holtzman wrote:
> I run rsync with the command:
>
> sudo rsync -vaHz --exclude '/proc' --exclude '/sys' --exclude '/media' /
> /media/disk
>

The z flag only compressed the data during transmission and not save the
data as compressed.

Try --delete option. It will delete the files in your pre-existing
backup which are not present anymore in the data you are trying to backup.

I usually use the option "-auv --progress --delete" to keep a back up in
a similar scenario. The "-C" may also be useful, it prevents standard
backup files and object files to be copied.

Regards.
--

Please reply to this list only. I read this list on its corresponding
newsgroup on gmane.org. Replies sent to my email address are just
filtered to a folder in my mailbox and get periodically deleted without
ever having been read.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-19-2009, 05:44 AM
Robert Holtzman
 
Default rsync backup size

Thanks to all who replied. You answered my questions and taught me a few
things.

Thanks again.

--
Bob Holtzman
"Bother", said Pooh, as he chambered another round...

--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 

Thread Tools




All times are GMT. The time now is 04:31 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org