Hi,

I have two 160Gb drives, one internal and one USB. I've partitioned
them the same and created an identical filesystem on the USB drive for
backing up my internal drive.

I'm using the following rsync command to make the backup:
sudo /usr/bin/ionice -c 3 /usr/bin/rsync -aAx --exclude suspend_file
--delete --delete-excluded --partial
--human-readable / /media/root-backup

however, after running this command sporadically for a few days, the USB
partition is now full, whereas my root partition isn't!

sda is internal, and sdd is external. sda7 is the one I'm interested
in:

$ sudo fdisk -l

Disk /dev/sda: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000080

Device Boot Start End Blocks Id System
/dev/sda1 1 11 88326 6 FAT16
/dev/sda2 * 12 4875 39070080 b W95 FAT32
/dev/sda3 4876 4888 104422+ 83 Linux
/dev/sda4 4889 19457 117025492+ 5 Extended
/dev/sda5 4889 7321 19543041 83 Linux
/dev/sda6 7322 7384 506016 83 Linux
/dev/sda7 7385 19457 96976341 83 Linux

Disk /dev/sdd: 160.0 GB, 160041885696 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x5d5d0036

Device Boot Start End Blocks Id System
/dev/sdd1 1 11 88326 6 FAT16
/dev/sdd2 12 4875 39070080 b W95 FAT32
/dev/sdd3 4876 4888 104422+ 83 Linux
/dev/sdd4 4889 19457 117025492+ 5 Extended
/dev/sdd5 4889 7321 19543041 83 Linux
/dev/sdd6 7322 7384 506016 83 Linux
/dev/sdd7 7385 19457 96976341 83 Linux

I just deleted a bunch of /var/tmp and distfiles to free up some space,
and ran the rsync again. Now it looks like this:

$ df -h
Filesystem Size Used Avail Use% Mounted on
rootfs 92G 81G 6.1G 93% /
/dev/sdd7 92G 89G 4.6M 100% /media/root-backup

/dev/sda3 99M 39M 55M 42% /boot
/dev/sdd3 99M 39M 55M 42% /media/boot-backup

I'm doing the /root backup from cron, but the /boot backup manually when
I make changes.

I thought perhaps the ext3 options were different (ie. different amount
of "reserved" space) but that would make the "Avail" columns different,
and shouldn't make the "Used" columns different.

any thoughts as to why my USB partition is full? thanks,
--
Iain Buchanan <iaindb at netspace dot net dot au>

Most people have two reasons for doing anything -- a good reason, and
the real reason.

Are you doing a full recursive copy of / from rootfs for sdd7 (aka cp -r /) if so, are the other partitions mounted as well?

Cheers

Kad

On Thu, May 13, 2010 at 6:51 PM, Iain Buchanan <iaindb@netspace.net.au> wrote:

Hi,



I have two 160Gb drives, one internal and one USB. *I've partitioned

them the same and created an identical filesystem on the USB drive for

backing up my internal drive.



I'm using the following rsync command to make the backup:

sudo /usr/bin/ionice -c 3 /usr/bin/rsync -aAx --exclude suspend_file

--delete --delete-excluded --partial

--human-readable / /media/root-backup



however, after running this command sporadically for a few days, the USB

partition is now full, whereas my root partition isn't!



sda is internal, and sdd is external. *sda7 is the one I'm interested

in:



$ sudo fdisk -l



Disk /dev/sda: 160.0 GB, 160041885696 bytes

255 heads, 63 sectors/track, 19457 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x00000080



* Device Boot * * *Start * * * * End * * *Blocks * Id *System

/dev/sda1 * * * * * * * 1 * * * * *11 * * * 88326 * *6 *FAT16

/dev/sda2 * * * * * * *12 * * * *4875 * *39070080 * *b *W95 FAT32

/dev/sda3 * * * * * *4876 * * * *4888 * * *104422+ *83 *Linux

/dev/sda4 * * * * * *4889 * * * 19457 * 117025492+ * 5 *Extended

/dev/sda5 * * * * * *4889 * * * *7321 * *19543041 * 83 *Linux

/dev/sda6 * * * * * *7322 * * * *7384 * * *506016 * 83 *Linux

/dev/sda7 * * * * * *7385 * * * 19457 * *96976341 * 83 *Linux



Disk /dev/sdd: 160.0 GB, 160041885696 bytes

255 heads, 63 sectors/track, 19457 cylinders

Units = cylinders of 16065 * 512 = 8225280 bytes

Sector size (logical/physical): 512 bytes / 512 bytes

I/O size (minimum/optimal): 512 bytes / 512 bytes

Disk identifier: 0x5d5d0036



* Device Boot * * *Start * * * * End * * *Blocks * Id *System

/dev/sdd1 * * * * * * * 1 * * * * *11 * * * 88326 * *6 *FAT16

/dev/sdd2 * * * * * * *12 * * * *4875 * *39070080 * *b *W95 FAT32

/dev/sdd3 * * * * * *4876 * * * *4888 * * *104422+ *83 *Linux

/dev/sdd4 * * * * * *4889 * * * 19457 * 117025492+ * 5 *Extended

/dev/sdd5 * * * * * *4889 * * * *7321 * *19543041 * 83 *Linux

/dev/sdd6 * * * * * *7322 * * * *7384 * * *506016 * 83 *Linux

/dev/sdd7 * * * * * *7385 * * * 19457 * *96976341 * 83 *Linux



I just deleted a bunch of /var/tmp and distfiles to free up some space,

and ran the rsync again. *Now it looks like this:



$ df -h

Filesystem * * * * * *Size *Used Avail Use% Mounted on

rootfs * * * * * * * * 92G * 81G *6.1G *93% /

/dev/sdd7 * * * * * * *92G * 89G *4.6M 100% /media/root-backup



/dev/sda3 * * * * * * *99M * 39M * 55M *42% /boot

/dev/sdd3 * * * * * * *99M * 39M * 55M *42% /media/boot-backup



I'm doing the /root backup from cron, but the /boot backup manually when

I make changes.



I thought perhaps the ext3 options were different (ie. different amount

of "reserved" space) but that would make the "Avail" columns different,

and shouldn't make the "Used" columns different.



any thoughts as to why my USB partition is full? *thanks,

--

Iain Buchanan <iaindb at netspace dot net dot au>



Most people have two reasons for doing anything -- a good reason, and

the real reason.

On Thu, 2010-05-13 at 20:22 -0700, Kaddeh wrote:
> Are you doing a full recursive copy of / from rootfs for sdd7 (aka cp
> -r /) if so, are the other partitions mounted as well?

[snip]

yes, but the rsync command "-x" or "--one-file-system" should stop rsync
traversing to different mounts so (I hope) this should only copy the one
partition.

thanks,
--
Iain Buchanan <iaindb at netspace dot net dot au>

When you have eliminated the impossible, whatever remains, however improbable,
must be the truth.
-- Sherlock Holmes, "The Sign of Four"

On Fri, 14 May 2010 11:21:02 +0930, Iain Buchanan wrote:

> I'm using the following rsync command to make the backup:
> sudo /usr/bin/ionice -c 3 /usr/bin/rsync -aAx --exclude suspend_file
> --delete --delete-excluded --partial
> --human-readable / /media/root-backup

As the rsync command is failing with disk full, files are not being
deleted. Try adding --delete-before to the options to have old files
cleaned up before copying new ones.


--
Neil Bothwick

A consultant is a person who borrows your watch, tells you what time it
is, pockets the watch, and sends you a bill for it.

On Fri, 2010-05-14 at 09:35 +0100, Neil Bothwick wrote:
> On Fri, 14 May 2010 11:21:02 +0930, Iain Buchanan wrote:
>
> > I'm using the following rsync command to make the backup:
> > sudo /usr/bin/ionice -c 3 /usr/bin/rsync -aAx --exclude suspend_file
> > --delete --delete-excluded --partial
> > --human-readable / /media/root-backup
>
> As the rsync command is failing with disk full, files are not being
> deleted. Try adding --delete-before to the options to have old files
> cleaned up before copying new ones.

that's what I thought initially, hence:

On Fri, 2010-05-14 at 11:21 +0930, Iain Buchanan wrote:
> I just deleted a bunch of /var/tmp and distfiles to free up some space,
> and ran the rsync again. Now it looks like this:
>
> $ df -h
> Filesystem Size Used Avail Use% Mounted on
> rootfs 92G 81G 6.1G 93% /
> /dev/sdd7 92G 89G 4.6M 100% /media/root-backup
>
> /dev/sda3 99M 39M 55M 42% /boot
> /dev/sdd3 99M 39M 55M 42% /media/boot-backup

So the last rsync didn't fail with "disk full" - it's got about 3G left
for use by root.

Any other ideas? thanks,
--
Iain Buchanan <iaindb at netspace dot net dot au>

Yesterday I was a dog. Today I'm a dog. Tomorrow I'll probably still
be a dog. Sigh! There's so little hope for advancement.
-- Snoopy

On Fri, May 14, 2010 at 7:11 PM, Iain Buchanan <iaindb@netspace.net.au> wrote:


On Fri, 2010-05-14 at 09:35 +0100, Neil Bothwick wrote:

> On Fri, 14 May 2010 11:21:02 +0930, Iain Buchanan wrote:

>

> > I'm using the following rsync command to make the backup:

> > sudo /usr/bin/ionice -c 3 /usr/bin/rsync -aAx --exclude suspend_file

> > --delete --delete-excluded --partial

> > --human-readable / /media/root-backup

>

> As the rsync command is failing with disk full, files are not being

> deleted. Try adding --delete-before to the options to have old files

> cleaned up before copying new ones.



that's what I thought initially, hence:



On Fri, 2010-05-14 at 11:21 +0930, Iain Buchanan wrote:

> I just deleted a bunch of /var/tmp and distfiles to free up some space,

> and ran the rsync again. *Now it looks like this:

>

> $ df -h

> Filesystem * * * * * *Size *Used Avail Use% Mounted on

> rootfs * * * * * * * * 92G * 81G *6.1G *93% /

> /dev/sdd7 * * * * * * *92G * 89G *4.6M 100% /media/root-backup

>

> /dev/sda3 * * * * * * *99M * 39M * 55M *42% /boot

> /dev/sdd3 * * * * * * *99M * 39M * 55M *42% /media/boot-backup



So the last rsync didn't fail with "disk full" - it's got about 3G left

for use by root.



Any other ideas? *thanks,

--

Iain Buchanan <iaindb at netspace dot net dot au>



Yesterday I was a dog. *Today I'm a dog. *Tomorrow I'll probably still

be a dog. Sigh! *There's so little hope for advancement.

* * * * * * * *-- Snoopy





Have you checked to see if it is following symlinks? Possibly add a -l option to "copy symlinks as symlinks"

On Sat, 2010-05-15 at 01:35 -0700, scott n-h wrote:

>
> Have you checked to see if it is following symlinks? Possibly add a -l
> option to "copy symlinks as symlinks"

good idea, I didn't have the -l option. Now I run rsync like this:

sudo /usr/bin/ionice -c 3 /usr/bin/rsync -aAlx --exclude suspend_file
--delete --delete-excluded --delete-before --partial --human-readable /
"${MOUNTPT}" >>"${LOGFILE}"

Note the "-l" AND "--delete-before".

However I'm STILL filling up the second drive for some unknown reason.

I've added "--exclude /usr/portage/distfiles" to the rsync options,
since there's no need to back up my distfiles, but I'd like to know why
it's not working...

--
Iain Buchanan <iain at pcorp dot com dot au>

It doesn't matter whether you win or lose -- until you lose.

So after I excluded distfiles from my rsync, I found that the two
partitions had roughly the same free space... strange! How could
excluding around 6G of distfiles make two copies of the same thing the
same size?

Well, it turns out I have the distfiles mounted with --bind to my
ftp/pub directory. And looking in the rsync man page:

-x, --one-file-system
...
Also keep in mind that rsync treats a "bind" mount to
the same device as being on the same filesystem.

So my distfiles were being copied in /usr/portage as well
as /home/ftp/pub!

Unfortunately the only way to get around it seems to be another
--exclude directive. At least I understand what's going on now

thanks for all the suggestions,
--
Iain Buchanan <iaindb at netspace dot net dot au>

Mr. Cole's Axiom:
The sum of the intelligence on the planet is a constant; the
population is growing.

On Mon, 17 May 2010 11:21:50 +0930, Iain Buchanan wrote:

> Well, it turns out I have the distfiles mounted with --bind to my
> ftp/pub directory. And looking in the rsync man page:

Why not set $DISTDIR to the true location of distfiles instead of using
bind mounts?


--
Neil Bothwick

Tribble math: * + * = ***********************************

On Mon, 17 May 2010 10:10:02 +0200, Neil Bothwick wrote about Re:
[gentoo-user] [SOLVED] identical drives, different free space!:

>On Mon, 17 May 2010 11:21:50 +0930, Iain Buchanan wrote:
>
>> Well, it turns out I have the distfiles mounted with --bind to my
>> ftp/pub directory. And looking in the rsync man page:
>
>Why not set $DISTDIR to the true location of distfiles instead of using
>bind mounts?

Because binding the directory to /home/ftp/pub makes the distfiles
available to the rest of one's network via anonymous ftp. I do the
same thing here, without the "pub" subdirectory, and exclude /home/ftp/
from my backups.
--
Regards,

Dave [RLU #314465]
================================================== ====================
dwnoon@ntlworld.com (David W Noon)
================================================== ====================