FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 02-07-2012, 03:30 PM
Les Mikesell
 
Default schily tools

On Tue, Feb 7, 2012 at 9:26 AM, Joerg Schilling
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
>
> When I implemented incremental restores for star in September 2004, I wrote a
> simple script for a incremental testcase and tested the deltas with
> ufsdump/ufsrestore, gtar and the star version at that time. Gtar was unable to
> deal with my testcase, so I stopped testing it any further.

My testcase for star was moving a subdirectory of what my backup runs
covered onto a mounted volume. Star failed and I stopped testing it
any further.

> If you like to discuss incrementals, you definitely need to discuss behavior at
> restore time and restoring incrementals definitely does not work correclty with
> gtar if you renamed directories.

If that is the issue I recall, you can recover without losing data.

> Star is used to do incremental backups/restores on a dayly base in Berlios
> since September 2004. Since Spring 2005, not a single problem was seen, so
> there are more that 2500 successful incremental restores that verify no problem
> even under stange conditions.

So it all that time you have not mounted a new volume somewhere? No
remote backups of hosts where someone else might add space? No one
ever wanted to restore onto a different mount topology than the one
where the backups were taken? Being able to do those things is the
reason I use tar instead of filesystem-dependent dump.

> This problem is not caused by compression or not, it is a general gtar bug that
> I reported in 1993 already. Nobody knows why it hits and the structure of the
> gtar sources makes it really hard to debug this problem. The FSF was interested
> to throw aywa gtar and replace it by star 10 years ago for this kind of
> problems in gtar.

So, you have a repeatable test case for this and no one has looked
into it? That's surprising considering the number of people who have
contributed to gnutar. And what drove the decision not to adopt
star?

> The following other problems are known with gtar:
>
> - * * * gtar is with aprox. 5% probability unable to read it's own
> * * * *continuation archives from multi volume archives. This cannot be fixed
> * * * *as it is caused by the concept used for multi volume archives in gtar.

I assume you mean the version where you let the tape drive hit the
end, not where you tell it the length. Does star always work in that
scenario?

> - * * * gtar has much less features than star

Unless you would like it to do incrementals properly across mount
points... And I thought there was another reason regarding features
that amanda used gtar as well - maybe it was the ability to quickly
estimate sizes of incrementals. If it had worked for amanda, I would
probably have been using it for ages. When I looked at it, it
couldn't because of missing features. These days I mostly use
backuppc with rsync as the transport since online access is so much
nicer than tapes and rsync obviously excels at detecting differences
in incrementals, but I suppose there is still a place for archives.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-07-2012, 04:50 PM
 
Default schily tools

Dennis Clarke <dclarke@blastwave.org> wrote:

> >> I don't think there is any such general consensus. Are you reading
> >> something that favors Solaris/*bsd over GNU based systems?
> >
> > Schily tools (and in special star) implement support for Linux specific
> > extensions. This is what you do not get from gtar at all. So why do Linux
> > distros prefer gtar even though there is no Linux support?
>
> Is there any correlation between this and the warning message I see during a
> bootstrap like so :
>
> Warning: *** /usr/src/linux/include contains broken include files ***
> Warning: *** /usr/src/linux/include is not used this reason ***
> Warning: This may result in the inability to use recent Linux kernel
> interfaces
>
> Warning: *** linux/ext2_fs.h is not usable at all ***
> Warning: *** This makes it impossible to support Linux file flags ***
> You may try to compile using 'make COPTX=-DTRY_EXT2_FS'

This is indeed one of the Linux specific issues. The Linux kernel guys create
defective include files. In other words:

The linux kernel include files that are needed in order to access certain
features (in this case the ext* extended file flags) cause a C compiler error
in case they are used from a userland program.

Jörg

--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni)
joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-07-2012, 05:03 PM
 
Default schily tools

Les Mikesell <lesmikesell@gmail.com> wrote:

> On Tue, Feb 7, 2012 at 9:26 AM, Joerg Schilling
> <Joerg.Schilling@fokus.fraunhofer.de> wrote:
> >
> > When I implemented incremental restores for star in September 2004, I wrote a
> > simple script for a incremental testcase and tested the deltas with
> > ufsdump/ufsrestore, gtar and the star version at that time. Gtar was unable to
> > deal with my testcase, so I stopped testing it any further.
>
> My testcase for star was moving a subdirectory of what my backup runs
> covered onto a mounted volume. Star failed and I stopped testing it
> any further.

So you tried to do something that cannot work for a filesystem oriented program.
This is not a problem from star but you just did not use star the right way.

Note that gtar uses a big database that is needed at the dump side and that
still does not hold enough data to let gtar work correctly.

Star on the other side just needs to remember dump dates and creates a full
data base et extract side when doing incremental restores. This is also more
reliable than what gtar does as all needed information for the restore is in
the archives.


> > If you like to discuss incrementals, you definitely need to discuss behavior at
> > restore time and restoring incrementals definitely does not work correclty with
> > gtar if you renamed directories.
>
> If that is the issue I recall, you can recover without losing data.

You are correct, you could in theory recover the data but this must be done
manually.

> > Star is used to do incremental backups/restores on a dayly base in Berlios
> > since September 2004. Since Spring 2005, not a single problem was seen, so
> > there are more that 2500 successful incremental restores that verify no problem
> > even under stange conditions.
>
> So it all that time you have not mounted a new volume somewhere? No
> remote backups of hosts where someone else might add space? No one
> ever wanted to restore onto a different mount topology than the one
> where the backups were taken? Being able to do those things is the
> reason I use tar instead of filesystem-dependent dump.

Star of course can do what you like. You just need to create more than one
backup once you split filesystems. It would be easy to handle if you like to
use it....


> > This problem is not caused by compression or not, it is a general gtar bug that
> > I reported in 1993 already. Nobody knows why it hits and the structure of the
> > gtar sources makes it really hard to debug this problem. The FSF was interested
> > to throw aywa gtar and replace it by star 10 years ago for this kind of
> > problems in gtar.
>
> So, you have a repeatable test case for this and no one has looked
> into it? That's surprising considering the number of people who have
> contributed to gnutar. And what drove the decision not to adopt
> star?

I had a repeatable case in 1993, but at that time, I send them an archive
created by star. In any case, there is more than one gtar user who would be
able and willing to provide gtar archives that trigger that case.

The reason for not adopting star was that RMS did send me a contract that was
illegal in Europe and RMS was unwilling to convert his contract into something
that I could legally sign.

BTW: there have been two attempts to replace gtar by star and both ended the
same way.

> > The following other problems are known with gtar:
> >
> > - * * * gtar is with aprox. 5% probability unable to read it's own
> > * * * *continuation archives from multi volume archives. This cannot be fixed
> > * * * *as it is caused by the concept used for multi volume archives in gtar.
>
> I assume you mean the version where you let the tape drive hit the
> end, not where you tell it the length. Does star always work in that
> scenario?

As far as I can tell, yes. Star uses a completely different method to verify
followup volumes and holds diffent sets of data that cannot cause this kind of
failure. Gtar fails when it splits an archive at a location that is inside the
(probably extended) tar header.

BTW: As star intentionally does not implement the "verification method" from
gtar, star is able to restore such multi volume archives.


> > - * * * gtar has much less features than star
>
> Unless you would like it to do incrementals properly across mount
> points... And I thought there was another reason regarding features
> that amanda used gtar as well - maybe it was the ability to quickly
> estimate sizes of incrementals. If it had worked for amanda, I would
> probably have been using it for ages. When I looked at it, it
> couldn't because of missing features. These days I mostly use
> backuppc with rsync as the transport since online access is so much
> nicer than tapes and rsync obviously excels at detecting differences
> in incrementals, but I suppose there is still a place for archives.

AFAIK, amanda has too few features or there are no people who are willing to
put efforts in adopting to star.

I currently cannot believe that there is really any important feature that is
missing in star.

Jörg

--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni)
joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-07-2012, 05:58 PM
Les Mikesell
 
Default schily tools

On Tue, Feb 7, 2012 at 12:03 PM, Joerg Schilling
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
>>> >
>> > When I implemented incremental restores for star in September 2004, I wrote a
>> > simple script for a incremental testcase and tested the deltas with
>> > ufsdump/ufsrestore, gtar and the star version at that time. Gtar was unable to
>> > deal with my testcase, so I stopped testing it any further.
>>
>> My testcase for star was moving a subdirectory of what my backup runs
>> covered onto a mounted volume. *Star failed and I stopped testing it
>> any further.
>
> So you tried to do something that cannot work for a filesystem oriented program.

It does work with GNUtar.

> This is not a problem from star but you just did not use star the right way.

No, star does not do what I need, so I don't use it at all.

> Note that gtar uses a big database that is needed at the dump side and that
> still does not hold enough data to let gtar work correctly.

How is it not correct?

> Star on the other side just needs to remember dump dates and creates a full
> data base et extract side when doing incremental restores.

So what happens when you don't exactly match the source mount tree
configuration when you try to restore? You complain about GNUtar not
working in some special case - how is that worse than star not working
in the very common case of moving some mount points around? I do
that all the time. I don't think I've ever done the weird sequence
that causes trouble with a gnutar incremental restore.

> This is also more
> reliable than what gtar does as all needed information for the restore is in
> the archives.

Except when it isn't, because it is tied to the filesystem, not the
directory tree structure. If I wanted filesystem dependencies I'd use
dump. I expect tar to follow directory trees and be agnostic to mount
points.

>> > If you like to discuss incrementals, you definitely need to discuss behavior at
>> > restore time and restoring incrementals definitely does not work correclty with
>> > gtar if you renamed directories.
>>
>> If that is the issue I recall, you can recover without losing data.
>
> You are correct, you could in theory recover the data but this must be done
> manually.

How do you recover from the mount point change case for star? If it
happens on the source side, do you lose data?

>> So it all that time you have not mounted a new volume somewhere? *No
>> remote backups of hosts where someone else might add space? *No one
>> ever wanted to restore onto a different mount topology than the one
>> where the backups were taken? * Being able to do those things is the
>> reason I use tar instead of filesystem-dependent dump.
>
> Star of course can do what you like. You just need to create more than one
> backup once you split filesystems. It would be easy to handle if you like to
> use it....

I handled it by pointing amanda at remote systems. And I expected it
to keep those systems backed up even if someone else mounted some new
disks in places where they needed some more space. I don't see how
star fits into that scheme.

> The reason for not adopting star was that RMS did send me a contract that was
> illegal in Europe and RMS was unwilling to convert his contract into something
> that I could legally sign.

Well, no one has ever accused RMS of being reasonable...

>> And I thought there was another reason regarding *features
>> that amanda used gtar as well - maybe it was the ability to quickly
>> estimate sizes of incrementals. *If it had worked for amanda, I would
>> probably have been using it for ages. *When I looked at it, it
>> couldn't because of missing features. *These days I mostly use
>> backuppc with rsync as the transport since online access is so much
>> nicer than tapes and rsync obviously excels at detecting differences
>> in incrementals, but I suppose there is still a place for archives.
>
> AFAIK, amanda has too few features or there are no people who are willing to
> put efforts in adopting to star.

Star simply does not do what amanda needs - or did not the last time I
looked. Amanda needs a way to quickly estimate the size of a run for
its brilliant feature of automatically balancing the mix of fulls and
incrementals to ensure that you get at least an incremental of every
target every night plus a full within the number of tapes in your
rotation. And it can't fail on incrementals just because someone
replaced a directory on a remote machine with a mount point.

> I currently cannot believe that there is really any important feature that is
> missing in star.

Those features obviously aren't important to you, but they are enough
to keep me - or any amanda user - from considering star. And amanda
made things from several machines fit on my one non-changer tape drive
every night for more than a decade with nothing on my part except
swapping the tape sometime during the day (and handled things
gracefully if I forgot). I don't think there was any alternative that
could have worked as well.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-08-2012, 11:20 AM
 
Default schily tools

Les Mikesell <lesmikesell@gmail.com> wrote:

> >> My testcase for star was moving a subdirectory of what my backup runs
> >> covered onto a mounted volume. *Star failed and I stopped testing it
> >> any further.
> >
> > So you tried to do something that cannot work for a filesystem oriented program.
>
> It does work with GNUtar.

In general, this does not work with gtar. You are exactly in the area that
caused my conclusion that gtar is not useful at all for incremental backups.


> > This is not a problem from star but you just did not use star the right way.
>
> No, star does not do what I need, so I don't use it at all.

Star does what you need, you are just using it the wrong way.


> > Note that gtar uses a big database that is needed at the dump side and that
> > still does not hold enough data to let gtar work correctly.
>
> How is it not correct?

As mentioned, gtar has major problems whe directory is envolved. How many
successful restores did you pass?


> > Star on the other side just needs to remember dump dates and creates a full
> > data base et extract side when doing incremental restores.
>
> So what happens when you don't exactly match the source mount tree
> configuration when you try to restore? You complain about GNUtar not
> working in some special case - how is that worse than star not working
> in the very common case of moving some mount points around? I do
> that all the time. I don't think I've ever done the weird sequence
> that causes trouble with a gnutar incremental restore.

The difference between gtar and star is that gtar never works correctly for all
possible changes, while star works correct and you just need to follow it's
constraints. As gtar does not work on a simple testcase, I am not willing to
trust in gtar's incrementals.

> > This is also more
> > reliable than what gtar does as all needed information for the restore is in
> > the archives.
>
> Except when it isn't, because it is tied to the filesystem, not the
> directory tree structure. If I wanted filesystem dependencies I'd use
> dump. I expect tar to follow directory trees and be agnostic to mount
> points.

It may be that you miss the fact that modern filesystems like e.g. ZFS may be
able to move a directory tree into a "new filesystem" without affecting ctime.

If gtar would be implemented correctly, it would need to maintain a huge
database for the lifetime of the filesystems tobackup.

Star on the other side only needs to remember the last backup dates for every
different incremental level.

You also seem to miss the point that in case you like to move a sub-tree into a
different filesystem (something that is extremely unlikely to happen with modern
filesystems), you need to do a lot of adminstrative work anyway. Adding a new
FS to the list of FSs in the backup system is a minor task in this context.

What star allows you to do is to have incrementals that are as reliable as
ufsdump but that are independent from the OS and the FS.


> > You are correct, you could in theory recover the data but this must be done
> > manually.
>
> How do you recover from the mount point change case for star? If it
> happens on the source side, do you lose data?

Star handles directory changes correctly, there is no loss of data.

If you remove a sub-tree on the source side, this is also done at incremental
restore time.

> >> So it all that time you have not mounted a new volume somewhere? *No
> >> remote backups of hosts where someone else might add space? *No one
> >> ever wanted to restore onto a different mount topology than the one
> >> where the backups were taken? * Being able to do those things is the
> >> reason I use tar instead of filesystem-dependent dump.
> >
> > Star of course can do what you like. You just need to create more than one
> > backup once you split filesystems. It would be easy to handle if you like to
> > use it....
>
> I handled it by pointing amanda at remote systems. And I expected it
> to keep those systems backed up even if someone else mounted some new
> disks in places where they needed some more space. I don't see how
> star fits into that scheme.

Well as mentioned before, if you reconfigure your server, you also need to
reconfigure the backup to match the new situation at the server.

> > AFAIK, amanda has too few features or there are no people who are willing to
> > put efforts in adopting to star.
>
> Star simply does not do what amanda needs - or did not the last time I
> looked. Amanda needs a way to quickly estimate the size of a run for
> its brilliant feature of automatically balancing the mix of fulls and
> incrementals to ensure that you get at least an incremental of every
> target every night plus a full within the number of tapes in your
> rotation. And it can't fail on incrementals just because someone
> replaced a directory on a remote machine with a mount point.

This is an unproven claim from you. I grant you that star supports all you need
for a reliable incremental backup and restore.

> > I currently cannot believe that there is really any important feature that is
> > missing in star.
>
> Those features obviously aren't important to you, but they are enough
> to keep me - or any amanda user - from considering star. And amanda

If amanda does not support star, it seems that the amanda people are not
interested in reliable backups.

Star supports everything you need, if Amanda does not support star, this is
obviously a filure at amanda's side.

Jörg

--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni)
joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-08-2012, 06:10 PM
Les Mikesell
 
Default schily tools

On Wed, Feb 8, 2012 at 6:20 AM, Joerg Schilling <
Joerg.Schilling@fokus.fraunhofer.de> wrote:

> Les Mikesell <lesmikesell@gmail.com> wrote:
>
>
(For bystanders following this conversion, it is only about the
non-standard extensions (--listed-incremental, etc.) to tar. Standard
modes are fine on either gnutar or star).


> >> My testcase for star was moving a subdirectory of what my backup runs
> > >> covered onto a mounted volume. Star failed and I stopped testing it
> > >> any further.
> > >
> > > So you tried to do something that cannot work for a filesystem
> oriented program.
> >
> > It does work with GNUtar.
>
> In general, this does not work with gtar. You are exactly in the area that
> caused my conclusion that gtar is not useful at all for incremental
> backups.
>
>
No, a general file-oriented case would handle FAT and NTFS filesystems, and
continue to work even if you mount one of those in the path of your planned
backup or restore. Star fails that.



> > > This is not a problem from star but you just did not use star the
> right way.
> >
> > No, star does not do what I need, so I don't use it at all.
>
> Star does what you need, you are just using it the wrong way.
>
>
I don't plan to arrange my life or my data around star's restricted
capabilities.

> > Note that gtar uses a big database that is needed at the dump side and
that
> > still does not hold enough data to let gtar work correctly.
>
> How is it not correct?

As mentioned, gtar has major problems whe directory is envolved. How many
> successful restores did you pass?
>

All of them. Even the contrived case you sent me when we had this same
conversation years ago. The saves always work and the only issue there
has ever been is when a restored directory is supposed to be deleted and
replaced by a file of the same name as an incremental update (or maybe the
other way around...). It's something that has never happened in years of
operations and recoverable if it does - just delete the offending item and
repeat the restore of the thing that ends up there. And it might be fixed
now - I haven't been concerned enough to check.

> The difference between gtar and star is that gtar never works correctly
for all

> possible changes, while star works correct and you just need to follow it's
> constraints. As gtar does not work on a simple testcase, I am not willing
> to
> trust in gtar's incrementals.
>

Here's the simple test case. Plug in a FAT-formatted USB key in the path
of your backup. Change something on it and repeat with your incremental
mode. Restore it with or without the key mounted in that position. If
that doesn't preserve the data in a way you can restore, I'm not willing to
trust it for my backups. Or if you are biased against FAT, use the
filesystem of your choice, just do it dynamically.

> Except when it isn't, because it is tied to the filesystem, not the
> directory tree structure. If I wanted filesystem dependencies I'd use
> dump. I expect tar to follow directory trees and be agnostic to mount
> points.

It may be that you miss the fact that modern filesystems like e.g. ZFS may
> be
> able to move a directory tree into a "new filesystem" without affecting
> ctime.
>
>
That breaks the definition of ctime, doesn't it? But it doesn't matter to
GNUtar because if it hits a directory in a location that doesn't match what
is listed in its -listed-incremental file it will back it up with
everything below it.

If gtar would be implemented correctly, it would need to maintain a huge
> database for the lifetime of the filesystems tobackup.
>

No, it needs a small file for each state of a tree where you plan to do a
subsequent higher level incremental. And amanda manages this for you.


> You also seem to miss the point that in case you like to move a sub-tree
> into a
> different filesystem (something that is extremely unlikely to happen with
> modern
> filesystems), you need to do a lot of adminstrative work anyway.


Sometimes it is, sometimes it is plugging in a usb device, and sometimes it
is someone else's administrative work, unrelated to the backup
configuration.

Adding a new
> FS to the list of FSs in the backup system is a minor task in this context.
>
> What star allows you to do is to have incrementals that are as reliable as
> ufsdump but that are independent from the OS and the FS.
>
>
That would be better stated as 'as filesytem dependent as ufsdump'. It is
not independent from the FS at all or I'd be able to point it at an
arbitrary directory without knowing or caring if it encompasses a whole
filesystem or has additional mounts. Or if the layout is the same during
the restore.

> I handled it by pointing amanda at remote systems. And I expected it
> > to keep those systems backed up even if someone else mounted some new
> > disks in places where they needed some more space. I don't see how
> > star fits into that scheme.
>
> Well as mentioned before, if you reconfigure your server, you also need to
> reconfigure the backup to match the new situation at the server.
>

They are different machines. Why should I need to know if someone else
moved /home to a separate volume just to correctly copy the data that has
changed since the last backup run?

> > AFAIK, amanda has too few features or there are no people who are
> willing to
> > > put efforts in adopting to star.
> >
> > Star simply does not do what amanda needs - or did not the last time I
> > looked. Amanda needs a way to quickly estimate the size of a run for
> > its brilliant feature of automatically balancing the mix of fulls and
> > incrementals to ensure that you get at least an incremental of every
> > target every night plus a full within the number of tapes in your
> > rotation. And it can't fail on incrementals just because someone
> > replaced a directory on a remote machine with a mount point.
>
> This is an unproven claim from you. I grant you that star supports all you
> need
> for a reliable incremental backup and restore.
>

It does, under the same restricted conditions dump would - and amanda
already works with dump. The reason I use tar instead of dump is to not be
restricted to those conditions.

> > I currently cannot believe that there is really any important feature
> that is
> > > missing in star.
> >
> > Those features obviously aren't important to you, but they are enough
> > to keep me - or any amanda user - from considering star. And amanda
>
> If amanda does not support star, it seems that the amanda people are not
> interested in reliable backups.
>

No, they already have dump for the things that star could do.



> Star supports everything you need, if Amanda does not support star, this is
> obviously a filure at amanda's side.
>

It just doesn't add capabilities the way GNUtar does so there is no need
for it. And I don't see any support for size estimates like dump provides,
which are very necessary for amanda to arrange each set of incremental and
full runs to fill a tape. Can you do anything that matches the speed of a
gnutar run of : 'tar --totals -cf /dev/null /path' ? I'll agree that what
GNUtar does in that special case is weird (or at least that the reason it
does it is weird) if you'll agree that it is necessary for practical
reasons.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-10-2012, 11:05 AM
 
Default schily tools

Les Mikesell <lesmikesell@gmail.com> wrote:


> > In general, this does not work with gtar. You are exactly in the area that
> > caused my conclusion that gtar is not useful at all for incremental
> > backups.
> >
> >
> No, a general file-oriented case would handle FAT and NTFS filesystems, and
> continue to work even if you mount one of those in the path of your planned
> backup or restore. Star fails that.

Well, I verified that gtar fails with even a simple case and I did not see any
problem with star since 6 years even with thousands of incremental restores.
So I am not sure what you are talkin about.

How many successful automatical incremental restores did you try with gtar so
far? 1/12 dozen?

Star's incrementals are file oriented and this is why it is able to offer a OS
and FS independent incremental backup.

Given the fact that gtar completely fails with it's incrementals even in simple
cases, I cannot understand why people still use gtar's incrementals. Note that
doing a backup is only half of the job and gtar fails with the restore...


> > Star does what you need, you are just using it the wrong way.
> >
> >
> I don't plan to arrange my life or my data around star's restricted
> capabilities.

This is a really strange claim, as you obviously follow the rules in case you
create a new filesystem.

Now please explain me why you are doing backups with a program that is known
to fail with incremental restores?


> As mentioned, gtar has major problems whe directory is envolved. How many
> > successful restores did you pass?
> >
>
> All of them. Even the contrived case you sent me when we had this same
> conversation years ago. The saves always work and the only issue there
> has ever been is when a restored directory is supposed to be deleted and
> replaced by a file of the same name as an incremental update (or maybe the
> other way around...). It's something that has never happened in years of
> operations and recoverable if it does - just delete the offending item and
> repeat the restore of the thing that ends up there. And it might be fixed
> now - I haven't been concerned enough to check.

Well gtar failed with my first simple test already, why do you still believe it
works?


> Here's the simple test case. Plug in a FAT-formatted USB key in the path
> of your backup. Change something on it and repeat with your incremental
> mode. Restore it with or without the key mounted in that position. If
> that doesn't preserve the data in a way you can restore, I'm not willing to
> trust it for my backups. Or if you are biased against FAT, use the
> filesystem of your choice, just do it dynamically.

Why do you like me to do some really strange test while your backup method is
know not to work at all?

Would'nt it be better to first switch to a working backup program?

As you stay with gtar, it seems that rather you are biased.

Also note that reliable backups need to use snapshots, so please rethink your
backup method under the right constraints.


> It may be that you miss the fact that modern filesystems like e.g. ZFS may
> > be
> > able to move a directory tree into a "new filesystem" without affecting
> > ctime.
> >
> >
> That breaks the definition of ctime, doesn't it? But it doesn't matter to
> GNUtar because if it hits a directory in a location that doesn't match what
> is listed in its -listed-incremental file it will back it up with
> everything below it.

No, ZFS is fully compatible with POSIX.

ZFS permits file renames aross different filesystems in case these filesystems
are on the same storage pool.

See POSIX for rename():

[EXDEV]
[CX] [Option Start] The links named by new and old are on different file
systems and the implementation does not support links between file systems.
[Option End]

So if really like you move a sub-tree into a separate filesystem, only the top
level directories are granted to get ctime updated.

BTW: Why should gtar work in your case when it already fails with incremental
restores for simple directory rename operations?


> If gtar would be implemented correctly, it would need to maintain a huge
> > database for the lifetime of the filesystems tobackup.
> >
>
> No, it needs a small file for each state of a tree where you plan to do a
> subsequent higher level incremental. And amanda manages this for you.

You seem to have a strange concept of what's a "small" file.

The file we are talking about may easily reach a size of several GB.

Star needs less that 100 bytes per level of incrementals.

So if you do just one level 0 dump and any amount of level 1 dumps, this star
state file will be less than 200 bytes per filesystem.

> Sometimes it is, sometimes it is plugging in a usb device, and sometimes it
> is someone else's administrative work, unrelated to the backup
> configuration.

If you like to do reliable backups, you need to snapshots anyway. Believe me
that the snapshot does not mirror the mount you are talking about, so it is
obvious that the gtar concept is even wrong if you asume a reliable overall
environment.


> > What star allows you to do is to have incrementals that are as reliable as
> > ufsdump but that are independent from the OS and the FS.
> >
> >
> That would be better stated as 'as filesytem dependent as ufsdump'. It is
> not independent from the FS at all or I'd be able to point it at an
> arbitrary directory without knowing or caring if it encompasses a whole
> filesystem or has additional mounts. Or if the layout is the same during
> the restore.

???


> > Well as mentioned before, if you reconfigure your server, you also need to
> > reconfigure the backup to match the new situation at the server.
> >
>
> They are different machines. Why should I need to know if someone else
> moved /home to a separate volume just to correctly copy the data that has
> changed since the last backup run?

Because such a change would affect snapshots that are definitely needed for
reliable backups.


> > This is an unproven claim from you. I grant you that star supports all you
> > need
> > for a reliable incremental backup and restore.
> >
>
> It does, under the same restricted conditions dump would - and amanda
> already works with dump. The reason I use tar instead of dump is to not be
> restricted to those conditions.

You told me that you use gtar and gtar has the limitation that incremental
restrores do not work in many cases.

I prefer to live with a method that may have any restriction, in case it is
able to support reliable incremental restores. This is what you get from star.



> > If amanda does not support star, it seems that the amanda people are not
> > interested in reliable backups.
> >
>
> No, they already have dump for the things that star could do.

Did you ever think about the fact that the various BSD dump descendants do not
share a common archive format?

Star being OS and FS independent allows you to e.g. restore any incremental on
any OS and does not force you to first manually port the actual "restore"
program to your current restore OS.


> > Star supports everything you need, if Amanda does not support star, this is
> > obviously a filure at amanda's side.
> >
>
> It just doesn't add capabilities the way GNUtar does so there is no need
> for it. And I don't see any support for size estimates like dump provides,
> which are very necessary for amanda to arrange each set of incremental and
> full runs to fill a tape. Can you do anything that matches the speed of a
> gnutar run of : 'tar --totals -cf /dev/null /path' ? I'll agree that what
> GNUtar does in that special case is weird (or at least that the reason it
> does it is weird) if you'll agree that it is necessary for practical
> reasons.

I cannot tell when gtar introduced this strange method that makes performance
tests hard....

Star introduced the -onull option more than 15 years ago and I asume that
potential users start with reading the man page.

One problem for you with star may be that star does not match the performance
of gtar.... star is aprox 2-3x faster than gtar ;-)

I hope you can live with this limitation.....


Jörg

--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni)
joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-10-2012, 06:01 PM
Les Mikesell
 
Default schily tools

On Fri, Feb 10, 2012 at 6:05 AM, Joerg Schilling <
Joerg.Schilling@fokus.fraunhofer.de> wrote:


> > No, a general file-oriented case would handle FAT and NTFS filesystems,
> and
> > continue to work even if you mount one of those in the path of your
> planned
> > backup or restore. Star fails that.
>
> Well, I verified that gtar fails with even a simple case and I did not see
> any
> problem with star since 6 years even with thousands of incremental
> restores.
> So I am not sure what you are talkin about.
>

Gtar doesn't fail, it just requires some extra steps to accommodate some
very unlikely circumstances. Your star runs will not work at all in very
likely circumstances (which you conveniently avoid testing).


> How many successful automatical incremental restores did you try with gtar
> so
> far? 1/12 dozen?
>

Enough to know it works.

Star's incrementals are file oriented and this is why it is able to offer a
> OS
> and FS independent incremental backup.
>
>
Our definitions of 'OS and FS independent' are very different. I expect
that to mean that I can actually change the FS and have things keep
working. Star doesn't. And it doesn't work over nfs, on fat, ntfs, and on
and on.

> I don't plan to arrange my life or my data around star's restricted

> > capabilities.
>
> This is a really strange claim, as you obviously follow the rules in case
> you
> create a new filesystem.
>
>
What rules? If you are FS independent, you follow a directory tree, not a
rulebook. I want the content of my files backed up whether they are stored
in a way that matches your restricted rules or not. I don't want to have
to take a whole filesystem. I don't want a run to be restricted to a
single filesystem.

Now please explain me why you are doing backups with a program that is known
> to fail with incremental restores?
>
>
In a word, amanda. If my backups don't fit on the tape there are no
backups at all, and the only reason for doing incrementals at all is that
you have to use them to make the data fit. I'm not interested in
calculating the right mix of incremental levels on a whole bunch of
directory trees to make each night's run fit. Amanda does that
automatically. Amanda can use dump or gnutar. Linus Torvald has claimed
that dump is not safe on live filesystems. I assume he's an expert on the
topic (and without looking at the code, I'd guess that star uses the same
approach to determine what to take in incrementals). That leaves gnutar.
I can live with a very unlikely chance that I'll have to give a few extra
commands to restore compared to not getting backups or having to compute
the size of incremental levels manually every day.

And then there was the fact that at the time I implemented amanda, star did
not do incremental restores anyway...



> Well gtar failed with my first simple test already, why do you still
> believe it
> works?
>
>
Because the data is on the tape and can be extracted if you move the
same-named but different object type thing out of the way first (and admit
that it wouldn't be there except in a contrived test). The commands to do
it might not be convenient, but it is much, much better than using a tool
that in many circumstances won't have the data at all, or can't restore it
onto a different mount topology.

> Here's the simple test case. Plug in a FAT-formatted USB key in the path
> > of your backup. Change something on it and repeat with your incremental
> > mode. Restore it with or without the key mounted in that position. If
> > that doesn't preserve the data in a way you can restore, I'm not willing
> to
> > trust it for my backups. Or if you are biased against FAT, use the
> > filesystem of your choice, just do it dynamically.
>
> Why do you like me to do some really strange test while your backup method
> is
> know not to work at all?
>

OK, here's a test that is a very common scenario. Boot a dual-boot
windows/linux laptop into linux, back it up in a way that includes the
mounted VFAT and NTFS partitions. Make changes on the VFAT or NTFS
locations under linux or windows. Make an incremental backup that includes
all the changes.


> Would'nt it be better to first switch to a working backup program?
>
> As you stay with gtar, it seems that rather you are biased.
>
>
Strictly a pragmatic choice. GNUtar works, where star did not.



> Also note that reliable backups need to use snapshots, so please rethink
> your
> backup method under the right constraints.
>
>
But FS snapshots take extra disk space and not all file systems provided
them back then. That does not make the file contents less important.


So if really like you move a sub-tree into a separate filesystem, only the
> top
> level directories are granted to get ctime updated.
>
> BTW: Why should gtar work in your case when it already fails with
> incremental
> restores for simple directory rename operations?
>
>
The one incremental restore inconvenience you found has nothing to do with
the way --listed-incremental backups work. They are not tied to
filesystems or mount points. They follow directory trees.

> If gtar would be implemented correctly, it would need to maintain a huge
> > database for the lifetime of the filesystems tobackup.
> >
>
> No, it needs a small file for each state of a tree where you plan to do a
> subsequent higher level incremental. And amanda manages this for you.

You seem to have a strange concept of what's a "small" file.
>
> The file we are talking about may easily reach a size of several GB.
>
>
So first we need room for filesystem snapshots, but now storing a file big
enough to hold the equivalent of a directory listing is a problem?

> > What star allows you to do is to have incrementals that are as reliable
as
> > ufsdump but that are independent from the OS and the FS.
> >
> >
> That would be better stated as 'as filesytem dependent as ufsdump'. It is
> not independent from the FS at all or I'd be able to point it at an
> arbitrary directory without knowing or caring if it encompasses a whole
> filesystem or has additional mounts. Or if the layout is the same during
> the restore.

???
>

Can you, using star, make incremental backups of a directory tree that work
without knowing./caring what file systems hold that tree or where the roots
or mount points are?

If you have incremental backups that were made under the restrictive
conditions that star requires, can you restore them correctly on a
different topology? For example, you backed up / from a small machine with
no mounted partitions. It breaks and you replace it with a new machine
that has an additional drive that you mount as /home. Can you restore the
incremental levels of the /home portion of the backup onto this replacement
system, along with some things still on the root partition?

>
> > No, they already have dump for the things that star could do.
>
> Did you ever think about the fact that the various BSD dump descendants do
> not
> share a common archive format?
>
>
Does that matter if I have the code for the one that reads my format?

Star being OS and FS independent allows you to e.g. restore any incremental
> on
> any OS and does not force you to first manually port the actual "restore"
> program to your current restore OS.
>
>
That would be nice if it were true. The most likely non-native target would
be windows. Should I plan on that working? What about the simpler case of
the same OS but an added mount point in the layout where I want a portion
of the restore to go?


I cannot tell when gtar introduced this strange method that makes
> performance
> tests hard....
>
>
I expect actual backups to be i/o bound to a point that whatever else they
do performance-wise is irrelevant.

Star introduced the -onull option more than 15 years ago and I asume that
> potential users start with reading the man page.
>
>
I didn't see an option for reporting size in the man page - it might
actually be adaptable if anyone still cared about tapes. But, I find the
man page very confusing. Most of the incremental options say it has to be
a single Posix-conforming FS but the -dump section talks about spanning
more than one.

Anyway it is mostly irrelevant for me now that online backups are feasible
in terms of disk space and bandwidth to offsite storage. It is so much
easier to click on a version of a file you want in backuppc's web interface
than to request the right series of offsite tapes to be shipped back and
restore them in the right order to get to the date it should have been
there.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-14-2012, 05:41 PM
 
Default schily tools

Les Mikesell <lesmikesell@gmail.com> wrote:

> > Well, I verified that gtar fails with even a simple case and I did not see
> > any
> > problem with star since 6 years even with thousands of incremental
> > restores.
> > So I am not sure what you are talkin about.
> >
>
> Gtar doesn't fail, it just requires some extra steps to accommodate some
> very unlikely circumstances. Your star runs will not work at all in very
> likely circumstances (which you conveniently avoid testing).

Trying to understand you leads me to the assumption that you of course know
that gtar fails. As you seem to claim that there is a workaround but don't
mention it, it seems that you like other people to fail when trying to restore
incremental backups that have been made with gtar.


> > How many successful automatical incremental restores did you try with gtar
> > so
> > far? 1/12 dozen?
> >
>
> Enough to know it works.

Well, I did one test only and I know that gtar does not do what it should when
restoring incrementals.

In addition, gtar incrementals are not space efficient:

If you e.g. have a 1 TB filesystem with one directory at top level and rename
that directory, you will end up with a 1 TB incremental if you use gtar.
Looking at the way gtar works, I would assume that gtar needs an intermediate
disk space of 2 TB to restore the data (if the restore works at all).

Star will create an incremental archive of 10kB for the same change and star
does not need additional space on the restore media.

Also note that star is able to backup ACLs and extended attributes but gtar
does not have the needed features to do so.


> Our definitions of 'OS and FS independent' are very different. I expect
> that to mean that I can actually change the FS and have things keep
> working. Star doesn't. And it doesn't work over nfs, on fat, ntfs, and on
> and on.

Do you like to tell everbody that it makes no sense to believe you, or why do
you write claims that are obviously wrong?

Star gives you FS and OS independence and star is even at least 30% faster
than ufsdump (under all conditions) or any fork that was created from ufsdump
(such es ext*..).


> What rules? If you are FS independent, you follow a directory tree, not a
> rulebook. I want the content of my files backed up whether they are stored
> in a way that matches your restricted rules or not. I don't want to have
> to take a whole filesystem. I don't want a run to be restricted to a
> single filesystem.

You again harm your credibility by writing false suggestions.

Star of course allows you to backup parts of a filesystem (e.g. a directory
sub-tree). It seems that your problem is that you are not willing to inform you
about any other program that gtar and it seems that you are not willing to
think about the results from gtar problems.


> In a word, amanda. If my backups don't fit on the tape there are no
> backups at all, and the only reason for doing incrementals at all is that
> you have to use them to make the data fit. I'm not interested in
> calculating the right mix of incremental levels on a whole bunch of
> directory trees to make each night's run fit. Amanda does that

You again harm your credibility by writing false suggestions.

Star is able to automatically deal with the unpredictable size of modern tapes
with HW compression, while amanda fails completely and needs to assume the
smallest possible size. This is because amanda does not make use of the related
features of the backup utility. Star can detect the actual end of a tape in
write mode on the fly and then request a media change.

BTW: what amanda does may make sense with gtar as gtar frequently fails to
accept own followup volumes but this is a problem that is absent with star.

> automatically. Amanda can use dump or gnutar. Linus Torvald has claimed
> that dump is not safe on live filesystems. I assume he's an expert on the
> topic (and without looking at the code, I'd guess that star uses the same
> approach to determine what to take in incrementals). That leaves gnutar.

Well, besides the fact that Torvalds is not known of being a filesystem expert
it is a bad idea to invert a true claim as you do here and believe that this is
still true.

ufsdump is not safe on life filesystems but gtar and star are not safe too.

The most insecure backup program for a life file system if ufsdump or it's
forks. This is because ufsdump first reads all fs meta data and later reads a
raw filesystem at block level without taking care of changes.

gtar is still insecure as it traverses the directory tree and is affected by
changes on that tree while the backup is running.

star is definitely less insecure than gtar as star does not need to keep a
database of the filesystem structure in order to be able to work.

In any case, I know of no serious sysadmin that makes backups from a life
filesystem without using filesystem snapshots. If you however use snapshots
that are needed for a reliable incremental backup, you cannot backup directory
trees the way you seem to like.....

Star matches the reliable case and if you use gtar without snapshots, you are
creating unreliable backups.



> I can live with a very unlikely chance that I'll have to give a few extra
> commands to restore compared to not getting backups or having to compute
> the size of incremental levels manually every day.
>
> And then there was the fact that at the time I implemented amanda, star did
> not do incremental restores anyway...

Star implements reliable incremental restores since 7 years, so you seem to be
living in a long gone past.


> > Well gtar failed with my first simple test already, why do you still
> > believe it
> > works?
> >
> >
> Because the data is on the tape and can be extracted if you move the
> same-named but different object type thing out of the way first (and admit
> that it wouldn't be there except in a contrived test). The commands to do
> it might not be convenient, but it is much, much better than using a tool
> that in many circumstances won't have the data at all, or can't restore it
> onto a different mount topology.

Well star always works fully automatically with incrementals.

What you like to do with gtar may require hours of manual intervention and is
still based on an unreliable basic method (as you cannot use the needed
snapshots).


> OK, here's a test that is a very common scenario. Boot a dual-boot
> windows/linux laptop into linux, back it up in a way that includes the
> mounted VFAT and NTFS partitions. Make changes on the VFAT or NTFS
> locations under linux or windows. Make an incremental backup that includes
> all the changes.

You are trying to do something in an unrelibale way that can be done reliably
and fully automatically using star instead of gtar. So why do you still use
gtar?


> > As you stay with gtar, it seems that rather you are biased.
> >
> >
> Strictly a pragmatic choice. GNUtar works, where star did not.

Well gtar does not work and all your replies admit (even if you try to hide
this fact) that gtar does not work automatically and that you need to add some
secret manual work that may take a long time in order to go on in case of
failures.

Star on the other side works reliably and automatically.


> But FS snapshots take extra disk space and not all file systems provided
> them back then. That does not make the file contents less important.

Just do not use filesystems that do not support snapshots for professional
work and if you are forced to do backups on a fs without snapshots at least
use the backup method with the least sensitivity against inconsistencies. This
preferred backup method is star if we are comparing ufsdump, gtar and star.


> > The file we are talking about may easily reach a size of several GB.
> >
> >
> So first we need room for filesystem snapshots, but now storing a file big
> enough to hold the equivalent of a directory listing is a problem?

There of course is a difference between the need to store some data for a few
minutes (this is what an incremental star backup takes) and the need to store
a lot of data for the whole life-cycle of the FS (which is what you need to do
with gtar).

Also note that many filesystems store the actual data for the snapshot in the
free blocks of the filesystem. In these cases, you do not even notice the need
for space to support the incremental.

> Can you, using star, make incremental backups of a directory tree that work
> without knowing./caring what file systems hold that tree or where the roots
> or mount points are?

Of course, if there are no mount points in that tree.

As mentioned before, the star method is enforced anyway if you like to have
reliable incrementals that need snapshots. So gtar will not do what you like in
case you implement reliable backups.


> > Did you ever think about the fact that the various BSD dump descendants do
> > not
> > share a common archive format?
> >
> >
> Does that matter if I have the code for the one that reads my format?

If you like to restore more meta data than UNIX had in 1981, this definitely
matters. Also note that star is at least 30% faster than ufsdump, so why do you
like to use ufsdump when you can have star?


> That would be nice if it were true. The most likely non-native target would
> be windows. Should I plan on that working? What about the simpler case of
> the same OS but an added mount point in the layout where I want a portion
> of the restore to go?

If you try to create backups on Win-DOS without using Cygwin or some equivalent
system, you are lost, as MS ignores standards and does not include st_ino in
struct stat. If you however use Cygwin, it works...


> I expect actual backups to be i/o bound to a point that whatever else they
> do performance-wise is irrelevant.

If you were true, why is gtar slower than star?

If you are backing up to a real tape, this may be extremely important because
being a bit too slow may cause the tape to fail streaming.

> Star introduced the -onull option more than 15 years ago and I asume that
> > potential users start with reading the man page.
> >
> >
> I didn't see an option for reporting size in the man page - it might
> actually be adaptable if anyone still cared about tapes. But, I find the
> man page very confusing. Most of the incremental options say it has to be
> a single Posix-conforming FS but the -dump section talks about spanning
> more than one.

If you did never work with star, why do you claim so many "failures" of star?

I recommend you to start using star and talk about it after you learn how it
works. Star implements so many features, that you need to use it for some days
to understand the concept, but I know nobody who did not switch to star after
understanding the concept.

Most of the features from star I am using on a daily base are missing in gtar.


Jörg

--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni)
joerg.schilling@fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/
URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-15-2012, 03:12 AM
Les Mikesell
 
Default schily tools

On Tue, Feb 14, 2012 at 12:41 PM, Joerg Schilling
<Joerg.Schilling@fokus.fraunhofer.de> wrote:
>>
>> Gtar doesn't fail, it just requires some extra steps to accommodate some
>> very unlikely circumstances. *Your star runs will not work at all in very
>> likely circumstances (which you conveniently avoid testing).
>
> Trying to understand you leads me to the assumption that you of course know
> that gtar fails. As you seem to claim that there is a workaround but don't
> mention it, it seems that you like other people to fail when trying to restore
> incremental backups that have been made with gtar.

The only thing I've ever seen fail was where the full or earlier
incremental contained a directory which was subsequently replaced with
a plain file with the same name that was included in a higher level
incremental. The gnutar code circa 2004 didn't correctly remove the
directory that should be overwritten during the incremental restore.
But the error was obvious and you already know how to remove a
directory. Do it, repeat the restore of the missing item and everyone
is happy. But that was a long time ago, maybe things have changed.

> Well, I did one test only and I know that gtar does not do what it should when
> restoring incrementals.

So you don't check for changes either?

> If you e.g. have a 1 TB filesystem with one directory at top level and rename
> that directory, you will end up with a 1 TB incremental if you use gtar.
> Looking at the way gtar works, I would assume that gtar needs an intermediate
> disk space of 2 TB to restore the data (if the restore works at all).
>
> Star will create an incremental archive of 10kB for the same change and star
> does not need additional space on the restore media.

Except that star won't do it at all if you want it to follow an
arbitrary directory tree.

>> Our definitions of 'OS and FS independent' *are very different. *I expect
>> that to mean that I can actually change the FS and have things keep
>> working. *Star doesn't. *And it doesn't work over nfs, on fat, ntfs, and on
>> and on.
>
> Do you like to tell everbody that it makes no sense to believe you, or why do
> you write claims that are obviously wrong?

Can it follow directory trees across mount points getting incrementals
right? If I'm wrong, show me the commands needed to make that work.

> Star of course allows you to backup parts of a filesystem (e.g. a directory
> sub-tree).

And follow across mount points?

>> In a word, amanda. * If my backups don't fit on the tape there are no
>> backups at all, and the only reason for doing incrementals at all is that
>> you have to use them to make the data fit. I'm not interested in
>> calculating the right mix of incremental levels on a whole bunch of
>> directory trees to make each night's run fit. *Amanda does that
>
> You again harm your credibility by writing false suggestions.
>
> Star is able to automatically deal with the unpredictable size of modern tapes
> with HW compression, while amanda fails completely and needs to assume the
> smallest possible size. This is because amanda does not make use of the related
> features of the backup utility. Star can detect the actual end of a tape in
> write mode on the fly and then request a media change.

I don't want a media change. No one is going to be there to change
it. I want the software to compute the mix of fulls and incrementals
that will fit my single daily tape. Amanda did that faithfully for
many years..

>> And then there was the fact that at the time I implemented amanda, star did
>> not do incremental restores anyway...
>
> Star implements reliable incremental restores since 7 years, so you seem to be
> living in a long gone past.

Yes, I believed what you said here:
http://blog.gmane.org/gmane.comp.archivers.star.user/month=20040701
and haven't had any more reason to check again than you have to check gnutar.
Amanda kept doing what I needed until the tape drives wore out many
years later, with no more attention than swapping the tape sometime
during the day.

> What you like to do with gtar may require hours of manual intervention and is
> still based on an unreliable basic method (as you cannot use the needed
> snapshots).

There is nothing that would prevent you from using snapshots with tar,
but a filesystem snapshot isn't really enough to ensure that your
application's view of the data is consistent anyway. Most things
don't care and the ones that do have application-level ways to dump
their data consistently.

> You are trying to do something in an unrelibale way that can be done reliably
> and fully automatically using star instead of gtar. So why do you still use
> gtar?

Because when I needed it, star wasn't even usable. Even if it had
been able to do an incremental restore, it didn't work with amanda.

>> So first we need room for filesystem snapshots, but now storing a file big
>> enough to hold the equivalent of a directory listing is a problem?
>
> There of course is a difference between the need to store some data for a few
> minutes (this is what an incremental star backup takes) and the need to store
> a lot of data for the whole life-cycle of the FS (which is what you need to do
> with gtar).

What good does being able to release that space do if you have to
reserve it anyway? I don't see how it has any other use.

>> Can you, using star, make incremental backups of a directory tree that work
>> without knowing./caring what file systems hold that tree or where the roots
>> or mount points are?
>
> Of course, if there are no mount points in that tree.

And if there are? I don't see why you consider gnutar's issue with a
directory being replaced with a file of the same name to be a big
problem and completely ignore the issue star has with a directory
being replaced by a mount point. I've done the latter much more
often than the former. It's not a reason to make backups of the file
contents fail.

>> I didn't see an option for reporting size in the man page - it might
>> actually be adaptable if anyone still cared about tapes. *But, *I find the
>> man page very confusing. *Most of the incremental options say it has to be
>> a single Posix-conforming FS but the -dump section talks about spanning
>> more than one.
>
> If you did never work with star, why do you claim so many "failures" of star?

One failing is enough. Isn't that why you dismissed gnutar? And for
me, being unable to do incrementals that cross mount points is a such
a failing. And if I understand it correctly, even if I kept changing
the backup runs to match the source mount layouts, I would not be
able to restore correctly if I wanted the replacement system to have a
different topology that would place a mount point in a location that
was previously a directory . Did I misunderstand that?

> I recommend you to start using star and talk about it after you learn how it
> works. Star implements so many features, that you need to use it for some days
> to understand the concept, but I know nobody who did not switch to star after
> understanding the concept.

I hope to never need tapes again, except possibly for archival storage
where incrementals would not be a factor. And going to disk targets,
rsync is much nicer than either gnutar or star.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 01:35 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org