FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 08-15-2010, 11:11 PM
Nganon
 
Default How to build a time machine on Gentoo

Hello all,
My first post on the list. I thought I would start with something that I startedto think of as 'essential' after losing 90GB of data. Now I have two main

questions in mind: what to and how to back up on gentoo most efficiently.
1. Apart from users' home directories and the followings, what should be backedup on a gentoo machine?*

/etc/portage//root/var/lib/portage...?
2. Erm..okay, I am gonna say,*what magic I want and then ask your way.I first started making gzipped tar balls as follows:


tar czpf /media/backups/userA-`date +%Y.%m.%d`.tgz -X userA-excludelist /etc
But these can get huge especially for home dirs. I also want safe dvd copies.*Though I can find enough space on the external drives, I don't trust them

any more. See above..sigh..(No I recovered about one third of it with testdisk/photorec *which names them as file000001 file00002.. and half them are zero sized.. which*quite justifies my agony)


Here is what I wanna do. I want to have only one big backup for, say,userA-2010.08.07.tgz and other small backup tars containing only thefiles/folders that were modified since last update, 2010.08.07, as

userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz, *userA-diff-2010.08.28.tgzetc. Now if I want to take the userA back to the future, 2010.08.21, *I want todo it by first extracting the huge tar userA-2010.08.07.tgz and then the tiny

backup userA-diff-2010-08-21.tgz.*
But the thing is I don't know how to do this. I am hoping maybe you can tell me*how to of it as well as a*better way of doing backups.*


By the way, since I want dvd backups as well, and I want to use +rw dvds so I*can*overwrite old backup after a while, what is best way of ensuring the*integrity*and safety of them. Is it a good idea to use truecrypt containers? Or*

nothing*tops signing and encrypting with gpg?
Thanks for any comment in advance.*
 
Old 08-16-2010, 12:15 AM
Alex Schuster
 
Default How to build a time machine on Gentoo

Nganon <nganon+gentoo@gmail.com> writes:

> Hello all,
>
> My first post on the list. I thought I would start with something
> that I started
> to think of as 'essential' after losing 90GB of data. Now I have two
> main questions in mind: what to and how to back up on gentoo most
> efficiently.
>
> 1. Apart from users' home directories and the followings, what should
> be backed
> up on a gentoo machine?
> /etc/portage/
> /root
> /var/lib/portage
> ...?

Wouldn't it be easier to just backup the whole / directory?
Excluding /home, /usr/portage and /var/tmp/portage?


> 2. Erm..okay, I am gonna say, what magic I want and then ask your way.
> I first started making gzipped tar balls as follows:
>
> tar czpf /media/backups/userA-`date +%Y.%m.%d`.tgz -X
> userA-excludelist /etc
>
> But these can get huge especially for home dirs. I also want safe dvd
> copies.
> Though I can find enough space on the external drives, I don't trust
> them any more. See above..sigh..(No I recovered about one third of it
> with testdisk/photorec
> which names them as file000001 file00002.. and half them are zero
> sized.. which
> quite justifies my agony)

Okay, but I don't trust DVDs. Although DVD-RAM is quite safe I heard.
But external disks are flexible, offer more space, and if you want more
security, just use yet another drive, so you are safe even if your main
drive and a backup drive fails.

I suggest you have a look at rdiff-backup. It gives you a 1:1 copy of
the source directory, but also does incremental backups, which are
stored (in compressed form) in an additional folder in the destination
directory. I would use this at least for things like /etc, where I
sometimes might want to retrieve an old version of a file. Similar to
your approach with big tar files and small ones containing the
increments.

I use a script for my backups, which I mentioned here on 2010-05-07,
subject 'Snackup'. It optionally creates LVM snapshots so I can make
backups from the running system, even if the source directory is
altered during the backup. This works on LVM only, though, and also
allows the volume to be LUKS-encrypted. It does backups by rdiff-backup,
rsync, cp, tar or dd. It may be overkill when not using the LVM
features, but still I suggest to use some script for backups, so one
does not always have to remember the backup commands. When I want to
update my backup, I enter something like 'snackup boot root home src',
and the script backs up my boot, root and home partition in the
background, and creates tar files each directory in /usr/src.


> By the way, since I want dvd backups as well, and I want to use +rw
> dvds so I can overwrite old backup after a while, what is best way of
> ensuring the integrity and safety of them. Is it a good idea to use
> truecrypt containers? Or nothing tops signing and encrypting with gpg?

I'd use DVD-RAM. The media is a little more expensive, but AFAIK they
were made with long-time backups in mind. And access is much easier,
you just copy the files as to an external drive, no need to burn ISOs.

Wonko
 
Old 08-16-2010, 08:36 AM
Marco
 
Default How to build a time machine on Gentoo

On Mon, Aug 16, 2010 at 1:11 AM, Nganon <nganon+gentoo@gmail.com> wrote:
> [...]
> Here is what I wanna do. I want to have only one big backup for, say,
> userA-2010.08.07.tgz and other small backup tars containing only the
> files/folders that were modified since last update, 2010.08.07, as
> userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz,
> *userA-diff-2010.08.28.tgz
> etc. Now if I want to take the userA back to the future, 2010.08.21, *I want
> to
> do it by first extracting the huge tar userA-2010.08.07.tgz and then the
> tiny
> backup userA-diff-2010-08-21.tgz.

backup2l can do exactly what you want:

http://backup2l.sourceforge.net/

HTH!

--
Marco
 
Old 08-16-2010, 10:27 AM
Nganon
 
Default How to build a time machine on Gentoo

On 16 August 2010 03:15, Alex Schuster <wonko@wonkology.org> wrote:



Nganon <nganon+gentoo@gmail.com> writes:



> Hello all,

>

> My first post on the list. I thought I would start with something

> that I started

> to think of as 'essential' after losing 90GB of data. Now I have two

> main questions in mind: what to and how to back up on gentoo most

> efficiently.

>

> 1. Apart from users' home directories and the followings, what should

> be backed

> up on a gentoo machine?

> /etc/portage/

> /root

> /var/lib/portage

> ...?



Wouldn't it be easier to just backup the whole / directory?

Excluding /home, /usr/portage and /var/tmp/portage?


Yes but that would not solve my huge backup file problem, would it?**



> 2. Erm..okay, I am gonna say, what magic I want and then ask your way.

> I first started making gzipped tar balls as follows:

>

> tar czpf /media/backups/userA-`date +%Y.%m.%d`.tgz -X

> userA-excludelist /etc

>

> But these can get huge especially for home dirs. I also want safe dvd

> copies.

> Though I can find enough space on the external drives, I don't trust

> them any more. See above..sigh..(No I recovered about one third of it

> with testdisk/photorec

> which names them as file000001 file00002.. and half them are zero

> sized.. which

> quite justifies my agony)



Okay, but I don't trust DVDs. Although DVD-RAM is quite safe I heard.

But external disks are flexible, offer more space, and if you want more

security, just use yet another drive, so you are safe even if your main

drive and a backup drive fails.


I did not give a thought to DVD-RAM before. Will give it a try. Thanks.

I suggest you have a look at rdiff-backup. It gives you a 1:1 copy of

the source directory, but also does incremental backups, which are

stored (in compressed form) in an additional folder in the destination

directory. I would use this at least for things like /etc, where I

sometimes might want to retrieve an old version of a file. Similar to

your approach with big tar files and small ones containing the

increments.


I just emerged and tried it. Seems like incremental backups was what I*was looking for. But from what I see, it is mirroring the src to dist and*storing*the metadata/stats of increments as archived. Maybe there is an*

option*to*archive increments along side their metadata as well. I will keep*playing*with it.




I use a script for my backups, which I mentioned here on 2010-05-07,

subject 'Snackup'. It optionally creates LVM snapshots so I can make

backups from the running system, even if the source directory is

altered during the backup. This works on LVM only, though, and also

allows the volume to be LUKS-encrypted. It does backups by rdiff-backup,

rsync, cp, tar or dd. It may be overkill when not using the LVM

features, but still I suggest to use some script for backups, so one

does not always have to remember the backup commands. When I want to

update my backup, I enter something like 'snackup boot root home src',

and the script backs up my boot, root and home partition in the

background, and creates tar files each directory in /usr/src.


I found the thread and script. I am gonna take a look at it. Once I decide on*how to*backup, I am gonna cron a script for it.*


> By the way, since I want dvd backups as well, and I want to use +rw

> dvds so I can overwrite old backup after a while, what is best way of

> ensuring the integrity and safety of them. Is it a good idea to use

> truecrypt containers? Or nothing tops signing and encrypting with gpg?



I'd use DVD-RAM. The media is a little more expensive, but AFAIK they

were made with long-time backups in mind. And access is much easier,

you just copy the files as to an external drive, no need to burn ISOs.



* * * *Wonko




Thanks for the advises.*
 
Old 08-16-2010, 10:30 AM
Nganon
 
Default How to build a time machine on Gentoo

On 16 August 2010 11:36, Marco <listworks@gmail.com> wrote:


On Mon, Aug 16, 2010 at 1:11 AM, Nganon <nganon+gentoo@gmail.com> wrote:

> [...]

> Here is what I wanna do. I want to have only one big backup for, say,

> userA-2010.08.07.tgz and other small backup tars containing only the

> files/folders that were modified since last update, 2010.08.07, as

> userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz,

> *userA-diff-2010.08.28.tgz

> etc. Now if I want to take the userA back to the future, 2010.08.21, *I want

> to

> do it by first extracting the huge tar userA-2010.08.07.tgz and then the

> tiny

> backup userA-diff-2010-08-21.tgz.



backup2l can do exactly what you want:



http://backup2l.sourceforge.net/



Nice one indeed. Exactly does what I want. It is also good that the backups*can be use without the program itself.
It does not seem to be updated since 2009 but I will give it a try.*


Thanks
 
Old 08-16-2010, 09:37 PM
Mick
 
Default How to build a time machine on Gentoo

On Monday 16 August 2010 11:30:36 Nganon wrote:
> On 16 August 2010 11:36, Marco <listworks@gmail.com> wrote:
> > On Mon, Aug 16, 2010 at 1:11 AM, Nganon
> > <nganon+gentoo@gmail.com<nganon%2Bgentoo@gmail.com >>
> >
> > wrote:
> > > [...]
> > > Here is what I wanna do. I want to have only one big backup for, say,
> > > userA-2010.08.07.tgz and other small backup tars containing only the
> > > files/folders that were modified since last update, 2010.08.07, as
> > > userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz,
> > >
> > > userA-diff-2010.08.28.tgz
> > >
> > > etc. Now if I want to take the userA back to the future, 2010.08.21, I
> >
> > want
> >
> > > to
> > > do it by first extracting the huge tar userA-2010.08.07.tgz and then
> > > the tiny
> > > backup userA-diff-2010-08-21.tgz.
> >
> > backup2l can do exactly what you want:
> >
> > http://backup2l.sourceforge.net/
>
> Nice one indeed. Exactly does what I want. It is also good that the backups
> can be use without the program itself.
>
> It does not seem to be updated since 2009 but I will give it a try.

Not sure if it's in an overlay, but I don't think it's in portage.

Run eix -l backup and see how many back up tools and scripts pop up.

I have been using tar, star and rsync. They all work and they can all make
incremental back ups. You'll find that a lot of the other 'smart' back up
applications are based on these anyway.

HTH.
--
Regards,
Mick
 
Old 08-16-2010, 11:53 PM
Thomas Yao
 
Default How to build a time machine on Gentoo

On Tue, Aug 17, 2010 at 5:37 AM, Mick <michaelkintzios@gmail.com> wrote:
> Not sure if it's in an overlay, but I don't think it's in portage.
>
> Run eix -l backup and see how many back up tools and scripts pop up.
>
> I have been using tar, star and rsync. *They all work and they can all make
> incremental back ups. *You'll find that a lot of the other 'smart' back up
> applications are based on these anyway.

A disk archive utility called dar is a great staff to backup dir trees and files

--
@ghosTM55
Mechanism, not policy
 
Old 08-17-2010, 11:29 AM
Nganon
 
Default How to build a time machine on Gentoo

On 17 August 2010 00:37, Mick <michaelkintzios@gmail.com> wrote:


On Monday 16 August 2010 11:30:36 Nganon wrote:

> On 16 August 2010 11:36, Marco <listworks@gmail.com> wrote:

> > On Mon, Aug 16, 2010 at 1:11 AM, Nganon

> > <nganon+gentoo@gmail.com<nganon%2Bgentoo@gmail.com >>

> >

> > wrote:

> > > [...]

> > > Here is what I wanna do. I want to have only one big backup for, say,

> > > userA-2010.08.07.tgz and other small backup tars containing only the

> > > files/folders that were modified since last update, 2010.08.07, as

> > > userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz,

> > >

> > > *userA-diff-2010.08.28.tgz

> > >

> > > etc. Now if I want to take the userA back to the future, 2010.08.21, *I

> >

> > want

> >

> > > to

> > > do it by first extracting the huge tar userA-2010.08.07.tgz and then

> > > the tiny

> > > backup userA-diff-2010-08-21.tgz.

> >

> > backup2l can do exactly what you want:

> >

> > http://backup2l.sourceforge.net/

>

> Nice one indeed. Exactly does what I want. It is also good that the backups

> can be use without the program itself.

>

> It does not seem to be updated since 2009 but I will give it a try.



Not sure if it's in an overlay, but I don't think it's in portage.


Its not in the protage, I found a ebuild proposal at bugs.gentoo but it*didnt even make it to sunrise.*



Run eix -l backup and see how many back up tools and scripts pop up.


I know there are dozens of them in the portage. searching for backup*does not answer me 'you can do that by these packages'.*Perhaps if NLP*is one day implemented in portage, usage of gentoo user lists will reduce*

by half.*

I have been using tar, star and rsync. *They all work and they can all make

incremental back ups. *You'll find that a lot of the other 'smart' back up

applications are based on these anyway.


I will take a look at star, thanks.*

HTH.

--

Regards,

Mick
 
Old 08-17-2010, 11:34 AM
Nganon
 
Default How to build a time machine on Gentoo

On 17 August 2010 02:53, Thomas Yao <t.yao426@gmail.com> wrote:



On Tue, Aug 17, 2010 at 5:37 AM, Mick <michaelkintzios@gmail.com> wrote:

> Not sure if it's in an overlay, but I don't think it's in portage.

>

> Run eix -l backup and see how many back up tools and scripts pop up.

>

> I have been using tar, star and rsync. *They all work and they can all make

> incremental back ups. *You'll find that a lot of the other 'smart' back up

> applications are based on these anyway.



A disk archive utility called dar is a great staff to backup dir trees and files



Looks like the -A switch is exactly what I am looking for:
** *This way you can make a first full backup, then many differential backup,*


** *each taking as reference the last backup made.*
Thanks!

--

@ghosTM55

Mechanism, not policy
 
Old 08-17-2010, 12:14 PM
Maximilian Bräutigam
 
Default How to build a time machine on Gentoo

Hi

Am 16.08.2010 01:11, schrieb Nganon:
> Hello all,
>
> My first post on the list. I thought I would start with something that I
> started
> to think of as 'essential' after losing 90GB of data. Now I have two main
> questions in mind: what to and how to back up on gentoo most efficiently.
>
> 1. Apart from users' home directories and the followings, what should be
> backed
> up on a gentoo machine?
> /etc/portage/
> /root
> /var/lib/portage
> ...?

You should backup all in / except
/tmp/*
/sys/*
/proc/*
/lost+found/*
/dev/*

I have no solution how to bzip or gzip your backups or how to make a dvd
backup, but I use "app-backup/rsnapshot" which uses rsync but implements
an intelligent rotating system that is done daily, weekly, monthly,
yearly according to your config. Of course you should store the backup
on another physical hdd.

By the way, since a new hdd of one TB is pretty cheap, think about
running your gentoo in a software RAID. Guides:

http://www.gentoo.org/doc/en/gentoo-x86+raid+lvm2-quickinstall.xml
http://en.gentoo-wiki.com/wiki/Software_RAID_Install

Kind regards,
der Max

> 2. Erm..okay, I am gonna say, what magic I want and then ask your way.
> I first started making gzipped tar balls as follows:
>
> tar czpf /media/backups/userA-`date +%Y.%m.%d`.tgz -X userA-excludelist /etc
>
> But these can get huge especially for home dirs. I also want safe dvd
> copies.
> Though I can find enough space on the external drives, I don't trust them
> any more. See above..sigh..(No I recovered about one third of it with
> testdisk/photorec
> which names them as file000001 file00002.. and half them are zero
> sized.. which
> quite justifies my agony)
>
> Here is what I wanna do. I want to have only one big backup for, say,
> userA-2010.08.07.tgz and other small backup tars containing only the
> files/folders that were modified since last update, 2010.08.07, as
> userA-diff-2010.08.14.tgz, userA-diff-2010.08.21.tgz,
> userA-diff-2010.08.28.tgz
> etc. Now if I want to take the userA back to the future, 2010.08.21, I
> want to
> do it by first extracting the huge tar userA-2010.08.07.tgz and then the
> tiny
> backup userA-diff-2010-08-21.tgz.
>
> But the thing is I don't know how to do this. I am hoping maybe you can
> tell me
> how to of it as well as a better way of doing backups.
>
> By the way, since I want dvd backups as well, and I want to use +rw dvds
> so I
> can overwrite old backup after a while, what is best way of ensuring the
> integrity and safety of them. Is it a good idea to use truecrypt
> containers? Or
> nothing tops signing and encrypting with gpg?
>
> Thanks for any comment in advance.
>
 

Thread Tools




All times are GMT. The time now is 12:43 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org