FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 02-24-2010, 08:18 AM
Eero Volotinen
 
Default Backup solution to backup /var/spool/imap above 150GB data

2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
> Hi
>
> We have* an issue with one of our clients , they have a mail server with
> the* /var/spool/imap partition as 150 GB . They need to take differential
> backup on to /backup partition ( a different HDD* of total 250 GB space )* .
> We have tried dar , rsync, rdiff and impasync . But its is not sufficing the
> need as to take a lot of time and consumes a lot of I/O .

So, you need to add more disk i/o? (so, add some disk space with faster raid?)

Take a look at: http://rdiff-backup.nongnu.org/

--
Eero
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-24-2010, 09:33 AM
Agnello George
 
Default Backup solution to backup /var/spool/imap above 150GB data

On Wed, Feb 24, 2010 at 3:08 PM, David Hrbáč <hrbac.conf@seznam.cz> wrote:

Dne 24.2.2010 10:00, Agnello George napsal(a):

> Hi

>

> We have *an issue with one of our clients , they have a mail server with

> the */var/spool/imap partition as 150 GB . They need to take differential

> backup on to /backup partition ( a different HDD *of total 250 GB space ) *.

> We have tried dar , rsync, rdiff and impasync . But its is not sufficing the

> need as to take a lot of time and consumes a lot of I/O .

>

> Is there any back up solution that you can think of , that can work in this

> situation *- open source or proprietary



Is seems to me, that you are using mbox format. So, differential backup

is hard to achieve. Migrate to maildir, every mail is a file, easy to

backup differentially.

David

backup directory structure is /var/spool/imap/a /adomain.com/a/agnello^dsouza/ *

--
Regards
Agnello D'souza



_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-24-2010, 09:38 AM
Eero Volotinen
 
Default Backup solution to backup /var/spool/imap above 150GB data

2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>
>
> On Wed, Feb 24, 2010 at 3:08 PM, David Hrbáč <hrbac.conf@seznam.cz> wrote:
>>
>> Dne 24.2.2010 10:00, Agnello George napsal(a):
>> > Hi
>> >
>> > We have *an issue with one of our clients , they have a mail server with
>> > the */var/spool/imap partition as 150 GB . They need to take
>> > differential
>> > backup on to /backup partition ( a different HDD *of total 250 GB space
>> > ) *.
>> > We have tried dar , rsync, rdiff and impasync . But its is not sufficing
>> > the
>> > need as to take a lot of time and consumes a lot of I/O .
>> >
>> > Is there any back up solution that you can think of , that can work in
>> > this
>> > situation *- open source or proprietary
>>
>> Is seems to me, that you are using mbox format. So, differential backup
>> is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> backup differentially.
>> David
>
> backup directory structure is /var/spool/imap/a
> /adomain.com/a/agnello^dsouza/
>

Well, does that directory contains one file or lot of files ?

Usually maildir structure is like dir++/tmp/current/new directories
and each message is in own file on mailbox all files are inside one
file.

--
Eero
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-24-2010, 10:21 AM
Agnello George
 
Default Backup solution to backup /var/spool/imap above 150GB data

2010/2/24 Eero Volotinen <eero.volotinen@iki.fi>

2010/2/24 Agnello George <agnello.dsouza@gmail.com>:

>

>

> On Wed, Feb 24, 2010 at 3:08 PM, David Hrbáč <hrbac.conf@seznam.cz> wrote:

>>

>> Dne 24.2.2010 10:00, Agnello George napsal(a):

>> > Hi

>> >

>> > We have *an issue with one of our clients , they have a mail server with

>> > the */var/spool/imap partition as 150 GB . They need to take

>> > differential

>> > backup on to /backup partition ( a different HDD *of total 250 GB space

>> > ) *.

>> > We have tried dar , rsync, rdiff and impasync . But its is not sufficing

>> > the

>> > need as to take a lot of time and consumes a lot of I/O .

>> >

>> > Is there any back up solution that you can think of , that can work in

>> > this

>> > situation *- open source or proprietary

>>

>> Is seems to me, that you are using mbox format. So, differential backup

>> is hard to achieve. Migrate to maildir, every mail is a file, easy to

>> backup differentially.

>> David

>

> backup directory structure is /var/spool/imap/a

> /adomain.com/a/agnello^dsouza/

>



Well, does that directory contains one file or lot of files ?



Usually maildir structure is like dir++/tmp/current/new directories

and each message is in own file on mailbox all files are inside one

file.


its in a maildir format and the structure slightly * different from what i mentioned earlier :*root@server1 ~]# ls -la /var/spool/imap/a/user/ajay/
total 5180

-rw------- 1 cyrus mail 34616 Feb 24 16:02 4790.
-rw------- 1 cyrus mail 4490 Feb 24 16:03 4791.
-rw------- 1 cyrus mail 199253 Feb 24 16:07 4792.
-rw------- 1 cyrus mail 22930 Feb 24 16:09 4793.
-rw------- 1 cyrus mail 8485 Feb 24 16:11 4794.

-rw------- 1 cyrus mail 12664 Feb 24 16:13 4795.
-rw------- 1 cyrus mail 4296 Feb 24 16:13 4796.
-rw------- 1 cyrus mail 5337 Feb 24 16:15 4797.
-rw------- 1 cyrus mail 111030 Feb 24 16:21 4798.
-rw------- 1 cyrus mail 4805500 Feb 24 16:23 4799.

-rw------- 1 cyrus mail 22920 Feb 24 16:23 cyrus.cache
-rw------- 1 cyrus mail 204 Dec 10 16:27 cyrus.header
-rw------- 1 cyrus mail 896 Feb 24 16:23 cyrus.index
-rw------- 1 cyrus mail 8669 Feb 24 11:28 cyrus.squat


this is Just a very small user and a example**
--
Regards
Agnello D'souza



_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-24-2010, 10:24 AM
Agnello George
 
Default Backup solution to backup /var/spool/imap above 150GB data

On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au> wrote:

On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:

> Dne 24.2.2010 10:00, Agnello George napsal(a):

> > We have *an issue with one of our clients , they have a mail server with

> > the */var/spool/imap partition as 150 GB . They need to take differential

> > backup on to /backup partition ( a different HDD *of total 250 GB space ) *.

> > We have tried dar , rsync, rdiff and impasync . But its is not sufficing the

> > need as to take a lot of time and consumes a lot of I/O .

> >

> > Is there any back up solution that you can think of , that can work in this

> > situation *- open source or proprietary

>

> Is seems to me, that you are using mbox format. So, differential backup

> is hard to achieve. Migrate to maildir, every mail is a file, easy to

> backup differentially.



rsync and rdiff should handle mbox format okay though. Though I agree Maildir

is generally nicer for differential backups.



Agnello, how long is "a lot of time"? A backup is always going to have to walk

the entire tree and checksum (or at least stat) every file, so there's a minimum

cost you're always going to have. How long does a 'find /var/spool/imap -ls'

take, for instance?



You might want to try brackup (http://code.google.com/p/brackup/). For very

large trees of relatively small files it seems to significantly out-perform

rsync-based backups. I've got brackup packages in my repository (see

http://www.openfusion.net/linux/openfusion_rpm_repository).



Cheers,

Gavin



is it possible with "*brackup "* to back it up to a different server on the same lan* *instead of /backup *. Is there any documentation on the same .

--
Regards
Agnello D'souza



_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-24-2010, 10:27 AM
Eero Volotinen
 
Default Backup solution to backup /var/spool/imap above 150GB data

2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>
>
> On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au> wrote:
>>
>> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
>> > Dne 24.2.2010 10:00, Agnello George napsal(a):
>> > > We have *an issue with one of our clients , they have a mail server
>> > > with
>> > > the */var/spool/imap partition as 150 GB . They need to take
>> > > differential
>> > > backup on to /backup partition ( a different HDD *of total 250 GB
>> > > space ) *.
>> > > We have tried dar , rsync, rdiff and impasync . But its is not
>> > > sufficing the
>> > > need as to take a lot of time and consumes a lot of I/O .
>> > >
>> > > Is there any back up solution that you can think of , that can work in
>> > > this
>> > > situation *- open source or proprietary
>> >
>> > Is seems to me, that you are using mbox format. So, differential backup
>> > is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> > backup differentially.
>>
>> rsync and rdiff should handle mbox format okay though. Though I agree
>> Maildir
>> is generally nicer for differential backups.
>>
>> Agnello, how long is "a lot of time"? A backup is always going to have to
>> walk
>> the entire tree and checksum (or at least stat) every file, so there's a
>> minimum
>> cost you're always going to have. How long does a 'find /var/spool/imap
>> -ls'
>> take, for instance?
>>
>> You might want to try brackup (http://code.google.com/p/brackup/). For
>> very
>> large trees of relatively small files it seems to significantly
>> out-perform
>> rsync-based backups. I've got brackup packages in my repository (see
>> http://www.openfusion.net/linux/openfusion_rpm_repository).
>>
>> Cheers,
>> Gavin
>>
> is it possible with "*brackup "* to back it up to a different server on the
> same lan* *instead of /backup *. Is there any documentation on the same .

rsync or rdiff-backup works on local disk or remote disk.(and other
backup methods too!)

--
Eero
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-24-2010, 10:36 AM
Agnello George
 
Default Backup solution to backup /var/spool/imap above 150GB data

On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen <eero.volotinen@iki.fi> wrote:

2010/2/24 Agnello George <agnello.dsouza@gmail.com>:

>

>

> On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au> wrote:

>>

>> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:

>> > Dne 24.2.2010 10:00, Agnello George napsal(a):

>> > > We have *an issue with one of our clients , they have a mail server

>> > > with

>> > > the */var/spool/imap partition as 150 GB . They need to take

>> > > differential

>> > > backup on to /backup partition ( a different HDD *of total 250 GB

>> > > space ) *.

>> > > We have tried dar , rsync, rdiff and impasync . But its is not

>> > > sufficing the

>> > > need as to take a lot of time and consumes a lot of I/O .

>> > >

>> > > Is there any back up solution that you can think of , that can work in

>> > > this

>> > > situation *- open source or proprietary

>> >

>> > Is seems to me, that you are using mbox format. So, differential backup

>> > is hard to achieve. Migrate to maildir, every mail is a file, easy to

>> > backup differentially.

>>

>> rsync and rdiff should handle mbox format okay though. Though I agree

>> Maildir

>> is generally nicer for differential backups.

>>

>> Agnello, how long is "a lot of time"? A backup is always going to have to

>> walk

>> the entire tree and checksum (or at least stat) every file, so there's a

>> minimum

>> cost you're always going to have. How long does a 'find /var/spool/imap

>> -ls'

>> take, for instance?

>>

>> You might want to try brackup (http://code.google.com/p/brackup/). For

>> very

>> large trees of relatively small files it seems to significantly

>> out-perform

>> rsync-based backups. I've got brackup packages in my repository (see

>> http://www.openfusion.net/linux/openfusion_rpm_repository).

>>

>> Cheers,

>> Gavin

>>

> is it possible with "*brackup "* to back it up to a different server on the

> same lan* *instead of /backup *. Is there any documentation on the same .



rsync or rdiff-backup works on local disk or remote disk.(and other

backup methods too!)


Does*http://code.google.com/p/brackup/ *also work in on remote machines .*
--
Regards

Agnello D'souza



_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-24-2010, 10:58 AM
Eero Volotinen
 
Default Backup solution to backup /var/spool/imap above 150GB data

2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>
>
> On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen <eero.volotinen@iki.fi>
> wrote:
>>
>> 2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>> >
>> >
>> > On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au>
>> > wrote:
>> >>
>> >> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
>> >> > Dne 24.2.2010 10:00, Agnello George napsal(a):
>> >> > > We have *an issue with one of our clients , they have a mail server
>> >> > > with
>> >> > > the */var/spool/imap partition as 150 GB . They need to take
>> >> > > differential
>> >> > > backup on to /backup partition ( a different HDD *of total 250 GB
>> >> > > space ) *.
>> >> > > We have tried dar , rsync, rdiff and impasync . But its is not
>> >> > > sufficing the
>> >> > > need as to take a lot of time and consumes a lot of I/O .
>> >> > >
>> >> > > Is there any back up solution that you can think of , that can work
>> >> > > in
>> >> > > this
>> >> > > situation *- open source or proprietary
>> >> >
>> >> > Is seems to me, that you are using mbox format. So, differential
>> >> > backup
>> >> > is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> >> > backup differentially.
>> >>
>> >> rsync and rdiff should handle mbox format okay though. Though I agree
>> >> Maildir
>> >> is generally nicer for differential backups.
>> >>
>> >> Agnello, how long is "a lot of time"? A backup is always going to have
>> >> to
>> >> walk
>> >> the entire tree and checksum (or at least stat) every file, so there's
>> >> a
>> >> minimum
>> >> cost you're always going to have. How long does a 'find /var/spool/imap
>> >> -ls'
>> >> take, for instance?
>> >>
>> >> You might want to try brackup (http://code.google.com/p/brackup/). For
>> >> very
>> >> large trees of relatively small files it seems to significantly
>> >> out-perform
>> >> rsync-based backups. I've got brackup packages in my repository (see
>> >> http://www.openfusion.net/linux/openfusion_rpm_repository).
>> >>
>> >> Cheers,
>> >> Gavin
>> >>
>> > is it possible with "*brackup "* to back it up to a different server on
>> > the
>> > same lan* *instead of /backup *. Is there any documentation on the same
>> > .
>>
>> rsync or rdiff-backup works on local disk or remote disk.(and other
>> backup methods too!)


Summary from webpage: "Flexible backup tool. Slices, dices, encrypts,
and sprays across the net, notably to Amazon's S3. "


--
Eero
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-24-2010, 12:07 PM
Agnello George
 
Default Backup solution to backup /var/spool/imap above 150GB data

On Wed, Feb 24, 2010 at 6:19 PM, Karanbir Singh <mail-lists@karan.org> wrote:

On 02/24/2010 11:21 AM, Agnello George wrote:

> -rw------- 1 cyrus mail 4805500 Feb 24 16:23 4799.

> -rw------- 1 cyrus mail 22920 Feb 24 16:23 cyrus.cache

> -rw------- 1 cyrus mail 204 Dec 10 16:27 cyrus.header

> -rw------- 1 cyrus mail 896 Feb 24 16:23 cyrus.index

> -rw------- 1 cyrus mail 8669 Feb 24 11:28 cyrus.squat

>

> this is Just a very small user and a example



About 90% of your problem is already solved here, you are using cyrus

which has built in mail level replication. All you need to do is setup a

lvm volume away from this main store and run your mail replica over to

it. then just backup using whatever tools you want.



Free win you get is online failover, backup in whatever manner you want!

yes just spoke to my senior and confrimed that this was alreday tried out a delayed replication *is possible . but the current suitation is we need to* take backup* on *the same server on a different partition /backup * *


--
Regards
Agnello D'souza



_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 02-24-2010, 12:57 PM
Agnello George
 
Default Backup solution to backup /var/spool/imap above 150GB data

On Wed, Feb 24, 2010 at 7:05 PM, Gavin Carr <gavin@openfusion.com.au> wrote:

On Wed, Feb 24, 2010 at 05:06:08PM +0530, Agnello George wrote:

> On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen <eero.volotinen@iki.fi>wrote:

> > 2010/2/24 Agnello George <agnello.dsouza@gmail.com>:

> > > On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au>

> > wrote:

> > >> You might want to try brackup (http://code.google.com/p/brackup/). For

> > >> very

> > >> large trees of relatively small files it seems to significantly

> > >> out-perform

> > >> rsync-based backups. I've got brackup packages in my repository (see

> > >> http://www.openfusion.net/linux/openfusion_rpm_repository).

> > >>

> > >> Cheers,

> > >> Gavin

> > >>

> > > is it possible with " brackup " *to back it up to a different server on

> > > the same lan * instead of /backup *. Is there any documentation on the

> > > same .

> >

> > rsync or rdiff-backup works on local disk or remote disk.(and other

> > backup methods too!)

>

> Does http://code.google.com/p/brackup/ *also work in on remote machines .



Brackup will backup to local disk, or remotely to ftp, sftp, Amazon S3, or

Rackspace CloudFiles targets/servers. So yes, on a lan you can backup over

ftp or sftp just fine.



Re docs, install brackup, 'man Brackup::Manual::Overview'. I've also written

a few blog posts on it: http://www.openfusion.net/tags/brackup.



Cheers,

Gavin



_______________________________________________

CentOS mailing list

CentOS@centos.org

http://lists.centos.org/mailman/listinfo/centos


it will take me some time to try this .. will get back on its output !! .. thanks*
--
Regards
Agnello D'souza



_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 03:15 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org