Backup solution to backup /var/spool/imap above 150GB data
2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
> Hi
>
> We have* an issue with one of our clients , they have a mail server with
> the* /var/spool/imap partition as 150 GB . They need to take differential
> backup on to /backup partition ( a different HDD* of total 250 GB space )* .
> We have tried dar , rsync, rdiff and impasync . But its is not sufficing the
> need as to take a lot of time and consumes a lot of I/O .
So, you need to add more disk i/o? (so, add some disk space with faster raid?)
Take a look at: http://rdiff-backup.nongnu.org/
--
Eero
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
02-24-2010, 09:33 AM
Agnello George
Backup solution to backup /var/spool/imap above 150GB data
On Wed, Feb 24, 2010 at 3:08 PM, David Hrbáč <hrbac.conf@seznam.cz> wrote:
Dne 24.2.2010 10:00, Agnello George napsal(a):
> Hi
>
> We have *an issue with one of our clients , they have a mail server with
> the */var/spool/imap partition as 150 GB . They need to take differential
> backup on to /backup partition ( a different HDD *of total 250 GB space ) *.
> We have tried dar , rsync, rdiff and impasync . But its is not sufficing the
> need as to take a lot of time and consumes a lot of I/O .
>
> Is there any back up solution that you can think of , that can work in this
> situation *- open source or proprietary
Is seems to me, that you are using mbox format. So, differential backup
is hard to achieve. Migrate to maildir, every mail is a file, easy to
backup differentially.
David
backup directory structure is /var/spool/imap/a /adomain.com/a/agnello^dsouza/ *
--
Regards
Agnello D'souza
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
02-24-2010, 09:38 AM
Eero Volotinen
Backup solution to backup /var/spool/imap above 150GB data
2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>
>
> On Wed, Feb 24, 2010 at 3:08 PM, David Hrbáč <hrbac.conf@seznam.cz> wrote:
>>
>> Dne 24.2.2010 10:00, Agnello George napsal(a):
>> > Hi
>> >
>> > We have *an issue with one of our clients , they have a mail server with
>> > the */var/spool/imap partition as 150 GB . They need to take
>> > differential
>> > backup on to /backup partition ( a different HDD *of total 250 GB space
>> > ) *.
>> > We have tried dar , rsync, rdiff and impasync . But its is not sufficing
>> > the
>> > need as to take a lot of time and consumes a lot of I/O .
>> >
>> > Is there any back up solution that you can think of , that can work in
>> > this
>> > situation *- open source or proprietary
>>
>> Is seems to me, that you are using mbox format. So, differential backup
>> is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> backup differentially.
>> David
>
> backup directory structure is /var/spool/imap/a
> /adomain.com/a/agnello^dsouza/
>
Well, does that directory contains one file or lot of files ?
Usually maildir structure is like dir++/tmp/current/new directories
and each message is in own file on mailbox all files are inside one
file.
--
Eero
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
02-24-2010, 10:21 AM
Agnello George
Backup solution to backup /var/spool/imap above 150GB data
2010/2/24 Eero Volotinen <eero.volotinen@iki.fi>
2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>
>
> On Wed, Feb 24, 2010 at 3:08 PM, David Hrbáč <hrbac.conf@seznam.cz> wrote:
>>
>> Dne 24.2.2010 10:00, Agnello George napsal(a):
>> > Hi
>> >
>> > We have *an issue with one of our clients , they have a mail server with
>> > the */var/spool/imap partition as 150 GB . They need to take
>> > differential
>> > backup on to /backup partition ( a different HDD *of total 250 GB space
>> > ) *.
>> > We have tried dar , rsync, rdiff and impasync . But its is not sufficing
>> > the
>> > need as to take a lot of time and consumes a lot of I/O .
>> >
>> > Is there any back up solution that you can think of , that can work in
>> > this
>> > situation *- open source or proprietary
>>
>> Is seems to me, that you are using mbox format. So, differential backup
>> is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> backup differentially.
>> David
>
> backup directory structure is /var/spool/imap/a
> /adomain.com/a/agnello^dsouza/
>
Well, does that directory contains one file or lot of files ?
Usually maildir structure is like dir++/tmp/current/new directories
and each message is in own file on mailbox all files are inside one
file.
its in a maildir format and the structure slightly * different from what i mentioned earlier :*root@server1 ~]# ls -la /var/spool/imap/a/user/ajay/
total 5180
-rw------- 1 cyrus mail 34616 Feb 24 16:02 4790.
-rw------- 1 cyrus mail 4490 Feb 24 16:03 4791.
-rw------- 1 cyrus mail 199253 Feb 24 16:07 4792.
-rw------- 1 cyrus mail 22930 Feb 24 16:09 4793.
-rw------- 1 cyrus mail 8485 Feb 24 16:11 4794.
-rw------- 1 cyrus mail 12664 Feb 24 16:13 4795.
-rw------- 1 cyrus mail 4296 Feb 24 16:13 4796.
-rw------- 1 cyrus mail 5337 Feb 24 16:15 4797.
-rw------- 1 cyrus mail 111030 Feb 24 16:21 4798.
-rw------- 1 cyrus mail 4805500 Feb 24 16:23 4799.
-rw------- 1 cyrus mail 22920 Feb 24 16:23 cyrus.cache
-rw------- 1 cyrus mail 204 Dec 10 16:27 cyrus.header
-rw------- 1 cyrus mail 896 Feb 24 16:23 cyrus.index
-rw------- 1 cyrus mail 8669 Feb 24 11:28 cyrus.squat
this is Just a very small user and a example**
--
Regards
Agnello D'souza
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
02-24-2010, 10:24 AM
Agnello George
Backup solution to backup /var/spool/imap above 150GB data
On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au> wrote:
On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
> Dne 24.2.2010 10:00, Agnello George napsal(a):
> > We have *an issue with one of our clients , they have a mail server with
> > the */var/spool/imap partition as 150 GB . They need to take differential
> > backup on to /backup partition ( a different HDD *of total 250 GB space ) *.
> > We have tried dar , rsync, rdiff and impasync . But its is not sufficing the
> > need as to take a lot of time and consumes a lot of I/O .
> >
> > Is there any back up solution that you can think of , that can work in this
> > situation *- open source or proprietary
>
> Is seems to me, that you are using mbox format. So, differential backup
> is hard to achieve. Migrate to maildir, every mail is a file, easy to
> backup differentially.
rsync and rdiff should handle mbox format okay though. Though I agree Maildir
is generally nicer for differential backups.
Agnello, how long is "a lot of time"? A backup is always going to have to walk
the entire tree and checksum (or at least stat) every file, so there's a minimum
cost you're always going to have. How long does a 'find /var/spool/imap -ls'
take, for instance?
You might want to try brackup (http://code.google.com/p/brackup/). For very
large trees of relatively small files it seems to significantly out-perform
rsync-based backups. I've got brackup packages in my repository (see
is it possible with "*brackup "* to back it up to a different server on the same lan* *instead of /backup *. Is there any documentation on the same .
--
Regards
Agnello D'souza
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
02-24-2010, 10:27 AM
Eero Volotinen
Backup solution to backup /var/spool/imap above 150GB data
2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>
>
> On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au> wrote:
>>
>> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
>> > Dne 24.2.2010 10:00, Agnello George napsal(a):
>> > > We have *an issue with one of our clients , they have a mail server
>> > > with
>> > > the */var/spool/imap partition as 150 GB . They need to take
>> > > differential
>> > > backup on to /backup partition ( a different HDD *of total 250 GB
>> > > space ) *.
>> > > We have tried dar , rsync, rdiff and impasync . But its is not
>> > > sufficing the
>> > > need as to take a lot of time and consumes a lot of I/O .
>> > >
>> > > Is there any back up solution that you can think of , that can work in
>> > > this
>> > > situation *- open source or proprietary
>> >
>> > Is seems to me, that you are using mbox format. So, differential backup
>> > is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> > backup differentially.
>>
>> rsync and rdiff should handle mbox format okay though. Though I agree
>> Maildir
>> is generally nicer for differential backups.
>>
>> Agnello, how long is "a lot of time"? A backup is always going to have to
>> walk
>> the entire tree and checksum (or at least stat) every file, so there's a
>> minimum
>> cost you're always going to have. How long does a 'find /var/spool/imap
>> -ls'
>> take, for instance?
>>
>> You might want to try brackup (http://code.google.com/p/brackup/). For
>> very
>> large trees of relatively small files it seems to significantly
>> out-perform
>> rsync-based backups. I've got brackup packages in my repository (see
>> http://www.openfusion.net/linux/openfusion_rpm_repository).
>>
>> Cheers,
>> Gavin
>>
> is it possible with "*brackup "* to back it up to a different server on the
> same lan* *instead of /backup *. Is there any documentation on the same .
rsync or rdiff-backup works on local disk or remote disk.(and other
backup methods too!)
--
Eero
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
02-24-2010, 10:36 AM
Agnello George
Backup solution to backup /var/spool/imap above 150GB data
On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen <eero.volotinen@iki.fi> wrote:
2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>
>
> On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au> wrote:
>>
>> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
>> > Dne 24.2.2010 10:00, Agnello George napsal(a):
>> > > We have *an issue with one of our clients , they have a mail server
>> > > with
>> > > the */var/spool/imap partition as 150 GB . They need to take
>> > > differential
>> > > backup on to /backup partition ( a different HDD *of total 250 GB
>> > > space ) *.
>> > > We have tried dar , rsync, rdiff and impasync . But its is not
>> > > sufficing the
>> > > need as to take a lot of time and consumes a lot of I/O .
>> > >
>> > > Is there any back up solution that you can think of , that can work in
>> > > this
>> > > situation *- open source or proprietary
>> >
>> > Is seems to me, that you are using mbox format. So, differential backup
>> > is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> > backup differentially.
>>
>> rsync and rdiff should handle mbox format okay though. Though I agree
>> Maildir
>> is generally nicer for differential backups.
>>
>> Agnello, how long is "a lot of time"? A backup is always going to have to
>> walk
>> the entire tree and checksum (or at least stat) every file, so there's a
>> minimum
>> cost you're always going to have. How long does a 'find /var/spool/imap
>> -ls'
>> take, for instance?
>>
>> You might want to try brackup (http://code.google.com/p/brackup/). For
>> very
>> large trees of relatively small files it seems to significantly
>> out-perform
>> rsync-based backups. I've got brackup packages in my repository (see
> is it possible with "*brackup "* to back it up to a different server on the
> same lan* *instead of /backup *. Is there any documentation on the same .
rsync or rdiff-backup works on local disk or remote disk.(and other
backup methods too!)
Does*http://code.google.com/p/brackup/ *also work in on remote machines .*
--
Regards
Agnello D'souza
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
02-24-2010, 10:58 AM
Eero Volotinen
Backup solution to backup /var/spool/imap above 150GB data
2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>
>
> On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen <eero.volotinen@iki.fi>
> wrote:
>>
>> 2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
>> >
>> >
>> > On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au>
>> > wrote:
>> >>
>> >> On Wed, Feb 24, 2010 at 10:38:32AM +0100, David Hrbáč wrote:
>> >> > Dne 24.2.2010 10:00, Agnello George napsal(a):
>> >> > > We have *an issue with one of our clients , they have a mail server
>> >> > > with
>> >> > > the */var/spool/imap partition as 150 GB . They need to take
>> >> > > differential
>> >> > > backup on to /backup partition ( a different HDD *of total 250 GB
>> >> > > space ) *.
>> >> > > We have tried dar , rsync, rdiff and impasync . But its is not
>> >> > > sufficing the
>> >> > > need as to take a lot of time and consumes a lot of I/O .
>> >> > >
>> >> > > Is there any back up solution that you can think of , that can work
>> >> > > in
>> >> > > this
>> >> > > situation *- open source or proprietary
>> >> >
>> >> > Is seems to me, that you are using mbox format. So, differential
>> >> > backup
>> >> > is hard to achieve. Migrate to maildir, every mail is a file, easy to
>> >> > backup differentially.
>> >>
>> >> rsync and rdiff should handle mbox format okay though. Though I agree
>> >> Maildir
>> >> is generally nicer for differential backups.
>> >>
>> >> Agnello, how long is "a lot of time"? A backup is always going to have
>> >> to
>> >> walk
>> >> the entire tree and checksum (or at least stat) every file, so there's
>> >> a
>> >> minimum
>> >> cost you're always going to have. How long does a 'find /var/spool/imap
>> >> -ls'
>> >> take, for instance?
>> >>
>> >> You might want to try brackup (http://code.google.com/p/brackup/). For
>> >> very
>> >> large trees of relatively small files it seems to significantly
>> >> out-perform
>> >> rsync-based backups. I've got brackup packages in my repository (see
>> >> http://www.openfusion.net/linux/openfusion_rpm_repository).
>> >>
>> >> Cheers,
>> >> Gavin
>> >>
>> > is it possible with "*brackup "* to back it up to a different server on
>> > the
>> > same lan* *instead of /backup *. Is there any documentation on the same
>> > .
>>
>> rsync or rdiff-backup works on local disk or remote disk.(and other
>> backup methods too!)
Summary from webpage: "Flexible backup tool. Slices, dices, encrypts,
and sprays across the net, notably to Amazon's S3. "
--
Eero
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
02-24-2010, 12:07 PM
Agnello George
Backup solution to backup /var/spool/imap above 150GB data
On Wed, Feb 24, 2010 at 6:19 PM, Karanbir Singh <mail-lists@karan.org> wrote:
On 02/24/2010 11:21 AM, Agnello George wrote:
> -rw------- 1 cyrus mail 4805500 Feb 24 16:23 4799.
> -rw------- 1 cyrus mail 22920 Feb 24 16:23 cyrus.cache
> -rw------- 1 cyrus mail 204 Dec 10 16:27 cyrus.header
> -rw------- 1 cyrus mail 896 Feb 24 16:23 cyrus.index
> -rw------- 1 cyrus mail 8669 Feb 24 11:28 cyrus.squat
>
> this is Just a very small user and a example
About 90% of your problem is already solved here, you are using cyrus
which has built in mail level replication. All you need to do is setup a
lvm volume away from this main store and run your mail replica over to
it. then just backup using whatever tools you want.
Free win you get is online failover, backup in whatever manner you want!
yes just spoke to my senior and confrimed that this was alreday tried out a delayed replication *is possible . but the current suitation is we need to* take backup* on *the same server on a different partition /backup * *
--
Regards
Agnello D'souza
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
02-24-2010, 12:57 PM
Agnello George
Backup solution to backup /var/spool/imap above 150GB data
On Wed, Feb 24, 2010 at 7:05 PM, Gavin Carr <gavin@openfusion.com.au> wrote:
On Wed, Feb 24, 2010 at 05:06:08PM +0530, Agnello George wrote:
> On Wed, Feb 24, 2010 at 4:57 PM, Eero Volotinen <eero.volotinen@iki.fi>wrote:
> > 2010/2/24 Agnello George <agnello.dsouza@gmail.com>:
> > > On Wed, Feb 24, 2010 at 4:13 PM, Gavin Carr <gavin@openfusion.com.au>
> > wrote:
> > >> You might want to try brackup (http://code.google.com/p/brackup/). For
> > >> very
> > >> large trees of relatively small files it seems to significantly
> > >> out-perform
> > >> rsync-based backups. I've got brackup packages in my repository (see
Backup solution to backup /var/spool/imap above 150GB data
*
