Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   CentOS (http://www.linux-archive.org/centos/)
-   -   RAID on Email Server (http://www.linux-archive.org/centos/174184-raid-email-server.html)

John R Pierce 10-10-2008 05:52 PM

RAID on Email Server
 
Matt wrote:

I have an email server running Exim, Dovecot, Spamassassin, Clam, etc.
on Centos 4.x 32bit. On occasion I have disk I/O problems. Its
handling several domains and alot of email. Its currently on a single
SATA drive. I am thinking of moving too 3 drives with RAID 1 for
redundancy. RAID 1 will help me on reads but do nothing on writes as
I understand. I am thinking the majority of my I/O is read though
not? I imagine quotta checks and all that being done and everytime a
user checks there email every message in the inbox must be read.

I guess I am asking if RAID 1 will help my I/O problem much?

[root@server ~]# w
12:04:02 up 2:01, 1 user, load average: 7.02, 7.47, 11.84
USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
root pts/0 208.92.169.4.ppp 11:25 0.00s 0.02s 0.00s w
[root@server ~]# vmstat
procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
r b swpd free buff cache si so bi bo in cs us sy id wa
0 2 0 1558496 456916 1087224 0 0 198 749 795 537 18 4 27 50

The above is when its running pretty good.



can you paste the output of `iostat -x 5 5` while its busy ? this will
show definateively how busy your disks are...

the first sample from vmstat, iostat, etc only shows the AVERAGE since
the system booted. the 2nd and beyond samples are the average over the
time intervals specified (5 5 means 5 seconds, 5 samples)


oh, if you don't have iostat, its part of package sysstat, so `yum
install sysstat`



_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Matt 10-10-2008 06:13 PM

RAID on Email Server
 
>> I have an email server running Exim, Dovecot, Spamassassin, Clam, etc.
>> on Centos 4.x 32bit. On occasion I have disk I/O problems. Its
>> handling several domains and alot of email. Its currently on a single
>> SATA drive. I am thinking of moving too 3 drives with RAID 1 for
>> redundancy. RAID 1 will help me on reads but do nothing on writes as
>> I understand. I am thinking the majority of my I/O is read though
>> not? I imagine quotta checks and all that being done and everytime a
>> user checks there email every message in the inbox must be read.
>>
>> I guess I am asking if RAID 1 will help my I/O problem much?
>>
>> [root@server ~]# w
>> 12:04:02 up 2:01, 1 user, load average: 7.02, 7.47, 11.84
>> USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
>> root pts/0 208.92.169.4.ppp 11:25 0.00s 0.02s 0.00s w
>> [root@server ~]# vmstat
>> procs -----------memory---------- ---swap-- -----io---- --system--
>> ----cpu----
>> r b swpd free buff cache si so bi bo in cs us sy
>> id wa
>> 0 2 0 1558496 456916 1087224 0 0 198 749 795 537 18 4
>> 27 50
>>
>> The above is when its running pretty good.
>>
>
> can you paste the output of `iostat -x 5 5` while its busy ? this will
> show definateively how busy your disks are...
> the first sample from vmstat, iostat, etc only shows the AVERAGE since the
> system booted. the 2nd and beyond samples are the average over the time
> intervals specified (5 5 means 5 seconds, 5 samples)
>
> oh, if you don't have iostat, its part of package sysstat, so `yum install
> sysstat`

Right now its running pretty good but here it is.

[root@server ~]# w
13:11:02 up 3:08, 2 users, load average: 4.03, 5.71, 5.51

avg-cpu: %user %nice %sys %iowait %idle
2.80 0.00 1.60 58.10 37.50

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s
avgrq-sz avgqu-sz await svctm %util
sda 0.60 142.80 67.20 170.20 678.40 2292.80 339.20
1146.40 12.52 118.53 615.66 4.21 99.92
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sda2 0.60 142.80 67.20 170.20 678.40 2292.80 339.20
1146.40 12.52 118.53 615.66 4.21 99.92
sdb 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
sdb1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00
dm-0 0.00 0.00 67.40 286.20 678.40 2289.60 339.20
1144.80 8.39 163.02 582.40 2.83 99.94
dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
0.00 0.00 0.00 0.00 0.00


sar also.

10:02:56 AM LINUX RESTART

10:10:04 AM CPU %user %nice %system %iowait %idle
10:20:01 AM all 13.87 0.00 2.45 59.20 24.49
10:30:03 AM all 22.26 0.00 3.68 53.51 20.54
10:40:01 AM all 20.58 0.00 3.78 55.40 20.24
10:50:04 AM all 22.20 0.00 5.23 52.67 19.91
11:00:05 AM all 21.58 0.00 4.72 51.81 21.89
11:10:01 AM all 18.14 0.00 4.52 56.91 20.43
11:20:03 AM all 21.42 0.00 4.59 47.20 26.79
11:30:02 AM all 19.22 0.00 4.48 53.86 22.44
11:40:04 AM all 17.59 0.00 4.82 51.61 25.98
11:50:02 AM all 15.88 0.00 4.67 45.74 33.71
12:00:01 PM all 13.32 0.00 2.73 25.72 58.23
12:10:02 PM all 16.98 0.00 4.54 53.14 25.35
12:20:01 PM all 17.31 0.00 3.45 47.80 31.44
12:30:01 PM all 19.45 0.00 4.08 36.47 40.00
12:40:01 PM all 13.79 0.00 4.39 44.83 36.99
12:50:01 PM all 12.18 0.00 3.93 30.16 53.73
01:00:01 PM all 11.53 0.00 2.38 20.96 65.12
Average: all 17.49 0.00 4.03 46.29 32.19

A while after the reboot it straightened its self out. Yesterday "w"
was indicatining load average of like 120 or more at times. Today
after reboot all is good.

Matt
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Jerry Franz 10-10-2008 06:21 PM

RAID on Email Server
 
Matt wrote:

Right now its running pretty good but here it is.
Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s
avgrq-sz avgqu-sz await svctm %util
sda 0.60 142.80 67.20 170.20 678.40 2292.80 339.20
1146.40 12.52 118.53 615.66 4.21 99.92



You have about 3 times more writing going on than reading. RAID1 isn't
going to do much for you. You might want to try four disks in a RAID10
instead. And put your mail folders and spool on partitions mounted using
'noatime'.


--
Benjamin Franz
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Rainer Duffner 10-10-2008 09:42 PM

RAID on Email Server
 
Matt schrieb:
> I have an email server running Exim, Dovecot, Spamassassin, Clam, etc.
> on Centos 4.x 32bit. On occasion I have disk I/O problems. Its
> handling several domains and alot of email. Its currently on a single
> SATA drive. I am thinking of moving too 3 drives with RAID 1 for
> redundancy. RAID 1 will help me on reads but do nothing on writes as
> I understand. I am thinking the majority of my I/O is read though
> not? I imagine quotta checks and all that being done and everytime a
> user checks there email every message in the inbox must be read.
>
> I guess I am asking if RAID 1 will help my I/O problem much?
>
> [root@server ~]# w
> 12:04:02 up 2:01, 1 user, load average: 7.02, 7.47, 11.84
> USER TTY FROM LOGIN@ IDLE JCPU PCPU WHAT
> root pts/0 208.92.169.4.ppp 11:25 0.00s 0.02s 0.00s w
> [root@server ~]# vmstat
> procs -----------memory---------- ---swap-- -----io---- --system-- ----cpu----
> r b swpd free buff cache si so bi bo in cs us sy id wa
> 0 2 0 1558496 456916 1087224 0 0 198 749 795 537 18 4 27 50
>
> The above is when its running pretty good.
>
> Matt
> _______________________________________________
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos
>


How many concurrent users?
If you stay with SATA, you will probably have to increase the amount of
disks in you storage-array (depending on the number of users).
A SATA disk does only 80 I/Os per second or so - and clamav+spamassassin
alone will consume a lot of these.
Try to put their working-directories on a swap-backed tmpfs.




cheers,
Rainer








_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Karanbir Singh 10-10-2008 11:52 PM

RAID on Email Server
 
Matt wrote:

I have an email server running Exim, Dovecot, Spamassassin, Clam, etc.


Firstly, I'd drop dovecot completely, cyrus-imapd has, for me, been a
lot faster and better optimised for situations where you have more than
a few hand full of users.


Secondly, exim configs out of the box on CentOS are *really* not
optimized for performance at all. If you handle more than a few thousand
emails an hour, I dont be surprised if exim is to blame for jamming your
i/o pipe just trying to work out which email to attempt delivery for next.



--
Karanbir Singh : http://www.karan.org/ : 2522219@icq
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Guy Boisvert 10-11-2008 02:32 AM

RAID on Email Server
 
Matt wrote:


Right now its running pretty good but here it is.

10:10:04 AM CPU %user %nice %system %iowait %idle
10:20:01 AM all 13.87 0.00 2.45 59.20 24.49
10:30:03 AM all 22.26 0.00 3.68 53.51 20.54
10:40:01 AM all 20.58 0.00 3.78 55.40 20.24


[ Stuff Deleted ... ]


01:00:01 PM all 11.53 0.00 2.38 20.96 65.12
Average: all 17.49 0.00 4.03 46.29 32.19

A while after the reboot it straightened its self out. Yesterday "w"
was indicatining load average of like 120 or more at times. Today
after reboot all is good.

Matt


Hi Matt,

Your %iowait seems high.

I had %iowait comparable to you with a single 200 Gigs 7200 RPM IDE drive.

Now we've upgraded this server: Opteron 2216 with 4 Gigs RAM, CentOS 64
v5.2) and an Adaptec 3405 plus 4 x 73 Gigs Seagate 15K RPM (RAID 10).
No more %iowait! I benched it at about 140 MB/s and random r/w are very
good on these drives.


We have 55 employees running Outlook connecting to Communicate with
MAPI. The dataset is about 100 Gigs. Communigate use MBOX files for
storage.



Guy Boisvert, ing.
IngTegration inc.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Matt 10-11-2008 02:46 AM

RAID on Email Server
 
>> I have an email server running Exim, Dovecot, Spamassassin, Clam, etc.
>
> Firstly, I'd drop dovecot completely, cyrus-imapd has, for me, been a lot
> faster and better optimised for situations where you have more than a few
> hand full of users.
>
> Secondly, exim configs out of the box on CentOS are *really* not optimized
> for performance at all. If you handle more than a few thousand emails an
> hour, I dont be surprised if exim is to blame for jamming your i/o pipe just
> trying to work out which email to attempt delivery for next.

This is a Directadmin type server. So I really have no options on
changing services. I am running exim version 4.60 and I wander if
updating to 4.69 may help out.

Matt
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Luke S Crawford 10-21-2008 12:43 AM

RAID on Email Server
 
Guy Boisvert <boisvert.guy@videotron.ca> writes:
> Your %iowait seems high.
>
> I had %iowait comparable to you with a single 200 Gigs 7200 RPM IDE drive.
>
> Now we've upgraded this server: Opteron 2216 with 4 Gigs RAM, CentOS
> 64 v5.2) and an Adaptec 3405 plus 4 x 73 Gigs Seagate 15K RPM (RAID
> 10). No more %iowait! I benched it at about 140 MB/s and random r/w
> are very good on these drives.

Yeah. without dangerous write-back caching, as far as performance
(as opposed to space) goes, expensive SAS disks (or even the expensive 15K
sata disks) will get you a lot more than any amount of RAID on the cheap
7500RPM sata disks.

Really, once you have the 15K disks, I'm sure a mirror would be fine, if
73G is enough.

The new (and really expensive) solid-state flash drives are also really good
for reducing your iowait.

The other thing to check is vmstat- if you have lots of si and so, well,
your disk is slow 'cause you are swapping a lot. add more ram and the problem
goes away.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


All times are GMT. The time now is 01:29 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.