RAID Cards performance issue
 

On Fri, Dec 05, 2008 at 12:24:49PM -0500, Nicolas Roussi wrote:
> Hi, I know that this topic is not the main subject matter of this list but I
> was wondering if anyone has any experience with RAID adapters and
> performance issues. I have already posted something on ubuntuforums.org (
> http://ubuntuforums.org/showthread.php?t=1001858). I would appreciate any
> input as to why the read speed on a 3ware 9650 RAID 5 would only be ~3MB/s
> and a PERC5 would be 10 times that much. Is there a way to troubleshoot this
> or maybe another benchmark for read and write speeds of my arrays?

Raid 5's terrible at a lot of things. RAID5's essentially a huge compromise,
and doesn't really do ANYTHING well. See:
http://en.wikipedia.org/wiki/RAID5#RAID_5

You'd be far, FAR better off with something like RAID0, or RAID10. You lose
half your drive space, but drives are cheap, and the performance gains are
huge.

Scott

--
Scott L. Balneaves | Today's weirdness is tomorrow's reason why.
Systems Department | -- Hunter S. Thompson
Legal Aid Manitoba |

--
edubuntu-users mailing list
edubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/edubuntu-users

RAID Cards performance issue
 

Scott Balneaves wrote:
> On Fri, Dec 05, 2008 at 12:24:49PM -0500, Nicolas Roussi wrote:
>> Hi, I know that this topic is not the main subject matter of this list but I
>> was wondering if anyone has any experience with RAID adapters and
>> performance issues. I have already posted something on ubuntuforums.org (
>> http://ubuntuforums.org/showthread.php?t=1001858). I would appreciate any
>> input as to why the read speed on a 3ware 9650 RAID 5 would only be ~3MB/s
>> and a PERC5 would be 10 times that much. Is there a way to troubleshoot this
>> or maybe another benchmark for read and write speeds of my arrays?

Are you running the latest firmware from 3ware?

>
> Raid 5's terrible at a lot of things. RAID5's essentially a huge compromise,
> and doesn't really do ANYTHING well. See:
> http://en.wikipedia.org/wiki/RAID5#RAID_5

Software raid5 get performance penalties but a hardware raid card with
sufficient cache memory (must be battery backed if you want to minimize
data loss) and processing power can do raid5 and perform as well as or
even better than raid10 depending on the number of drives involved.

>
> You'd be far, FAR better off with something like RAID0, or RAID10. You lose
> half your drive space, but drives are cheap, and the performance gains are
> huge.
>

RAID0? The guy has lost one drive already and you tell him RAID0 as a
choice for performance? Even if I don't care about the data, I would not
want to go through the trouble of replacing and recreating the array
everytime a disk decides it does not want to play anymore.

--
edubuntu-users mailing list
edubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/edubuntu-users

RAID Cards performance issue
 

On Mon, Dec 08, 2008 at 08:21:38AM +0800, Christopher Chan wrote:

> Software raid5 get performance penalties but a hardware raid card with
> sufficient cache memory (must be battery backed if you want to minimize
> data loss) and processing power can do raid5 and perform as well as or
> even better than raid10 depending on the number of drives involved.

No, RAID5's a compromise. If it's a compromise in software on the server,
it'll be just as much of a compromise on a dedicated controller, where
it'll be implemented in software running on the card's controller.
I've run tests myself on 3Ware controllers, and they are MUCH slower in
a RAID5 config than either a RAID1 or RAID10 config. This isn't a smack
at 3ware controllers: I use them myself, and they're great. Solid, dependable
raid controllers. It's just a limitation of RAID5. RAID5 tries to be
everything to everybody ("More Space!!" "Fault Tolerant!!" "Less Filling!!")
and in the end, doesn't really satisfy anyone.

> RAID0? The guy has lost one drive already and you tell him RAID0 as a
> choice for performance? Even if I don't care about the data, I would not
> want to go through the trouble of replacing and recreating the array
> everytime a disk decides it does not want to play anymore.

Sorry, I meant RAID1 not RAID0.

Cheers,
Scott

--
Scott L. Balneaves | Perilous to all of us are the devices of an art
Systems Department | deeper than we ourselves possess.
Legal Aid Manitoba | -- Gandalf [J.R.R. Tolkien, "Lord of the Rings"]

--
edubuntu-users mailing list
edubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/edubuntu-users

RAID Cards performance issue
 

Scott Balneaves wrote:
> On Mon, Dec 08, 2008 at 08:21:38AM +0800, Christopher Chan wrote:
>
>> Software raid5 get performance penalties but a hardware raid card with
>> sufficient cache memory (must be battery backed if you want to minimize
>> data loss) and processing power can do raid5 and perform as well as or
>> even better than raid10 depending on the number of drives involved.
>
> No, RAID5's a compromise. If it's a compromise in software on the server,
> it'll be just as much of a compromise on a dedicated controller, where
> it'll be implemented in software running on the card's controller.

When I say 'Software raid5 get performance penalties', that is in
relation to the performance you can get with hardware raid cards. That
is, in this day and age.

Popular hardware raid cards using the puny and useless Intel i960
processor a decade ago were absolutely creamed by software raid5 in
performance.

Even if you do have the processing power on the hardware raid card, you
still need sufficient buffering for the processor as the tests done by
the Gleb research group in the link below will tell you.

http://www.chemistry.wustl.edu/~gelb/castle_raid.html

The 3ware 850x series has no cache memory. They will therefore perform
absolutely poorly in raid5 mode. As the tests indicate, hardware raid5
performance is really poor on the 3ware 8506. However, when they
switched the card into JBOD mode and used Linux software raid5 to
implement raid5, the performance they got was comparable to the 3ware
hardware raid10 figures. If that is not an argument for raid5
performance, I don't know what is.

Today, bus traffic on the mainboard gives software raid5 performance
penalties compared to a hardware raid card doing raid5 since the disks
are directly connected to the raid board/processor. But the raid board
needs sufficient processing power and onboard cache to be able to pull
off the performance.

> I've run tests myself on 3Ware controllers, and they are MUCH slower in
> a RAID5 config than either a RAID1 or RAID10 config. This isn't a smack
> at 3ware controllers: I use them myself, and they're great. Solid, dependable
> raid controllers. It's just a limitation of RAID5. RAID5 tries to be
> everything to everybody ("More Space!!" "Fault Tolerant!!" "Less Filling!!")
> and in the end, doesn't really satisfy anyone.

Your problem is most probably because you did tests with the 3ware 750x
or 850x series. I know those boards suck at raid5 from personal
experience. I also know that newer 3ware boards with cache memory have
solved the raid5 performance disparity. I had a ten disk raid5 array on
a 3ware 9550 that was loaned to me and I used it as a mail queue and it
rocked. I was the MTA guy at Outblaze Ltd. at that time.

Here is a newer test that also uses six disks like the Gleb research
group did but with a different controller that has both the processing
power and sufficient cache memory.

http://www.linux.com/feature/140734

Not surprisingly, raid5 beat the pants off raid10. Why? For six disks,
on raid5, you have all six disks for input and output. On raid10, you
need to make three mirrors and so you are effectively reduced to three
disks for input and output. Given sufficient resources on the raid
board, raid5 with six spindles will beat raid0 with 3 spindles. Raid5 is
effectively raid0 + uber processing. Therefore, if 'uber processing'
keeps up, you are really doing an unfair raid0 with 6 logical disks
versus raid0 with 3 logical disks knockout.

You also get software raid performance comparison in that article and
you will see what I have said about software raid performance suffering
performance penalties proven except for one thing that I found most
interesting...software raid 5 got the best read performance and that
simply blows my mind away.

Enjoy.

--
edubuntu-users mailing list
edubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/edubuntu-users

RAID Cards performance issue
 

On Tue, Dec 09, 2008 at 09:40:44AM +0800, Christopher Chan wrote:
> Not surprisingly, raid5 beat the pants off raid10. Why? For six disks,
> on raid5, you have all six disks for input and output. On raid10, you
> need to make three mirrors and so you are effectively reduced to three
> disks for input and output.

Personally, I'd argue that's a broken RAID10, since, for reading at least, you
should be able to read off BOTH sides of the mirror, typically doubling your
read speed.

I've seen study after study indicating that, for most applications, RAID10
outperforms RAID5, and my own personal experience is that way as well. I
suspect we're quickly edging into the realm of religious discussion, so we'll
just leave things pat, and say, "Your Mileage May Vary, personal testing on
your hardware setup will determine the best course of action".

That being said, and since you're obviously up on RAID5, any suggestions for
the original poster?

Scott

--
Scott L. Balneaves | Grass is the forgiveness of nature - her constant
Systems Department | benediction. Forests decay, harvests perish, flowers
Legal Aid Manitoba | vanish, but grass is immortal. -- Brian Ingalls

--
edubuntu-users mailing list
edubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/edubuntu-users

RAID Cards performance issue
 

Scott Balneaves wrote:
> On Tue, Dec 09, 2008 at 09:40:44AM +0800, Christopher Chan wrote:
>> Not surprisingly, raid5 beat the pants off raid10. Why? For six disks,
>> on raid5, you have all six disks for input and output. On raid10, you
>> need to make three mirrors and so you are effectively reduced to three
>> disks for input and output.
>
> Personally, I'd argue that's a broken RAID10, since, for reading at least, you
> should be able to read off BOTH sides of the mirror, typically doubling your
> read speed.

AFAIK, that sort of thing seems to be only done on Linux software raid.
Not sure if any hardware raid card actually does the same thing as
reading off all disks in a quasi raid0 way.

>
> I've seen study after study indicating that, for most applications, RAID10
> outperforms RAID5, and my own personal experience is that way as well. I
> suspect we're quickly edging into the realm of religious discussion, so we'll
> just leave things pat, and say, "Your Mileage May Vary, personal testing on
> your hardware setup will determine the best course of action".

I was just pointing out that performance is not necessarily an issue
with raid5. A lot of factors can change the answer as to what is the
fastest raid implementation I will get on this set of hardware and
specifically those factors are whether there is sufficient processing
power and cache memory on the raid card. This is not an argument for
raid5 being the best solution there is.

A six disk raid5 array can take one disk going down. A six disk raid10
array can take up to 3 disks going down if you are fortunate (or maybe
not so fortunate) and in the tests in that Linux.com article, the raid10
array was doing better with three spindles than the raid5 array with six
spindles comparatively speaking. Like you said, it is a compromise.
3ware raid10/raid1 sync/rebuild code is very good as they keep markers
so you would not necessarily have to resync all data in a mirror. I am
not sure if Linux md raid1 has caught on that count yet.

>
> That being said, and since you're obviously up on RAID5, any suggestions for
> the original poster?

I am not up on raid5. Man, I stay away from it! That ten disk raid5
array on a 3ware 9550 I used as a mail queue was given to me not only to
handle a temporary issue but also to silence me on my criticism of
raid5. That was when I learnt that the lack of cache memory was a
primary reason why a 3ware 850x board has really poor performance when
doing raid5. I would not touch raid5 with Linux software raid. Things
have probably changed now though.

As for the OP, I asked the OP whether he was running the latest firmware
from 3ware. Then there is whether he is using the 3ware board on a
tested motherboard as listed on 3ware's website. After that it is try
the latest driver from 3ware...all the general procedures one can take
to try to find the problem. I am no longer in a high i/o required
environment and so I have not touched a 3ware or any other raid card in
a few years. You won't be getting any special insight from me on
problems with anything newer than them 750x/850x 3ware boards.

--
edubuntu-users mailing list
edubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/edubuntu-users

RAID Cards performance issue
 

Thanks for all the replies.
Christopher, I am not using the latest firmware from 3ware but I enabled* "Write Cache" and set the storsave profile to "Performance" and now I am getting ~14 MB/s instead of 3. I will try to update the firmware over the weekend and 3ware support told me to update the codeset and drivers. Right now I cant do that since I messed up my installation trying to install the realtek audio drivers and nobody can log in. As soon as I fix that I will try and update the drivers of the raid card. Will post with* the results.


Thanks again.
--
Nicolas Roussi

--
edubuntu-users mailing list
edubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/edubuntu-users