FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 06-07-2008, 09:05 PM
"Mag Gam"
 
Default RAID for large disks

Thanks for the responses all.

I want RAID 5 but without mirroring. The data is important but not that important.
I am planning to use LVM.

If the controller creates a stripe size of 16k, do I need to do anything special with physical extends (in pvcreate or vgcreate) ?

Do I need to do anything specific when creating a LV? I plan on striping my LV to create extra spindles. Do I need to create my ext3 filesystem with any particular settings? I am looking for a optimal tuning guide with emphasis on performace versus redudancy.




On Sat, Jun 7, 2008 at 4:27 PM, Andrew M.A. Cater <amacater@galactic.demon.co.uk> wrote:

On Sat, Jun 07, 2008 at 12:52:24PM -0400, Mag Gam wrote:

> With the RAID array I am planning to use RAID 5 so my data is still

> protected. My confusion is going with RAID striping (picking the right

> size). Also, Does the filesystem layout need to be specific when I do

> striping? If I am using 128k stripes, should I start my filesystem on 129k

> and end with max-(128+1k)?

>



You have four or five considerations.



You mentioned you were going to use your 12 disks as two RAID arrays.



If one is going to be for your data and one for a backup of that data -

2 x RAID 5 and then RAID1 [5 x 500 = ~2.5TB mirrored].



If you need maximum data storage - all your disks in one array in RAID

5.



11 x 500, one spare - 5.5TB but you rely on the spare



If you need data resilience - all your disks in one array in RAID 6 or

RAID 10



Hardware RAID control is lovely - but you may need battery backup on

some cards to avoid problems on delayed writes. Hardware RAID control

also ties you to one manufacturer's cards and/or recovery utilities if a

RAID fails and you have to recover data.



If you go the hardware route: take the card defaults.



Linux mdadm works well and, under some circumstances, can approach the

performance of a dedicated hardware RAID card - disks can be swapped

into any Linux box to recover the RAID.



You can then add LVM on top.



HTH,



Andy





--

To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org

with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 06-07-2008, 09:52 PM
Ron Johnson
 
Default RAID for large disks

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/07/08 11:15, Ron Johnson wrote:
[snip]
>
> Otherwise, use RAID 0, 10, 0+1 or 5.

My mistake: not RAID 0, but RAID 1.

- --
Ron Johnson, Jr.
Jefferson LA USA

"Kittens give Morbo gas. In lighter news, the city of New New
York is doomed."
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFISwMtS9HxQb37XmcRAvOVAKDhG4KyiUAmmckRFK41Wq qrqspcIACgvw0x
8nBmVNCqJboNcsQU6/AKIkk=
=j/5L
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 06-07-2008, 09:56 PM
Ron Johnson
 
Default RAID for large disks

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/07/08 16:05, Mag Gam wrote:
> Thanks for the responses all.
>
> I want RAID 5 but without mirroring. The data is important but not that
> important.

Ummm, there is NO mirroring in RAID 5. Never has been.

> I am planning to use LVM.
>
> If the controller creates a stripe size of 16k, do I need to do anything
> special with physical extends (in pvcreate or vgcreate) ?

No. LVM is at a completely separate (and "higher) level than the
h/w RAID controller.

> Do I need to do anything specific when creating a LV? I plan on striping
> my LV to create extra spindles. Do I need to create my ext3 filesystem
> with any particular settings?

No. ext3 is at a higher level than LVM.

> I am looking for a optimal tuning guide
> with emphasis on performace versus redudancy.

Read speed or write speed?

The tenor of your questions and comments indicates that you *really*
need to learn more about the different types of RAID.

If you want:
- - best R/W speed and no redundancy, use RAID 0 (striping),
- - best R/W speed, full redundancy, use RAID 10 or RAID 0+1.
- - worst W speed, great R speed and low "RAID overhead", use RAID 5.

- --
Ron Johnson, Jr.
Jefferson LA USA

"Kittens give Morbo gas. In lighter news, the city of New New
York is doomed."
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFISwPwS9HxQb37XmcRAg5bAJ9U94WrcPbtwJsSmxIaoV W72o+OFQCfUPYA
mkOuxzaRoWAjGkaLBSqWIVw=
=VyM9
-----END PGP SIGNATURE-----


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 06-07-2008, 10:19 PM
"Damon L. Chesser"
 
Default RAID for large disks

On Sat, 2008-06-07 at 17:05 -0400, Mag Gam wrote:
> Thanks for the responses all.
>
> I want RAID 5 but without mirroring. The data is important but not
> that important.
> I am planning to use LVM.
>
> If the controller creates a stripe size of 16k, do I need to do
> anything special with physical extends (in pvcreate or vgcreate) ?
> Do I need to do anything specific when creating a LV? I plan on
> striping my LV to create extra spindles. Do I need to create my ext3
> filesystem with any particular settings? I am looking for a optimal
> tuning guide with emphasis on performace versus redudancy.

If you use hardware raid, the bios of the box, the OS and all
applications will not be aware of any of the underlining RAID setup
parameters. If you use two RAID5, your OS will only know it will have
sda and sdb. On top of this you will put your file system (or lvm) and
none of these things will know anything about the hardware. Nothing.
Not how many HDs you have, not what scsi bus they are on, not how fast
they are. Not even that one of them dies (all though I have seen scsi
errors show up in /var/log/messages for one HD failure in a hdwr RAID,
go figure). All you need to know at the level of the fs/LVM is you will
have two hd. You can not have /boot on lvm (technically you can, but
not the version used by etch, lenny or sid) so you will need at least
one non-lvm partition.

--
Damon L. Chesser
damon@damtek.com
http://www.linkedin.com/in/dchesser
 
Old 06-08-2008, 12:03 AM
Richard Hector
 
Default RAID for large disks

On Sat, 2008-06-07 at 11:15 -0500, Ron Johnson wrote:

> Striping is a GREAT idea IFF you want serious speed, but don't care
> about your data. If one of the disks goes flaky, *all* the data on
> the stripeset goes poof.
>
> So, *never* use striping on a production server!! Unless you hate
> the company, are vindictive, and are about to leave.

'Production' can be many things. There are many applications that could
well be 'production', but which can recreate the data relatively easily,
which would be reasonable candidates for striping. A squid proxy cache
would be one; an intermediate state in some file conversion process
might be another.

Richard



--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 06-08-2008, 12:04 AM
"Mag Gam"
 
Default RAID for large disks

Thanks thats the exact same question I have.

Does this page, http://www.redhat.com/archives/linux-lvm/2006-October/msg00014.html, hold any validity? The poster makes a good argument, but by seeing Damon's response it makes no sense to go thru the trouble. I would be willing to try this if I get some assistance...







On Sat, Jun 7, 2008 at 7:39 PM, Brian McKee <brian.mckee@gmail.com> wrote:

On Sat, Jun 7, 2008 at 8:27 AM, Mag Gam <magawake@gmail.com> wrote:

>

> I have a RAID controller with 256MB of on board cache and its connected to

> 12 500GB SATA disks. I am planning to create 2 RAID groups (6 disks each),

> but I don't know what is the optimal stripe size should be.

>

> Also, once I stripe on the RAID controller I am planning to use LVM. Is

> striping a good idea? What should I consider for the filesystem?



I think you are confusing stripe and stride (and others may not

realize it's still a stripe in RAID5 even though it's not a mirror).



This email seems to be apropos -

<http://www.redhat.com/archives/linux-lvm/2006-October/msg00014.html>

but I dont' have any advice for you.



Brian
 
Old 06-08-2008, 12:16 AM
"Damon L. Chesser"
 
Default RAID for large disks

On Sat, 2008-06-07 at 20:04 -0400, Mag Gam wrote:
> Thanks thats the exact same question I have.
>
> Does this page,
> http://www.redhat.com/archives/linux-lvm/2006-October/msg00014.html,
> hold any validity? The poster makes a good argument, but by seeing
> Damon's response it makes no sense to go thru the trouble. I would be
> willing to try this if I get some assistance...

I don't know if it is worth the trouble or not. However your hardware
RAID controller will have a "geometry" to it. That geometry will be
listed in the controller bios. If you really feel like a file server
needs to be this tuned, you can use those parameters to set it up. The
OP of that email could not have been talking about a raid5 via hardware,
but most likely a software raid. Again, I have never worried or had to
worry that much about the geometry and I suspect, aside from academic
curiosity, you need not worry about it either. But what the heck. Try
it, test it, then just go with defaults and test it again. See which is
better.
>
>

>
--
Damon L. Chesser
damon@damtek.com
http://www.linkedin.com/in/dchesser
 
Old 06-08-2008, 12:18 AM
Mike Bird
 
Default RAID for large disks

On Sat June 7 2008 17:04:02 Mag Gam wrote:
> Does this page,
> http://www.redhat.com/archives/linux-lvm/2006-October/msg00014.html, hold
> any validity? The poster makes a good argument, but by seeing Damon's
> response it makes no sense to go thru the trouble. I would be willing to
> try this if I get some assistance...

If performance is uber-critical for your application then you
need to benchmark various different configurations under
realistic multitasking loads.

For most applications, it's really not worth worrying about
the details at this level.

--Mike Bird


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 06-08-2008, 11:33 AM
"Mag Gam"
 
Default RAID for large disks

Again, I appreciate the responses.

Damon:

I am dealing with HW RAID. I looked for the "geometry" for my controller, but could not find it.

http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=e n&cc=us&docIndexId=64179&taskId=101&prodTypeId=329 290&prodSeriesId=1157686


I am very curious about the geometry too...

*I don't know enough to pick and choose the optimal setting. Since I am working in the academic field, I would like to really understand this "geometry setting". Can someone please elaborate on this topic?


TIA


On Sat, Jun 7, 2008 at 8:18 PM, Mike Bird <mgb-debian@yosemite.net> wrote:

On Sat June 7 2008 17:04:02 Mag Gam wrote:

> Does this page,

> http://www.redhat.com/archives/linux-lvm/2006-October/msg00014.html, hold

> any validity? The poster makes a good argument, but by seeing Damon's

> response it makes no sense to go thru the trouble. I would be willing to

> try this if I get some assistance...



If performance is uber-critical for your application then you

need to benchmark various different configurations under

realistic multitasking loads.



For most applications, it's really not worth worrying about

the details at this level.



--Mike Bird





--

To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org

with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 06-08-2008, 02:14 PM
"Damon L. Chesser"
 
Default RAID for large disks

On Sun, 2008-06-08 at 07:33 -0400, Mag Gam wrote:
> Again, I appreciate the responses.
>
> Damon:
>
> I am dealing with HW RAID. I looked for the "geometry" for my
> controller, but could not find it.
>
> http://h20000.www2.hp.com/bizsupport/TechSupport/DocumentIndex.jsp?contentType=SupportManual&lang=e n&cc=us&docIndexId=64179&taskId=101&prodTypeId=329 290&prodSeriesId=1157686
>
> I am very curious about the geometry too...
>
> I don't know enough to pick and choose the optimal setting. Since I
> am working in the academic field, I would like to really understand
> this "geometry setting". Can someone please elaborate on this topic?
>
> TIA

Mag,

It looks like your controller does not let you set very much manually:
page 13 shows you can set the Stripe Size. At this point, jump in and
play. But again, I don't think it will amount to a hill of beans. The
controller "masks" all reads and writes to the physical drives and the
OS is ignorant of the underlining details. IF you needed to set it up a
certain way, you would KNOW it. And even so, it looks like the only
things you can adjust with this controller is the stripe size. In
almost all cases the "optimal" setting is the default of the controller
when you use it's setup "wizard" thingy, what ever it may be called.
All other deviations are for very specific instructions in some manual
for some application (or you spend much time bench testing to arrive at
what works best for you with that given hardware). Some advice: unless
you are being graded in some way on the maximum through put of this
server: Jump in, install it and be done.

We (IT) don't know the "optimum" settings of such things. We play with
it per some set of (specific) directions or we have a test box we can
bench test to meet some objective with. This concept is a moving,
slippery thing. It all depends on your network throughput, latency, cpu
load, bus load, I/O of every other component, I/O of the controller,
it's (the controller) memory, physical hd read/write speed and probably
a few more I can't think of right now.

Kill this beast, install the os using defaults for the HDs, see if you
can serve up the files at a rate that works for you, if not, look it
over again.

BTW, on the next model server you get, everything you learn here will
not be valid unless it is the exact same set of hardware. That is why I
would not fret over this, there is no "great maxim" to be learned except
"can I set up THIS box to work at the rate I need?" Sometimes the
answer is no, but that falls onto the procurement end of the deal.

A DBA will spend much time telling a sys admin what strip to put onto a
RAID (using a hardware controller), but that is from the application mfg
having benched tested a specific model of server with very specific
hardware to arrive at the best throughput possible with a given hardware
load out. This is not your situation. The only answer is to test.

Anyway, that is my 2C worth.


--
Damon L. Chesser
damon@damtek.com
http://www.linkedin.com/in/dchesser
 

Thread Tools




All times are GMT. The time now is 04:22 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org