Question about LVM and RAID
 

Hi All,

I have an external drive cage which has been configured with two
separate RAID 5 arrays. I then used LVM to create two PVs, and then
added the volumes together under one VG. The whole shebang is mounted
on one file system (/srv).

What would happen if one of the RAID arrays failed (e.g. two drives die
in RAID 5 array 1)? Would the data be safe, would I lose all data, or
would I just lose the data that was on the failed array?

I believe I would only lose the data on the failed array, but a friend
believe I would lose the whole lot.

Thanks in advance!

Regards,

Ranbir
--
Kanwar Ranbir Sandhu
Linux 2.6.22.9-61.fc6 i686 GNU/Linux
11:48:40 up 13 days, 10:21, 1 user, load average: 0.27, 0.12, 0.10


--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Question about LVM and RAID
 

Hi,

I suppose you have only made a JBOD with LVM - no further RAID0 or such.

With JBOD, at least the data on the left RAID5-set should be safe. The
data on the failed array and the bits and pieces which were on both
arrays would be lost. Fragmentation and the size of data could be an
issue - e.g. a 200 GB file on 2 x 160GB arrays would mean more than half
is lost... ;(


I think your friend meant you had made a RAID0 above the RAID5-sets,
which would indeed mean the whole thing would be lost.


Regards,
rainer


Kanwar Ranbir Sandhu wrote:

Hi All,

I have an external drive cage which has been configured with two
separate RAID 5 arrays. I then used LVM to create two PVs, and then
added the volumes together under one VG. The whole shebang is mounted
on one file system (/srv).

What would happen if one of the RAID arrays failed (e.g. two drives die
in RAID 5 array 1)? Would the data be safe, would I lose all data, or
would I just lose the data that was on the failed array?

I believe I would only lose the data on the failed array, but a friend
believe I would lose the whole lot.

Thanks in advance!

Regards,

Ranbir



--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Question about LVM and RAID
 

Kanwar Ranbir Sandhu wrote:
> Hi All,
>
> I have an external drive cage which has been configured with two
> separate RAID 5 arrays. I then used LVM to create two PVs, and then
> added the volumes together under one VG. The whole shebang is mounted
> on one file system (/srv).
>
> What would happen if one of the RAID arrays failed (e.g. two drives die
> in RAID 5 array 1)? Would the data be safe, would I lose all data, or
> would I just lose the data that was on the failed array?
>
> I believe I would only lose the data on the failed array, but a friend
> believe I would lose the whole lot.
>
> Thanks in advance!
>
> Regards,
>
> Ranbir
Because they are one file system, it is hard to predict exactly what
you would lose, but in any case, recovering the data from the
remaining array would be interesting. I could be wrong, but I don't
believe there is anything in the LVM code that would prevent a file
from spanning the two RAID arrays making up the VG. So you can
figure you are going to lose any files that span the drives. (You
may be able to recover part of the file...)

You may want to start looking into LVM recovery tools now, before
you run into a problem. (And keep good backups.)

Mikkel
--

Do not meddle in the affairs of dragons,
for thou art crunchy and taste good with Ketchup!

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Question about LVM and RAID
 

rainer wrote:



Kanwar Ranbir Sandhu wrote:

Hi All,

I have an external drive cage which has been configured with two
separate RAID 5 arrays. I then used LVM to create two PVs, and then
added the volumes together under one VG. The whole shebang is mounted
on one file system (/srv).

What would happen if one of the RAID arrays failed (e.g. two drives die
in RAID 5 array 1)? Would the data be safe, would I lose all data, or
would I just lose the data that was on the failed array?

I believe I would only lose the data on the failed array, but a friend
believe I would lose the whole lot.

Thanks in advance!

Regards,

Ranbir




> Hi,
>
> I suppose you have only made a JBOD with LVM - no further RAID0 or such.
>
> With JBOD, at least the data on the left RAID5-set should be safe. The
> data on the failed array and the bits and pieces which were on both
> arrays would be lost. Fragmentation and the size of data could be an
> issue - e.g. a 200 GB file on 2 x 160GB arrays would mean more than half
> is lost... ;(
>
> I think your friend meant you had made a RAID0 above the RAID5-sets,
> which would indeed mean the whole thing would be lost.
>
> Regards,
> rainer
>

To add to this.

A RAID 5 array is dependent on the number of drives where the data is
spread across. If there are 3 drives, then two failures is more than
enough. If you have 6 drives, then two drives may be okay.


It would be more useful to explain the RAID setup with the hardware,
number of drives per RAID and how they are configured.



--
Robin Laing

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Question about LVM and RAID
 

RAID5 (if it really is one) ALWAYS has one drive´s capacity as spare...
the spare blocks are just distributed on the disks, thus avoiding the
bottleneck of a single spare drive (these would be raid levels 3 and 4).


what you meant was RAID6/ADG, a semi-proprietary stuff rather found on
hardware controllers, e.g. hp smartarrays. these do calculate a parity
for each n blocks, and for "n blocks+parity" generate a second parity
block. All these blocks are distributed evenly on all drives in the array.


The thing with ADG is the rebuild time - for example the RAIDs at work
have about 20 drives each (300 gig); the rebuild time on those is about
1 gb per hour minimum (when there is heavy activity on the raid set).
that would mean 300 hours without any protection (when using raid5) !
instead, with raid6/adg there still is one parity left.
bad thing, though, is the raid controller has to calculate a lot of
parities. furthermore, the cost is rather high with 2 disks´ worth of
parity. Most of the time, such setups use RAID10 (mirror and stripe),
which uses much cheaper controllers and offers more performance.


sorry for off-topic :)

Robin Laing schrieb:

rainer wrote:



Kanwar Ranbir Sandhu wrote:

Hi All,

I have an external drive cage which has been configured with two
separate RAID 5 arrays. I then used LVM to create two PVs, and then
added the volumes together under one VG. The whole shebang is mounted
on one file system (/srv).

What would happen if one of the RAID arrays failed (e.g. two drives die
in RAID 5 array 1)? Would the data be safe, would I lose all data, or
would I just lose the data that was on the failed array?

I believe I would only lose the data on the failed array, but a friend
believe I would lose the whole lot.

Thanks in advance!

Regards,

Ranbir




> Hi,
>
> I suppose you have only made a JBOD with LVM - no further RAID0 or
such.

>
> With JBOD, at least the data on the left RAID5-set should be safe. The
> data on the failed array and the bits and pieces which were on both
> arrays would be lost. Fragmentation and the size of data could be an
> issue - e.g. a 200 GB file on 2 x 160GB arrays would mean more than
half

> is lost... ;(
>
> I think your friend meant you had made a RAID0 above the RAID5-sets,
> which would indeed mean the whole thing would be lost.
>
> Regards,
> rainer
>

To add to this.

A RAID 5 array is dependent on the number of drives where the data is
spread across. If there are 3 drives, then two failures is more than
enough. If you have 6 drives, then two drives may be okay.


It would be more useful to explain the RAID setup with the hardware,
number of drives per RAID and how they are configured.





--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Question about LVM and RAID
 

On Thu, 2007-11-29 at 21:39 +0100, moi wrote:
> RAID5 (if it really is one) ALWAYS has one drive´s capacity as spare...
> the spare blocks are just distributed on the disks, thus avoiding the
> bottleneck of a single spare drive (these would be raid levels 3 and 4).
>
> what you meant was RAID6/ADG, a semi-proprietary stuff rather found on
> hardware controllers, e.g. hp smartarrays. these do calculate a parity
> for each n blocks, and for "n blocks+parity" generate a second parity
> block. All these blocks are distributed evenly on all drives in the array.
>
> The thing with ADG is the rebuild time - for example the RAIDs at work
> have about 20 drives each (300 gig); the rebuild time on those is about
> 1 gb per hour minimum (when there is heavy activity on the raid set).
> that would mean 300 hours without any protection (when using raid5) !
> instead, with raid6/adg there still is one parity left.
> bad thing, though, is the raid controller has to calculate a lot of
> parities. furthermore, the cost is rather high with 2 disks´ worth of
> parity. Most of the time, such setups use RAID10 (mirror and stripe),
> which uses much cheaper controllers and offers more performance.
>
> sorry for off-topic :)

Two remarks:
A. Modern RAID5 (be that software and/or hardware controller) build far
faster then 1GB/h (291KB/s!??!?!).
I timed my own sever (6 250GB drives in software RAID5) at ~12MBps
(42GB/h) load and ~90MB/s (324GB/h) idle.
B. The Linux kernel has built in software RAID6 support; while slower
then the RAID5 implementation, the performance hit is noticeable but not
devastating and given the added price (1 250/320/etc GB SATA drive)
RAID6 is indeed a fair option if you require two-failed-disk support.

- Gilboa

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Question about LVM and RAID
 

On Thu, 2007-11-29 at 11:54 -0500, Kanwar Ranbir Sandhu wrote:
> Hi All,
>
> I have an external drive cage which has been configured with two
> separate RAID 5 arrays. I then used LVM to create two PVs, and then
> added the volumes together under one VG. The whole shebang is mounted
> on one file system (/srv).

Ouch... Bad configuration.
If you're looking for two dead disk support, test the Linux' kernel
software RAID6 support.

>
> What would happen if one of the RAID arrays failed (e.g. two drives die
> in RAID 5 array 1)? Would the data be safe, would I lose all data, or
> would I just lose the data that was on the failed array?

Two drives in either RAID will most likely kill your LVM.

>
> I believe I would only lose the data on the failed array, but a friend
> believe I would lose the whole lot.


It's a matter of blind luck, but any LV that sits on the dead RAID5
array will be taken with it.

- Gilboa

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Question about LVM and RAID
 

> > What would happen if one of the RAID arrays failed (e.g. two drives die
> > in RAID 5 array 1)? Would the data be safe, would I lose all data, or
> > would I just lose the data that was on the failed array?
>
> Two drives in either RAID will most likely kill your LVM.

If you need two drive reliability you need RAID6 or depending on the
tradeoffs you want to make RAID 5 + hot spare.

For performance you want RAID 1 + 0 (possibly + hot spares)

--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Question about LVM and RAID
 

On Sun, 2007-12-02 at 14:29 +0200, Gilboa Davara wrote:
> > What would happen if one of the RAID arrays failed (e.g. two drives die
> > in RAID 5 array 1)? Would the data be safe, would I lose all data, or
> > would I just lose the data that was on the failed array?
>
> Two drives in either RAID will most likely kill your LVM.

Guess the friend was right, although I had a feeling he would be. Well,
there isn't much data on the arrays yet, so I think I might just rebuild
it as one big RAID 5 instead.

> It's a matter of blind luck, but any LV that sits on the dead RAID5
> array will be taken with it.

Ok. I need to mount all the storage on one file system, so having two
arrays won't work (I want to mount everything on /srv).

Thanks to everyone for the input!

Regards,

Ranbir
--
Kanwar Ranbir Sandhu
Linux 2.6.22.9-61.fc6 i686 GNU/Linux
22:39:02 up 16 days, 21:11, 1 user, load average: 0.77, 0.58, 0.53


--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list

Question about LVM and RAID
 

Hi,



the (B) part is new to me - I just had my rhce course and they didn´t
mention it in any way... Ok, I always do raid in hardware when using
servers, for desktop class hardware I prefer doing mirroring.



Regarding (A): the 1 GB rebuild rate is a (hp) worst-case szenario when
sizing solutions - in fact, it may well be a dozen times faster.

Nevertheless, I prefer to think of the worst possible case - in your
case, it would be at least 8 hours before the rebuild was done. That
would mean 8 hours without protection when using raid5; enough time for
the next drive to fail... Furthermore the impact on your CPU will be
rather big...



Thanks for the info anyway

rainer







Gilboa Davara wrote:

On Thu, 2007-11-29 at 21:39 +0100, moi wrote:


RAID5 (if it really is one) ALWAYS has one drive´s capacity as spare...
the spare blocks are just distributed on the disks, thus avoiding the
bottleneck of a single spare drive (these would be raid levels 3 and 4).

what you meant was RAID6/ADG, a semi-proprietary stuff rather found on
hardware controllers, e.g. hp smartarrays. these do calculate a parity
for each n blocks, and for "n blocks+parity" generate a second parity
block. All these blocks are distributed evenly on all drives in the array.

The thing with ADG is the rebuild time - for example the RAIDs at work
have about 20 drives each (300 gig); the rebuild time on those is about
1 gb per hour minimum (when there is heavy activity on the raid set).
that would mean 300 hours without any protection (when using raid5) !
instead, with raid6/adg there still is one parity left.
bad thing, though, is the raid controller has to calculate a lot of
parities. furthermore, the cost is rather high with 2 disks´ worth of
parity. Most of the time, such setups use RAID10 (mirror and stripe),
which uses much cheaper controllers and offers more performance.

sorry for off-topic :)



Two remarks:
A. Modern RAID5 (be that software and/or hardware controller) build far
faster then 1GB/h (291KB/s!??!?!).
I timed my own sever (6 250GB drives in software RAID5) at ~12MBps
(42GB/h) load and ~90MB/s (324GB/h) idle.
B. The Linux kernel has built in software RAID6 support; while slower
then the RAID5 implementation, the performance hit is noticeable but not
devastating and given the added price (1 250/320/etc GB SATA drive)
RAID6 is indeed a fair option if you require two-failed-disk support.

- Gilboa







--
fedora-list mailing list
fedora-list@redhat.com
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list