FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 09-04-2012, 11:10 AM
Tony Molloy
 
Default Advice on partitioning a Dell MD1200 disk array

Hi,

I've just got possession of a Dell PE R720 with 2 MD1200 disk
enclosures.

Both MD1200 are fully populated with 12 x 3 TB disks

The system will basically be a student file-server running CentOS 6.x
serving various size files from small c programs to multi gigabyte
audio and video files over GB ethernet.

The first MD1200 will be configured as the NFS disk. The requirements
are for 6 fixed equally sized partitions, one for each cohort of
students. For this I was thinking of splitting the MD1200 into 2 RAID5
arrays with a hot spare each. Then partitioning each into 3 ext4
partitions.

The second MD1200 will be used to backup the first, using BackupPC and
for other storage purposes.

As I won't know the storage requirements for the "backup partition"
and they will probably change over time anyway. I was thinking of
using LVM for it. So how to partition the MD1200 for LVM. I don't want
to put all 12 disks in a RAID5 and put a LVM volume on it. Can I
split it into 2 RAID5 and have a LVM volume spanning both.

Any suggestions.

Just remember I'm due to retire at the end of this month so this will
be my last big job for the Dept. And due to financial constraints I
will not be replaced. So I will be handing this machine over to a co-
worker who is basically a Windoze admin with only a basic knowledge of
Linux so nothing too fancy. ;-)

Thanks,

Tony


--

Tony Molloy

CTO, Dept. of Comp. Sci.
University of Limerick
Limerick.
Ireland
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-04-2012, 11:44 AM
Götz Reinicke
 
Default Advice on partitioning a Dell MD1200 disk array

Hi Tony,

because I suggest just something very general I post off list

From my POV as I'm currently facing similar setups with different
hardware rolling back from fine granular setups to simple 'bigger' less
complex configurations. (we do have 6 iscsi storages from 2TB (sun ZFS)
up to 32 TB)

keep it small and simple!

I think you are very familiar with the general problems of big HW raids
and big filesystems like rebuild or check times, but splitting up and
adding more complex layers like multiple raids joining in lvm etc. makes
debugging and general handling very hard.

On the other hand, I checked and read a lot about filesystems the last
days being faced with serving user windows samba profiles with lot of
small files and big video/audio data etc.

Long story short:

I usually do one raidvolume per hardware raid box; e.g. we use 16*1TB
drives. Raid6 or Raid5 with spare. I did not notice big performance
differences.

I use LVM to make partitions or I prefer using just one big partition.

I tried xfs and ext4 and will go with ext4 as some test went better for
my setup and from what I read it looks not bad

I think you can combine block level devices (like multiple raid boxes)
by LVM into one bigger LV.

And last but not least: The CPU/RAM/Network of the host serving the
files is also very important!

I noticed, that the same iscsi storage got about 70MB/s on a new server
(xeon multicore), while on the old fileserver it just got up to 40MB/s.



my2cents regards . Götz



May be worth reading:

http://www.techforce.com.br/news/linux_blog/lvm_raid_xfs_ext3_tuning_for_small_files_parallel_ i_o_on_debian#.UEPSI1RqYso

http://monolight.cc/2011/02/linux-filesystems-small-file-performance-on-hdds/

http://unix.stackexchange.com/questions/28756/what-is-the-most-high-performance-linux-filesystem-for-storing-a-lot-of-small-fi

https://wiki.archlinux.org/index.php/Ext3

https://wiki.archlinux.org/index.php/Ext4#Tips_and_tricks



Am 04.09.12 13:10, schrieb Tony Molloy:
>
> Hi,
>
> I've just got possession of a Dell PE R720 with 2 MD1200 disk
> enclosures.
>
> Both MD1200 are fully populated with 12 x 3 TB disks
>
> The system will basically be a student file-server running CentOS 6.x
> serving various size files from small c programs to multi gigabyte
> audio and video files over GB ethernet.
>
> The first MD1200 will be configured as the NFS disk. The requirements
> are for 6 fixed equally sized partitions, one for each cohort of
> students. For this I was thinking of splitting the MD1200 into 2 RAID5
> arrays with a hot spare each. Then partitioning each into 3 ext4
> partitions.
>
> The second MD1200 will be used to backup the first, using BackupPC and
> for other storage purposes.
>
> As I won't know the storage requirements for the "backup partition"
> and they will probably change over time anyway. I was thinking of
> using LVM for it. So how to partition the MD1200 for LVM. I don't want
> to put all 12 disks in a RAID5 and put a LVM volume on it. Can I
> split it into 2 RAID5 and have a LVM volume spanning both.
>
> Any suggestions.
>
> Just remember I'm due to retire at the end of this month so this will
> be my last big job for the Dept. And due to financial constraints I
> will not be replaced. So I will be handing this machine over to a co-
> worker who is basically a Windoze admin with only a basic knowledge of
> Linux so nothing too fancy. ;-)
>
> Thanks,
>
> Tony
>
>
> --
>
> Tony Molloy
>
> CTO, Dept. of Comp. Sci.
> University of Limerick
> Limerick.
> Ireland
> _______________________________________________
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos
>


--
Götz Reinicke
IT-Koordinator

Tel. +49 7141 969 82 420
Fax +49 7141 969 55 420
E-Mail goetz.reinicke@filmakademie.de

Filmakademie Baden-Württemberg GmbH
Akademiehof 10
71638 Ludwigsburg
www.filmakademie.de

Eintragung Amtsgericht Stuttgart HRB 205016

Vorsitzender des Aufsichtsrats: Jürgen Walter MdL
Staatssekretär im Ministerium für Wissenschaft,
Forschung und Kunst Baden-Württemberg

Geschäftsführer: Prof. Thomas Schadt

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-04-2012, 01:37 PM
Blake Hudson
 
Default Advice on partitioning a Dell MD1200 disk array

Tony Molloy wrote the following on 9/4/2012 6:10 AM:
> Hi,
>
> I've just got possession of a Dell PE R720 with 2 MD1200 disk
> enclosures.
>
> Both MD1200 are fully populated with 12 x 3 TB disks
>
> The system will basically be a student file-server running CentOS 6.x
> serving various size files from small c programs to multi gigabyte
> audio and video files over GB ethernet.
>
> The first MD1200 will be configured as the NFS disk. The requirements
> are for 6 fixed equally sized partitions, one for each cohort of
> students. For this I was thinking of splitting the MD1200 into 2 RAID5
> arrays with a hot spare each. Then partitioning each into 3 ext4
> partitions.

I think that sounds like a balanced approach. With 12 SATA drives you're
bound to run into failures during the life of these devices (3-5 yrs).
By splitting the disks into 2 arrays, a failure of one disk will only
degrade 1 array, affecting less users. Rebuild times will be reduced if
there is a failure and performance for multi-user workloads should be
increased by separating into multiple arrays.

>
> The second MD1200 will be used to backup the first, using BackupPC and
> for other storage purposes.
>
> As I won't know the storage requirements for the "backup partition"
> and they will probably change over time anyway. I was thinking of
> using LVM for it. So how to partition the MD1200 for LVM. I don't want
> to put all 12 disks in a RAID5 and put a LVM volume on it. Can I
> split it into 2 RAID5 and have a LVM volume spanning both.
I've never found any value in LVM. To me, it adds another layer of
complexity or room for bugs to creep in. Arrays can be resized, GPT or
MSDOS partitions can be resized, file systems can be resized, your
hardware can do the spanning that LVM provides, why would you want to
add another layer of abstraction? For similar reasons, I would stick to
a file system that comes stock with your OS, ext4 or XFS should be fine,
as they are both well supported in CentOS.
> Any suggestions.
>
> Just remember I'm due to retire at the end of this month so this will
> be my last big job for the Dept. And due to financial constraints I
> will not be replaced. So I will be handing this machine over to a co-
> worker who is basically a Windoze admin with only a basic knowledge of
> Linux so nothing too fancy. ;-)
>

At first, I was going to recommend 1 hot spare, but if you think the new
admin may not have as much time, or you want things a little more
simple, then making two identical arrays is probably going to be easier.
If he's not as familiar with your hardware, then I would print out a
cheat sheet on how to rebuild the array, check the status (MegaCLI?,
OpenManage?), etc. As always, document what you have, how you put it
together, and who to call or where to look if he's stuck.

--Blake
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-04-2012, 02:20 PM
"Paul Tader"
 
Default Advice on partitioning a Dell MD1200 disk array

>
> The second MD1200 will be used to backup the first, using BackupPC and
> for other storage purposes.
>
> As I won't know the storage requirements for the "backup partition"
> and they will probably change over time anyway. I was thinking of
> using LVM for it. So how to partition the MD1200 for LVM. I don't want
> to put all 12 disks in a RAID5 and put a LVM volume on it. Can I
> split it into 2 RAID5 and have a LVM volume spanning both.
>

Just a thought about the backup volume.

Because it going to be used for backups and your retirement (congrats!) I
would configure one partition and no volume management on top. Why?
Because backups are like insurance policies. That are not fun and no one
reviews them, but when you need them, you really hope that everything is
in order. So you don't want the next admin to find out several months or
years down the road that the backup software ran out of space and has been
failing to back anything up for the previous few months.

_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-04-2012, 06:07 PM
Tony Molloy
 
Default Advice on partitioning a Dell MD1200 disk array

On Tuesday 04 September 2012 12:44:26 Götz Reinicke wrote:
> Hi Tony,
>
> because I suggest just something very general I post off list
>
> From my POV as I'm currently facing similar setups with different
> hardware rolling back from fine granular setups to simple 'bigger'
> less complex configurations. (we do have 6 iscsi storages from 2TB
> (sun ZFS) up to 32 TB)
>
> keep it small and simple!
>
> I think you are very familiar with the general problems of big HW
> raids and big filesystems like rebuild or check times, but
> splitting up and adding more complex layers like multiple raids
> joining in lvm etc. makes debugging and general handling very
> hard.
>
> On the other hand, I checked and read a lot about filesystems the
> last days being faced with serving user windows samba profiles
> with lot of small files and big video/audio data etc.
>
> Long story short:
>
> I usually do one raidvolume per hardware raid box; e.g. we use
> 16*1TB drives. Raid6 or Raid5 with spare. I did not notice big
> performance differences.
>
> I use LVM to make partitions or I prefer using just one big
> partition.
>
> I tried xfs and ext4 and will go with ext4 as some test went better
> for my setup and from what I read it looks not bad
>
> I think you can combine block level devices (like multiple raid
> boxes) by LVM into one bigger LV.
>
> And last but not least: The CPU/RAM/Network of the host serving the
> files is also very important!
>
> I noticed, that the same iscsi storage got about 70MB/s on a new
> server (xeon multicore), while on the old fileserver it just got
> up to 40MB/s.
>
>
>
> my2cents regards . Götz
>
>
>
> May be worth reading:
>
> http://www.techforce.com.br/news/linux_blog/lvm_raid_xfs_ext3_tunin
> g_for_small_files_parallel_i_o_on_debian#.UEPSI1Rq Yso
>
> http://monolight.cc/2011/02/linux-filesystems-small-file-performanc
> e-on-hdds/
>
> http://unix.stackexchange.com/questions/28756/what-is-the-most-high
> -performance-linux-filesystem-for-storing-a-lot-of-small-fi
>
> https://wiki.archlinux.org/index.php/Ext3
>
> https://wiki.archlinux.org/index.php/Ext4#Tips_and_tricks
>
> Am 04.09.12 13:10, schrieb Tony Molloy:
> > Hi,
> >
> > I've just got possession of a Dell PE R720 with 2 MD1200 disk
> > enclosures.
> >
> > Both MD1200 are fully populated with 12 x 3 TB disks
> >
> > The system will basically be a student file-server running CentOS
> > 6.x serving various size files from small c programs to multi
> > gigabyte audio and video files over GB ethernet.
> >
> > The first MD1200 will be configured as the NFS disk. The
> > requirements are for 6 fixed equally sized partitions, one for
> > each cohort of students. For this I was thinking of splitting the
> > MD1200 into 2 RAID5 arrays with a hot spare each. Then
> > partitioning each into 3 ext4 partitions.
> >
> > The second MD1200 will be used to backup the first, using
> > BackupPC and for other storage purposes.
> >
> > As I won't know the storage requirements for the "backup
> > partition" and they will probably change over time anyway. I was
> > thinking of using LVM for it. So how to partition the MD1200 for
> > LVM. I don't want to put all 12 disks in a RAID5 and put a LVM
> > volume on it. Can I split it into 2 RAID5 and have a LVM volume
> > spanning both.
> >
> > Any suggestions.
> >
> > Just remember I'm due to retire at the end of this month so this
> > will be my last big job for the Dept. And due to financial
> > constraints I will not be replaced. So I will be handing this
> > machine over to a co- worker who is basically a Windoze admin
> > with only a basic knowledge of Linux so nothing too fancy. ;-)
> >
> > Thanks,
> >
> > Tony
> >
> >
> > --
> >
> > Tony Molloy
> >
> > CTO, Dept. of Comp. Sci.
> > University of Limerick
> > Limerick.
> > Ireland
> > _______________________________________________
> > CentOS mailing list
> > CentOS@centos.org
> > http://lists.centos.org/mailman/listinfo/centos
>
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-04-2012, 06:25 PM
Nate Duehr
 
Default Advice on partitioning a Dell MD1200 disk array

On Sep 4, 2012, at 5:10 AM, Tony Molloy <tony.molloy@ul.ie> wrote:

> Just remember I'm due to retire at the end of this month so this will
> be my last big job for the Dept. And due to financial constraints I
> will not be replaced. So I will be handing this machine over to a co-
> worker who is basically a Windoze admin with only a basic knowledge of
> Linux so nothing too fancy. ;-)

Hand him the machine and tell him to load Windows on it or whatever he wants to maintain for the next X years, relax for 30 days and enjoy retirement. It's his problem now. :-) :-) ;-)

Nate
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-04-2012, 06:55 PM
Tony Molloy
 
Default Advice on partitioning a Dell MD1200 disk array

On Tuesday 04 September 2012 19:25:05 Nate Duehr wrote:
> On Sep 4, 2012, at 5:10 AM, Tony Molloy <tony.molloy@ul.ie> wrote:
> > Just remember I'm due to retire at the end of this month so this
> > will be my last big job for the Dept. And due to financial
> > constraints I will not be replaced. So I will be handing this
> > machine over to a co- worker who is basically a Windoze admin
> > with only a basic knowledge of Linux so nothing too fancy. ;-)
>
> Hand him the machine and tell him to load Windows on it or whatever
> he wants to maintain for the next X years, relax for 30 days and
> enjoy retirement. It's his problem now. :-) :-) ;-)
>
> Nate


Sounds good but it's not the Linux way is it ;-)

Anyway I want to force him to become a better person, to grow and
expand his knowledge and in the end maybe drag him away from the evil
empire to the land of the free !!!!!

LOL

Tony
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-05-2012, 07:02 PM
Tony Molloy
 
Default Advice on partitioning a Dell MD1200 disk array

On Tuesday 04 September 2012 15:20:26 Paul Tader wrote:
> > The second MD1200 will be used to backup the first, using
> > BackupPC and for other storage purposes.
> >
> > As I won't know the storage requirements for the "backup
> > partition" and they will probably change over time anyway. I was
> > thinking of using LVM for it. So how to partition the MD1200 for
> > LVM. I don't want to put all 12 disks in a RAID5 and put a LVM
> > volume on it. Can I split it into 2 RAID5 and have a LVM volume
> > spanning both.
>
> Just a thought about the backup volume.
>
> Because it going to be used for backups and your retirement
> (congrats!) I would configure one partition and no volume
> management on top. Why? Because backups are like insurance
> policies. That are not fun and no one reviews them, but when you
> need them, you really hope that everything is in order. So you
> don't want the next admin to find out several months or years down
> the road that the backup software ran out of space and has been
> failing to back anything up for the previous few months.
>
> _______________________________________________
> CentOS mailing list
> CentOS@centos.org
> http://lists.centos.org/mailman/listinfo/centos
>


Thanks to Gotz, Blake, Paul and Nate for your advice.

I was pretty happy with the split RAID 5 for the NFS partitions on the
first MD1200.

I think I'll go with the simple solution for the backup MD1200. A
single RAID 5 encompassing all 12 disks probably with 2 hot spares.
Put an ext4 on top of that without the complexity of LVM.

Thanks,

Tony

aside Paul, unfortunately my retirement is not voluntary. It's due to
a spinal cord injury but such is life ;-)
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 09-05-2012, 07:46 PM
Dean Jones
 
Default Advice on partitioning a Dell MD1200 disk array

On Wed, Sep 5, 2012 at 12:02 PM, Tony Molloy <tony.molloy@ul.ie> wrote:
> On Tuesday 04 September 2012 15:20:26 Paul Tader wrote:
>> > The second MD1200 will be used to backup the first, using
>> > BackupPC and for other storage purposes.
>> >
>> > As I won't know the storage requirements for the "backup
>> > partition" and they will probably change over time anyway. I was
>> > thinking of using LVM for it. So how to partition the MD1200 for
>> > LVM. I don't want to put all 12 disks in a RAID5 and put a LVM
>> > volume on it. Can I split it into 2 RAID5 and have a LVM volume
>> > spanning both.
>>
>> Just a thought about the backup volume.
>>
>> Because it going to be used for backups and your retirement
>> (congrats!) I would configure one partition and no volume
>> management on top. Why? Because backups are like insurance
>> policies. That are not fun and no one reviews them, but when you
>> need them, you really hope that everything is in order. So you
>> don't want the next admin to find out several months or years down
>> the road that the backup software ran out of space and has been
>> failing to back anything up for the previous few months.
>>
>> _______________________________________________
>> CentOS mailing list
>> CentOS@centos.org
>> http://lists.centos.org/mailman/listinfo/centos
>>
>
>
> Thanks to Gotz, Blake, Paul and Nate for your advice.
>
> I was pretty happy with the split RAID 5 for the NFS partitions on the
> first MD1200.
>
> I think I'll go with the simple solution for the backup MD1200. A
> single RAID 5 encompassing all 12 disks probably with 2 hot spares.
> Put an ext4 on top of that without the complexity of LVM.
>
> Thanks,
>
> Tony
>
> aside Paul, unfortunately my retirement is not voluntary. It's due to
> a spinal cord injury but such is life ;-)


I would highly suggest RAID 6 if you are dealing with drives over 2TB
in size. Errors during rebuild happen and rebuild times for 2TB+
drives can get pretty long, especially if you are building one 12 disk
RAID.

Also important if a non-savvy admin is going to take over
administration. What if they pull the wrong drive during a
replacement on a failed RAID5 array?
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 11:51 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org