FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 04-13-2011, 03:19 PM
Emmanuel Noobadmin
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On 4/13/11, Rudi Ahlers <Rudi@softdux.com> wrote:
>> to expand the array
>
> I haven't had problems doing it this way yet.

I finally figured out my mistake creating the raid devices and got a
working RAID 0 on two RAID 1 arrays. But I wasn't able to add another
RAID 1 component to the array with the error

mdadm: add new device failed for /dev/md/mdr1_3 as 2: Invalid argument

Googling up this indicates that it's the expected result trying to add
a new device to a RAID 0 array. Could you or anybody else please share
what's the trick to achieving this?
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 04-13-2011, 04:45 PM
Ross Walker
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On Wed, Apr 13, 2011 at 11:19 AM, Emmanuel Noobadmin
<centos.admin@gmail.com> wrote:
> On 4/13/11, Rudi Ahlers <Rudi@softdux.com> wrote:
>>> to expand the array
>>
>> I haven't had problems doing it this way yet.
>
> I finally figured out my mistake creating the raid devices and got a
> working RAID 0 on two RAID 1 arrays. But I wasn't able to add another
> RAID 1 component to the array with the error
>
> mdadm: add new device failed for /dev/md/mdr1_3 as 2: Invalid argument
>
> Googling up this indicates that it's the expected result trying to add
> a new device to a RAID 0 array. Could you or anybody else please share
> what's the trick to achieving this?

You can't expand a mdraid raid0.

I believe you can expand a mdraid raid10,5,6, but not raid0.

If you want to use separate raid1 devices instead of mdraid's raid10
implementation then use LVM, add them to a VG then stripe the LVs
across the different PVs.

The only problem with that is restriping existing LVs across new PVs
is difficult to the point that it is often better to create new LVs
with the proper striping. You can do it though using 'lvresize' to
change the stripe width, but it won't give a linear striping for
existing data and the LV will eventually fill the first PVs causing
all data to only be written to the last PV.

I often find it handy to have a "backup" raid1 disk on the system
that's big enough to hold the contents of the largest LV, then dump
the production LV to the backup, blow away the production, recreate
with the new stripe size, then restore the data back. This backup
volume could be an iSCSI volume exported from another server that does
have the capacity if there isn't any in the host.

-Ross
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 04-14-2011, 04:51 AM
Emmanuel Noobadmin
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On 4/14/11, Ross Walker <rswwalker@gmail.com> wrote:
> You can't expand a mdraid raid0.
>
> I believe you can expand a mdraid raid10,5,6, but not raid0.

That was what I thought previously when looking into this and weighing
the pros/cons of using RAID 10 vs RAID 5.

But earlier this week, from the 40TB Filesystem thread, Rudi stated
that he has a RAID 0 on RAID 1 setup that he can expand and Brandon
confirmed that it is possible to expand RAID 0.

http://forum.nginx.org/read.php?24,190407,190764#msg-190764

Since I've read that some features such as creating RAID 10 directly
were not in the man pages, and another person in the mdadm list
implied that a raid 10 like array could be achieved using some
creative RAID 5 layout, I assumed that perhaps it was possible to do
that, just that there was some undocumented trick or specific manner
the arrays had to be setup.

I'm was stuck trying to decide whether to go for the cheaper RAID 5
setup and possibly getting killed by the IOPS penalty and the risk
associated with rebuild time, or figure out a way to use the
recommended RAID 10 setup with a smaller usable capacity for the
budget but do so with the ability to expand in the near future. So
really hoping that it could be done.


> I often find it handy to have a "backup" raid1 disk on the system
> that's big enough to hold the contents of the largest LV, then dump
> the production LV to the backup, blow away the production, recreate
> with the new stripe size, then restore the data back. This backup
> volume could be an iSCSI volume exported from another server that does
> have the capacity if there isn't any in the host.

That sounds like a solution since each LV shouldn't be too massive.
This seems to imply I would have to bring down the service running off
the LV during the recreation but I suppose I since the storage is
planned to be exported over iSCSI in the first place, I could simply
export the backup copy while expanding the original.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 04-14-2011, 05:51 AM
John R Pierce
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On 04/13/11 9:51 PM, Emmanuel Noobadmin wrote:
> I'm was stuck trying to decide whether to go for the cheaper RAID 5
> setup and possibly getting killed by the IOPS penalty and the risk
> associated with rebuild time, or figure out a way to use the
> recommended RAID 10 setup with a smaller usable capacity for the
> budget but do so with the ability to expand in the near future. So
> really hoping that it could be done.

since this is the centos list, I really didn't want to suggest this, but
if I was building a 20 or 40TB or whatever storage server, I do believe
I'd be strongly consider using Solaris, or one of its variants like
OpenIndiana, with ZFS.

ZFS was engineered from the ground up to scale to zetabytes

# zpool create archive mirror c2t0d0 c2t1d0 mirror c2t2d0 c2t3d0 mirror
c3t0d0 c3t1d0 mirror c3t2d0 c3t2d0 ..... spare c?t?d0 c?t?d0

done. available for use in a few seconds. default mountpoint is /archive

# zfs create -o mountpoint=/u01 archive/u01
# zfs create -o mountpoint=/u02 archive/u02

creates a couple more filesystems that share the free space, mounted as
/u01 and /u02

adding more disks?

# zpool add archive mirror c7t0d0 c7t1d0 mirror c7t2d0 c7t3d0


_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 04-14-2011, 12:54 PM
Christopher Chan
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On Thursday, April 14, 2011 01:51 PM, John R Pierce wrote:
> On 04/13/11 9:51 PM, Emmanuel Noobadmin wrote:
>> I'm was stuck trying to decide whether to go for the cheaper RAID 5
>> setup and possibly getting killed by the IOPS penalty and the risk
>> associated with rebuild time, or figure out a way to use the
>> recommended RAID 10 setup with a smaller usable capacity for the
>> budget but do so with the ability to expand in the near future. So
>> really hoping that it could be done.
>
> since this is the centos list, I really didn't want to suggest this, but
> if I was building a 20 or 40TB or whatever storage server, I do believe
> I'd be strongly consider using Solaris, or one of its variants like
> OpenIndiana, with ZFS.

Special cases warrant exemption.

I, too, run OpenIndiana...

Use the right tool for the job.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 04-14-2011, 02:07 PM
Emmanuel Noobadmin
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On 4/14/11, John R Pierce <pierce@hogranch.com> wrote:
> since this is the centos list, I really didn't want to suggest this, but
> if I was building a 20 or 40TB or whatever storage server, I do believe
> I'd be strongly consider using Solaris, or one of its variants like
> OpenIndiana, with ZFS.
>
> ZFS was engineered from the ground up to scale to zetabytes

I was actually considering this but then came news that Oracle was
killing OpenSolaris and likely to be pushing OCFS so decided I
probably don't want to have something come bite me a year or two down
the road. I'm not sure how things developed since then though.

But based on your recommendation and Christopher Chan's, it would seem
like you guys don't think that long term support/updates would be an
issue for ZFS?
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 04-14-2011, 02:44 PM
Christopher Chan
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On Thursday, April 14, 2011 10:11 PM, Ray Van Dolson wrote:
> On Thu, Apr 14, 2011 at 10:07:55PM +0800, Emmanuel Noobadmin wrote:
>> On 4/14/11, John R Pierce<pierce@hogranch.com> wrote:
>>> since this is the centos list, I really didn't want to suggest this, but
>>> if I was building a 20 or 40TB or whatever storage server, I do believe
>>> I'd be strongly consider using Solaris, or one of its variants like
>>> OpenIndiana, with ZFS.
>>>
>>> ZFS was engineered from the ground up to scale to zetabytes
>>
>> I was actually considering this but then came news that Oracle was
>> killing OpenSolaris and likely to be pushing OCFS so decided I
>> probably don't want to have something come bite me a year or two down
>> the road. I'm not sure how things developed since then though.
>>
>> But based on your recommendation and Christopher Chan's, it would seem
>> like you guys don't think that long term support/updates would be an
>> issue for ZFS?
>
> ZFS and OCFS play in different spaces. And ZFS is going nowhere... if
> you want to use on an "open" OS, OpenIndiana may be a good bet, but
> you're best short-term / "mature" option would be Nexenta or Solaris
> Express.
>

Huh? What gives Nexenta a better advantage over OpenIndiana? They are
both in the same boat. Both will have to migrate to illumos and move
away from the last OpenSolaris ON release. Oh, Nexenta has a company
backing it? Makes no different when both projects will be using the same
core image. Now, if OpenIndiana resists using illumos, then you will
have a case for Nexenta over OpenIndiana.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 04-14-2011, 02:56 PM
Christopher Chan
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On Thursday, April 14, 2011 10:07 PM, Emmanuel Noobadmin wrote:
> On 4/14/11, John R Pierce<pierce@hogranch.com> wrote:
>> since this is the centos list, I really didn't want to suggest this, but
>> if I was building a 20 or 40TB or whatever storage server, I do believe
>> I'd be strongly consider using Solaris, or one of its variants like
>> OpenIndiana, with ZFS.
>>
>> ZFS was engineered from the ground up to scale to zetabytes
>
> I was actually considering this but then came news that Oracle was
> killing OpenSolaris and likely to be pushing OCFS so decided I
> probably don't want to have something come bite me a year or two down
> the road. I'm not sure how things developed since then though.

/me is so happy that the Indiana project surfaced.


>
> But based on your recommendation and Christopher Chan's, it would seem
> like you guys don't think that long term support/updates would be an
> issue for ZFS?

I have found ZFS to be very much stable on OpenSolaris stable releases
and have not had any issues on OpenIndiana 147 with 9 1TB disks in a
raidz2 pool. There are reports of those who had problems when upgrading
but I however have not - whether difference in env/procedure was
responsible I do not know but those chaps got help on the Openindiana ml
with their zfs problems.

I have used OpenSolaris since 2008.05 stable release without issues.
Long term support/updates is not an absolute certainty at the moment
with regards to OpenIndiana but that is only in the terms of packages
outside the core image since illumos is guaranteed.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 04-14-2011, 06:46 PM
John R Pierce
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On 04/14/11 7:44 AM, Christopher Chan wrote:
> Now, if OpenIndiana resists using illumos...

openindiana is under the Illumos project umbrella. They aren't going to
use anything else.

someone suggested Solaris Express, that has no patches or updates unless
you subscribe to annual support at several $1000/year/CPU socket.


_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 04-15-2011, 12:43 AM
Christopher Chan
 
Default Expanding RAID 10 array, WAS: 40TB File System Recommendations

On Friday, April 15, 2011 02:46 AM, John R Pierce wrote:
> On 04/14/11 7:44 AM, Christopher Chan wrote:
>> Now, if OpenIndiana resists using illumos...
>
> openindiana is under the Illumos project umbrella. They aren't going to
> use anything else.

Eh? I was under the impression that they are separate and that Garrett
Damore was rather unhappy with the initial direction of OpenIndiana in
not preparing for an illumos release. 148 is still not illumos as far as
I know.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 10:29 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org