FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 05-21-2008, 03:32 PM
"Craig Simpson"
 
Default Round Robin vs Active/Passive

My Hitachi AMS200 is an Active/Passive array says Hitachi.
By looking at asm13 I see all my paths active. Did use the
"path_grouping_policy multibus" when creating that alias.

The LUNs that are picked up and not aliased in /etc/multipath.conf seem
to show an active/passive setup.

Wondering "How" the [active] or [active] [enables] setup is figured out
by multipath.


(this is a different machine, but with no alias I [active] & [enabled]
path.
mpath5 (1HITACHI_D60090910036) dm-10 HITACHI,DF600F
[size=5.5G][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 0:0:1:36 sdq 65:0 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 0:0:0:36 sdf 8:80 [active][undef]


This one was aliased with below settings in /etc/multipath.conf
Multipath -l
asm13 (1HITACHI_730600240012) dm-53 HITACHI,DF600F
[size=64G][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][active]
\_ 0:0:1:12 sdba 67:64 [active][undef]
\_ 1:0:0:12 sdco 69:192 [active][undef]
\_ 1:0:1:12 sdec 128:64 [active][undef]
\_ 0:0:0:12 sdm 8:192 [active][undef]




>From /etc/multipath.conf:
multipath {
wwid 1HITACHI_730600240012
alias asm13
path_grouping_policy multibus
path_checker readsector0
path_selector "round-robin 0"
failback immediate
}




>From /etc/multipath.conf:
defaults {
udev_dir /dev
polling_interval 10
selector "round-robin 0"
path_grouping_policy multibus
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout /bin/true
path_checker readsector0
rr_min_io 100
rr_weight priorities
failback immediate
no_path_retry fail
user_friendly_name yes
}






Craig




-----Original Message-----
From: dm-devel-bounces@redhat.com [mailto:dm-devel-bounces@redhat.com]
On Behalf Of Craig Simpson
Sent: Wednesday, May 21, 2008 7:48 AM
To: device-mapper development
Subject: [dm-devel] Round Robin vs Active/Passive



For multipathd and dm is round-robin the only mode for multipathing? The
array we have as a newer Hitachi AMS200 and Active/Passive says the
documentation.

Using round-robin seems to work fine with it. Have 4 paths and they are
all sharing 1/4 of the load.

Are there other options that can be set in /etc/multipath.conf? I only
know of round-robin.

Craig

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-21-2008, 06:23 PM
Tore Anderson
 
Default Round Robin vs Active/Passive

* Craig Simpson
>
> My Hitachi AMS200 is an Active/Passive array says Hitachi.
> By looking at asm13 I see all my paths active. Did use the
> "path_grouping_policy multibus" when creating that alias.

The AMS200 is indeed an active/passive array, but it's "fakes"
active/active behaviour - if the passive controller receives an I/O
operation it will redirect it internally to the active one which will
process it and return it back to the passive controller, which in turn
returns it back to the initiator, which have no idea that this happens
at all.

So you can use it as a true active/active array, but I'd recommend
against it for two reasons; first, there might be a slight processing
overhead to route I/O through the passive controller (as well as a
slight increase in latency), second, you might risk saturating the
interconnect between the controllers with re-routed I/O if you have lots
of volumes using the array in this way (this might or might not be a
real problem depending on how the hardware is built).

So what you should do is to distinguish between paths to the active
controller and run round-robin on all of these, while having fail-over
to the set of paths to the passive controller. An example on how this
looks:

mysql (36006016034301f0004582492ab21dd11)
[size=40 GB][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=2][active]
\_ 4:0:2:0 sds 65:32 [active][ready]
\_ 3:0:2:0 sdu 65:64 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 4:0:3:0 sdr 65:16 [active][ready]
\_ 3:0:3:0 sdt 65:48 [active][ready]

I/O is here balanced between sds and sdu, which have the highest
priority. sdr and sdt will only be used should both sds and sdu fail.
This is accomplished by the following two configuration settings:

path_grouping_policy group_by_prio
prio_callout "/sbin/mpath_prio_emc_silent /dev/%n"

(This is an EMC array.)

You should be able to do the same using mpath_prio_hdc_modular as the
prio_callout. Last I checked this callout wasn't actually able to
determine which controller is the preferred for a given volume (one of
the reasons I bought an EMC instead), but did a simplistic check which
was something along the lines of "controller 0 is preferred for all
volumes with an even LUN; controller 1 for all volumes with an odd
LUN". So even though this probably won't match reality unless you take
care to configure the AMS accordingly, you will get the desired effect -
round robin between the paths to one controller, failover to the paths
to the other. The AMS is also clever enough to understand that if
you're only sending I/O to the passive controller it will automatically
change the ownership of the volume to the controller actually receiving
I/O, so you won't have the problem of I/O being re-routed between
controllers.

The downside is that you can't decide which controller is the preferred
one for a given volume, so if you have two highly active volumes with
odd LUNs and two mostly idle one with even LUNs you won't be able to
split the load equally between the controllers.

Regards,
--
Tore Anderson

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-21-2008, 08:21 PM
"Craig Simpson"
 
Default Round Robin vs Active/Passive

Amazing Info, thanks!

Changed my Defaults to this:

defaults {
udev_dir /dev
polling_interval 10
selector "round-robin 0"
path_grouping_policy group_by_prio
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout /sbin/mpath_prio_hds_modular
path_checker readsector0
rr_min_io 100
rr_weight priorities
failback immediate
no_path_retry fail
user_friendly_name yes
}


So figure I don't need to include anything in my aliases, since the
defaults are set.

multipath {
wwid 1HITACHI_D60090910032
alias asm01
}

Did a multipathd -k
And a reconfigure


But when doing a multipath -l Not sure if it looks correct:

asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
[size=32G][features=0][hwhandler=0]
\_ round-robin 0 [prio=0][enabled]
\_ 0:0:1:32 sdm 8:192 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 0:0:0:32 sdb 8:16 [active][undef]



Also a multipathd> show topology

reload: asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
[size=32G ][features=0 ][hwhandler=0 ]
\_ round-robin 0 [prio=1][enabled]
\_ 0:0:1:32 sdm 8:192 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 0:0:0:32 sdb 8:16 [active][ready]



Looks like I have [enabled] [enabled] ...
But it should be [active] [enabled]


Thanks for any feedback.

Craig




-----Original Message-----
From: dm-devel-bounces@redhat.com [mailto:dm-devel-bounces@redhat.com]
On Behalf Of Tore Anderson
Sent: Wednesday, May 21, 2008 11:24 AM
To: device-mapper development
Cc: Michael Denney
Subject: Re: [dm-devel] Round Robin vs Active/Passive

* Craig Simpson
>
> My Hitachi AMS200 is an Active/Passive array says Hitachi.
> By looking at asm13 I see all my paths active. Did use the
> "path_grouping_policy multibus" when creating that alias.

The AMS200 is indeed an active/passive array, but it's "fakes"
active/active behaviour - if the passive controller receives an I/O
operation it will redirect it internally to the active one which will
process it and return it back to the passive controller, which in turn
returns it back to the initiator, which have no idea that this happens
at all.

So you can use it as a true active/active array, but I'd recommend
against it for two reasons; first, there might be a slight processing
overhead to route I/O through the passive controller (as well as a
slight increase in latency), second, you might risk saturating the
interconnect between the controllers with re-routed I/O if you have lots
of volumes using the array in this way (this might or might not be a
real problem depending on how the hardware is built).

So what you should do is to distinguish between paths to the active
controller and run round-robin on all of these, while having fail-over
to the set of paths to the passive controller. An example on how this
looks:

mysql (36006016034301f0004582492ab21dd11)
[size=40 GB][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=2][active]
\_ 4:0:2:0 sds 65:32 [active][ready]
\_ 3:0:2:0 sdu 65:64 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 4:0:3:0 sdr 65:16 [active][ready]
\_ 3:0:3:0 sdt 65:48 [active][ready]

I/O is here balanced between sds and sdu, which have the highest
priority. sdr and sdt will only be used should both sds and sdu fail.
This is accomplished by the following two configuration settings:

path_grouping_policy group_by_prio
prio_callout "/sbin/mpath_prio_emc_silent /dev/%n"

(This is an EMC array.)

You should be able to do the same using mpath_prio_hdc_modular as the
prio_callout. Last I checked this callout wasn't actually able to
determine which controller is the preferred for a given volume (one of
the reasons I bought an EMC instead), but did a simplistic check which
was something along the lines of "controller 0 is preferred for all
volumes with an even LUN; controller 1 for all volumes with an odd
LUN". So even though this probably won't match reality unless you take
care to configure the AMS accordingly, you will get the desired effect -
round robin between the paths to one controller, failover to the paths
to the other. The AMS is also clever enough to understand that if
you're only sending I/O to the passive controller it will automatically
change the ownership of the volume to the controller actually receiving
I/O, so you won't have the problem of I/O being re-routed between
controllers.

The downside is that you can't decide which controller is the preferred
one for a given volume, so if you have two highly active volumes with
odd LUNs and two mostly idle one with even LUNs you won't be able to
split the load equally between the controllers.

Regards,
--
Tore Anderson

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-21-2008, 09:01 PM
Tore Anderson
 
Default Round Robin vs Active/Passive

Hi,

* Craig Simpson

> Amazing Info, thanks!

Glad I could help.

> Changed my Defaults to this:
>
> defaults {
> udev_dir /dev
> polling_interval 10
> selector "round-robin 0"
> path_grouping_policy group_by_prio
> getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
> prio_callout /sbin/mpath_prio_hds_modular
> path_checker readsector0
> rr_min_io 100
> rr_weight priorities
> failback immediate
> no_path_retry fail
> user_friendly_name yes
> }

You need

prio_callout "/sbin/mpath_prio_hds_modular /dev/%n"

for the priority to be determined correctly.

Anyway I'm a bit surprised that you need to specify these things, the
AMS series do have a entry in hwtable.c in multipath-tools 0.4.8 at
least. Running an old version maybe? They don't differ much from what
you have there, though.

> So figure I don't need to include anything in my aliases, since the
> defaults are set.

You figure correctly.

> Did a multipathd -k
> And a reconfigure

You also need to actually reload the multipath maps in the kernel, by
invoking e.g. "multipath -v2".

> But when doing a multipath -l Not sure if it looks correct:
>
> asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
> [size=32G][features=0][hwhandler=0]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:1:32 sdm 8:192 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:0:32 sdb 8:16 [active][undef]

You need to use "multipath -ll" (two l's) for it to show you the
priority, but it looks like multipathd have everything figured out:

> Also a multipathd> show topology
>
> reload: asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
> [size=32G ][features=0 ][hwhandler=0 ]
> \_ round-robin 0 [prio=1][enabled]
> \_ 0:0:1:32 sdm 8:192 [active][ready]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:0:32 sdb 8:16 [active][ready]

It's a bit strange that it actually is able to determine the priority,
considering that you have the prio_callout set incorrectly in your
defaults section.

I suspect that the default values from hwtable.c comes into play and
overrides your settings anyway. You can check this by running "show
config" from inside "multipathd -k" - if you have a device section for
your vendor HITACHI, product DF.* there that might be what's going on.

> Looks like I have [enabled] [enabled] ...
> But it should be [active] [enabled]

Have you sent any I/O to the device after the configuration change? The
PG doesn't transition from enabled to active before some regular I/O has
been sent there. Just reading some data from it should suffice.

Regards,
--
Tore Anderson

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-21-2008, 09:49 PM
"Craig Simpson"
 
Default Round Robin vs Active/Passive

OK, I did notice that my Multipath Tools are a little old

[root@wpe02 sbin]# rpm -qa |grep -i multipath
device-mapper-multipath-0.4.7-12.el5_1.3

Running Oracle Linux 5. Which in truth is:

[root@wpe02 sbin]# cat /etc/redhat-release
Enterprise Linux Enterprise Linux Server release 5.1 (Carthage)

Guess I could grab the latest multipath tools from RedHat for ES 5.1.


Looks like I am on track now.

I looked in
/usr/share/doc/device-mapper-multipath-0.4.7/multipath.conf.defaults
And see the below. I guess that is what multipath is trying to use.
# device {
# vendor "HITACHI"
# product "DF.*"
# getuid_callout "/sbin/scsi_id -g -u -s
/block/%n"
# prio_callout "/sbin/mpath_prio_hds_modular
%d"
# features "0"
# hardware_handler "0"
# path_grouping_policy group_by_prio
# failback immediate
# rr_weight uniform
# rr_min_io 1000
# path_checker readsector0
# }

Created my own version of defaults in /etc/multipath.conf from that:
defaults {
udev_dir /dev
polling_interval 10
selector "round-robin 0"
path_grouping_policy group_by_prio
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_hds_modular /dev/%n"
path_checker readsector0
rr_min_io 1000
rr_weight uniform
failback immediate
no_path_retry fail
user_friendly_name yes
}



Then for an alias I use this:
multipath {
wwid 1HITACHI_D60090910032
alias asm01
}


Output from multipath -ll
asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
[size=32G][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
\_ 0:0:1:32 sdm 8:192 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 0:0:0:32 sdb 8:16 [active][ready]



>From a multipathd -k, "show config"
device {
vendor HITACHI
product DF.*
prio_callout mpath_prio_hds_modular %d
}

So looks like maybe it is incorrect there?
Usually if that is messing up, it shows in /var/log/messages.


THANKS THANKS THANKS Tore!!!!!!!! Would be lost without the help!

Craig



-----Original Message-----
From: dm-devel-bounces@redhat.com [mailto:dm-devel-bounces@redhat.com]
On Behalf Of Tore Anderson
Sent: Wednesday, May 21, 2008 2:01 PM
To: device-mapper development
Cc: Michael Denney
Subject: Re: [dm-devel] Round Robin vs Active/Passive

Hi,

* Craig Simpson

> Amazing Info, thanks!

Glad I could help.

> Changed my Defaults to this:
>
> defaults {
> udev_dir /dev
> polling_interval 10
> selector "round-robin 0"
> path_grouping_policy group_by_prio
> getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
> prio_callout /sbin/mpath_prio_hds_modular
> path_checker readsector0
> rr_min_io 100
> rr_weight priorities
> failback immediate
> no_path_retry fail
> user_friendly_name yes
> }

You need

prio_callout "/sbin/mpath_prio_hds_modular /dev/%n"

for the priority to be determined correctly.

Anyway I'm a bit surprised that you need to specify these things, the
AMS series do have a entry in hwtable.c in multipath-tools 0.4.8 at
least. Running an old version maybe? They don't differ much from what
you have there, though.

> So figure I don't need to include anything in my aliases, since the
> defaults are set.

You figure correctly.

> Did a multipathd -k
> And a reconfigure

You also need to actually reload the multipath maps in the kernel, by
invoking e.g. "multipath -v2".

> But when doing a multipath -l Not sure if it looks correct:
>
> asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
> [size=32G][features=0][hwhandler=0]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:1:32 sdm 8:192 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:0:32 sdb 8:16 [active][undef]

You need to use "multipath -ll" (two l's) for it to show you the
priority, but it looks like multipathd have everything figured out:

> Also a multipathd> show topology
>
> reload: asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
> [size=32G ][features=0 ][hwhandler=0 ]
> \_ round-robin 0 [prio=1][enabled]
> \_ 0:0:1:32 sdm 8:192 [active][ready]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:0:32 sdb 8:16 [active][ready]

It's a bit strange that it actually is able to determine the priority,
considering that you have the prio_callout set incorrectly in your
defaults section.

I suspect that the default values from hwtable.c comes into play and
overrides your settings anyway. You can check this by running "show
config" from inside "multipathd -k" - if you have a device section for
your vendor HITACHI, product DF.* there that might be what's going on.

> Looks like I have [enabled] [enabled] ...
> But it should be [active] [enabled]

Have you sent any I/O to the device after the configuration change? The
PG doesn't transition from enabled to active before some regular I/O has
been sent there. Just reading some data from it should suffice.

Regards,
--
Tore Anderson

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-21-2008, 11:41 PM
"Craig Simpson"
 
Default Round Robin vs Active/Passive

All is Beautiful now. Not logging any errors to messages so looks like "
mpath_prio_hds_modular" is getting called correctly.

Multipath -ll
asm14 (1HITACHI_730600240013) dm-55 HITACHI,DF600F
[size=64G][features=0][hwhandler=0]
\_ round-robin 0 [prio=2][active]
\_ 1:0:0:13 sdcp 69:208 [active][ready]
\_ 0:0:0:13 sdn 8:208 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 0:0:1:13 sdbb 67:80 [active][ready]
\_ 1:0:1:13 sded 128:80 [active][ready]

Thanks,
Craig




-----Original Message-----
From: Craig Simpson
Sent: Wednesday, May 21, 2008 2:49 PM
To: device-mapper development
Cc: Michael Denney; Geoff Quan; Kevin Koplar; Craig Simpson
Subject: RE: [dm-devel] Round Robin vs Active/Passive




OK, I did notice that my Multipath Tools are a little old

[root@wpe02 sbin]# rpm -qa |grep -i multipath
device-mapper-multipath-0.4.7-12.el5_1.3

Running Oracle Linux 5. Which in truth is:

[root@wpe02 sbin]# cat /etc/redhat-release
Enterprise Linux Enterprise Linux Server release 5.1 (Carthage)

Guess I could grab the latest multipath tools from RedHat for ES 5.1.


Looks like I am on track now.

I looked in
/usr/share/doc/device-mapper-multipath-0.4.7/multipath.conf.defaults
And see the below. I guess that is what multipath is trying to use.
# device {
# vendor "HITACHI"
# product "DF.*"
# getuid_callout "/sbin/scsi_id -g -u -s
/block/%n"
# prio_callout "/sbin/mpath_prio_hds_modular
%d"
# features "0"
# hardware_handler "0"
# path_grouping_policy group_by_prio
# failback immediate
# rr_weight uniform
# rr_min_io 1000
# path_checker readsector0
# }

Created my own version of defaults in /etc/multipath.conf from that:
defaults {
udev_dir /dev
polling_interval 10
selector "round-robin 0"
path_grouping_policy group_by_prio
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_hds_modular /dev/%n"
path_checker readsector0
rr_min_io 1000
rr_weight uniform
failback immediate
no_path_retry fail
user_friendly_name yes
}



Then for an alias I use this:
multipath {
wwid 1HITACHI_D60090910032
alias asm01
}


Output from multipath -ll
asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
[size=32G][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
\_ 0:0:1:32 sdm 8:192 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 0:0:0:32 sdb 8:16 [active][ready]



>From a multipathd -k, "show config"
device {
vendor HITACHI
product DF.*
prio_callout mpath_prio_hds_modular %d
}

So looks like maybe it is incorrect there?
Usually if that is messing up, it shows in /var/log/messages.


THANKS THANKS THANKS Tore!!!!!!!! Would be lost without the help!

Craig



-----Original Message-----
From: dm-devel-bounces@redhat.com [mailto:dm-devel-bounces@redhat.com]
On Behalf Of Tore Anderson
Sent: Wednesday, May 21, 2008 2:01 PM
To: device-mapper development
Cc: Michael Denney
Subject: Re: [dm-devel] Round Robin vs Active/Passive

Hi,

* Craig Simpson

> Amazing Info, thanks!

Glad I could help.

> Changed my Defaults to this:
>
> defaults {
> udev_dir /dev
> polling_interval 10
> selector "round-robin 0"
> path_grouping_policy group_by_prio
> getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
> prio_callout /sbin/mpath_prio_hds_modular
> path_checker readsector0
> rr_min_io 100
> rr_weight priorities
> failback immediate
> no_path_retry fail
> user_friendly_name yes
> }

You need

prio_callout "/sbin/mpath_prio_hds_modular /dev/%n"

for the priority to be determined correctly.

Anyway I'm a bit surprised that you need to specify these things, the
AMS series do have a entry in hwtable.c in multipath-tools 0.4.8 at
least. Running an old version maybe? They don't differ much from what
you have there, though.

> So figure I don't need to include anything in my aliases, since the
> defaults are set.

You figure correctly.

> Did a multipathd -k
> And a reconfigure

You also need to actually reload the multipath maps in the kernel, by
invoking e.g. "multipath -v2".

> But when doing a multipath -l Not sure if it looks correct:
>
> asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
> [size=32G][features=0][hwhandler=0]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:1:32 sdm 8:192 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:0:32 sdb 8:16 [active][undef]

You need to use "multipath -ll" (two l's) for it to show you the
priority, but it looks like multipathd have everything figured out:

> Also a multipathd> show topology
>
> reload: asm01 (1HITACHI_D60090910032) dm-6 HITACHI,DF600F
> [size=32G ][features=0 ][hwhandler=0 ]
> \_ round-robin 0 [prio=1][enabled]
> \_ 0:0:1:32 sdm 8:192 [active][ready]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:0:32 sdb 8:16 [active][ready]

It's a bit strange that it actually is able to determine the priority,
considering that you have the prio_callout set incorrectly in your
defaults section.

I suspect that the default values from hwtable.c comes into play and
overrides your settings anyway. You can check this by running "show
config" from inside "multipathd -k" - if you have a device section for
your vendor HITACHI, product DF.* there that might be what's going on.

> Looks like I have [enabled] [enabled] ...
> But it should be [active] [enabled]

Have you sent any I/O to the device after the configuration change? The
PG doesn't transition from enabled to active before some regular I/O has
been sent there. Just reading some data from it should suffice.

Regards,
--
Tore Anderson

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-22-2008, 08:24 AM
Domenico Viggiani
 
Default Round Robin vs Active/Passive

* Tore Anderson
>
> The AMS200 is indeed an active/passive array, but it's "fakes"
> active/active behaviour - if the passive controller receives
> an I/O operation it will redirect it internally to the active
> one which will process it and return it back to the passive
> controller, which in turn returns it back to the initiator,
> which have no idea that this happens at all.
>
> So you can use it as a true active/active array, but I'd
> recommend against it for two reasons; first, there might be
> a slight processing overhead to route I/O through the passive
> controller (as well as a slight increase in latency), second,
> you might risk saturating the interconnect between the
> controllers with re-routed I/O if you have lots of volumes
> using the array in this way (this might or might not be a
> real problem depending on how the hardware is built).
>
> So what you should do is to distinguish between paths to the
> active controller and run round-robin on all of these, while
> having fail-over to the set of paths to the passive
> controller. An example on how this
> looks:
>
> mysql (36006016034301f0004582492ab21dd11)
> [size=40 GB][features=1 queue_if_no_path][hwhandler=0] \_
> round-robin 0 [prio=2][active] \_ 4:0:2:0 sds 65:32
> [active][ready] \_ 3:0:2:0 sdu 65:64 [active][ready] \_
> round-robin 0 [prio=0][enabled] \_ 4:0:3:0 sdr 65:16
> [active][ready] \_ 3:0:3:0 sdt 65:48 [active][ready]
>
> I/O is here balanced between sds and sdu, which have the
> highest priority. sdr and sdt will only be used should both
> sds and sdu fail.
> This is accomplished by the following two configuration settings:
>
> path_grouping_policy group_by_prio
> prio_callout "/sbin/mpath_prio_emc_silent /dev/%n"
>
> (This is an EMC array.)
>
> You should be able to do the same using
> mpath_prio_hdc_modular as the prio_callout. Last I checked
> this callout wasn't actually able to determine which
> controller is the preferred for a given volume (one of the
> reasons I bought an EMC instead), but did a simplistic check
> which was something along the lines of "controller 0 is
> preferred for all volumes with an even LUN; controller 1 for
> all volumes with an odd LUN". So even though this probably
> won't match reality unless you take care to configure the AMS
> accordingly, you will get the desired effect - round robin
> between the paths to one controller, failover to the paths to
> the other. The AMS is also clever enough to understand that
> if you're only sending I/O to the passive controller it will
> automatically change the ownership of the volume to the
> controller actually receiving I/O, so you won't have the
> problem of I/O being re-routed between controllers.
>
> The downside is that you can't decide which controller is the
> preferred one for a given volume, so if you have two highly
> active volumes with odd LUNs and two mostly idle one with
> even LUNs you won't be able to split the load equally between
> the controllers.

Sorry for question: is this how new ALUA mode works for EMC Clariion CX
arrays?
Are default settings suitable for this new failover mode?

I just upgraded my CX700 to FLARE26 with ALUA mode...

Thanks
--
Domenico Viggiani

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-22-2008, 11:57 AM
Tore Anderson
 
Default Round Robin vs Active/Passive

* Domenico Viggiani

> Sorry for question: is this how new ALUA mode works for EMC Clariion
> CX arrays?

Yes, that's right. Except for the fact that mpath_prio_emc will
correctly detect the preferred controller, while mpath_prio_hds_modular
only checks even/odd LUNs.

> Are default settings suitable for this new failover mode?

You don't have to use the EMC specific hardware handler or path checker
any longer. This is what I use for my CX3:

device {
vendor DGC
product *
product_blacklist LUNZ
path_grouping_policy group_by_prio
path_checker tur
no_path_retry queue
prio_callout "/sbin/mpath_prio_emc /dev/%n"
failback immediate
}

Note that the host is no longer able to explicitly trespass the volume
between controllers. I actually see that as an advantage, especially in
cluster environments. If the host wants to change controllers it can
simply do so and wait for the CX to implicitly trespass the volume (due
to the I/O coming mostly to the passive controller). This works very
well, and I consider it a huge improvement over the old
Passive-Not-Ready mode you had to use earlier (hwhandler "emc").

Note that there's also a new ALUA-specific hardware handler available
now. I never tried it, so I can't tell how it differs from my setup.

Regards,
--
Tore Anderson

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-22-2008, 12:00 PM
Tore Anderson
 
Default Round Robin vs Active/Passive

* Craig Simpson

> All is Beautiful now. Not logging any errors to messages so looks like "
> mpath_prio_hds_modular" is getting called correctly.
>
> Multipath -ll
> asm14 (1HITACHI_730600240013) dm-55 HITACHI,DF600F
> [size=64G][features=0][hwhandler=0]
> \_ round-robin 0 [prio=2][active]
> \_ 1:0:0:13 sdcp 69:208 [active][ready]
> \_ 0:0:0:13 sdn 8:208 [active][ready]
> \_ round-robin 0 [prio=0][enabled]
> \_ 0:0:1:13 sdbb 67:80 [active][ready]
> \_ 1:0:1:13 sded 128:80 [active][ready]

Yes, you can see from this output that sdcp and sdn have a prio of 1,
placing them in the same priority group (which in sum have a prio of 2),
so the prio-callout definitely works as it is supposed to. Everything
looks good, now you should try yanking some fibres (with lots of I/O
activity running) to make sure that it actually is able to handle a
failure scenario. Good luck.

Regards,
--
Tore Anderson

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-23-2008, 07:16 AM
Hannes Reinecke
 
Default Round Robin vs Active/Passive

Hi Domenico,

On Thu, May 22, 2008 at 10:24:52AM +0200, Domenico Viggiani wrote:
> * Tore Anderson
> >
> > path_grouping_policy group_by_prio
> > prio_callout "/sbin/mpath_prio_emc_silent /dev/%n"
> >
> > (This is an EMC array.)
> >
[ .. ]
>
> Sorry for question: is this how new ALUA mode works for EMC Clariion CX
> arrays?
> Are default settings suitable for this new failover mode?
>
> I just upgraded my CX700 to FLARE26 with ALUA mode...
>
No. Alua is completely different. You have to use

prio_callout "/sbin/mpath_prio_alua /dev/%n"

for this.
Although the normal EMC configuration continues to work, too.
And also note that you have to change the failover mode
to '4' to enable ALUA on the Clariion.

Cheers,

Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 N�g
GF: Markus Rex, HRB 16746 (AG N�g)

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 09:15 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org