FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 11-15-2011, 02:34 PM
Gianluca Cecchi
 
Default info on enabling only one path with rdac and DS4700

Hello,
with RH EL 5.7 x86_64 and updates:
# uname -r
2.6.18-274.7.1.el5
# rpm -q device-mapper-multipath
device-mapper-multipath-0.4.7-46.el5_7.1

I'm on a IBM HS21 blade connected to a DS4700 storage array.
Without doing nothing in multipath.conf but blacklisting the internal
disk, I get this kind of configuration:
# multipath -l
mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
[size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=0][active]
\_ 3:0:0:1 sdb 8:16 [active][undef]
\_ 4:0:0:1 sdc 8:32 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:1:1 sdd 8:48 [active][undef]
\_ 4:0:1:1 sde 8:64 [active][undef]

Is there any config option I can force to use only one path at a time
in the active controller, instead of two (sdb and sdc in my case) in
round robin as I see in iostat?

Also, I noticed that output with "-ll" shows "200" as priority of
paths for the active controller, while "-l" shows 0 for both (see
above)...
Is this a bug or expected difference due to command different switch/verbosity?
(I see similar behaviour also with another server connected to a
DS6800 storage array)
# multipath -ll
mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
[size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=200][active]
\_ 3:0:0:1 sdb 8:16 [active][ready]
\_ 4:0:0:1 sdc 8:32 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:1:1 sdd 8:48 [active][ghost]
\_ 4:0:1:1 sde 8:64 [active][ghost]

I would like to use default multipath instead of LSI provided rdac, found at
http://www.lsi.com/sep/Pages/rdac/ds4000.aspx
version 09.03.0C05.0504 , dated 7/18/2011 with:
New linux kernel versions supported with this release:
- SLES 10-SP3: 2.6.16.60-0.54.5
- SLES 10-SP4: 2.6.16.60-0.85.1
- SLES 11.1 : 2.6.32.12-0.7
- RHEL5-u5 : 2.6.18-194
- RHEL5-u6 : 2.6.18-238
- RHEL6 : 2.6.32-71

In fact I'm using 2.6.18-274.7.1.el5 that is not listed...
Any known cons in using device-mapper-multipath with IBM,1814?

Thanks in advance,
Gianluca

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-15-2011, 03:01 PM
"Moger, Babu"
 
Default info on enabling only one path with rdac and DS4700

> -----Original Message-----
> From: Gianluca Cecchi [mailto:gianluca.cecchi@gmail.com]
> Sent: Tuesday, November 15, 2011 9:34 AM
> To: device-mapper development
> Subject: [dm-devel] info on enabling only one path with rdac and DS4700
>
> Hello,
> with RH EL 5.7 x86_64 and updates:
> # uname -r
> 2.6.18-274.7.1.el5
> # rpm -q device-mapper-multipath
> device-mapper-multipath-0.4.7-46.el5_7.1
>
> I'm on a IBM HS21 blade connected to a DS4700 storage array.
> Without doing nothing in multipath.conf but blacklisting the internal
> disk, I get this kind of configuration:
> # multipath -l
> mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
> [size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
> \_ round-robin 0 [prio=0][active]
> \_ 3:0:0:1 sdb 8:16 [active][undef]
> \_ 4:0:0:1 sdc 8:32 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> \_ 3:0:1:1 sdd 8:48 [active][undef]
> \_ 4:0:1:1 sde 8:64 [active][undef]

This output looks good to me. I don’t see any problem..

>
> Is there any config option I can force to use only one path at a time
> in the active controller, instead of two (sdb and sdc in my case) in
> round robin as I see in iostat?

Not sure about the purpose of this. Either you have to blacklist by device node(sdb or sdc) or change your config to see only one path to the controller.

>
> Also, I noticed that output with "-ll" shows "200" as priority of
> paths for the active controller, while "-l" shows 0 for both (see
> above)...
> Is this a bug or expected difference due to command different
> switch/verbosity?

It is not a bug. It will not run all the commands if you run with "-l"

> (I see similar behaviour also with another server connected to a
> DS6800 storage array)
> # multipath -ll
> mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
> [size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
> \_ round-robin 0 [prio=200][active]
> \_ 3:0:0:1 sdb 8:16 [active][ready]
> \_ 4:0:0:1 sdc 8:32 [active][ready]
> \_ round-robin 0 [prio=0][enabled]
> \_ 3:0:1:1 sdd 8:48 [active][ghost]
> \_ 4:0:1:1 sde 8:64 [active][ghost]
>
> I would like to use default multipath instead of LSI provided rdac,
> found at
> http://www.lsi.com/sep/Pages/rdac/ds4000.aspx
> version 09.03.0C05.0504 , dated 7/18/2011 with:
> New linux kernel versions supported with this release:
> - SLES 10-SP3: 2.6.16.60-0.54.5
> - SLES 10-SP4: 2.6.16.60-0.85.1
> - SLES 11.1 : 2.6.32.12-0.7
> - RHEL5-u5 : 2.6.18-194
> - RHEL5-u6 : 2.6.18-238
> - RHEL6 : 2.6.32-71
>
> In fact I'm using 2.6.18-274.7.1.el5 that is not listed...
> Any known cons in using device-mapper-multipath with IBM,1814?
>
> Thanks in advance,
> Gianluca
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-15-2011, 07:10 PM
Johannes Hirte
 
Default info on enabling only one path with rdac and DS4700

On Tuesday 15 November 2011 16:34:20 Gianluca Cecchi wrote:
> Hello,
> with RH EL 5.7 x86_64 and updates:
> # uname -r
> 2.6.18-274.7.1.el5
> # rpm -q device-mapper-multipath
> device-mapper-multipath-0.4.7-46.el5_7.1
>
> I'm on a IBM HS21 blade connected to a DS4700 storage array.
> Without doing nothing in multipath.conf but blacklisting the internal
> disk, I get this kind of configuration:
> # multipath -l
> mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
> [size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
> \_ round-robin 0 [prio=0][active]
> \_ 3:0:0:1 sdb 8:16 [active][undef]
> \_ 4:0:0:1 sdc 8:32 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> \_ 3:0:1:1 sdd 8:48 [active][undef]
> \_ 4:0:1:1 sde 8:64 [active][undef]
>
> Is there any config option I can force to use only one path at a time
> in the active controller, instead of two (sdb and sdc in my case) in
> round robin as I see in iostat?
>
> Also, I noticed that output with "-ll" shows "200" as priority of
> paths for the active controller, while "-l" shows 0 for both (see
> above)...
> Is this a bug or expected difference due to command different switch/verbosity?
> (I see similar behaviour also with another server connected to a
> DS6800 storage array)
> # multipath -ll
> mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
> [size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
> \_ round-robin 0 [prio=200][active]
> \_ 3:0:0:1 sdb 8:16 [active][ready]
> \_ 4:0:0:1 sdc 8:32 [active][ready]
> \_ round-robin 0 [prio=0][enabled]
> \_ 3:0:1:1 sdd 8:48 [active][ghost]
> \_ 4:0:1:1 sde 8:64 [active][ghost]
>
> I would like to use default multipath instead of LSI provided rdac, found at
> http://www.lsi.com/sep/Pages/rdac/ds4000.aspx
> version 09.03.0C05.0504 , dated 7/18/2011 with:
> New linux kernel versions supported with this release:
> - SLES 10-SP3: 2.6.16.60-0.54.5
> - SLES 10-SP4: 2.6.16.60-0.85.1
> - SLES 11.1 : 2.6.32.12-0.7
> - RHEL5-u5 : 2.6.18-194
> - RHEL5-u6 : 2.6.18-238
> - RHEL6 : 2.6.32-71
>
> In fact I'm using 2.6.18-274.7.1.el5 that is not listed...
> Any known cons in using device-mapper-multipath with IBM,1814?

The rdac driver from LSI won't even compile, IIRC. With the dm-multipath driver you have to disable AVT on the DS4700. Otherwise the rdac module won't work with the storage.

regards,
Johannes

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-15-2011, 07:38 PM
Christophe Varoqui
 
Default info on enabling only one path with rdac and DS4700

> I'm on a IBM HS21 blade connected to a DS4700 storage array.
> Without doing nothing in multipath.conf but blacklisting the internal
> disk, I get this kind of configuration:
> # multipath -l
> mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
> [size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
> \_ round-robin 0 [prio=0][active]
> \_ 3:0:0:1 sdb 8:16 [active][undef]
> \_ 4:0:0:1 sdc 8:32 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> \_ 3:0:1:1 sdd 8:48 [active][undef]
> \_ 4:0:1:1 sde 8:64 [active][undef]
>
> Is there any config option I can force to use only one path at a time
> in the active controller, instead of two (sdb and sdc in my case) in
> round robin as I see in iostat?
>
What ever the reason you'd want to do that, setting the path group
policy to failover would create 4 1-path groups. I'd have 1 active paths
and 3 spares.

Regards,
--
Christophe Varoqui
OpenSVC - Tools to scale
http://www.opensvc.com/

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-15-2011, 09:35 PM
Gianluca Cecchi
 
Default info on enabling only one path with rdac and DS4700

On Tue, Nov 15, 2011 at 9:38 PM, Christophe Varoqui wrote:

>>
>> Is there any config option I can force to use only one path at a time
>> in the active controller, instead of two (sdb and sdc in my case) in
>> round robin as I see in iostat?
>>
> What ever the reason you'd want to do that, setting the path group
> policy to failover would create 4 1-path groups. I'd have 1 active paths
> and 3 spares.
>

Hi,
thank you all for the helpful answers.
Some more details to explain better:
- this system is a reinstall where before there was RH EL 4.x 32bit with rdac
rdac experience was a pain in terms of being not up2date and also
providing sometimes "unnecessary" kernel panic
(eg when running "mppUtil -o" to show options...)
So I would like now to go through device-mapper-multipath on rh el 5 x86_64

- This storage array is not under my total control
It is used by other systems, mainly Windows systems and other rh el
(these ones using using rdac...)
Win admins told me that standard drivers installation for that OS
keeps by default only one of the two paths of the active controller
for I/O operations.
And normally they keep this default as config.
So I wanted to investigate the chance to mimic this behaviour with multipath...

- One possible reason to force one fixed path as primary could be to
"statically" balance overall I/O to the storage array using one path
for some of the systems and the other one for the remaining ones
Possibly same results using round robin for each server connected, IMO

- I'm going to check tomorrow about AVT current configuration as noted
inside one of the answers.
BTW: on the storage array there are snmp traps configured.
After enabling multipath and rebooting the system, I still see the
trap below, sent by the storage (only one)... Could it be related to
this kind of configuration option at storage side?

Here the snmp message
Summary
Node ID: XXXX_DS4700
Host IP Address:
Host ID: Out-of-Band
Event Error Code: 4011
Event occurred: 15-nov-2011 14.49.36
Event Message: Logical Drive not on preferred path due to ADT/RDAC failover
Component type: Controller
Component location: Controller in slot B

- On the presented LUN I configured a PV, VG, LV and ext4 fs (not system fs)
At reboot at host side I see messages related to duplicated PV IDs for
the paths (sdb, sdc, sdd, sde): they comes before vg activation and
before multipathd start...
Is this normal, because at the first vgscan run during boot, multipath
configuration has not been instantiated yet..?
I have to check, but I don't remember similar messages with eh el 5.7
in other SAN configurations, where the VG is not a system VG....

Thanks again

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-16-2011, 09:56 AM
Gianluca Cecchi
 
Default info on enabling only one path with rdac and DS4700

In the mean time, while waiting for further answers, I successfully
tested the failover group policy config
My way of proceedings was this, hope correct.
I didn't find any mention of IBM 1814 in multipath.conf.annotated...
I did find inside it
#defaults {
...
# # name : path_grouping_policy
# # scope : multipath
# # desc : the default path grouping policy to apply to unspecified
# # multipaths
# # values : failover = 1 path per priority group
# # multibus = all valid paths in 1 priority group
# # group_by_serial = 1 priority group per detected serial
# # number
# # group_by_prio = 1 priority group per path priority
# # value
# # group_by_node_name = 1 priority group per target node name
# # default : failover
# #
# path_grouping_policy multibus

this seemed a bit misleading because it is not clear to me if the
default then would be multibus or failover...

Setting failover as the group policy inside the defaults {} section of
multipath.conf didn't work.
defaults {
user_friendly_names yes
path_grouping_policy failover
}

.. probably overwritten by device itself config default?

Anyway I tried another "path"

# multipahd -k
multipathd> show config
...
found now IBM 1814 inside the output with these specs (donna if would
have found a similar result in a more general section in
multipath.conf.annotated ...)

#devices {
# device {
# vendor "IBM"
# product "1814"
# path_grouping_policy group_by_prio
# getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
# path_selector "round-robin 0"
# path_checker rdac
# features "0"
# hardware_handler "1 rdac"
# prio_callout "/sbin/mpath_prio_rdac /dev/%n"
# failback immediate
# rr_weight uniform
# no_path_retry queue
# rr_min_io 1000
# }
#

So I wanted only to change the path_grouping_policy parameter and I
set this in my multipath.conf


devices {
device {
vendor "IBM"
product "1814"
path_grouping_policy failover
}

}

Then
# multipath -v2 -d
and then committed (after deactivating fs and VG, donna if necessary...) with
# multipath -v2

Now I have indeed:
# multipath -l
mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
[size=3.4T][features=1 queue_if_no_path][hwhandler=0][rw]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:0:1 sdb 8:16 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 4:0:0:1 sdc 8:32 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:1:1 sdd 8:48 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 4:0:1:1 sde 8:64 [active][undef]

Tried this command
# time dd if=/dev/zero of=testfile2 bs=1024k count=102400
102400+0 records in
102400+0 records out
107374182400 bytes (107 GB) copied, 651.872 seconds, 165 MB/s

real 10m51.934s
user 0m0.048s
sys 3m41.549s

And iostat confirmed only sdb was used during I/O operation
I got about 5% performance gain, as yesterday
# time dd if=/dev/zero of=testfile bs=1024k count=102400
102400+0 records in
102400+0 records out
107374182400 bytes (107 GB) copied, 686.96 seconds, 156 MB/s

real 11m27.010s
user 0m0.070s
sys 3m33.813s

But it could also be related to different I/O going through the DS4700 today.

BTW: I would expect "multipathd -k" --> show config to be updated
after changing the policy, but it continues to give
path_grouping_policy group_by_prio
for the IBM 1814
Probably it is a static display... any way to show the current one
with this command?

Sorry for the many answers... probably more targeted at dm-user ml...

Gianluca

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-16-2011, 12:24 PM
Johannes Hirte
 
Default info on enabling only one path with rdac and DS4700

On Tuesday 15 November 2011 23:35:53 Gianluca Cecchi wrote:
> - I'm going to check tomorrow about AVT current configuration as noted
> inside one of the answers.
> BTW: on the storage array there are snmp traps configured.
> After enabling multipath and rebooting the system, I still see the
> trap below, sent by the storage (only one)... Could it be related to
> this kind of configuration option at storage side?
>
> Here the snmp message
> Summary
> Node ID: XXXX_DS4700
> Host IP Address:
> Host ID: Out-of-Band
> Event Error Code: 4011
> Event occurred: 15-nov-2011 14.49.36
> Event Message: Logical Drive not on preferred path due to ADT/RDAC failover
> Component type: Controller
> Component location: Controller in slot B

Yes, this is because the rdac module detected the LUN in AVT mode and refused
to work with it. This will happen every time you access a ghost path without
rdac.

> - On the presented LUN I configured a PV, VG, LV and ext4 fs (not system fs)
> At reboot at host side I see messages related to duplicated PV IDs for
> the paths (sdb, sdc, sdd, sde): they comes before vg activation and
> before multipathd start...
> Is this normal, because at the first vgscan run during boot, multipath
> configuration has not been instantiated yet..?
> I have to check, but I don't remember similar messages with eh el 5.7
> in other SAN configurations, where the VG is not a system VG....

You should avoid to access the sdX directly. If you need to run lvm before
multipath is up, you can blacklist the sdX in the lvm.conf.


regards,
Johannes

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-23-2011, 04:22 PM
Gianluca Cecchi
 
Default info on enabling only one path with rdac and DS4700

On Wed, Nov 16, 2011 at 2:24 PM, Johannes Hirte wrote:
[snip]
> Yes, this is because the rdac module detected the LUN in AVT mode and refused
> to work with it. This will happen every time you access a ghost path without
> rdac.

>
>> - On the presented LUN I configured a PV, VG, LV and ext4 fs (not system fs)
>> At reboot at host side I see messages related to duplicated PV IDs for
>> the paths (sdb, sdc, sdd, sde): they comes before vg activation and
>> before multipathd start...
>> Is this normal, because at the first vgscan run during boot, multipath
>> configuration has not been instantiated yet..?
>> I have to check, but I don't remember similar messages with eh el 5.7
>> in other SAN configurations, where the VG is not a system VG....
>
> You should avoid to access the sdX directly. If you need to run lvm before
> multipath is up, you can blacklist the sdX in the lvm.conf.


So I configured:
- LUN on DS4700 as LNXCLVMWARE that I found should disable AVT
- multipath as standard without any particular setting (I only
blacklisted the internal disk)

At the start time of the system I get (both in console and then I
found it in /var/log/messages too):

Nov 23 17:32:56 testserver kernel: sdc:end_request: I/O error, dev
sdb, sector 0
Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdb,
logical block 0
Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdc, sector 0
Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdc,
logical block 0
Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdb, sector 0
Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdb,
logical block 0
Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdc, sector 0
Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdc,
logical block 0
Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdb, sector 0
Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdb,
logical block 0
Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdc, sector 0
Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdc,
logical block 0
Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdb, sector 0
Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdb,
logical block 0
Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdc, sector 0
Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdc,
logical block 0

This happens for sdb ad sdc only (probably passive controller disk paths?)

And this other ones:
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb, sector 0
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb,
sector 7320493952
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb,
sector 7320494064
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb, sector 0
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb, sector 8
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb, sector 0
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdc, sector 0
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdc,
sector 7320493952
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdc,
sector 7320494064
Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdc, sector 0
...
then
Nov 23 17:33:00 testserver kernel: device-mapper: multipath: version
1.0.6 loaded
Nov 23 17:33:00 testserver kernel: sd 3:0:0:1: rdac: LUN 1 (unowned)
Nov 23 17:33:00 testserver kernel: sd 3:0:1:1: rdac: LUN 1 (owned)
Nov 23 17:33:00 testserver kernel: sd 4:0:0:1: rdac: LUN 1 (unowned)
Nov 23 17:33:00 testserver kernel: sd 4:0:1:1: rdac: LUN 1 (owned)
Nov 23 17:33:01 testserver kernel: rdac: device handler registered
Nov 23 17:33:01 testserver kernel: device-mapper: multipath: Using
scsi_dh module scsi_dh_rdac for failover/failback and device
management.
Nov 23 17:33:01 testserver kernel: device-mapper: multipath
round-robin: version 1.0.0 loaded
Nov 23 17:33:01 testserver kernel: sd 3:0:0:1: rdac: array
Z1_BEIC_DS4700, ctlr 0, queueing MODE_SELECT command
Nov 23 17:33:01 testserver kernel: sd 3:0:0:1: rdac: array
Z1_BEIC_DS4700, ctlr 0, MODE_SELECT completed
Nov 23 17:33:01 testserver kernel: sd 4:0:0:1: rdac: array
Z1_BEIC_DS4700, ctlr 0, queueing MODE_SELECT command
Nov 23 17:33:01 testserver kernel: sd 4:0:0:1: rdac: array
Z1_BEIC_DS4700, ctlr 0, MODE_SELECT completed
Nov 23 17:33:01 testserver kernel: end_request: I/O error, dev sdd,
sector 7320494072
Nov 23 17:33:01 testserver kernel: printk: 4 messages suppressed.
Nov 23 17:33:01 testserver kernel: Buffer I/O error on device sdd,
logical block 915061759
Nov 23 17:33:01 testserver kernel: end_request: I/O error, dev sde,
sector 7320494072
Nov 23 17:33:02 testserver kernel: printk: 49 messages suppressed.
Nov 23 17:33:02 testserver kernel: Buffer I/O error on device sde,
logical block 0
Nov 23 17:33:02 testserver kernel: Buffer I/O error on device sde,
logical block 2
Nov 23 17:33:02 testserver kernel: Buffer I/O error on device sde,
logical block 3

And then no other I/O error messages. Donna if this is avoidable or not....

So after complete startup the situation is:
[root@testserver ~]# multipath -l
mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
[size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=0][active]
\_ 3:0:0:1 sdb 8:16 [active][undef]
\_ 4:0:0:1 sdc 8:32 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 3:0:1:1 sdd 8:48 [active][undef]
\_ 4:0:1:1 sde 8:64 [active][undef]

When I activate the LVM on mpath1 PV and mount the file system:
Nov 23 17:34:31 testserver kernel: EXT4-fs (dm-7): mounted filesystem
with ordered data mode
Nov 23 17:34:47 testserver kernel: JBD: barrier-based sync failed on
dm-7-8 - disabling barriers
--> donna if it is to be intended as a problem

I instantiate I/O without problems.

Then I test to change active controller for the lun at DS4700 side
during a running I/O session (dd seq read of 10Gb) , and I get
Nov 23 17:43:02 testserver kernel: end_request: I/O error, dev sdb,
sector 2110328
Nov 23 17:43:02 testserver kernel: device-mapper: multipath: Failing path 8:16.
Nov 23 17:43:02 testserver multipathd: 8:16: mark as failed
Nov 23 17:43:02 testserver multipathd: mpath1: remaining active paths: 3
Nov 23 17:43:02 testserver multipathd: dm-6: add map (uevent)
Nov 23 17:43:02 testserver multipathd: dm-6: devmap already registered
Nov 23 17:43:03 testserver kernel: end_request: I/O error, dev sdc,
sector 2110328
Nov 23 17:43:03 testserver kernel: device-mapper: multipath: Failing path 8:32.
Nov 23 17:43:03 testserver multipathd: dm-6: add map (uevent)
Nov 23 17:43:03 testserver multipathd: dm-6: devmap already registered
Nov 23 17:43:03 testserver multipathd: 8:32: mark as failed
Nov 23 17:43:03 testserver multipathd: mpath1: remaining active paths: 2
Nov 23 17:43:06 testserver multipathd: sdb: rdac checker reports path is ghost
Nov 23 17:43:06 testserver multipathd: 8:16: reinstated
Nov 23 17:43:06 testserver multipathd: mpath1: remaining active paths: 3
Nov 23 17:43:06 testserver kernel: device-mapper: multipath: Using
scsi_dh module scsi_dh_rdac for failover/failback and device
management.
Nov 23 17:43:06 testserver multipathd: mpath1: load table [0
7320494080 multipath 0 1 rdac 2 1 round-robin 0 3 1 8:32 1000 8:48
1000 8:64 1000 round-robin 0 1 1 8:16
Nov 23 17:43:06 testserver multipathd: dm-6: add map (uevent)
Nov 23 17:43:06 testserver multipathd: dm-6: devmap already registered
Nov 23 17:43:06 testserver kernel: device-mapper: multipath: Failing path 8:32.
Nov 23 17:43:06 testserver multipathd: dm-6: add map (uevent)
Nov 23 17:43:06 testserver multipathd: dm-6: devmap already registered
Nov 23 17:43:06 testserver multipathd: dm-6: add map (uevent)
Nov 23 17:43:06 testserver multipathd: dm-6: devmap already registered
Nov 23 17:43:07 testserver multipathd: sdc: rdac checker reports path is ghost
Nov 23 17:43:07 testserver multipathd: 8:32: reinstated
Nov 23 17:43:07 testserver kernel: sd 4:0:0:1: rdac: array
Z1_BEIC_DS4700, ctlr 0, queueing MODE_SELECT command
Nov 23 17:43:07 testserver multipathd: mpath1: remaining active paths: 4
Nov 23 17:43:07 testserver kernel: device-mapper: multipath: Using
scsi_dh module scsi_dh_rdac for failover/failback and device
management.
Nov 23 17:43:08 testserver kernel: sd 4:0:0:1: rdac: array
Z1_BEIC_DS4700, ctlr 0, MODE_SELECT completed
Nov 23 17:43:08 testserver multipathd: mpath1: load table [0
7320494080 multipath 0 1 rdac 2 1 round-robin 0 2 1 8:48 1000 8:64
1000 round-robin 0 2 1 8:32 1000 8:16
Nov 23 17:43:08 testserver multipathd: dm-6: add map (uevent)
Nov 23 17:43:08 testserver multipathd: dm-6: devmap already registered
Nov 23 17:43:08 testserver multipathd: dm-6: add map (uevent)
Nov 23 17:43:08 testserver multipathd: dm-6: devmap already registered
Nov 23 17:43:08 testserver kernel: sd 3:0:1:1: rdac: array
Z1_BEIC_DS4700, ctlr 1, queueing MODE_SELECT command
Nov 23 17:43:10 testserver kernel: sd 3:0:1:1: rdac: array
Z1_BEIC_DS4700, ctlr 1, MODE_SELECT completed
Nov 23 17:43:10 testserver kernel: sd 4:0:1:1: rdac: array
Z1_BEIC_DS4700, ctlr 1, queueing MODE_SELECT command
Nov 23 17:43:11 testserver kernel: sd 4:0:1:1: rdac: array
Z1_BEIC_DS4700, ctlr 1, MODE_SELECT completed
Nov 23 17:43:12 testserver multipathd: sdd: rdac checker reports path is up
Nov 23 17:43:12 testserver multipathd: 8:48: reinstated
Nov 23 17:43:12 testserver multipathd: sde: rdac checker reports path is up
Nov 23 17:43:12 testserver multipathd: 8:64: reinstated

The overall increased time is 3-4 seconds for a 1 minute I/O period
Without failover:
[root@testserver ~]# time dd if=/testfs/testfile bs=1024k count=10000
of=/dev/null
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 61.9951 seconds, 169 MB/s

real 1m1.996s
user 0m0.002s
sys 0m7.088s

With change of active controller in the mid:
[root@testserver ~]# time dd if=/testfs/testfile1 bs=1024k count=10000
of=/dev/null
10000+0 records in
10000+0 records out
10485760000 bytes (10 GB) copied, 65.6529 seconds, 160 MB/s

real 1m5.654s
user 0m0.007s
sys 0m7.175s

So, quite good, and without error at user side.

at the end the multipath config is this:
[root@testserver ~]# multipath -l
mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
[size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
\_ round-robin 0 [prio=0][active]
\_ 3:0:1:1 sdd 8:48 [active][undef]
\_ 4:0:1:1 sde 8:64 [active][undef]
\_ round-robin 0 [prio=0][enabled]
\_ 4:0:0:1 sdc 8:32 [active][undef]
\_ 3:0:0:1 sdb 8:16 [active][undef]

Questions:

1) Can I conclude it is ok as a configuration? Or any other tests to carry on?
I confirm I didn't get any snmp trap from the ds4700 as happened before...

2) At the moment I put this in lvm.conf to whitelist the root volume
groups and blacklist the san individual paths and then delete the
.cache file and reboot
filter = [ "a/dev/mapper/.*/", "a/dev/sda/", "a/dev/sda2/", "r/dev/sd.*/" ]
Is this ok?
If root PV is on sda2, do I need to whitelist both sda and sda2 or only sda2?

3) Based on messages during failover, is it true that I can avoid
explicitly put scsi_dh in initrd?
If I create initrd this way:
mkinitrd /boot/initrd-$(uname -r)-scsi_dh.img $(uname -r) --preload=scsi_dh_rdac
I get this difference:
[root@testserver ~]# diff /tmp/new/init /tmp/current/init
44,51d43
< echo "Loading scsi_mod.ko module"
< insmod /lib/scsi_mod.ko
< echo "Loading sd_mod.ko module"
< insmod /lib/sd_mod.ko
< echo "Loading scsi_dh.ko module"
< insmod /lib/scsi_dh.ko
< echo "Loading scsi_dh_rdac.ko module"
< insmod /lib/scsi_dh_rdac.ko
62a55,58
> echo "Loading scsi_mod.ko module"
> insmod /lib/scsi_mod.ko
> echo "Loading sd_mod.ko module"
> insmod /lib/sd_mod.ko

or will it help in any way?
BTW: The I/O tests above were done with standard initrd (so the > side
of the diff without the scsi_dh_rdac)
I only run the mkinitrd to sort out how would have been create the init file...

4) the san lun is 3.4Tb and I'm going to add another one of about 5Tb
In messages I see this
Nov 23 17:32:58 testserver kernel: sde : very big device. try to use
READ CAPACITY(16).

I found in an old kernel ml post that actually it should mean "trying
to use" ... so only informational message.
Can anyone confirm this?

Thanks again in advance for your help,
Gianluca

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-28-2011, 01:46 PM
"Moger, Babu"
 
Default info on enabling only one path with rdac and DS4700

> -----Original Message-----
> From: Gianluca Cecchi [mailto:gianluca.cecchi@gmail.com]
> Sent: Wednesday, November 23, 2011 11:22 AM
> To: device-mapper development
> Subject: Re: [dm-devel] info on enabling only one path with rdac and
> DS4700
>
> On Wed, Nov 16, 2011 at 2:24 PM, Johannes Hirte wrote:
> [snip]
> > Yes, this is because the rdac module detected the LUN in AVT mode and
> refused
> > to work with it. This will happen every time you access a ghost path
> without
> > rdac.
>
> >
> >> - On the presented LUN I configured a PV, VG, LV and ext4 fs (not
> system fs)
> >> At reboot at host side I see messages related to duplicated PV IDs
> for
> >> the paths (sdb, sdc, sdd, sde): they comes before vg activation and
> >> before multipathd start...
> >> Is this normal, because at the first vgscan run during boot,
> multipath
> >> configuration has not been instantiated yet..?
> >> I have to check, but I don't remember similar messages with eh el
> 5.7
> >> in other SAN configurations, where the VG is not a system VG....
> >
> > You should avoid to access the sdX directly. If you need to run lvm
> before
> > multipath is up, you can blacklist the sdX in the lvm.conf.
>
>
> So I configured:
> - LUN on DS4700 as LNXCLVMWARE that I found should disable AVT
> - multipath as standard without any particular setting (I only
> blacklisted the internal disk)
>
> At the start time of the system I get (both in console and then I
> found it in /var/log/messages too):
>
> Nov 23 17:32:56 testserver kernel: sdc:end_request: I/O error, dev
> sdb, sector 0
> Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdb,
> logical block 0
> Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdc,
> sector 0
> Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdc,
> logical block 0
> Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdb,
> sector 0
> Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdb,
> logical block 0
> Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdc,
> sector 0
> Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdc,
> logical block 0
> Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdb,
> sector 0
> Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdb,
> logical block 0
> Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdc,
> sector 0
> Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdc,
> logical block 0
> Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdb,
> sector 0
> Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdb,
> logical block 0
> Nov 23 17:32:56 testserver kernel: end_request: I/O error, dev sdc,
> sector 0
> Nov 23 17:32:56 testserver kernel: Buffer I/O error on device sdc,
> logical block 0
>
> This happens for sdb ad sdc only (probably passive controller disk
> paths?)
>
> And this other ones:
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb,
> sector 0
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb,
> sector 7320493952
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb,
> sector 7320494064
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb,
> sector 0
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb,
> sector 8
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdb,
> sector 0
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdc,
> sector 0
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdc,
> sector 7320493952
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdc,
> sector 7320494064
> Nov 23 17:32:58 testserver kernel: end_request: I/O error, dev sdc,
> sector 0
> ...
> then
> Nov 23 17:33:00 testserver kernel: device-mapper: multipath: version
> 1.0.6 loaded
> Nov 23 17:33:00 testserver kernel: sd 3:0:0:1: rdac: LUN 1 (unowned)
> Nov 23 17:33:00 testserver kernel: sd 3:0:1:1: rdac: LUN 1 (owned)
> Nov 23 17:33:00 testserver kernel: sd 4:0:0:1: rdac: LUN 1 (unowned)
> Nov 23 17:33:00 testserver kernel: sd 4:0:1:1: rdac: LUN 1 (owned)
> Nov 23 17:33:01 testserver kernel: rdac: device handler registered
> Nov 23 17:33:01 testserver kernel: device-mapper: multipath: Using
> scsi_dh module scsi_dh_rdac for failover/failback and device
> management.
> Nov 23 17:33:01 testserver kernel: device-mapper: multipath
> round-robin: version 1.0.0 loaded
> Nov 23 17:33:01 testserver kernel: sd 3:0:0:1: rdac: array
> Z1_BEIC_DS4700, ctlr 0, queueing MODE_SELECT command
> Nov 23 17:33:01 testserver kernel: sd 3:0:0:1: rdac: array
> Z1_BEIC_DS4700, ctlr 0, MODE_SELECT completed
> Nov 23 17:33:01 testserver kernel: sd 4:0:0:1: rdac: array
> Z1_BEIC_DS4700, ctlr 0, queueing MODE_SELECT command
> Nov 23 17:33:01 testserver kernel: sd 4:0:0:1: rdac: array
> Z1_BEIC_DS4700, ctlr 0, MODE_SELECT completed
> Nov 23 17:33:01 testserver kernel: end_request: I/O error, dev sdd,
> sector 7320494072
> Nov 23 17:33:01 testserver kernel: printk: 4 messages suppressed.
> Nov 23 17:33:01 testserver kernel: Buffer I/O error on device sdd,
> logical block 915061759
> Nov 23 17:33:01 testserver kernel: end_request: I/O error, dev sde,
> sector 7320494072
> Nov 23 17:33:02 testserver kernel: printk: 49 messages suppressed.
> Nov 23 17:33:02 testserver kernel: Buffer I/O error on device sde,
> logical block 0
> Nov 23 17:33:02 testserver kernel: Buffer I/O error on device sde,
> logical block 2
> Nov 23 17:33:02 testserver kernel: Buffer I/O error on device sde,
> logical block 3
>
> And then no other I/O error messages. Donna if this is avoidable or
> not....
>
> So after complete startup the situation is:
> [root@testserver ~]# multipath -l
> mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
> [size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
> \_ round-robin 0 [prio=0][active]
> \_ 3:0:0:1 sdb 8:16 [active][undef]
> \_ 4:0:0:1 sdc 8:32 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> \_ 3:0:1:1 sdd 8:48 [active][undef]
> \_ 4:0:1:1 sde 8:64 [active][undef]
>
> When I activate the LVM on mpath1 PV and mount the file system:
> Nov 23 17:34:31 testserver kernel: EXT4-fs (dm-7): mounted filesystem
> with ordered data mode
> Nov 23 17:34:47 testserver kernel: JBD: barrier-based sync failed on
> dm-7-8 - disabling barriers
> --> donna if it is to be intended as a problem
>
> I instantiate I/O without problems.
>
> Then I test to change active controller for the lun at DS4700 side
> during a running I/O session (dd seq read of 10Gb) , and I get
> Nov 23 17:43:02 testserver kernel: end_request: I/O error, dev sdb,
> sector 2110328
> Nov 23 17:43:02 testserver kernel: device-mapper: multipath: Failing
> path 8:16.
> Nov 23 17:43:02 testserver multipathd: 8:16: mark as failed
> Nov 23 17:43:02 testserver multipathd: mpath1: remaining active paths:
> 3
> Nov 23 17:43:02 testserver multipathd: dm-6: add map (uevent)
> Nov 23 17:43:02 testserver multipathd: dm-6: devmap already registered
> Nov 23 17:43:03 testserver kernel: end_request: I/O error, dev sdc,
> sector 2110328
> Nov 23 17:43:03 testserver kernel: device-mapper: multipath: Failing
> path 8:32.
> Nov 23 17:43:03 testserver multipathd: dm-6: add map (uevent)
> Nov 23 17:43:03 testserver multipathd: dm-6: devmap already registered
> Nov 23 17:43:03 testserver multipathd: 8:32: mark as failed
> Nov 23 17:43:03 testserver multipathd: mpath1: remaining active paths:
> 2
> Nov 23 17:43:06 testserver multipathd: sdb: rdac checker reports path
> is ghost
> Nov 23 17:43:06 testserver multipathd: 8:16: reinstated
> Nov 23 17:43:06 testserver multipathd: mpath1: remaining active paths:
> 3
> Nov 23 17:43:06 testserver kernel: device-mapper: multipath: Using
> scsi_dh module scsi_dh_rdac for failover/failback and device
> management.
> Nov 23 17:43:06 testserver multipathd: mpath1: load table [0
> 7320494080 multipath 0 1 rdac 2 1 round-robin 0 3 1 8:32 1000 8:48
> 1000 8:64 1000 round-robin 0 1 1 8:16
> Nov 23 17:43:06 testserver multipathd: dm-6: add map (uevent)
> Nov 23 17:43:06 testserver multipathd: dm-6: devmap already registered
> Nov 23 17:43:06 testserver kernel: device-mapper: multipath: Failing
> path 8:32.
> Nov 23 17:43:06 testserver multipathd: dm-6: add map (uevent)
> Nov 23 17:43:06 testserver multipathd: dm-6: devmap already registered
> Nov 23 17:43:06 testserver multipathd: dm-6: add map (uevent)
> Nov 23 17:43:06 testserver multipathd: dm-6: devmap already registered
> Nov 23 17:43:07 testserver multipathd: sdc: rdac checker reports path
> is ghost
> Nov 23 17:43:07 testserver multipathd: 8:32: reinstated
> Nov 23 17:43:07 testserver kernel: sd 4:0:0:1: rdac: array
> Z1_BEIC_DS4700, ctlr 0, queueing MODE_SELECT command
> Nov 23 17:43:07 testserver multipathd: mpath1: remaining active paths:
> 4
> Nov 23 17:43:07 testserver kernel: device-mapper: multipath: Using
> scsi_dh module scsi_dh_rdac for failover/failback and device
> management.
> Nov 23 17:43:08 testserver kernel: sd 4:0:0:1: rdac: array
> Z1_BEIC_DS4700, ctlr 0, MODE_SELECT completed
> Nov 23 17:43:08 testserver multipathd: mpath1: load table [0
> 7320494080 multipath 0 1 rdac 2 1 round-robin 0 2 1 8:48 1000 8:64
> 1000 round-robin 0 2 1 8:32 1000 8:16
> Nov 23 17:43:08 testserver multipathd: dm-6: add map (uevent)
> Nov 23 17:43:08 testserver multipathd: dm-6: devmap already registered
> Nov 23 17:43:08 testserver multipathd: dm-6: add map (uevent)
> Nov 23 17:43:08 testserver multipathd: dm-6: devmap already registered
> Nov 23 17:43:08 testserver kernel: sd 3:0:1:1: rdac: array
> Z1_BEIC_DS4700, ctlr 1, queueing MODE_SELECT command
> Nov 23 17:43:10 testserver kernel: sd 3:0:1:1: rdac: array
> Z1_BEIC_DS4700, ctlr 1, MODE_SELECT completed
> Nov 23 17:43:10 testserver kernel: sd 4:0:1:1: rdac: array
> Z1_BEIC_DS4700, ctlr 1, queueing MODE_SELECT command
> Nov 23 17:43:11 testserver kernel: sd 4:0:1:1: rdac: array
> Z1_BEIC_DS4700, ctlr 1, MODE_SELECT completed
> Nov 23 17:43:12 testserver multipathd: sdd: rdac checker reports path
> is up
> Nov 23 17:43:12 testserver multipathd: 8:48: reinstated
> Nov 23 17:43:12 testserver multipathd: sde: rdac checker reports path
> is up
> Nov 23 17:43:12 testserver multipathd: 8:64: reinstated
>
> The overall increased time is 3-4 seconds for a 1 minute I/O period
> Without failover:
> [root@testserver ~]# time dd if=/testfs/testfile bs=1024k count=10000
> of=/dev/null
> 10000+0 records in
> 10000+0 records out
> 10485760000 bytes (10 GB) copied, 61.9951 seconds, 169 MB/s
>
> real 1m1.996s
> user 0m0.002s
> sys 0m7.088s
>
> With change of active controller in the mid:
> [root@testserver ~]# time dd if=/testfs/testfile1 bs=1024k count=10000
> of=/dev/null
> 10000+0 records in
> 10000+0 records out
> 10485760000 bytes (10 GB) copied, 65.6529 seconds, 160 MB/s
>
> real 1m5.654s
> user 0m0.007s
> sys 0m7.175s
>
> So, quite good, and without error at user side.
>
> at the end the multipath config is this:
> [root@testserver ~]# multipath -l
> mpath1 (3600a0b80005012440000093e4a55cf33) dm-6 IBM,1814 FAStT
> [size=3.4T][features=1 queue_if_no_path][hwhandler=1 rdac][rw]
> \_ round-robin 0 [prio=0][active]
> \_ 3:0:1:1 sdd 8:48 [active][undef]
> \_ 4:0:1:1 sde 8:64 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> \_ 4:0:0:1 sdc 8:32 [active][undef]
> \_ 3:0:0:1 sdb 8:16 [active][undef]
>
> Questions:
>
> 1) Can I conclude it is ok as a configuration? Or any other tests to
> carry on?
> I confirm I didn't get any snmp trap from the ds4700 as happened
> before...

Your configuration looks good to me.

>
> 2) At the moment I put this in lvm.conf to whitelist the root volume
> groups and blacklist the san individual paths and then delete the
> .cache file and reboot
> filter = [ "a/dev/mapper/.*/", "a/dev/sda/", "a/dev/sda2/",
> "r/dev/sd.*/" ]
> Is this ok?
> If root PV is on sda2, do I need to whitelist both sda and sda2 or only
> sda2?
>
> 3) Based on messages during failover, is it true that I can avoid
> explicitly put scsi_dh in initrd?
> If I create initrd this way:
> mkinitrd /boot/initrd-$(uname -r)-scsi_dh.img $(uname -r) --
> preload=scsi_dh_rdac
> I get this difference:
> [root@testserver ~]# diff /tmp/new/init /tmp/current/init
> 44,51d43
> < echo "Loading scsi_mod.ko module"
> < insmod /lib/scsi_mod.ko
> < echo "Loading sd_mod.ko module"
> < insmod /lib/sd_mod.ko
> < echo "Loading scsi_dh.ko module"
> < insmod /lib/scsi_dh.ko
> < echo "Loading scsi_dh_rdac.ko module"
> < insmod /lib/scsi_dh_rdac.ko
> 62a55,58
> > echo "Loading scsi_mod.ko module"
> > insmod /lib/scsi_mod.ko
> > echo "Loading sd_mod.ko module"
> > insmod /lib/sd_mod.ko
>
> or will it help in any way?

Having scsi_dh_rdac in initrd will help to get rid of the initial I/O errors you are seeing.

> BTW: The I/O tests above were done with standard initrd (so the > side
> of the diff without the scsi_dh_rdac)
> I only run the mkinitrd to sort out how would have been create the init
> file...
>
> 4) the san lun is 3.4Tb and I'm going to add another one of about 5Tb
> In messages I see this
> Nov 23 17:32:58 testserver kernel: sde : very big device. try to use
> READ CAPACITY(16).
>
> I found in an old kernel ml post that actually it should mean "trying
> to use" ... so only informational message.
> Can anyone confirm this?

It is only informational..

>
> Thanks again in advance for your help,
> Gianluca
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-28-2011, 03:04 PM
Gianluca Cecchi
 
Default info on enabling only one path with rdac and DS4700

On Mon, Nov 28, 2011 at 3:46 PM, Moger, Babu wrote:
>> 3) Based on messages during failover, is it true that I can avoid
>> explicitly put scsi_dh in initrd?
>> If I create initrd this way:
>> mkinitrd /boot/initrd-$(uname -r)-scsi_dh.img $(uname -r) --
>> preload=scsi_dh_rdac
>> I get this difference:
>> [root@testserver ~]# diff /tmp/new/init /tmp/current/init
>> 44,51d43
>> < echo "Loading scsi_mod.ko module"
>> < insmod /lib/scsi_mod.ko
>> < echo "Loading sd_mod.ko module"
>> < insmod /lib/sd_mod.ko
>> < echo "Loading scsi_dh.ko module"
>> < insmod /lib/scsi_dh.ko
>> < echo "Loading scsi_dh_rdac.ko module"
>> < insmod /lib/scsi_dh_rdac.ko
>> 62a55,58
>> > echo "Loading scsi_mod.ko module"
>> > insmod /lib/scsi_mod.ko
>> > echo "Loading sd_mod.ko module"
>> > insmod /lib/sd_mod.ko
>>
>> or will it help in any way?
>
> Having scsi_dh_rdac in initrd will help to get rid of the initial I/O errors you are seeing.
>

Actually not.... see screen shots of console start at
https://docs.google.com/open?id=0BwoPbcrMv8mvMWRlYmVhM2MtY2JkYi00ZDMzLWIxM WMtMDM0YzZjMWU0NGE1
and
https://docs.google.com/open?id=0BwoPbcrMv8mvM2RjMDg1MWUtYzVmZC00OTA1LTljN jAtZGZmMjM5NDlhYTI4
and also dmesg giving for example for sdb and sdc:

scsi 3:0:0:1: rdac: LUN 1 (unowned)
sdb : very big device. try to use READ CAPACITY(16).
SCSI device sdb: 7320494080 512-byte hdwr sectors (3748093 MB)
sdb: Write Protect is off
sdb: Mode Sense: 77 00 10 08
SCSI device sdb: drive cache: write back w/ FUA
sdb : very big device. try to use READ CAPACITY(16).
SCSI device sdb: 7320494080 512-byte hdwr sectors (3748093 MB)
sdb: Write Protect is off
sdb: Mode Sense: 77 00 10 08
SCSI device sdb: drive cache: write back w/ FUA
sdb:<3>Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 0
Dev sdb: unable to read RDB block 0
Buffer I/O error on device sdb, logical block 0
Buffer I/O error on device sdb, logical block 0
unable to read partition table
sd 3:0:0:1: Attached scsi disk sdb
Vendor: IBM Model: 1814 FAStT Rev: 0916
Type: Direct-Access ANSI SCSI revision: 05
scsi 3:0:1:1: rdac: LUN 1 (owned)
scsi 3:0:1:1: rdac: LUN 1 (owned)
sdc : very big device. try to use READ CAPACITY(16).
SCSI device sdc: 7320494080 512-byte hdwr sectors (3748093 MB)
sdc: Write Protect is off
sdc: Mode Sense: 77 00 10 08
SCSI device sdc: drive cache: write back w/ FUA
sdc : very big device. try to use READ CAPACITY(16).
SCSI device sdc: 7320494080 512-byte hdwr sectors (3748093 MB)
sdc: Write Protect is off
sdc: Mode Sense: 77 00 10 08
SCSI device sdc: drive cache: write back w/ FUA
sdc: unknown partition table
sd 3:0:1:1: Attached scsi disk sdc
...
GSI 23 sharing vector 0x72 and IRQ 23
ACPI: PCI Interrupt 0000:04:00.0[A] -> GSI 17 (level, low) -> IRQ 114
eth0: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit
133MHz found at mem da000000, IRQ 114, node addr 00215e2fd0e2
ACPI: PCI Interrupt 0000:06:00.0[A] -> GSI 19 (level, low) -> IRQ 90
eth1: Broadcom NetXtreme II BCM5708 1000Base-SX (B2) PCI-X 64-bit
133MHz found at mem d8000000, IRQ 90, node addr 00215e2fd0e4
printk: 69 messages suppressed.
Buffer I/O error on device sdb, logical block 0
...
md: Autodetecting RAID arrays.
md: autorun ...
md: ... autorun DONE.
device-mapper: multipath: version 1.0.6 loaded
device-mapper: multipath: Using scsi_dh module scsi_dh_rdac for
failover/failback and device management.
device-mapper: multipath round-robin: version 1.0.0 loaded
printk: 4 messages suppressed.
Buffer I/O error on device sdb, logical block 915061759
...

So if I have to get yet the I/O errors, having no advantage, I'd
prefer to stay stick with the default initrd...

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 02:37 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org