FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 06-21-2012, 11:54 PM
Brian Bunker
 
Default why doesn't multipathd use the device?

Hello everyone,
We have hit this problem a couple of times now on our RHEL 6.2 initiators using our FC target. We have many dm devices which should have 4 paths to each. Sometimes we see that some of the dm devices do not have the 4 paths. The underlying /dev/sd devices do exist but they are not used by the dm device. Here is what I see:
mpathbbt (3624a9370f9b69331bb3d16650001000b) dm-11 PURE,FlashArray
size=500G features='0' hwhandler='0' wp=rw
`-+- policy='round-robin 0' prio=1 status=active
|- 1:0:1:12 sds 65:32 active ready running
`- 1:0:0:12 sdae 65:224 active ready running
However there are devices for the remaining two paths they are just not being used by multipathd:
/dev/sg18 1 0 1 12 0 /dev/sds PURE FlashArray 100
/dev/sg30 1 0 0 12 0 /dev/sdae PURE FlashArray 100
/dev/sg39 0 0 1 12 0 /dev/sdan PURE FlashArray 100
/dev/sg48 0 0 0 12 0 /dev/sdaw PURE FlashArray 100
You can see by the LUN serial number that the devices do point to the same LUN on the array. I am including just the LUN serial number, page 0x80, in the interest of space page 0x83's NAA number also matches for the 4 devices:
root@r13init31 ~]# sg_inq -p 0x80 /dev/sg18VPD INQUIRY: Unit serial number pageUnit serial number: F9B69331BB3D16650001000B
[root@r13init31 ~]# sg_inq -p 0x80 /dev/sg30VPD INQUIRY: Unit serial number pageUnit serial number: F9B69331BB3D16650001000B
[root@r13init31 ~]# sg_inq -p 0x80 /dev/sg39VPD INQUIRY: Unit serial number pageUnit serial number: F9B69331BB3D16650001000B
[root@r13init31 ~]# sg_inq -p 0x80 /dev/sg48VPD INQUIRY: Unit serial number pageUnit serial number: F9B69331BB3D16650001000B
I can see in the syslog that there are syslog messages about each of the devices which don't get added to the multipath device like this:Jun 21 09:23:50 r13init31 kernel: sd 0:0:1:12: [sdan] Attached SCSI disk
Jun 21 09:23:50 r13init31 multipathd: sdan: add path (uevent)
Jun 21 09:23:50 r13init31 multipathd: mpathbbt: failed in domap for addition of new path sdan
My multipath tools version is:device-mapper-multipath-libs-0.4.9-46.el6_2.2.x86_64device-mapper-multipath-0.4.9-46.el6_2.2.x86_64
Any ideas why the paths are being rejected by domap?
Thanks,Brian

Brian Bunkerbrian@purestorage.com



--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 06-22-2012, 05:44 AM
Christophe Varoqui
 
Default why doesn't multipathd use the device?

On jeu., 2012-06-21 at 16:54 -0700, Brian Bunker wrote:
> Hello everyone,
>
>
> We have hit this problem a couple of times now on our RHEL 6.2
> initiators using our FC target. We have many dm devices which should
> have 4 paths to each. Sometimes we see that some of the dm devices do
> not have the 4 paths. The underlying /dev/sd devices do exist but they
> are not used by the dm device. Here is what I see:
>
>
> mpathbbt (3624a9370f9b69331bb3d16650001000b) dm-11 PURE,FlashArray
> size=500G features='0' hwhandler='0' wp=rw
> `-+- policy='round-robin 0' prio=1 status=active
> |- 1:0:1:12 sds 65:32 active ready running
> `- 1:0:0:12 sdae 65:224 active ready running
>
>
> However there are devices for the remaining two paths they are just
> not being used by multipathd:
>
>
> /dev/sg18 1 0 1 12 0 /dev/sds PURE FlashArray 100
> /dev/sg30 1 0 0 12 0 /dev/sdae PURE FlashArray 100
> /dev/sg39 0 0 1 12 0 /dev/sdan PURE FlashArray 100
> /dev/sg48 0 0 0 12 0 /dev/sdaw PURE FlashArray 100
>
>
> You can see by the LUN serial number that the devices do point to the
> same LUN on the array. I am including just the LUN serial number, page
> 0x80, in the interest of space page 0x83's NAA number also matches for
> the 4 devices:
>
>
> root@r13init31 ~]# sg_inq -p 0x80 /dev/sg18
> VPD INQUIRY: Unit serial number page
> Unit serial number: F9B69331BB3D16650001000B
>
>
> [root@r13init31 ~]# sg_inq -p 0x80 /dev/sg30
> VPD INQUIRY: Unit serial number page
> Unit serial number: F9B69331BB3D16650001000B
>
>
> [root@r13init31 ~]# sg_inq -p 0x80 /dev/sg39
> VPD INQUIRY: Unit serial number page
> Unit serial number: F9B69331BB3D16650001000B
>
>
> [root@r13init31 ~]# sg_inq -p 0x80 /dev/sg48
> VPD INQUIRY: Unit serial number page
> Unit serial number: F9B69331BB3D16650001000B

>
> Any ideas why the paths are being rejected by domap?
>
Have you tried the "group_by_serial" path grouping policy ?
There might be funny side effects if the wwid discovered on paths is not
the same though ...

Can you post the multipath -v3 output concerning those 4 paths
discovery.

Regard,
cvaroqui


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 06-22-2012, 05:00 PM
Brian Bunker
 
Default why doesn't multipathd use the device?

We are currently using multibus as a path_grouping_policy since all of our ports are active and there is no need to weight any different than others. The array is active / active.

In poking around the multipath source files I am assuming that this is what is happening in domap since that is the only path that returns a negative number to the caller in main.c in multipathd:

if (lock_multipath(mpp, 1)) {
condlog(3, "%s: failed to create map (in use)",
mpp->alias);
return DOMAP_RETRY;
}

I guess that it didn't log since the log level of multipath was lower and that is why you want me to run at '-v 3'? If it is true that it believes that the map is in use what does that mean?

Thanks,
Brian

On Jun 21, 2012, at 10:44 PM, Christophe Varoqui wrote:

> On jeu., 2012-06-21 at 16:54 -0700, Brian Bunker wrote:
>> Hello everyone,
>>
>>
>> We have hit this problem a couple of times now on our RHEL 6.2
>> initiators using our FC target. We have many dm devices which should
>> have 4 paths to each. Sometimes we see that some of the dm devices do
>> not have the 4 paths. The underlying /dev/sd devices do exist but they
>> are not used by the dm device. Here is what I see:
>>
>>
>> mpathbbt (3624a9370f9b69331bb3d16650001000b) dm-11 PURE,FlashArray
>> size=500G features='0' hwhandler='0' wp=rw
>> `-+- policy='round-robin 0' prio=1 status=active
>> |- 1:0:1:12 sds 65:32 active ready running
>> `- 1:0:0:12 sdae 65:224 active ready running
>>
>>
>> However there are devices for the remaining two paths they are just
>> not being used by multipathd:
>>
>>
>> /dev/sg18 1 0 1 12 0 /dev/sds PURE FlashArray 100
>> /dev/sg30 1 0 0 12 0 /dev/sdae PURE FlashArray 100
>> /dev/sg39 0 0 1 12 0 /dev/sdan PURE FlashArray 100
>> /dev/sg48 0 0 0 12 0 /dev/sdaw PURE FlashArray 100
>>
>>
>> You can see by the LUN serial number that the devices do point to the
>> same LUN on the array. I am including just the LUN serial number, page
>> 0x80, in the interest of space page 0x83's NAA number also matches for
>> the 4 devices:
>>
>>
>> root@r13init31 ~]# sg_inq -p 0x80 /dev/sg18
>> VPD INQUIRY: Unit serial number page
>> Unit serial number: F9B69331BB3D16650001000B
>>
>>
>> [root@r13init31 ~]# sg_inq -p 0x80 /dev/sg30
>> VPD INQUIRY: Unit serial number page
>> Unit serial number: F9B69331BB3D16650001000B
>>
>>
>> [root@r13init31 ~]# sg_inq -p 0x80 /dev/sg39
>> VPD INQUIRY: Unit serial number page
>> Unit serial number: F9B69331BB3D16650001000B
>>
>>
>> [root@r13init31 ~]# sg_inq -p 0x80 /dev/sg48
>> VPD INQUIRY: Unit serial number page
>> Unit serial number: F9B69331BB3D16650001000B
>
>>
>> Any ideas why the paths are being rejected by domap?
>>
> Have you tried the "group_by_serial" path grouping policy ?
> There might be funny side effects if the wwid discovered on paths is not
> the same though ...
>
> Can you post the multipath -v3 output concerning those 4 paths
> discovery.
>
> Regard,
> cvaroqui
>
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

Brian Bunker
brian@purestorage.com




--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 06-26-2012, 09:24 PM
Brian Bunker
 
Default why doesn't multipathd use the device?

OK, I have the output of 'multipath -v3' and the output of 'dmsetup table'. I don't see the smoking gun though.

multipath -v3
Jun 26 13:46:40 | Found matching wwid [3624a937056ea3c50504e404700010009] in bindings file. Setting alias to mpathbkp
Jun 26 13:46:40 | sdq: ownership set to mpathbkp
Jun 26 13:46:40 | sdq: not found in pathvec
Jun 26 13:46:40 | sdq: mask = 0xc
Jun 26 13:46:40 | sdq: get_state
Jun 26 13:46:40 | sdq: state = running
Jun 26 13:46:40 | sdq: state = 3
Jun 26 13:46:40 | sdq: state = running
Jun 26 13:46:40 | sdq: const prio = 1
Jun 26 13:46:40 | sdac: ownership set to mpathbkp
Jun 26 13:46:40 | sdac: not found in pathvec
Jun 26 13:46:40 | sdac: mask = 0xc
Jun 26 13:46:40 | sdac: get_state
Jun 26 13:46:40 | sdac: state = running
Jun 26 13:46:40 | sdac: state = 3
Jun 26 13:46:40 | sdac: state = running
Jun 26 13:46:40 | sdac: const prio = 1
Jun 26 13:46:40 | sdal: ownership set to mpathbkp
Jun 26 13:46:40 | sdal: not found in pathvec
Jun 26 13:46:40 | sdal: mask = 0xc
Jun 26 13:46:40 | sdal: get_state
Jun 26 13:46:40 | sdal: state = running
Jun 26 13:46:40 | sdal: state = 3
Jun 26 13:46:40 | sdal: state = running
Jun 26 13:46:40 | sdal: const prio = 1
Jun 26 13:46:40 | sdau: ownership set to mpathbkp
Jun 26 13:46:40 | sdau: not found in pathvec
Jun 26 13:46:40 | sdau: mask = 0xc
Jun 26 13:46:40 | sdau: get_state
Jun 26 13:46:40 | sdau: state = running
Jun 26 13:46:40 | sdau: state = 3
Jun 26 13:46:40 | sdau: state = running
Jun 26 13:46:40 | sdau: const prio = 1
Jun 26 13:46:40 | mpathbkp: pgfailover = -1 (internal default)
Jun 26 13:46:40 | mpathbkp: pgpolicy = multibus (config file default)
Jun 26 13:46:40 | mpathbkp: selector = round-robin 0 (internal default)
Jun 26 13:46:40 | mpathbkp: features = 0 (internal default)
Jun 26 13:46:40 | mpathbkp: hwhandler = 0 (internal default)
Jun 26 13:46:40 | mpathbkp: rr_weight = 1 (internal default)
Jun 26 13:46:40 | mpathbkp: minio = 1 rq (config file default)
Jun 26 13:46:40 | mpathbkp: no_path_retry = NONE (internal default)
Jun 26 13:46:40 | pg_timeout = NONE (internal default)
Jun 26 13:46:40 | mpathbkp: set ACT_RELOAD (path group topology change)
reload: mpathbkp (3624a937056ea3c50504e404700010009) undef PURE,FlashArray
size=500G features='0' hwhandler='0' wp=undef
`-+- policy='round-robin 0' prio=1 status=undef
|- 1:0:1:10 sdq 65:0 active ready running
|- 1:0:0:10 sdac 65:192 active ready running
|- 0:0:1:10 sdal 66:80 undef ready running
`- 0:0:0:10 sdau 66:224 undef ready running

dmsetup table:
mpathbkg: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 8:48 1 8:96 1 65:80 1 8:144 1
mpathbkf: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 8:16 1 8:64 1 65:48 1 8:112 1
mpathbkq: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 65:32 1 65:224 1 66:112 1 67:0 1
mpathbkp: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 65:0 1 65:192 1 66:80 1 66:224 1
mpathbko: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 8:240 1 65:176 1 66:64 1 66:208 1
mpathbkn: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 65:96 1 8:160 1 65:240 1 66:128 1
mpathbkm: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 65:16 1 65:208 1 66:96 1 66:240 1
mpathbkl: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 8:208 1 65:144 1 66:32 1 66:176 1
mpathbkk: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 65:112 1 8:176 1 66:0 1 66:144 1
mpathbkj: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 65:128 1 8:192 1 66:160 1 66:16 1
mpathbki: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 8:224 1 65:160 1 66:48 1 66:192 1
mpathbkh: 0 1048576000 multipath 0 0 1 1 round-robin 0 4 1 8:32 1 8:80 1 8:128 1 65:64 1

Thanks,
Brian

On Jun 22, 2012, at 12:08 PM, Christophe Varoqui wrote:

> On ven., 2012-06-22 at 10:00 -0700, Brian Bunker wrote:
>> We are currently using multibus as a path_grouping_policy since all of our ports are active and there is no need to weight any different than others. The array is active / active.
>>
>> In poking around the multipath source files I am assuming that this is what is happening in domap since that is the only path that returns a negative number to the caller in main.c in multipathd:
>>
>> if (lock_multipath(mpp, 1)) {
>> condlog(3, "%s: failed to create map (in use)",
>> mpp->alias);
>> return DOMAP_RETRY;
>> }
>>
>> I guess that it didn't log since the log level of multipath was lower and that is why you want me to run at '-v 3'? If it is true that it believes that the map is in use what does that mean?
>>
>
> Usual suspects are other device maps layered on these paths or
> filesystems. 'dmsetup table' will help confirm the former, df the later.
>
> Regards,
> cvaroqui
>
>

Brian Bunker
brian@purestorage.com




--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 09:34 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org