FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 07-28-2010, 08:45 AM
Gianluca Cecchi
 
Default How to associate pci id with wwpn for an hba

Hello,
yesterday I had a problem where a server with 2 Qlogic HBA detected only 2 of the 4 paths it should see.
Each HBA is connected to a different fc switch.
Each FC switch is connected to the 2 controllers of an IBM DS6800 storage array.

So that in general each disk is seen by 4 paths.
System is rh el 5.5 x86_64
Something like this for each mpath device normally:

mpath1 (3600507630efe0b0c0000000000000601) dm-8 IBM,1750500
[size=15G][features=1 queue_if_no_path][hwhandler=0][rw]

\_ round-robin 0 [prio=0][active]
*\_ 1:0:3:2* sdao 66:128 [active][undef]
*\_ 2:0:3:2* sdaq 66:160 [active][undef]
\_ round-robin 0 [prio=0][enabled]
*\_ 1:0:2:2* sdd* 8:48** [active][undef]
*\_ 2:0:2:2* sdp* 8:240* [active][undef]


Yesterday one server was able to see only 2 paths for each mpath device.
I had something like this:
mpath1 (3600507630efe0b0c0000000000000601) dm-8 IBM,1750500

[size=15G][features=1 queue_if_no_path][hwhandler=0][rw]

\_ round-robin 0 [prio=0][active]

*\_ 1:0:3:2* sdao 66:128 [active][undef]


\_ round-robin 0 [prio=0][enabled]

*\_ 1:0:2:2* sdd* 8:48** [active][undef]


I had some problems to identify PCI ID <--> WWPN <--> disk devices associations so that I could give the correct wwpn involved in the problem to the SAN guys for analysis and resolution.

I took these steps. Could anyone confirm they are ok or provide other analysys information?


a) From above output of "multipath -l", and also from output of "lsscsi" command
..
[1:0:2:2]*** disk*** IBM***** 1750500********* .508* /dev/sdd
[1:0:2:3]*** disk*** IBM***** 1750500********* .508* /dev/sde

..
[1:0:3:0]*** disk*** IBM***** 1750500********* .508* /dev/sdal
[1:0:3:1]*** disk*** IBM***** 1750500********* .508* /dev/sdan
[1:0:3:2]*** disk*** IBM***** 1750500********* .508* /dev/sdao

with only lines of kind "1:0:x:x" I concluded that I had one adapter correctly seeing its 2 paths and the other one seeing nothing.


b) In /var/log/messages I had this, only for one adapter
Jul 27 17:54:58 orastud1 kernel: qla2xxx 0000:06:00.0: SNS scan failed -- assuming zero-entry result...

that confirmed somehow a) conclusion

Now I had to give wwpn of hba with pci id 0000:06:00.0

I found that this was not so obvious (at least for me)

c) to see my two wwpn:
# for i in 1 2 ; do echo "host$i $(cat /sys/class/fc_host/host$i/port_name)"; done
host1 0x21000024ff288e04
host2 0x21000024ff288e05


ok: how to directly connect hostx with pci 0000:06:00.0 ?
lspci* gives:
06:00.0 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)
06:00.1 Fibre Channel: QLogic Corp. ISP2532-based 8Gb Fibre Channel to PCI Express HBA (rev 02)


but no "-vv" switch gives the wwpn for them
Can I surely say that host1 <--> 06:00.0 and host2 <--> 06:00.1 ?
What if the first number in general is not the same (for example first hba 05:00.0 and second 06:00.0)?


d) Basically I went through something like:
# ls -d /sys/class/fc_transport/target*/device/*/block* | grep sdao
/sys/class/fc_transport/target1:0:3/device/1:0:3:2/block:sdao
# ls -d /sys/class/fc_transport/target*/device/*/block* | grep sdd

/sys/class/fc_transport/target1:0:2/device/1:0:2:2/block:sdd

Can I say that host1 <--> target1 ?
And so that in my case the adapter not seeing the LUNs was host2 ----> wpn 0x21000024ff288e05 ?

Could I directly say from point a) that the hostx involved is host2 as "multipath -l" only shows 1:x:y:z devices?

And so that problematic wpn was 0x21000024ff288e05?
Anyway, how to associate hostx with pci id?

Thanks in advance,
Gianluca


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 07-28-2010, 11:26 AM
Jonathan Barber
 
Default How to associate pci id with wwpn for an hba

On 28 July 2010 09:45, Gianluca Cecchi <gianluca.cecchi@gmail.com> wrote:
> Hello,
> yesterday I had a problem where a server with 2 Qlogic HBA detected only 2
> of the 4 paths it should see.
> Each HBA is connected to a different fc switch.
> Each FC switch is connected to the 2 controllers of an IBM DS6800 storage
> array.
> So that in general each disk is seen by 4 paths.
> System is rh el 5.5 x86_64
> Something like this for each mpath device normally:
>
> mpath1 (3600507630efe0b0c0000000000000601) dm-8 IBM,1750500
> [size=15G][features=1 queue_if_no_path][hwhandler=0][rw]
> \_ round-robin 0 [prio=0][active]
> *\_ 1:0:3:2* sdao 66:128 [active][undef]
> *\_ 2:0:3:2* sdaq 66:160 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> *\_ 1:0:2:2* sdd* 8:48** [active][undef]
> *\_ 2:0:2:2* sdp* 8:240* [active][undef]
>
> Yesterday one server was able to see only 2 paths for each mpath device.
> I had something like this:
> mpath1 (3600507630efe0b0c0000000000000601) dm-8 IBM,1750500
> [size=15G][features=1 queue_if_no_path][hwhandler=0][rw]
> \_ round-robin 0 [prio=0][active]
> *\_ 1:0:3:2* sdao 66:128 [active][undef]
> \_ round-robin 0 [prio=0][enabled]
> *\_ 1:0:2:2* sdd* 8:48** [active][undef]
>
> I had some problems to identify PCI ID <--> WWPN <--> disk devices
> associations so that I could give the correct wwpn involved in the problem
> to the SAN guys for analysis and resolution.

[snip]

> Could I directly say from point a) that the hostx involved is host2 as
> "multipath -l" only shows 1:x:y:z devices?
> And so that problematic wpn was 0x21000024ff288e05?
> Anyway, how to associate hostx with pci id?

# lspci | grep QLogic
0d:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to
PCI Express HBA (rev 03)
0d:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to
PCI Express HBA (rev 03)

# ls -l /sys/class/fc_host/host?/device
lrwxrwxrwx 1 root root 0 Jul 28 11:54 /sys/class/fc_host/host1/device
-> ../../../devices/pci0000:00/0000:00:06.0/0000:0b:00.0/0000:0c:09.0/0000:0d:00.1/host1
lrwxrwxrwx 1 root root 0 Jul 28 11:55 /sys/class/fc_host/host0/device
-> ../../../devices/pci0000:00/0000:00:06.0/0000:0b:00.0/0000:0c:09.0/0000:0d:00.0/host0

(note the 0d:00.0 and 0d:00.1 PCI ID in the symlink targets)

And I believe:
# ls -l /sys/block/*/device

will show you the relationship between the block device and the FC
port you were looking for in d).

I would interpret your missing block devices as being each of your
target HBA seeing only one of the storage controllers, and not both -
but check the output of "ls -l /sys/block/{sdao,sdd}/device" to make
sure.

HTH.

>
> Thanks in advance,
> Gianluca

--
Jonathan Barber <jonathan.barber@gmail.com>

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 07-28-2010, 01:51 PM
Gianluca Cecchi
 
Default How to associate pci id with wwpn for an hba

On Wed, Jul 28, 2010 at 1:26 PM, Jonathan Barber wrote:




# lspci | grep QLogic

0d:00.0 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to

PCI Express HBA (rev 03)

0d:00.1 Fibre Channel: QLogic Corp. ISP2432-based 4Gb Fibre Channel to

PCI Express HBA (rev 03)



# ls -l /sys/class/fc_host/host?/device

lrwxrwxrwx 1 root root 0 Jul 28 11:54 /sys/class/fc_host/host1/device

-> ../../../devices/pci0000:00/0000:00:06.0/0000:0b:00.0/0000:0c:09.0/0000:0d:00.1/host1

lrwxrwxrwx 1 root root 0 Jul 28 11:55 /sys/class/fc_host/host0/device

-> ../../../devices/pci0000:00/0000:00:06.0/0000:0b:00.0/0000:0c:09.0/0000:0d:00.0/host0



(note the 0d:00.0 and 0d:00.1 PCI ID in the symlink targets)



And I believe:

# ls -l /sys/block/*/device



will show you the relationship between the block device and the FC

port you were looking for in d).



I would interpret your missing block devices as being each of your

target HBA seeing only one of the storage controllers, and not both -

but check the output of "ls -l /sys/block/{sdao,sdd}/device" to make

sure.





Thanks for your answer Jonathan,
In my case I got
Jul 27 17:54:58 orastud1 kernel: qla2xxx 0000:06:00.0: SNS scan failed -- assuming zero-entry result...

and I'm sure both the controllers were visible as this is the passive node of a 2 nodes cluster with same SAN visibility between nodes.

And the other node (the active one) had same fc-switch config and no problem at all with the paths.
*
Now the problem is solved for this node (probably it was a matter of a changed but not activated config... because a group of SAN disks was swapped) and I get:


# ls -l /sys/class/fc_host/host?/device
lrwxrwxrwx 1 root root 0 Jul 27 18:16 /sys/class/fc_host/host1/device -> ../../../devices/pci0000:00/0000:00:07.0/0000:06:00.0/host1
lrwxrwxrwx 1 root root 0 Jul 27 18:16 /sys/class/fc_host/host2/device -> ../../../devices/pci0000:00/0000:00:07.0/0000:06:00.1/host2


]# ls -l /sys/block/*/device
lrwxrwxrwx 1 root root 0 Jul 27 18:16 /sys/block/sdaa/device -> ../../devices/pci0000:00/0000:00:07.0/0000:06:00.1/host2/rport-2:0-2/target2:0:2/2:0:2:12
lrwxrwxrwx 1 root root 0 Jul 27 18:16 /sys/block/sdab/device -> ../../devices/pci0000:00/0000:00:07.0/0000:06:00.1/host2/rport-2:0-2/target2:0:2/2:0:2:13

lrwxrwxrwx 1 root root 0 Jul 27 18:16 /sys/block/sdac/device -> ../../devices/pci0000:00/0000:00:07.0/0000:06:00.0/host1/rport-1:0-2/target1:0:2/1:0:2:13
lrwxrwxrwx 1 root root 0 Jul 27 18:16 /sys/block/sdad/device -> ../../devices/pci0000:00/0000:00:07.0/0000:06:00.1/host2/rport-2:0-2/target2:0:2/2:0:2:14

[snipped... long output]

I can see that all the host1 lines contain 06:00.0 (and 1:0:x:y devices at the end), while all the host2 lines contain 06:00.1 (and 2:0:x:y for device at the end)

In case of similar situation I would only have to see if

a) I have a mix of host1 and host2 disks lines
==> each hba is seeing only one path;

b) I have only hostx lines
==> I have only 1 hba involved in LUN visibility problems: the hosty missing in output above (and related pci id)...

and then with
cat /sys/class/fc_host/hosty/port_name
I would get its wwpn to pass to SAN guys

Thanks!

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 12:25 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org