dmsetup status not clear

03-04-2009, 09:35 PM

I have raid1 setup using nvidia raid controller.

00:08.0 IDE interface: nVidia Corporation CK804 Serial ATA Controller
(rev f3) (prog-if 85 [Master SecO PriO])
Subsystem: Sun Microsystems Computer Corp. Device 5348
Flags: bus master, 66MHz, fast devsel, latency 0, IRQ 23
I/O ports at 09e0 [size=8]
I/O ports at 0be0 [size=4]
I/O ports at 0960 [size=8]
I/O ports at 0b60 [size=4]
I/O ports at c000 [size=16]
Memory at fe02b000 (32-bit, non-prefetchable) [size=4K]
Capabilities: <access denied>
Kernel driver in use: sata_nv
Kernel modules: ata_generic, sata_nv, pata_acpi

When I take bad disk out I get this message in /var/log/messages

Mar 4 17:20:11 ftp0 kernel: [ 2095.666527] ata4: hard resetting link
Mar 4 17:20:12 ftp0 kernel: [ 2096.389020] ata4: SATA link down
(SStatus 0 SControl 300)
Mar 4 17:20:17 ftp0 kernel: [ 2101.388029] ata4: hard resetting link
Mar 4 17:20:17 ftp0 kernel: [ 2101.708031] ata4: SATA link down
(SStatus 0 SControl 300)
Mar 4 17:20:22 ftp0 kernel: [ 2106.709016] ata4: hard resetting link
Mar 4 17:20:22 ftp0 kernel: [ 2107.029018] ata4: SATA link down
(SStatus 0 SControl 300)
Mar 4 17:20:22 ftp0 kernel: [ 2107.029027] ata4.00: disabled
Mar 4 17:20:22 ftp0 kernel: [ 2107.029038] ata4: EH complete
Mar 4 17:20:22 ftp0 kernel: [ 2107.029461] ata4.00: detaching (SCSI 3:0:0:0)
Mar 4 17:20:22 ftp0 kernel: [ 2107.091392] sd 3:0:0:0: [sdb] Result:
hostbyte=DID_NO_CONNECT driverbyte=DRIVER_OK,SUGGEST_OK
Mar 4 17:20:22 ftp0 kernel: [ 2107.092546] sd 3:0:0:0: [sdb]
Synchronizing SCSI cache
Mar 4 17:20:22 ftp0 kernel: [ 2107.092591] sd 3:0:0:0: [sdb] Result:
hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK
Mar 4 17:20:22 ftp0 kernel: [ 2107.092594] sd 3:0:0:0: [sdb] Stopping disk
Mar 4 17:20:22 ftp0 kernel: [ 2107.092601] sd 3:0:0:0: [sdb] START_STOP FAILED
Mar 4 17:20:22 ftp0 kernel: [ 2107.092602] sd 3:0:0:0: [sdb] Result:
hostbyte=DID_BAD_TARGET driverbyte=DRIVER_OK,SUGGEST_OK

And the first two lines from dmesg that did not show up in messages file

[ 2095.513781] ata4: exception Emask 0x10 SAct 0x0 SErr 0x1810000
action 0xe frozen
[ 2095.602104] ata4: SError: { PHYRdyChg LinkSeq TrStaTrns }
[ 2095.666527] ata4: hard resetting link

Now I inserted a new disk and I get this message from messages file

Mar 4 17:26:21 ftp0 kernel: [ 2465.606247] ata4: hard resetting link
Mar 4 17:26:25 ftp0 kernel: [ 2470.136026] ata4: SATA link up 3.0
Gbps (SStatus 123 SControl 300)
Mar 4 17:26:25 ftp0 kernel: [ 2470.160260] ata4.00: ATA-7: HITACHI
HDS7280SASUN80G 0638M70W3M, PF2OA6EA, max UDMA/133
Mar 4 17:26:25 ftp0 kernel: [ 2470.160265] ata4.00: 156301488
sectors, multi 0: LBA48 NCQ (depth 0/32)
Mar 4 17:26:26 ftp0 kernel: [ 2470.176260] ata4.00: configured for UDMA/133
Mar 4 17:26:26 ftp0 kernel: [ 2470.176267] ata4: EH complete
Mar 4 17:26:26 ftp0 kernel: [ 2470.176368] scsi 3:0:0:0:
Direct-Access ATA HITACHI HDS7280S PF2O PQ: 0 ANSI: 5
Mar 4 17:26:26 ftp0 kernel: [ 2470.177508] sd 3:0:0:0: [sdc]
156301488 512-byte hardware sectors (80026 MB)
Mar 4 17:26:26 ftp0 kernel: [ 2470.177524] sd 3:0:0:0: [sdc] Write
Protect is off
Mar 4 17:26:26 ftp0 kernel: [ 2470.177549] sd 3:0:0:0: [sdc] Write
cache: disabled, read cache: enabled, doesn't support DPO or FUA
Mar 4 17:26:26 ftp0 kernel: [ 2470.177621] sd 3:0:0:0: [sdc]
156301488 512-byte hardware sectors (80026 MB)
Mar 4 17:26:26 ftp0 kernel: [ 2470.177633] sd 3:0:0:0: [sdc] Write
Protect is off
Mar 4 17:26:26 ftp0 kernel: [ 2470.177657] sd 3:0:0:0: [sdc] Write
cache: disabled, read cache: enabled, doesn't support DPO or FUA
Mar 4 17:26:26 ftp0 kernel: [ 2470.177660] sdc: sdc1 sdc2 < sdc5 >
Mar 4 17:26:26 ftp0 kernel: [ 2470.215634] sd 3:0:0:0: [sdc] Attached SCSI disk
Mar 4 17:26:26 ftp0 kernel: [ 2470.215936] sd 3:0:0:0: Attached scsi
generic sg1 type 0

Why is the new disk showing up as `sdc' instead of `sdb' and why is
the write cache disabled?

Of course similar two lines below showing up only in dmesg and not in
messages file

[ 2465.456628] ata4: exception Emask 0x10 SAct 0x0 SErr 0x150000
action 0xe frozen
[ 2465.543901] ata4: SError: { PHYRdyChg CommWake Dispar }
[ 2465.606247] ata4: hard resetting link

Also, how can I find out if/when the raid1 sync completed? I do not
follow what the dmsetup status is saying

~$ sudo dmsetup status
nvidia_eeffhbef5: 0 6458067 linear
nvidia_eeffhbef1: 0 149838192 linear
nvidia_eeffhbef: 0 156301486 mirror 2 8:0 8:16 1183/1193 1 AD 1 core

--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

03-05-2009, 03:20 PM

> Why is the new disk showing up as `sdc' instead of `sdb' and why is

I can't comment on the naming - but in regards to the write cache - that is
the information the kernel gets from the device using SCSI commands (to be
exact it MODE_SENSE). The device has it disabled. You can fiddle with MODE_SELECT
command if you want to enable it - but keep in mind that when you have a RAID
it is advised to turn cache off. Unless your computer has a UPS connected so that
you can flush the data immediately and turn off caching.

> the write cache disabled?
>
>
> Of course similar two lines below showing up only in dmesg and not in
> messages file
>
>
> [ 2465.456628] ata4: exception Emask 0x10 SAct 0x0 SErr 0x150000
> action 0xe frozen
> [ 2465.543901] ata4: SError: { PHYRdyChg CommWake Dispar }
> [ 2465.606247] ata4: hard resetting link
>
That is OK.
>
> Also, how can I find out if/when the raid1 sync completed? I do not
> follow what the dmsetup status is saying

cat /proc/dmstatus should give you some details.

>
>
> ~$ sudo dmsetup status
> nvidia_eeffhbef5: 0 6458067 linear
> nvidia_eeffhbef1: 0 149838192 linear
> nvidia_eeffhbef: 0 156301486 mirror 2 8:0 8:16 1183/1193 1 AD 1 core

That is just the static mapping it has setup. Where it says that disks 8:0 and 8:16
are part of the of your RAID. Do a 'ls /dev/ | grep 8' and you will see what
the 8:0, and 8:16 correspond to.

>
>
> --
> Asif Iqbal
> PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

03-05-2009, 03:35 PM

On Thu, Mar 5, 2009 at 11:20 AM, Konrad Rzeszutek
<konrad@virtualiron.com> wrote:
>> Why is the new disk showing up as `sdc' instead of `sdb' *and why is
>
> I can't comment on the naming - but in regards to the write cache - that is
> the information the kernel gets from the device using SCSI commands (to be
> exact it MODE_SENSE). The device has it disabled. You can fiddle with MODE_SELECT
> command if you want to enable it - but keep in mind that when you have a RAID
> it is advised to turn cache off. Unless your computer has a UPS connected so that
> you can flush the data immediately and turn off caching.
>
>> the write cache disabled?
>>
>>
>> Of course similar two lines below showing up only in dmesg and not in
>> messages file
>>
>>
>> [ 2465.456628] ata4: exception Emask 0x10 SAct 0x0 SErr 0x150000
>> action 0xe frozen
>> [ 2465.543901] ata4: SError: { PHYRdyChg CommWake Dispar }
>> [ 2465.606247] ata4: hard resetting link
>>
> That is OK.
>>
>> Also, how can I find out if/when the raid1 sync completed? I do not
>> follow what the dmsetup status is saying
>
> cat /proc/dmstatus should give you some details.

~# cat /proc/dmstatus
cat: /proc/dmstatus: No such file or directory

>
>>
>>
>> ~$ sudo dmsetup status
>> nvidia_eeffhbef5: 0 6458067 linear
>> nvidia_eeffhbef1: 0 149838192 linear
>> nvidia_eeffhbef: 0 156301486 mirror 2 8:0 8:16 1183/1193 1 AD 1 core
>
> That is just the static mapping it has setup. Where it says that disks 8:0 and 8:16
> are part of the of your RAID. Do a 'ls /dev/ | grep 8' and you will see what
> the 8:0, and 8:16 correspond to.

iqbala@ftp0:~$ ls -al /dev/ | awk '$5 ~ /8/ {print}'
drwxr-xr-x 2 root root 12680 2009-03-05 11:30 char
drwxr-xr-x 3 root root 180 2009-03-05 11:29 input
crw------- 1 root root 108, 0 2009-03-04 11:58 ppp
brw-rw---- 1 root disk 8, 0 2009-03-05 11:29 sda
brw-rw---- 1 root disk 8, 1 2009-03-05 11:29 sda1
brw-rw---- 1 root disk 8, 2 2009-03-05 11:29 sda2
brw-rw---- 1 root disk 8, 5 2009-03-05 11:29 sda5
brw-rw---- 1 root disk 8, 16 2009-03-05 11:29 sdb
brw-rw---- 1 root disk 8, 17 2009-03-05 11:29 sdb1
brw-rw---- 1 root disk 8, 18 2009-03-05 11:29 sdb2
brw-rw---- 1 root disk 8, 21 2009-03-05 11:29 sdb5

I see 8:0 is sda and 8:16 is sbd, cool!

The dmsetup status has a little different output now

iqbala@ftp0:~$ sudo dmsetup status
nvidia_eeffhbef5: 0 6458067 linear
nvidia_eeffhbef1: 0 149838192 linear
nvidia_eeffhbef: 0 156301486 mirror 2 8:0 8:16 1193/1193 1 AA 1 core

It was '1183/1193 1 AD 1 core' and now '1193/1193 1 AA 1 core'

What are the differences? Is there some doc where I can read more about this?

>
>>
>>
>> --
>> Asif Iqbal
>> PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
>> A: Because it messes up the order in which people normally read text.
>> Q: Why is top-posting such a bad thing?
>>
>> --
>> dm-devel mailing list
>> dm-devel@redhat.com
>> https://www.redhat.com/mailman/listinfo/dm-devel
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>

--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

03-05-2009, 04:15 PM

> > cat /proc/dmstatus should give you some details.
>
> ~# cat /proc/dmstatus
> cat: /proc/dmstatus: No such file or directory

Ugh. /proc/mdstat I think.

.. snip ..
> iqbala@ftp0:~$ sudo dmsetup status
> nvidia_eeffhbef5: 0 6458067 linear
> nvidia_eeffhbef1: 0 149838192 linear
> nvidia_eeffhbef: 0 156301486 mirror 2 8:0 8:16 1193/1193 1 AA 1 core
>
> It was '1183/1193 1 AD 1 core' and now '1193/1193 1 AA 1 core'
>
> What are the differences? Is there some doc where I can read more about this?

It is divided in two groups. The first is the
type and the size (0 156301486 mirror): meaning 156301486 sectors, or:
76319MB, and the kernel module that oversears this device is 'dm-mirror'.

The next group of data is from 'mirror_status' function (from the dm-raid1 module, which
actually registers itself as 'dm-mirror'). Take a look
at drivers/md/dm-raid1.c and the above mentioned function - that should
give you an idea what is what.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

03-05-2009, 06:29 PM

On Thu, Mar 5, 2009 at 12:15 PM, Konrad Rzeszutek
<konrad@virtualiron.com> wrote:
>> > cat /proc/dmstatus should give you some details.
>>
>> ~# cat /proc/dmstatus
>> cat: /proc/dmstatus: No such file or directory
>
> Ugh. /proc/mdstat I think.

# cat /proc/mdstat
cat: /proc/mdstat: No such file or directory

>
>
> .. snip ..
>> iqbala@ftp0:~$ sudo dmsetup status
>> nvidia_eeffhbef5: 0 6458067 linear
>> nvidia_eeffhbef1: 0 149838192 linear
>> nvidia_eeffhbef: 0 156301486 mirror 2 8:0 8:16 1193/1193 1 AA 1 core
>>
>> It was '1183/1193 1 AD 1 core' and now '1193/1193 1 AA 1 core'
>>
>> What are the differences? Is there some doc where I can read more about this?
>
> It is divided in two groups. The first is the
> type and the size (0 156301486 mirror): meaning 156301486 sectors, or:
> 76319MB, and the kernel module that oversears this device is 'dm-mirror'.
>
> The next group of data is from 'mirror_status' function (from the dm-raid1 module, which
> actually registers itself as 'dm-mirror'). Take a look
> at drivers/md/dm-raid1.c and the above mentioned function - that should
> give you an idea what is what.
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>

--
Asif Iqbal
PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel