FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 05-16-2008, 07:35 PM
Christophe Varoqui
 
Default Read-Only devices, multipath and more

Hi,

I'll try to summarize the facts about how linux kernel (scsi and
device-mapper) and userspace (multipath-tools) handles devices
RO->RW->RO changes.

The sample setup is 2 x EMC Symmetrix systems, with a pair of logical
unit configured for synchronous inter-system replication (SRDF). In a
normal situation, the LU receiving the data updates (the R2) is RO,
while the LU emitting the data updates (the R1) is RW. Let's take this
state as a starting point.

Server S1 sees R1. Server S2 sees R2

---

t0)
storage state: R1 (RW) <- sync -> R2 (RO)
action: Linux system on S2 freshly rebooted.
fact: scsi paths to R2 are reported read-only by "hdparm -r"
fact: device-mapper refuse to load a RW multipath table on these paths
(ie without libdevmapper:set_task_set_ro())

question: should change this behaviour to allow to load a RW multipath
table on these paths and let the IO be failed by the storage hardware ?

t1)
storage state: R1 (RW) <- split -> R2 (RW)
action: none
fact: scsi paths to R2 are still reported read-only by "hdparm -r"
fact: device-mapper still refuse to load a RW multipath table on these
paths
fact: if a RO multipath table was loaded at t0, it is still RO at t1

question: shouldn't the write protection change be detected by the scsi
kernel subsystem, or should we implement a userspace device polling ?
question: how do we detect from userspace the device write protection
change ? (trying to load a multipath devmap is not a good test).
sg-utils maybe ? in multipathd or in a separe daemon, as the issue
extends beyond multipathing ?

action: echo 1>/sys/block/sd{a,b}/device/rescan
fact: the write protection flags are updated to the correct RW state and
multipath then works as expected

question: is there a softer way to update the write proctection flags ?

t2)
storage state: R1 (RW) <- resync -> R2 (RO)
action: none
fact: the RW multipath devmap is still loaded

question: so why not permit to load it in the first place ?

---

This scenario shows there is an annoying lack of consistency and
symmetry in the Linux behaviour. I'm willing to implement whatever is
expected from the multipath-tools. But can we define what is expected ?

Alasdair proposed to add more explicit table loading ioctl return code
when the failure is due to this ready-only paths issue (E_ROFS for
example). Which comes short of solving the RO->RW devmap promotion

Please advise.
Christophe Varoqui (keep me on cc

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-20-2008, 02:48 PM
Alasdair G Kergon
 
Default Read-Only devices, multipath and more

On Fri, May 16, 2008 at 09:35:44PM +0200, christophe varoqui wrote:
> fact: device-mapper refuse to load a RW multipath table on these paths
> (ie without libdevmapper:set_task_set_ro())
> question: should change this behaviour to allow to load a RW multipath
> table on these paths and let the IO be failed by the storage hardware ?

Yes. But relax this *only* for multipath targets, where the whole idea
is to "hide" path problems from the layer above. It would not make sense
in my view to do this for other DM targets.

> Alasdair proposed to add more explicit table loading ioctl return code
> when the failure is due to this ready-only paths issue (E_ROFS for
> example).

I think this should be done too (for non-multipath and other errors).

Alasdair
--
agk@redhat.com

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 05-20-2008, 09:09 PM
Christophe Varoqui
 
Default Read-Only devices, multipath and more

Le mardi 20 mai 2008 * 15:48 +0100, Alasdair G Kergon a écrit :
> On Fri, May 16, 2008 at 09:35:44PM +0200, christophe varoqui wrote:
> > fact: device-mapper refuse to load a RW multipath table on these paths
> > (ie without libdevmapper:set_task_set_ro())
> > question: should change this behaviour to allow to load a RW multipath
> > table on these paths and let the IO be failed by the storage hardware ?
>
> Yes. But relax this *only* for multipath targets, where the whole idea
> is to "hide" path problems from the layer above.

Fair enough.

> It would not make sense
> in my view to do this for other DM targets.
>

During my testing I remember seeing a RW linear map loaded over a RO
multipath map (through a genuine vgchange -ay, the PV being the
multipath).

> > Alasdair proposed to add more explicit table loading ioctl return code
> > when the failure is due to this ready-only paths issue (E_ROFS for
> > example).
>
> I think this should be done too (for non-multipath and other errors).
>
Right, I'm about the publish a changeset for supporting the whole RO
notion in the multipath-tools :
1/ printing the RO/RW information
2/ fallbacking to RO when RW loading fails
3/ new flag to force map reloading, which takes care of the RO->RW
promotion and RW->RO demotion scenarii

I'll adapt 2/ to your new return codes when available.


Thanks for caring,
cvaroqui

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 05:42 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org