FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 03-18-2010, 04:24 PM
Alasdair G Kergon
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Thu, Mar 18, 2010 at 05:13:22PM +0100, Kay Sievers wrote:
> How about doing a two-stage instantiation instead of mangling events
> to work around this setup model? Instead of creating the mentioned
> event/sys inconsistencies, did you think about not registering the
> "dead" blockdev until it is ready to be used as a blockdev? Like you
> would allocate the dm instance in the kernel, but only register the
> blockdev with the block subsystem when the device is resumed/the table
> loaded. That way, only after the device is usable, it would get
> registered and appear in /dev, /sys and proc.

Still going around in circles, I'm afraid.

That looked like even bigger block layer surgery last time we considered it.
By all means, look at it yet again...

We're creating trees of devices that reference each other (need to know
minor numbers) and have constraints about which parts of the code may
allocate new memory.

Alasdair

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-18-2010, 08:35 PM
Milan Broz
 
Default Send KOBJ_ADD event after dm resume ioctl.

On 03/18/2010 05:13 PM, Kay Sievers wrote:
> On Thu, Mar 18, 2010 at 14:58, Milan Broz <mbroz@redhat.com> wrote:
>> Block layer sends ADD uevent when new device is initialised.
>>
>> But the device-mapper block devices are more complex,
>> initialisation consists of allocating underlying device and
>> loading mapping table.
>>
>> Because from the userspace all block devices should behave
>> the same, patch defines new flag indicating that ADD event
>> should be suppressed in block layer.
>>
>> If the flag is set, caller then take full responsibility
>> for enabling and sending events later when device is ready
>> to use.

Hi Kay,

First - the patch is intentionally simple, because it is was the goal.

Call it hack if you want - the goal was fix events to fit
the udev rules specification of events, not rewrite block layer
initialisation.
("Why the dm devices need special handling of ADD event" problem.)

Now you are saying, that it is not enough.
Well I am not sure if I understand these objections:

> This will disconnect /sys, /dev and /proc from the flow of events. We
> rather like to keep them in sync. Device enumeration will find devices
> which have never been announced before. It will also find devices
> where "remove" was sent, but the device is still there. I don't think
> this disconnect is really acceptable from a driver core perspective
> and its consumers. On systems without devtmpfs, there will be no
> device node for the dm device while there is already the full sysfs
> entry and udev would be idle (settle returns), but the state in /sys
> not fully reflected in /dev, which is what we should avoid.

If it is problem it is already there for ages.
The device node appears immediately when device is usable.

Device without mapping table is not usable.

The information about device is not yet in /partitions, it is present
in /sysfs, but the device have zero size
(size is defined by table which is not yet loaded)
This is _current_ state and I think it was this way since 2.6.0 kernel.
No change with this patch.

What's the real problem there?

And why device-mapper need separate allocation of major:minor pair
and then load table in two separate steps:

device-mapper devices can be more complex that one simple device,
it constructs devices by referencing other devices in the table.

An example are snapshots - you have the origin device, (several) COW(s)
and these are linked together and must be activated
(read: resumed) as a tree - together.

I understand that you can invent another model which fit better udev
and I am sure that if device-mapper is designed now, udev integration
is one of the prerequisite. But it pre-dates udev.
Of course the final state is that some functions will be integrated into
block layer and the special "device-mapper" add-on disappears together
with this problem... :-)

> How about doing a two-stage instantiation instead of mangling events
> to work around this setup model? Instead of creating the mentioned
> event/sys inconsistencies, did you think about not registering the
> "dead" blockdev until it is ready to be used as a blockdev? Like you
> would allocate the dm instance in the kernel, but only register the
> blockdev with the block subsystem when the device is resumed/the table
> loaded. That way, only after the device is usable, it would get
> registered and appear in /dev, /sys and proc.

See above - if you mean registering device = allocating major:minor pair,
it is not possible to move it to resume.
Not because I am ignorant but because it is basic design principle
of the device-mapper.

Probably we can achieve it by duplicating/rewriting a lot of block layer
code - but I am sure you discussed this with Alasdair several times.

Mangling events... why you think it is mangling events? It just move ADD
event to proper place, when the device becomes ready.
Sysfs entry always appears before it - just the time interval
is slightly different here (but resume is called immediately after create).

I am trying to use udev view of problem - that's why I pushed to
add sysfs entries for dm for example. And it worked.

But if your statement is "it is your broken activation model" as you said
in some discussion, I can do nothing, just disagree - it is different model,
not broken.
And we have to find compromise how to work with it together.

Thanks,
Milan
--
mbroz@redhat.com

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 08:49 AM
Milan Broz
 
Default Send KOBJ_ADD event after dm resume ioctl.

On 03/19/2010 10:24 AM, Kay Sievers wrote:
> On Fri, Mar 19, 2010 at 10:06, Lars Ellenberg <lars.ellenberg@linbit.com> wrote:
>> Would introducing a KOBJ_READY_TO_BE_CONSUMED_BY_UDEV help?
>
> No, that's what "change" is for, and we already have these "change"
> events for dm. Udev does not care if the device is ready or not, it
> synchronizes /sys and /dev, and that works just fine with "change"
> events.

Udev (rules) do not not care if the device is ready?

That's really news for me. So you are basically saying that dm ADD event is ok,
and it is problem of udev rules that they react here on ADD event and run
various scan over not-yet-ready device because it should wait for CHANGE?

ok, then I am wasting time with fixing the dm ADD event. We have this already.

Mea culpa. Sorry for spam then:-)

Milan
--
mbroz@redhat.com

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 10:50 AM
Hannes Reinecke
 
Default Send KOBJ_ADD event after dm resume ioctl.

Milan Broz wrote:
> On 03/19/2010 11:16 AM, Kay Sievers wrote:
>
>> There are several subsystems that depend on updating everything with
>> "change" events when device configurations change. There is nothing
>> inherently wrong with this approach, as long as subsystems send the
>> proper "change" events and don't try to hide anything they have
>> registered.
>
> ok, this is perfectly fine with dm devices, CHANGE announces all
> changes. Just I am not sure if all consumers of events (and separate
> rules authors) know about that, I saw so many problems with
> failing something when wrongly reacting to ADD event...
>
> Also it means that after ADD the by-uuid* and similar symlinks
> cannot be yet trusted - if the UUID is read from device and device
> is not yet ready.
>
Correct. We try to take care of that one by
a) not running something like 'vol_id' for 'ADD' events on dm devices
and
b) really making sure the dm device can be read from when receiving
CHANGE event.

> Well. And what should happen if anyone generate
> artificial CHANGE event before the real first CHANGE event comes from
> subsystem? (yes, I am looking at you, OPTIONS+="watch" thing for example)
>
You have to check the device anyway. 'CHANGE' literally just means that,
ie something has changed with the device state.
We used to have 'ONLINE' and 'OFFLINE' events for block devices, but
that got modified to the more generic 'CHANGE' event.

There is _no_ indication what has changed with the device state; any
program etc must check for itself if the device is in a usable state.
(Normally there won't even be any additional environment variables with
a 'CHANGE' event, so one has to look in sysfs anyway to find out what
has changed).

> According to above, rules must be written such way that every ADD/CHANGE
> event must expect that device is not ready, so it can create only
> partial info in udev, is it correct?
Correct.

> (this is of course no problem if the rules are the same for both cases)
>
Well, yes and no.
We (read: SUSE) are trying to make sure to _not_ run any programs likely
to read data from the disk when receiving 'ADD' events from these devices.

You might get lucky here when the 'ADD' event is in fact a fake as it
was triggered externally, but this is not something I'd bank on.
(And if someone is faking 'ADD' events he can as well do it properly
and fake the corresponding 'CHANGE' events, too)

HTH.

Cheers,

Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 11:00 AM
Alasdair G Kergon
 
Default Send KOBJ_ADD event after dm resume ioctl.

We've been over all this ground before, but here it is again FWIW

On Fri, Mar 19, 2010 at 11:16:15AM +0100, Kay Sievers wrote:
> It will not wait, the tools will just fail to open the device, and
> udev will only create the basic device node, but not possible
> metadata-based symlinks or anything like that. That is expected
> behavior, udev will handle just fine. The needed information will only
> be readable with the next "change" event, and be fully updated with
> every "change" event after that.

But while userspace code responding to the uevents is fiddling with the
new node deciding what to do, the userspace code (e.g. lvm2) that is
creating the device tree is continuing to manipulate it as these
are multi-step processes. If code triggered by the uevents runs
alongside the lvm2 code, stuff fails. (E.g. lvm2 wants exclusive access
which it can't get because something reacting to the uevent opened the
device temporarily - even though we could have told it not to bother as
it would find nothing of interest on it yet - a concept of 'ignored'
or 'private' devices). So we require a synchronisation mechanism so
that the lvm2 code can wait for *everything* triggered by the uevents to
finish before proceeding. (Or alternatively a mechanism to flag that
other code reacting to uevents should ignore these devices at this
stage.) We attempted to add the synchronisation ourselves
(95-dm-notify.rules) but I think you pointed out there are still things
outside our control that can be triggered without a mechanism to wait
for them to finish.

Alasdair

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 11:12 AM
Alasdair G Kergon
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Fri, Mar 19, 2010 at 11:16:15AM +0100, Kay Sievers wrote:
> It will not wait, the tools will just fail to open the device, and

In some cases the tools are wasting their time even trying to open
the device - we know in advance that it is pointless and should have
a mechanism in place so they can be told not to bother.

E.g. Flagging a device as 'private' as discussed in an earlier mail.

Alasdair

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 11:27 AM
Alasdair G Kergon
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Fri, Mar 19, 2010 at 10:42:58AM +0100, Hannes Reinecke wrote:
> No need to fiddle around with events here. It'll
> just serve to confuse userspace.

Userspace is already well-confused and doing things that look silly from
a high-level viewpoint and none of the potential solutions we've
proposed to the various problems has been acceptable yet.

So we widened the scope of our thinking, and suggested if we just break
this kobject/uevent connection in the kernel, it might give us easy
solutions to several of the seemingly-intractable
userspace-processing-of-uevent problems at once.

Alasdair

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 12:17 PM
Peter Rajnoha
 
Default Send KOBJ_ADD event after dm resume ioctl.

On 03/19/2010 01:16 PM, Kay Sievers wrote:
> On Fri, Mar 19, 2010 at 13:12, Alasdair G Kergon <agk@redhat.com> wrote:
>> E.g. Flagging a device as 'private' as discussed in an earlier mail.
>
> Sure, that can all be done. Plain udev leaves dm and md devices
> completely to the subsystem own rules. It might be other tools which
> interfere here, and that might need to be synchronized to match on
> some values in the udev properties or sysfs.
>
> Kay

Great. We have the flags in uevents, we have udev properties. We can't
put them directly in sysfs because of the reasons we already discussed
(it's also extended information related to exact device-mapper subsystem,
not easily accessible in the middle of processing from outer space).

Do we have these flags/properties available when processing the event
originated in the "watch rule" or "udevadm trigger". No. The information
is cleared from udev db and inaccessible.

That's what we need and what we tried to ask for (besides trying to solve
that problem with the ADD event).

Peter

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 12:24 PM
Peter Rajnoha
 
Default Send KOBJ_ADD event after dm resume ioctl.

On 03/19/2010 10:24 AM, Kay Sievers wrote:
> No, that's what "change" is for, and we already have these "change"
> events for dm. Udev does not care if the device is ready or not, it
> synchronizes /sys and /dev, and that works just fine with "change"
> events.

CHANGE events, not quite... We can't even rely on these.

Just to mention, there's also a CHANGE event generated when
read-only flag is set for a device (this is not managed by
device-mapper of course). This one is generated even before
the actual CHANGE event that is generated when DM device is
ready to be used.

Peter

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 12:47 PM
Alasdair G Kergon
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Fri, Mar 19, 2010 at 02:43:13PM +0100, Kay Sievers wrote:
> Sure, but as mentioned earlier, these events are just expected to
> fail, and update the current udev state, if they can't retrieve the
> needed information or find out that the device in not usable.

They have to 'fail' in a controlled way that does not interfere
with any other processing that may already be happening.

Alasdair

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 09:09 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org