FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 03-18-2010, 03:13 PM
Kay Sievers
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Thu, Mar 18, 2010 at 14:58, Milan Broz <mbroz@redhat.com> wrote:
> Block layer sends ADD uevent when new device is initialised.
>
> But the device-mapper block devices are more complex,
> initialisation consists of allocating underlying device and
> loading mapping table.
>
> Because from the userspace all block devices should behave
> the same, patch defines new flag indicating that ADD event
> should be suppressed in block layer.
>
> If the flag is set, caller then take full responsibility
> for enabling and sending events later when device is ready
> to use.

This will disconnect /sys, /dev and /proc from the flow of events. We
rather like to keep them in sync. Device enumeration will find devices
which have never been announced before. It will also find devices
where "remove" was sent, but the device is still there. I don't think
this disconnect is really acceptable from a driver core perspective
and its consumers. On systems without devtmpfs, there will be no
device node for the dm device while there is already the full sysfs
entry and udev would be idle (settle returns), but the state in /sys
not fully reflected in /dev, which is what we should avoid.

How about doing a two-stage instantiation instead of mangling events
to work around this setup model? Instead of creating the mentioned
event/sys inconsistencies, did you think about not registering the
"dead" blockdev until it is ready to be used as a blockdev? Like you
would allocate the dm instance in the kernel, but only register the
blockdev with the block subsystem when the device is resumed/the table
loaded. That way, only after the device is usable, it would get
registered and appear in /dev, /sys and proc.

Thanks,
Kay

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 07:27 AM
Kay Sievers
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Thu, Mar 18, 2010 at 22:35, Milan Broz <mbroz@redhat.com> wrote:

> But if your statement is "it is your broken activation model" as you said
> in some discussion, I can do nothing, just disagree - it is different model,
> not broken.

Sure, if you are fine with that model, I'm fine with it too. You are
the one sending patches to mangle basic driver-core definitions to
paper-over some issues dm seem to have with it. I just object to such
core changes, not to the way dm is doing things.

I have no problem with dm creating "dead" devices, it's like this
since a long time, but please don't try to fake things in the driver
core to make it look different from what it is. We don't want /sys and
/dev and events to be out of sync, like non-"add"-ed devices which are
fully created in /sys, or "remove"-d devices which are still fully
populated in /sys.

/sys is the direct export of kernel objects, if you create objects,
they appear, and they get announced. If you don't want them to be
announced at that time, just don't register them at that time.

Don't get me wrong, I'm not asking you to change the current state how
dm is doing things, I just object to the patch which inconsistently
tries to fake events, which do not match the state in /sys.

Thanks,
Kay

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 08:10 AM
Milan Broz
 
Default Send KOBJ_ADD event after dm resume ioctl.

On 03/19/2010 09:27 AM, Kay Sievers wrote:
> On Thu, Mar 18, 2010 at 22:35, Milan Broz <mbroz@redhat.com> wrote:

> /sys is the direct export of kernel objects, if you create objects,
> they appear, and they get announced. If you don't want them to be
> announced at that time, just don't register them at that time.

Well, again, I used something what is already used (not on many places,
but it is there), just search for dev_set_uevent_suppress().

See

/* delay uevents, until we scanned partition table */
dev_set_uevent_suppress(ddev, 1);

... (part table scan etc. it reads disk, so there can be
significant delay if device retries read for example)

+ /* caller will send ADD event later */
+ if (disk->flags & GENHD_FL_SUPPRESS_ADD_EVENT)
+ return;
+
/* announce disk after possible partitions are created */
dev_set_uevent_suppress(ddev, 0);
kobject_uevent(&ddev->kobj, KOBJ_ADD);


So the comment says that base device is announced after all
partitions devices are created.

I thought it is exactly the same model I used - so there is for
some time unannounced fully created "dead" device in sysfs.

Probably I am still missing where the problem is - the ADD
event is sent later, so the problem in time interval?
Or the atomicity of the call?

Well, if it is not acceptable (is this what you want to say?),
what do you suggest?

Not use add_disk()/del_gendisk() in dm core and rewrite it?
This seems like change a lot of code. And after that someone
surely will say "why dm implements this differently".

But I think it always need a change in device core code.

Thanks,
Milan
--
mbroz@redhat.com

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 08:22 AM
Kay Sievers
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Fri, Mar 19, 2010 at 10:10, Milan Broz <mbroz@redhat.com> wrote:
> On 03/19/2010 09:27 AM, Kay Sievers wrote:
>> On Thu, Mar 18, 2010 at 22:35, Milan Broz <mbroz@redhat.com> wrote:
>
>> /sys is the direct export of kernel objects, if you create objects,
>> they appear, and they get announced. If you don't want them to be
>> announced at that time, just don't register them at that time.
>
> Well, again, I used something what is already used (not on many places,
> but it is there), just search for dev_set_uevent_suppress().
>
> See

Sure, I see. It's code I added myself.

> /* delay uevents, until we scanned partition table */
> dev_set_uevent_suppress(ddev, 1);
>
> ... (part table scan etc. it reads disk, so there can be
> significant delay if device retries read for example)

Come on. It's the same code path, and it's for proper sysfs timing,
not for some it-can-happen-any-time action triggered from userspace.

> So the comment says that base device is announced after all
> partitions devices are created.

Sure, but that's not in any sense what matches your patch, and
"remove" is still fully handled by the core.

> Well, if it is not acceptable (is this what you want to say?),
> what do you suggest?

Leave it as it is, or fix the driver core _interaction_, not the core itself.

> Not use add_disk()/del_gendisk() in dm core and rewrite it?
> This seems like change a lot of code. And after that someone
> surely will say "why dm implements this differently".

Why would anybody say that. You create the _always_visible_ objects
when they are supposed to be visible, that would be all you do. I'm
not saying you need to do that, but you better don't fake around in
driver core events like your patch tries to do.

> But I think it always need a change in device core code.

Sure, we can do all reasonable changes here, inconsistent /sys and
event state we try to avoid for many good reasons.

Kay

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 08:24 AM
Kay Sievers
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Fri, Mar 19, 2010 at 10:06, Lars Ellenberg <lars.ellenberg@linbit.com> wrote:
> Would introducing a KOBJ_READY_TO_BE_CONSUMED_BY_UDEV help?

No, that's what "change" is for, and we already have these "change"
events for dm. Udev does not care if the device is ready or not, it
synchronizes /sys and /dev, and that works just fine with "change"
events.

Kay

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 08:42 AM
Hannes Reinecke
 
Default Send KOBJ_ADD event after dm resume ioctl.

Milan Broz wrote:
> Block layer sends ADD uevent when new device is initialised.
>
> But the device-mapper block devices are more complex,
> initialisation consists of allocating underlying device and
> loading mapping table.
>
> Because from the userspace all block devices should behave
> the same, patch defines new flag indicating that ADD event
> should be suppressed in block layer.
>
> If the flag is set, caller then take full responsibility
> for enabling and sending events later when device is ready
> to use.
>

Please. DONT.

driver core sends out 'ADD' uevents whenever a device
is added (ie made visible) in /sys.
There is _no_ guarantee that a device is usable at this
point, hence there is is 'CHANGE' event which those device
afflicted by this sent out to signal to userspace the
device is now ready for use.

You are not alone here, S/390 DASD and qeth driver
behave the same.

And udev (and multipath :-) are pretty much aware of
the fact. And so should other userspace tools.

No need to fiddle around with events here. It'll
just serve to confuse userspace.

Cheers,

Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 09:16 AM
Kay Sievers
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Fri, Mar 19, 2010 at 10:49, Milan Broz <mbroz@redhat.com> wrote:
> On 03/19/2010 10:24 AM, Kay Sievers wrote:
>> On Fri, Mar 19, 2010 at 10:06, Lars Ellenberg <lars.ellenberg@linbit.com> wrote:
>>> Would introducing a KOBJ_READY_TO_BE_CONSUMED_BY_UDEV help?
>>
>> No, that's what "change" is for, and we already have these "change"
>> events for dm. Udev does not care if the device is ready or not, it
>> synchronizes /sys and /dev, and that works just fine with "change"
>> events.
>
> Udev (rules) do not not care if the device is ready?
>
> That's really news for me. So you are basically saying that dm ADD event is ok,
> and it is problem of udev rules that they react here on ADD event and run
> various scan over not-yet-ready device because it should wait for CHANGE?

It will not wait, the tools will just fail to open the device, and
udev will only create the basic device node, but not possible
metadata-based symlinks or anything like that. That is expected
behavior, udev will handle just fine. The needed information will only
be readable with the next "change" event, and be fully updated with
every "change" event after that.

Usually, it does not make much sense to distinguish between "add" and
"change" in userspace/udev rules. The called udev rules are the same,
and should just check a subsystem-provided property if they should go
ahead or ignore the device.

Udev itself handles "add" and "change" exactly the same way. The only
exception is network device renaming, which only runs on "add".

There are several subsystems that depend on updating everything with
"change" events when device configurations change. There is nothing
inherently wrong with this approach, as long as subsystems send the
proper "change" events and don't try to hide anything they have
registered.

Kay

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 10:14 AM
Milan Broz
 
Default Send KOBJ_ADD event after dm resume ioctl.

On 03/19/2010 11:16 AM, Kay Sievers wrote:

> There are several subsystems that depend on updating everything with
> "change" events when device configurations change. There is nothing
> inherently wrong with this approach, as long as subsystems send the
> proper "change" events and don't try to hide anything they have
> registered.

ok, this is perfectly fine with dm devices, CHANGE announces all
changes. Just I am not sure if all consumers of events (and separate
rules authors) know about that, I saw so many problems with
failing something when wrongly reacting to ADD event...

Also it means that after ADD the by-uuid* and similar symlinks
cannot be yet trusted - if the UUID is read from device and device
is not yet ready.

Well. And what should happen if anyone generate
artificial CHANGE event before the real first CHANGE event comes from
subsystem? (yes, I am looking at you, OPTIONS+="watch" thing for example)

According to above, rules must be written such way that every ADD/CHANGE
event must expect that device is not ready, so it can create only
partial info in udev, is it correct?
(this is of course no problem if the rules are the same for both cases)

Milan
--
mbroz@redhat.com

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 10:44 AM
Kay Sievers
 
Default Send KOBJ_ADD event after dm resume ioctl.

On Fri, Mar 19, 2010 at 12:14, Milan Broz <mbroz@redhat.com> wrote:
> On 03/19/2010 11:16 AM, Kay Sievers wrote:
>
>> There are several subsystems that depend on updating everything with
>> "change" events when device configurations change. There is nothing
>> inherently wrong with this approach, as long as subsystems send the
>> proper "change" events and don't try to hide anything they have
>> registered.
>
> ok, this is perfectly fine with dm devices, CHANGE announces all
> changes. Just I am not sure if all consumers of events (and separate
> rules authors) know about that, I saw so many problems with
> failing something when wrongly reacting to ADD event...
>
> Also it means that after ADD the by-uuid* and similar symlinks
> cannot be yet trusted - if the UUID is read from device and device
> is not yet ready.

Yes, such links will not be created, or even be removed, if we can
not/or no longer retrieve the needed information from the device.

> Well. And what should happen if anyone generate
> artificial CHANGE event before the real first CHANGE event comes from
> subsystem? (yes, I am looking at you, OPTIONS+="watch" thing for example)

We retrieve the device state and if it's not ready, we just exit. We
used to do that by calling dmsetup and checking the state of the
device. If the device is ready we create all the stuff, if not, we
just skip the rules. A later event from the subsystem might do the
same stuff again, udev preserves everything that does not get updated,
and this is usually cheap enough and such events do not happen at a
high frequency.

In the simplest case, which will not really apply here, rules could
just check if the size is not "0", and only go ahead if it found a
device which has readable stuff on it.

Ideally, we just gather the needed information from the subsystem
always in the same way, and react on the event the same way,
regardless if it is the first, the second, or any later event.

> According to above, rules must be written such way that every ADD/CHANGE
> event must expect that device is not ready, so it can create only
> partial info in udev, is it correct?
> (this is of course no problem if the rules are the same for both cases)

Yeah, usually it does not create any meta-information besides the
primary device node.

Kay

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-19-2010, 11:08 AM
Milan Broz
 
Default Send KOBJ_ADD event after dm resume ioctl.

On 03/19/2010 12:44 PM, Kay Sievers wrote:
> On Fri, Mar 19, 2010 at 12:14, Milan Broz <mbroz@redhat.com> wrote:
>> On 03/19/2010 11:16 AM, Kay Sievers wrote:
>> Well. And what should happen if anyone generate
>> artificial CHANGE event before the real first CHANGE event comes from
>> subsystem? (yes, I am looking at you, OPTIONS+="watch" thing for example)
>
> We retrieve the device state and if it's not ready, we just exit.

How? By scanning/opening it?

Scan on uninitialised device can lock the device and break initialisation,
"-EBUSY" - isn't it race?

So all applications, which run some kind of configuration of device
should expect that device can be randomly locked by udev rules...

[Yes, it happens. I solved several problems in cryptsetup where various
udev scans open and locks keyslot device.
(Ignoring the fact that it contains key material, in this case it was
not security problem - the material is still obfuscated.)
These are now masked in udev rules, but I do not like this approach much.]

> Yeah, usually it does not create any meta-information besides the
> primary device node.

Unfortunately it is not the DM/LVM etc case - we had always symlinks
and long device names etc.

Anyway, this name/uuid information is in sysfs now.
But not the VG/LV symlinks info which is pure userspace abstraction...

Milan
--
mbroz@redhat.com

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 08:44 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org