FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 03-29-2011, 01:20 PM
Mike Snitzer
 
Default Please revert a91a2785b20

On Tue, Mar 29 2011 at 2:59am -0400,
Jens Axboe <jaxboe@fusionio.com> wrote:

> On 2011-03-29 01:03, Mike Snitzer wrote:
> > On Mon, Mar 28 2011 at 6:43pm -0400,
> > Thomas Gleixner <tglx@linutronix.de> wrote:
> >
> >> Forgot to cc Jens and fixed up the borked mail address of Mike which
> >> I took from the changelog
> >>
> >> On Tue, 29 Mar 2011, Thomas Gleixner wrote:
> >>
> >>> Out of the blue all my perfectly fine working test machines which use
> >>> RAID stopped working with the very helpful error message:
> >>>
> >>> md/raid1:md1: active with 2 out of 2 mirrors
> >>> md: pers->run() failed ...
> >>>
> >>> Reverting a91a2785b20 fixes the problem.
> >>>
> >>> Neither the subject line:
> >>>
> >>> block: Require subsystems to explicitly allocate bio_set integrity mempool
> >>>
> >>> nor the changelog have any hint why that wreckage is in any way
> >>> sensible.
> >>>
> >>> The wreckage happens due to:
> >>>
> >>> - md_integrity_register(mddev);
> >>> - return 0;
> >>> + return md_integrity_register(mddev);
> >
> > Right, a kernel.org BZ was filed on this here:
> > https://bugzilla.kernel.org/show_bug.cgi?id=32062
> >
> > Martin is working to "conjure up a patch" to fix the common case where
> > no devices in the MD have DIF/DIX capabilities.
>
> And I see that has been merged now. So all is good?

Yes, commit 89078d572e clearly addresses the immediate concern.

But I think we have a related issue that needs discussion, given that an
integrity profile mismatch will cause MD's assemble to fail (rather than
warn and continue to assemble without integrity support).

DM doesn't fail to load a DM device due to a integrity profile mismatch;
it just emits a warning and continues.

In contrast, MD will now disallow adding a normal disk (without
integrity support) to an array that has historically had a symmetric
integrity profile across all members.

So this begs the question: what is the correct approach?

At the moment I prefer the more forgiving approach that DM provides.
But I can appreciate the other school of thought where a mismatch is a
hard failure during device assembly (avoids possibility of falling back
to not using integrity capabilities).

Mike

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-29-2011, 01:35 PM
James Bottomley
 
Default Please revert a91a2785b20

On Tue, 2011-03-29 at 09:20 -0400, Mike Snitzer wrote:
> On Tue, Mar 29 2011 at 2:59am -0400,
> Jens Axboe <jaxboe@fusionio.com> wrote:
>
> > On 2011-03-29 01:03, Mike Snitzer wrote:
> > > On Mon, Mar 28 2011 at 6:43pm -0400,
> > > Thomas Gleixner <tglx@linutronix.de> wrote:
> > >
> > >> Forgot to cc Jens and fixed up the borked mail address of Mike which
> > >> I took from the changelog
> > >>
> > >> On Tue, 29 Mar 2011, Thomas Gleixner wrote:
> > >>
> > >>> Out of the blue all my perfectly fine working test machines which use
> > >>> RAID stopped working with the very helpful error message:
> > >>>
> > >>> md/raid1:md1: active with 2 out of 2 mirrors
> > >>> md: pers->run() failed ...
> > >>>
> > >>> Reverting a91a2785b20 fixes the problem.
> > >>>
> > >>> Neither the subject line:
> > >>>
> > >>> block: Require subsystems to explicitly allocate bio_set integrity mempool
> > >>>
> > >>> nor the changelog have any hint why that wreckage is in any way
> > >>> sensible.
> > >>>
> > >>> The wreckage happens due to:
> > >>>
> > >>> - md_integrity_register(mddev);
> > >>> - return 0;
> > >>> + return md_integrity_register(mddev);
> > >
> > > Right, a kernel.org BZ was filed on this here:
> > > https://bugzilla.kernel.org/show_bug.cgi?id=32062
> > >
> > > Martin is working to "conjure up a patch" to fix the common case where
> > > no devices in the MD have DIF/DIX capabilities.
> >
> > And I see that has been merged now. So all is good?
>
> Yes, commit 89078d572e clearly addresses the immediate concern.
>
> But I think we have a related issue that needs discussion, given that an
> integrity profile mismatch will cause MD's assemble to fail (rather than
> warn and continue to assemble without integrity support).
>
> DM doesn't fail to load a DM device due to a integrity profile mismatch;
> it just emits a warning and continues.
>
> In contrast, MD will now disallow adding a normal disk (without
> integrity support) to an array that has historically had a symmetric
> integrity profile across all members.
>
> So this begs the question: what is the correct approach?

That's easy: Anything that would cause a change in the previously
specified user profile for the array is an error (principle of least
surprise) so it can't happen automatically. Silently adding a non
integrity device would be nasty for the admin because they could think
they were adding an integrity disk when they weren't (for a variety of
reasons, like some problem with the HBA or incorrect settings on the
disk). So adding a non-integrity device to an integrity profile array
should return a soft error, with an explanation. However, the user
should be able to force the array online and change the profile. How
it's done (tooling or kernel), I'm not particularly bothered.

James


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-29-2011, 01:42 PM
"Martin K. Petersen"
 
Default Please revert a91a2785b20

>>>>> "Mike" == Mike Snitzer <snitzer@redhat.com> writes:

Mike,

Mike> But I think we have a related issue that needs discussion, given
Mike> that an integrity profile mismatch will cause MD's assemble to
Mike> fail (rather than warn and continue to assemble without integrity
Mike> support).

Mike> DM doesn't fail to load a DM device due to a integrity profile
Mike> mismatch; it just emits a warning and continues.

Mike> In contrast, MD will now disallow adding a normal disk (without
Mike> integrity support) to an array that has historically had a
Mike> symmetric integrity profile across all members.

You would invalidate all your existing integrity metadata, tagging,
etc. on existing metadevice members. That seems to be a policy decision,
so if we go down that path it would have to be keyed off a force
assembly option passed down from userland tooling. Turning off features
and/or losing metadata really should not be done without the user's
explicit consent.

Also, let's assume you run an integrity-aware app on a DM device and you
add a non-integrity drive. The DM device is then no longer capable of
carrying integrity metadata out to storage. What happens to the app?
What about outstanding writes with metadata attached?

Good discussion topic for next week, methinks...

--
Martin K. Petersen Oracle Linux Engineering

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 04-05-2011, 02:09 AM
NeilBrown
 
Default Please revert a91a2785b20

On Tue, 29 Mar 2011 09:42:08 -0400 "Martin K. Petersen"
<martin.petersen@oracle.com> wrote:

> >>>>> "Mike" == Mike Snitzer <snitzer@redhat.com> writes:
>
> Mike,
>
> Mike> But I think we have a related issue that needs discussion, given
> Mike> that an integrity profile mismatch will cause MD's assemble to
> Mike> fail (rather than warn and continue to assemble without integrity
> Mike> support).
>
> Mike> DM doesn't fail to load a DM device due to a integrity profile
> Mike> mismatch; it just emits a warning and continues.
>
> Mike> In contrast, MD will now disallow adding a normal disk (without
> Mike> integrity support) to an array that has historically had a
> Mike> symmetric integrity profile across all members.
>
> You would invalidate all your existing integrity metadata, tagging,
> etc. on existing metadevice members. That seems to be a policy decision,
> so if we go down that path it would have to be keyed off a force
> assembly option passed down from userland tooling. Turning off features
> and/or losing metadata really should not be done without the user's
> explicit consent.

I've been distracted by other things for a while so I'm just looking at this
now, and I've never really paid much attention to the 'integrity' stuff
(seems all wrong to me anyway) so I might need some help coming up to
speed...

My reading of data-integrity.txt suggest that the IMD consists of two basic
components.
One is a fixed-format chunk which contains a light-weight checksum possibly
with some other summary information (address?). This is created at
whichever level doesn't trust the levels below, verified by the drive
firmware, and written to disk (together with whatever much stronger CRC the
drive firmware wants. It can then be verified on read by anyone who cares.

It seems to me that if one device in the array doesn't support this, then it
cannot be visible above the array but can still be computed and checked below
the array level, which is nearly as safe(??)
So changing an array from homogeneous to mixed just moves where the checksum
is calculated - not a big deal (maybe).

The other component of the IMD is an application tag which is not understood
or checked by the drive firmware (though presumably it is included in the
light-weight checksum so minimal checking is possible). This is used by the
filesystem to get an extra few bytes of data per-block which is known to be
updated atomically with the rest of the block.
This is currently completely unsupported by any redundant md array as there
is no attempt to copy this info when recovering to a spare etc.

So for this part of the IMD, RAID0 and LINEAR are the only levels that might
support it, and they don't have new devices added while active. LINEAR can
be extended by adding a device but that is used so rarely that I'm not really
fussed exactly how it gets handled.

So it seem to me there is no justification for disallowing the adding of a
device to an active array just because of some incompatibility with integrity
management.

Am I missing something???


Longer term - it is conceivable that RAID1/RAID10 could be taught to copy
integrity data, and we might even be able to make RAID5/6 handle it, though
the idea certainly doesn't appeal to me.

In that case we really need to know whether the sysadmin is expecting
integrity support or not. It think it is only justified to refuse an
explicit request to add a device to an array if we know it to be incompatible
with some other request.
I don't know how integrity should be requested - maybe mkfs, maybe mount,
maybe ioctl... maybe an mdadm option.
Once md finds out that it has been explicitly requested the array could be
flagged to say that integrity is in use, and then we could do all the extra
work to provide it, and reject new devices which don't support it.


Does any of that make sense? Is there something I have completely
misunderstood?

>
> Also, let's assume you run an integrity-aware app on a DM device and you
> add a non-integrity drive. The DM device is then no longer capable of
> carrying integrity metadata out to storage. What happens to the app?
> What about outstanding writes with metadata attached?
>
> Good discussion topic for next week, methinks...
>

Yeah, have fun - but remember that not all storage people are there :-)

NeilBrown

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 10:08 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org