FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 09-01-2010, 10:31 AM
Mikulas Patocka
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

On Mon, 30 Aug 2010, Mike Snitzer wrote:

> Hi,
>
> On Mon, Aug 30 2010 at 11:45am -0400,
> Tejun Heo <tj@kernel.org> wrote:
>
> > Here's a version w/ BUG_ON() added. Once the queue hang issue is
> > tracked down, I'll refresh the whole series and repost.
>
> When you next send out the refreshed series, could you cc dm-devel any
> patches that touch DM directly or block patches that are required to
> support DM?
>
> It'd be great to cc Mikulas Patocka on those patches too (I cc'd him).
> Mikulas will be reviewing these DM patches now that we have something
> that works with your larger FLUSH+FUA patchset.
>
> Thanks,
> Mike

My recommended approach to this (on non-request-based dm) is to simply let
the current barrier infrastructure be as it is --- you don't need to
change it now, you can simply map FUA write to barrier write and FLUSH to
zero-data barrier --- and it won't cause any data corruption. It will just
force unneeded I/O queue draining.

Once FLUSH+FUA interface is finalized and committed upstream, we can
remove that I/O queue draining from dm to improve performance.

Mikulas

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-01-2010, 11:20 AM
Tejun Heo
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

Hello,

On 09/01/2010 12:31 PM, Mikulas Patocka wrote:
> My recommended approach to this (on non-request-based dm) is to simply let
> the current barrier infrastructure be as it is --- you don't need to
> change it now, you can simply map FUA write to barrier write and FLUSH to
> zero-data barrier --- and it won't cause any data corruption. It will just
> force unneeded I/O queue draining.
>
> Once FLUSH+FUA interface is finalized and committed upstream, we can
> remove that I/O queue draining from dm to improve performance.

Unfortunately, it doesn't work that way. The current dm
implementation depends on block layer holding the queue while a
barrier sequence is in progress which the new implementation doesn't
do anymore (the whole point of this conversion BTW).

Thanks.

--
tejun

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-01-2010, 12:12 PM
Mikulas Patocka
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

On Wed, 1 Sep 2010, Tejun Heo wrote:

> Hello,
>
> On 09/01/2010 12:31 PM, Mikulas Patocka wrote:
> > My recommended approach to this (on non-request-based dm) is to simply let
> > the current barrier infrastructure be as it is --- you don't need to
> > change it now, you can simply map FUA write to barrier write and FLUSH to
> > zero-data barrier --- and it won't cause any data corruption. It will just
> > force unneeded I/O queue draining.
> >
> > Once FLUSH+FUA interface is finalized and committed upstream, we can
> > remove that I/O queue draining from dm to improve performance.
>
> Unfortunately, it doesn't work that way. The current dm
> implementation depends on block layer holding the queue while a
> barrier sequence is in progress which the new implementation doesn't
> do anymore (the whole point of this conversion BTW).

That may be true for request-based dm (I don't know).

But bio-based dm doesn't depend on it, I wrote it and I didn't rely on
that.

Mikulas

> Thanks.
>
> --
> tejun
>

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-01-2010, 12:42 PM
Tejun Heo
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

Hello,

On 09/01/2010 02:12 PM, Mikulas Patocka wrote:
> That may be true for request-based dm (I don't know).

Oh, okay, this part of thread was for request based dm, so I assumed
you were talking about it.

> But bio-based dm doesn't depend on it, I wrote it and I didn't rely on
> that.

If you look at the two patches for bio-based ones. The first one is
basically what you're talking about w/ s/barrier/flush/ renames and
dropping of -EOPNOTSUPP. It doesn't really change the mechanism much.
If you don't feel comfortable about the second one, we sure can
postpone it but it's still quite away from the next merge window and
what would be the point of delaying it?

Thanks.

--
tejun

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-01-2010, 12:54 PM
Mike Snitzer
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

On Wed, Sep 01 2010 at 8:42am -0400,
Tejun Heo <tj@kernel.org> wrote:

> Hello,
>
> On 09/01/2010 02:12 PM, Mikulas Patocka wrote:
> > That may be true for request-based dm (I don't know).
>
> Oh, okay, this part of thread was for request based dm, so I assumed
> you were talking about it.
>
> > But bio-based dm doesn't depend on it, I wrote it and I didn't rely on
> > that.
>
> If you look at the two patches for bio-based ones. The first one is
> basically what you're talking about w/ s/barrier/flush/ renames and
> dropping of -EOPNOTSUPP. It doesn't really change the mechanism much.
> If you don't feel comfortable about the second one, we sure can
> postpone it but it's still quite away from the next merge window and
> what would be the point of delaying it?

Right, we have a window of opportunity to sort this out now. No sense
in wasting it.

Mike

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-01-2010, 03:20 PM
Mike Snitzer
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

On Wed, Sep 01 2010 at 8:12am -0400,
Mikulas Patocka <mpatocka@redhat.com> wrote:

> On Wed, 1 Sep 2010, Tejun Heo wrote:
>
> > Hello,
> >
> > On 09/01/2010 12:31 PM, Mikulas Patocka wrote:
> > > My recommended approach to this (on non-request-based dm) is to simply let
> > > the current barrier infrastructure be as it is --- you don't need to
> > > change it now, you can simply map FUA write to barrier write and FLUSH to
> > > zero-data barrier --- and it won't cause any data corruption. It will just
> > > force unneeded I/O queue draining.
> > >
> > > Once FLUSH+FUA interface is finalized and committed upstream, we can
> > > remove that I/O queue draining from dm to improve performance.
> >
> > Unfortunately, it doesn't work that way. The current dm
> > implementation depends on block layer holding the queue while a
> > barrier sequence is in progress which the new implementation doesn't
> > do anymore (the whole point of this conversion BTW).
>
> That may be true for request-based dm (I don't know).
>
> But bio-based dm doesn't depend on it, I wrote it and I didn't rely on
> that.

Mikulas,

Current bio-based barrier support also defers IO if a flush is in
progress. See _dm_request:

/*
* If we're suspended or the thread is processing barriers
* we have to queue this io for later.
*/

Tejun also shared the following:

"bio based implementation also uses dm_wait_for_completion() and
DMF_QUEUE_IO_TO_THREAD to plug all the follow up bio's while flush is in
progress, which sucks for throughput but successfully avoids starvation."

here:
https://www.redhat.com/archives/dm-devel/2010-August/msg00174.html

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-01-2010, 03:35 PM
Mikulas Patocka
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

On Wed, 1 Sep 2010, Mike Snitzer wrote:

> On Wed, Sep 01 2010 at 8:12am -0400,
> Mikulas Patocka <mpatocka@redhat.com> wrote:
>
> > On Wed, 1 Sep 2010, Tejun Heo wrote:
> >
> > > Hello,
> > >
> > > On 09/01/2010 12:31 PM, Mikulas Patocka wrote:
> > > > My recommended approach to this (on non-request-based dm) is to simply let
> > > > the current barrier infrastructure be as it is --- you don't need to
> > > > change it now, you can simply map FUA write to barrier write and FLUSH to
> > > > zero-data barrier --- and it won't cause any data corruption. It will just
> > > > force unneeded I/O queue draining.
> > > >
> > > > Once FLUSH+FUA interface is finalized and committed upstream, we can
> > > > remove that I/O queue draining from dm to improve performance.
> > >
> > > Unfortunately, it doesn't work that way. The current dm
> > > implementation depends on block layer holding the queue while a
> > > barrier sequence is in progress which the new implementation doesn't
> > > do anymore (the whole point of this conversion BTW).
> >
> > That may be true for request-based dm (I don't know).
> >
> > But bio-based dm doesn't depend on it, I wrote it and I didn't rely on
> > that.
>
> Mikulas,
>
> Current bio-based barrier support also defers IO if a flush is in
> progress. See _dm_request:

I know. But it doesn't hurt with flush/fua requests. It just lowers
performance (it defers i/os when it doesn't have to) but doesn't damage
data.

So I think that we can let it be this way until flush/fua patch is
finalized.

Mikulas

> /*
> * If we're suspended or the thread is processing barriers
> * we have to queue this io for later.
> */
>
> Tejun also shared the following:
>
> "bio based implementation also uses dm_wait_for_completion() and
> DMF_QUEUE_IO_TO_THREAD to plug all the follow up bio's while flush is in
> progress, which sucks for throughput but successfully avoids starvation."
>
> here:
> https://www.redhat.com/archives/dm-devel/2010-August/msg00174.html
>

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-01-2010, 05:07 PM
Mike Snitzer
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

On Wed, Sep 01 2010 at 11:35am -0400,
Mikulas Patocka <mpatocka@redhat.com> wrote:

>
>
> On Wed, 1 Sep 2010, Mike Snitzer wrote:
>
> > On Wed, Sep 01 2010 at 8:12am -0400,
> > Mikulas Patocka <mpatocka@redhat.com> wrote:
> >
> > > On Wed, 1 Sep 2010, Tejun Heo wrote:
> > >
> > > > Hello,
> > > >
> > > > On 09/01/2010 12:31 PM, Mikulas Patocka wrote:
> > > > > My recommended approach to this (on non-request-based dm) is to simply let
> > > > > the current barrier infrastructure be as it is --- you don't need to
> > > > > change it now, you can simply map FUA write to barrier write and FLUSH to
> > > > > zero-data barrier --- and it won't cause any data corruption. It will just
> > > > > force unneeded I/O queue draining.
> > > > >
> > > > > Once FLUSH+FUA interface is finalized and committed upstream, we can
> > > > > remove that I/O queue draining from dm to improve performance.
> > > >
> > > > Unfortunately, it doesn't work that way. The current dm
> > > > implementation depends on block layer holding the queue while a
> > > > barrier sequence is in progress which the new implementation doesn't
> > > > do anymore (the whole point of this conversion BTW).
> > >
> > > That may be true for request-based dm (I don't know).
> > >
> > > But bio-based dm doesn't depend on it, I wrote it and I didn't rely on
> > > that.
> >
> > Mikulas,
> >
> > Current bio-based barrier support also defers IO if a flush is in
> > progress. See _dm_request:
>
> I know. But it doesn't hurt with flush/fua requests. It just lowers
> performance (it defers i/os when it doesn't have to) but doesn't damage
> data.
>
> So I think that we can let it be this way until flush/fua patch is
> finalized.

Neither Tejun nor I see the point in waiting when we have a window of
time to address the issues now. We want DM to realize the benefit
associated with the kernel-wide FLUSH+FUA conversion too.

You're not explaining your reluctance to tackle review of this DM
FLUSH+FUA conversion. Converting DM can (and already has) exposed some
shortcomings in the the kernel-wide FLUSH+FUA conversion.

We can't afford for these kernel-wide changes to go in and have DM left
trying to fix some fundamental kernel issue outside of DM after the
fact.

So why wait? This debate has distracted you from just reviewing the
code. The bio-based DM changes are fairly straight-forward.

Mike

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-01-2010, 06:59 PM
Mike Snitzer
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

On Wed, Sep 01 2010 at 1:07pm -0400,
Mike Snitzer <snitzer@redhat.com> wrote:

> On Wed, Sep 01 2010 at 11:35am -0400,
> Mikulas Patocka <mpatocka@redhat.com> wrote:
>
> >
> > On Wed, 1 Sep 2010, Mike Snitzer wrote:
> > >
> > > Mikulas,
> > >
> > > Current bio-based barrier support also defers IO if a flush is in
> > > progress. See _dm_request:
> >
> > I know. But it doesn't hurt with flush/fua requests. It just lowers
> > performance (it defers i/os when it doesn't have to) but doesn't damage
> > data.
> >
> > So I think that we can let it be this way until flush/fua patch is
> > finalized.
>
> Neither Tejun nor I see the point in waiting when we have a window of
> time to address the issues now. We want DM to realize the benefit
> associated with the kernel-wide FLUSH+FUA conversion too.

But we can meet in the middle. I've reordered the DM FLUSH+FUA patches
so that the more intrusive bio-based relaxed ordering patch is at the
very end.

My hope was that the request-based deadlock I'm seeing would disappear
if that relaxed ordering patch wasn't applied. Unfortunately, I still
see the hang.

Anyway, I've made the patches available here:
http://people.redhat.com/msnitzer/patches/flush-fua/

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-02-2010, 03:22 AM
Mike Snitzer
 
Default dm: implement REQ_FLUSH/FUA support for request-based dm

On Wed, Sep 01 2010 at 2:59pm -0400,
Mike Snitzer <snitzer@redhat.com> wrote:

> But we can meet in the middle. I've reordered the DM FLUSH+FUA patches
> so that the more intrusive bio-based relaxed ordering patch is at the
> very end.
>
> My hope was that the request-based deadlock I'm seeing would disappear
> if that relaxed ordering patch wasn't applied. Unfortunately, I still
> see the hang.

Turns out I can reproduce the hang on a stock 2.6.36-rc3 (without _any_
FLUSH+FUA patches)!

I'll try to pin-point the root cause but I think my test is somehow
exposing a bug in my virt setup.

So this hang is definitely starting to look like a red herring.

Tejun,

This news should clear the way for you to re-post your patches. I think
it would be best if you reordered the DM patches like I did here in this
series: http://people.redhat.com/msnitzer/patches/flush-fua/series

In particular, the dm-relax-ordering-of-bio-based-flush-implementation
patch should go at the end. I think it makes for a more logical
evolution of the DM code.

Thanks,
Mike

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 03:28 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org