Riches Jr, RobertX M [robertx.m.riches.jr@intel.com] wrote:
> However, I am able to reproduce the symptom using the standard dm-linear module supplied with the x86_64 kernel that comes with Ubuntu Karmic. With the attached script, the first 'od' on /dev/vg1/lv1, done before 'dmsetup remove', shows the first part of the file contains 0x00. A second 'od' after 'dmsetup remove' shows the file contains random data.
>
> I didn't see any queues or caches in dm-linear that would need to be flushed.
>
> Any other suggestions?

If you write to a block device devA and read from it on the same system,
you would get the expected data. In your case, you have devA and devB
(devB is a mapped device on devA, but that doesn't matter), you write to
devA and read from devB. You won't see what you wrote because of block
device caching. Try fsync() on the block device fred01 before copying or
try suspend on the linear dev (fred01 in your case).

Thanks, Malahal.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Alasdair G Kergon [agk@redhat.com] wrote:
> On Wed, Jun 30, 2010 at 04:53:23PM -0700, Riches Jr, RobertX M wrote:
> > I didn't see any queues or caches in dm-linear that would need to be flushed.
>
> It's your test script that is not behaving as you expect
>
> Add conv=odirect to dd

You mean flags rather than conv? conv just converts ascii to ebcdic etc.
oflag=direct would do direct I/O on the writes and iflag=direct should do
direct I/O on reads.

Thanks, Malahal.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Riches Jr, RobertX M [robertx.m.riches.jr@intel.com] wrote:
> Many thanks to those who responded with helpful suggestions. A couple of the suggestions were sufficient to make my test script behave as I expect--to liberate the data that had been stuck in the page cache twilight zone. :-)
>
> For the benefit of anyone who might come across the archives with a similar problem, here's what worked and what didn't, with all actions taken between writing and reading:
>
> - Using 'dd -iflag=direct' to read from /dev/vg1/lv1 (the underlying device) worked.
>
> - Doing 'blockdev --flushbufs /dev/vg1/lv1' (on the underlying device) worked.
>
> - Doing 'blockdev --flushbufs /dev/mapper/fred01' (on the device exposed by dm-linear or my module) did not solve the problem.

I wonder why flushing the underlying device worked but not the actual
device itself. I expected the opposite! :-(

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel

Malahal Naineni wrote:
> Riches Jr, RobertX M [robertx.m.riches.jr@intel.com] wrote:
>> Many thanks to those who responded with helpful suggestions. A couple of the suggestions were sufficient to make my test script behave as I expect--to liberate the data that had been stuck in the page cache twilight zone. :-)
>>
>> For the benefit of anyone who might come across the archives with a similar problem, here's what worked and what didn't, with all actions taken between writing and reading:
>>
>> - Using 'dd -iflag=direct' to read from /dev/vg1/lv1 (the underlying device) worked.
>>
>> - Doing 'blockdev --flushbufs /dev/vg1/lv1' (on the underlying device) worked.
>>
>> - Doing 'blockdev --flushbufs /dev/mapper/fred01' (on the device exposed by dm-linear or my module) did not solve the problem.
>
> I wonder why flushing the underlying device worked but not the actual
> device itself. I expected the opposite! :-(
>
Why, no. That's exactly the point.
The device keeps the underlying block devices pinned in the cache, so any page invalidation can only be triggered
by the device itself, not the underlying ones. Any writes to the underlying devices won't trigger a page
invalidation, so they'll stay in the cache forever.
Hence you need to flush them explicitely.

Not that I'll recommend that. It'll play silly buggers with the page cache.

Cheers,

Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel