FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor


 
 
LinkBack Thread Tools
 
Old 03-23-2012, 01:12 PM
"Kasatkin, Dmitry"
 
Default dm-bufio

On Fri, Mar 23, 2012 at 1:26 PM, Kasatkin, Dmitry
<dmitry.kasatkin@intel.com> wrote:
> On Fri, Mar 23, 2012 at 1:10 PM, Zdenek Kabelac <zkabelac@redhat.com> wrote:
>> Dne 23.3.2012 12:01, Kasatkin, Dmitry napsal(a):
>>> Hello,
>>>
>>> When using dm-bufio and dm-io in general, how to ensure that all dirty
>>> buffers are written to the storage when machine reboots?
>>> suspend hooks could be used, but they are not called on reboot, only
>>> when suspending/removing the target...
>>>
>>
>> You mean you reboot without running *'sync' command?
>>
>> And yes - on reboot you should properly unmount devices - so you should
>> see removal of target on your shutdown sequence - *I believe Fedora currently
>> tries to support switch to some shutdown ramdisk, so all filesystem and
>> devices might be properly unmounted and destroyed.
>>
>
> Hello,
>
> Thanks for response.
> I use bufio to store some data on block device.
> It is not mounted in anyway. My target just use it to load/store data.
> When machine reboots, I want to be sure that bufio written all dirty buffers...
>
> - Dmitry
>

At the moment, I have reboot notifier which does the following

dm_bufio_write_dirty_buffers(d->bufio);
sync_blockdev(d->dev->bdev);
blkdev_issue_flush(d->dev->bdev, GFP_KERNEL, NULL);

without first line on the next boot I got corrupted/not updated blocks.
and I am not sure if I need last 2 lines...

- Dmitry

>
>> Zdenek
>>
>>

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-23-2012, 01:21 PM
Mike Snitzer
 
Default dm-bufio

On Fri, Mar 23 2012 at 10:12am -0400,
Kasatkin, Dmitry <dmitry.kasatkin@intel.com> wrote:

> On Fri, Mar 23, 2012 at 1:26 PM, Kasatkin, Dmitry
> <dmitry.kasatkin@intel.com> wrote:
> > On Fri, Mar 23, 2012 at 1:10 PM, Zdenek Kabelac <zkabelac@redhat.com> wrote:
> >> Dne 23.3.2012 12:01, Kasatkin, Dmitry napsal(a):
> >>> Hello,
> >>>
> >>> When using dm-bufio and dm-io in general, how to ensure that all dirty
> >>> buffers are written to the storage when machine reboots?
> >>> suspend hooks could be used, but they are not called on reboot, only
> >>> when suspending/removing the target...
> >>>
> >>
> >> You mean you reboot without running *'sync' command?
> >>
> >> And yes - on reboot you should properly unmount devices - so you should
> >> see removal of target on your shutdown sequence - *I believe Fedora currently
> >> tries to support switch to some shutdown ramdisk, so all filesystem and
> >> devices might be properly unmounted and destroyed.
> >>
> >
> > Hello,
> >
> > Thanks for response.
> > I use bufio to store some data on block device.
> > It is not mounted in anyway. My target just use it to load/store data.
> > When machine reboots, I want to be sure that bufio written all dirty buffers...
> >
> > - Dmitry
> >
>
> At the moment, I have reboot notifier which does the following
>
> dm_bufio_write_dirty_buffers(d->bufio);
> sync_blockdev(d->dev->bdev);
> blkdev_issue_flush(d->dev->bdev, GFP_KERNEL, NULL);
>
> without first line on the next boot I got corrupted/not updated blocks.
> and I am not sure if I need last 2 lines...

Are you cleanly removing the target from the kernel before reboot
(e.g. dmsetup remove devname)?

As long as your target's .dtr is making sure to flush all outstanding IO
(like your reboot notifier does) you should be fine.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-23-2012, 01:29 PM
"Kasatkin, Dmitry"
 
Default dm-bufio

On Fri, Mar 23, 2012 at 4:21 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Fri, Mar 23 2012 at 10:12am -0400,
> Kasatkin, Dmitry <dmitry.kasatkin@intel.com> wrote:
>
>> On Fri, Mar 23, 2012 at 1:26 PM, Kasatkin, Dmitry
>> <dmitry.kasatkin@intel.com> wrote:
>> > On Fri, Mar 23, 2012 at 1:10 PM, Zdenek Kabelac <zkabelac@redhat.com> wrote:
>> >> Dne 23.3.2012 12:01, Kasatkin, Dmitry napsal(a):
>> >>> Hello,
>> >>>
>> >>> When using dm-bufio and dm-io in general, how to ensure that all dirty
>> >>> buffers are written to the storage when machine reboots?
>> >>> suspend hooks could be used, but they are not called on reboot, only
>> >>> when suspending/removing the target...
>> >>>
>> >>
>> >> You mean you reboot without running *'sync' command?
>> >>
>> >> And yes - on reboot you should properly unmount devices - so you should
>> >> see removal of target on your shutdown sequence - *I believe Fedora currently
>> >> tries to support switch to some shutdown ramdisk, so all filesystem and
>> >> devices might be properly unmounted and destroyed.
>> >>
>> >
>> > Hello,
>> >
>> > Thanks for response.
>> > I use bufio to store some data on block device.
>> > It is not mounted in anyway. My target just use it to load/store data.
>> > When machine reboots, I want to be sure that bufio written all dirty buffers...
>> >
>> > - Dmitry
>> >
>>
>> At the moment, I have reboot notifier which does the following
>>
>> * * * dm_bufio_write_dirty_buffers(d->bufio);
>> * * * sync_blockdev(d->dev->bdev);
>> * * * blkdev_issue_flush(d->dev->bdev, GFP_KERNEL, NULL);
>>
>> without first line on the next boot I got corrupted/not updated blocks.
>> and I am not sure if I need last 2 lines...
>
> Are you cleanly removing the target from the kernel before reboot
> (e.g. dmsetup remove devname)?
>
> As long as your target's .dtr is making sure to flush all outstanding IO
> (like your reboot notifier does) you should be fine.
>

The target contains rootfs... On reboot, it is remounted read-only.
I cannot remove it...

Sometime ago I had "message" operation "sync", to sync backing devices.
But reboot notifier looks nice... It is automatically called.

> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-23-2012, 01:32 PM
Mike Snitzer
 
Default dm-bufio

On Fri, Mar 23 2012 at 10:29am -0400,
Kasatkin, Dmitry <dmitry.kasatkin@intel.com> wrote:

> On Fri, Mar 23, 2012 at 4:21 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> > On Fri, Mar 23 2012 at 10:12am -0400,
> > Kasatkin, Dmitry <dmitry.kasatkin@intel.com> wrote:
> >
> >> On Fri, Mar 23, 2012 at 1:26 PM, Kasatkin, Dmitry
> >> <dmitry.kasatkin@intel.com> wrote:
> >> > On Fri, Mar 23, 2012 at 1:10 PM, Zdenek Kabelac <zkabelac@redhat.com> wrote:
> >> >> Dne 23.3.2012 12:01, Kasatkin, Dmitry napsal(a):
> >> >>> Hello,
> >> >>>
> >> >>> When using dm-bufio and dm-io in general, how to ensure that all dirty
> >> >>> buffers are written to the storage when machine reboots?
> >> >>> suspend hooks could be used, but they are not called on reboot, only
> >> >>> when suspending/removing the target...
> >> >>>
> >> >>
> >> >> You mean you reboot without running *'sync' command?
> >> >>
> >> >> And yes - on reboot you should properly unmount devices - so you should
> >> >> see removal of target on your shutdown sequence - *I believe Fedora currently
> >> >> tries to support switch to some shutdown ramdisk, so all filesystem and
> >> >> devices might be properly unmounted and destroyed.
> >> >>
> >> >
> >> > Hello,
> >> >
> >> > Thanks for response.
> >> > I use bufio to store some data on block device.
> >> > It is not mounted in anyway. My target just use it to load/store data.
> >> > When machine reboots, I want to be sure that bufio written all dirty buffers...
> >> >
> >> > - Dmitry
> >> >
> >>
> >> At the moment, I have reboot notifier which does the following
> >>
> >> * * * dm_bufio_write_dirty_buffers(d->bufio);
> >> * * * sync_blockdev(d->dev->bdev);
> >> * * * blkdev_issue_flush(d->dev->bdev, GFP_KERNEL, NULL);
> >>
> >> without first line on the next boot I got corrupted/not updated blocks.
> >> and I am not sure if I need last 2 lines...
> >
> > Are you cleanly removing the target from the kernel before reboot
> > (e.g. dmsetup remove devname)?
> >
> > As long as your target's .dtr is making sure to flush all outstanding IO
> > (like your reboot notifier does) you should be fine.
> >
>
> The target contains rootfs... On reboot, it is remounted read-only.
> I cannot remove it...
>
> Sometime ago I had "message" operation "sync", to sync backing devices.
> But reboot notifier looks nice... It is automatically called.

OK.

As an aside, just curious: what does your target do?

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-23-2012, 02:17 PM
"Kasatkin, Dmitry"
 
Default dm-bufio

On Fri, Mar 23, 2012 at 4:32 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Fri, Mar 23 2012 at 10:29am -0400,
> Kasatkin, Dmitry <dmitry.kasatkin@intel.com> wrote:
>
>> On Fri, Mar 23, 2012 at 4:21 PM, Mike Snitzer <snitzer@redhat.com> wrote:
>> > On Fri, Mar 23 2012 at 10:12am -0400,
>> > Kasatkin, Dmitry <dmitry.kasatkin@intel.com> wrote:
>> >
>> >> On Fri, Mar 23, 2012 at 1:26 PM, Kasatkin, Dmitry
>> >> <dmitry.kasatkin@intel.com> wrote:
>> >> > On Fri, Mar 23, 2012 at 1:10 PM, Zdenek Kabelac <zkabelac@redhat.com> wrote:
>> >> >> Dne 23.3.2012 12:01, Kasatkin, Dmitry napsal(a):
>> >> >>> Hello,
>> >> >>>
>> >> >>> When using dm-bufio and dm-io in general, how to ensure that all dirty
>> >> >>> buffers are written to the storage when machine reboots?
>> >> >>> suspend hooks could be used, but they are not called on reboot, only
>> >> >>> when suspending/removing the target...
>> >> >>>
>> >> >>
>> >> >> You mean you reboot without running *'sync' command?
>> >> >>
>> >> >> And yes - on reboot you should properly unmount devices - so you should
>> >> >> see removal of target on your shutdown sequence - *I believe Fedora currently
>> >> >> tries to support switch to some shutdown ramdisk, so all filesystem and
>> >> >> devices might be properly unmounted and destroyed.
>> >> >>
>> >> >
>> >> > Hello,
>> >> >
>> >> > Thanks for response.
>> >> > I use bufio to store some data on block device.
>> >> > It is not mounted in anyway. My target just use it to load/store data.
>> >> > When machine reboots, I want to be sure that bufio written all dirty buffers...
>> >> >
>> >> > - Dmitry
>> >> >
>> >>
>> >> At the moment, I have reboot notifier which does the following
>> >>
>> >> * * * dm_bufio_write_dirty_buffers(d->bufio);
>> >> * * * sync_blockdev(d->dev->bdev);
>> >> * * * blkdev_issue_flush(d->dev->bdev, GFP_KERNEL, NULL);
>> >>
>> >> without first line on the next boot I got corrupted/not updated blocks.
>> >> and I am not sure if I need last 2 lines...
>> >
>> > Are you cleanly removing the target from the kernel before reboot
>> > (e.g. dmsetup remove devname)?
>> >
>> > As long as your target's .dtr is making sure to flush all outstanding IO
>> > (like your reboot notifier does) you should be fine.
>> >
>>
>> The target contains rootfs... On reboot, it is remounted read-only.
>> I cannot remove it...
>>
>> Sometime ago I had "message" operation "sync", to sync backing devices.
>> But reboot notifier looks nice... It is automatically called.
>
> OK.
>
> As an aside, just curious: what does your target do?

I have experimented with integrity protection.

Thanks.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-23-2012, 03:22 PM
Mikulas Patocka
 
Default dm-bufio

On Fri, 23 Mar 2012, Kasatkin, Dmitry wrote:

> When using dm-bufio and dm-io in general, how to ensure that all dirty

Regarding dm-io, you don't have to flush dirty buffers because dm-io does
no caching. You only need to flush hardware disk cache with
blkdev_issue_flush.

> At the moment, I have reboot notifier which does the following
>
> dm_bufio_write_dirty_buffers(d->bufio);
> sync_blockdev(d->dev->bdev);
> blkdev_issue_flush(d->dev->bdev, GFP_KERNEL, NULL);
>
> without first line on the next boot I got corrupted/not updated blocks.
> and I am not sure if I need last 2 lines...

You can drop sync_blockdev(d->dev->bdev) --- sync_blockdev flushes kernel
buffer cache and dm-bufio doesn't use the kernel buffer cache (you only
need sync_blockdev if you use kernel buffer cache on this device for
something else). If you drop sync_blockdev, you can also drop
blkdev_issue_flush, because dm_bufio_write_dirty_buffers already issues
the flush request.

BTW. how are you going to deal with kernel crashes or power faults?

It is much better to use journaling, phase tree, crash counts or other
method to maintain metadata integrity insted of reboot notifier --- these
methods maintain integrity even in case of unexpected crash.

Mikulas

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-23-2012, 08:32 PM
Mike Snitzer
 
Default dm-bufio

On Fri, Mar 23 2012 at 11:17am -0400,
Kasatkin, Dmitry <dmitry.kasatkin@intel.com> wrote:

> On Fri, Mar 23, 2012 at 4:32 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> > As an aside, just curious: what does your target do?
>
> I have experimented with integrity protection.

Interesting. Are you aware that a new integrity target has been
developed and is approaching upstream inclussion? It is called
dm-verity.

Mikulas posted the most recent version here:
http://www.redhat.com/archives/dm-devel/2012-March/msg00133.html

And here is the userspace setup code:
http://www.redhat.com/archives/dm-devel/2012-March/msg00134.html

The original docs from Google's initial dm-verity are mostly applicable
(but need to be refreshed and included with the new version):
http://www.redhat.com/archives/dm-devel/2012-February/msg00119.html

dm-verity already has 2 formats, a 3rd could be added if you found the
existing formats somehow lacking, see:
http://www.redhat.com/archives/dm-devel/2012-March/msg00143.html

Mike

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-23-2012, 10:58 PM
"Kasatkin, Dmitry"
 
Default dm-bufio

On Fri, Mar 23, 2012 at 6:22 PM, Mikulas Patocka <mpatocka@redhat.com> wrote:
>
>
> On Fri, 23 Mar 2012, Kasatkin, Dmitry wrote:
>
>> When using dm-bufio and dm-io in general, how to ensure that all dirty
>
> Regarding dm-io, you don't have to flush dirty buffers because dm-io does
> no caching. You only need to flush hardware disk cache with
> blkdev_issue_flush.
>
>> At the moment, I have reboot notifier which does the following
>>
>> * * * dm_bufio_write_dirty_buffers(d->bufio);
>> * * * sync_blockdev(d->dev->bdev);
>> * * * blkdev_issue_flush(d->dev->bdev, GFP_KERNEL, NULL);
>>
>> without first line on the next boot I got corrupted/not updated blocks.
>> and I am not sure if I need last 2 lines...
>
> You can drop sync_blockdev(d->dev->bdev) --- sync_blockdev flushes kernel
> buffer cache and dm-bufio doesn't use the kernel buffer cache (you only
> need sync_blockdev if you use kernel buffer cache on this device for
> something else). If you drop sync_blockdev, you can also drop
> blkdev_issue_flush, because dm_bufio_write_dirty_buffers already issues
> the flush request.
>
> BTW. how are you going to deal with kernel crashes or power faults?
>
> It is much better to use journaling, phase tree, crash counts or other
> method to maintain metadata integrity insted of reboot notifier --- these
> methods maintain integrity even in case of unexpected crash.
>
> Mikulas
>

Hi,

Thanks for clarification.
Indeed everything works just with dm_bufio_write_dirty_buffers().
Reboot notifier is to issue the flush only..
As I understand, dm-bufio will do the flush but currently once per 10 seconds.

if data on the block device and metadata on other block device get out
of sync, what you can do then?
how journal helps then?

- Dmitry

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-23-2012, 11:04 PM
"Kasatkin, Dmitry"
 
Default dm-bufio

On Fri, Mar 23, 2012 at 11:32 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Fri, Mar 23 2012 at 11:17am -0400,
> Kasatkin, Dmitry <dmitry.kasatkin@intel.com> wrote:
>
>> On Fri, Mar 23, 2012 at 4:32 PM, Mike Snitzer <snitzer@redhat.com> wrote:
>> > As an aside, just curious: what does your target do?
>>
>> I have experimented with integrity protection.
>
> Interesting. *Are you aware that a new integrity target has been
> developed and is approaching upstream inclussion? *It is called
> dm-verity.
>
> Mikulas posted the most recent version here:
> http://www.redhat.com/archives/dm-devel/2012-March/msg00133.html
>
> And here is the userspace setup code:
> http://www.redhat.com/archives/dm-devel/2012-March/msg00134.html
>
> The original docs from Google's initial dm-verity are mostly applicable
> (but need to be refreshed and included with the new version):
> http://www.redhat.com/archives/dm-devel/2012-February/msg00119.html
>
> dm-verity already has 2 formats, a 3rd could be added if you found the
> existing formats somehow lacking, see:
> http://www.redhat.com/archives/dm-devel/2012-March/msg00143.html
>
> Mike

Hi,

Yes.. I am aware of dm-verity target.
It suites well for read-only cases.
It is questionable how tree-based approach will work with read-write.
Each single update will cause whole tree recalculation.

- Dmitry

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 03-24-2012, 12:07 AM
Mikulas Patocka
 
Default dm-bufio

On Sat, 24 Mar 2012, Kasatkin, Dmitry wrote:

> Hi,
>
> Thanks for clarification.
> Indeed everything works just with dm_bufio_write_dirty_buffers().
> Reboot notifier is to issue the flush only..
> As I understand, dm-bufio will do the flush but currently once per 10 seconds.
>
> if data on the block device and metadata on other block device get out
> of sync, what you can do then?
> how journal helps then?
>
> - Dmitry

It depends what you're trying to do.

If you're trying to do something like "dm-verity", but with a possibility
to write to the device, there are several possibilities:

* keep two checksums per 512-byte sector, the old checksum and the new
checksum. If you update the block, you update the new checksum, sync the
metadata device and then write to the data device (obviously you need to
batch this update-sync-write for many blocks write concurrently to get
decent performance). When you verify block, you allow either checksum to
match. When you sync caches on the data device, you must forget all the
old checksums.

* use journaling, write data block and its checksum to a journal. If the
computer crashes, you just replay the journal (so you don't care what data
was present at that place, you overwrite it with data from the journal).
The downside is that this doubles required i/o throughput, you should have
journal and data on different devices.

* do nothing and rebuild the checksums in case of crash. It is simplest,
but it doesn't protect from data damages that happen due to the crash (for
example, some SSDs corrupt its metadata on power failure and you can't
detect this if you rebuild checksums after a power failure).

> Yes.. I am aware of dm-verity target.
> It suites well for read-only cases.
> It is questionable how tree-based approach will work with read-write.
> Each single update will cause whole tree recalculation.

A write would recalculate hashes only in the branch from tree bottom to
tree top. The obvious downside is that there is no protection from crash.


BTW. regarding that reboot notifier with
"dm_bufio_write_dirty_buffers(d->bufio)" ... there could be another
problem ... what if other reboot notifier (maybe for a completely
different driver) writes to the device after
"dm_bufio_write_dirty_buffers(d->bufio)" was performed?

- would it be possible to install your notifier again?

- or turn into a synchronous updates? --- i.e. set a flag in your reboot
notifier and if the flag is on, call
"dm_bufio_write_dirty_buffers(d->bufio)" after every write.

Mikulas

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 09:20 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org