Fix a crash when block device is read and block size is changed at the same time
Mikulas Patocka <firstname.lastname@example.org> writes:
> On Tue, 25 Sep 2012, Jeff Moyer wrote:
>> Jeff Moyer <email@example.com> writes:
>> > Mikulas Patocka <firstname.lastname@example.org> writes:
>> >> Hi Jeff
>> >> Thanks for testing.
>> >> It would be interesting ... what happens if you take the patch 3, leave
>> >> "struct percpu_rw_semaphore bd_block_size_semaphore" in "struct
>> >> block_device", but remove any use of the semaphore from fs/block_dev.c? -
>> >> will the performance be like unpatched kernel or like patch 3? It could be
>> >> that the change in the alignment affects performance on your CPU too, just
>> >> differently than on my CPU.
>> > It turns out to be exactly the same performance as with the 3rd patch
>> > applied, so I guess it does have something to do with cache alignment.
>> > Here is the patch (against vanilla) I ended up testing. Let me know if
>> > I've botched it somehow.
>> > So, I next up I'll play similar tricks to what you did (padding struct
>> > block_device in all kernels) to eliminate the differences due to
>> > structure alignment and provide a clear picture of what the locking
>> > effects are.
>> After trying again with the same padding you used in the struct
>> bdev_inode, I see no performance differences between any of the
>> patches. I tried bumping up the number of threads to saturate the
>> number of cpus on a single NUMA node on my hardware, but that resulted
>> in lower IOPS to the device, and hence consumption of less CPU time.
>> So, I believe my results to be inconclusive.
> For me, the fourth patch with RCU-based locks performed better, so I am
> submitting that.
>> After talking with Vivek about the problem, he had mentioned that it
>> might be worth investigating whether bd_block_size could be protected
>> using SRCU. I looked into it, and the one thing I couldn't reconcile is
>> updating both the bd_block_size and the inode->i_blkbits at the same
>> time. It would involve (afaiui) adding fields to both the inode and the
>> block_device data structures and using rcu_assign_pointer and
>> rcu_dereference to modify and access the fields, and both fields would
>> need to protected by the same struct srcu_struct. I'm not sure whether
>> that's a desirable approach. When I started to implement it, it got
>> ugly pretty quickly. What do others think?
> Using RCU doesn't seem sensible to me (except for lock implementation, as
> it is in patch 4). The major problem is that the block layer reads
> blocksize multiple times and when different values are read, a crash may
> happen - RCU doesn't protect you against that - if you read a variable
> multiple times in a RCU-protected section, you can still get different
SRCU is sleepable, so could be (I think) used in the same manner as your
rw semaphore. The only difference is that it would require changing the
bd_blocksize and the i_blkbits to pointers and protecting them both with
the same srcu struct. Then, the inode i_blkbits would also need to be
special cased, so that we only require such handling when it is
associated with a block device. It got messy.
> If we wanted to use RCU, we would have to read blocksize just once and
> pass the value between all functions involved - that would result in a
> massive code change.
If we did that, we wouldn't need rcu at all, would we?
dm-devel mailing list