Block cleanups
Incorporated various review feedback, and I reordered/reworked the patches to
make them more coherent. This also has the latest version of the closure code, incorporating review feedback from Joe and Tejun. I may rewrite the bio splitting stuff to not depend on it so it can go in sooner (Tejun is not a fan of closures), but IMO the patch is better for using them. First 5 patches: Kill bi_destructor Next 3 patches: Better bio splitting Next 3 patches: Rework bio cloning to not clone entire bio vec Tejun also pointed out that with the bio splitting improvements we ought to be able to get rid of merge_bvec_fn, so I started on that. Those patches are definitely a WIP, though. For most of the md code, it looks like things will almost just work - they already had bio_pair_split() calls to handle single page bios that crossed bvec boundries, but bio_pair_split() now splits arbitrary bios so things should just work. (I just thought of one problem - previously, in practice you'd never need to split a bio more than once, but that's no longer true - so the bio_pair_split() calls do need to be stuck in a loop, and the as is the patches aren't correct.) For raid5 though I couldn't find any splitting in the existing code - it looks like it's depending on merge_bvec_fn to make sure bios never need to be split. Adding bio splitting looks to be not quite trivial - Neil, any thoughts? For dm, it was already splitting multi page bios when necessary, it looked like its merge_bvec_fn was mostly for propagating device limits - which my generic_make_request() patch handles. Any dm folks want to take a look? There are only 4 merge_bvec_fns left in my kernel: drivers/md/raid5.c - raid5_mergeable_bvec() drivers/block/drbd/drbd_req.c - drbd_merge_bvec() drivers/block/pktcdvd.c - pkt_merge_bvec() drivers/block/rbd.c - rbd_merge_bvec() I haven't looked at the last three, but if the maintainers want to help out a bit it should be pretty easy to get rid of merge_bvec_fn entirely. Kent Overstreet (16): block: Generalized bio pool freeing dm: Use bioset's front_pad for dm_rq_clone_bio_info block: Add bio_reset() pktcdvd: Switch to bio_kmalloc() block: Kill bi_destructor block: Add an explicit bio flag for bios that own their bvec block: Rename bio_split() -> bio_pair_split() block: Rework bio splitting block: Add bio_clone_kmalloc() block: Add bio_clone_bioset() block: Only clone bio vecs that are in use Closures Make generic_make_request handle arbitrarily large bios Gut bio_add_page() md: Kill merge_bvec_fn()s dm: Kill merge_bvec_fn() Documentation/block/biodoc.txt | 5 - block/blk-core.c | 126 ++++++- drivers/block/drbd/drbd_req.c | 18 +- drivers/block/osdblk.c | 3 +- drivers/block/pktcdvd.c | 121 +++---- drivers/block/rbd.c | 8 +- drivers/md/dm-crypt.c | 9 - drivers/md/dm-io.c | 11 - drivers/md/dm.c | 118 +------ drivers/md/linear.c | 52 +-- drivers/md/md.c | 44 +-- drivers/md/raid0.c | 70 +--- drivers/md/raid1.c | 34 -- drivers/md/raid10.c | 100 +----- drivers/target/target_core_iblock.c | 9 - fs/bio-integrity.c | 44 --- fs/bio.c | 437 ++++++++++++----------- fs/exofs/ore.c | 5 +- include/linux/bio.h | 43 ++- include/linux/blk_types.h | 9 +- include/linux/blkdev.h | 3 + include/linux/closure.h | 658 +++++++++++++++++++++++++++++++++++ lib/Kconfig.debug | 8 + lib/Makefile | 2 +- lib/closure.c | 344 ++++++++++++++++++ 25 files changed, 1482 insertions(+), 799 deletions(-) create mode 100644 include/linux/closure.h create mode 100644 lib/closure.c -- 1.7.9.3.327.g2980b -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
Block cleanups
CHANGES SINCE LAST VERSION:
Split off the "make generic_make_request() handle arbitrary size bios" - this patch series now just kills bi_destructor and reworks bio splitting. Split out the "rework bio splitting" patch based on Boaz's feedback - now I've got one patch that introduces the new bio_split() and another that rewrites bio_pair_split(), which makes the patches much more readable. Going to try and finish off the rest of the patches that get rid of merge_bvec_fn, but that'll be a separate patch series. Kent Overstreet (12): block: Generalized bio pool freeing dm: Use bioset's front_pad for dm_rq_clone_bio_info block: Add bio_reset() pktcdvd: Switch to bio_kmalloc() block: Kill bi_destructor block: Add an explicit bio flag for bios that own their bvec block: Rename bio_split() -> bio_pair_split() block: Introduce new bio_split() block: Rework bio_pair_split() block: Add bio_clone_kmalloc() block: Add bio_clone_bioset() block: Only clone bio vecs that are in use Documentation/block/biodoc.txt | 5 - block/blk-core.c | 10 +- drivers/block/drbd/drbd_main.c | 13 +-- drivers/block/drbd/drbd_req.c | 18 +-- drivers/block/osdblk.c | 3 +- drivers/block/pktcdvd.c | 121 ++++++----------- drivers/block/rbd.c | 8 +- drivers/md/dm-crypt.c | 9 -- drivers/md/dm-io.c | 11 -- drivers/md/dm.c | 60 ++------- drivers/md/linear.c | 6 +- drivers/md/md.c | 44 +------ drivers/md/raid0.c | 8 +- drivers/md/raid10.c | 23 +--- drivers/target/target_core_iblock.c | 9 -- fs/bio-integrity.c | 44 ------ fs/bio.c | 264 ++++++++++++++++++++++++----------- fs/exofs/ore.c | 5 +- include/linux/bio.h | 37 ++--- include/linux/blk_types.h | 9 +- 20 files changed, 281 insertions(+), 426 deletions(-) -- 1.7.7.3 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
Block cleanups
Various cleanups for the generic block layer - the patches should be pretty
self explanatory. CHANGES SINCE LAST VERSION: Review feedback - should all be noted in the patch descriptions. Fixed retarded rebase conflicts. Added some acked-bys. Kent Overstreet (12): block: Generalized bio pool freeing dm: Use bioset's front_pad for dm_rq_clone_bio_info block: Add bio_reset() pktcdvd: Switch to bio_kmalloc() block: Kill bi_destructor block: Add an explicit bio flag for bios that own their bvec block: Rename bio_split() -> bio_pair_split() block: Introduce new bio_split() block: Rework bio_pair_split() block: Add bio_clone_kmalloc() block: Add bio_clone_bioset() block: Only clone bio vecs that are in use Documentation/block/biodoc.txt | 5 - block/blk-core.c | 10 +- drivers/block/drbd/drbd_main.c | 13 +-- drivers/block/drbd/drbd_req.c | 18 +-- drivers/block/osdblk.c | 3 +- drivers/block/pktcdvd.c | 73 +++------- drivers/block/rbd.c | 8 +- drivers/md/dm-crypt.c | 9 -- drivers/md/dm-io.c | 11 -- drivers/md/dm.c | 60 ++------- drivers/md/linear.c | 6 +- drivers/md/md.c | 44 +------ drivers/md/raid0.c | 8 +- drivers/md/raid10.c | 23 +--- drivers/target/target_core_iblock.c | 9 -- fs/bio-integrity.c | 44 ------ fs/bio.c | 259 +++++++++++++++++++++++------------ fs/exofs/ore.c | 5 +- include/linux/bio.h | 37 ++--- include/linux/blk_types.h | 12 ++- 20 files changed, 256 insertions(+), 401 deletions(-) -- 1.7.7.3 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
Block cleanups
Buncha changes in this version.
I came up with a new, much better solution to the "bio allocation from bio sets under generic_make_request()" problem. Previously, I was working around this in all of my code that allocated bios; it'd mask out GFP_WAIT if current->bio_list != NULL, then punt to workqueue if the allocation failed - so as to avoid the deadlock from allocating multiple bios from the same bio set. This approach worked, but it required it to be explicitly handled in any stacking driver that might split a bio an arbitrary number of times - less than ideal. My new patch inverts this - when we go to allocate a bio and we're blocking another bio from being submitted, we punt the bio(s) that are being blocked off to a workqueue to be submitted. This means we only have to handle it in one place, in bio_alloc_bioset() - stack block drivers no longer have to do anything special (I got to delete some code from bcache, and dm wasn't handling this at all before). Other big change - I consolidated bio allocation. Now bio_alloc(), bio_kmalloc(), bio_clone(), and bio_clone_kmalloc() are all one line wrappers. Incorporated a bunch of review feedback, most of which I hopefully remembered to note in the patch descriptions this time. Kent Overstreet (13): block: Generalized bio pool freeing dm: Use bioset's front_pad for dm_rq_clone_bio_info block: Add bio_reset() pktcdvd: Switch to bio_kmalloc() block: Kill bi_destructor block: Consolidate bio_alloc_bioset(), bio_kmalloc() block: Avoid deadlocks with bio allocation by stacking drivers block: Add an explicit bio flag for bios that own their bvec block: Rename bio_split() -> bio_pair_split() block: Introduce new bio_split() block: Rework bio_pair_split() block: Add bio_clone_bioset(), bio_clone_kmalloc() block: Only clone bio vecs that are in use Documentation/block/biodoc.txt | 5 - block/blk-core.c | 10 +- drivers/block/drbd/drbd_main.c | 13 +- drivers/block/drbd/drbd_req.c | 18 +- drivers/block/osdblk.c | 3 +- drivers/block/pktcdvd.c | 73 ++----- drivers/block/rbd.c | 8 +- drivers/md/dm-crypt.c | 9 - drivers/md/dm-io.c | 11 - drivers/md/dm.c | 68 ++---- drivers/md/linear.c | 6 +- drivers/md/md.c | 44 +--- drivers/md/raid0.c | 8 +- drivers/md/raid10.c | 23 +- drivers/target/target_core_iblock.c | 9 - fs/bio-integrity.c | 45 ---- fs/bio.c | 419 +++++++++++++++++++++++------------- fs/exofs/ore.c | 5 +- include/linux/bio.h | 155 +++++++------ include/linux/blk_types.h | 16 +- 20 files changed, 434 insertions(+), 514 deletions(-) -- 1.7.12 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
Block cleanups
On Wed, Aug 22, 2012 at 10:03:57AM -0700, Kent Overstreet wrote:
[..] > Kent Overstreet (13): > block: Generalized bio pool freeing > dm: Use bioset's front_pad for dm_rq_clone_bio_info > block: Add bio_reset() > pktcdvd: Switch to bio_kmalloc() > block: Kill bi_destructor > block: Consolidate bio_alloc_bioset(), bio_kmalloc() This patch series says "Block Cleanups" in the subject. I think that it can be divided in two parts. First part seems to be cleanup where we get rid of ->bi_destructor and merge kmalloc() and allocation from bio_set. Second part seems to be more of enhancing bio splitting functionality and not necessarily a cleanup. So may be breaking this series in two parts makes sense. First part is cleanup and second part is enhanced bio splitting capabilities of block layer. Thanks Vivek > block: Avoid deadlocks with bio allocation by stacking drivers > block: Add an explicit bio flag for bios that own their bvec > block: Rename bio_split() -> bio_pair_split() > block: Introduce new bio_split() > block: Rework bio_pair_split() > block: Add bio_clone_bioset(), bio_clone_kmalloc() > block: Only clone bio vecs that are in use > > Documentation/block/biodoc.txt | 5 - > block/blk-core.c | 10 +- > drivers/block/drbd/drbd_main.c | 13 +- > drivers/block/drbd/drbd_req.c | 18 +- > drivers/block/osdblk.c | 3 +- > drivers/block/pktcdvd.c | 73 ++----- > drivers/block/rbd.c | 8 +- > drivers/md/dm-crypt.c | 9 - > drivers/md/dm-io.c | 11 - > drivers/md/dm.c | 68 ++---- > drivers/md/linear.c | 6 +- > drivers/md/md.c | 44 +--- > drivers/md/raid0.c | 8 +- > drivers/md/raid10.c | 23 +- > drivers/target/target_core_iblock.c | 9 - > fs/bio-integrity.c | 45 ---- > fs/bio.c | 419 +++++++++++++++++++++++------------- > fs/exofs/ore.c | 5 +- > include/linux/bio.h | 155 +++++++------ > include/linux/blk_types.h | 16 +- > 20 files changed, 434 insertions(+), 514 deletions(-) > > -- > 1.7.12 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
Block cleanups
On Thu, Aug 23, 2012 at 02:00:41PM -0400, Vivek Goyal wrote:
> On Wed, Aug 22, 2012 at 10:03:57AM -0700, Kent Overstreet wrote: > > [..] > > Kent Overstreet (13): > > block: Generalized bio pool freeing > > dm: Use bioset's front_pad for dm_rq_clone_bio_info > > block: Add bio_reset() > > pktcdvd: Switch to bio_kmalloc() > > block: Kill bi_destructor > > block: Consolidate bio_alloc_bioset(), bio_kmalloc() > > This patch series says "Block Cleanups" in the subject. I think that > it can be divided in two parts. First part seems to be cleanup where > we get rid of ->bi_destructor and merge kmalloc() and allocation > from bio_set. > > Second part seems to be more of enhancing bio splitting functionality > and not necessarily a cleanup. So may be breaking this series in two > parts makes sense. First part is cleanup and second part is enhanced > bio splitting capabilities of block layer. Yeah, that's a good idea - will do. -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
Block cleanups
Since v7:
Split off the deadlock fix, this is now just cleanups Noticed a minor issue with bio integrity freeing, and decided the best way to fix it was to just make it use bi_pool directly which simplifies the code anyways. Tested the integrity stuff with the scsi debug module. Cleaned up bio_reset()/bio_free() a bit, introduce a __bio_free() helper used by both and made bio_free() static. Kent Overstreet (8): block: Generalized bio pool freeing block: Ues bi_pool for bio_integrity_alloc() dm: Use bioset's front_pad for dm_rq_clone_bio_info block: Add bio_reset() pktcdvd: Switch to bio_kmalloc() block: Kill bi_destructor block: Consolidate bio_alloc_bioset(), bio_kmalloc() block: Add bio_clone_bioset(), bio_clone_kmalloc() Documentation/block/biodoc.txt | 5 - block/blk-core.c | 10 +- drivers/block/drbd/drbd_main.c | 13 +-- drivers/block/osdblk.c | 3 +- drivers/block/pktcdvd.c | 52 ++-------- drivers/md/dm-crypt.c | 16 +-- drivers/md/dm-io.c | 11 -- drivers/md/dm.c | 72 ++++--------- drivers/md/md.c | 44 +------- drivers/target/target_core_iblock.c | 9 -- fs/bio-integrity.c | 44 +++----- fs/bio.c | 196 +++++++++++++++--------------------- fs/exofs/ore.c | 5 +- include/linux/bio.h | 44 +++++--- include/linux/blk_types.h | 27 +++-- 15 files changed, 183 insertions(+), 368 deletions(-) -- 1.7.12 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
Block cleanups
Since v8: Just a few minor comment/patch descriptions changes
Kent Overstreet (9): block: Generalized bio pool freeing block: Ues bi_pool for bio_integrity_alloc() dm: Use bioset's front_pad for dm_rq_clone_bio_info block: Add bio_reset() pktcdvd: Switch to bio_kmalloc() block: Add bio_reset() block: Kill bi_destructor block: Consolidate bio_alloc_bioset(), bio_kmalloc() block: Add bio_clone_bioset(), bio_clone_kmalloc() Documentation/block/biodoc.txt | 5 - block/blk-core.c | 10 +- drivers/block/drbd/drbd_main.c | 13 +-- drivers/block/osdblk.c | 3 +- drivers/block/pktcdvd.c | 52 ++------- drivers/md/dm-crypt.c | 16 +-- drivers/md/dm-io.c | 11 -- drivers/md/dm.c | 72 ++++-------- drivers/md/md.c | 44 +------- drivers/target/target_core_iblock.c | 9 -- fs/bio-integrity.c | 44 +++----- fs/bio.c | 220 +++++++++++++++++------------------- fs/exofs/ore.c | 5 +- include/linux/bio.h | 44 +++++--- include/linux/blk_types.h | 27 +++-- 15 files changed, 207 insertions(+), 368 deletions(-) -- 1.7.12 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
Block cleanups
Screwed up the bio_reset() patch in the last patch series when I went to edit
the description, fixed that here. Only other change is the dm patch - made the front_pad conditional on DM_TYPE_BIO_BASED. Kent Overstreet (8): block: Generalized bio pool freeing block: Ues bi_pool for bio_integrity_alloc() dm: Use bioset's front_pad for dm_rq_clone_bio_info block: Add bio_reset() pktcdvd: Switch to bio_kmalloc() block: Kill bi_destructor block: Consolidate bio_alloc_bioset(), bio_kmalloc() block: Add bio_clone_bioset(), bio_clone_kmalloc() Documentation/block/biodoc.txt | 5 - block/blk-core.c | 10 +- drivers/block/drbd/drbd_main.c | 13 +-- drivers/block/osdblk.c | 3 +- drivers/block/pktcdvd.c | 52 ++------- drivers/md/dm-crypt.c | 16 +-- drivers/md/dm-io.c | 11 -- drivers/md/dm.c | 74 ++++--------- drivers/md/md.c | 44 +------- drivers/target/target_core_iblock.c | 9 -- fs/bio-integrity.c | 44 +++----- fs/bio.c | 206 ++++++++++++++++-------------------- fs/exofs/ore.c | 5 +- include/linux/bio.h | 44 +++++--- include/linux/blk_types.h | 27 +++-- 15 files changed, 195 insertions(+), 368 deletions(-) -- 1.7.12 -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
Block cleanups
Hello, guys.
(cc'ing Jens, Alasdair and Neil) On Thu, Sep 06, 2012 at 03:34:54PM -0700, Kent Overstreet wrote: > Screwed up the bio_reset() patch in the last patch series when I went to edit > the description, fixed that here. > > Only other change is the dm patch - made the front_pad conditional on > DM_TYPE_BIO_BASED. > > Kent Overstreet (8): > block: Generalized bio pool freeing > block: Ues bi_pool for bio_integrity_alloc() > dm: Use bioset's front_pad for dm_rq_clone_bio_info > block: Add bio_reset() > pktcdvd: Switch to bio_kmalloc() > block: Kill bi_destructor > block: Consolidate bio_alloc_bioset(), bio_kmalloc() > block: Add bio_clone_bioset(), bio_clone_kmalloc() This series looks good to me now. If someone can ack the dm patch, I think it's good to go. Jens, what do you think? Thanks. -- tejun -- dm-devel mailing list dm-devel@redhat.com https://www.redhat.com/mailman/listinfo/dm-devel |
| All times are GMT. The time now is 09:50 AM. |
VBulletin, Copyright ©2000 - 2013, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.