FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor


 
 
LinkBack Thread Tools
 
Old 05-25-2012, 08:25 PM
Kent Overstreet
 
Default Block cleanups

Incorporated various review feedback, and I reordered/reworked the patches to
make them more coherent.

This also has the latest version of the closure code, incorporating review
feedback from Joe and Tejun. I may rewrite the bio splitting stuff to not
depend on it so it can go in sooner (Tejun is not a fan of closures), but IMO
the patch is better for using them.

First 5 patches: Kill bi_destructor

Next 3 patches: Better bio splitting

Next 3 patches: Rework bio cloning to not clone entire bio vec

Tejun also pointed out that with the bio splitting improvements we ought to be
able to get rid of merge_bvec_fn, so I started on that. Those patches are
definitely a WIP, though.

For most of the md code, it looks like things will almost just work - they
already had bio_pair_split() calls to handle single page bios that crossed bvec
boundries, but bio_pair_split() now splits arbitrary bios so things should just work.

(I just thought of one problem - previously, in practice you'd never need to
split a bio more than once, but that's no longer true - so the
bio_pair_split() calls do need to be stuck in a loop, and the as is the
patches aren't correct.)

For raid5 though I couldn't find any splitting in the existing code - it looks
like it's depending on merge_bvec_fn to make sure bios never need to be split.
Adding bio splitting looks to be not quite trivial - Neil, any thoughts?

For dm, it was already splitting multi page bios when necessary, it looked like
its merge_bvec_fn was mostly for propagating device limits - which my
generic_make_request() patch handles. Any dm folks want to take a look?

There are only 4 merge_bvec_fns left in my kernel:

drivers/md/raid5.c - raid5_mergeable_bvec()
drivers/block/drbd/drbd_req.c - drbd_merge_bvec()
drivers/block/pktcdvd.c - pkt_merge_bvec()
drivers/block/rbd.c - rbd_merge_bvec()

I haven't looked at the last three, but if the maintainers want to help out a
bit it should be pretty easy to get rid of merge_bvec_fn entirely.

Kent Overstreet (16):
block: Generalized bio pool freeing
dm: Use bioset's front_pad for dm_rq_clone_bio_info
block: Add bio_reset()
pktcdvd: Switch to bio_kmalloc()
block: Kill bi_destructor
block: Add an explicit bio flag for bios that own their bvec
block: Rename bio_split() -> bio_pair_split()
block: Rework bio splitting
block: Add bio_clone_kmalloc()
block: Add bio_clone_bioset()
block: Only clone bio vecs that are in use
Closures
Make generic_make_request handle arbitrarily large bios
Gut bio_add_page()
md: Kill merge_bvec_fn()s
dm: Kill merge_bvec_fn()

Documentation/block/biodoc.txt | 5 -
block/blk-core.c | 126 ++++++-
drivers/block/drbd/drbd_req.c | 18 +-
drivers/block/osdblk.c | 3 +-
drivers/block/pktcdvd.c | 121 +++----
drivers/block/rbd.c | 8 +-
drivers/md/dm-crypt.c | 9 -
drivers/md/dm-io.c | 11 -
drivers/md/dm.c | 118 +------
drivers/md/linear.c | 52 +--
drivers/md/md.c | 44 +--
drivers/md/raid0.c | 70 +---
drivers/md/raid1.c | 34 --
drivers/md/raid10.c | 100 +-----
drivers/target/target_core_iblock.c | 9 -
fs/bio-integrity.c | 44 ---
fs/bio.c | 437 ++++++++++++-----------
fs/exofs/ore.c | 5 +-
include/linux/bio.h | 43 ++-
include/linux/blk_types.h | 9 +-
include/linux/blkdev.h | 3 +
include/linux/closure.h | 658 +++++++++++++++++++++++++++++++++++
lib/Kconfig.debug | 8 +
lib/Makefile | 2 +-
lib/closure.c | 344 ++++++++++++++++++
25 files changed, 1482 insertions(+), 799 deletions(-)
create mode 100644 include/linux/closure.h
create mode 100644 lib/closure.c

--
1.7.9.3.327.g2980b

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 07-24-2012, 08:11 PM
Kent Overstreet
 
Default Block cleanups

CHANGES SINCE LAST VERSION:

Split off the "make generic_make_request() handle arbitrary size bios" - this
patch series now just kills bi_destructor and reworks bio splitting.

Split out the "rework bio splitting" patch based on Boaz's feedback - now I've
got one patch that introduces the new bio_split() and another that rewrites
bio_pair_split(), which makes the patches much more readable.

Going to try and finish off the rest of the patches that get rid of
merge_bvec_fn, but that'll be a separate patch series.

Kent Overstreet (12):
block: Generalized bio pool freeing
dm: Use bioset's front_pad for dm_rq_clone_bio_info
block: Add bio_reset()
pktcdvd: Switch to bio_kmalloc()
block: Kill bi_destructor
block: Add an explicit bio flag for bios that own their bvec
block: Rename bio_split() -> bio_pair_split()
block: Introduce new bio_split()
block: Rework bio_pair_split()
block: Add bio_clone_kmalloc()
block: Add bio_clone_bioset()
block: Only clone bio vecs that are in use

Documentation/block/biodoc.txt | 5 -
block/blk-core.c | 10 +-
drivers/block/drbd/drbd_main.c | 13 +--
drivers/block/drbd/drbd_req.c | 18 +--
drivers/block/osdblk.c | 3 +-
drivers/block/pktcdvd.c | 121 ++++++-----------
drivers/block/rbd.c | 8 +-
drivers/md/dm-crypt.c | 9 --
drivers/md/dm-io.c | 11 --
drivers/md/dm.c | 60 ++-------
drivers/md/linear.c | 6 +-
drivers/md/md.c | 44 +------
drivers/md/raid0.c | 8 +-
drivers/md/raid10.c | 23 +---
drivers/target/target_core_iblock.c | 9 --
fs/bio-integrity.c | 44 ------
fs/bio.c | 264 ++++++++++++++++++++++++-----------
fs/exofs/ore.c | 5 +-
include/linux/bio.h | 37 ++---
include/linux/blk_types.h | 9 +-
20 files changed, 281 insertions(+), 426 deletions(-)

--
1.7.7.3

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 08-06-2012, 10:08 PM
Kent Overstreet
 
Default Block cleanups

Various cleanups for the generic block layer - the patches should be pretty
self explanatory.

CHANGES SINCE LAST VERSION:

Review feedback - should all be noted in the patch descriptions. Fixed
retarded rebase conflicts. Added some acked-bys.

Kent Overstreet (12):
block: Generalized bio pool freeing
dm: Use bioset's front_pad for dm_rq_clone_bio_info
block: Add bio_reset()
pktcdvd: Switch to bio_kmalloc()
block: Kill bi_destructor
block: Add an explicit bio flag for bios that own their bvec
block: Rename bio_split() -> bio_pair_split()
block: Introduce new bio_split()
block: Rework bio_pair_split()
block: Add bio_clone_kmalloc()
block: Add bio_clone_bioset()
block: Only clone bio vecs that are in use

Documentation/block/biodoc.txt | 5 -
block/blk-core.c | 10 +-
drivers/block/drbd/drbd_main.c | 13 +--
drivers/block/drbd/drbd_req.c | 18 +--
drivers/block/osdblk.c | 3 +-
drivers/block/pktcdvd.c | 73 +++-------
drivers/block/rbd.c | 8 +-
drivers/md/dm-crypt.c | 9 --
drivers/md/dm-io.c | 11 --
drivers/md/dm.c | 60 ++-------
drivers/md/linear.c | 6 +-
drivers/md/md.c | 44 +------
drivers/md/raid0.c | 8 +-
drivers/md/raid10.c | 23 +---
drivers/target/target_core_iblock.c | 9 --
fs/bio-integrity.c | 44 ------
fs/bio.c | 259 +++++++++++++++++++++++------------
fs/exofs/ore.c | 5 +-
include/linux/bio.h | 37 ++---
include/linux/blk_types.h | 12 ++-
20 files changed, 256 insertions(+), 401 deletions(-)

--
1.7.7.3

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 08-22-2012, 05:03 PM
Kent Overstreet
 
Default Block cleanups

Buncha changes in this version.

I came up with a new, much better solution to the "bio allocation from
bio sets under generic_make_request()" problem. Previously, I was
working around this in all of my code that allocated bios; it'd mask out
GFP_WAIT if current->bio_list != NULL, then punt to workqueue if the
allocation failed - so as to avoid the deadlock from allocating multiple
bios from the same bio set.

This approach worked, but it required it to be explicitly handled in any
stacking driver that might split a bio an arbitrary number of times -
less than ideal.

My new patch inverts this - when we go to allocate a bio and we're
blocking another bio from being submitted, we punt the bio(s) that are
being blocked off to a workqueue to be submitted. This means we only
have to handle it in one place, in bio_alloc_bioset() - stack block
drivers no longer have to do anything special (I got to delete some code
from bcache, and dm wasn't handling this at all before).

Other big change - I consolidated bio allocation. Now bio_alloc(),
bio_kmalloc(), bio_clone(), and bio_clone_kmalloc() are all one line
wrappers.

Incorporated a bunch of review feedback, most of which I hopefully
remembered to note in the patch descriptions this time.

Kent Overstreet (13):
block: Generalized bio pool freeing
dm: Use bioset's front_pad for dm_rq_clone_bio_info
block: Add bio_reset()
pktcdvd: Switch to bio_kmalloc()
block: Kill bi_destructor
block: Consolidate bio_alloc_bioset(), bio_kmalloc()
block: Avoid deadlocks with bio allocation by stacking drivers
block: Add an explicit bio flag for bios that own their bvec
block: Rename bio_split() -> bio_pair_split()
block: Introduce new bio_split()
block: Rework bio_pair_split()
block: Add bio_clone_bioset(), bio_clone_kmalloc()
block: Only clone bio vecs that are in use

Documentation/block/biodoc.txt | 5 -
block/blk-core.c | 10 +-
drivers/block/drbd/drbd_main.c | 13 +-
drivers/block/drbd/drbd_req.c | 18 +-
drivers/block/osdblk.c | 3 +-
drivers/block/pktcdvd.c | 73 ++-----
drivers/block/rbd.c | 8 +-
drivers/md/dm-crypt.c | 9 -
drivers/md/dm-io.c | 11 -
drivers/md/dm.c | 68 ++----
drivers/md/linear.c | 6 +-
drivers/md/md.c | 44 +---
drivers/md/raid0.c | 8 +-
drivers/md/raid10.c | 23 +-
drivers/target/target_core_iblock.c | 9 -
fs/bio-integrity.c | 45 ----
fs/bio.c | 419 +++++++++++++++++++++++-------------
fs/exofs/ore.c | 5 +-
include/linux/bio.h | 155 +++++++------
include/linux/blk_types.h | 16 +-
20 files changed, 434 insertions(+), 514 deletions(-)

--
1.7.12

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 08-23-2012, 06:00 PM
Vivek Goyal
 
Default Block cleanups

On Wed, Aug 22, 2012 at 10:03:57AM -0700, Kent Overstreet wrote:

[..]
> Kent Overstreet (13):
> block: Generalized bio pool freeing
> dm: Use bioset's front_pad for dm_rq_clone_bio_info
> block: Add bio_reset()
> pktcdvd: Switch to bio_kmalloc()
> block: Kill bi_destructor
> block: Consolidate bio_alloc_bioset(), bio_kmalloc()

This patch series says "Block Cleanups" in the subject. I think that
it can be divided in two parts. First part seems to be cleanup where
we get rid of ->bi_destructor and merge kmalloc() and allocation
from bio_set.

Second part seems to be more of enhancing bio splitting functionality
and not necessarily a cleanup. So may be breaking this series in two
parts makes sense. First part is cleanup and second part is enhanced
bio splitting capabilities of block layer.

Thanks
Vivek

> block: Avoid deadlocks with bio allocation by stacking drivers
> block: Add an explicit bio flag for bios that own their bvec
> block: Rename bio_split() -> bio_pair_split()
> block: Introduce new bio_split()
> block: Rework bio_pair_split()
> block: Add bio_clone_bioset(), bio_clone_kmalloc()
> block: Only clone bio vecs that are in use
>
> Documentation/block/biodoc.txt | 5 -
> block/blk-core.c | 10 +-
> drivers/block/drbd/drbd_main.c | 13 +-
> drivers/block/drbd/drbd_req.c | 18 +-
> drivers/block/osdblk.c | 3 +-
> drivers/block/pktcdvd.c | 73 ++-----
> drivers/block/rbd.c | 8 +-
> drivers/md/dm-crypt.c | 9 -
> drivers/md/dm-io.c | 11 -
> drivers/md/dm.c | 68 ++----
> drivers/md/linear.c | 6 +-
> drivers/md/md.c | 44 +---
> drivers/md/raid0.c | 8 +-
> drivers/md/raid10.c | 23 +-
> drivers/target/target_core_iblock.c | 9 -
> fs/bio-integrity.c | 45 ----
> fs/bio.c | 419 +++++++++++++++++++++++-------------
> fs/exofs/ore.c | 5 +-
> include/linux/bio.h | 155 +++++++------
> include/linux/blk_types.h | 16 +-
> 20 files changed, 434 insertions(+), 514 deletions(-)
>
> --
> 1.7.12

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 08-24-2012, 12:46 PM
Kent Overstreet
 
Default Block cleanups

On Thu, Aug 23, 2012 at 02:00:41PM -0400, Vivek Goyal wrote:
> On Wed, Aug 22, 2012 at 10:03:57AM -0700, Kent Overstreet wrote:
>
> [..]
> > Kent Overstreet (13):
> > block: Generalized bio pool freeing
> > dm: Use bioset's front_pad for dm_rq_clone_bio_info
> > block: Add bio_reset()
> > pktcdvd: Switch to bio_kmalloc()
> > block: Kill bi_destructor
> > block: Consolidate bio_alloc_bioset(), bio_kmalloc()
>
> This patch series says "Block Cleanups" in the subject. I think that
> it can be divided in two parts. First part seems to be cleanup where
> we get rid of ->bi_destructor and merge kmalloc() and allocation
> from bio_set.
>
> Second part seems to be more of enhancing bio splitting functionality
> and not necessarily a cleanup. So may be breaking this series in two
> parts makes sense. First part is cleanup and second part is enhanced
> bio splitting capabilities of block layer.

Yeah, that's a good idea - will do.

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-05-2012, 08:27 PM
Kent Overstreet
 
Default Block cleanups

Since v7:

Split off the deadlock fix, this is now just cleanups

Noticed a minor issue with bio integrity freeing, and decided the best way to
fix it was to just make it use bi_pool directly which simplifies the code
anyways.

Tested the integrity stuff with the scsi debug module.

Cleaned up bio_reset()/bio_free() a bit, introduce a __bio_free() helper used
by both and made bio_free() static.

Kent Overstreet (8):
block: Generalized bio pool freeing
block: Ues bi_pool for bio_integrity_alloc()
dm: Use bioset's front_pad for dm_rq_clone_bio_info
block: Add bio_reset()
pktcdvd: Switch to bio_kmalloc()
block: Kill bi_destructor
block: Consolidate bio_alloc_bioset(), bio_kmalloc()
block: Add bio_clone_bioset(), bio_clone_kmalloc()

Documentation/block/biodoc.txt | 5 -
block/blk-core.c | 10 +-
drivers/block/drbd/drbd_main.c | 13 +--
drivers/block/osdblk.c | 3 +-
drivers/block/pktcdvd.c | 52 ++--------
drivers/md/dm-crypt.c | 16 +--
drivers/md/dm-io.c | 11 --
drivers/md/dm.c | 72 ++++---------
drivers/md/md.c | 44 +-------
drivers/target/target_core_iblock.c | 9 --
fs/bio-integrity.c | 44 +++-----
fs/bio.c | 196 +++++++++++++++---------------------
fs/exofs/ore.c | 5 +-
include/linux/bio.h | 44 +++++---
include/linux/blk_types.h | 27 +++--
15 files changed, 183 insertions(+), 368 deletions(-)

--
1.7.12

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-06-2012, 09:32 PM
Kent Overstreet
 
Default Block cleanups

Since v8: Just a few minor comment/patch descriptions changes

Kent Overstreet (9):
block: Generalized bio pool freeing
block: Ues bi_pool for bio_integrity_alloc()
dm: Use bioset's front_pad for dm_rq_clone_bio_info
block: Add bio_reset()
pktcdvd: Switch to bio_kmalloc()
block: Add bio_reset()
block: Kill bi_destructor
block: Consolidate bio_alloc_bioset(), bio_kmalloc()
block: Add bio_clone_bioset(), bio_clone_kmalloc()

Documentation/block/biodoc.txt | 5 -
block/blk-core.c | 10 +-
drivers/block/drbd/drbd_main.c | 13 +--
drivers/block/osdblk.c | 3 +-
drivers/block/pktcdvd.c | 52 ++-------
drivers/md/dm-crypt.c | 16 +--
drivers/md/dm-io.c | 11 --
drivers/md/dm.c | 72 ++++--------
drivers/md/md.c | 44 +-------
drivers/target/target_core_iblock.c | 9 --
fs/bio-integrity.c | 44 +++-----
fs/bio.c | 220 +++++++++++++++++-------------------
fs/exofs/ore.c | 5 +-
include/linux/bio.h | 44 +++++---
include/linux/blk_types.h | 27 +++--
15 files changed, 207 insertions(+), 368 deletions(-)

--
1.7.12

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-06-2012, 10:34 PM
Kent Overstreet
 
Default Block cleanups

Screwed up the bio_reset() patch in the last patch series when I went to edit
the description, fixed that here.

Only other change is the dm patch - made the front_pad conditional on
DM_TYPE_BIO_BASED.

Kent Overstreet (8):
block: Generalized bio pool freeing
block: Ues bi_pool for bio_integrity_alloc()
dm: Use bioset's front_pad for dm_rq_clone_bio_info
block: Add bio_reset()
pktcdvd: Switch to bio_kmalloc()
block: Kill bi_destructor
block: Consolidate bio_alloc_bioset(), bio_kmalloc()
block: Add bio_clone_bioset(), bio_clone_kmalloc()

Documentation/block/biodoc.txt | 5 -
block/blk-core.c | 10 +-
drivers/block/drbd/drbd_main.c | 13 +--
drivers/block/osdblk.c | 3 +-
drivers/block/pktcdvd.c | 52 ++-------
drivers/md/dm-crypt.c | 16 +--
drivers/md/dm-io.c | 11 --
drivers/md/dm.c | 74 ++++---------
drivers/md/md.c | 44 +-------
drivers/target/target_core_iblock.c | 9 --
fs/bio-integrity.c | 44 +++-----
fs/bio.c | 206 ++++++++++++++++--------------------
fs/exofs/ore.c | 5 +-
include/linux/bio.h | 44 +++++---
include/linux/blk_types.h | 27 +++--
15 files changed, 195 insertions(+), 368 deletions(-)

--
1.7.12

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 09-06-2012, 11:48 PM
Tejun Heo
 
Default Block cleanups

Hello, guys.

(cc'ing Jens, Alasdair and Neil)

On Thu, Sep 06, 2012 at 03:34:54PM -0700, Kent Overstreet wrote:
> Screwed up the bio_reset() patch in the last patch series when I went to edit
> the description, fixed that here.
>
> Only other change is the dm patch - made the front_pad conditional on
> DM_TYPE_BIO_BASED.
>
> Kent Overstreet (8):
> block: Generalized bio pool freeing
> block: Ues bi_pool for bio_integrity_alloc()
> dm: Use bioset's front_pad for dm_rq_clone_bio_info
> block: Add bio_reset()
> pktcdvd: Switch to bio_kmalloc()
> block: Kill bi_destructor
> block: Consolidate bio_alloc_bioset(), bio_kmalloc()
> block: Add bio_clone_bioset(), bio_clone_kmalloc()

This series looks good to me now. If someone can ack the dm patch, I
think it's good to go. Jens, what do you think?

Thanks.

--
tejun

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 07:15 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org