FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.

» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

LinkBack Thread Tools
Old 08-24-2011, 11:05 PM
Mike Snitzer
Default dm: share mempools for all request-based DM devices

Request-based DM requires considerably larger mempool reserves than
bio-based DM given that it must clone requests (which may contain
multiple bios) rather than simply cloning bios.

It was reported that bio-based dm-multipath was able to be tested with
1024 mpath devices and failover/failback under load on a server with
only 2 GB of memory. The same test fails with request-based
dm-multipath because of the increase in bio reserves (biovec-256).

Patches are available here but I'm too ashamed to actually post them
to the list:

In the end it would likely be best if we simply automatically scaled
RESERVED_REQUEST_BASED_IOS back if the system has low amounts of
memory (say 128 if 4GB, 64 if 3GB, 32 if 2GB, 16 if 1GB) -- this would
result in ~1600MB, ~800MB, ~400MB, and ~200MB (respectively) of slab
reserves for 1024 multipath devices (or 600, 300, 150, 75 MB
respectively if integrity support isn't needed -- yeah bip-256 brings
the pain given objperslab=1 and pagesperslab=2!).

Additional Background:

The relevant slab use for bio-based vs request-based for 1024 devices

bio-0 3.19921875 MB
biovec-256 64.0 MB
bip-256 128.0 MB
dm_io 1.7265625 MB
dm_target_io 0.44140625 MB

total: 197.3671875 MB

bio-0 51.19921875 MB
biovec-256 1024.0 MB
bip-256 2048.0 MB
dm_mpath_io 5.06640625 MB
dm_rq_clone_bio_info 5.06640625 MB
dm_rq_target_io 102.3984375 MB

total: 3235.73046875 MB

Above was generated with: bio_vs_rq_slab_usage.py 1024 256

Commit a91a2785b200 ("block: Require subsystems to explicitly allocate
bio_set integrity mempool") eliminated unnecessary slab use (bip-256
reserves for all 1024 devices consume 2 GB!) but the system still failed
to boot because of multipathd consuming too much memory.

Mike Snitzer (4):
dm: share mempools for all request-based DM devices
dm table: disallow stacking request-based DM devices
dm: add reserved_rq_based_ios module parameter

drivers/md/dm-table.c | 9 +++
drivers/md/dm.c | 166 +++++++++++++++++++++++++++++++++++++++++++------
2 files changed, 156 insertions(+), 19 deletions(-)

dm-devel mailing list

Thread Tools

All times are GMT. The time now is 04:17 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org