FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 01-12-2012, 11:55 PM
Jagan Reddy
 
Default dm-thin vs lvm performance

Hi,I *recently started using dm-thin module and first of all, thank you for the good work. It works great, well documented and the comments in the code are very useful. *I tried to run some performance tests using a high performance flash device and the performance of dm thin is about 30% of LVM performance on a full allocated thin provisioned volume. ( ie after all pages/blocks are allocated using dd).
Performance test does only reads and no writes.*
To reproduce the scenario easily, I tried same operations using ramdisk and dd.*
To read 3.2G of data using dd,*
** Raw Ramdisk takes - 2.5 seconds** LVM Lun - takes 4 seconds** TP Lun ( after fully allocated) - takes 18 seconds


Steps involved------------------1. Linux 3.2.0.rc4 version from*kernel.org*3.2.0-rc4 #1 SMP Fri Dec 16 13:25:02 EST 2011 x86_64 x86_64 x86_64 GNU/Linux2.*LVM2.2.02.883. compiling and installing dm thin modules.4. Create a ramdisk of size 4G.
Ramdisk Raw Read Test--------------------------[root@lab-dm-cn4 ~]# dd
of=/dev/null if=/dev/ram bs=4096 count=786432786432+0 records in786432+0 records out3221225472 bytes (3.2 GB) copied, 2.56607 s, 1.3 GB/s
LVM SETUP------------------------
Create PV with 8M metadata on ramdisk , chunksize=8M in VG, lv uses 100%VG.
root@lab-dm-cn4 ~]# dd of=/dev/null if=/dev/6C067CWX00132-1/engineering bs=4096 count=$BLOCKS** * * * * * * * * * * * * * * *786432+0 records in** * * * * * * * * * * * * * * *786432+0 records out** * * * * * * * * * * * * * * *3221225472 bytes (3.2 GB) copied, 4.06687 s, 792
MB/s


DM THIN SETUP-----------------------Create metadata device of size 8M and data device of size 3G using dm-linear on ramdisk.Create TP Pool using the above devices with page/block size of 8M.Create TP LUN in the TP Pool.Using dd, write 4k to every page in the TP LUN so that all pages are allocated.using "dmsetup status" verify that all pages in the pool are allocated.
[root@lab-dm-cn4 ~]# dd of=/dev/null if=/dev/mapper/thin1 bs=4096 count=$BLOCKS** * * * * * * * * * * * * * * * * * * * *786432+0 records in** * * * * * * * * * * * * * * * * * * * *786432+0
records out** * * * * * * * * * * * * * * * * * * * *3221225472 bytes (3.2 GB) copied, 18.2166 s, 177 MB/s


I have attached the output file and the script file used for this test. *Any ideas / suggestions on what may be the bottle neck in dm-thin performance is much appreciated.
Thanks,Jagan.


[root@lab-dm-cn4 ~]# #constants
[root@lab-dm-cn4 ~]# let ONEMEG=1024*2; #translated to sectors
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #user defined
[root@lab-dm-cn4 ~]# DMSETUP=/home/jagan/src/LVM2.2.02.88/tools/dmsetup
[root@lab-dm-cn4 ~]# LUN=1
[root@lab-dm-cn4 ~]# META_DEVICE_START_SECTOR=0 #on $DEVICE_PATH this is where meta data starts
[root@lab-dm-cn4 ~]# DEVICE_PATH=/dev/ram
[root@lab-dm-cn4 ~]# let META_DEV_SIZE=1*1024*$ONEMEG; #1G translated to sectors used in pvcreate command
[root@lab-dm-cn4 ~]# let PAGESIZE=8*$ONEMEG #page size is 8M in sectors
[root@lab-dm-cn4 ~]# let DATA_DEV_SIZE=3*1024*$ONEMEG+$PAGESIZE; #3G translated to sectors
[root@lab-dm-cn4 ~]# let LVM_DEVICE_SIZE=$META_DEV_SIZE+$DATA_DEV_SIZE;
[root@lab-dm-cn4 ~]# let DD_BS=4*1024 # dd block size = 4k bytes
[root@lab-dm-cn4 ~]# let BLOCKS=($DATA_DEV_SIZE-$PAGESIZE)*512/$DD_BS
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #system defined
[root@lab-dm-cn4 ~]# let DATA_DEVICE_START_SECTOR=$META_DEVICE_START_SECTOR +$META_DEV_SIZE
[root@lab-dm-cn4 ~]# let PAGES=$DATA_DEV_SIZE/$PAGESIZE
[root@lab-dm-cn4 ~]# let SEEK=$PAGESIZE*512/$DD_BS
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# echo $DMSETUP LUNID=$LUN DEVICE=$DEVICE_PATH DEVICE_START=$META_DEVICE_START_SECTOR META_DEV_SIZE=$META_DEV_SIZE DATA_DEV_SIZE=$DATA_DEV_SIZE PAGESIZE=$PAGESIZE DD_BS=$DD_BS DATA_DEVICE_START_SECTOR=$DATA_DEVICE_START_SECTOR PAGES=$PAGES SEEK=$SEEK BLOCKS=$BLOCKS LVM_DEVICE_SIZE=$LVM_DEVICE_SIZE
/home/jagan/src/LVM2.2.02.88/tools/dmsetup LUNID=1 DEVICE=/dev/ram DEVICE_START=0 META_DEV_SIZE=2097152 DATA_DEV_SIZE=6307840 PAGESIZE=16384 DD_BS=4096 DATA_DEVICE_START_SECTOR=2097152 PAGES=385 SEEK=2048 BLOCKS=786432 LVM_DEVICE_SIZE=8404992
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# pvcreate -M2 --dataalignment 4K --metadatacopies 1 --metadatasize 8M --zero y -v /dev/ram
Set up physical volume for "/dev/ram" with 8388608 available sectors
Zeroing start of device /dev/ram
Writing physical volume data to disk "/dev/ram"
Physical volume "/dev/ram" successfully created
[root@lab-dm-cn4 ~]# vgcreate -l 16384 -s 8M -c n -A n 6C067CWX00132-1 /dev/ram
WARNING: This metadata update is NOT backed up
Volume group "6C067CWX00132-1" successfully created
[root@lab-dm-cn4 ~]# lvcreate -A n -C n -l 100%VG -Z y -n engineering -r None 6C067CWX00132-1
WARNING: This metadata update is NOT backed up
WARNING: This metadata update is NOT backed up
Logical volume "engineering" created
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# # run dd test on several devices
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #write something
[root@lab-dm-cn4 ~]# dd if=/dev/zero of=/dev/6C067CWX00132-1/engineering bs=4096 count=$BLOCKS
786432+0 records in
786432+0 records out
3221225472 bytes (3.2 GB) copied, 4.63318 s, 695 MB/s
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #note the time on lvm device
[root@lab-dm-cn4 ~]# dd of=/dev/null if=/dev/6C067CWX00132-1/engineering bs=4096 count=$BLOCKS
786432+0 records in
786432+0 records out
3221225472 bytes (3.2 GB) copied, 4.06687 s, 792 MB/s
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# lvremove -f /dev/6C067CWX00132-1/engineering
Logical volume "engineering" successfully removed
[root@lab-dm-cn4 ~]# vgremove 6C067CWX00132-1
Volume group "6C067CWX00132-1" successfully removed
[root@lab-dm-cn4 ~]# pvremove /dev/ram
Labels on physical volume "/dev/ram" successfully wiped
[root@lab-dm-cn4 ~]# lvdisplay
No volume groups found
[root@lab-dm-cn4 ~]# vgdisplay
No volume groups found
[root@lab-dm-cn4 ~]# pvdisplay
[root@lab-dm-cn4 ~]# ls -l /dev/mapper
total 0
crw-rw----. 1 root root 10, 236 Jan 11 11:34 control
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #constants
[root@lab-dm-cn4 ~]# let ONEMEG=1024*2; #translated to sectors
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #user defined
[root@lab-dm-cn4 ~]# DMSETUP=/home/jagan/src/LVM2.2.02.88/tools/dmsetup
[root@lab-dm-cn4 ~]# LUN=1
[root@lab-dm-cn4 ~]# META_DEVICE_START_SECTOR=0 #on $DEVICE_PATH this is where meta data starts
[root@lab-dm-cn4 ~]# DEVICE_PATH=/dev/ram
[root@lab-dm-cn4 ~]# let META_DEV_SIZE=8*$ONEMEG; #8M translated to sectors
[root@lab-dm-cn4 ~]# let DATA_DEV_SIZE=3*1024*$ONEMEG; #3G translated to sectors
[root@lab-dm-cn4 ~]# let PAGESIZE=8*$ONEMEG #page size is 8M in sectors
[root@lab-dm-cn4 ~]# let DD_BS=4*1024 # dd block size = 4k bytes
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #system defined
[root@lab-dm-cn4 ~]# let DATA_DEVICE_START_SECTOR=$META_DEVICE_START_SECTOR +$META_DEV_SIZE
[root@lab-dm-cn4 ~]# let PAGES=$DATA_DEV_SIZE/$PAGESIZE
[root@lab-dm-cn4 ~]# let SEEK=$PAGESIZE*512/$DD_BS # how many blocks we need to seek in dd to go to next page
[root@lab-dm-cn4 ~]# let BLOCKS=$DATA_DEV_SIZE*512/$DD_BS
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# echo $DMSETUP LUNID=$LUN DEVICE=$DEVICE_PATH DEVICE_START=$META_DEVICE_START_SECTOR META_DEV_SIZE=$META_DEV_SIZE DATA_DEV_SIZE=$DATA_DEV_SIZE PAGESIZE=$PAGESIZE DD_BS=$DD_BS DATA_DEVICE_START_SECTOR=$DATA_DEVICE_START_SECTOR PAGES=$PAGES SEEK=$SEEK BLOCKS=$BLOCKS
/home/jagan/src/LVM2.2.02.88/tools/dmsetup LUNID=1 DEVICE=/dev/ram DEVICE_START=0 META_DEV_SIZE=16384 DATA_DEV_SIZE=6291456 PAGESIZE=16384 DD_BS=4096 DATA_DEVICE_START_SECTOR=16384 PAGES=384 SEEK=2048 BLOCKS=786432
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# echo 0 $META_DEV_SIZE linear $DEVICE_PATH $META_DEVICE_START_SECTOR | $DMSETUP create tp_meta_disk$LUN
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #zero out first 4k of meta data disk
[root@lab-dm-cn4 ~]# dd if=/dev/zero of=/dev/mapper/tp_meta_disk$LUN bs=4096 count=1
1+0 records in
1+0 records out
4096 bytes (4.1 kB) copied, 3.9319e-05 s, 104 MB/s
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #create data disk
[root@lab-dm-cn4 ~]# echo 0 $DATA_DEV_SIZE linear $DEVICE_PATH $DATA_DEVICE_START_SECTOR | $DMSETUP create tp_data_disk$LUN
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #create thin pool
[root@lab-dm-cn4 ~]# $DMSETUP create pool$LUN --table "0 $DATA_DEV_SIZE thin-pool /dev/mapper/tp_meta_disk$LUN /dev/mapper/tp_data_disk$LUN $PAGESIZE 0 " #1G page with 100 with no water mark
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #verify that 0 pages are allocated
[root@lab-dm-cn4 ~]# $DMSETUP status pool$LUN
0 6291456 thin-pool 0 9/2048 0/384 -
[root@lab-dm-cn4 ~]# $DMSETUP message /dev/mapper/pool$LUN 0 "create_thin 0"
[root@lab-dm-cn4 ~]# $DMSETUP create thin$LUN --table "0 $DATA_DEV_SIZE thin /dev/mapper/pool$LUN 0"
[root@lab-dm-cn4 ~]# #by now no pages should be allocated
[root@lab-dm-cn4 ~]# $DMSETUP status pool$LUN
0 6291456 thin-pool 0 10/2048 0/384 -
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #touch all pages on thin provisoned data disk so that all pages are allocated
[root@lab-dm-cn4 ~]# i=0;while [ $i -lt $PAGES ]; do let j=$i*$SEEK; echo $i; dd if=/dev/zero of=/dev/mapper/thin$LUN bs=4096 seek=$j count=1; let i=$i+1; done >&/dev/null
[root@lab-dm-cn4 ~]# #by now all pages should be allocated
[root@lab-dm-cn4 ~]# $DMSETUP status pool$LUN
0 6291456 thin-pool 0 18/2048 384/384 -
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #note the time on thin device
[root@lab-dm-cn4 ~]# dd of=/dev/null if=/dev/mapper/thin1 bs=4096 count=$BLOCKS
786432+0 records in
786432+0 records out
3221225472 bytes (3.2 GB) copied, 18.2166 s, 177 MB/s
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]#
[root@lab-dm-cn4 ~]# #cleanup
[root@lab-dm-cn4 ~]# $DMSETUP remove thin$LUN
[root@lab-dm-cn4 ~]# $DMSETUP message /dev/mapper/pool$LUN 0 "delete 0"
[root@lab-dm-cn4 ~]# $DMSETUP remove pool$LUN
[root@lab-dm-cn4 ~]# $DMSETUP remove tp_data_disk$LUN
[root@lab-dm-cn4 ~]# $DMSETUP remove tp_meta_disk$LUN
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-16-2012, 11:42 AM
Joe Thornber
 
Default dm-thin vs lvm performance

Hi Jagan,

On Thu, Jan 12, 2012 at 04:55:16PM -0800, Jagan Reddy wrote:
> Hi,

> I *recently started using dm-thin module and first of all, thank you
> for the good work. It works great, well documented and the comments
> in the code are very useful. *I tried to run some performance tests
> using a high performance flash device and the performance of dm thin
> is about 30% of LVM performance on a full allocated thin provisioned
> volume. ( ie after all pages/blocks are allocated using
> dd). Performance test does only reads and no writes.*

Thanks very much for taking the time to try thinp out properly.
People testing different scenarios is very useful to us.

You're probably aware that the thinp test suite is available here:

https://github.com/jthornber/thinp-test-suite

I've added a little set of tests that recreate your scenario here:

https://github.com/jthornber/thinp-test-suite/blob/master/ramdisk_tests.rb

I used a 2G ramdisk for these tests, and a variety of thinp block
sizes and 'dd' options. I'll just summarise the main results, also I
should point out that my testing was done on a VM hosted on a 4G
machine, so the machine was under a lot of memory pressure and there
was a lot of variance in the benchmarks.

writes across various volumes
-----------------------------

write1
------

Testing write performance.

dd if=/dev/zero of=/dev/mapper/<thin> oflags=direct bs=64M

zeroing new blocks turned on.

thinp block size = 64k

| Linear | 2.2 G/s |
| Unprovisioned thin | 1.4 G/s |
| Provisioned thin | 1.9 |
| Snap totally shared | 1.5 G/s |
| Snap no sharing | 1.9 |

Pretty good. Not showing the drastic drop that you were seeing. The
small thinp block size means the snaps perform nicely (not many
copy-on-writes).

write2
------

As test1, but with 8M thinp block size as in your tests.

| Linear | 2.2 G/s |
| Unprovisioned thin | 1.5 G/s |
| Provisioned thin | 2.2 |
| Snap totally shared | 882 Ms |
| Snap no sharing | 2.2 |

Good results, breaking sharing performance is down because the large
block size mean there will be more actual copying incurred.

write3
------

As test2, but no oflags=direct option to dd.

| Linear | 900 M/s |
| Unprovisioned thin | 579 M/s |
| Provisioned thin | 694 M/s |
| Snap totally shared | 510 M/s |
| Snap no sharing | 654 M/s |

Alarming. Results are similar for thinp block size of 64k.

read1
-----

Testing read performance.

dd if=/dev/mapper/<thin> of=/dev/null iflags=direct bs=64M

thinp block size = 64k

| Linear | 3.3 G/s |
| Provisioned thin | 2.7 G/s |
| Snap no sharing | 2.8 G/s |


read2
-----

read1 but with 8M thinp block size

| Linear | 3.3 G/s |
| Provisioned thin | 3.2 G/s |
| Snap no sharing | 3.3 G/s |

read3
-----

As read2, but without the iflags=direct option to 'dd'.

| Linear | 1.0 G/s |
| Provisioned thin | 594 M/s |
| Snap no sharing | 605 M/s |



I think there are a couple of conclusions we can draw from this:

i) dd isn't great for benchmarking block devices
ii) if you are going to use it, then make sure you use O_DIRECT

Using an instrumented kernel, I've confirmed that these read tests
are taking the fast path through the code. The mapping tables are all
in memory. The bio_map function is always returning DM_MAPIO_REMAPPED.

Will you be using a filesystem on these block devices?

- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-18-2012, 06:30 PM
Jagan Reddy
 
Default dm-thin vs lvm performance

Joe,*Thanks for looking into the issue and running the tests and suggesting to use "direct" flag. I do see a difference with "direct" flag using dd. However the difference is significant when using bs=64M compared to bs=4k.*
dd-blocksize *flags *dm-thin * * LVMoutput **64M
* * * * * * * *none * 179 MB/s * * * * 1GB/s * * * * **64M * * * * * * * *direct * 2.4GB/s * * * * * 3.6GB/s * * * *4k * * * * * * * * * none * 179 MB/s * * * * *965MB/s * * *4k * * * * * * * * * direct * 193MB/s * * * * *1.7GB/s * * * *
In all the tests, dm-thin performance is below lvm.*I used 8M pages as we are not planning to use snapshots right now. *
We are planning to use the dm-thin module in a high end flash device that supports upto 300k IOPS/second. Hence *we are doing some performance tests. I wanted to try more parallel async IO instead of dd. A good program I found on web is*

fsbench.filesystems.org/bench/aio-stress.c

(if you are aware of other tools to test parallel async io using multiple threads, please let me know. I will give it a try.)

I compiled it with libaio and libpth.*
Then I ran the test on LVM lun. Throughput is 9563.56MB/s.*Fully
allocated TP LUN is*5399.81MB/s
I used "perf top" to see where most of the time is spent. Lot of time is spent in*_raw_spin_lock. I poked the code little bit and I see that*dm_thin_find_block returns EWOULDBLOCK and hence the bio gets deferred. It looks like spin locks are used when putting the bio on deferred list and getting it back from the deferred list. I see that EWOULDBLOCK is returned from*dm_thin_find_block ->*dm_btree_lookup->btree_lookup_raw->ro_step->bn_read_lock->dm_tm_read_lock->dm_bm_read_try_lock->bl_down_read_nonblock.
I tried to change the code little bit so
that*dm_thin_find_block uses array as a cache and looks up btree only if not found in the array.With that change, I see TP performance go way up.
Fully allocated TP LUN with array lookup is 9387.17MB/s. With array, no bios are being deferred. However when I used the array, I see kernel hang after several runs ( yet to debug the reason).
Code changes are initialize array in pool_ctr() in dm-thin.c
extern dm_block_t
blockmap[1000];
printk("blockmap array inited");** * * *for(i=0;i<1000;i++)** * * *{** * * * *blockmap[i]=0xffffffff;** * * *}

dm_thin_find_block() in dm-thin-metadata.c
at the begining:** * * if(blockmap[block]==0xffffffff)** * * *{** * * * * * * *printk("found unassigned block %llu
",block);** * * *}** * * *else** * * *{** * * * * * * *result->block =
blockmap[block];** * * * * * * *result->shared = 0;** * * * * * * *return 0;** * * *}
at the end before returning:** * * if(r == 0)** * * *{** * * * * * * *blockmap[block] = result->block ;** * * *}

Note that the array changes are only for my testing and not generic enough at all.
*Command I used for getting the throughput numbers are
i=0; while [ $i -lt 100 ] ; do ./aio-stress.exe *-O -o 1 -c 16 -t 16 -d 256 /dev/mapper/thin1 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done *| awk 'BEGIN{i=0.0} {i+=$0} END{print i/100}'

Thanks,Jagan.
From: Joe Thornber <thornber@redhat.com>
To: Jagan Reddy <gjmsreddy@yahoo.com>; device-mapper development <dm-devel@redhat.com>
Sent: Monday, January 16, 2012 4:42 AM
Subject: Re: [dm-devel] dm-thin vs lvm performance


Hi Jagan,

On Thu, Jan 12, 2012 at 04:55:16PM -0800, Jagan Reddy wrote:
> Hi,

> I *recently started using dm-thin module and first of all, thank you
> for the good work. It works great, well documented and the comments
> in the code are very useful. *I tried to run some performance tests
> using a high performance flash device and the performance of dm thin
> is about 30% of LVM performance on a full allocated thin provisioned
> volume. ( ie after all pages/blocks are allocated using
> dd). Performance test does only reads and no writes.*

Thanks very much for taking the time to try thinp out properly.
People testing different scenarios is very useful to us.

You're probably aware that the thinp test suite is available here:

* * https://github.com/jthornber/thinp-test-suite

I've added a little set of tests that recreate your scenario here:

* * https://github.com/jthornber/thinp-test-suite/blob/master/ramdisk_tests.rb

I used a 2G ramdisk for these tests, and a variety of thinp block
sizes and 'dd' options.* I'll just summarise the main results, also I
should point out that my testing was done on a VM hosted on a 4G
machine, so the machine was under a lot of memory pressure and there
was a lot of variance in the benchmarks.

writes across various volumes
-----------------------------

write1
------

Testing write performance.

dd if=/dev/zero of=/dev/mapper/<thin> oflags=direct bs=64M

zeroing new blocks turned on.

thinp block size = 64k

| Linear*
* * * * * * | 2.2 G/s |
| Unprovisioned thin* | 1.4 G/s |
| Provisioned thin* * | 1.9* * |
| Snap totally shared | 1.5 G/s |
| Snap no sharing* * | 1.9* * |

Pretty good.* Not showing the drastic drop that you were seeing.* The
small thinp block size means the snaps perform nicely (not many
copy-on-writes).

write2
------

As test1, but with 8M thinp block size as in your tests.

| Linear* * * * * * * | 2.2 G/s |
| Unprovisioned thin* | 1.5 G/s |
| Provisioned thin* * | 2.2* * |
| Snap totally shared | 882 Ms* |
| Snap no sharing* * | 2.2* * |

Good results, breaking sharing performance is down because the large
block size mean there will be more actual copying incurred.

write3
------

As test2, but
no oflags=direct option to dd.

| Linear* * * * * * * | 900 M/s |
| Unprovisioned thin* | 579 M/s |
| Provisioned thin* * | 694 M/s |
| Snap totally shared | 510 M/s |
| Snap no sharing* * | 654 M/s |

Alarming.* Results are similar for thinp block size of 64k.

read1
-----

Testing read performance.

dd if=/dev/mapper/<thin> of=/dev/null iflags=direct bs=64M

thinp block size = 64k

| Linear* * * * * * * | 3.3 G/s |
| Provisioned thin* * | 2.7 G/s |
| Snap no sharing* * | 2.8 G/s |


read2
-----

read1 but with 8M thinp block size

| Linear* * * * * * * | 3.3 G/s |
| Provisioned thin* * | 3.2 G/s |
| Snap no sharing* * | 3.3 G/s |

read3
-----

As read2, but
without the iflags=direct option to 'dd'.

| Linear* * * * * * * | 1.0 G/s |
| Provisioned thin* * | 594 M/s |
| Snap no sharing* * | 605 M/s |



I think there are a couple of conclusions we can draw from this:

i) dd isn't great for benchmarking block devices
ii) if you are going to use it, then make sure you use O_DIRECT

Using an instrumented kernel, I've confirmed that these read tests
are taking the fast path through the code.* The mapping tables are all
in memory.* The bio_map function is always returning DM_MAPIO_REMAPPED.

Will you be using a filesystem on these block devices?

- Joe


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-20-2012, 04:03 PM
Joe Thornber
 
Default dm-thin vs lvm performance

Hi Jagan,

On Wed, Jan 18, 2012 at 11:30:54AM -0800, Jagan Reddy wrote:
> Joe,
> *Thanks for looking into the issue and running the tests and suggesting to use "direct" flag. I do see a difference with "direct" flag using dd. However the difference is significant when using bs=64M compared to bs=4k.*

I've spent a couple of days tinkering with aio-stress and thinp on
ramdisks. More tests can be found here:

https://github.com/jthornber/thinp-test-suite/blob/master/ramdisk_tests.rb

It appears that wiping the device (ie. to ensure total allocation) is
causing the issue, and what's more this is a more general problem than
just thinp.

For instance see this test:

def test_linear_aio_stress
linear_table = Table.new(Linear.new(@volume_size, @data_dev, 0))
@dm.with_dev(linear_table) do |linear_dev|
aio_stress(linear_dev)
wipe_device(linear_dev) # cause slow down
aio_stress(linear_dev)
end
end

For me, the first run of aio_stress manages a throughput of ~9G/s.
After the wipe, which is just a simple dd across the device,
performance drops to ~5.5 G/s. Also throughput on the device under
the linear target also drops. Permanently.

I don't know if this is specific to aio, or a more general slowdown.

Once we have got to the bottom of this I've written a couple of
experimental patches that we can try to boost read performance
further.

- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-20-2012, 04:33 PM
Joe Thornber
 
Default dm-thin vs lvm performance

On Fri, Jan 20, 2012 at 05:03:35PM +0000, Joe Thornber wrote:
> For me, the first run of aio_stress manages a throughput of ~9G/s.
> After the wipe, which is just a simple dd across the device,
> performance drops to ~5.5 G/s. Also throughput on the device under
> the linear target also drops. Permanently.
>
> I don't know if this is specific to aio, or a more general slowdown.

It's just me being stupid. For the initial pass the ramdisk has no
memory backing it and is just returning zeroes.

Back to testing ...

- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-20-2012, 05:07 PM
Joe Thornber
 
Default dm-thin vs lvm performance

Jagan,

On Fri, Jan 20, 2012 at 05:33:34PM +0000, Joe Thornber wrote:
> Back to testing ...

Here are my aio-stress results for various devices running on top of a
ramdisk:

| Device stack | M/s |
+------------------------+------+
| raw ramdisk | 5440 |
| linear | 5431 |
| 2 stacked linear | 5304 |
| pool device | 5351 |
| linear stacked on pool | 5243 |
| thin | 5375 |

I also tried the thin test after disabling the block locking in
dm-block-manager.c, and using a 128 entry cache local to the thin
device. Neither made any difference to the 'thin' result.

Things for you to check:

i) Are you benchmarking with a ramdisk, or your flash device? In both
cases I think you need to make sure the device has allocated backing
store before you run any tests. Remember that Discards may well
removing this backing.

ii) You claim that io is getting deferred. This will happen when you do
the initial wipe of the thin device to force provisioning. But for
the aio-stress tests you should get nothing deferred (I instrumented
to confirm this).


- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-20-2012, 08:06 PM
Jagan Reddy
 
Default dm-thin vs lvm performance

Joe,Thanks for spending time with aio_stress(much apperciated) and the information.** A week back I ran the tests on flash device and saw lower numbers. However all my runs since last week have been with ramdisk only. I run the following command to make sure ramdisk has backing store.
[root@lab-dm-cn4 ~]# dd if=/dev/zero of=/dev/ram bs=512 oflag=direct count=`blockdev --getsize /dev/ram`
8388608+0 records in
8388608+0 records out
4294967296 bytes (4.3 GB) copied, 19.2371 s, 223 MB/s

One interesting thing I notice is raw ramdisk has 9000+MB/s throughput in my tests and 5440MB/s throughput in your tests. I remember you mentioning that you are running tests in a VM with 4G memory,
while I run the rests on a standalone server with 16 quad-core processors and 12G or ram (out of that 12G, I carve out 4G ramdisk). Attached are the cpu and memory information. Could that be causing an issue?

Thanks,
Jagan.

From: Joe Thornber <thornber@redhat.com>
To: Jagan Reddy <gjmsreddy@yahoo.com>; device-mapper development <dm-devel@redhat.com>
Sent: Friday, January 20, 2012 10:07 AM
Subject: Re: [dm-devel] dm-thin vs lvm performance


Jagan,

On Fri, Jan 20, 2012 at 05:33:34PM +0000, Joe Thornber wrote:
> Back to testing ...

Here are my aio-stress results for various devices running on top of a
ramdisk:

| Device stack* * * * * | M/s* |
+------------------------+------+*
| raw ramdisk* * * * * * | 5440 |
| linear* * * * * * * * | 5431 |
| 2 stacked linear* * * | 5304 |
| pool device* * * * * * | 5351 |
| linear stacked on pool | 5243 |
| thin* * * * * * * * * | 5375 |

I also tried the thin test after disabling the block locking in
dm-block-manager.c, and using a 128 entry cache local to the thin
device.* Neither made any difference to the 'thin' result.

Things for you to check:

i)* Are you benchmarking
with a ramdisk, or your flash device?* In both
* * cases I think you need to make sure the device has allocated backing
* * store before you run any tests.* Remember that Discards may well
* * removing this backing.

ii) You claim that io is getting deferred.* This will happen when you do
* * the initial wipe of the thin device to force provisioning.* But for
* * the aio-stress tests you should get nothing deferred (I instrumented
* * to confirm this).


- Joe


--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-22-2012, 07:24 PM
Joe Thornber
 
Default dm-thin vs lvm performance

On Fri, Jan 20, 2012 at 01:06:08PM -0800, Jagan Reddy wrote:
> One interesting thing I notice is raw ramdisk has 9000+MB/s throughput in my tests and 5440MB/s throughput in your tests.

Yes, and if the ramdisk is unallocated the throughput tests on my machine also give ~9G/s throughput. Can you please double check that it's allocated. eg, are you running all the tests on the same ram disk? If you run the thin test first and then the raw test does the raw performance change?

> I remember you mentioning that you are running tests in a VM with 4G
> memory, while I run the rests on a standalone server with 16
> quad-core processors and 12G or ram (out of that 12G, I carve out
> 4G ramdisk). Attached are the cpu and memory information. Could
> that be causing an issue?

Well memory performance will have an effect. But I'd expect your
number to be consistent with mine.

- Joe

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-24-2012, 10:58 PM
Jagan Reddy
 
Default dm-thin vs lvm performance

Joe,*I was running more tests to get more data.
*To answer your questions, I always use dd to write zeroes to ramdisk( you can also see this in the script I attached). So the ramdisk is allocated. I use
dd if=/dev/zero of=/dev/ram oflag=direct bs=512 count=`blockdev --getsize /dev/ram`
I get 9000+MB/s on allocated ramdisk using aio_stress. I was running the tests on same ramdisk. The performance doesn't change if i run thin test first and then raw test.
I modified the test to divide 4G ramdisk into 2 ramdisks. ( ramdisk1=2G ramdisk2=2G). Then I create tp lun on ramdisk1 and lvm lun on ramdisk2.


Numbers I see

ramdisk 10125 MB/s
ramdisk1 9690.55
ramdisk2 9548.89
unallocated tp lun 2257.67
fully allocated tp lun 5742.14
ramdisk 10288.2

ramdisk1 9721.8

ramdisk2 9415.25

Data not written lvm lun 9258.24

Data written lvm lun 9126.04
I have attached the script I use and the output. I wonder if* using 16 quad core processor and doing lot of of parallel operations is causing the issue to be reproduced.
Thanks,Jagan.



From: Joe Thornber <thornber@redhat.com>
To: Jagan Reddy <gjmsreddy@yahoo.com>
Cc: device-mapper development <dm-devel@redhat.com>
Sent: Sunday, January 22, 2012 12:24 PM
Subject: Re: [dm-devel] dm-thin vs lvm performance


On Fri, Jan 20, 2012 at 01:06:08PM -0800, Jagan Reddy wrote:
> One interesting thing I notice is raw ramdisk has 9000+MB/s throughput in my tests and 5440MB/s throughput in your tests.

Yes, and if the ramdisk is unallocated the throughput tests on my machine also give ~9G/s throughput.* Can you please double check that it's allocated.* eg, are you running all the tests on the same ram disk?* If you run the thin test first and then the raw test does the raw performance change?

> I remember you mentioning that you are running tests in a VM with 4G
> memory, while I run the rests on a standalone server with 16
> quad-core processors and 12G or ram (out of that 12G, I carve out
> 4G ramdisk). Attached are the cpu and memory information. Could
> that be causing an issue?

Well memory performance will have an effect.* But I'd expect your
number to be consistent with mine.

-
Joe


[root@lab-dm-cn4 md]# DEVICE_PATH=/dev/ram
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# rm -f $DEVICE_PATH
[root@lab-dm-cn4 md]# mknod -m 660 $DEVICE_PATH b 1 1
[root@lab-dm-cn4 md]# #check the size of ramdisk
[root@lab-dm-cn4 md]# #it should be 4G as we booted the kernel with ramdisk_size=4194304
[root@lab-dm-cn4 md]# echo checking ramdisk size
checking ramdisk size
[root@lab-dm-cn4 md]# blockdev --getsize $DEVICE_PATH
8388608
[root@lab-dm-cn4 md]# echo zeroing out ramdisk
zeroing out ramdisk
[root@lab-dm-cn4 md]# dd if=/dev/zero of=$DEVICE_PATH oflag=direct bs=512 count=`blockdev --getsize $DEVICE_PATH`
8388608+0 records in
8388608+0 records out
4294967296 bytes (4.3 GB) copied, 13.8445 s, 310 MB/s
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# #let's divide the ramdisk into 2 linear ramdisks of 2G in size
[root@lab-dm-cn4 md]# let fullramdisksize=`blockdev --getsize /dev/ram`
[root@lab-dm-cn4 md]# let ramdisksize=$fullramdisksize/2
[root@lab-dm-cn4 md]# echo 0 $ramdisksize linear $DEVICE_PATH 0 | $DMSETUP create ramdisk1
[root@lab-dm-cn4 md]# echo 0 $ramdisksize linear $DEVICE_PATH $ramdisksize | $DMSETUP create ramdisk2
[root@lab-dm-cn4 md]# echo checking ramdisk1 size
checking ramdisk1 size
[root@lab-dm-cn4 md]# blockdev --getsize /dev/mapper/ramdisk1
4194304
[root@lab-dm-cn4 md]# echo checking ramdisk2 size
checking ramdisk2 size
[root@lab-dm-cn4 md]# blockdev --getsize /dev/mapper/ramdisk2
4194304
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# #constants
[root@lab-dm-cn4 md]# let ONEMEG=1024*2; #translated to sectors
[root@lab-dm-cn4 md]# DEVICE_PATH=/dev/mapper/ramdisk1
[root@lab-dm-cn4 md]# #user defined
[root@lab-dm-cn4 md]# DMSETUP=/home/jagan/src/LVM2.2.02.88/tools/dmsetup
[root@lab-dm-cn4 md]# LUN=1
[root@lab-dm-cn4 md]# META_DEVICE_START_SECTOR=0 #on $DEVICE_PATH this is where meta data starts
[root@lab-dm-cn4 md]# let META_DEV_SIZE=8*$ONEMEG; #8M translated to sectors
[root@lab-dm-cn4 md]# let DATA_DEV_SIZE=1*1024*$ONEMEG; #1G translated to sectors
[root@lab-dm-cn4 md]# let PAGESIZE=64*2 #page size is 64k in sectors
[root@lab-dm-cn4 md]# let DD_BS=4*1024 # dd block size = 4k bytes
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# #system defined
[root@lab-dm-cn4 md]# let DATA_DEVICE_START_SECTOR=$META_DEVICE_START_SECTOR +$META_DEV_SIZE
[root@lab-dm-cn4 md]# let PAGES=$DATA_DEV_SIZE/$PAGESIZE
[root@lab-dm-cn4 md]# let SEEK=$PAGESIZE*512/$DD_BS # how many blocks we need to seek in dd to go to next page
[root@lab-dm-cn4 md]# let BLOCKS=$DATA_DEV_SIZE*512/$DD_BS
[root@lab-dm-cn4 md]# noofruns=20
[root@lab-dm-cn4 md]# AIO_PATH=/root/aio-stress.exe
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo $DMSETUP LUNID=$LUN DEVICE=$DEVICE_PATH DEVICE_START=$META_DEVICE_START_SECTOR META_DEV_SIZE=$META_DEV_SIZE DATA_DEV_SIZE=$DATA_DEV_SIZE PAGESIZE=$PAGESIZE DD_BS=$DD_BS DATA_DEVICE_START_SECTOR=$DATA_DEVICE_START_SECTOR PAGES=$PAGES SEEK=$SEEK BLOCKS=$BLOCKS
/home/jagan/src/LVM2.2.02.88/tools/dmsetup LUNID=1 DEVICE=/dev/mapper/ramdisk1 DEVICE_START=0 META_DEV_SIZE=16384 DATA_DEV_SIZE=2097152 PAGESIZE=128 DD_BS=4096 DATA_DEVICE_START_SECTOR=16384 PAGES=16384 SEEK=16 BLOCKS=262144
[root@lab-dm-cn4 md]# echo 0 $META_DEV_SIZE linear $DEVICE_PATH $META_DEVICE_START_SECTOR | $DMSETUP create tp_meta_disk$LUN
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# #zero out first 4k of meta data disk
[root@lab-dm-cn4 md]# dd if=/dev/zero of=/dev/mapper/tp_meta_disk$LUN bs=4096 count=1
1+0 records in
1+0 records out
4096 bytes (4.1 kB) copied, 3.7874e-05 s, 108 MB/s
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# #create data disk
[root@lab-dm-cn4 md]# echo 0 $DATA_DEV_SIZE linear $DEVICE_PATH $DATA_DEVICE_START_SECTOR | $DMSETUP create tp_data_disk$LUN
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# #create thin pool
[root@lab-dm-cn4 md]# $DMSETUP create pool$LUN --table "0 $DATA_DEV_SIZE thin-pool /dev/mapper/tp_meta_disk$LUN /dev/mapper/tp_data_disk$LUN $PAGESIZE 0 " #1G page with 100 with no water mark
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# #verify that 0 pages are allocated
[root@lab-dm-cn4 md]# echo verify that 0 pages are allocated
verify that 0 pages are allocated
[root@lab-dm-cn4 md]# $DMSETUP status pool$LUN
0 2097152 thin-pool 0 10/2048 0/16384 -
[root@lab-dm-cn4 md]# $DMSETUP message /dev/mapper/pool$LUN 0 "create_thin 0"
[root@lab-dm-cn4 md]# $DMSETUP create thin$LUN --table "0 $DATA_DEV_SIZE thin /dev/mapper/pool$LUN 0"
[root@lab-dm-cn4 md]# echo by now no pages should be allocated
by now no pages should be allocated
[root@lab-dm-cn4 md]# $DMSETUP status pool$LUN
0 2097152 thin-pool 0 11/2048 0/16384 -
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# DEVICE_PATH=/dev/mapper/ramdisk2
[root@lab-dm-cn4 md]# let LVM_DEVICE_SIZE=$META_DEV_SIZE+$DATA_DEV_SIZE;
[root@lab-dm-cn4 md]# let BLOCKS=($DATA_DEV_SIZE-$PAGESIZE)*512/$DD_BS
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# #system defined
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo $DMSETUP LUNID=$LUN DEVICE=$DEVICE_PATH DEVICE_START=$META_DEVICE_START_SECTOR META_DEV_SIZE=$META_DEV_SIZE DATA_DEV_SIZE=$DATA_DEV_SIZE PAGESIZE=$PAGESIZE DD_BS=$DD_BS DATA_DEVICE_START_SECTOR=$DATA_DEVICE_START_SECTOR PAGES=$PAGES SEEK=$SEEK BLOCKS=$BLOCKS LVM_DEVICE_SIZE=$LVM_DEVICE_SIZE
/home/jagan/src/LVM2.2.02.88/tools/dmsetup LUNID=1 DEVICE=/dev/mapper/ramdisk2 DEVICE_START=0 META_DEV_SIZE=16384 DATA_DEV_SIZE=2097152 PAGESIZE=128 DD_BS=4096 DATA_DEVICE_START_SECTOR=16384 PAGES=16384 SEEK=16 BLOCKS=262128 LVM_DEVICE_SIZE=2113536
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# pvcreate -M2 --dataalignment 4K --metadatacopies 1 --metadatasize 8M --zero y -v /dev/mapper/ramdisk2
Set up physical volume for "/dev/mapper/ramdisk2" with 4194304 available sectors
Zeroing start of device /dev/mapper/ramdisk2
Writing physical volume data to disk "/dev/mapper/ramdisk2"
Physical volume "/dev/mapper/ramdisk2" successfully created
[root@lab-dm-cn4 md]# vgcreate -l 16384 -s 64k -c n -A n 6C067CWX00132-1 /dev/mapper/ramdisk2
WARNING: This metadata update is NOT backed up
Volume group "6C067CWX00132-1" successfully created
[root@lab-dm-cn4 md]# lvcreate -A n -C n -l 100%VG -Z y -n engineering -r None 6C067CWX00132-1
Using reduced mirror region size of 128 sectors
WARNING: This metadata update is NOT backed up
WARNING: This metadata update is NOT backed up
Logical volume "engineering" created
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of ramdisk
measuring performance of ramdisk
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/ram 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
10125
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of ramdisk1
measuring performance of ramdisk1
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/mapper/ramdisk1 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
9690.55
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of ramdisk2
measuring performance of ramdisk2
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/mapper/ramdisk2 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
9548.89
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of un-allocated tp lun
measuring performance of un-allocated tp lun
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/mapper/thin$LUN 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
2257.67
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo touching all pages in tp lun to make it fully allocated
touching all pages in tp lun to make it fully allocated
[root@lab-dm-cn4 md]# i=0;while [ $i -lt $PAGES ]; do let j=$i*$SEEK; echo $i; dd if=/dev/zero of=/dev/mapper/thin$LUN bs=4096 seek=$j count=1; let i=$i+1; done >&/dev/null
[root@lab-dm-cn4 md]# echo now all pages should be allocated
now all pages should be allocated
[root@lab-dm-cn4 md]# #touch all pages on thin provisoned data disk so that all pages are allocated
[root@lab-dm-cn4 md]# $DMSETUP status pool$LUN
0 2097152 thin-pool 0 147/2048 16384/16384 -
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of fully-allocated tp lun
measuring performance of fully-allocated tp lun
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/mapper/thin$LUN 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
5742.14
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of ramdisk
measuring performance of ramdisk
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/ram 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
10288.2
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of ramdisk1
measuring performance of ramdisk1
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/mapper/ramdisk1 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
9721.8
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of ramdisk2
measuring performance of ramdisk2
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/mapper/ramdisk2 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
9415.25
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of data-not-written lvm lun
measuring performance of data-not-written lvm lun
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/mapper/6C067CWX00132--1-engineering 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
9258.24
[root@lab-dm-cn4 md]# #write something
[root@lab-dm-cn4 md]# dd if=/dev/zero of=/dev/mapper/6C067CWX00132--1-engineering bs=4096 count=$BLOCKS
262128+0 records in
262128+0 records out
1073676288 bytes (1.1 GB) copied, 1.96007 s, 548 MB/s
[root@lab-dm-cn4 md]#
[root@lab-dm-cn4 md]# echo measuring performance of data-written lvm lun
measuring performance of data-written lvm lun
[root@lab-dm-cn4 md]# i=0; while [ $i -lt $noofruns ] ; do $AIO_PATH -O -o 1 -c 16 -t 16 -d 256 /dev/mapper/6C067CWX00132--1-engineering 2>&1 | grep throughput | cut -f3 -d ' ' | cut -f2 -d '(' | cut -f1 -d ' ' ; let i=$i+1; done | awk -v runs=$noofruns 'BEGIN{i=0.0} {i+=$0} END{print i/runs}'
9126.04
[root@lab-dm-cn4 md]#
--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 01-27-2012, 01:58 PM
Joe Thornber
 
Default dm-thin vs lvm performance

On Tue, Jan 24, 2012 at 03:58:35PM -0800, Jagan Reddy wrote:
> Joe,
> *I was running more tests to get more data.

Could I trouble you to grab my test suite [1], add your machine to the
config.rb file and then:

./run_tests -t /RamDisk/

and see if those results are different too please?

- Joe


[1] https://github.com/jthornber/thinp-test-suite

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 05:03 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org