FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian ISP

 
 
LinkBack Thread Tools
 
Old 08-10-2011, 11:03 AM
mimo
 
Default slow lvm

Hi, I've been having problems with a really slow lvm for a while. I think it started after I had replaced a broken hard disk and done pvmove. I've checked the drives with hdparm and they all seem normal. I've moved all the drives to a similar box and it's still equally slow. I've even added a new drive and created a lvm just on that and it still is very slow and just shows 100% utilisation of the drive but very little throughput (max 30k blocks/second according to munin).
I think the only think all of the configurations that I've tried have in common is that I started off with a linux software raid 1 with one missing drive. So here's my disk layout.

md4 : active raid1 sdd4[0]
1885150680 blocks super 1.2 [2/1] [U_]

--- Logical volume ---
LV Name /dev/vg0/lv0
VG Name vg0
LV UUID BEGmO7-Oi4O-DID5-4qs4-pvMp-1nnD-sHDz53
LV Write Access read/write
LV Status available
# open 1
LV Size 1.76 TiB
Current LE 460241
Segments 1
Allocation inherit
Read ahead sectors auto
- currently set to 256
Block device 253:1

This is a fresh install of Debian squeeze. Before changing the disk it was Lenny. Any help really appreciate. I've spent about a month on this now.

Thanks,

mimo


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201108101203.51095.mimo@restoel.net">http://lists.debian.org/201108101203.51095.mimo@restoel.net
 
Old 08-11-2011, 06:48 AM
Keith Edmunds
 
Default slow lvm

Do the disks have 1K sectors or 4K sectors?

--
"You can have everything in life you want if you help enough other people
get what they want" - Zig Ziglar.

Who did you help today?


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110811074845.36c1a10c@ws.the.cage">http://lists.debian.org/20110811074845.36c1a10c@ws.the.cage
 
Old 08-11-2011, 08:22 AM
Matus UHLAR - fantomas
 
Default slow lvm

On 11.08.11 07:48, Keith Edmunds wrote:

Do the disks have 1K sectors or 4K sectors?


you apparently mean 512B or 4K. Note that some disks say they have 512B
while they have 4K (but you can't find informations about that).


disks with "advanced disk format" have 4K sectors. Many other newre
have it too


e.g. seagate es.2 1T seem to have 4K sectors too, although I haven't
found any mention of that


(badblocks in write mode with cuoncurrent block count rounded to 4K was
much faster than not-rounded)


--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Microsoft dick is soft to do no harm


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110811082220.GB671@fantomas.sk">http://lists.debian.org/20110811082220.GB671@fantomas.sk
 
Old 08-11-2011, 10:37 AM
mimo
 
Default slow lvm

On Thursday 11 August 2011 07:48:45 Keith Edmunds wrote:
> Do the disks have 1K sectors or 4K sectors?

It says:

fdisk -l /dev/sdd

Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x0005f6d7

Device Boot Start End Blocks Id System
/dev/sdd1 * 1 1216 9764864 fd Linux raid autodetect
/dev/sdd2 1216 2432 9765888 fd Linux raid autodetect
/dev/sdd3 2432 8511 48828416 fd Linux raid autodetect
/dev/sdd4 8511 243201 1885151840+ fd Linux raid autodetect


And the first one I tried with

fdisk -l /dev/sda

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x000e0666

Device Boot Start End Blocks Id System
/dev/sda1 * 1 1216 9764864 fd Linux raid autodetect
/dev/sda2 1216 2432 9765888 fd Linux raid autodetect
/dev/sda3 2432 8511 48828416 fd Linux raid autodetect
/dev/sda4 8511 243201 1885151840+ fd Linux raid autodetect


mimo


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201108111137.05387.mimo@restoel.net">http://lists.debian.org/201108111137.05387.mimo@restoel.net
 
Old 08-11-2011, 10:50 AM
Matus UHLAR - fantomas
 
Default slow lvm

On Thursday 11 August 2011 07:48:45 Keith Edmunds wrote:

Do the disks have 1K sectors or 4K sectors?


On 11.08.11 11:37, mimo wrote:

fdisk -l /dev/sdd

hmmm, since when does fdisk show that?



Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes


It's 2T, it should be 4k/sec


Sector size (logical/physical): 512 bytes / 4096 bytes


See? It is...


I/O size (minimum/optimal): 4096 bytes / 4096 bytes


that's why partisions should be rounded on 4K limit.


Device Boot Start End Blocks Id System
/dev/sdd1 * 1 1216 9764864 fd Linux raid autodetect
/dev/sdd2 1216 2432 9765888 fd Linux raid autodetect
/dev/sdd3 2432 8511 48828416 fd Linux raid autodetect
/dev/sdd4 8511 243201 1885151840+ fd Linux raid autodetect


the 'fdisk -lu /dev/sdd' will tell you boundaris on sector range.


fdisk -l /dev/sda

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


This disk tells it has 512B sectors, but that may be just configuration
and reality might be different.

Therefore you could try with 4K rounding too.


Disk identifier: 0x000e0666

Device Boot Start End Blocks Id System
/dev/sda1 * 1 1216 9764864 fd Linux raid autodetect
/dev/sda2 1216 2432 9765888 fd Linux raid autodetect
/dev/sda3 2432 8511 48828416 fd Linux raid autodetect
/dev/sda4 8511 243201 1885151840+ fd Linux raid autodetect


--
Matus UHLAR - fantomas, uhlar@fantomas.sk ; http://www.fantomas.sk/
Warning: I wish NOT to receive e-mail advertising to this address.
Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu.
Micro$oft random number generator: 0, 0, 0, 4.33e+67, 0, 0, 0...


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110811105038.GA3732@fantomas.sk">http://lists.debian.org/20110811105038.GA3732@fantomas.sk
 
Old 08-11-2011, 11:52 AM
Bachsau WebWorX
 
Default slow lvm

I hate those. They should just have made it report it's real sector size
and wait for controllers and drivers to support it.


Now, you have to make your partitions to start at megabyte boundaries to
get full speed, so that the sectors oft the file system come to lie on
the physical sectors, or else it will need read out every sector before
writing to it.


Am 11.08.2011 12:50, schrieb Matus UHLAR - fantomas:

On Thursday 11 August 2011 07:48:45 Keith Edmunds wrote:

Do the disks have 1K sectors or 4K sectors?


On 11.08.11 11:37, mimo wrote:

fdisk -l /dev/sdd

hmmm, since when does fdisk show that?



Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes


It's 2T, it should be 4k/sec


Sector size (logical/physical): 512 bytes / 4096 bytes


See? It is...


I/O size (minimum/optimal): 4096 bytes / 4096 bytes


that's why partisions should be rounded on 4K limit.


Device Boot Start End Blocks Id System
/dev/sdd1 * 1 1216 9764864 fd Linux raid autodetect
/dev/sdd2 1216 2432 9765888 fd Linux raid autodetect
/dev/sdd3 2432 8511 48828416 fd Linux raid autodetect
/dev/sdd4 8511 243201 1885151840+ fd Linux raid autodetect


the 'fdisk -lu /dev/sdd' will tell you boundaris on sector range.


fdisk -l /dev/sda

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes
255 heads, 63 sectors/track, 243201 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


This disk tells it has 512B sectors, but that may be just configuration
and reality might be different.
Therefore you could try with 4K rounding too.


Disk identifier: 0x000e0666

Device Boot Start End Blocks Id System
/dev/sda1 * 1 1216 9764864 fd Linux raid autodetect
/dev/sda2 1216 2432 9765888 fd Linux raid autodetect
/dev/sda3 2432 8511 48828416 fd Linux raid autodetect
/dev/sda4 8511 243201 1885151840+ fd Linux raid autodetect





--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4E43C284.8090807@bachsau.com">http://lists.debian.org/4E43C284.8090807@bachsau.com
 
Old 08-11-2011, 01:40 PM
mimo
 
Default slow lvm

On Thursday 11 August 2011 12:52:36 Bachsau WebWorX wrote:
> Now, you have to make your partitions to start at megabyte boundaries to
> get full speed, so that the sectors oft the file system come to lie on
> the physical sectors, or else it will need read out every sector before
> writing to it.

Thanks for everyone's help. I found something about these "Green" drives going to park every 8 secs so I used

hdparm -S 242 /dev/sdd

to change the standby. That brought a tiny improvement. So the next step I deleted the whole lvm and used some other advice by running

cfdisk -h 224 -s 56 /dev/sdd

Which gives a partition table that isn't at all divisible by 8 or start on 2048 but performance had improved quite dramatically. Originally I got 71MB/s when writing one large 20GB file. Writing to a raw partition I now get 117MB/s (also with lvm). Is that what I should be achieving?

I just wonder whether I've got it right now as the partition table looks plain weird..

fdisk -lu /dev/sdd

Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes
224 heads, 56 sectors/track, 311465 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x0005f6d7

Device Boot Start End Blocks Id System
/dev/sdd1 * 56 19531007 9765476 fd Linux raid autodetect
/dev/sdd2 19531008 39062015 9765504 fd Linux raid autodetect
/dev/sdd3 39062016 136717055 48827520 fd Linux raid autodetect
/dev/sdd4 136717056 3907016959 1885149952 fd Linux raid autodetect


mimo


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201108111440.03903.mimo@restoel.net">http://lists.debian.org/201108111440.03903.mimo@restoel.net
 
Old 08-11-2011, 02:22 PM
Bachsau NetWorX
 
Default slow lvm

Why does it look weird to you? Every start sector is divisible by 8 so
you should be fine.


Am 11.08.2011 15:40, schrieb mimo:

On Thursday 11 August 2011 12:52:36 Bachsau WebWorX wrote:

Now, you have to make your partitions to start at megabyte boundaries to
get full speed, so that the sectors oft the file system come to lie on
the physical sectors, or else it will need read out every sector before
writing to it.


Thanks for everyone's help. I found something about these "Green" drives going to park every 8 secs so I used

hdparm -S 242 /dev/sdd

to change the standby. That brought a tiny improvement. So the next step I deleted the whole lvm and used some other advice by running

cfdisk -h 224 -s 56 /dev/sdd

Which gives a partition table that isn't at all divisible by 8 or start on 2048 but performance had improved quite dramatically. Originally I got 71MB/s when writing one large 20GB file. Writing to a raw partition I now get 117MB/s (also with lvm). Is that what I should be achieving?

I just wonder whether I've got it right now as the partition table looks plain weird..

fdisk -lu /dev/sdd

Disk /dev/sdd: 2000.4 GB, 2000398934016 bytes
224 heads, 56 sectors/track, 311465 cylinders, total 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disk identifier: 0x0005f6d7

Device Boot Start End Blocks Id System
/dev/sdd1 * 56 19531007 9765476 fd Linux raid autodetect
/dev/sdd2 19531008 39062015 9765504 fd Linux raid autodetect
/dev/sdd3 39062016 136717055 48827520 fd Linux raid autodetect
/dev/sdd4 136717056 3907016959 1885149952 fd Linux raid autodetect


mimo





--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4E43E5AE.8010003@bachsau.com">http://lists.debian.org/4E43E5AE.8010003@bachsau.com
 
Old 08-30-2011, 03:39 PM
mimo
 
Default slow lvm

Hi again and thanks to everyone who replied to this,


I had done some benchmarking on the individual partitions, md devices, and lvm device and it all seemed fine initially after aligning the partitions on 4k, I then rebuilt my raid 1 based lvm and it seemed fine for starters but now I'm back to the initial problems. In particular doing hard links (and deletes) in the xfs file system that lives on the lvm seems slower than before, especially when it hits the two WD20EARS drives (though I've created the xfs with a block size of 4096). As an additional complication there are two more Seagate drives in the lvm which perform as usual (but they are smaller, 1TB and 750GB respectively). I guess in a way it could be said that it's also helpful to have them as a comparison. The most notable difference in my graphs is the iowait on the WD disks which seems to be close to 100% quite a bit.

I'm attaching my graphs sda and sdd are the WD drives, sdb, sdc are Seagate.

http://mimo.gn.apc.org/stuff/blue.html (sorry for the external link but the list doesn't allow attachments)

Any help very much appreciated.

Thanks,

mimo



--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 201108301639.38818.mimo@restoel.net">http://lists.debian.org/201108301639.38818.mimo@restoel.net
 
Old 08-31-2011, 02:00 PM
Marek Podmaka
 
Default slow lvm

Hello,

WD20EARS are the Green drives. Are also the Seagate drives "green"?
Those WD have rpm stated as 5400-7200, so they are surely slower than
any non-green true 7200 rpm disk. They have also some other
energy-saving techniques which can make them slower (although they can
have comparable sustained sequential throughput).
I don't know your exact configuration, but generally it is not good to
mix disks with different speeds on one filesystem.

Tuesday, August 30, 2011, 17:39:38, mimo wrote:

> I'm attaching my graphs sda and sdd are the WD drives, sdb, sdc are Seagate.

> http://mimo.gn.apc.org/stuff/blue.html (sorry for the external link
> but the list doesn't allow attachments)





--
bYE, Marki


--
To UNSUBSCRIBE, email to debian-isp-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1185931746.20110831160024@marki-online.net">http://lists.debian.org/1185931746.20110831160024@marki-online.net
 

Thread Tools




All times are GMT. The time now is 01:28 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org