FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 02-08-2010, 11:27 PM
Neil Bothwick
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

On Mon, 8 Feb 2010 14:34:01 -0600, Paul Hartman wrote:

> Thanks for the info everyone, but do you understand the agony I am now
> suffering at the fact that all disk in my system (including all parts
> of my RAID5) are starting on sector 63 and I don't have sufficient
> free space (or free time) to repartition them?

With the RAID, you could fail one disk, repartition, re-add it, rinse and
repeat. But that doesn't take care of the time issue.

> I am really curious
> if there are any gains to be made on my own system...

Me too, so post back after you've done it ;-)


--
Neil Bothwick

Barth's Distinction:
There are two types of people: those who divide people into two types, and
those who don't.
 
Old 02-08-2010, 11:47 PM
Stroller
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

On 9 Feb 2010, at 00:05, Frank Steinmetzger wrote:

...

- probably about the worst value it could be.


Hm.... what about those first 62 sectors?


If I'm understanding correctly, then the drive will *always* have to
start at the 63rd sector, then swing back round and start reading a
1st sector, for every read larger than 1 byte.


This will result in a minimum of one extra rotation of the disk's
platter for every read, and instead of reading larger data
contiguously the effect will be like a *completely*, least-optimally
fragmented filesystem.


I may be mistaken on this - if that's the case I would love to be
corrected.


The results shown by Valmor & Mark are *two orders of magnitude
faster* when the partitions are correctly aligned.


Stroller.
 
Old 02-09-2010, 01:20 AM
Willie Wong
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

On Tue, Feb 09, 2010 at 01:05:11AM +0100, Frank Steinmetzger wrote:
> Am Sonntag, 7. Februar 2010 schrieb Mark Knecht:
>
> > Hi Willie,
> > OK - it turns out if I start fdisk using the -u option it show me
> > sector numbers. Looking at the original partition put on just using
> > default values it had the starting sector was 63
>
> Same here.
>
> > - probably about the worst value it could be.
>
> Hm.... what about those first 62 sectors?

It is possible you can use some of those; I never tried. That's a
negligible amount of space on modern harddrives anyway. And actually,
starting on sector number 63 means that you are skipping 63 sectors,
not 62, since LBA numbering starts with 0.

Historically there is a reason for all drives coming with default
formatting with the first partition at section 63. Sector 0 is the
MBR, which you shouldn't overwrite. MSDOS and all Windows up to XP
requires the partitions be aligned on Cylinder boundary. So it is
safest to just partition the drive, by default, such that the first
partition starts at LBA 63, or the 64th sector, or the first sector of
the second cylinder.

Actually, this is why Western Digital et al are releasing this flood
of 4K physical sector discs now. Windows XP has been EOLed and Vista
and up supports partitioning not on cylinder boundary. If Windows XP
still had support, this order of magnitude inefficiency wouldn't have
been overlooked by most consumers.

> I bought this 500GB drive for my laptop recently and did a fresh partitioning
> scheme on it, and then rsynced the filesystems of the old, smaller drive onto
> it. The first two partitions are ntfs, but I believe they also use cluster
> sizes of 4k by default. So technically I could repartition everything and
> then restore the contents from my backup drive.

Are you sharing the harddrive with a Windows operating system?
Especially Windows XP? There are reports that Windows XP supports
partitioning not aligned to cylinder boundary. However, if you are
dual booting you will almost surely be fscked if you try that. I had
some fun earlier last year when I did everything else right but
couldn't figure out why my laptop tells me it cannot find the
operating system when I tried to dual boot.

> Though the result justifies your decision, I would have though one has to
> start at 65, unless the disk starts counting its sectors at 0.

I've always assumed by default that computer programmers starts
counting at 0. Mathematicians, on the other hand, varies: analysts
start at 0 or minus infinity; number theorists at 1; algebraists at 1
for groups but 0 for rings; and logicians start counting at the empty
set.

Cheers,

W
--
Willie W. Wong wwong@math.princeton.edu
Data aequatione quotcunque fluentes quantitae involvente fluxiones invenire
et vice versa ~~~ I. Newton
 
Old 02-09-2010, 01:48 AM
Frank Steinmetzger
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

Am Dienstag, 9. Februar 2010 schrieb Mark Knecht:

> 4) Everything I've done so far leave me with messages about partition
> 1 not ending on a cylinder boundary. Googling on that one says don't
> worry about it. I don't know...

Would that be when there’s a + sign behind the end sector? I believe to
remember that _my_ fdisk didn’t show this warning, only parted did.

Anyway, mine's like this, just to throw it into the pot to the others
( those # are added by me to show their respective use )

eisen # fdisk -l -u /dev/sda

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders, total 976773168 sectors
Units = sectors of 1 * 512 = 512 bytes
Disk identifier: 0x80178017

Device Boot Start End Blocks Id System
/dev/sda1 * 63 25157789 12578863+ 7 HPFS/NTFS # Windows
/dev/sda2 25157790 88084394 31463302+ 7 HPFS/NTFS # Win Games
/dev/sda3 88084395 127941659 19928632+ 83 Linux # /
/dev/sda4 127941660 976768064 424413202+ 5 Extended
/dev/sda5 127941723 288816569 80437423+ 83 Linux # /home
/dev/sda6 288816633 780341309 245762338+ 83 Linux # music
/dev/sda7 813113973 976703804 81794916 83 Linux # X-Plane =o)
/dev/sda8 * 976703868 976768064 32098+ 83 Linux # /boot
/dev/sda9 780341373 813113909 16386268+ 7 HPFS/NTFS # Win7 test

--
Gruß | Greetings | Qapla'
begin signature_virus
Hi! I’m a signature virus.
Please copy me to your signature to help me spread.
end
 
Old 02-09-2010, 06:47 AM
"J. Roeleveld"
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

On Monday 08 February 2010 21:34:01 Paul Hartman wrote:
> On Mon, Feb 8, 2010 at 12:52 PM, Valmor de Almeida <val.gentoo@gmail.com>
wrote:
> > Mark Knecht wrote:
> > [snip]
> >
> >> This has been helpful for me. I'm glad Valmor is getting better
> >> results also.
> >
> > [snip]
> >
> > These 4k-sector drives can be problematic when upgrading older
> > computers. For instance, my laptop BIOS would not boot from the toshiba
> > drive I mentioned earlier. However when used as an external usb drive, I
> > could boot gentoo. Since I have been using this drive as backup storage
> > I did not investigate the reason for the lower speed. I am happy to get
> > a factor of 8 in speed up now after you did the research
> >
> > Thanks for your postings.
>
> Thanks for the info everyone, but do you understand the agony I am now
> suffering at the fact that all disk in my system (including all parts
> of my RAID5) are starting on sector 63 and I don't have sufficient
> free space (or free time) to repartition them? I am really curious
> if there are any gains to be made on my own system...
>
> Next time I partition I will definitely pay attention to this, and
> feel foolish that I didn't pay attention before. Thanks.
>

I have similar disks in my new system and was lucky that I was still in the
testing phase and hadn't filled the disks yet.
After changing the partitions to start at sector 64, the creation of the
RAID-5 set went from around 22 hours to 9 hours.

I also get a much higher throughput (in the range of at least 4 times faster),
so I would recommend doing the change if you can.

I now only need to figure out the best way to configure LVM over this to get
the best performance from it. Does anyone know of a decent way of figuring
this out?
I got 6 disks in Raid-5.

Thanks,

Joost Roeleveld
 
Old 02-09-2010, 11:46 AM
Stroller
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

On 9 Feb 2010, at 00:27, Neil Bothwick wrote:


On Mon, 8 Feb 2010 14:34:01 -0600, Paul Hartman wrote:

Thanks for the info everyone, but do you understand the agony I am
now

suffering at the fact that all disk in my system (including all parts
of my RAID5) are starting on sector 63 and I don't have sufficient
free space (or free time) to repartition them?


With the RAID, you could fail one disk, repartition, re-add it,
rinse and

repeat. But that doesn't take care of the time issue.


Aren't you thinking of LVM, or something?

Stroller.
 
Old 02-09-2010, 12:34 PM
Neil Bothwick
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

On Tue, 9 Feb 2010 12:46:40 +0000, Stroller wrote:

> > With the RAID, you could fail one disk, repartition, re-add it,
> > rinse and
> > repeat. But that doesn't take care of the time issue.
>
> Aren't you thinking of LVM, or something?

No. The very nature of RAID is redundancy, so you could remove one disk
from the array to modify its setup then replace it.


--
Neil Bothwick

"One World, One Web, One Program" - Microsoft Promotional Ad
"Ein Volk, Ein Reich, Ein Fuhrer" - Adolf Hitler
 
Old 02-09-2010, 12:35 PM
Volker Armin Hemmann
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

On Dienstag 09 Februar 2010, Stroller wrote:
> On 9 Feb 2010, at 00:27, Neil Bothwick wrote:
> > On Mon, 8 Feb 2010 14:34:01 -0600, Paul Hartman wrote:
> >> Thanks for the info everyone, but do you understand the agony I am
> >> now
> >> suffering at the fact that all disk in my system (including all parts
> >> of my RAID5) are starting on sector 63 and I don't have sufficient
> >> free space (or free time) to repartition them?
> >
> > With the RAID, you could fail one disk, repartition, re-add it,
> > rinse and
> > repeat. But that doesn't take care of the time issue.
>
> Aren't you thinking of LVM, or something?
>
> Stroller.

no
 
Old 02-09-2010, 12:57 PM
"J. Roeleveld"
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

On Tuesday 09 February 2010 13:46:40 Stroller wrote:
> On 9 Feb 2010, at 00:27, Neil Bothwick wrote:
> > On Mon, 8 Feb 2010 14:34:01 -0600, Paul Hartman wrote:
> >> Thanks for the info everyone, but do you understand the agony I am
> >> now
> >> suffering at the fact that all disk in my system (including all parts
> >> of my RAID5) are starting on sector 63 and I don't have sufficient
> >> free space (or free time) to repartition them?
> >
> > With the RAID, you could fail one disk, repartition, re-add it,
> > rinse and
> > repeat. But that doesn't take care of the time issue.
>
> Aren't you thinking of LVM, or something?
>
> Stroller.
>

Not sure where LVM would fit into this, as then you'd need to offload the data
from that PV (Physical Volume) to a different PV first.

With Raid (NOT striping) you can remove one disk, leaving the Raid-array in a
reduced state. Then repartition the disk you removed, repartition and then re-
add the disk to the array.
Wait for the rebuild to complete and do the same with the next disk in the
array.
Eg: (for a 3-disk raid5):
1) remove disk-1 from raid
2) repartition disk-1
3) add disk-1 as new disk to raid
4) wait for the synchronisation to finish
5) remove disk-2 from raid
6) repartition disk-2
7) add disk-2 as new disk to raid
8) wait for the synchronisation to finish
9) remove disk-3 from raid
10) repartition disk-3
11) add disk-3 as new disk to raid
12) wait for the synchronisation to finish

(These steps can easily be adapted for any size and type of raid, apart from
striping/raid-0)

I do, however, see a potential problem, if you repartition starting from
sector 64 instead of from sector 63, the disk has 1 sector less, which means
4KB less in size.
The Raid-array may not accept the re-partitioned disk back into the array
because it's not big enough for the array.

I had this issue with an older system once where I replaced a dead 80GB (Yes,
I did say "old" ) with a new 80GB drive. This drive was actually a few KB
smaller in size and the RAID would refuse to accept it.

--
Joost Roeleveld
 
Old 02-09-2010, 02:11 PM
Stroller
 
Default 1-Terabyte drives - 4K sector sizes? -> bar performance so far

On 9 Feb 2010, at 13:57, J. Roeleveld wrote:

...
With Raid (NOT striping) you can remove one disk, leaving the Raid-
array in a
reduced state. Then repartition the disk you removed, repartition
and then re-

add the disk to the array.


Exactly. Except the partitions extend, in the same positions, across
all the disks.


You cannot remove one disk from the array and repartition it, because
the partition is across the array, not the disk. The single disk,
removed from a RAID 5 (specified by Paul Hartman) array does not
contain any partitions, just one stripe of them.


I apologise if I'm misunderstanding something here, or if your RAID
works differently to mine.


Stroller.
 

Thread Tools




All times are GMT. The time now is 06:59 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org