FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 10-07-2010, 03:06 PM
Zhang Weiwu
 
Default tools to improve harddisk performance by short-stroking?

Thanks to the national holiday (Beijing) I begin to read some article
marked for free-time reading a few years ago. One of them is short stroking.

http://www.tomshardware.com/reviews/short-stroking-hdd,2157.html

The article is awfully long just to give a simple idea: by only using
the first 20% of hard disk space you get about 3 to 4 times of hard-disk
performance gain. The less you use your hard disk space, the more
performance gain you get, you can get as much as 5 times faster HDD by
using less than 10% of hard-disk.

The article says, the only disadvantage of this method is you don't use
the rest of the hard disk.

Finish reading the article it makes obvious to me that, if this
"technology" is really so powerful, it should have already been
implemented in OSes, like Linux, without necessarily abandoning the slow
part of hard-disk space but instead put rarely used data there.

Technically it cannot be too difficult to design file system tools in
the way that it tends to put rarely-accessed files to the end of the
partition that holds the file system. I am sure I have many files on my
computer with atime a few years ago (e.g. man page for hier; files I
moved to Trash a few years ago), they can be moved to the bottom of HDD
space with a performance gain.

Since the knowledge seems to be decades old (some references to this
idea are in 1990s under different names), if the article's
many-time-hdd-performance-gain is true, in the decade there should
already been harddisk performance speedup tool that moves rarely
accessed file to the bottom of file system once a week or so and get me
a 3 time boost of HDD performance.

Question is how to find such a tool? A google search for such tool end
up in vain, that seem to suggest the tool doesn't exist, which means the
article is probably wrong.

Note that I read about methods of partitioning harddisk to get
short-stroking advantage, that wouldn't be optimal, because human has to
decide what are frequently accessed files (put them to first partition)
and what are rarely accessed files (put them on later partitions), and a
tool can do this thing much better than human. Besides, human doing
partition optimization can only choose one of /var, /usr, /home, /tmp
and "swap" as "rarely accessed", the fact is none of them are rarely
accessed, while most of them contain rarely accessed files. That's a
very bad granularity compare to what specialized tool could do.

Thanks in advance for comments!


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4CADE20C.40205@realss.com">http://lists.debian.org/4CADE20C.40205@realss.com
 
Old 10-07-2010, 07:05 PM
 
Default tools to improve harddisk performance by short-stroking?

>
>
>
>---- Original Message ----
>From: zhangweiwu@realss.com
>To: debian-user@lists.debian.org
>Subject: RE: tools to improve harddisk performance by short-stroking?
>Date: Thu, 07 Oct 2010 23:06:52 +0800
>
>>Thanks to the national holiday (Beijing) I begin to read some
>article
>>marked for free-time reading a few years ago. One of them is short
>stroking.
>>
>>http://www.tomshardware.com/reviews/short-stroking-hdd,2157.html
>>
>>The article is awfully long just to give a simple idea: by only
>using
>>the first 20% of hard disk space you get about 3 to 4 times of
>hard-disk
>>performance gain. The less you use your hard disk space, the more
>>performance gain you get, you can get as much as 5 times faster HDD
>by
>>using less than 10% of hard-disk.
>>
>>The article says, the only disadvantage of this method is you don't
>use
>>the rest of the hard disk.
>>
>>Finish reading the article it makes obvious to me that, if this
>>"technology" is really so powerful, it should have already been
>>implemented in OSes, like Linux, without necessarily abandoning the
>slow
>>part of hard-disk space but instead put rarely used data there.
>>
>>Technically it cannot be too difficult to design file system tools
>in
>>the way that it tends to put rarely-accessed files to the end of the
>>partition that holds the file system. I am sure I have many files on
>my
>>computer with atime a few years ago (e.g. man page for hier; files I
>>moved to Trash a few years ago), they can be moved to the bottom of
>HDD
>>space with a performance gain.
>>
>>Since the knowledge seems to be decades old (some references to this
>>idea are in 1990s under different names), if the article's
>>many-time-hdd-performance-gain is true, in the decade there should
>>already been harddisk performance speedup tool that moves rarely
>>accessed file to the bottom of file system once a week or so and get
>me
>>a 3 time boost of HDD performance.
>>
>>Question is how to find such a tool? A google search for such tool
>end
>>up in vain, that seem to suggest the tool doesn't exist, which means
>the
>>article is probably wrong.
>>
>>Note that I read about methods of partitioning harddisk to get
>>short-stroking advantage, that wouldn't be optimal, because human
>has to
>>decide what are frequently accessed files (put them to first
>partition)
>>and what are rarely accessed files (put them on later partitions),
>and a
>>tool can do this thing much better than human. Besides, human doing
>>partition optimization can only choose one of /var, /usr, /home,
>/tmp
>>and "swap" as "rarely accessed", the fact is none of them are rarely
>>accessed, while most of them contain rarely accessed files. That's a
>>very bad granularity compare to what specialized tool could do.
>>
>>Thanks in advance for comments!
>>
>>
Just a guess but the short stroking advantage was likely for disk
access in which the arm returns to some neutral position after
access. OTOH if the access is the usual elevator algorithm in which
the arm stays where it is assuming the next access will be nearby the
short stroking will be ineffective.
Larry
>>--
>>To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
>>with a subject of "unsubscribe". Trouble? Contact
>listmaster@lists.debian.org
>>Archive: http://lists.debian.org/4CADE20C.40205@realss.com
>>
>>



--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 380-2201010471959968@netptc.net">http://lists.debian.org/380-2201010471959968@netptc.net
 
Old 10-07-2010, 08:03 PM
Ron Johnson
 
Default tools to improve harddisk performance by short-stroking?

On 10/07/2010 10:06 AM, Zhang Weiwu wrote:

Thanks to the national holiday (Beijing) I begin to read some article
marked for free-time reading a few years ago. One of them is short stroking.

http://www.tomshardware.com/reviews/short-stroking-hdd,2157.html

The article is awfully long just to give a simple idea: by only using
the first 20% of hard disk space you get about 3 to 4 times of hard-disk
performance gain. The less you use your hard disk space, the more
performance gain you get, you can get as much as 5 times faster HDD by
using less than 10% of hard-disk.

The article says, the only disadvantage of this method is you don't use
the rest of the hard disk.

Finish reading the article it makes obvious to me that, if this
"technology" is really so powerful, it should have already been
implemented in OSes, like Linux, without necessarily abandoning the slow
part of hard-disk space but instead put rarely used data there.


That's "disk file location optimization".



Technically it cannot be too difficult to design file system tools in
the way that it tends to put rarely-accessed files to the end of the
partition that holds the file system.


Says who?

You'd need to add accounting complexity to the kernel (where would
it put the accounting data?), and some utility would have to "know"
the intimate details of your h/w (JBOD, fraction of a JBOD, lvm,
RAID, what kind of RAID, how many spindles, etc, etc) and determine,
on multi-drive devices like RAID and lvm, what exactly is the
*interior* of a drive?


It's a *really* complicated problem.

--
Seek truth from facts.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4CAE279A.2060304@cox.net">http://lists.debian.org/4CAE279A.2060304@cox.net
 
Old 10-08-2010, 12:33 AM
Zhang Weiwu
 
Default tools to improve harddisk performance by short-stroking?

On 2010年10月08日 04:03, Ron Johnson wrote:
>
> You'd need to add accounting complexity to the kernel (where would it
> put the accounting data?)

I had been too brief, but if you read the article I referred to, it
works best only in case you put rarely accessed file to the posterior,
not 'infrequently accessed file' to the posterior.

To find out infrequently accessed file you need accounting. To find
rarely accessed file you only need to look at atime. The difference is
due to there are two factors slows down: disk spinning speed and head
movement speed. If there is only the disk spin speed factor, then put
infrequent file to the bottom helps a lot; if counting in the second
factor, than only putting rarely accessed files at the bottom helps a
lot. Suppose you have one infrequently accessed file at the posterior,
and it's accessed, than the moment accessing it, access to every other
file slows down as the head have to travel back afterwards, that's why
the article suggest not to put any data at all to the posterior (even
infrequent ones), while I think not putting any data at the posterior
should have the same performance gain of putting only rarely accessed
file there.



--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4CAE66BD.4060702@realss.com">http://lists.debian.org/4CAE66BD.4060702@realss.com
 
Old 10-08-2010, 12:40 AM
Ron Johnson
 
Default tools to improve harddisk performance by short-stroking?

On 10/07/2010 07:33 PM, Zhang Weiwu wrote:

On 2010年10月08日 04:03, Ron Johnson wrote:


You'd need to add accounting complexity to the kernel (where would it
put the accounting data?)


I had been too brief, but if you read the article I referred to, it
works best only in case you put rarely accessed file to the posterior,
not 'infrequently accessed file' to the posterior.



I'm sure that this is a case of ESL, but you apparently do not
realize that "rarely" == "infrequently".


Also, that still does not define what "posterior" of a RAID set is.

--
Seek truth from facts.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4CAE689B.50906@cox.net">http://lists.debian.org/4CAE689B.50906@cox.net
 
Old 10-08-2010, 04:42 AM
Tom Furie
 
Default tools to improve harddisk performance by short-stroking?

On Fri, Oct 08, 2010 at 08:33:01AM +0800, Zhang Weiwu wrote:

> To find out infrequently accessed file you need accounting. To find
> rarely accessed file you only need to look at atime.

How do you determine the atime without accessing the file, or keeping
some sort of accounting? How do you update that accounting without
accessing the file? What happens when a bunch of files are seen to be
rarely accessed? What happens when you start using some of these files?

How can you tell how often a file is accessed just by looking at the
last time it was accessed?

Just some food for thought.

Cheers,
Tom
 
Old 10-09-2010, 09:45 PM
Andrei Popescu
 
Default tools to improve harddisk performance by short-stroking?

On Jo, 07 oct 10, 23:06:52, Zhang Weiwu wrote:

> Finish reading the article it makes obvious to me that, if this
> "technology" is really so powerful, it should have already been
> implemented in OSes, like Linux, without necessarily abandoning the slow
> part of hard-disk space but instead put rarely used data there.

IMVHO, I think at least part of the speed increase is based on the fact
that the head(s) *never* travel to the "back" of the harddisk.

Regards,
Andrei
--
Offtopic discussions among Debian users and developers:
http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic
 
Old 10-13-2010, 06:26 AM
Zhang Weiwu
 
Default tools to improve harddisk performance by short-stroking?

On 2010年10月10日 05:45, Andrei Popescu wrote:

IMVHO, I think at least part of the speed increase is based on the fact
that the head(s) *never* travel to the "back" of the harddisk.




A brief search found most of the files on my harddisks are *never*
accessed (since the Debian system installation).
 
Old 10-13-2010, 06:32 AM
Zhang Weiwu
 
Default tools to improve harddisk performance by short-stroking?

I noticed I can find out files that were never accessed after the 1st
month I installed my debian without any accounting mechanism but only by
atime. In fact, if I pick up a random file on my system, the chance it
was never accessed since after 1month of debian installation is a bit
higher than it was.

On 2010年10月08日 12:42, Tom Furie wrote:
> How do you determine the atime without accessing the file
Checking a file's atime is not considered accessing it.

almustafa@orphalese:~$ find Music/ -atime +147
Music/Flamenco. Una antologia de cante y baile andaluces - Poema de la guitarra (rito y geografia del Toque) - Vol. 1 de 4.avi
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/06-hans_zimmer-hes_killed_the_dog_again.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/08-hans_zimmer-not_in_blood_but_in_bond.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/10-hans_zimmer-panic_sheer_bloody_panic.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/07-hans_zimmer-marital_sabotage.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/05-hans_zimmer-data_data_data.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/09-hans_zimmer-ah_putrefaction.mp3
almustafa@orphalese:~$ sleep 10
almustafa@orphalese:~$ find Music/ -atime +147
Music/Flamenco. Una antologia de cante y baile andaluces - Poema de la guitarra (rito y geografia del Toque) - Vol. 1 de 4.avi
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/06-hans_zimmer-hes_killed_the_dog_again.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/08-hans_zimmer-not_in_blood_but_in_bond.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/10-hans_zimmer-panic_sheer_bloody_panic.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/07-hans_zimmer-marital_sabotage.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/05-hans_zimmer-data_data_data.mp3
Music/Hans_Zimmer-Sherlock_Holmes_OST-2010-FRAY/09-hans_zimmer-ah_putrefaction.mp3



--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4CB55270.9010200@realss.com">http://lists.debian.org/4CB55270.9010200@realss.com
 

Thread Tools




All times are GMT. The time now is 05:08 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org