FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 09-05-2012, 03:28 PM
Florian Philipp
 
Default aligning SSD partitions

Am 05.09.2012 14:55, schrieb Adam Carter:
>>> I might also add, I see no speed improvements in putting portages work
>>> directory on tmpfs. I have tested this a few times and the difference
>>> in compile times is just not there.
>>
>> Probably because with 16GB everything stays cached anyway.
>
> Would it still be useful to use tmpfs if you wanted to keep drive
> writes down on an SSD? I imagine even tho its cached it would get
> written to disk after a timer expires, so while it doesn't affect
> performance it does reduce the drive life.
>

Yes. For ext{3,4}, this timer is controlled by the "commit=x" mount flag
(default: 5 seconds).

IIRC, app-laptop/laptop-mode-tools sets this to 30 so I guess it is safe
to use large values (besides the obvious risk of loosing data on system
crashes).

Regards,
Florian Philipp
 
Old 09-05-2012, 04:30 PM
Volker Armin Hemmann
 
Default aligning SSD partitions

Am Mittwoch, 5. September 2012, 09:23:58 schrieb Neil Bothwick:
> On Tue, 4 Sep 2012 20:42:56 -0400, Philip Webb wrote:
> > What is the best line for /etc/fstab ? The only example I have is :
> > 'tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0'
> >
> > This doesn't seem to limit the size in any way.
>
> man mount explains it all, but the option you want is size, which
> defaults to 50%. I use 80% which is what gives the somewhat odd size of
> 13GB. This is based on physical RAM, but tmpfs will use swap if there is
> not enough memory.

swap destroys your performance and interactivity so throughly - that is
something you really want to avoid.

--
#163933
 
Old 09-05-2012, 05:02 PM
Neil Bothwick
 
Default aligning SSD partitions

On Wed, 05 Sep 2012 18:30:24 +0200, Volker Armin Hemmann wrote:

> > man mount explains it all, but the option you want is size, which
> > defaults to 50%. I use 80% which is what gives the somewhat odd size
> > of 13GB. This is based on physical RAM, but tmpfs will use swap if
> > there is not enough memory.
>
> swap destroys your performance and interactivity so throughly - that is
> something you really want to avoid.

Absolutely, but it beats the hell out of an OOM. I prefer to have a
system with plenty of swap and using none of it.


--
Neil Bothwick

The quickest way to a man's heart is through his sternum.
 
Old 09-05-2012, 05:54 PM
Dale
 
Default aligning SSD partitions

Michael Mol wrote:
> On Wed, Sep 5, 2012 at 11:17 AM, Neil Bothwick <neil@digimed.co.uk> wrote:
>> On Wed, 05 Sep 2012 07:52:45 -0500, Dale wrote:
>>
>>>>> I might also add, I see no speed improvements in putting portages
>>>>> work directory on tmpfs. I have tested this a few times and the
>>>>> difference in compile times is just not there.
>>>> Probably because with 16GB everything stays cached anyway.
>>> I cleared the cache between the compiles. This is the command I use:
>>>
>>> echo 3 > /proc/sys/vm/drop_caches
>> But you are still using the RAM as disk cache during the emerge, the data
>> doesn't stay around long enough to need to get written to disk with so
>> much RAM for cache.
> Indeed. Try setting the mount to write-through to see the difference.
>
>

When I run that command, it clears all the cache. It is the same as if
I rebooted. Certainly you are not thinking that cache survives a reboot?

If you are talking about ram on the drive itself, well, when it is on
tmpfs, it is not on the drive to be cached. That's the whole point of
tmpfs is to get the slow drive out of the way. By the way, there are
others that ran tests with the same results. It just doesn't speed up
anything since drives are so much faster nowadays.

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
 
Old 09-05-2012, 06:02 PM
Dale
 
Default aligning SSD partitions

Peter Humphrey wrote:
> On Wednesday 05 September 2012 13:02:01 Dale wrote:
>
>> I find that after a big update, like KDE, it helps to defrag /usr.
> Interesting. I've just run sudo e4defrag -c /usr and got a fragmentation
> of zero. That's after upgrading KDE last week.
>
> Then I ran it on all the nine ext4 partitions here and only two had
> nonzero fragmentations; one was 1 and the other 2.
>
> Looks like I can forget about it on this box.
>

I have to say that here, it is not a whole lot of fragmentation but it
does seem a bit faster afterwards. I guess it depends on what is
fragmented and such. I sometimes wonder if it defrags itself. Even
when I watch the fsck when booting, all the ext4 partitions have a very
small percentage of fragmentation. My /boot which is ext2 is fragmented
as heck. lol I'm not worried about it tho. ;-) When I was using
reiserfs, it was always a good bit of fragmentation.

Just thought it was worth a mention since this is the first time I saw a
Linux defrag tool.

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
 
Old 09-05-2012, 07:01 PM
Paul Hartman
 
Default aligning SSD partitions

On Wed, Sep 5, 2012 at 1:02 PM, Dale <rdalek1967@gmail.com> wrote:
> Peter Humphrey wrote:
>> On Wednesday 05 September 2012 13:02:01 Dale wrote:
>>
>>> I find that after a big update, like KDE, it helps to defrag /usr.
>> Interesting. I've just run sudo e4defrag -c /usr and got a fragmentation
>> of zero. That's after upgrading KDE last week.
>>
>> Then I ran it on all the nine ext4 partitions here and only two had
>> nonzero fragmentations; one was 1 and the other 2.
>>
>> Looks like I can forget about it on this box.
>>
>
> I have to say that here, it is not a whole lot of fragmentation but it
> does seem a bit faster afterwards. I guess it depends on what is
> fragmented and such. I sometimes wonder if it defrags itself. Even
> when I watch the fsck when booting, all the ext4 partitions have a very
> small percentage of fragmentation. My /boot which is ext2 is fragmented
> as heck. lol I'm not worried about it tho. ;-) When I was using
> reiserfs, it was always a good bit of fragmentation.
>
> Just thought it was worth a mention since this is the first time I saw a
> Linux defrag tool.

I think almost all linux defrag tools/techniques deal with file
fragmentation only, that is to say one file with more than 1 extent,
but don't deal with filesystem fragmentation (10000 small files
scattered all over the drive, rather than written contiguously). So
I'm not surprised that Peter did not see fragmentation after
installing KDE.

AFAIK almost all that modern defrag tools do is just copy the file,
allocating the whole file at once in the copy process, and if that new
copy has fewer extents than the old copy, it fills in the data, then
removes the original file. The concept is not entirely dissimilar to
the old "backup, format, restore" defrag process.

Over the years I have used a poor-man's version of that concept to
defrag files. Just move it to another drive (or -- even better -- a
ramdrive/tmpfs), then move it back to disk (with a tool that performs
preallocation).

There is a userland defrag tool that does exactly this, on any
filesystem. It is called "shake".

Typically I only see fragmentation on large files that were copied
from a slow source (over the network/internet), or bittorrent clients
that do not preallocate space, etc. Any kind of streaming file that
was written, huge multi-gigabyte video recording files, that kind of
stuff. But the key to avoiding file fragmentation is preallocation...
 
Old 09-05-2012, 08:46 PM
Dale
 
Default aligning SSD partitions

Paul Hartman wrote:
> On Wed, Sep 5, 2012 at 1:02 PM, Dale <rdalek1967@gmail.com> wrote:
>>
>> I have to say that here, it is not a whole lot of fragmentation but it
>> does seem a bit faster afterwards. I guess it depends on what is
>> fragmented and such. I sometimes wonder if it defrags itself. Even
>> when I watch the fsck when booting, all the ext4 partitions have a very
>> small percentage of fragmentation. My /boot which is ext2 is fragmented
>> as heck. lol I'm not worried about it tho. ;-) When I was using
>> reiserfs, it was always a good bit of fragmentation.
>>
>> Just thought it was worth a mention since this is the first time I saw a
>> Linux defrag tool.
> I think almost all linux defrag tools/techniques deal with file
> fragmentation only, that is to say one file with more than 1 extent,
> but don't deal with filesystem fragmentation (10000 small files
> scattered all over the drive, rather than written contiguously). So
> I'm not surprised that Peter did not see fragmentation after
> installing KDE.
>
> AFAIK almost all that modern defrag tools do is just copy the file,
> allocating the whole file at once in the copy process, and if that new
> copy has fewer extents than the old copy, it fills in the data, then
> removes the original file. The concept is not entirely dissimilar to
> the old "backup, format, restore" defrag process.
>
> Over the years I have used a poor-man's version of that concept to
> defrag files. Just move it to another drive (or -- even better -- a
> ramdrive/tmpfs), then move it back to disk (with a tool that performs
> preallocation).
>
> There is a userland defrag tool that does exactly this, on any
> filesystem. It is called "shake".
>
> Typically I only see fragmentation on large files that were copied
> from a slow source (over the network/internet), or bittorrent clients
> that do not preallocate space, etc. Any kind of streaming file that
> was written, huge multi-gigabyte video recording files, that kind of
> stuff. But the key to avoiding file fragmentation is preallocation...
>
>

I used shake before but it just didn't seem to work right for me. I
found a script that does something and it seems to work for the most
part but still not great or anything. I just like the way ext4 works.
Heck, I liked it before I found the defrag tool. I've had this install
for a while and it has never had much fragmentation even before the
tool. So, I find it funny that they make a tool that really isn't
needed very much. :/

Dale

:-) :-)

--
I am only responsible for what I said ... Not for what you understood or how you interpreted my words!
 
Old 09-05-2012, 09:22 PM
Paul Hartman
 
Default aligning SSD partitions

On Wed, Sep 5, 2012 at 3:46 PM, Dale <rdalek1967@gmail.com> wrote:
> Paul Hartman wrote:
>> On Wed, Sep 5, 2012 at 1:02 PM, Dale <rdalek1967@gmail.com> wrote:
>>>
>>> I have to say that here, it is not a whole lot of fragmentation but it
>>> does seem a bit faster afterwards. I guess it depends on what is
>>> fragmented and such. I sometimes wonder if it defrags itself. Even
>>> when I watch the fsck when booting, all the ext4 partitions have a very
>>> small percentage of fragmentation. My /boot which is ext2 is fragmented
>>> as heck. lol I'm not worried about it tho. ;-) When I was using
>>> reiserfs, it was always a good bit of fragmentation.
>>>
>>> Just thought it was worth a mention since this is the first time I saw a
>>> Linux defrag tool.
>> I think almost all linux defrag tools/techniques deal with file
>> fragmentation only, that is to say one file with more than 1 extent,
>> but don't deal with filesystem fragmentation (10000 small files
>> scattered all over the drive, rather than written contiguously). So
>> I'm not surprised that Peter did not see fragmentation after
>> installing KDE.
>>
>> AFAIK almost all that modern defrag tools do is just copy the file,
>> allocating the whole file at once in the copy process, and if that new
>> copy has fewer extents than the old copy, it fills in the data, then
>> removes the original file. The concept is not entirely dissimilar to
>> the old "backup, format, restore" defrag process.
>>
>> Over the years I have used a poor-man's version of that concept to
>> defrag files. Just move it to another drive (or -- even better -- a
>> ramdrive/tmpfs), then move it back to disk (with a tool that performs
>> preallocation).
>>
>> There is a userland defrag tool that does exactly this, on any
>> filesystem. It is called "shake".
>>
>> Typically I only see fragmentation on large files that were copied
>> from a slow source (over the network/internet), or bittorrent clients
>> that do not preallocate space, etc. Any kind of streaming file that
>> was written, huge multi-gigabyte video recording files, that kind of
>> stuff. But the key to avoiding file fragmentation is preallocation...
>>
>>
>
> I used shake before but it just didn't seem to work right for me. I
> found a script that does something and it seems to work for the most
> part but still not great or anything. I just like the way ext4 works.
> Heck, I liked it before I found the defrag tool. I've had this install
> for a while and it has never had much fragmentation even before the
> tool. So, I find it funny that they make a tool that really isn't
> needed very much. :/

I think shake's default options might require extended attributes
enabled in your mounted fs. It also has some thresholds for file size
and age that cause it to skip certain files, unless you tell it
otherwise. I haven't used it in quite a long time, to be honest.
 
Old 09-05-2012, 10:36 PM
Peter Humphrey
 
Default aligning SSD partitions

On Wednesday 05 September 2012 21:46:59 Dale wrote:

> So, I find it funny that they make a tool that really isn't needed very
> much. :/

Call it belt-and-braces if you like. (That's UK braces, which I think
are US suspenders.)

--
Rgds
Peter
 
Old 09-05-2012, 11:42 PM
Neil Bothwick
 
Default aligning SSD partitions

On Wed, 05 Sep 2012 12:54:51 -0500, Dale wrote:

> >>>>> I might also add, I see no speed improvements in putting portages
> >>>>> work directory on tmpfs. I have tested this a few times and the
> >>>>> difference in compile times is just not there.
> >>>> Probably because with 16GB everything stays cached anyway.
> >>> I cleared the cache between the compiles. This is the command I
> >>> use:
> >>>
> >>> echo 3 > /proc/sys/vm/drop_caches
> >> But you are still using the RAM as disk cache during the emerge, the
> >> data doesn't stay around long enough to need to get written to disk
> >> with so much RAM for cache.
> > Indeed. Try setting the mount to write-through to see the difference.

> When I run that command, it clears all the cache. It is the same as if
> I rebooted. Certainly you are not thinking that cache survives a
> reboot?

You clear the cache between the two emerge runs, not during them.

> If you are talking about ram on the drive itself, well, when it is on
> tmpfs, it is not on the drive to be cached. That's the whole point of
> tmpfs is to get the slow drive out of the way. By the way, there are
> others that ran tests with the same results. It just doesn't speed up
> anything since drives are so much faster nowadays.

Drives are still orders of magnitude slower than RAM, that's why using
swap is so slow. What appears to be happening here is that because
files are written and then read again in short succession, they are still
in the kernel's disk cache, so the speed of the disk is irrelevant. Bear
in mind that tmpfs is basically a cached disk without the disk, so you
are effectively comparing the same thing twice.


--
Neil Bothwick

Theory is when you know everything, but nothing works.
Reality is when everything works, but you don't know why.
However, usually theory and reality are mixed together :
Nothing works, and nobody knows why not.
 

Thread Tools




All times are GMT. The time now is 06:58 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org