FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 06-27-2010, 08:27 AM
Florian Philipp
 
Default ext4 reserved space and defragmentation?

Am 27.06.2010 09:52, schrieb Shaochun Wang:
> Hi:
>
> I want to tune my ext4 filesystem of NAS data partition to free its
> reserved space by using "tune2fs -m0 <partition>". By reading the
> manual of tune2fs, I observed the following words:
>
> "Reserving some number of filesystem blocks for use by
> privileged processes is done to avoid filesystem
> fragmentation"
>
> It means that filesystem defragmentation need such reserved blocks to
> work properly, am I right? If so, can I make the reserved blocks a
> little because the default 5% is too much. My NAS filesystem is about
> 7x1.5T, then 5% means a lot of space.
>

It's not meant for defragmentation but to avoid fragmentation in the
first place. Simply speaking, as long as there is enough free space on
the disk, even if it is not accessible by an ordinary user, the system
can put new data in any place it seems fit.

If the disk is really near 100% full, the system has to squeeze new data
in every small hole somewhere on the disk. If this is done over a longer
time, maybe by deleting files and creating new ones, it massively
increases fragmentation.

Besides, reserving some space for root can save your rear-end in case
some user fills up your root partition. As long as root's processes
still have a bit of disk space available, he can still log in and clean
up the mess.

I agree with you that reserving 5% of several TB is too much for these
purposes. I think reducing it to a few 100MB is sufficient.

Hope this helps,
Florian Philipp
 
Old 06-27-2010, 08:45 AM
Stroller
 
Default ext4 reserved space and defragmentation?

On 27 Jun 2010, at 08:52, Shaochun Wang wrote:

... "Reserving some number of filesystem blocks for use by
privileged processes is done to avoid filesystem
fragmentation"

It means that filesystem defragmentation need such reserved blocks to
work properly, am I right? If so, can I make the reserved blocks a
little because the default 5% is too much. My NAS filesystem is about
7x1.5T, then 5% means a lot of space.


I'm pretty sure that just means that Linux will try to put files in
contiguous sectors, so they're not fragmented, and that as you run out
of space it's generally harder to do that.


But I would imagine this is particularly the case with the occasional
large file on a typical filesystem cluttered with small files - if you
have a 1TB drive and save 9 100GB movie files on it, the remaining
free space is going to be contiguous, anyway.


Whilst it would be interesting to do some real world testing on big
hard drives fulla porn, you can safely set the reserved space to 0%
and forget about it. That message has been there since ext2 and if you
streaming suddenly starts to stutter when your filesystem is 99% full,
well, you were going to add another drive to the array, anyway,
weren't you? Add it in and expand the filesystem and see if that makes
any difference.


Interestingly, I've just done an fsck on my ext4 media array and it
shows as 83.8% non-contiguous. It is 1.4TB with 272G or 19% free. I
can only assume this is because I also use it for backups, and have a
couple of directories on there of many much smaller files.


Stroller.
 
Old 06-27-2010, 09:23 AM
Dale
 
Default ext4 reserved space and defragmentation?

Stroller wrote:


I'm pretty sure that just means that Linux will try to put files in
contiguous sectors, so they're not fragmented, and that as you run out
of space it's generally harder to do that.


But I would imagine this is particularly the case with the occasional
large file on a typical filesystem cluttered with small files - if you
have a 1TB drive and save 9 100GB movie files on it, the remaining
free space is going to be contiguous, anyway.


Whilst it would be interesting to do some real world testing on big
hard drives fulla porn, you can safely set the reserved space to 0%
and forget about it. That message has been there since ext2 and if you
streaming suddenly starts to stutter when your filesystem is 99% full,
well, you were going to add another drive to the array, anyway,
weren't you? Add it in and expand the filesystem and see if that makes
any difference.


Interestingly, I've just done an fsck on my ext4 media array and it
shows as 83.8% non-contiguous. It is 1.4TB with 272G or 19% free. I
can only assume this is because I also use it for backups, and have a
couple of directories on there of many much smaller files.


Stroller.





I have a 750Gb drive that I put mostly movies on. I have NCIS and some
that I got from youtube of old TV shows plus some regular files like OOo
docs. I just ran fragck on that a while ago and got this:


72.499201913381% non contiguous files, 2.38433317962776 average fragments.

I also have my /boot partition which has a few kernels on it and their
config files. I get this for that partition:


78.4313725490196% non contiguous files, 4.72549019607843 average fragments.

Both of those have lots of free space still. About half way on the
750Gb drive and 75% free space on /boot. I cleaned it out a while back
and got rid of some old kernels and configs.


My freshly copied /usr directory comes in as this:

3.52929844927837% non contiguous files, 0.524852607939654 average fragments.

That includes the portage tree by the way. However, there is not much
difference when portage is unmounted either. Again, freshly copied just
this morning and only synced once so far.


I just freshly transfered my OS from one drive to another. /boot and
the data drive was untouched tho. I don't have a place large enough to
transfer my data partition. My question is this, isn't there a point
where there will be fragmentation no matter what file system you use?
After all, some files are going to be fragmented because of size and
some are going to be fragmented because they are edited and such.


By the way, resierfs for everything except /boot and portage which uses
ext2 and ext3 respectively.


Dale

:-) :-)
 
Old 06-27-2010, 10:21 AM
Alan McKinnon
 
Default ext4 reserved space and defragmentation?

On Sunday 27 June 2010 10:27:13 Florian Philipp wrote:
> Besides, reserving some space for root can save your rear-end in case
> some user fills up your root partition. As long as root's processes
> still have a bit of disk space available, he can still log in and clean
> up the mess.
>
> I agree with you that reserving 5% of several TB is too much for these
> purposes. I think reducing it to a few 100MB is sufficient.

This is correct, ext2 was set up like that in the beginning and it hasn't
changed since.

Those were the days when 200M drives were BIG and reserving 10M for root was
enough for the purpose.


--
alan dot mckinnon at gmail dot com
 

Thread Tools




All times are GMT. The time now is 01:26 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org