FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu User

 
 
LinkBack Thread Tools
 
Old 02-15-2009, 12:14 PM
Lie Ryan
 
Default Performance characteristic of various filesystem

Do anyone know the performance characteristic of various file systems
(ext2, ext3, ntfs, etc)?

I'm not interested in how fast read/write is in the file system, rather I
want to know how the filesystem copes with large numbers of files.

Since some time ago, I've started zipping up folders with large number of
small files to reduce the number of files. Recently, I've been wondering
whether this practice is actually useful at all, especially since zipping
up files would make it difficult to find duplicates with fdupes.

So, I want to know whether my zipping things up would actually give any
performance improvements or not.

TIA.


--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-15-2009, 01:00 PM
"Anthony M. Rasat"
 
Default Performance characteristic of various filesystem

Lie Ryan wrote:
>I'm not interested in how fast read/write is in the file system, rather I
want to know how the filesystem copes with large numbers of files.

I will talk this purely out of my own experience instead of presenting statistic figures.

Ext2 and ext3 are slow. That's for sure. And handling large number of files going to be pain in the a$$ with these two. And they could lose your file during bad crashes.

I had better experience with xfs but if only you had large number of large files (movies, lots of 700MB+ files). However I think xfs was not fast on large number of small files. My horror experience of crashes in xfs made me turned away from it.

Now mainly I'm using Reiserfs and JFS. Reiserfs is fast handling small and large files. However I did experience file losses on Reiserfs during crash. JFS in the other hand is not as fast as Reiserfs but not as slow as ext2/ext3. Large number of small files is better handled by JFS than ext2/ext3. I never had file loss with JFS. Yet.

Now, I usually employ Reiserfs on desktops and JFS on servers. But that's just me. I could be bias on both case.

--

Regards,

Anthony M. Rasat
Manager - Technical, Network and Support Division
PT. Jawa Pos National Network
Graha Pena Jawa Pos Group Building, 5th floor
Jln. Raya Kebayoran Lama 12, Jakarta Selatan 12210
Indonesia.-
Phone 02132185562
Phone 081574217035
Fax 02153651465
Web http://www.jpnn.com
--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 
Old 02-15-2009, 01:57 PM
Rashkae
 
Default Performance characteristic of various filesystem

Lie Ryan wrote:
> Do anyone know the performance characteristic of various file systems
> (ext2, ext3, ntfs, etc)?
>
> I'm not interested in how fast read/write is in the file system, rather I
> want to know how the filesystem copes with large numbers of files.
>
> Since some time ago, I've started zipping up folders with large number of
> small files to reduce the number of files. Recently, I've been wondering
> whether this practice is actually useful at all, especially since zipping
> up files would make it difficult to find duplicates with fdupes.
>
> So, I want to know whether my zipping things up would actually give any
> performance improvements or not.
>
> TIA.

You aren't really going to see performance benefits from zipping up your
small files.

Ext3, by default, does not use B_tree index, and will become very
cumbersome if you try putting more than 10,000 files in a single
directory. I'm sorry that I don't remember all the steps, but
basically, you use tune2fs to add dir_index option to your ext3 file
system then fsck with an option that makes it re-index your files.

This should make ext3 nearly as efficient as Reiser for file retrieval.
(Ext3 is still very slow at file deletion, however.) As a caveat, when
you do this, program that try to read all files from a directory (say,
for example, if you use zip or tar) will no longer process the files in
inode order. That is to say, if you zip a directory with 5000 files,
chances are, the order that zip reads the files will be the same order
in which they were written to the drive, so the hard drive head doesn't
have to move much. However, once you add dir_index, the 5000 files are
instead processed in a seemingly random hash order, and that causes the
hard drive head to thrash, decimating your i/o throughput. (Note that
ReiserFS has this same problem as well.)

XFS and JFS don't suffer from this same problem to nearly the same
degree, but they come with their own caveats. I would suggest not using
them unless you really need their performance characteristics.



--
ubuntu-users mailing list
ubuntu-users@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-users
 

Thread Tools




All times are GMT. The time now is 08:08 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org