ext3_dx_add_entry: Directory index full!
 

On Sun, May 18, 2008 at 05:36:02PM +0200, Bernie Innocenti wrote:
>
> Some background: I'm moving users' Maildirs to a separate filesystem tuned
> for small files to increase performance. One of our users intentionally
> collected spam for 5 years in one folder and likes it this way.
> We could easily work it around, but first I'd like to understand whether
> the particular parameters we used trigger a bug in ext3 or if we're just
> hitting a (possibly undocumented) limit.

No, not a bug, but a limit. Ext3's hash directores are limited to a
depth of 3 blocks, which normally isn't a problem if you are using a
4k blocksize, since each internal node is small; only 8 bytes. So you
have a fanout of 508 for each internal node, and two internal nodes
gets you to over 250,000 4k directory blocks. But with a 1k
blocksize, the internal node fanout is only 124, so that only gets you
a bit more than 15,000 1k directory blocks.

We could remove this limit at some point; the problem is that Daniel
Phillip's original code had this as a limitation, and fixing it would
mean replacing the tree implementation. We actually have some code
from Lustre that we could use for this purpose, but to date we've been
focused on some other higher priority items for ext4.

- Ted

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3_dx_add_entry: Directory index full!
 

On Sun, May 18, 2008 at 05:38:58PM +0200, Bernie Innocenti wrote:
> Bernie Innocenti wrote:
>> On 2.6.24.4-64.fc8, I createed and mounted a filesystem like this:
>> mke2fs -m0 -b 1024 -R stride=64 -I 128 -i 2048 -j -L mail -O
>> dir_index,sparse_super -v /dev/sdc1
>
> I cannot reproduce it any more if I reformat omitting "-b 1024".
> Maybe it would reappear with 200K * 4 = 800K files?

Using a filesystem with 4k blocks, and assuming the filenames are of
the same average length, you should be able to get approximately 200k
* (4**3) = 12.8 million files in a single directory. If you use a 2k
block filesystem, the limit will be approximately 200k * (2**3) = 1.6
million files in a single directory.

Regards,

- Ted

P.S. Past a certain point, you really don't want to have that many
files in a Maildir directory; if the user is never going to be
deleting his SPAM, then you should seriously think about using a Unix
mbox style storage scheme. Even with a 1k block filesystem, at 12
million files you'll be wasting 6 gigabytes of disk space of slack
space that is totally being wasted since the whole point of using
Maildir is to make it easy to delete or replace individual mail
messages. If you want to archive all of your SPAM, why use a Maildir
format mbox at all?

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3_dx_add_entry: Directory index full!
 

On Mon, May 19, 2008 at 01:01:57AM +0200, Stefano Fedrigo wrote:
>
> So, if I understand correctly, with a 1024 bytes blocksize, dir_index, and
> inode size of 128 byte, the maximum number of files in a directory is
> 123008. With 4k blocks this limit rises to 8,258,048 files?

It depends on the length of the directory entries, and how full the
various directory blocks end up getting (which is a function of the
directory names used and the per-filesystem hash seed). But in
general, the maximum limit goes up as the cube of the blocksize. So a
4k filesystem can store roughly 64 times as many files ; a filesystem
using 16k blocks (say, on a Power or IA64 architecture) will be able
to store roughly 4,096 as many files in a single directory. (So
around 819 million files in a single directory, using the original
maildir example).

Seriously, though, past a certain point, if you really want to store
that many small datums, you should really consider a database....

- Ted

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3_dx_add_entry: Directory index full!
 

On Sun, 2008-05-18 at 17:04 -0400, Theodore Tso wrote:
> P.S. Past a certain point, you really don't want to have that many
> files in a Maildir directory; if the user is never going to be
> deleting his SPAM, then you should seriously think about using a Unix
> mbox style storage scheme. Even with a 1k block filesystem, at 12
> million files you'll be wasting 6 gigabytes of disk space of slack
> space that is totally being wasted since the whole point of using
> Maildir is to make it easy to delete or replace individual mail
> messages. If you want to archive all of your SPAM, why use a Maildir
> format mbox at all?
>
Cyrus, which I am using on ext3, has a maildir like format in which each
message is a separate file. (It might even be maildir, but I think
not). This is what has led to my very slow directory traversal times on
backup.

Cyrus does not offer a choice of formats in the sense of switching to
something like mbox, and it is intended to be for large scale use.
Functionally, it is a mail database.

I suspect many other systems, even those offering a choice of format,
won't let you mix different formats. So if you like maildir for some
stuff, you may need to use it for all.

So it seems to me it would be useful if the filesystem supported such
useage patterns well.

Ross Boylan

[I cut out most of the distribution list.]

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3_dx_add_entry: Directory index full!
 

My log is showing errors like
May 1 05:14:17 corn kernel: [6822807.017625] EXT3-fs warning (device dm-7): ext3_dx_add_entry: Directory index full!

Judging from the minor device numbers in /dev/mapper, that corresponds to my mail spool.

My searches suggest that the problem indicates an individual directory
has too many files in it. There does not seem to be a general shortage
of inodes or space. df -i shows IUse of 11% for that partition and df
says 88% of the ~37G is in use.

First question: what happens in these circumstances? Are files lost?
Search and indexing are inefficient? The program trying to write the
file gets an error (it's cyrus. 10 seconds after the errors shown above,
the log has "cyrus/master[8178]: process 11850 exited, status 98".
Usually the status is 0.)?

Second: is there a way to find what directory is causing the problem?

Third: How can I fix this?

I'm running a stock Debian Lenny 2.6.26-2-686 kernel.

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3_dx_add_entry: Directory index full!
 

On 2010-05-02, at 02:57, Ross Boylan wrote:
My log is showing errors like
> May 1 05:14:17 corn kernel: [6822807.017625] EXT3-fs warning (device dm-7): ext3_dx_add_entry: Directory index full!
>
> Judging from the minor device numbers in /dev/mapper, that corresponds to my mail spool.
>
> My searches suggest that the problem indicates an individual directory
> has too many files in it. There does not seem to be a general shortage
> of inodes or space. df -i shows IUse of 11% for that partition and df
> says 88% of the ~37G is in use.

The directory is probably at least 10M files, though it might also suffer from the random create/delete cycle of the mail spool directory.

> First question: what happens in these circumstances? Are files lost?
> Search and indexing are inefficient?

It _should_ be the latter, though I haven't actually looked into it closely.

> Second: is there a way to find what directory is causing the problem?

Patch the error message to print the inode number and dentry name, and submit it here.

> Third: How can I fix this?

e2fsck -fD on the filesystem (unmounted of course)

Cheers, Andreas






_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users