FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > EXT3 Users

 
 
LinkBack Thread Tools
 
Old 03-26-2010, 06:16 PM
Eric Sandeen
 
Default Ext4 and large (>8TB) files

On 03/26/2010 01:52 PM, Arun Nair wrote:
> Hi -
>
> (I apologize for the ext4 question in an ext3 mailer, but I couldn't
> find a user list for ext4.)

linux-ext4@vger.kernel.org but that's ok.

> Per my understanding, ext4 can support file sizes upto 16 TiB if you use
> 4k blocks. I have a logical volume which uses ext4 with a 4k block size
> but I am unable to create files that are 8TiB (8796093022208 bytes) or
> larger.
>
> [root@camanoe] ls -l
> total 8589935388
> -rw-rw---- 1 root root 8796093022207 2010-03-26 11:43 bigfile
>
> [root@camanoe] echo x >> bigfile
> -bash: echo: write error: File too large

Perhaps echo isn't using largefile semantics? Is this the first
test you did, or is echo the simple testcase, and something else
failed?

It works for me on rawhide x86_64:

create a file with blocks past 8T:
# xfs_io -F -f -c "pwrite 8T 1M" bigfile
wrote 1048576/1048576 bytes at offset 8796093022208
1 MiB, 256 ops; 0.0000 sec (206.313 MiB/sec and 52816.1750 ops/sec)

echo more into it:
# echo x >> bigfile

it really is that big:
# ls -lh bigfile
-rw-------. 1 root root 8.1T Mar 26 14:13 bigfile

I don't have an x86 box to test quickly; try something besides echo,
is what I'd suggest - xfs_io would work, or probably dd (with
conv=notrunc if you want to append)

-Eric

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 03-26-2010, 07:50 PM
Arun Nair
 
Default Ext4 and large (>8TB) files

Eric,
Thanks for the quick reply... see my responses inline...
On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <sandeen@redhat.com> wrote:


On 03/26/2010 01:52 PM, Arun Nair wrote:

> Hi -

>

> (I apologize for the ext4 question in an ext3 mailer, but I couldn't

> find a user list for ext4.)



linux-ext4@vger.kernel.org *but that's ok.

Saw that but thought it was a dev-only list, sorry. Next time *




> Per my understanding, ext4 can support file sizes upto 16 TiB if you use

> 4k blocks. I have a logical volume which uses ext4 with a 4k block size

> but I am unable to create files that are 8TiB (8796093022208 bytes) or

> larger.

>

> [root@camanoe] ls -l

> total 8589935388

> -rw-rw---- 1 root root 8796093022207 2010-03-26 11:43 bigfile

>

> [root@camanoe] echo x >> bigfile

> -bash: echo: write error: File too large



Perhaps echo isn't using largefile semantics? *Is this the first

test you did, or is echo the simple testcase, and something else

failed?

It's the simple test case. We found the problem when MySQL failed to expand its ibdata file beyond 8 TB. I then tried dd as well with notrunc like you mentioned, same error:


[root@camanoe]# dd oflag=append conv=notrunc if=/dev/zero of=bigfile bs=1 count=1dd: writing `bigfile': File too large1+0 records in0+0 records out
0 bytes (0 B) copied, 0.000234712 s, 0.0 kB/s




It works for me on rawhide x86_64:



create a file with blocks past 8T:

# xfs_io -F -f -c "pwrite 8T 1M" bigfile

wrote 1048576/1048576 bytes at offset 8796093022208

1 MiB, 256 ops; 0.0000 sec (206.313 MiB/sec and 52816.1750 ops/sec)



echo more into it:

# echo x >> bigfile



it really is that big:

# ls -lh bigfile

-rw-------. 1 root root 8.1T Mar 26 14:13 bigfile



I don't have an x86 box to test quickly; try something besides echo,

is what I'd suggest - xfs_io would work, or probably dd (with

conv=notrunc if you want to append)

dd fails as mentioned above. xfs_io errors too:[root@camanoe]# xfs_io -F -f -c "pwrite 8T 1M" bigfile2pwrite64: File too large





-Eric




BTW, my system is NOT 64-bit but my guess is this doesn't affect max file size?

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 

Thread Tools




All times are GMT. The time now is 01:23 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org