On 03/26/2010 03:50 PM, Arun Nair wrote:
> Eric,
>
> Thanks for the quick reply... see my responses inline...
>
> On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <sandeen@redhat.com
> <mailto:sandeen@redhat.com>> wrote:
>
> On 03/26/2010 01:52 PM, Arun Nair wrote:
> > Hi -
> >
> > (I apologize for the ext4 question in an ext3 mailer, but I couldn't
> > find a user list for ext4.)
>
> linux-ext4@vger.kernel.org <mailto:linux-ext4@vger.kernel.org>
> but that's ok.
>
>
> Saw that but thought it was a dev-only list, sorry. Next time
*shrug* I think user questions are welcome too. At least I don't mind.
...
> dd fails as mentioned above. xfs_io errors too:
> [root@camanoe]# xfs_io -F -f -c "pwrite 8T 1M" bigfile2
> pwrite64: File too large
/* Sanity check against vm- & vfs- imposed limits */
if (res > upper_limit)
res = upper_limit;
return res;
}
and:
/* Page cache limit. The filesystems should put that into their s_maxbytes
limits, otherwise bad things can happen in VM. */
#if BITS_PER_LONG==32
#define MAX_LFS_FILESIZE (((u64)PAGE_CACHE_SIZE << (BITS_PER_LONG-1))-1)
so it's only giving us 31 bits of pages, not 32. This limits it to 8T
on a 32-bit machine with 4k pages.
I'm not honestly sure if there is anything in the vm that can't actually
cope with a 32-bit offset... but until proven otherwise, probably not
going to change this without a lot of testing & inspection.
-Eric
_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
03-26-2010, 09:05 PM
Arun Nair
Ext4 and large (>8TB) files
Eric -
So I'm guessing switching the system to 64-bit would fix this for us. How about increasing the block size from the current 4k? Would that be an option too?
Thanks much,
Arun
On Fri, Mar 26, 2010 at 2:10 PM, Eric Sandeen <sandeen@redhat.com> wrote:
On 03/26/2010 03:50 PM, Arun Nair wrote:
> Eric,
>
> Thanks for the quick reply... see my responses inline...
>
> On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <sandeen@redhat.com
> <mailto:sandeen@redhat.com>> wrote:
>
> * * On 03/26/2010 01:52 PM, Arun Nair wrote:
> * * > Hi -
> * * >
> * * > (I apologize for the ext4 question in an ext3 mailer, but I couldn't
so it's only giving us 31 bits of pages, not 32. *This limits it to 8T
on a 32-bit machine with 4k pages.
I'm not honestly sure if there is anything in the vm that can't actually
cope with a 32-bit offset... but until proven otherwise, probably not
going to change this without a lot of testing & inspection.
-Eric
_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
03-26-2010, 09:38 PM
Andreas Dilger
Ext4 and large (>8TB) files
On 2010-03-26, at 16:05, Arun Nair wrote:
So I'm guessing switching the system to 64-bit would fix this for
us. How about increasing the block size from the current 4k? Would
that be an option too?
Not in the near future, unless you are running on PPC/ARM/SPARC that
can also handle large pages.
On Fri, Mar 26, 2010 at 2:10 PM, Eric Sandeen <sandeen@redhat.com>
wrote:
On 03/26/2010 03:50 PM, Arun Nair wrote:
> Eric,
>
> Thanks for the quick reply... see my responses inline...
>
> On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <sandeen@redhat.com
> <mailto:sandeen@redhat.com>> wrote:
>
> On 03/26/2010 01:52 PM, Arun Nair wrote:
> > Hi -
> >
> > (I apologize for the ext4 question in an ext3 mailer, but I
couldn't
> > find a user list for ext4.)
>
> linux-ext4@vger.kernel.org <mailto:linux-
ext4@vger.kernel.org>
> but that's ok.
>
>
> Saw that but thought it was a dev-only list, sorry. Next time
*shrug* I think user questions are welcome too. At least I don't
mind.
...
> dd fails as mentioned above. xfs_io errors too:
> [root@camanoe]# xfs_io -F -f -c "pwrite 8T 1M" bigfile2
> pwrite64: File too large
/* Sanity check against vm- & vfs- imposed limits */
if (res > upper_limit)
res = upper_limit;
return res;
}
and:
/* Page cache limit. The filesystems should put that into their
s_maxbytes
limits, otherwise bad things can happen in VM. */
#if BITS_PER_LONG==32
#define MAX_LFS_FILESIZE (((u64)PAGE_CACHE_SIZE <<
(BITS_PER_LONG-1))-1)
so it's only giving us 31 bits of pages, not 32. This limits it to 8T
on a 32-bit machine with 4k pages.
I'm not honestly sure if there is anything in the vm that can't
actually
cope with a 32-bit offset... but until proven otherwise, probably not
going to change this without a lot of testing & inspection.
-Eric
_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
Cheers, Andreas
_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
03-27-2010, 12:32 AM
Arun Nair
Ext4 and large (>8TB) files
Ok, so I guess ext4 with 64-bit, or another filesystem for us.*Thanks Andreas & Eric for all the help.
On Fri, Mar 26, 2010 at 3:38 PM, Andreas Dilger <adilger@dilger.ca> wrote:
On 2010-03-26, at 16:05, Arun Nair wrote:
So I'm guessing switching the system to 64-bit would fix this for us. How about increasing the block size from the current 4k? Would that be an option too?
Not in the near future, unless you are running on PPC/ARM/SPARC that can also handle large pages.
On Fri, Mar 26, 2010 at 2:10 PM, Eric Sandeen <sandeen@redhat.com> wrote:
On 03/26/2010 03:50 PM, Arun Nair wrote:
> Eric,
>
> Thanks for the quick reply... see my responses inline...
>
> On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <sandeen@redhat.com
> <mailto:sandeen@redhat.com>> wrote:
>
> * * On 03/26/2010 01:52 PM, Arun Nair wrote:
> * * > Hi -
> * * >
> * * > (I apologize for the ext4 question in an ext3 mailer, but I couldn't
_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
03-27-2010, 03:04 AM
Andreas Dilger
Ext4 and large (>8TB) files
On 2010-03-26, at 19:32, Arun Nair wrote:
Ok, so I guess ext4 with 64-bit, or another filesystem for us.
Thanks Andreas & Eric for all the help.
No, I don't think another filesystem will help, on a 32-bit host. The
limit that ext4 is reporting is the VM page cache limit for a single
file, and has nothing to do with ext4 itself.
On Fri, Mar 26, 2010 at 3:38 PM, Andreas Dilger <adilger@dilger.ca>
wrote:
On 2010-03-26, at 16:05, Arun Nair wrote:
So I'm guessing switching the system to 64-bit would fix this for
us. How about increasing the block size from the current 4k? Would
that be an option too?
Not in the near future, unless you are running on PPC/ARM/SPARC that
can also handle large pages.
On Fri, Mar 26, 2010 at 2:10 PM, Eric Sandeen <sandeen@redhat.com>
wrote:
On 03/26/2010 03:50 PM, Arun Nair wrote:
> Eric,
>
> Thanks for the quick reply... see my responses inline...
>
> On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <sandeen@redhat.com
> <mailto:sandeen@redhat.com>> wrote:
>
> On 03/26/2010 01:52 PM, Arun Nair wrote:
> > Hi -
> >
> > (I apologize for the ext4 question in an ext3 mailer, but I
couldn't
> > find a user list for ext4.)
>
> linux-ext4@vger.kernel.org <mailto:linux-
ext4@vger.kernel.org>
> but that's ok.
>
>
> Saw that but thought it was a dev-only list, sorry. Next time
*shrug* I think user questions are welcome too. At least I don't
mind.
...
> dd fails as mentioned above. xfs_io errors too:
> [root@camanoe]# xfs_io -F -f -c "pwrite 8T 1M" bigfile2
> pwrite64: File too large
/* Sanity check against vm- & vfs- imposed limits */
if (res > upper_limit)
res = upper_limit;
return res;
}
and:
/* Page cache limit. The filesystems should put that into their
s_maxbytes
limits, otherwise bad things can happen in VM. */
#if BITS_PER_LONG==32
#define MAX_LFS_FILESIZE (((u64)PAGE_CACHE_SIZE <<
(BITS_PER_LONG-1))-1)
so it's only giving us 31 bits of pages, not 32. This limits it to 8T
on a 32-bit machine with 4k pages.
I'm not honestly sure if there is anything in the vm that can't
actually
cope with a 32-bit offset... but until proven otherwise, probably not
going to change this without a lot of testing & inspection.
-Eric
_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
Cheers, Andreas
Cheers, Andreas
_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
03-27-2010, 03:16 AM
Arun Nair
Ext4 and large (>8TB) files
Ah. Got it, thanks.
On Fri, Mar 26, 2010 at 9:04 PM, Andreas Dilger <adilger@dilger.ca> wrote:
On 2010-03-26, at 19:32, Arun Nair wrote:
Ok, so I guess ext4 with 64-bit, or another filesystem for us. Thanks Andreas & Eric for all the help.
No, I don't think another filesystem will help, on a 32-bit host. *The limit that ext4 is reporting is the VM page cache limit for a single file, and has nothing to do with ext4 itself.
On Fri, Mar 26, 2010 at 3:38 PM, Andreas Dilger <adilger@dilger.ca> wrote:
On 2010-03-26, at 16:05, Arun Nair wrote:
So I'm guessing switching the system to 64-bit would fix this for us. How about increasing the block size from the current 4k? Would that be an option too?
Not in the near future, unless you are running on PPC/ARM/SPARC that can also handle large pages.
On Fri, Mar 26, 2010 at 2:10 PM, Eric Sandeen <sandeen@redhat.com> wrote:
On 03/26/2010 03:50 PM, Arun Nair wrote:
> Eric,
>
> Thanks for the quick reply... see my responses inline...
>
> On Fri, Mar 26, 2010 at 12:16 PM, Eric Sandeen <sandeen@redhat.com
> <mailto:sandeen@redhat.com>> wrote:
>
> * * On 03/26/2010 01:52 PM, Arun Nair wrote:
> * * > Hi -
> * * >
> * * > (I apologize for the ext4 question in an ext3 mailer, but I couldn't
_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
03-29-2010, 06:06 PM
Eric Sandeen
Ext4 and large (>8TB) files
Andreas Dilger wrote:
> On 2010-03-26, at 19:32, Arun Nair wrote:
>> Ok, so I guess ext4 with 64-bit, or another filesystem for us. Thanks
>> Andreas & Eric for all the help.
>
> No, I don't think another filesystem will help, on a 32-bit host. The
> limit that ext4 is reporting is the VM page cache limit for a single
> file, and has nothing to do with ext4 itself.
Well, for what it's worth, xfs doesn't use MAX_LFS_FILESIZE for s_maxbytes, and:
# mkfs.xfs -dfile,name=fsfile,size=5g
...
# mount -o loop fsfile mnt/
# cd mnt/
# truncate --size 17592186044415 bigfile
# ls -lh bigfile
-rw-r--r--. 1 root root 16T Mar 29 14:03 bigfile
# uname -m
i686
it is possible to create a > 8T file offset.
Now, whether the vm is really happy with this probably remains to be seen;
this is the sort of thing that breaks without constant testing, IMHO.
I'd certainly suggest that a 64-bit box is the best way to go if at all possible.
-Eric
_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
Ext4 and large (>8TB) files
