FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 10-04-2011, 09:38 PM
Farkas Levente
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On 10/04/2011 05:30 PM, Eric Sandeen wrote:
>>> XFS has been proven at this scale on Linux for a very long time, is all.
>>
>> the why rh do NOT support it in 32 bit? there're still system that
>> should have to run on 32 bit:-(
>
> 32-bit machines have a 32-bit index into the page cache; on x86, that limits
> us to 16T for XFS, as well. So 32-bit is really not that interesting for
> large filesystem use.
>
> If you need really scalable filesystems, I'd suggest a 64-bit machine.

i mean if you support xfs and think it's better then ext4 why not
support it on rhel 32bit?

--
Levente "Si vis pacem para bellum!"
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-04-2011, 10:47 PM
"Richard W.M. Jones"
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On Tue, Oct 04, 2011 at 11:38:18PM +0200, Farkas Levente wrote:
> On 10/04/2011 05:30 PM, Eric Sandeen wrote:
> >>> XFS has been proven at this scale on Linux for a very long time, is all.
> >>
> >> the why rh do NOT support it in 32 bit? there're still system that
> >> should have to run on 32 bit:-(
> >
> > 32-bit machines have a 32-bit index into the page cache; on x86, that limits
> > us to 16T for XFS, as well. So 32-bit is really not that interesting for
> > large filesystem use.
> >
> > If you need really scalable filesystems, I'd suggest a 64-bit machine.
>
> i mean if you support xfs and think it's better then ext4 why not
> support it on rhel 32bit?

This is a question you should direct through Red Hat's support
channels.

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
libguestfs lets you edit virtual machines. Supports shell scripting,
bindings from many languages. http://libguestfs.org
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-04-2011, 11:19 PM
Przemek Klosowski
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On 10/03/2011 06:33 PM, Eric Sandeen wrote:
> On 10/3/11 5:13 PM, Richard W.M. Jones wrote:
>> On Mon, Oct 03, 2011 at 04:11:28PM -0500, Eric Sandeen wrote:
>>> testing something more real-world (20T ... 500T?) might still be interesting.
>>
>> Here's my test script:
>>
>> qemu-img create -f qcow2 test1.img 500T&&
>> guestfish -a test1.img
>> memsize 4096 : run :
>> part-disk /dev/vda gpt : mkfs ext4 /dev/vda1
...
>>
>> At 100T it doesn't run out of memory, but the man behind the curtain
>> starts to show. The underlying qcow2 file grows to several gigs and I
>> had to kill it. I need to play with the lazy init features of ext4.
>>
>> Rich.
>>
>
> Bleah. Care to use xfs?

WHy not btrfs? I am testing a 24TB physical server and ext4 creation
took forever while btrfs was almost instant. I understand it's still
experimental (I hear storing virtual disk images on btrfs still has
unresolved performance problems) but vm disk storage should be fine.
FWIW I have been using btrfs as my /home at home for some time now;
so far so good.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-04-2011, 11:53 PM
Ric Wheeler
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On 10/04/2011 07:19 PM, Przemek Klosowski wrote:
> On 10/03/2011 06:33 PM, Eric Sandeen wrote:
>> On 10/3/11 5:13 PM, Richard W.M. Jones wrote:
>>> On Mon, Oct 03, 2011 at 04:11:28PM -0500, Eric Sandeen wrote:
>>>> testing something more real-world (20T ... 500T?) might still be interesting.
>>> Here's my test script:
>>>
>>> qemu-img create -f qcow2 test1.img 500T&&
>>> guestfish -a test1.img
>>> memsize 4096 : run :
>>> part-disk /dev/vda gpt : mkfs ext4 /dev/vda1
> ...
>>> At 100T it doesn't run out of memory, but the man behind the curtain
>>> starts to show. The underlying qcow2 file grows to several gigs and I
>>> had to kill it. I need to play with the lazy init features of ext4.
>>>
>>> Rich.
>>>
>> Bleah. Care to use xfs?
> WHy not btrfs? I am testing a 24TB physical server and ext4 creation
> took forever while btrfs was almost instant. I understand it's still
> experimental (I hear storing virtual disk images on btrfs still has
> unresolved performance problems) but vm disk storage should be fine.
> FWIW I have been using btrfs as my /home at home for some time now;
> so far so good.

Creating an XFS file system is also a matter of seconds (both xfs and btrfs do
dynamic inode allocation).

Note that ext4 has a new feature that allows inodes to be initialized in the
background, so you will see much quicker mkfs.ext4 times as well

ric

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-05-2011, 08:01 AM
Farkas Levente
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On 10/05/2011 12:47 AM, Richard W.M. Jones wrote:
> On Tue, Oct 04, 2011 at 11:38:18PM +0200, Farkas Levente wrote:
>> On 10/04/2011 05:30 PM, Eric Sandeen wrote:
>>>>> XFS has been proven at this scale on Linux for a very long time, is all.
>>>>
>>>> the why rh do NOT support it in 32 bit? there're still system that
>>>> should have to run on 32 bit:-(
>>>
>>> 32-bit machines have a 32-bit index into the page cache; on x86, that limits
>>> us to 16T for XFS, as well. So 32-bit is really not that interesting for
>>> large filesystem use.
>>>
>>> If you need really scalable filesystems, I'd suggest a 64-bit machine.
>>
>> i mean if you support xfs and think it's better then ext4 why not
>> support it on rhel 32bit?
>
> This is a question you should direct through Red Hat's support
> channels.

i'm just like to ask Erik's opinion (who seems to be the fs people at rh:-)

--
Levente "Si vis pacem para bellum!"
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-05-2011, 08:05 AM
Farkas Levente
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On 10/05/2011 01:19 AM, Przemek Klosowski wrote:
> On 10/03/2011 06:33 PM, Eric Sandeen wrote:
>> On 10/3/11 5:13 PM, Richard W.M. Jones wrote:
>>> On Mon, Oct 03, 2011 at 04:11:28PM -0500, Eric Sandeen wrote:
>>>> testing something more real-world (20T ... 500T?) might still be interesting.
>>>
>>> Here's my test script:
>>>
>>> qemu-img create -f qcow2 test1.img 500T&&
>>> guestfish -a test1.img
>>> memsize 4096 : run :
>>> part-disk /dev/vda gpt : mkfs ext4 /dev/vda1
> ...
>>>
>>> At 100T it doesn't run out of memory, but the man behind the curtain
>>> starts to show. The underlying qcow2 file grows to several gigs and I
>>> had to kill it. I need to play with the lazy init features of ext4.
>>>
>>> Rich.
>>>
>>
>> Bleah. Care to use xfs?
>
> WHy not btrfs? I am testing a 24TB physical server and ext4 creation
> took forever while btrfs was almost instant. I understand it's still
> experimental (I hear storing virtual disk images on btrfs still has
> unresolved performance problems) but vm disk storage should be fine.
> FWIW I have been using btrfs as my /home at home for some time now;
> so far so good.

imho if the bfrts maintainer said it's not ready for prime time then
it's not ready for production use.

--
Levente "Si vis pacem para bellum!"
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-05-2011, 11:04 AM
Ric Wheeler
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On 10/05/2011 04:01 AM, Farkas Levente wrote:
> On 10/05/2011 12:47 AM, Richard W.M. Jones wrote:
>> On Tue, Oct 04, 2011 at 11:38:18PM +0200, Farkas Levente wrote:
>>> On 10/04/2011 05:30 PM, Eric Sandeen wrote:
>>>>>> XFS has been proven at this scale on Linux for a very long time, is all.
>>>>> the why rh do NOT support it in 32 bit? there're still system that
>>>>> should have to run on 32 bit:-(
>>>> 32-bit machines have a 32-bit index into the page cache; on x86, that limits
>>>> us to 16T for XFS, as well. So 32-bit is really not that interesting for
>>>> large filesystem use.
>>>>
>>>> If you need really scalable filesystems, I'd suggest a 64-bit machine.
>>> i mean if you support xfs and think it's better then ext4 why not
>>> support it on rhel 32bit?
>> This is a question you should direct through Red Hat's support
>> channels.
> i'm just like to ask Erik's opinion (who seems to be the fs people at rh:-)
>

Eric is our technical lead for file system and works in the broader file system
team.

What Red Hat supports is determined by lots of things - some of them technical,
some of them practical.

Practically, we try to focus our testing and resources on the most common
platforms for enterprise users. 32 bit is not that common and certainly not a
reasonable choice for large file systems. Most new enterprise class servers are
64 bit these days and have been for years.

I can say that as Eric's manager if that helps

Just to be clear, this is a *Fedora* list, not a Red Hat or RHEL list, so what
considerations we as a community make about what is supported in fedora are
different. In Fedora, we do worry more about supporting legacy platforms so 32
bit support will go on longer. We still do have concerns about getting
sufficient testing/qa resources to validate each platform.

thanks!

Ric

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-05-2011, 02:58 PM
Eric Sandeen
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On 10/4/11 6:53 PM, Ric Wheeler wrote:
> On 10/04/2011 07:19 PM, Przemek Klosowski wrote:
>> On 10/03/2011 06:33 PM, Eric Sandeen wrote:
>>> On 10/3/11 5:13 PM, Richard W.M. Jones wrote:
>>>> On Mon, Oct 03, 2011 at 04:11:28PM -0500, Eric Sandeen wrote:
>>>>> testing something more real-world (20T ... 500T?) might still be interesting.
>>>> Here's my test script:
>>>>
>>>> qemu-img create -f qcow2 test1.img 500T&&
>>>> guestfish -a test1.img
>>>> memsize 4096 : run :
>>>> part-disk /dev/vda gpt : mkfs ext4 /dev/vda1
>> ...
>>>> At 100T it doesn't run out of memory, but the man behind the curtain
>>>> starts to show. The underlying qcow2 file grows to several gigs and I
>>>> had to kill it. I need to play with the lazy init features of ext4.
>>>>
>>>> Rich.
>>>>
>>> Bleah. Care to use xfs?
>> WHy not btrfs? I am testing a 24TB physical server and ext4 creation
>> took forever while btrfs was almost instant. I understand it's still
>> experimental (I hear storing virtual disk images on btrfs still has
>> unresolved performance problems) but vm disk storage should be fine.
>> FWIW I have been using btrfs as my /home at home for some time now;
>> so far so good.
>
> Creating an XFS file system is also a matter of seconds (both xfs and btrfs do
> dynamic inode allocation).
>
> Note that ext4 has a new feature that allows inodes to be initialized in the
> background, so you will see much quicker mkfs.ext4 times as well

right; for large ext4 fs use (or testing), try

# mkfs.ext4 -E lazy_itable_init=1 /dev/blah

this will cause it to skip inode table initialization, and speed up mkfs a LOT.
It'll also keep sparse test images smaller.

IMHO this should probably be made default above a certain size.

The tradeoff is that inode table initialization happens in kernelspace, post-mount -
with efforts made to do it in the background, and not impact other I/O too much.

-Eric

> ric
>

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-05-2011, 03:42 PM
Eric Sandeen
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On 10/5/11 9:58 AM, Eric Sandeen wrote:
> On 10/4/11 6:53 PM, Ric Wheeler wrote:

...

>> Note that ext4 has a new feature that allows inodes to be initialized in the
>> background, so you will see much quicker mkfs.ext4 times as well
>
> right; for large ext4 fs use (or testing), try
>
> # mkfs.ext4 -E lazy_itable_init=1 /dev/blah
>
> this will cause it to skip inode table initialization, and speed up mkfs a LOT.
> It'll also keep sparse test images smaller.
>
> IMHO this should probably be made default above a certain size.
>
> The tradeoff is that inode table initialization happens in kernelspace, post-mount -
> with efforts made to do it in the background, and not impact other I/O too much.

Sorry, Lukas reminds me that this should already be the default mode, with a
new enough kernel and new enough e2fsprogs. Rawhide should meet those criteria.

-Eric

> -Eric
>
>> ric
>>
>

--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 
Old 10-05-2011, 05:54 PM
"Richard W.M. Jones"
 
Default Heads up: e2fsprogs-1.42-WIP-0702 pushed to rawhide

On Wed, Oct 05, 2011 at 09:58:59AM -0500, Eric Sandeen wrote:
> right; for large ext4 fs use (or testing), try
>
> # mkfs.ext4 -E lazy_itable_init=1 /dev/blah
>
> this will cause it to skip inode table initialization, and speed up
> mkfs a LOT. It'll also keep sparse test images smaller.
>
> IMHO this should probably be made default above a certain size.

You almost preempted my question: Could I use this for every ext4
filesystem I make? Honestly from a virt / libguestfs p.o.v. it sounds
like something we should always do.

> The tradeoff is that inode table initialization happens in
> kernelspace, post-mount - with efforts made to do it in the
> background, and not impact other I/O too much.

Is there a circumstance where this is bad? I'm thinking perhaps a
case where you create a filesystem and immediately try to populate it
with lots of files.

Rich.

--
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
virt-top is 'top' for virtual machines. Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://et.redhat.com/~rjones/virt-top
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
 

Thread Tools




All times are GMT. The time now is 04:34 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org