FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > CentOS > CentOS

 
 
LinkBack Thread Tools
 
Old 12-23-2009, 02:47 PM
carlopmart
 
Default OT:Which filesystem to use with large files

Hi all,

Recently I have installed a centOS 5.4 server to use as a home NAS server. I need
to use large files (8GB minimum) inside of it to serve via iSCSI services. Which
filesystem do you recommends me to reach maximum performance: xfs, ext3, ext4, gfs2
....??

Thanks.

--
CL Martinez
carlopmart {at} gmail {d0t} com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 12-23-2009, 02:57 PM
Les Mikesell
 
Default OT:Which filesystem to use with large files

carlopmart wrote:
> Hi all,
>
> Recently I have installed a centOS 5.4 server to use as a home NAS server. I need
> to use large files (8GB minimum) inside of it to serve via iSCSI services. Which
> filesystem do you recommends me to reach maximum performance: xfs, ext3, ext4, gfs2
> ....??

None of them should have a problem with large files and any differences
in the filesystem metadata handling will be covered up by the underlying
disk access speed of a large amount of data per file. The bigger
differences are in how fast they can create or delete large numbers of
files.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 12-23-2009, 03:39 PM
carlopmart
 
Default OT:Which filesystem to use with large files

Les Mikesell wrote:
> carlopmart wrote:
>> Hi all,
>>
>> Recently I have installed a centOS 5.4 server to use as a home NAS server. I need
>> to use large files (8GB minimum) inside of it to serve via iSCSI services. Which
>> filesystem do you recommends me to reach maximum performance: xfs, ext3, ext4, gfs2
>> ....??
>
> None of them should have a problem with large files and any differences
> in the filesystem metadata handling will be covered up by the underlying
> disk access speed of a large amount of data per file. The bigger
> differences are in how fast they can create or delete large numbers of
> files.
>

Thanks Les. then if I would to use sparse files to create these large files, which
can be the best form: dd of qcow2 format for example??

--
CL Martinez
carlopmart {at} gmail {d0t} com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 12-23-2009, 03:54 PM
Kwan Lowe
 
Default OT:Which filesystem to use with large files

On Wed, Dec 23, 2009 at 10:47 AM, carlopmart <carlopmart@gmail.com> wrote:
> Hi all,
>
> *Recently I have installed a centOS 5.4 server to use as a home NAS server. I need
> to use large files (8GB minimum) inside of it to serve via iSCSI services. Which
> filesystem do you recommends me to reach maximum performance: xfs, ext3, ext4, gfs2
> ....??

I don't know if this is still true, but when I last checked a couple
years ago, the recommendation was for LVM device backed iSCSI targets.

http://osdir.com/ml/linux.iscsi.tgt.devel/2008-09/msg00000.html

With LVMs you'd of course lose the flexibility of file-backed targets
and the ability to do sparse files are you're intending..

dd if=/dev/zero of=iqn.2009-12.com.mydomain:storage.disk01.foo.foo
bs=1 count=0 seek=16G
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 12-23-2009, 04:01 PM
carlopmart
 
Default OT:Which filesystem to use with large files

Kwan Lowe wrote:
> On Wed, Dec 23, 2009 at 10:47 AM, carlopmart <carlopmart@gmail.com> wrote:
>> Hi all,
>>
>> Recently I have installed a centOS 5.4 server to use as a home NAS server. I need
>> to use large files (8GB minimum) inside of it to serve via iSCSI services. Which
>> filesystem do you recommends me to reach maximum performance: xfs, ext3, ext4, gfs2
>> ....??
>
> I don't know if this is still true, but when I last checked a couple
> years ago, the recommendation was for LVM device backed iSCSI targets.
>
> http://osdir.com/ml/linux.iscsi.tgt.devel/2008-09/msg00000.html
>
> With LVMs you'd of course lose the flexibility of file-backed targets
> and the ability to do sparse files are you're intending..
>
> dd if=/dev/zero of=iqn.2009-12.com.mydomain:storage.disk01.foo.foo
> bs=1 count=0 seek=16G

LVM was my first option and performance it is very very good with iSCSI, but backup
and restore it is a problem with LVM. For these reason I need to use large files on
this server...


--
CL Martinez
carlopmart {at} gmail {d0t} com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 12-23-2009, 04:29 PM
Les Mikesell
 
Default OT:Which filesystem to use with large files

carlopmart wrote:
> Kwan Lowe wrote:
>> On Wed, Dec 23, 2009 at 10:47 AM, carlopmart <carlopmart@gmail.com> wrote:
>>> Hi all,
>>>
>>> Recently I have installed a centOS 5.4 server to use as a home NAS server. I need
>>> to use large files (8GB minimum) inside of it to serve via iSCSI services. Which
>>> filesystem do you recommends me to reach maximum performance: xfs, ext3, ext4, gfs2
>>> ....??
>> I don't know if this is still true, but when I last checked a couple
>> years ago, the recommendation was for LVM device backed iSCSI targets.
>>
>> http://osdir.com/ml/linux.iscsi.tgt.devel/2008-09/msg00000.html
>>
>> With LVMs you'd of course lose the flexibility of file-backed targets
>> and the ability to do sparse files are you're intending..
>>
>> dd if=/dev/zero of=iqn.2009-12.com.mydomain:storage.disk01.foo.foo
>> bs=1 count=0 seek=16G
>
> LVM was my first option and performance it is very very good with iSCSI, but backup
> and restore it is a problem with LVM. For these reason I need to use large files on
> this server...

Doesn't sparse file use leave you in danger of (a) overcommiting the
actual available space, and (b) badly fragmenting the on-disk locations
when the space is actually allocated? I think xfs has some support for
allocating sparse space at creation time without waiting for real
writes, but I don't know how to use it.

--
Les Mikesell
lesmikesell@gmail.com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 12-23-2009, 09:46 PM
Kwan Lowe
 
Default OT:Which filesystem to use with large files

>> With LVMs you'd of course lose the flexibility of file-backed targets
>> and the ability to do sparse files are you're intending..
>>
>> dd if=/dev/zero of=iqn.2009-12.com.mydomain:storage.disk01.foo.foo
>> bs=1 count=0 seek=16G
>
> LVM was my first option and performance it is very very good with iSCSI, but backup
> and restore it is a problem with LVM. For these reason I need to use large files on
> this server...

I actually prefer backing up LVMs.. I use the snapshot feature which
means I can backup a live volume. Works well.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 12-23-2009, 09:50 PM
carlopmart
 
Default OT:Which filesystem to use with large files

Kwan Lowe wrote:
>>> With LVMs you'd of course lose the flexibility of file-backed targets
>>> and the ability to do sparse files are you're intending..
>>>
>>> dd if=/dev/zero of=iqn.2009-12.com.mydomain:storage.disk01.foo.foo
>>> bs=1 count=0 seek=16G
>> LVM was my first option and performance it is very very good with iSCSI, but backup
>> and restore it is a problem with LVM. For these reason I need to use large files on
>> this server...
>
> I actually prefer backing up LVMs.. I use the snapshot feature which
> means I can backup a live volume. Works well.

But I can't use snapshot feature because under these lvm partitions there are ZFS,
NTFS and so on filesystems that linux can't access ...



--
CL Martinez
carlopmart {at} gmail {d0t} com
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 12-23-2009, 10:09 PM
Kwan Lowe
 
Default OT:Which filesystem to use with large files

>> I actually prefer backing up LVMs.. I use the snapshot feature which
>> means I can backup a live volume. Works well.
>
> But I can't use snapshot feature because under these lvm partitions there are ZFS,
> NTFS and so on filesystems that linux can't access ...

Not sure that I'm understanding.. The snapshots are block level, so no
knowledge of the filesystem on the LVs are needed. In fact, some are
raw devices and don't have filesystems at all.
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 
Old 12-23-2009, 10:14 PM
MHR
 
Default OT:Which filesystem to use with large files

On Wed, Dec 23, 2009 at 2:50 PM, carlopmart <carlopmart@gmail.com> wrote:
> Kwan Lowe wrote:
>>
>> I actually prefer backing up LVMs.. I use the snapshot feature which
>> means I can backup a live volume. Works well.
>
> But I can't use snapshot feature because under these lvm partitions there are ZFS,
> NTFS and so on filesystems that linux can't access ...
>

There is a perfectly usable NTFS support file system available for
CentOS/Linux - I use it from time to time for disk backups.

Take a look at NTFS-fuse.

BTW, volume backups generally do not require knowledge of the
underlying file system structures, so this shouldn't be an issue
anyway.

mhr
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos
 

Thread Tools




All times are GMT. The time now is 10:55 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org