FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 05-04-2012, 07:08 PM
Chris Knadle
 
Default 2TB USB hard drive for backing up: XFS info

On Monday, April 30, 2012 10:53:46, Camaleón wrote:
> On Sun, 29 Apr 2012 20:16:58 +0200, Martin Steigerwald wrote:
> > Am Dienstag, 24. April 2012 schrieb Camaleón:
...
> > XFS might also have long file check times.
>
> I still have not tried this so I can't really tell but what I've heard on
> XFS is not comparable to ext3, I mean, in regards with the time it takes
> to perform a filesystem check.

On XFS, fsck is literally a no-op -- it does *not* fsck the filesystem.

The steps to actually fsck a XFS filesystem:
- mount the XFS filesystem to replay the log
- unmount the XFS filesystem that was just mounted
- run xfs_check on the filesystem's device
- if necessary run xfs_repair on the filesystem's device

Note this means running 'xfs_check' is done when the filesystem is not
mounted. _Supposedly_ it can also be run if the filesystem is mounted read-
only, but in practice I find it's best (and easier) to run the XFS commands
from a LiveCD. The xfs_check and xfs_repair operations are incredibly fast --
even for a 500GB filesystem it's usually only takes about 10 or 15 seconds.
Speed is generally what XFS is good at, *except* when it comes to deletion of
a large number of files -- that's where it's slow.

Also in practice I find that any kernel crash or hard-power-off corrupts XFS
to at least some extent requiring an xfs_check and xfs_repair, so I have to
make sure to keep a LiveCD on hand to be able to do this.

This gets even more interesting when running XFS on top of LUKS encryption.
I'm currently doing that, and I have had to do an xfs_repair operation -- it
involves running cryptsetup manually at the command line within a LiveCD and
then running xfs_repair on the newly created unencrypted device. [And of
course you have to know to look and make sure the LiveCD contains those
utilities.] Definitely an interesting experience.

The main reason I've been running XFS is for speed -- even on top of LUKS I'm
finding XFS is able to do sustained 40MB/s transfers over 1Gb ethernet, where
ext4 on the same box is not able to sustain that. However ext4 is more
reliable and easier to deal with, because it's able to run an fsck at boot
time and without neeting a LiveCD to fix it. ;-)

-- Chris

--
Chris Knadle
Chris.Knadle@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74
 
Old 05-04-2012, 09:31 PM
Camaleón
 
Default 2TB USB hard drive for backing up: XFS info

On Fri, 04 May 2012 15:08:57 -0400, Chris Knadle wrote:

> On Monday, April 30, 2012 10:53:46, Camaleón wrote:
>> On Sun, 29 Apr 2012 20:16:58 +0200, Martin Steigerwald wrote:
>> > Am Dienstag, 24. April 2012 schrieb Camaleón:
> ...
>> > XFS might also have long file check times.
>>
>> I still have not tried this so I can't really tell but what I've heard
>> on XFS is not comparable to ext3, I mean, in regards with the time it
>> takes to perform a filesystem check.
>
> On XFS, fsck is literally a no-op -- it does *not* fsck the filesystem.

Well, at least there are "xfsprogs".

> The steps to actually fsck a XFS filesystem:
> - mount the XFS filesystem to replay the log
> - unmount the XFS filesystem that was just mounted
> - run xfs_check on the filesystem's device
> - if necessary run xfs_repair on the filesystem's device

You mean you don't even notice a file system inconsistency until it
royally crashes or even something worse? Oh.

> Note this means running 'xfs_check' is done when the filesystem is not
> mounted. _Supposedly_ it can also be run if the filesystem is mounted
> read- only, but in practice I find it's best (and easier) to run the XFS
> commands from a LiveCD. The xfs_check and xfs_repair operations are
> incredibly fast -- even for a 500GB filesystem it's usually only takes
> about 10 or 15 seconds. Speed is generally what XFS is good at, *except*
> when it comes to deletion of a large number of files -- that's where
> it's slow.

So... is that you don't find it suitable for a standard "/" partition?

I mean, if it's better don't analyze an XFS partition when is mounted
read-only, that can be really a no-no for many installations running
24/365.

> Also in practice I find that any kernel crash or hard-power-off corrupts
> XFS to at least some extent requiring an xfs_check and xfs_repair, so I
> have to make sure to keep a LiveCD on hand to be able to do this.

I've also heard about terrific stories of data lose after an unexpected
power failure on volumes running on XFS but as I said before, I have no
direct experience with this file system so I can't comment.

> This gets even more interesting when running XFS on top of LUKS
> encryption. I'm currently doing that, and I have had to do an xfs_repair
> operation -- it involves running cryptsetup manually at the command line
> within a LiveCD and then running xfs_repair on the newly created
> unencrypted device. [And of course you have to know to look and make
> sure the LiveCD contains those utilities.] Definitely an interesting
> experience.

File systems need a twist :-)

> The main reason I've been running XFS is for speed -- even on top of
> LUKS I'm finding XFS is able to do sustained 40MB/s transfers over 1Gb
> ethernet, where ext4 on the same box is not able to sustain that.
> However ext4 is more reliable and easier to deal with, because it's able
> to run an fsck at boot time and without neeting a LiveCD to fix it. ;-)

Not bad numbers.

In the event I give XFS a whirl it will be over my "/data" partition,
that's for sure... and fortunately all of my system have UPS units on
behind O:-)

Greetings,

--
Camaleón


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: jo1hra$3du$19@dough.gmane.org">http://lists.debian.org/jo1hra$3du$19@dough.gmane.org
 
Old 05-05-2012, 06:55 PM
Chris Knadle
 
Default 2TB USB hard drive for backing up: XFS info

On Friday, May 04, 2012 17:31:23, Camaleón wrote:
> On Fri, 04 May 2012 15:08:57 -0400, Chris Knadle wrote:
> > On Monday, April 30, 2012 10:53:46, Camaleón wrote:
> >> On Sun, 29 Apr 2012 20:16:58 +0200, Martin Steigerwald wrote:
> >> > Am Dienstag, 24. April 2012 schrieb Camaleón:
...
> > The steps to actually fsck a XFS filesystem:
> > - mount the XFS filesystem to replay the log
> > - unmount the XFS filesystem that was just mounted
> > - run xfs_check on the filesystem's device
> > - if necessary run xfs_repair on the filesystem's device
>
> You mean you don't even notice a file system inconsistency until it
> royally crashes or even something worse? Oh.

Yes that's corret -- XFS does /not/ warn you at boot time (nor mount time) if
the state of the filesystem is inconsistent. YOU have to know to check that
(or find out "the hard way"), and if you use XFS for "/" you somehow need to
know to do it from a LiveCD. :-/ [This is rarely explained.]

> > Note this means running 'xfs_check' is done when the filesystem is not
> > mounted. _Supposedly_ it can also be run if the filesystem is mounted
> > read- only, but in practice I find it's best (and easier) to run the XFS
> > commands from a LiveCD. The xfs_check and xfs_repair operations are
> > incredibly fast -- even for a 500GB filesystem it's usually only takes
> > about 10 or 15 seconds. Speed is generally what XFS is good at, *except*
> > when it comes to deletion of a large number of files -- that's where
> > it's slow.
>
> So... is that you don't find it suitable for a standard "/" partition?

Hmm. Having given that a thought -- yeah I think that would be a good idea
and I might be happier using ext4 for "/" and keep using XFS for /home. On my
next reinstall of my laptop (whatever year that will be :-P) I might try that
and see how I like it.

> I mean, if it's better don't analyze an XFS partition when is mounted
> read-only, that can be really a no-no for many installations running
> 24/365.

Yes. And in addition nounting a "/" XFS partition read-only to run xfs_repair
on is fairly tricky. Even when booting up into single-user mode it's still
necessary to shut down several processes, which last I recall also includes
the sshd daemon. :-/

I'm currently using XFS for "/" on a remote server I have no physical access
to, and I'm finding this is a problem because I don't have a good way of
running xfs_check and xfs_repair on it while the system is running. The
hosting company supports remote KVM and bootup to a LiveCD of your choice, and
I think this is basically what I'd have to resort to using if I wanted to run
xfs_repair on "/" on that box.

> > Also in practice I find that any kernel crash or hard-power-off corrupts
> > XFS to at least some extent requiring an xfs_check and xfs_repair, so I
> > have to make sure to keep a LiveCD on hand to be able to do this.
>
> I've also heard about terrific stories of data lose after an unexpected
> power failure on volumes running on XFS but as I said before, I have no
> direct experience with this file system so I can't comment.

I've experienced some data loss due to unclean shutdowns, so I can verify that
that's possible. I have not yet had any "massive" data loss, thankfully.

...
> > The main reason I've been running XFS is for speed -- even on top of
> > LUKS I'm finding XFS is able to do sustained 40MB/s transfers over 1Gb
> > ethernet, where ext4 on the same box is not able to sustain that.
> > However ext4 is more reliable and easier to deal with, because it's able
> > to run an fsck at boot time and without neeting a LiveCD to fix it. ;-)
>
> Not bad numbers.

BTW these numbers are with 1Gb ethernet /without/ using jumbo frames -- this
is because the unmanaged switch I'm using doesn't support them.

> In the event I give XFS a whirl it will be over my "/data" partition,
> that's for sure... and fortunately all of my system have UPS units on
> behind O:-)

All of the systems I'm using XFS on have UPSes on them too -- yet I find
systems I run go down hard once in a blue moon, so IMHO a UPS won't completely
save you from needing to run xfs_check and xfs_repair occasionally.

-- Chris

--
Chris Knadle
Chris.Knadle@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74
 
Old 05-06-2012, 09:19 AM
Andrei POPESCU
 
Default 2TB USB hard drive for backing up: XFS info

On Vi, 04 mai 12, 15:08:57, Chris Knadle wrote:
>
> Note this means running 'xfs_check' is done when the filesystem is not
> mounted. _Supposedly_ it can also be run if the filesystem is mounted read-
> only, but in practice I find it's best (and easier) to run the XFS commands
> from a LiveCD. The xfs_check and xfs_repair operations are incredibly fast --
> even for a 500GB filesystem it's usually only takes about 10 or 15 seconds.

Yep, just did that now.

> Speed is generally what XFS is good at, *except* when it comes to deletion of
> a large number of files -- that's where it's slow.

On advise of a list subscriber I have added the 'delaylog' mount option.
This is supposed to help if you have a new enough kernel.

> Also in practice I find that any kernel crash or hard-power-off corrupts XFS
> to at least some extent requiring an xfs_check and xfs_repair, so I have to
> make sure to keep a LiveCD on hand to be able to do this.

I using xfs only on my laptop, so I have hard power-off only if the
kernel crashes. I just did the xfs_check and xfs_repair now, but they
didn't find any problems.

Kind regards,
Andrei
--
Offtopic discussions among Debian users and developers:
http://lists.alioth.debian.org/mailman/listinfo/d-community-offtopic
 
Old 05-06-2012, 10:50 PM
Chris Knadle
 
Default 2TB USB hard drive for backing up: XFS info

On Sunday, May 06, 2012 05:19:20, Andrei POPESCU wrote:
> On Vi, 04 mai 12, 15:08:57, Chris Knadle wrote:
...
> > Speed is generally what XFS is good at, *except* when it comes to
> > deletion of a large number of files -- that's where it's slow.
>
> On advise of a list subscriber I have added the 'delaylog' mount option.
> This is supposed to help if you have a new enough kernel.

It took me a while to find a reference for what this setting does. Having
read the paragraph in the link below [it's question #40 when using the
contents at the top of the page], it seems there's a speed benefit but also a
risk of additional corruption in the case of an unclean shutdown.

http://www.xfs.org/index.php/XFS_FAQ#Q:_I_want_to_tune_my_XFS_filesystems_for_. 3Csomething.3E

As for the kernel -- I'm currently using Linux 3.3.4. [I've been custom
compiling my own kernel using make-kpkg from the 'kernel-package' package for
a long time now.]

> > Also in practice I find that any kernel crash or hard-power-off corrupts
> > XFS to at least some extent requiring an xfs_check and xfs_repair, so I
> > have to make sure to keep a LiveCD on hand to be able to do this.
>
> I using xfs only on my laptop, so I have hard power-off only if the
> kernel crashes. I just did the xfs_check and xfs_repair now, but they
> didn't find any problems.

The last time I had a hard-power-off both of the XFS filesystems on my laptop
came up clean also, however I did have corruption and had to clean out about a
hundred files in /lost+found. [Mostly temp files, but 'debsums -s' also
reported files missing from packages, so I needed to reinstall several
packages to fix that.]

-- Chris

--
Chris Knadle
Chris.Knadle@coredump.us
GPG Key: 4096R/0x1E759A726A9FDD74
 

Thread Tools




All times are GMT. The time now is 04:05 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org