FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 03-09-2009, 06:48 PM
Paul E Condon
 
Default Advice about ext3, please (An experiment & results)

On 2009-03-09_11:19:38, Stefan Monnier wrote:
> > Does anyone here power off their computer without first shutting it down?
> > Maybe, but after having to spend time repairing the system and/or rebuilding
> > it or losing data they most likely don't anymore.
>
> Even if you're careful, you'll still occasionally lose power, and your
> machine will still occasionally crash. That's why people have developed
> file systems that can withstand even such situations (using journalling
> or a log structure).
>
> > Neither ext2 nor ext3 were designed to be used in this manor.
>
> ext3 was specifically designed for these situations.
>

I did an experiment. I was worried particularly about an accidental
unplugging of USB before umount. What happens when you pull the plug
is that the kernel issues an alert on all functioning consoles and
terminal emulators to the effect journaling has failed. It is hard,
very hard to be unaware that you, the operator, have made a mistake.

The message doesn't -tell- you what to do, but what I think one should
do is plug in the USB drive again and do fsck on the device. When fsck
runs, in immediately reruns the journal and fixes metadata
inconsistencies. What it may not do is actually write data that was
waiting in some buffer. (fsck has a option to force a full check even
if the fast, incomplete check using the journal indicates that things
are fixed. I did this, and fsck found no detailed errors either.)

In an archiving application, which is what I am interested in, the
data is a stable record that has been prepared for archiving, so
rerunning the archiving software fixes any incomplete writes. With less
robust file systems, the file system can become corrupted to the point
that it must be reformatted. That would be really not good.

The computer did have to be re-booted before the fsck. the kernel behaved
as it a journal error was equivalent to a kerneloops. So pulling the plug
accidentally is a fairly big deal in a situation where 100% up-time is
required, But, again there are ways to design around this, like having
the USB drive connected to a computer that is dedicated to archiving, and
is expected to go down from time to time due to operator error.

I haven't looked into power failure during archiving, but I reason that
this is not a problem -for- -archiving-. The record that has been prepared
for archive is not being modified when the power fails, so there is little
chance that it will be damaged. The file system on the USB maybe damaged,
but, with the journal, can be fixed before it is remounted to resume the
archiving process. So I conclude ext3 is good for archiving.

--
Paul E Condon
pecondon@mesanetworks.net


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-11-2009, 04:15 PM
Stefan Monnier
 
Default Advice about ext3, please (An experiment & results)

> The message doesn't -tell- you what to do, but what I think one should
> do is plug in the USB drive again and do fsck on the device. When fsck
> runs, in immediately reruns the journal and fixes metadata
> inconsistencies.

Mounting the device would have done the same thing (even if mounted
read-only, actually: you'd get a message along the lines of "enabling
read-write while replaying the journal"). So running fsck is really not
needed in this case.

> What it may not do is actually write data that was waiting in some
> buffer. (fsck has a option to force a full check even if the fast,
> incomplete check using the journal indicates that things are
> fixed. I did this, and fsck found no detailed errors either.)

Indeed, fsck can still be useful if you want to force a full check
(which mount won't do).

> The computer did have to be re-booted before the fsck.

AFAIK this depends on many things. In my experience, the system is
still perfectly usable afterwards, except that it still has some
"pending operations" for that now-non-existent device, so you may be
unable to unmount the drive and re-inserting the USB drive will usually
give it some new device name (because the old one is still in use).
So it has never been enough to force me to reboot, but if you do it
often you will eventually need to reboot.

So, yes, unplugging your USB key while it's still mounted is to be
avoided, and even more so while it's being written to.


Stefan


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-11-2009, 04:36 PM
Lisi Reisz
 
Default Advice about ext3, please (An experiment & results)

On Wednesday 11 March 2009 17:15:44 Stefan Monnier wrote:
> So, yes, unplugging your USB key while it's still mounted is to be
> avoided, and even more so while it's being written to.

The OP asked about about a USB external HDD, not a key. I have not tested the
theory, but I have always understood that keys are particularly vulnerable.
To physical damage if pulled out prematurely, not just damage to the
filesystem.

Lisi


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-11-2009, 04:43 PM
Johannes Wiedersich
 
Default Advice about ext3, please (An experiment & results)

Lisi Reisz wrote:
> I have not tested the
> theory, but I have always understood that keys are particularly vulnerable.
> To physical damage if pulled out prematurely, not just damage to the
> filesystem.

Why so?

Johannes


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-11-2009, 04:56 PM
Lisi Reisz
 
Default Advice about ext3, please (An experiment & results)

On Wednesday 11 March 2009 17:43:33 Johannes Wiedersich wrote:
> Lisi Reisz wrote:
> > I have not tested the
> > theory, but I have always understood that keys are particularly
> > vulnerable. To physical damage if pulled out prematurely, not just damage
> > to the filesystem.
>
> Why so?

As I say, I have not tested this out myself. But I have been told it quite
often. I don't know much about what is inside a USB key, and assumed that it
was something to do with that. I have certainly had a USB key die when
someone else yanked it out prematurely, but that could have been coincidence.

Lisi



--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-11-2009, 05:40 PM
Johannes Wiedersich
 
Default Advice about ext3, please (An experiment & results)

Lisi Reisz wrote:
> On Wednesday 11 March 2009 17:43:33 Johannes Wiedersich wrote:
>> Lisi Reisz wrote:
>>> I have not tested the
>>> theory, but I have always understood that keys are particularly
>>> vulnerable. To physical damage if pulled out prematurely, not just damage
>>> to the filesystem.
>> Why so?
>
> As I say, I have not tested this out myself. But I have been told it quite
> often. I don't know much about what is inside a USB key, and assumed that it
> was something to do with that. I have certainly had a USB key die when
> someone else yanked it out prematurely, but that could have been coincidence.

Sorry for my misunderstanding. As you wrote about a 'theory' I thought
you had some facts (although untested facts).

Cheers,

Johannes


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-12-2009, 01:44 AM
Stefan Monnier
 
Default Advice about ext3, please (An experiment & results)

>> So, yes, unplugging your USB key while it's still mounted is to be
>> avoided, and even more so while it's being written to.

> The OP asked about about a USB external HDD, not a key. I have not
> tested the theory, but I have always understood that keys are
> particularly vulnerable. To physical damage if pulled out
> prematurely, not just damage to the filesystem.

I do not know that it makes any difference, tho it's quite possible that
flash keys might be more delicate, because their basic "modify disk
block" operation may temporarily invalidate other blocks. Note that the
risk is in "remove while it's writing" rather than "remove without
unmounting" (tho the difference between the two is probably irrelevant
to the end user).


Stefan

PS: typically flash memory is made up of "eraseblocks" that are much
larger than a disk block, so depending on the way your flash key works,
writing a single block (512bytes) of your disk may end up doing "read
the surrounding eraseblock; erase it, rewrite it with the new content of
that particular block", so if the operation gets interrupted right after
the erase, you may end up losing a whole bunch of nearby (but maybe
unrelated) blocks.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-12-2009, 11:44 PM
Emanoil Kotsev
 
Default Advice about ext3, please (An experiment & results)

Stefan Monnier wrote:

>
> PS: typically flash memory is made up of "eraseblocks" that are much
> larger than a disk block, so depending on the way your flash key works,
> writing a single block (512bytes) of your disk may end up doing "read
> the surrounding eraseblock; erase it, rewrite it with the new content of
> that particular block", so if the operation gets interrupted right after
> the erase, you may end up losing a whole bunch of nearby (but maybe
> unrelated) blocks.

Hi,

my experience with ext3 on external disk is excellent. I've been testing it
last year with raid and the raid itself on usb was not very reliable,
though I still have 2x200GB raided usb disks running for more than an year.

But I've never had a problem with ext3 on a single usb drive

an issue with the flash drives is their life cycle. they support about
100000 writes or so in average - there was article I read recently

regards


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-14-2009, 02:16 AM
Stefan Monnier
 
Default Advice about ext3, please (An experiment & results)

> an issue with the flash drives is their life cycle. they support about
> 100000 writes or so in average - there was article I read recently

For large enough drives, 100000 writes will take several years
of constant write access. So I wouldn't worry about it.


Stefan




--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-16-2009, 11:48 PM
Emanoil Kotsev
 
Default Advice about ext3, please (An experiment & results)

Stefan Monnier wrote:

>> an issue with the flash drives is their life cycle. they support about
>> 100000 writes or so in average - there was article I read recently
>
> For large enough drives, 100000 writes will take several years
> of constant write access. So I wouldn't worry about it.
>
>
> Stefan

Well several years is not very precise. I was thinking to let my firewall
run on a CF drive. The last one served for 10years, so ...

thanks


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 05:27 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org