FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > EXT3 Users

 
 
LinkBack Thread Tools
 
Old 09-24-2008, 08:23 AM
Ron Yorston
 
Default ext3 zerofree option and RedHat back port?

Andreas Dilger <adilger@sun.com> wrote:
>> Ulf Zimmermann wrote:
>> Ok, so you really want to zero the unused blocks in-place, and e2image
>> writing out a new sparsified image isn't a ton of help.
>>
>> The tool does that, I guess - but only on an unmounted or RO-mounted
>> filesystem, right? (plus I'd triple-check that it's doing things
>> correctly, opening a block device and splatting zeros around, one hopes
>> that it is!)
>
>That is WAY to scary for me on a mounted filesystem. It is racy if the
>blocks become allocated.

The 1.0.0 version of the zerofree utility only worked on unmounted
filesystems, but then someone suggested that it should be safe on
a read-only mount. Is that not so?

Ron

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 09-24-2008, 11:19 AM
Ric Wheeler
 
Default ext3 zerofree option and RedHat back port?

Ulf Zimmermann wrote:

-----Original Message-----
From: Eric Sandeen [mailto:sandeen@redhat.com]
Sent: 09/23/2008 20:30
To: Ulf Zimmermann
Cc: Theodore Tso; ext3-users@redhat.com
Subject: Re: ext3 zerofree option and RedHat back port?

Ulf Zimmermann wrote:



Reason I asked is this. We use currently 3Par S400 and E200 as SAN
arrays. The new T400 and T800 has a built in chip to do more


intelligent


thin provisioning but I believe even the S400 and E200 we have will


free


on the SAN level a block of a thin provisioned volume if it gets


zero'ed


out. Haven't gotten around yet to test it, but I am planning on. We


are


currently using 3 different file system types, one is a propriety


from


Onstor for their Bobcats (NFS/CIFS heads) where I believe I have
observed just freeing of SAN level blocks. The two other are EXT3


and


OCFS2.


Ok, so you really want to zero the unused blocks in-place, and e2image
writing out a new sparsified image isn't a ton of help.

The tool does that, I guess - but only on an unmounted or RO-mounted
filesystem, right? (plus I'd triple-check that it's doing things
correctly, opening a block device and splatting zeros around, one


hopes


that it is!)

But in any case the util itself is simple enough that building (or


even


packaging) for fedora/EPEL should be trivial.

(FWIW, there is work upstream for filesystems to actually communicate
freed blocks to the underlying storage, just for this purpose...)

-Eric



I am going to try it out by hand. Create a thin provisioned volume,
write random crap to it, then zero the blocks. See if that shrinks the
physical allocated space.

Ulf.





Note that there is work on getting file systems to use the new TRIM (for
S-ATA drives) and its equivalent proposed standard in T10 SCSI for
arrays which will give you this automatically. David Woodhouse was
pushing patches for TRIM, we are still thinking about the SCSI versions...


ric

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 09-24-2008, 01:31 PM
Theodore Tso
 
Default ext3 zerofree option and RedHat back port?

On Wed, Sep 24, 2008 at 09:12:37AM +0100, Ron Yorston wrote:
> "Ulf Zimmermann" <ulf@openlane.com> wrote:
> >Can anyone tell me if the zerofree option for ext3 has been back ported
> >to RedHat EL4 or EL5?
>
> I used to maintain backports of zerofree (the kernel patch, not the
> utility) to EL4 and EL5, but since I wasn't actually using them I gave
> up. The last RPMs I have are from December of last year. Contact me
> directly if you want them.
>
> I don't recommend the ext3 patch as it hasn't seen much use. I regularly
> use the ext2 version (on Fedora 9), but be warned that Ted has expressed
> concerns about it.

I just searched my sent-mail archives for the last 5 years, and I
can't find any references to "zerofree" previous to this mail thread.
Maybe I commented about them under some other name.

Having quickly looked at the ext3 patch here:

http://osdir.com/ml/file-systems.ext3.user/2006-09/msg00026.html

...the big thing I will note is that if you crash after a file is
deleted, but before the journal transaction is committed, the file may
end up being cleared but not deleted. This may or may not be
problematic for your appication; in particular, if the file deletion
was implied with the intent of doing an atomic replacement of some
critical file, i.e. such as a vipw script which does:

cp /etc/passwd /etc/passwd.vipw
vi /etc/passwd.vipw
<sanity check /etc/passwd.vipw for correctness>
# atomically update /etc/passwd
mv /etc/passwd.vipw /etc/passwd

... and you crash before the transaction is commited but after the
"mv" command has run, you could end up with a partially or completely
zero'ed /etc/passwd file. Some might call that unfortunate. :-)

I will admit that the chances of this happening are somewhat remote,
but in terms of potential issues that would have to be fixed before
such a patch could be included in mainline, or before (I suspect) Red
Hat would feel comfortable taking responsibility for their customers'
data after such a patch were committed, that would probably be a real
issue. The code for supporting the "trim" command could also be used
to implement a proper zero-free command, but it gets tricky, since the
blocks in question would have to be remembered until the commit block
is written out, and then only zero'ed (or trimmed) right after the
commit has happened, but before the pinned block bitmaps are released
(which would allow the block allocator to allocate to the blocks that
had just been released).

- Ted

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 09-24-2008, 02:35 PM
Ron Yorston
 
Default ext3 zerofree option and RedHat back port?

Theodore Tso <tytso@mit.edu> wrote:
>I just searched my sent-mail archives for the last 5 years, and I
>can't find any references to "zerofree" previous to this mail thread.
>Maybe I commented about them under some other name.
>
>Having quickly looked at the ext3 patch here:
>
> http://osdir.com/ml/file-systems.ext3.user/2006-09/msg00026.html

Your response is in the same thread:

http://osdir.com/ml/file-systems.ext3.user/2006-09/msg00031.html

Unless that was some other Theodore Tso.

>..the big thing I will note is that if you crash after a file is
>deleted, but before the journal transaction is committed, the file may
>end up being cleared but not deleted.

Indeed, that was the concern last time. The ext3 patch hasn't changed
significantly since then because, truth be told, I don't entirely
understand journalling and was unable to fix it up. The ext2 patch now
writes out the zeroed blocks immediately, which may or may not help.

The latest versions of the patches are available on my website:

http://intgat.tigress.co.uk/rmy/uml/sparsify.html

Ron

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 09-24-2008, 03:04 PM
Eric Sandeen
 
Default ext3 zerofree option and RedHat back port?

Ron Yorston wrote:
> "Ulf Zimmermann" <ulf@openlane.com> wrote:
>> Can anyone tell me if the zerofree option for ext3 has been back ported
>> to RedHat EL4 or EL5?
>
> I used to maintain backports of zerofree (the kernel patch, not the
> utility) to EL4 and EL5, but since I wasn't actually using them I gave
> up. The last RPMs I have are from December of last year. Contact me
> directly if you want them.
>
> I don't recommend the ext3 patch as it hasn't seen much use. I regularly
> use the ext2 version (on Fedora 9), but be warned that Ted has expressed
> concerns about it.

oh, whoops - I guess my google-fu is weak, I searched for zerofree and
assumed we were talking about the userspace utility I found ...

/me runs off to look at that patch...

-Eric

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 09-24-2008, 03:19 PM
Theodore Tso
 
Default ext3 zerofree option and RedHat back port?

On Wed, Sep 24, 2008 at 03:35:32PM +0100, Ron Yorston wrote:
> Your response is in the same thread:
>
> http://osdir.com/ml/file-systems.ext3.user/2006-09/msg00031.html
>
> Unless that was some other Theodore Tso.

Hmm, I must have sent that from a non-primary computer, so it wasn't
in my sent-mail archive. My bad. :-)

- Ted

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 
Old 09-24-2008, 04:23 PM
"Ulf Zimmermann"
 
Default ext3 zerofree option and RedHat back port?

> -----Original Message-----
> From: Eric Sandeen [mailto:sandeen@redhat.com]
> Sent: 09/24/2008 08:05
> To: Ron Yorston
> Cc: Ulf Zimmermann; ext3-users@redhat.com
> Subject: Re: ext3 zerofree option and RedHat back port?
>
> Ron Yorston wrote:
> > "Ulf Zimmermann" <ulf@openlane.com> wrote:
> >> Can anyone tell me if the zerofree option for ext3 has been back
> ported
> >> to RedHat EL4 or EL5?
> >
> > I used to maintain backports of zerofree (the kernel patch, not the
> > utility) to EL4 and EL5, but since I wasn't actually using them I
> gave
> > up. The last RPMs I have are from December of last year. Contact
me
> > directly if you want them.
> >
> > I don't recommend the ext3 patch as it hasn't seen much use. I
> regularly
> > use the ext2 version (on Fedora 9), but be warned that Ted has
> expressed
> > concerns about it.
>
> oh, whoops - I guess my google-fu is weak, I searched for zerofree and
> assumed we were talking about the userspace utility I found ...
>
> /me runs off to look at that patch...
>
> -Eric

Sorry, I meant the mount option for zero'ing blocks which are getting
freed.

Ulf.


_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users
 

Thread Tools




All times are GMT. The time now is 05:02 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org