ext3 zerofree option and RedHat back port?
 

Can anyone tell me if the zerofree option for ext3 has been back ported
to RedHat EL4 or EL5?

Regards, Ulf.

---------------------------------------------------------------------
OPENLANE Inc., T: 650-532-6382, F: 650-532-6441
4600 Bohannon Drive, Suite 100, Menlo Park, CA 94025
---------------------------------------------------------------------


_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3 zerofree option and RedHat back port?
 

Ulf Zimmermann wrote:
> Can anyone tell me if the zerofree option for ext3 has been back ported
> to RedHat EL4 or EL5?

there appears to be no backporting to do; it's a single .c file that
makes simple use (I assume...) of libext2...

But no, it's not in Fedora, EPEL, or RHEL. Builds fine on my rhel5 box.

If you wanted to, you could be the maintainer for Fedora, and put it
into EPEL, which would make it available for RHEL :)

-Eric

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3 zerofree option and RedHat back port?
 

On Tue, Sep 23, 2008 at 10:13:20AM -0500, Eric Sandeen wrote:
> Ulf Zimmermann wrote:
> > Can anyone tell me if the zerofree option for ext3 has been back ported
> > to RedHat EL4 or EL5?
>
> there appears to be no backporting to do; it's a single .c file that
> makes simple use (I assume...) of libext2...
>
> But no, it's not in Fedora, EPEL, or RHEL. Builds fine on my rhel5 box.
>
> If you wanted to, you could be the maintainer for Fedora, and put it
> into EPEL, which would make it available for RHEL :)

Or it would be roughly a 5 line change to e2image (3 for option
parsing, 1 for the usage line, and 1 to the if statement in
write_raw_image_file() :-) to add an option to extend the "raw dump"
functionality to also dump the data blocks of files, at which point it
would create a sparse file containing only the used blocks in the
filesystem for you, automatically.

- Ted

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3 zerofree option and RedHat back port?
 

Theodore Tso wrote:
> On Tue, Sep 23, 2008 at 10:13:20AM -0500, Eric Sandeen wrote:
>> Ulf Zimmermann wrote:
>>> Can anyone tell me if the zerofree option for ext3 has been back ported
>>> to RedHat EL4 or EL5?
>> there appears to be no backporting to do; it's a single .c file that
>> makes simple use (I assume...) of libext2...
>>
>> But no, it's not in Fedora, EPEL, or RHEL. Builds fine on my rhel5 box.
>>
>> If you wanted to, you could be the maintainer for Fedora, and put it
>> into EPEL, which would make it available for RHEL :)
>
> Or it would be roughly a 5 line change to e2image (3 for option
> parsing, 1 for the usage line, and 1 to the if statement in
> write_raw_image_file() :-) to add an option to extend the "raw dump"
> functionality to also dump the data blocks of files, at which point it
> would create a sparse file containing only the used blocks in the
> filesystem for you, automatically.
>
> - Ted

hey that sounds even better than a random collection of single-purpose
utilities! ;)

(But I suppose the original util had the other useful purpose of
scrubbing free blocks even if you don't intend to compress the fs image...)

-Eric

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3 zerofree option and RedHat back port?
 

> -----Original Message-----
> From: Eric Sandeen [mailto:sandeen@redhat.com]
> Sent: 09/23/2008 10:02
> To: Theodore Tso
> Cc: Ulf Zimmermann; ext3-users@redhat.com
> Subject: Re: ext3 zerofree option and RedHat back port?
>
> Theodore Tso wrote:
> > On Tue, Sep 23, 2008 at 10:13:20AM -0500, Eric Sandeen wrote:
> >> Ulf Zimmermann wrote:
> >>> Can anyone tell me if the zerofree option for ext3 has been back
> ported
> >>> to RedHat EL4 or EL5?
> >> there appears to be no backporting to do; it's a single .c file
that
> >> makes simple use (I assume...) of libext2...
> >>
> >> But no, it's not in Fedora, EPEL, or RHEL. Builds fine on my rhel5
> box.
> >>
> >> If you wanted to, you could be the maintainer for Fedora, and put
it
> >> into EPEL, which would make it available for RHEL :)
> >
> > Or it would be roughly a 5 line change to e2image (3 for option
> > parsing, 1 for the usage line, and 1 to the if statement in
> > write_raw_image_file() :-) to add an option to extend the "raw dump"
> > functionality to also dump the data blocks of files, at which point
> it
> > would create a sparse file containing only the used blocks in the
> > filesystem for you, automatically.
> >
> > - Ted
>
> hey that sounds even better than a random collection of single-purpose
> utilities! ;)
>
> (But I suppose the original util had the other useful purpose of
> scrubbing free blocks even if you don't intend to compress the fs
> image...)
>
> -Eric

Reason I asked is this. We use currently 3Par S400 and E200 as SAN
arrays. The new T400 and T800 has a built in chip to do more intelligent
thin provisioning but I believe even the S400 and E200 we have will free
on the SAN level a block of a thin provisioned volume if it gets zero'ed
out. Haven't gotten around yet to test it, but I am planning on. We are
currently using 3 different file system types, one is a propriety from
Onstor for their Bobcats (NFS/CIFS heads) where I believe I have
observed just freeing of SAN level blocks. The two other are EXT3 and
OCFS2.


Ulf Zimmermann


_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3 zerofree option and RedHat back port?
 

Ulf Zimmermann wrote:

> Reason I asked is this. We use currently 3Par S400 and E200 as SAN
> arrays. The new T400 and T800 has a built in chip to do more intelligent
> thin provisioning but I believe even the S400 and E200 we have will free
> on the SAN level a block of a thin provisioned volume if it gets zero'ed
> out. Haven't gotten around yet to test it, but I am planning on. We are
> currently using 3 different file system types, one is a propriety from
> Onstor for their Bobcats (NFS/CIFS heads) where I believe I have
> observed just freeing of SAN level blocks. The two other are EXT3 and
> OCFS2.

Ok, so you really want to zero the unused blocks in-place, and e2image
writing out a new sparsified image isn't a ton of help.

The tool does that, I guess - but only on an unmounted or RO-mounted
filesystem, right? (plus I'd triple-check that it's doing things
correctly, opening a block device and splatting zeros around, one hopes
that it is!)

But in any case the util itself is simple enough that building (or even
packaging) for fedora/EPEL should be trivial.

(FWIW, there is work upstream for filesystems to actually communicate
freed blocks to the underlying storage, just for this purpose...)

-Eric

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3 zerofree option and RedHat back port?
 

> -----Original Message-----
> From: Eric Sandeen [mailto:sandeen@redhat.com]
> Sent: 09/23/2008 20:30
> To: Ulf Zimmermann
> Cc: Theodore Tso; ext3-users@redhat.com
> Subject: Re: ext3 zerofree option and RedHat back port?
>
> Ulf Zimmermann wrote:
>
> > Reason I asked is this. We use currently 3Par S400 and E200 as SAN
> > arrays. The new T400 and T800 has a built in chip to do more
> intelligent
> > thin provisioning but I believe even the S400 and E200 we have will
> free
> > on the SAN level a block of a thin provisioned volume if it gets
> zero'ed
> > out. Haven't gotten around yet to test it, but I am planning on. We
> are
> > currently using 3 different file system types, one is a propriety
> from
> > Onstor for their Bobcats (NFS/CIFS heads) where I believe I have
> > observed just freeing of SAN level blocks. The two other are EXT3
and
> > OCFS2.
>
> Ok, so you really want to zero the unused blocks in-place, and e2image
> writing out a new sparsified image isn't a ton of help.
>
> The tool does that, I guess - but only on an unmounted or RO-mounted
> filesystem, right? (plus I'd triple-check that it's doing things
> correctly, opening a block device and splatting zeros around, one
hopes
> that it is!)
>
> But in any case the util itself is simple enough that building (or
even
> packaging) for fedora/EPEL should be trivial.
>
> (FWIW, there is work upstream for filesystems to actually communicate
> freed blocks to the underlying storage, just for this purpose...)
>
> -Eric

I am going to try it out by hand. Create a thin provisioned volume,
write random crap to it, then zero the blocks. See if that shrinks the
physical allocated space.

Ulf.


_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3 zerofree option and RedHat back port?
 

On Sep 23, 2008 22:30 -0500, Eric Sandeen wrote:
> Ulf Zimmermann wrote:
> Ok, so you really want to zero the unused blocks in-place, and e2image
> writing out a new sparsified image isn't a ton of help.
>
> The tool does that, I guess - but only on an unmounted or RO-mounted
> filesystem, right? (plus I'd triple-check that it's doing things
> correctly, opening a block device and splatting zeros around, one hopes
> that it is!)

That is WAY to scary for me on a mounted filesystem. It is racy if the
blocks become allocated.

Instead, what I always do when creating a sparse image for e2fsck test
cases is just "dd if=/dev/zero of=/mnt/fs/zeroes bs=64k; rm /mnt/fs/zeroes"
until the filesystem is full, then the file is deleted. This will
leave blocks "empty" for the free space in the filesystem without
any special tools.

Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3 zerofree option and RedHat back port?
 

"Ulf Zimmermann" <ulf@openlane.com> wrote:
>Can anyone tell me if the zerofree option for ext3 has been back ported
>to RedHat EL4 or EL5?

I used to maintain backports of zerofree (the kernel patch, not the
utility) to EL4 and EL5, but since I wasn't actually using them I gave
up. The last RPMs I have are from December of last year. Contact me
directly if you want them.

I don't recommend the ext3 patch as it hasn't seen much use. I regularly
use the ext2 version (on Fedora 9), but be warned that Ted has expressed
concerns about it.

Ron

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

ext3 zerofree option and RedHat back port?
 

"Ulf Zimmermann" <ulf@openlane.com> wrote:
>Reason I asked is this. We use currently 3Par S400 and E200 as SAN
>arrays. The new T400 and T800 has a built in chip to do more intelligent
>thin provisioning but I believe even the S400 and E200 we have will free
>on the SAN level a block of a thin provisioned volume if it gets zero'ed
>out. Haven't gotten around yet to test it, but I am planning on. We are
>currently using 3 different file system types, one is a propriety from
>Onstor for their Bobcats (NFS/CIFS heads) where I believe I have
>observed just freeing of SAN level blocks. The two other are EXT3 and
>OCFS2.

Interesting. A similar case I've seen recently is s3backer, a FUSE
filesystem that keeps its blocks as objects in Amazon S3:

http://code.google.com/p/s3backer/

Blocks of zeroes aren't actually stored, so they suggest using zerofree
to get rid of non-zero deleted blocks and avoid being charged for them.

Ron

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users