FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian dpkg

 
 
LinkBack Thread Tools
 
Old 04-17-2011, 08:05 PM
 
Default Experiment with using 7z for gzip compression of packages

Dear developers,
I did experiment with using 7z for gzip compression for control.tar.gz and data.tar.gz components of the deb packages. Then I tried also recompress all gzip files inside the data.tar.gz and finally also tried the xz for comparison. Here are results:

0) 1132 packages (I used my apt cache): 1 076 539kB - 100%
1) recompress the control.tar.gz and data.tar.gz: 1 038 391kB - 96.5%
2) + recompress all *.gz files inside the data.tar.gz: 1 022 712kB - 95.0%
3) like 1) but with xz: 752 202kB - 69.9%
4) like 2) but with xz: 727 946kB - 67.6%

Are the numbers for 1) and 2) interesting enough to do something in this direction?

Have a nice day,
TonyMi


Notes:
I used "7z a -tgzip -mx9 file.gz file".
My "recompress" scripts are very easy and cannot handle correctly all situations so the result is not 100% precise.


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 50710.2585.1926-25342-898104751-1303070753@post.cz">http://lists.debian.org/50710.2585.1926-25342-898104751-1303070753@post.cz
 
Old 04-17-2011, 08:21 PM
Jonathan Nieder
 
Default Experiment with using 7z for gzip compression of packages

Hi,

tonda.misek@post.cz wrote:

> I did experiment with using 7z for gzip compression
[...]
> 0) 1132 packages (I used my apt cache): 1 076 539kB - 100%
> 1) recompress the control.tar.gz and data.tar.gz: 1 038 391kB - 96.5%
[...]
> I used "7z a -tgzip -mx9 file.gz file".
> My "recompress" scripts are very easy and cannot handle correctly all
> situations so the result is not 100% precise.

I wonder what 7z is doing to get this ~4.5% improvement. Could gzip learn
an option to use the same trick?


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110417202124.GA32237@elie">http://lists.debian.org/20110417202124.GA32237@elie
 
Old 04-17-2011, 09:05 PM
Christian Leber
 
Default Experiment with using 7z for gzip compression of packages

On Sun, Apr 17, 2011 at 10:05:53PM +0200, tonda.misek@post.cz wrote:

Hello Tonda

> 0) 1132 packages (I used my apt cache): 1 076 539kB - 100%
> 1) recompress the control.tar.gz and data.tar.gz: 1 038 391kB - 96.5%
> 2) + recompress all *.gz files inside the data.tar.gz: 1 022 712kB - 95.0%
> 3) like 1) but with xz: 752 202kB - 69.9%
> 4) like 2) but with xz: 727 946kB - 67.6%
>
> Are the numbers for 1) and 2) interesting enough to do something in this direction?

Surprising that the 7zip gzip compression is not more well known, it is available since serveal years as advancecomp.
(i just looked it up and i used to 2004 to reduce the live filesystem size of knoppix)

Afaik the difference is that is tries out a few parameters for each junk and uses the best one.

It would be nice if you could try out something else:
decompress all *.gz files in the data.tar and compress the .tar with xz,
this could possibly yield better compression due to the big directory size of lzma.
(after extraction the file would have to be gziped... but nevertheless it would be interesting)

Christian

--
Christian Leber <christian@leber.de> or <leber@uni-hd.de>


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110417210510.GA13995@core.home">http://lists.debian.org/20110417210510.GA13995@core.home
 
Old 04-26-2011, 08:59 PM
 
Default Experiment with using 7z for gzip compression of packages

OK, so another results, I used different data, because the apt cache is moving target and I not do backup of the last data...

0) 129 files: 268 706 070B - 100%
1) xz for control.tar.xz and data.tar.xz, all *.gz inside debs ungziped: 186 814 608B - 69.5%
2) 7z-gzip for control.tar.gz, xz for data.tar.xz, all *.gz inside debs ungziped: 186 822 308B - 69.5%
3) 7z-gzip for control.tar.gz, xz for data.tar.xz, all *.gz inside debs recompressed to xz: 187 681 956B - 69.8%
4) like 1) + PNG optimization (advdef, optipng, pngcrush, pngopt): 186 750 504B - 69.5%

Decompression of all gzip files inside the deb will save 0.3%.
Recompression of control.tar to xz gives little smaller size overall (some files bigger some smaller).
PNG optimization gives also little smaller size.

Note that I am not dpkg developer either zlib developer, I am only user.
TonyMi

> ------------ Původn* zpráva ------------
> Od: Christian Leber <christian@leber.de>
> Předmět: Re: Experiment with using 7z for gzip compression of packages
> Datum: 17.4.2011 23:14:57
> ----------------------------------------
> On Sun, Apr 17, 2011 at 10:05:53PM +0200, tonda.misek@post.cz wrote:
>
> Hello Tonda
>
> > 0) 1132 packages (I used my apt cache): 1 076 539kB - 100%
> > 1) recompress the control.tar.gz and data.tar.gz: 1 038 391kB - 96.5%
> > 2) + recompress all *.gz files inside the data.tar.gz: 1 022 712kB - 95.0%
> > 3) like 1) but with xz: 752 202kB - 69.9%
> > 4) like 2) but with xz: 727 946kB - 67.6%
> >
> > Are the numbers for 1) and 2) interesting enough to do something in this
> direction?
>
> Surprising that the 7zip gzip compression is not more well known, it is
> available since serveal years as advancecomp.
> (i just looked it up and i used to 2004 to reduce the live filesystem size of
> knoppix)
>
> Afaik the difference is that is tries out a few parameters for each junk and
> uses the best one.
>
> It would be nice if you could try out something else:
> decompress all *.gz files in the data.tar and compress the .tar with xz,
> this could possibly yield better compression due to the big directory size of
> lzma.
> (after extraction the file would have to be gziped... but nevertheless it would
> be interesting)
>
> Christian
>
> --
> Christian Leber <christian@leber.de> or <leber@uni-hd.de>
>
>
> --
> To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
> with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
> Archive: http://lists.debian.org/20110417210510.GA13995@core.home
>
>
>
>


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 51051.2878.1524-25200-439704607-1303851596@post.cz">http://lists.debian.org/51051.2878.1524-25200-439704607-1303851596@post.cz
 

Thread Tools




All times are GMT. The time now is 03:11 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org