FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 05-08-2008, 01:43 PM
Doug Goldstein
 
Default RFC: lzma tarball usage

Ciaran McCreesh wrote:

On Thu, 08 May 2008 09:32:34 -0400
Doug Goldstein <cardoe@gentoo.org> wrote:


Ciaran McCreesh wrote:


On Thu, 08 May 2008 09:17:08 -0400
Doug Goldstein <cardoe@gentoo.org> wrote:


It's troubling to me that projects are using lzma when it's on disk
format isn't even final and the project has security issues.


You mean projects like 'GNU tar'?


As far as I know Ciaran, all GNU projects have switched or are in the
process of switching to lzma over bzip2. I believe the issue in

question which prompted this original e-mail was due to coreutils.
But I could be wrong.



You miss my point. GNU tar sometimes changes its on disk format (and
will be doing so again at some point for xattrs), and it's had security
issues.


Fair enough. However, newer GNU tar's are able to untar the older
formats. If you read the lzma changelogs, it appears to imply that newer
ones won't be able to read older formats. The changelog specifically
states if a user they are handling the issue "gracefully" by telling the
user to upgrade or downgrade their lzma.

--
gentoo-dev@lists.gentoo.org mailing list
 
Old 05-08-2008, 02:30 PM
 
Default RFC: lzma tarball usage

Ciaran McCreesh <ciaran.mccreesh@googlemail.com> writes:

> You miss my point. GNU tar sometimes changes its on disk format (and
> will be doing so again at some point for xattrs)

It's not really important to the discussion, but...

The TAR format is designed as such that on disk formats can be extended
without breaking entirely backward compatibility. For what it's worth,
extended attributes and ACLs are already supported by star (Schilling's)
and bsdtar (libarchive). The fact that GNU tar doesn't support them at
the moment is a different problem. On the other hand, even if the GNU
tar does not support reading those attributes, it does handle them
gracefully, warning the user of unknown extended headers, and then
proceeding to unpack the data without preserving the extended
attributes.

So what Doug said stands perfectly and does not interest GNU tar at all.

--
Diego "Flameeyes" Petten˛
http://blog.flameeyes.eu/
 
Old 05-08-2008, 02:33 PM
Robert Buchholz
 
Default RFC: lzma tarball usage

On Thursday 08 May 2008, Doug Goldstein wrote:
> Additionally to follow myself up, I believe one of the security
> issues was execution of arbitrary data either when untarred or just
> decompressed (assuming a *specially crafted lzma file).

Can you please point me to the location where this is mentioned. I read
through the lzma git log, and I all I could find was data corruption
(which usually is not a security issue) and the mention of the
word "security" inside the announcement.

Thanks,
Robert
 
Old 05-08-2008, 06:45 PM
Mart Raudsepp
 
Default RFC: lzma tarball usage

On K, 2008-05-07 at 15:34 +0200, Fabian Groffen wrote:
> On 07-05-2008 16:23:12 +0300, Mart Raudsepp wrote:
> > This is a plea and also a request for comments on the matter of
> > using .tar.lzma tarballs or not, and for what packages this is
> > acceptable and for what not.
>
> Just as a little background:
> GNU chose to switch from bzip2 to lzma, for it produces smaller files
> (less bandwith) and decompresses faster.
>
> They no longer provide the bzip2 versions of archives for newer releases
> IIRC, so it's either tar.gz or tar.lzma.
>
> > I'd be happy if some other unpacker is used than lzma-utils - one that
> > does not depend on libstdc++ - I'm sure it can be done, heck it's done
> > in integrated form in some other projects in less than a couple
> > kilobytes of code for the unpacking from a VFS. Meanwhile please
> > consider using the upstream provided .tar.gz tarballs instead and not
> > roll patchsets in .lzma just cause you can.
>
> See above why it might not just be "'cause you can".

"and not roll patchsets in .lzma just cause you can". Cause you can
applies to patchsets mostly. But using .tar.lzma instead of .tar.gz is
also a "because they are available and therefore I can use it"
neglecting the issues of

a) on-disk format is supposedly not even finalized; high potential
breakage of packages in existing ebuilds once lzma-utils gets updated
b) The currently used decompressor package links to libstdc++ (and
portage uses lzma, not lzmadec) unconditionally for most components
´╗┐c) Potential security issues; details needed, but for other reasons it
makes sense to ban .tar.lzma's until a new C only rewritten lzma-utils
comes along anyway
d) too early adoption in critical system packages - once above issues
are solved, higher levels should be using it first, before critical
system packages (for example shows in the circular dep hell with m4)
e) It has been suggested the support should have been added with new
EAPI instead of local build deps (some of which are missing, for
instance in the hand-rolled for-no-reason-whatsoever .tar.lzma format
net-tools doesn't have a dep in addition to using lzma for no good
reason)

Probably some more.
Base-system, please stop using .tar.lzma for now, thank you.


--
Mart Raudsepp
Gentoo Developer
Mail: leio@gentoo.org
Weblog: http://planet.gentoo.org/developers/leio
 
Old 05-08-2008, 07:09 PM
Fabian Groffen
 
Default RFC: lzma tarball usage

On 08-05-2008 21:45:00 +0300, Mart Raudsepp wrote:
> d) too early adoption in critical system packages - once above issues
> are solved, higher levels should be using it first, before critical
> system packages (for example shows in the circular dep hell with m4)

been there, done that.

> e) It has been suggested the support should have been added with new
> EAPI instead of local build deps (some of which are missing, for
> instance in the hand-rolled for-no-reason-whatsoever .tar.lzma format
> net-tools doesn't have a dep in addition to using lzma for no good
> reason)

Chill, relax and cool down. Instead, just ask those who decided to
follow upstream why and if they have even thought about the issues you
brought up.


--
Fabian Groffen
Gentoo on a different level
--
gentoo-dev@lists.gentoo.org mailing list
 
Old 05-08-2008, 07:17 PM
"Santiago M. Mola"
 
Default RFC: lzma tarball usage

On Thu, May 8, 2008 at 9:09 PM, Fabian Groffen <grobian@gentoo.org> wrote:
>
> > e) It has been suggested the support should have been added with new
> > EAPI instead of local build deps (some of which are missing, for
> > instance in the hand-rolled for-no-reason-whatsoever .tar.lzma format
> > net-tools doesn't have a dep in addition to using lzma for no good
> > reason)
>
> Chill, relax and cool down. Instead, just ask those who decided to
> follow upstream why and if they have even thought about the issues you
> brought up.
>

Note that we're also speaking about downstream lzma archives. Like in
sys-apps/net-tools, where lzma hasn't been adopted even by upstream.

Regards,
--
Santiago M. Mola
Jabber ID: cooldwind@gmail.com
--
gentoo-dev@lists.gentoo.org mailing list
 
Old 05-08-2008, 07:21 PM
Mart Raudsepp
 
Default RFC: lzma tarball usage

On N, 2008-05-08 at 21:09 +0200, Fabian Groffen wrote:
> > e) It has been suggested the support should have been added with new
> > EAPI instead of local build deps (some of which are missing, for
> > instance in the hand-rolled for-no-reason-whatsoever .tar.lzma format
> > net-tools doesn't have a dep in addition to using lzma for no good
> > reason)
>
> Chill, relax and cool down.

Well, I said how it is. I don't see anything in it that indicates I am
so upset and angry that I need to do these things. I did however loose
hours of work time, but that's life.

> Instead, just ask those who decided to
> follow upstream why and if they have even thought about the issues you
> brought up.

This is what I am doing with this as well, in addition to the bug
reports. But as this is widespread to at least 4-6 system packages, I
brought it up here as well to ensure this is not something I have to
fight against in overlays and time wastes continuously in the future.
Oh and net-tools has not distributed anything in .tar.lzma, so this has
nothing to do with following upstream in any shape or form in this case.

--
Mart Raudsepp
Gentoo Developer
Mail: leio@gentoo.org
Weblog: http://planet.gentoo.org/developers/leio
 
Old 05-09-2008, 01:04 AM
Ryan Hill
 
Default RFC: lzma tarball usage

On Thu, 08 May 2008 09:17:08 -0400
Doug Goldstein <cardoe@gentoo.org> wrote:

> Ryan Hill wrote:
> > The new lzma-utils codebase uses liblzma, written in C. It's at the
> > alpha stage but supposedly supports encoding/decoding the current
> > lzma format "well enough" (;P). It probably has some fun bugs to
> > find and squish.
> >
> > http://sf.net/mailarchive/forum.php?thread_name=200804251652.58484.lasse.col lin%40tukaani.org&forum_name=lzmautils-announce

> According to the mailing list this change was done to fix security
> holes in the format and also resulted in a slightly different format
> that's incompatible with the previous verion. So lzma 5.x and higher
> will be a different on disk format. It's troubling to me that
> projects are using lzma when it's on disk format isn't even final and
> the project has security issues.

The current format is fine. It's the new format that has
design/security issues. Yes the formats are incompatible, but so
are .tar.lzma and .7z, which are both lzma. Either way I was just
offering it as a data point. I have no real opinion one way or the
other.


--
fonts, gcc-porting, by design, by neglect
mips, treecleaner, for a fact or just for effect
wxwidgets @ gentoo EFFD 380E 047A 4B51 D2BD C64F 8AA8 8346 F9A4 0662
 
Old 05-09-2008, 08:37 AM
James Cloos
 
Default RFC: lzma tarball usage

>>>>> "Doug" == Doug Goldstein <cardoe@gentoo.org> writes:

Doug> If you read the lzma changelogs, it appears to imply that newer
Doug> ones won't be able to read older formats. The changelog
Doug> specifically states if a user they are handling the issue
Doug> "gracefully" by telling the user to upgrade or downgrade their
Doug> lzma.

My understanding is that the new utils will be able to uncompress
current archives, but will not be able to create them and also will
be unable to convert current archives to the new format w/o a full
(and time-consuming) uncompress/compress cycle.

-JimC
--
James Cloos <cloos@jhcloos.com> OpenPGP: 1024D/ED7DAEA6
--
gentoo-dev@lists.gentoo.org mailing list
 
Old 05-10-2008, 07:32 AM
Mike Frysinger
 
Default RFC: lzma tarball usage

On Wednesday 07 May 2008, Fabian Groffen wrote:
> m4, that one gave me some headaches, because lzma-utils required some
> eautoreconf, which introduced a nice cycle.

must have been a prefix-only bug as the version in the tree never did
-mike
 

Thread Tools




All times are GMT. The time now is 07:18 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ę2007 - 2008, www.linux-archive.org