FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo Development

 
 
LinkBack Thread Tools
 
Old 01-19-2012, 08:19 AM
"Paweł Hajdan, Jr."
 
Default doubtful about libjpeg-turbo vs. libjpeg binary compatibility

While dealing with <https://bugs.gentoo.org/show_bug.cgi?id=393471> I
started discussing with developers working on libjpeg-turbo support in
WebKit, and I learned that despite
<http://archives.gentoo.org/gentoo-dev/msg_e67bedf25dd178ec09a325a1220724e6.xml>
libjpeg-turbo is not necessarily binary compatible with libjpeg, and
even with different versions of itself.

Here's why. See
<http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/trunk/libjpeg.txt?revision=299&view=markup>
and search for "as a shared library". I'll paste the relevant quote here
anyway:

> While you can build the JPEG library as a shared library if the whim strikes
> you, we don't really recommend it. The trouble with shared libraries is that
> at some point you'll probably try to substitute a new version of the library
> without recompiling the calling applications. That generally doesn't work
> because the parameter struct declarations usually change with each new
> version. In other words, the library's API is *not* guaranteed binary
> compatible across versions; we only try to ensure source-code compatibility.

The particular problem with www-client/chromium is caused by
<http://trac.webkit.org/changeset/101286/trunk/Source/WebCore/platform/image-decoders/jpeg/JPEGImageDecoder.cpp>
which sort of "hardcodes" in the compiled binary whether it was compiled
against libjpeg-turbo with swizzle support (whatever that is) or not.

The real problem here is that with above "no guarantee" of binary
compatibility such a solution may be considered legitimate, especially
that for e.g. Google Chrome the bundled copy of libjpeg-turbo is always
used.

What do you guys think we should do with this on the Gentoo side? At
this moment I've just reverted the mentioned change in
www-client/chromium ebuild, but it's not feasible to keep a local patch
too long (it needs rebasing too often).

I was thinking about changing SONAMES, which would trigger rebuilds make
things work, but then don't we rely on specific libjpeg SONAMES for
binary-only software to work? Or does such binary-only software just use
libjpeg-6b?

Are there some other solutions we could apply on the Gentoo side? The
main point here is that Chromium upstream considers
<http://trac.webkit.org/changeset/101286/trunk/Source/WebCore/platform/image-decoders/jpeg/JPEGImageDecoder.cpp>
a legitimate change, and based on the referenced
<http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/trunk/libjpeg.txt?revision=299&view=markup>
disclaimer about no guarantee of binary compatibility, I think it makes
sense.
 
Old 01-19-2012, 08:35 AM
Samuli Suominen
 
Default doubtful about libjpeg-turbo vs. libjpeg binary compatibility

On 01/19/2012 11:19 AM, "Paweł Hajdan, Jr." wrote:

While dealing with<https://bugs.gentoo.org/show_bug.cgi?id=393471> I
started discussing with developers working on libjpeg-turbo support in
WebKit, and I learned that despite
<http://archives.gentoo.org/gentoo-dev/msg_e67bedf25dd178ec09a325a1220724e6.xml>
libjpeg-turbo is not necessarily binary compatible with libjpeg, and
even with different versions of itself.

Here's why. See
<http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/trunk/libjpeg.txt?revision=299&view=markup>
and search for "as a shared library". I'll paste the relevant quote here
anyway:


While you can build the JPEG library as a shared library if the whim strikes
you, we don't really recommend it. The trouble with shared libraries is that
at some point you'll probably try to substitute a new version of the library
without recompiling the calling applications. That generally doesn't work
because the parameter struct declarations usually change with each new
version. In other words, the library's API is *not* guaranteed binary
compatible across versions; we only try to ensure source-code compatibility.


The particular problem with www-client/chromium is caused by
<http://trac.webkit.org/changeset/101286/trunk/Source/WebCore/platform/image-decoders/jpeg/JPEGImageDecoder.cpp>
which sort of "hardcodes" in the compiled binary whether it was compiled
against libjpeg-turbo with swizzle support (whatever that is) or not.

The real problem here is that with above "no guarantee" of binary
compatibility such a solution may be considered legitimate, especially
that for e.g. Google Chrome the bundled copy of libjpeg-turbo is always
used.

What do you guys think we should do with this on the Gentoo side?At


always use system libraries and i'm in process of dropping keywording
from media-libs/jpeg wrt[1] since we have no need for source slot of it


[1] http://bugs.gentoo.org/398909

both providers will be left in the virtual/jpeg, but only libjpeg-turbo
will be keyworded (and thus supported), eliminating rest of the issues
raised here


- Samuli


this moment I've just reverted the mentioned change in
www-client/chromium ebuild, but it's not feasible to keep a local patch
too long (it needs rebasing too often).

I was thinking about changing SONAMES, which would trigger rebuilds make
things work, but then don't we rely on specific libjpeg SONAMES for
binary-only software to work? Or does such binary-only software just use
libjpeg-6b?

Are there some other solutions we could apply on the Gentoo side? The
main point here is that Chromium upstream considers
<http://trac.webkit.org/changeset/101286/trunk/Source/WebCore/platform/image-decoders/jpeg/JPEGImageDecoder.cpp>
a legitimate change, and based on the referenced
<http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/trunk/libjpeg.txt?revision=299&view=markup>
disclaimer about no guarantee of binary compatibility, I think it makes
sense.
 
Old 01-19-2012, 08:45 AM
Michał Górny
 
Default doubtful about libjpeg-turbo vs. libjpeg binary compatibility

On Thu, 19 Jan 2012 10:19:27 +0100
""Paweł Hajdan, Jr."" <phajdan.jr@gentoo.org> wrote:

> While dealing with <https://bugs.gentoo.org/show_bug.cgi?id=393471> I
> started discussing with developers working on libjpeg-turbo support in
> WebKit, and I learned that despite
> <http://archives.gentoo.org/gentoo-dev/msg_e67bedf25dd178ec09a325a1220724e6.xml>
> libjpeg-turbo is not necessarily binary compatible with libjpeg, and
> even with different versions of itself.
>
> Here's why. See
> <http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/trunk/libjpeg.txt?revision=299&view=markup>
> and search for "as a shared library". I'll paste the relevant quote
> here anyway:
>
> > While you can build the JPEG library as a shared library if the
> > whim strikes you, we don't really recommend it. The trouble with
> > shared libraries is that at some point you'll probably try to
> > substitute a new version of the library without recompiling the
> > calling applications. That generally doesn't work because the
> > parameter struct declarations usually change with each new
> > version. In other words, the library's API is *not* guaranteed
> > binary compatible across versions; we only try to ensure
> > source-code compatibility.
>
> The particular problem with www-client/chromium is caused by
> <http://trac.webkit.org/changeset/101286/trunk/Source/WebCore/platform/image-decoders/jpeg/JPEGImageDecoder.cpp>
> which sort of "hardcodes" in the compiled binary whether it was
> compiled against libjpeg-turbo with swizzle support (whatever that
> is) or not.
>
> The real problem here is that with above "no guarantee" of binary
> compatibility such a solution may be considered legitimate, especially
> that for e.g. Google Chrome the bundled copy of libjpeg-turbo is
> always used.
>
> What do you guys think we should do with this on the Gentoo side? At
> this moment I've just reverted the mentioned change in
> www-client/chromium ebuild, but it's not feasible to keep a local
> patch too long (it needs rebasing too often).
>
> I was thinking about changing SONAMES, which would trigger rebuilds
> make things work, but then don't we rely on specific libjpeg SONAMES
> for binary-only software to work? Or does such binary-only software
> just use libjpeg-6b?

Hmm, does this mean the ABI differs on runtime compilation options?
SONAME should be changed with new versions, not options. If upstream
does allow things like that, that's bad. If chromium uses that, it's
even worse.

I'd go with the simplest solution which is either enabling
the particular configure option unconditionally or disabling it.
If possible -- synced with SONAME change. But it should be upstream
SONAME change (considering they do change SONAMEs when releasing
binary-incompatible versions); we don't really want to go the Debian
way, keeping our own SONAME cycles.

--
Best regards,
Michał Górny
 
Old 01-19-2012, 08:58 AM
"Paweł Hajdan, Jr."
 
Default doubtful about libjpeg-turbo vs. libjpeg binary compatibility

On 1/19/12 10:45 AM, Michał Górny wrote:
> Hmm, does this mean the ABI differs on runtime compilation options?

No, at least I'm not aware of such a thing.

I'd sum it up as "libjpeg-turbo is not binary-compatible with libjpeg
and also with a different version of itself, and is not supposed to be".

> SONAME should be changed with new versions, not options. If upstream
> does allow things like that, that's bad.

I think upstream keeps the SONAME at "libjpeg.so.8" . And as indicated
by <https://bugs.gentoo.org/show_bug.cgi?id=393471#c3> and the next
comment, libjpeg-turbo-1.1.1 and libjpeg-1.1.90 shouldn't have the same
SONAME (they're not binary compatible wrt JCS extensions Chromium uses).

> If chromium uses that, it's even worse.

That's what I initially thought, but the "as a shared library" paragraph
I quoted from
<http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/trunk/libjpeg.txt?revision=299&view=markup>
strongly suggests it's legitimate to do what they've done.

> But it should be upstream SONAME change (considering they do change
> SONAMEs when releasing binary-incompatible versions); we don't really
> want to go the Debian way, keeping our own SONAME cycles.

I think OpenBSD also has its own SONAME cycles, which is not necessarily
too bad (except for binary-only software). In Gentoo, we also have our
own SONAME for V8 JavaScript engine (upstream only provides
infrastructure for setting SONAME, i.e. a build system switch).
 
Old 01-19-2012, 03:56 PM
Mike Frysinger
 
Default doubtful about libjpeg-turbo vs. libjpeg binary compatibility

On Thursday 19 January 2012 04:35:43 Samuli Suominen wrote:
> On 01/19/2012 11:19 AM, "Paweł Hajdan, Jr." wrote:
> > While dealing with<https://bugs.gentoo.org/show_bug.cgi?id=393471> I
> > started discussing with developers working on libjpeg-turbo support in
> > WebKit, and I learned that despite
> > <http://archives.gentoo.org/gentoo-dev/msg_e67bedf25dd178ec09a325a1220724
> > e6.xml> libjpeg-turbo is not necessarily binary compatible with libjpeg,
> > and even with different versions of itself.
> >
> > Here's why. See
> > <http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/trunk/libj
> > peg.txt?revision=299&view=markup> and search for "as a shared library".
> > I'll paste the relevant quote here
> >
> > anyway:
> >> While you can build the JPEG library as a shared library if the whim
> >> strikes you, we don't really recommend it. The trouble with shared
> >> libraries is that at some point you'll probably try to substitute a new
> >> version of the library without recompiling the calling applications.
> >> That generally doesn't work because the parameter struct declarations
> >> usually change with each new version. In other words, the library's
> >> API is *not* guaranteed binary compatible across versions; we only try
> >> to ensure source-code compatibility.
> >
> > The particular problem with www-client/chromium is caused by
> > <http://trac.webkit.org/changeset/101286/trunk/Source/WebCore/platform/im
> > age-decoders/jpeg/JPEGImageDecoder.cpp> which sort of "hardcodes" in the
> > compiled binary whether it was compiled against libjpeg-turbo with
> > swizzle support (whatever that is) or not.
> >
> > The real problem here is that with above "no guarantee" of binary
> > compatibility such a solution may be considered legitimate, especially
> > that for e.g. Google Chrome the bundled copy of libjpeg-turbo is always
> > used.
> >
> > What do you guys think we should do with this on the Gentoo side?At
>
> always use system libraries

that doesn't help. the libjpeg turbo peeps themselves have said they don't
guarantee compatibility across their own versions.

> and i'm in process of dropping keywording
> from media-libs/jpeg wrt[1] since we have no need for source slot of it

err, no. libjpeg-turbo doesn't provide the older SONAME's like jpeg does.
those SLOT's aren't going anywhere (SLOT!=0).

history has shown that the canonical version stays around while the
derivatives come and go. so until the seemingly braindead ABI policies of
libjpeg-turbo can get sorted out, i don't think we can drop KEYWORDS on SLOT=0
jpeg.
-mike
 
Old 01-19-2012, 04:02 PM
Samuli Suominen
 
Default doubtful about libjpeg-turbo vs. libjpeg binary compatibility

On 01/19/2012 06:56 PM, Mike Frysinger wrote:

On Thursday 19 January 2012 04:35:43 Samuli Suominen wrote:

On 01/19/2012 11:19 AM, "Paweł Hajdan, Jr." wrote:

While dealing with<https://bugs.gentoo.org/show_bug.cgi?id=393471> I
started discussing with developers working on libjpeg-turbo support in
WebKit, and I learned that despite
<http://archives.gentoo.org/gentoo-dev/msg_e67bedf25dd178ec09a325a1220724
e6.xml> libjpeg-turbo is not necessarily binary compatible with libjpeg,
and even with different versions of itself.

Here's why. See
<http://libjpeg-turbo.svn.sourceforge.net/viewvc/libjpeg-turbo/trunk/libj
peg.txt?revision=299&view=markup> and search for "as a shared library".
I'll paste the relevant quote here

anyway:

While you can build the JPEG library as a shared library if the whim
strikes you, we don't really recommend it. The trouble with shared
libraries is that at some point you'll probably try to substitute a new
version of the library without recompiling the calling applications.
That generally doesn't work because the parameter struct declarations
usually change with each new version. In other words, the library's
API is *not* guaranteed binary compatible across versions; we only try
to ensure source-code compatibility.


The particular problem with www-client/chromium is caused by
<http://trac.webkit.org/changeset/101286/trunk/Source/WebCore/platform/im
age-decoders/jpeg/JPEGImageDecoder.cpp> which sort of "hardcodes" in the
compiled binary whether it was compiled against libjpeg-turbo with
swizzle support (whatever that is) or not.

The real problem here is that with above "no guarantee" of binary
compatibility such a solution may be considered legitimate, especially
that for e.g. Google Chrome the bundled copy of libjpeg-turbo is always
used.

What do you guys think we should do with this on the Gentoo side?At


always use system libraries


that doesn't help. the libjpeg turbo peeps themselves have said they don't
guarantee compatibility across their own versions.


it's forward compatible, which is all we should care about


and i'm in process of dropping keywording
from media-libs/jpeg wrt[1] since we have no need for source slot of it


err, no. libjpeg-turbo doesn't provide the older SONAME's like jpeg does.
those SLOT's aren't going anywhere (SLOT!=0).


i said "source slot" which was supposed to mean SLOT="0" alone, the
SLOT="62" will still have keywording, SLOT="7" is not used anything in
tree but can still have the keywording (unintresting to this topic)



history has shown that the canonical version stays around while the
derivatives come and go.


and the ones before never had this level of people behind it, and
projects where people get paid to get it working, like fedora



so until the seemingly braindead ABI policies of
libjpeg-turbo can get sorted out, i don't think we can drop KEYWORDS on SLOT=0
jpeg.


the only thing that's really broken is building against libjpeg-turbo,
and then switching to ijg's jpeg without rebuilding the package


and downgrading of libjpeg-turbo

both of which are not worth of supporting
 
Old 01-19-2012, 04:16 PM
"Paweł Hajdan, Jr."
 
Default doubtful about libjpeg-turbo vs. libjpeg binary compatibility

On 1/19/12 6:02 PM, Samuli Suominen wrote:
> On 01/19/2012 06:56 PM, Mike Frysinger wrote:
>> that doesn't help. the libjpeg turbo peeps themselves have said they
>> don't
>> guarantee compatibility across their own versions.
>
> it's forward compatible, which is all we should care about

Just a note: that'd require me to DEPEND on a recent enough version of
libjpeg-turbo in the www-client/chromium ebuild, which would mean either:

a) changing the virtual/jpeg dependency to >=libjpeg-turbo-...

b) adding a versioned virtual/jpeg to require a recent enough
libjpeg-turbo (but then what about libjpeg versions - it can easily
become complicated)

c) similar to a) but also adding || ( >=libjpeg-turbo-... libjpeg )

With proper SONAMEs in libjpeg-turbo that would be a non-issue (bump the
SONAME on incompatible change, revdep-rebuild does the rest).

> the only thing that's really broken is building against libjpeg-turbo,
> and then switching to ijg's jpeg without rebuilding the package

I'm fine with not supporting that, provided keywords for libjpeg are
dropped (except the 62 slot iirc).

> and downgrading of libjpeg-turbo

I think this one should "just work", or at least not allow obvious
mistakes. See my a) b) c) notes above in this e-mail for possible
solutions and ideal SONAME.
 
Old 01-19-2012, 04:42 PM
Samuli Suominen
 
Default doubtful about libjpeg-turbo vs. libjpeg binary compatibility

On 01/19/2012 07:16 PM, "Paweł Hajdan, Jr." wrote:

On 1/19/12 6:02 PM, Samuli Suominen wrote:

On 01/19/2012 06:56 PM, Mike Frysinger wrote:

that doesn't help. the libjpeg turbo peeps themselves have said they
don't
guarantee compatibility across their own versions.


it's forward compatible, which is all we should care about


Just a note: that'd require me to DEPEND on a recent enough version of
libjpeg-turbo in the www-client/chromium ebuild, which would mean either:

a) changing the virtual/jpeg dependency to>=libjpeg-turbo-...


will be done soon as 1.2.0 is released and stabilized, i'd like to skip
1.1.90



and downgrading of libjpeg-turbo


I think this one should "just work", or at least not allow obvious
mistakes. See my a) b) c) notes above in this e-mail for possible
solutions and ideal SONAME.



a) is fine, preventing any downgrades. a fatal check, like glibc and
qt4 has to prevent downgrading is an option too, but a bit overkill imho
 
Old 01-30-2012, 07:25 AM
"Paweł Hajdan, Jr."
 
Default doubtful about libjpeg-turbo vs. libjpeg binary compatibility

On 1/19/12 6:42 PM, Samuli Suominen wrote:
>> a) changing the virtual/jpeg dependency to>=libjpeg-turbo-...
>
> will be done soon as 1.2.0 is released and stabilized, i'd like to skip
> 1.1.90

Sounds good to me.

> a) is fine, preventing any downgrades. a fatal check, like glibc and
> qt4 has to prevent downgrading is an option too, but a bit overkill imho

Agreed, in my opinion such a check would be a bad idea. People might
have reasons to temporarily downgrade libjpeg-turbo.

An elog notice could be worthwhile though.
 

Thread Tools




All times are GMT. The time now is 08:26 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org