FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Development

 
 
LinkBack Thread Tools
 
Old 05-23-2008, 09:13 AM
Andrew Haley
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Colin Walters wrote:
> On Thu, May 22, 2008 at 5:33 PM, Felix Schwarz
> <felix.schwarz@oss.schwarz.eu> wrote:
>
>> c) How to get the correct vm type? On i386 there is a client and a server
>> vm. Is there a way I can "just" get the client vm directory (and as a
>> fallback the server vm)?
>
> Hm, if the project needs to poke at the internal VM libraries I'm not
> sure there's a clean way to do that, but an OpenJDK hacker might be
> able to give you an answer.

This is a bit mysterious. I guess I don't understand why PyLucene
would want to poke at the internal VM libraries.

>> d) JCC uses JNI so the library paths must be set correctly at runtime.
>> Unfortunately, the OpenJDK package does not add its paths to
>> /etc/ld.so.conf.d (did I miss something?) Is there another workaround
>> besides using rpath (bad, see a) or filing a bug against OpenJDK?

OpenJDK doesn't need to add its paths to /etc/ld.so.conf.d.
It calls (for example)
putenv("LD_LIBRARY_PATH=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:
/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:
/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64" + the old path.)
and re-execs java.

> Right now, you should hardcode the paths to the library in your package. See:
> http://fedoraproject.org/wiki/Packaging/Java#head-61a3ee0d05ff616ef9be2021b489610e036fd932
> Specifically, "If the JNI-using code calls System.loadLibrary you'll
> have to patch it to use System.load, passing it the full path to the
> dynamic shared object."
>
> For an example of this see
> http://cvs.fedoraproject.org/viewcvs/devel/javasqlite/
>
> This is necessary until we get multilib-awareness into OpenJDK upstream.

Interesting. What multilib-awareness do you think we need? It's not
immediately clear to me where the beinefit would be.

Andrew.

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 05-23-2008, 12:58 PM
Andrew Haley
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Colin Walters wrote:
> On Fri, May 23, 2008 at 5:13 AM, Andrew Haley <aph@redhat.com> wrote:
>> Interesting. What multilib-awareness do you think we need? It's not
>> immediately clear to me where the beinefit would be.
>
> Code calling System.loadLibrary (i.e. most software out there that
> wants to dlopen) would work.

Well yes, obviously, but in what way would multilib-awareness help?
Even if you have a mixture of 32-bit and 64-bit VMs in the same system,
the VMs themselves wouldn't need to be multilib-aware.

Andrew.

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 05-23-2008, 01:19 PM
Andrew Haley
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Colin Walters wrote:
> On Fri, May 23, 2008 at 8:58 AM, Andrew Haley <aph@redhat.com> wrote:
>> Colin Walters wrote:
>>> On Fri, May 23, 2008 at 5:13 AM, Andrew Haley <aph@redhat.com> wrote:
>>>> Interesting. What multilib-awareness do you think we need? It's not
>>>> immediately clear to me where the beinefit would be.
>>> Code calling System.loadLibrary (i.e. most software out there that
>>> wants to dlopen) would work.
>> Well yes, obviously, but in what way would multilib-awareness help?
>> Even if you have a mixture of 32-bit and 64-bit VMs in the same system,
>> the VMs themselves wouldn't need to be multilib-aware.
>
> I didn't write that section of the Java guidelines, so perhaps I'm
> misinterpreting it. But I read "...modifying IcedTea to look for JNI
> shared objects in %{_libdir}/jni" as "multilib-aware". But maybe the
> operative change is the /jni and not the %{libdir}, so the
> IcedTea/OpenJDK change is really to be "JPackage layout aware"?

That sounds much more likely. OpenJDK already looks in the correct libdir,
and adding "%{_libdir}/jni" would be a simple change.

The code that sets the path is here in java_md.c:

sprintf(new_runpath, "LD_LIBRARY_PATH="
"%s:"
"%s/lib/%s:"
"%s/../lib/%s",
jvmpath,
#ifdef DUAL_MODE
jrepath, ((wanted==64)?BIG_ARCH:SMALL_ARCH),
jrepath, ((wanted==64)?BIG_ARCH:SMALL_ARCH)
#else
jrepath, arch,
jrepath, arch
#endif
);

Andrew.


--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 06-02-2008, 12:26 PM
Felix Schwarz
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Colin Walters wrote:

On Thu, May 22, 2008 at 5:33 PM, Felix Schwarz
<felix.schwarz@oss.schwarz.eu> wrote:

I'm trying to package JCC/PyLucene. Now that OpenJDK made it into Fedora
(and EPEL 5), PyLucene with JCC could possibly be included with Fedora.


Wow, JCC is terrifying. Interacting with Lucene via Jython would be...saner =)


In my understanding Jython better suited/the best possibility if I want to
call Python from Java code. If the main program is in Python, using Jython
will restrict you on a Python feature set that is roughly the same as in
Python 2.2.


This would make PyLucene worthless for many programmers - e.g. most of my
libraries won't work as they require at least Python 2.3 (Python 2.2 did not
even had "True" and "False"!) and a big percentage needs even 2.4 or 2.5.


So I think JCC is basically the right thing to do as this is the only way you
can always use the latest Python features (even Python packages that are
written in C) and the latest Java (GCJ always had threading issues and is
generally hard to debug).


Furthermore using JCC you can even use Java from C++ without too much hassle -
quite cool I think :-)



d) JCC uses JNI so the library paths must be set correctly at runtime.
Unfortunately, the OpenJDK package does not add its paths to
/etc/ld.so.conf.d (did I miss something?) Is there another workaround
besides using rpath (bad, see a) or filing a bug against OpenJDK?


Right now, you should hardcode the paths to the library in your package. See:
http://fedoraproject.org/wiki/Packaging/Java#head-61a3ee0d05ff616ef9be2021b489610e036fd932
Specifically, "If the JNI-using code calls System.loadLibrary you'll
have to patch it to use System.load, passing it the full path to the
dynamic shared object."

For an example of this see
http://cvs.fedoraproject.org/viewcvs/devel/javasqlite/


Sorry, my wording was not detailed enough. JCC does "JNI the other way round"
so it calls Java from C++. Therefore there is no System.loadLibrary which
could be patched. Instead I have to rely on the standard linker configuration
(or use rpath).


Felix Schwarz


--
Felix Schwarz
Dipl.-Informatiker

Gubener Str. 38
10243 Berlin
Germany

www.schwarz.eu - software development and consulting

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 06-02-2008, 12:31 PM
Felix Schwarz
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Andrew Haley wrote:

Colin Walters wrote:

On Thu, May 22, 2008 at 5:33 PM, Felix Schwarz
<felix.schwarz@oss.schwarz.eu> wrote:


d) JCC uses JNI so the library paths must be set correctly at runtime.
Unfortunately, the OpenJDK package does not add its paths to
/etc/ld.so.conf.d (did I miss something?) Is there another workaround
besides using rpath (bad, see a) or filing a bug against OpenJDK?


OpenJDK doesn't need to add its paths to /etc/ld.so.conf.d.
It calls (for example)

putenv("LD_LIBRARY_PATH=/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server:
/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64:
/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/../lib/amd64" + the old path.)
and re-execs java.


Unfortunately, I don't see a possibility for me to do the same as OpenJDK does
currently (maybe I'm just too blind to see). python-jcc is a Python library
and how can I re-exec arbitrary programs which don't expect that?


Furthermore setting LD_LIBRARY_PATH is only useful for new processes so I
can't just add this env variable in the library (before loading the binary
module).


I filed a new review request so you may add comments directly in this bug:
https://bugzilla.redhat.com/show_bug.cgi?id=449360

Thanks for your input :-)
Felix Schwarz

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 06-02-2008, 01:13 PM
Andrew Haley
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Felix Schwarz wrote:

> So I think JCC is basically the right thing to do as this is the only
> way you can always use the latest Python features (even Python packages
> that are written in C) and the latest Java (GCJ always had threading
> issues and is generally hard to debug).

I'm not going to let this one past. gcj does not have "threading issues",
whatever that means, and I don't agree that it's hard to debug. The latter
is a matter of opinion, but the former a matter of fact.

>> Specifically, "If the JNI-using code calls System.loadLibrary you'll
>> have to patch it to use System.load, passing it the full path to the
>> dynamic shared object."
>>
>> For an example of this see
>> http://cvs.fedoraproject.org/viewcvs/devel/javasqlite/
>
> Sorry, my wording was not detailed enough. JCC does "JNI the other way
> round" so it calls Java from C++. Therefore there is no
> System.loadLibrary which could be patched. Instead I have to rely on the
> standard linker configuration (or use rpath).

Or use a full path in dlopen(). What libraries does JCC need to open?
Just libjvm, or others?

Andrew.

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 06-02-2008, 01:35 PM
Felix Schwarz
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Andrew Haley schrieb:

Felix Schwarz wrote:


So I think JCC is basically the right thing to do as this is the only
way you can always use the latest Python features (even Python packages
that are written in C) and the latest Java (GCJ always had threading
issues and is generally hard to debug).


I'm not going to let this one past. gcj does not have "threading issues",
whatever that means, and I don't agree that it's hard to debug. The latter
is a matter of opinion, but the former a matter of fact.


You are correct that threading inside of gcj was seemingly not that bad. I
should be more precise in my wording because these "threading issues" were
related to Python's threading - with a gcj compiled Lucene you had to be quite
careful what Python threading implementation you use (something that caused
some problems with CherryPy/TurboGears).


Specifically, "If the JNI-using code calls System.loadLibrary you'll
have to patch it to use System.load, passing it the full path to the
dynamic shared object."

For an example of this see
http://cvs.fedoraproject.org/viewcvs/devel/javasqlite/

Sorry, my wording was not detailed enough. JCC does "JNI the other way
round" so it calls Java from C++. Therefore there is no
System.loadLibrary which could be patched. Instead I have to rely on the
standard linker configuration (or use rpath).


Or use a full path in dlopen().


I may be wrong but it seems that JCC does do any dlopens. Instead the code
is just linked against the Java libraries. I may be wrong on this, though.


What libraries does JCC need to open? Just libjvm, or others?


libjava.so and libjvm.so.

fs

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 06-02-2008, 01:48 PM
Andrew Haley
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Felix Schwarz wrote:
> Andrew Haley schrieb:
>> Felix Schwarz wrote:
>>
>>>> Specifically, "If the JNI-using code calls System.loadLibrary you'll
>>>> have to patch it to use System.load, passing it the full path to the
>>>> dynamic shared object."
>>>>
>>>> For an example of this see
>>>> http://cvs.fedoraproject.org/viewcvs/devel/javasqlite/
>>> Sorry, my wording was not detailed enough. JCC does "JNI the other way
>>> round" so it calls Java from C++. Therefore there is no
>>> System.loadLibrary which could be patched. Instead I have to rely on the
>>> standard linker configuration (or use rpath).
>>
>> Or use a full path in dlopen().
>
> I may be wrong but it seems that JCC does do any dlopens. Instead the code
> is just linked against the Java libraries. I may be wrong on this, though.
>
>> What libraries does JCC need to open? Just libjvm, or others?
>
> libjava.so and libjvm.so.

OK, I understand now. I'm going to ping a few of my colleagues to see
what they think we should do.

Andrew.

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 06-02-2008, 03:41 PM
Andrew Haley
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Andrew Haley wrote:
> Felix Schwarz wrote:
>> Andrew Haley schrieb:
>>> Felix Schwarz wrote:
>>>
>>>>> Specifically, "If the JNI-using code calls System.loadLibrary you'll
>>>>> have to patch it to use System.load, passing it the full path to the
>>>>> dynamic shared object."
>>>>>
>>>>> For an example of this see
>>>>> http://cvs.fedoraproject.org/viewcvs/devel/javasqlite/
>>>> Sorry, my wording was not detailed enough. JCC does "JNI the other way
>>>> round" so it calls Java from C++. Therefore there is no
>>>> System.loadLibrary which could be patched. Instead I have to rely on the
>>>> standard linker configuration (or use rpath).
>>> Or use a full path in dlopen().
>> I may be wrong but it seems that JCC does do any dlopens. Instead the code
>> is just linked against the Java libraries. I may be wrong on this, though.
>>
>>> What libraries does JCC need to open? Just libjvm, or others?
>> libjava.so and libjvm.so.
>
> OK, I understand now. I'm going to ping a few of my colleagues to see
> what they think we should do.

I've pinged. Please open a Bugzilla item and we'll look into adding
libjvm.so and its dependencies to /etc/ld.so.conf.d and/or ld.so.cache.
This will be in the Fedora 10 timeframe. We'll also have to talk to
upstream about this.

Thanks,
Andrew.

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 06-02-2008, 06:21 PM
Felix Schwarz
 
Default PyLucene: How to package libraries that link against OpenJDK libraries?

Hi Andrew,

Andrew Haley wrote:

I've pinged. Please open a Bugzilla item and we'll look into adding
libjvm.so and its dependencies to /etc/ld.so.conf.d and/or ld.so.cache.
This will be in the Fedora 10 timeframe. We'll also have to talk to
upstream about this.


Thanks for caring about this. Bugzilla issue is:
https://bugzilla.redhat.com/show_bug.cgi?id=449456

Do you have any advise on how to proceed with python-jcc (bz 449360) in the
mean time?


Felix Schwarz

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 

Thread Tools




All times are GMT. The time now is 04:34 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org