Hello!

Please cc: me on replies, since I'm not subsribed to this list.

I'm currently looking at JSPWiki, a wiki-system written in Java. Debian
(and [K]Ubuntu) contains a very old version: 2.5.139 vs. 2.6.3. While
trying to build a new package, I detected that JSPWiki contains a lot of
third-parts .jar-files in lib/ and tests/lib/, which are already
provided by separate packages in Debian:
lib/activation.jar usr/share/java/activation.jar
lib/commons-codec-1.3.jar usr/share/java/commons-codec.jar
lib/commons-httpclient-3.0.1.jar usr/share/java/commons-httpclient.jar
lib/commons-lang-2.3.jar usr/share/java/commons-lang.jar
lib/commons-logging-api.jar usr/share/java/commons-logging-api.jar
lib/jaxen.jar usr/share/java/jaxen.jar
lib/jdom.jar usr/share/java/jdom1.jar
lib/jsp-api.jar usr/share/java/jsp-api.jar
lib/log4j-1.2.14.jar usr/share/java/log4j-1.2.jar
lib/lucene-highlighter.jar usr/share/java/lucene-highlighter.jar
lib/lucene.jar usr/share/java/lucene-core.jar
lib/nekohtml.jar usr/share/java/nekohtml.jar
lib/oro.jar usr/share/java/oro.jar
lib/oscache.jar usr/share/java/oscache.jar
lib/servlet-api.jar usr/share/java/servlet-2.3.jar
lib/mail.jar usr/share/java/gnumail.jar
tests/lib/hsqldb.jar usr/share/java/hsqldb.jar
tests/lib/xml-apis-1.0.b2.jar usr/share/java/xml-apis.jar
tests/lib/servlet.jar usr/share/java/servlet-2.3.jar
tests/lib/junit.jar usr/share/java/junit.jar
tests/lib/xercesImpl-2.6.2.jar usr/share/java/xercesImpl.jar

I now ask myself, when and were should I replace those libraries:
1. Only in the final debian/jspwiki/ tree (but build with the original
.jar files)
2. Each time during the debian/rules run
3. Once in the .orig.tar.gz (the download is only availabe as .zip)

Currently I have a mix of 2. and 3. and strip these .jar-files from the
.orig.tar.gz, but add and remove the symbolic links only during the
debian/rules run.

-rw-rw-r-- 1 12945644 16. Jul 17:45 jspwiki_2.6.3.orig2.tar.gz (vanilla)
-rw-rw-r-- 1 8784024 16. Jul 23:32 jspwiki_2.6.3.orig.tar.gz (stripped)

This saves 4 MiB and also tests, if JSPWiki is buildable with the
replacements libs. What do you think?


There are also some more .jar files, which aren't packaged separately
yet:
lib/akismet-java-1.02.jar
lib/ecs.jar
lib/freshcookies-security-0.54.jar
lib/jakarta-tablibs-standard-1.1.2.jar
lib/jakarta-taglibs-jstl-1.1.2.jar
lib/jrcs-diff.jar
lib/jsonrpc-1.0.jar
lib/jug-asl-2.0.0.jar
lib/multipartrequest.jar
lib/sandler.jar
lib/xmlrpc.jar (some < 3.1 version)
tests/lib/custom_rhino.jar
tests/lib/selenium-java-client-driver-0.9.2.jar
tests/lib/selenium-server-0.9.2-patched.jar
The files in tests/lib/ are not shipped inside the final Debian package,
the files in lib/ are. I'm not aware of any other Debian package
shipping them also, so is it okay to include them in jspwiki only or do
I need to split them into their own packages?


Also, if I strip those .jar-files from the .orig.tar.gz, do I still need
to keep their copyright and license-info in debian/copyright?


My current working version is available from
http://pint.pmhahn.de/pmhahn/debian/lenny/j/jspwiki/

BYtE
Philipp
--
Philipp Matthias Hahn <pmhahn@debian.org>
GPG/PGP: 9A540E39 @ keyrings.debian.org


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Hi Philipp,


I now ask myself, when and were should I replace those libraries:
1. Only in the final debian/jspwiki/ tree (but build with the original
.jar files)
2. Each time during the debian/rules run
3. Once in the .orig.tar.gz (the download is only availabe as .zip)


I think Debian policy doesn't allow you to depend on pre-built jars even
during build phase.


IMHO you should strip the jars from the original tar directly after
download so that they do not end up in the source package. This is the
only way you can make sure that you'll never (even unknowingly) use
non-free prebuilt binaries.


See [1] and [2] for a similar case today.

You'll then BuildDepend on Debian packages that contain the jars you
require for build and put these jars in your build-classpath. There are
different ways to do this. I currently use cdbs which provides an env
variable that you simply fill with the jars you need for your build.
Matthew has done a script called java-propose-classpath which is part of
the javahelper package. This may also prove useful.


If you have an exotic upstream build system it might be necessary to use
symlinks for build but in most cases this will probably not be necessary.


See [3] for a simple example on how I used cdbs/debhelper with ant.

If you need symlinks for runtime I propose you add them through
debhelper's dh_links during the install phase.


I am new to Debian and not a DD. So maybe one of the experienced DDs
will want to check my statement?


Florian

[1] http://lists.debian.org/debian-java/2008/07/msg00022.html
[2] http://lists.debian.org/debian-java/2008/07/msg00026.html
[3] http://mentors.debian.net/debian/pool/main/j/jaranalyzer/


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

On Thu, Jul 17, 2008 at 7:41 AM, Florian Grandel <jerico.dev@gmail.com> wrote:
> Hi Philipp,
>
>> I now ask myself, when and were should I replace those libraries:
>> 1. Only in the final debian/jspwiki/ tree (but build with the original
>> .jar files)
>> 2. Each time during the debian/rules run
>> 3. Once in the .orig.tar.gz (the download is only availabe as .zip)
>
> I think Debian policy doesn't allow you to depend on pre-built jars even
> during build phase.
>
> IMHO you should strip the jars from the original tar directly after download
> so that they do not end up in the source package. This is the only way you
> can make sure that you'll never (even unknowingly) use non-free prebuilt
> binaries.

What is wrong with cleaning up prebuilt jar files in clean target in
debian/rules? If you use this approach you don't have to repack the
upstream tarball every time there is a new version.
Is there any policy against this approach? If yes, can you please
point me to it?


Onkar


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Hi Onkar,

Onkar Shinde wrote:

What is wrong with cleaning up prebuilt jar files in clean target in
debian/rules? If you use this approach you don't have to repack the
upstream tarball every time there is a new version.
Is there any policy against this approach? If yes, can you please
point me to it?
I'm not sure there is any *explicit* policy against this, but the
definition of the clean target is that it brings the build environment
in the same state as it was before the build. Well, before the build,
the jars were there... So, basically, you land in a confusing (because
not done in any other package) and undefined state, not to think about
problems you might get with diffs on binary files.

I think it would be a no-go for any mentoring DD (something I am not).

Hope this helps,
Eric




Onkar





--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Onkar Shinde wrote:

On Thu, Jul 17, 2008 at 7:41 AM, Florian Grandel <jerico.dev@gmail.com> wrote:


Hi Philipp,



I now ask myself, when and were should I replace those libraries:
1. Only in the final debian/jspwiki/ tree (but build with the original
.jar files)
2. Each time during the debian/rules run
3. Once in the .orig.tar.gz (the download is only availabe as .zip)


I think Debian policy doesn't allow you to depend on pre-built jars even
during build phase.

IMHO you should strip the jars from the original tar directly after download
so that they do not end up in the source package. This is the only way you
can make sure that you'll never (even unknowingly) use non-free prebuilt
binaries.



What is wrong with cleaning up prebuilt jar files in clean target in
debian/rules? If you use this approach you don't have to repack the
upstream tarball every time there is a new version.

You want to completely avoid shipping upstream's bundled jars, since
quite often the upstream has a lot more lenient
policy towards what they consider 'open source' then what debian
considers to be acceptable for redistribution.


Example: Apache Foundation projects and BCL-licensed JARs prior to the
drafting of the third party license policy a few years ago.


cheers,
dalibor topic

Is there any policy against this approach? If yes, can you please
point me to it?


Onkar






--
************************************************** *****************
Dalibor Topic Tel: (+49 40) 23 646 738
Java F/OSS Ambassador AIM: robiladonaim
Sun Microsystems GmbH Mobile: (+49 177) 2664 192
Nagelsweg 55 http://openjdk.java.net
D-20097 Hamburg mailtoalibor.Topic@sun.com
Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten
Amtsgericht München: HRB 161028
Geschäftsführer: Thomas Schröder, Wolfgang Engels, Dr. Roland Bömer
Vorsitzender des Aufsichtsrates: Martin Häring


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

On Thu, Jul 17, 2008 at 2:01 PM, Eric Lavarde <deb@zorglub.s.bawue.de> wrote:
> Hi Onkar,
>
> Onkar Shinde wrote:
>>
>> What is wrong with cleaning up prebuilt jar files in clean target in
>> debian/rules? If you use this approach you don't have to repack the
>> upstream tarball every time there is a new version.
>> Is there any policy against this approach? If yes, can you please
>> point me to it?
>
> I'm not sure there is any *explicit* policy against this, but the definition
> of the clean target is that it brings the build environment in the same
> state as it was before the build. Well, before the build, the jars were
> there... So, basically, you land in a confusing (because not done in any
> other package) and undefined state, not to think about problems you might
> get with diffs on binary files.
> I think it would be a no-go for any mentoring DD (something I am not).


If there is no policy about this then I guess this good time to have one.
Because I am always told (in #ubuntu-motu) to keep the orig.tar.gz as
close as possible to upstream tarball and hence use clean target to
delete jar files.

I am still learning lot of things about packaging so I would like to
get habituated to right approach. And if the approach is backed by a
policy then even better.


Onkar


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

On Thu Jul 17 00:36, Philipp Matthias Hahn wrote:

> Also, if I strip those .jar-files from the .orig.tar.gz, do I still need
> to keep their copyright and license-info in debian/copyright?
>

Hi Philipp,

If not explicit policy then it is the norm (and required if there are
any licence issues as has been mentioned) to repack the tarball.

This happens once before packaging (per-release) and not in the normal
build procedure.

In this case you should change the upstream version (normally by
appending .dfsg, so the tarball would be foo_1.2.3.dfsg.orig.tar.gz, and
the version of the package would be 1.2.3.dfsg-1 etc).

You also must document this stripping process in the copyright file,
but you shouldn't include the licences of the files which have been
removed. A recommended practice is to include a get-orig-source target
in the rules file which can be used to obtain debian source from a
pristine tarball and possibly download the original as well. This helps
document what you have done and makes your life easier for new upstream
releases.

I have provided in the javahelper suite as part of jh_makepkg (which
creates a template package from upstream source) a --clean option which
will guess at which files need to be removed. Obviously, this is just an
aid, you will need to check it has done the right thing.

Matt

--
Matthew Johnson

Hi Onkar,

Onkar Shinde schrieb:

If there is no policy about this then I guess this good time to have one.
Because I am always told (in #ubuntu-motu) to keep the orig.tar.gz as
close as possible to upstream tarball and hence use clean target to
delete jar files.

I am still learning lot of things about packaging so I would like to
get habituated to right approach. And if the approach is backed by a
policy then even better.


I think you have to differentiate here between the binaries (e.g. jars)
that have been pre-built upstream and the binaries you produce during
your own build.


IMO binaries that come within the source tarball and have been built
/upstream/ should be stripped out with a get-orig-source target in
debian/rules. This is a practice I have seen in several Debian Java
packages now and which I like for the reasons I laid out in my earlier post.


The binaries that you build /yourself/ during the build process should
be stripped out within the clean target so that you return to the state
your folder hierarchy was in before the build.


Hope this helps to clarify the issue.

Florian


--
To UNSUBSCRIBE, email to debian-java-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org