Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Ubuntu Development (http://www.linux-archive.org/ubuntu-development/)
-   -   Packaging large Java software stacks ? (http://www.linux-archive.org/ubuntu-development/234586-packaging-large-java-software-stacks.html)

Thierry Carrez 01-26-2009 08:49 AM

Packaging large Java software stacks ?
 
Fellow developers,

It is difficult to integrate the large recent Java software stacks
(Glassfish, Geronimo...) in a Linux distribution in general. The key
reason is that most of those stacks require very precise versions of
libraries (JARs) to run and to build. They won't work with the latest
version of libraries as those might change APIs and/or key behavior.
Java developers are used to pick specific JAR versions and assemble the
exact needed stack, they don't want to care about sharing their
dependencies with other packages, or about dependencies being upgraded.
Tools like Maven help them in this endeavor, and they rely very heavily
on this external code : dozens of those JARs are usually needed at
runtime, hundreds of those at build-time.

That makes most attempts to properly package large Java software in
Ubuntu to fail for two reasons.

(1) Need precisely-versioned artifacts
Those stacks need, as build dependencies and as runtime dependencies,
very precise versions of JARs. They won't build or run with a different
one. Using a more recent one might break functionality in a creative
way. A maven-based build will sometimes require 6 different versions of
the same JAR. In our packages, we usually offer only one version,
in corner cases one minor version for each major version. In some cases
the Java software will run/build with ours, in most cases it won't.

(2) Building entirely from source
Java "compilation" (in fact bytecoding) requires lots of JARs to be
present, because they are used to check all external method signatures.
Geronimo and Glassfish builds will require hundreds of them. Packaging
all those build dependencies from source is a huge work, and each of
those build dependencies, in turn, will require more. Combined with the
first problem, this makes packaging those stacks too much work for so
little to gain (they are usually easily installable by
downloading/unpacking the upstream tarball).

I am looking at ways to solve these issues, and would very much like to
hear ideas from our awesome community :)

Potential solutions to solve (1) include using a parallel delivery
mechanism for JARs, separated from the usual Ubuntu packages, something
that could hook into Maven repositories... or get ready to create a
separate package for each version of each JAR.

Potential solutions to solve (2) include evolving our build-from-source
policy to accept that JARs that would only be used as method-checking
media during Java compilation to be considered part of the source...

A solution to workaround both problems would be to avoid targeting our
built-from-source repositories for such Java software and pack them with
their binary dependencies (or as binary directly)...

Please let me know what you think of all this problem. I'm pretty sure
the Java world will one day gets its senses back and have a more
distribution-friendly dependency system, but we probably need a
solution until such day happens... or forget about such Java software
altogether.

--
Thierry Carrez
Ubuntu server team

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Daniel Holbach 01-26-2009 09:10 AM

Packaging large Java software stacks ?
 
Thierry Carrez schrieb:
> Potential solutions to solve (1) include using a parallel delivery
> mechanism for JARs, separated from the usual Ubuntu packages, something
> that could hook into Maven repositories... or get ready to create a
> separate package for each version of each JAR.

While I don't know much about Java, JARs and Java packaging, this sounds
most like the most ubuntuic or debianic alternative. I realise that it
will take huge amount of work and a very active Java team that manages
the source packages that are out-of-date, etc. very well.

I'm sure there's a lot of interest in getting this solved, but as far as
I know we'd need more documentation or "agree on" example packages to
make this work easier. What do you think?


> Please let me know what you think of all this problem. I'm pretty sure
> the Java world will one day gets its senses back and have a more
> distribution-friendly dependency system, but we probably need a
> solution until such day happens... or forget about such Java software
> altogether.

The only way I see to go about this is very careful bitching about
breaking compatibility over and over again. We've done this successfully
in other parts of the Open Source community. :-)

Have a great day,
Daniel

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Thierry Carrez 01-26-2009 11:17 AM

Packaging large Java software stacks ?
 
Daniel Holbach wrote:
> Thierry Carrez schrieb:
>> Potential solutions to solve (1) include using a parallel delivery
>> mechanism for JARs, separated from the usual Ubuntu packages, something
>> that could hook into Maven repositories... or get ready to create a
>> separate package for each version of each JAR.
>
> While I don't know much about Java, JARs and Java packaging, this sounds
> most like the most ubuntuic or debianic alternative. I realise that it
> will take huge amount of work and a very active Java team that manages
> the source packages that are out-of-date, etc. very well.

Creating a separate package for each version of each JAR is indeed the
most debianic approach, however I'm not sure it's realistic. Glassfish
alone needs ~230 separate versions of dependencies to build (of which we
have ~30 in the tree currently). Add the dependencies to those
dependencies to the mix, and the 1000 mark is probably not very far. And
Glassfish has a rather clean dependency approach compared to Geronimo...

> I'm sure there's a lot of interest in getting this solved, but as far as
> I know we'd need more documentation or "agree on" example packages to
> make this work easier. What do you think?

I'm not sure there is so much interest. It's quite easy to install Java
software from upstream binary tarballs. Even with tomcat6 packaged, lots
of Java developers are still unpacking binary tarballs to install their
own Tomcat...

>> Please let me know what you think of all this problem. I'm pretty sure
>> the Java world will one day gets its senses back and have a more
>> distribution-friendly dependency system, but we probably need a
>> solution until such day happens... or forget about such Java software
>> altogether.
>
> The only way I see to go about this is very careful bitching about
> breaking compatibility over and over again. We've done this successfully
> in other parts of the Open Source community. :-)

Yes, we still have to advocate the right way. Note that from Java
developers point of view, their way of doing things (including precise
external libraries as part of the package) is the right way, and our way
(trying to run several things on top of the same pile of libraries) is
wrong.

--
Thierry Carrez
Ubuntu server team

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Daniel Holbach 01-26-2009 11:35 AM

Packaging large Java software stacks ?
 
I'll add this disclaimer once for all the thread now: I hope to not
sound too authoritative. If somebody knows a cleverer way, speak up. :-)

Thierry Carrez schrieb:
> Creating a separate package for each version of each JAR is indeed the
> most debianic approach, however I'm not sure it's realistic. Glassfish
> alone needs ~230 separate versions of dependencies to build (of which we
> have ~30 in the tree currently). Add the dependencies to those
> dependencies to the mix, and the 1000 mark is probably not very far. And
> Glassfish has a rather clean dependency approach compared to Geronimo...

I have a set of questions:
- How many of these do we have already?
- How many of these ever change?
- How many of these break compatibility?
- How hard is the dependency? Is this always "== JAR_VERSION" or often
a ">= JAR_VERSION" too?


> I'm not sure there is so much interest.

If it's "hard-core scientific rocket science work", it's going to be
tough, but if we can demonstrate how it's done and clearly articulate
what needs doing, I'm sure we can generate some interest here. There's a
lot of people on the world who use Java and who would be happy to have
it distributed by their distribution just like the rest of quality
software that we ship.


> It's quite easy to install Java
> software from upstream binary tarballs. Even with tomcat6 packaged, lots
> of Java developers are still unpacking binary tarballs to install their
> own Tomcat...

I'm not sure if a previous discussion of this ended up with differing
outcome, but I think there's no framework for us to allow this.


> Yes, we still have to advocate the right way. Note that from Java
> developers point of view, their way of doing things (including precise
> external libraries as part of the package) is the right way, and our way
> (trying to run several things on top of the same pile of libraries) is
> wrong.

Maybe there needs to be a "Java Open Source Distribution Interest Group"
or something. The other distributions will have the same problem like we do.

Open Source Distributions essentially value and care about:
- building from open source
- shipping built binaries and being authoritative about what is shipped
and installed on the users machines
- stable interfaces and a clear indication (versioning, tools, etc) of
changes
- <add more items here>

It might be a long winding road (and sound a bit demanding too), but if
Java wants to be a "first class citizen" in the Open Source world, this
needs to be clear.

Have a great day,
Daniel

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Thierry Carrez 01-26-2009 12:30 PM

Packaging large Java software stacks ?
 
Daniel Holbach wrote:
> Thierry Carrez schrieb:
>> Creating a separate package for each version of each JAR is indeed the
>> most debianic approach, however I'm not sure it's realistic. Glassfish
>> alone needs ~230 separate versions of dependencies to build (of which we
>> have ~30 in the tree currently). Add the dependencies to those
>> dependencies to the mix, and the 1000 mark is probably not very far. And
>> Glassfish has a rather clean dependency approach compared to Geronimo...
>
> I have a set of questions:
> - How many of these do we have already?

I would say around 500. 99% of them are providing a single version
(usually a recent one).

> - How many of these ever change?

Most of them still regularly change.

> - How many of these break compatibility?

That's a difficult one to answer, since there is no API versioning in
JARs, but apparently a lot of new releases do. The Geronimo developers
told me the slightest version change frequently breaks the TCK JavaEE
compliance tests, and using the latest is usually not the best. To get
an idea of the (runtime) components versioning complexity, see [1]. The
-Gnnnnnn versions means it's a Geronimo-patched version of the JAR that
is used.

[1] http://cwiki.apache.org/GMOxDOC22/component-versions.html

> - How hard is the dependency? Is this always "== JAR_VERSION" or often
> a ">= JAR_VERSION" too?

Dependencies are "== JAR_VERSION" for Maven-based projects. Ant-based
things are generally friendlier.

Note that the JARs needed at runtime are usually shipped together with
the software in the upstream binary release. They really consider JARs
as pre-written blocks of their own code rather than as external deps...
The Geronimo guys monitor security issues in all components and release
a new version of the stack every time there is a security issue with
some external thing.

>> Yes, we still have to advocate the right way. Note that from Java
>> developers point of view, their way of doing things (including precise
>> external libraries as part of the package) is the right way, and our way
>> (trying to run several things on top of the same pile of libraries) is
>> wrong.
>
> Maybe there needs to be a "Java Open Source Distribution Interest Group"
> or something. The other distributions will have the same problem like we do.
>
> Open Source Distributions essentially value and care about:
> - building from open source
> - shipping built binaries and being authoritative about what is shipped
> and installed on the users machines
> - stable interfaces and a clear indication (versioning, tools, etc) of
> changes
> - <add more items here>

The JPackage project has been trying to do that, provide a sufficiently
common platform to try to influence smaller upstream projects. The
bigger projects (like Geronimo) have much more brittle stacks: they
don't see the point of going through a very complex path while they
already have a hard job assembling their stack without caring about
anything else.

--
Thierry Carrez
Ubuntu server team

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Martin Pitt 01-26-2009 03:00 PM

Packaging large Java software stacks ?
 
Hello Thierry,

Thierry Carrez [2009-01-26 10:49 +0100]:
> It is difficult to integrate the large recent Java software stacks
> (Glassfish, Geronimo...) in a Linux distribution in general.

You impressively described why packaging those "properly", i. e. with
separate libraries and build dependencies, is not really sensible, if
at all possible. It doesn't sound like we could support those kind of
builds in our usual "backport patches" way of working anyway, if these
components get constantly changed.

However, do we actually need to? It doesn't sound like we would ever
put those into main, just making it available conveniently for Ubuntu
users?

> Potential solutions to solve (1) include using a parallel delivery
> mechanism for JARs, separated from the usual Ubuntu packages, something
> that could hook into Maven repositories... or get ready to create a
> separate package for each version of each JAR.

So is Maven a system to pull Java libraries/projects from external
well-known sources, much like Python eggs or Ruby gems? If those
repositories already exist and are maintained, then we should rather
think about integrating them better than trying to do the sysiphos
work of packaging them again properly.

> A solution to workaround both problems would be to avoid targeting our
> built-from-source repositories for such Java software and pack them with
> their binary dependencies (or as binary directly)...

If I remember correctly, e. g. our glassfish package is a giant bundle
of jars, i. e. includes everything you need to build and run the
package. At this level, packaging seems to be feasible, and working
(however ugly it might be wrt. "proper" packaging).

> Please let me know what you think of all this problem. I'm pretty sure
> the Java world will one day gets its senses back and have a more
> distribution-friendly dependency system

Stable APIs have worked very well in the C, perl, and Python world. I
can't imagine why that should intrinsically not be possible in the
Java world. But for projects where the changes are really as dramatic
as you point out, I agree that we currently just have to do the "giant
jar bundle" workaround, if we need to package it in the first place.

However, we do have a fair bunch of Java libraries in the
repositories, and even in main, and their API doesn't seem to change
that often (if at all). So is this really just a problem of the two
big projects you mentioned (glassfish/geronimo) or a wide-spread
phenomenon, and the packages we have in main are just the "good"
exceptions?

Martin
--
Martin Pitt | http://www.piware.de
Ubuntu Developer (www.ubuntu.com) | Debian Developer (www.debian.org)

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Emmet Hikory 01-27-2009 02:36 AM

Packaging large Java software stacks ?
 
Thierry Carrez wrote:
> It is difficult to integrate the large recent Java software stacks
> (Glassfish, Geronimo...) in a Linux distribution in general. The key
> reason is that most of those stacks require very precise versions of
> libraries (JARs) to run and to build. They won't work with the latest
> version of libraries as those might change APIs and/or key behavior.
> Java developers are used to pick specific JAR versions and assemble the
> exact needed stack, they don't want to care about sharing their
> dependencies with other packages, or about dependencies being upgraded.
> Tools like Maven help them in this endeavor, and they rely very heavily
> on this external code : dozens of those JARs are usually needed at
> runtime, hundreds of those at build-time.
>
> That makes most attempts to properly package large Java software in
> Ubuntu to fail for two reasons.
>
> (1) Need precisely-versioned artifacts
> Those stacks need, as build dependencies and as runtime dependencies,
> very precise versions of JARs. They won't build or run with a different
> one. Using a more recent one might break functionality in a creative
> way. A maven-based build will sometimes require 6 different versions of
> the same JAR. In our packages, we usually offer only one version,
> in corner cases one minor version for each major version. In some cases
> the Java software will run/build with ours, in most cases it won't.

In the case of libraries in other languages (most commonly C), we
regularly port applications to work with the preferred version of the
libraries we ship. When preparing new versions of C libraries, the API
and ABI are checked, with the binary package name changed where they
differ, to better ensure compatibility. Perhaps we could use the Java
Introspection methods to generate some API report, and version Java
libraries based on changing APIs? In the case where an incompatibility
is not an API change, what sort of differences are encountered? Might
these be considered bugs? Is there a case where two applications depend
on the same API, but would break if used with different versions
providing that API?

> (2) Building entirely from source
> Java "compilation" (in fact bytecoding) requires lots of JARs to be
> present, because they are used to check all external method signatures.
> Geronimo and Glassfish builds will require hundreds of them. Packaging
> all those build dependencies from source is a huge work, and each of
> those build dependencies, in turn, will require more. Combined with the
> first problem, this makes packaging those stacks too much work for so
> little to gain (they are usually easily installable by
> downloading/unpacking the upstream tarball).

Well, the same could be said for any interpreted language (python,
ruby, perl, etc.), and to a large degree for compiled languages (which
are often sufficiently portable that `./configure && make && make
install` works). One of the points of building from source is to be
able to verify that the provided source matches the provided binary, and
allow our users to make further modifications to the software in
accordance with the license. In a purely self-serving manner, shipping
the source from which the package was built significantly reduces the
effort required for distribution-local patches, most notably for
security patches.

> Potential solutions to solve (1) include using a parallel delivery
> mechanism for JARs, separated from the usual Ubuntu packages, something
> that could hook into Maven repositories... or get ready to create a
> separate package for each version of each JAR.

The last option is what is chosen for libraries in other languages,
where we ship two or three different upstream versions while waiting for
ports to occur. Where possible, we try to restrict this to less than
three, and wherever possible to only one.

> Potential solutions to solve (2) include evolving our build-from-source
> policy to accept that JARs that would only be used as method-checking
> media during Java compilation to be considered part of the source...
>
> A solution to workaround both problems would be to avoid targeting our
> built-from-source repositories for such Java software and pack them with
> their binary dependencies (or as binary directly)...

This has historically been acceptable for some packages in
multiverse, but without using the built-from-source method, it is hard
for us to comply with licenses that insist that source code be provided
on request as we cannot know that the source provided generates the
binary provided. This is further complicated in terms of defect
management: without being able to ourselves collect a source tree we
know to be capable of generating a given binary object, it becomes very
difficult for us to address any user-reported problems. This is
compounded if there are embedded libraries, as a patch to fix a bug in a
library may need to be repeated for several packages embedding that
library. Even if such patches are generated, but the use of our build
tools to generate the package has not been tested, we cannot be
confident of our ability to regenerate a suitable package to distribute
to users.

Note also that including binary objects in a source limits our
ability to patch them using our existing toolset. Variances from
upstream distributed software are included in the diff,gz, which is
restricted to unicode artifacts. The typical model to handle binary
patches is to ship the replacement blob uuencoded, and uudecode at
build-time for installation. When changing objects where the preferred
form of modification is indeed binary (e.g. PNG files), this is
acceptable. Where changing objects where the preferred form of
modification are separate source, we would need to find a way to
indicate from which source the replacement blob was generated (which is
more than just patching the provided source, especially in the case of
binary-only embedded libraries).

In summary, I'd recommend shipping such stacks that do not require
redistribution of source as binary objects in multiverse until such time
as they can be built reliably from source within the distribution, and
discussing the difficulties of extending the offered license to others
with upstream for those stacks licensed with source-distribution
licenses (e.g. GPL) if the entire application cannot be built without
binary objects within the distribution (as I'm not convinced these can
be shipped, although I'm not an archive-admin, so it's not my decision,
and I'm certainly not providing any legal advice).

My personal experience in working with upstream to get Java software
packaged is that upstream is quite willing to help identify what pieces
are missing in Ubuntu, help identify which patches may have been applied
to any embedded libraries, and test their software against provided
system libraries once available (even where versions differ). While
this still leaves significant work in getting the necessary components
packaged, I believe that these discussions lead to better sharing of
patches between upstreams, and overall a better quality of the Java
stack within Ubuntu.

--
Emmet HIKORY


--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Thierry Carrez 01-27-2009 07:22 AM

Packaging large Java software stacks ?
 
Martin Pitt wrote:

> You impressively described why packaging those "properly", i. e. with
> separate libraries and build dependencies, is not really sensible, if
> at all possible. It doesn't sound like we could support those kind of
> builds in our usual "backport patches" way of working anyway, if these
> components get constantly changed.
>
> However, do we actually need to? It doesn't sound like we would ever
> put those into main, just making it available conveniently for Ubuntu
> users?

The top stacks (Geronimo) might become targets for main, based on our
support strategy. I agree with you it would be simpler if they weren't.

>> Potential solutions to solve (1) include using a parallel delivery
>> mechanism for JARs, separated from the usual Ubuntu packages, something
>> that could hook into Maven repositories... or get ready to create a
>> separate package for each version of each JAR.
>
> So is Maven a system to pull Java libraries/projects from external
> well-known sources, much like Python eggs or Ruby gems? If those
> repositories already exist and are maintained, then we should rather
> think about integrating them better than trying to do the sysiphos
> work of packaging them again properly.

Maven is a project management tool that handles build-time dependencies,
rather than a more classic runtime library delivery mechanism. Remember
Java "binary releases" tarballs usually include all the needed external
JARs directly, so there is no need for a runtime dependency delivery
system. However Maven makes use of artifact repositories from well-known
sources so we could try to leverage those.

>> A solution to workaround both problems would be to avoid targeting our
>> built-from-source repositories for such Java software and pack them with
>> their binary dependencies (or as binary directly)...
>
> If I remember correctly, e. g. our glassfish package is a giant bundle
> of jars, i. e. includes everything you need to build and run the
> package. At this level, packaging seems to be feasible, and working
> (however ugly it might be wrt. "proper" packaging).

Yes, that's how glassfishv2 was packaged (in multiverse). For
glassfishv3, an attempt to make a "proper" packaging is under way but
hits the two issues I described in this post.

>> Please let me know what you think of all this problem. I'm pretty sure
>> the Java world will one day gets its senses back and have a more
>> distribution-friendly dependency system
>
> Stable APIs have worked very well in the C, perl, and Python world. I
> can't imagine why that should intrinsically not be possible in the
> Java world.

There is no reason why it would be impossible in the Java world. It's
just that the Java platform doesn't help (yet) and the Java developers
have decided not to care about "common library dependencies" but to just
consider them as "pre-written bricks included in my code".

> But for projects where the changes are really as dramatic
> as you point out, I agree that we currently just have to do the "giant
> jar bundle" workaround, if we need to package it in the first place.
> However, we do have a fair bunch of Java libraries in the
> repositories, and even in main, and their API doesn't seem to change
> that often (if at all). So is this really just a problem of the two
> big projects you mentioned (glassfish/geronimo) or a wide-spread
> phenomenon, and the packages we have in main are just the "good"
> exceptions?

I think the Geronimo case is extreme, given the number of dependencies
and the brittle nature of JavaEE TCK compliance. But for all projects
built with Maven, you will need the precise artifact versions to build,
or force it into using a slightly different version, without any easy
way of telling if they are at least API-compatible. This may just fail
to compile, or fail in strange ways at runtime... This is definitely
something you can work on and test for small Maven-based projects, but
for medium-sized ones it will be much more difficult, and you can't
count on upstream support (they decided to use Maven to avoid caring
about those issues).

--
Thierry Carrez
Ubuntu server team

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Thierry Carrez 01-27-2009 08:17 AM

Packaging large Java software stacks ?
 
Emmet Hikory wrote:
> Thierry Carrez wrote:
>> (1) Need precisely-versioned artifacts
>> Those stacks need, as build dependencies and as runtime dependencies,
>> very precise versions of JARs. They won't build or run with a different
>> one. Using a more recent one might break functionality in a creative
>> way. A maven-based build will sometimes require 6 different versions of
>> the same JAR. In our packages, we usually offer only one version,
>> in corner cases one minor version for each major version. In some cases
>> the Java software will run/build with ours, in most cases it won't.
>
> In the case of libraries in other languages (most commonly C), we
> regularly port applications to work with the preferred version of the
> libraries we ship. When preparing new versions of C libraries, the API
> and ABI are checked, with the binary package name changed where they
> differ, to better ensure compatibility. Perhaps we could use the Java
> Introspection methods to generate some API report, and version Java
> libraries based on changing APIs?

Yes, that would be an option. I know Sun wants to go in a more
stable-API-oriented direction in the future, having acknowledged that
the current JAR version numbers don't really mean anything useful.

> In the case where an incompatibility
> is not an API change, what sort of differences are encountered? Might
> these be considered bugs? Is there a case where two applications depend
> on the same API, but would break if used with different versions
> providing that API?

When some project requires a precise version of a JAR, you don't really
know if that's :
- because it relies on a specific API
- because there is a bugfix they rely on
- because newer features introduce a regression
- because Maven uses =JAR_VERSION types of dependencies by default

There might exist some Java library projects with strict versioning
rules, like x.y.z where x.y defines API and .z defines bugfixes inside a
stable API. But the general case is that they are allowed to (and will)
break API and behavior for every version (no matter how minor), since
(1) Maven helps developers in depending on a specific JAR version and
(2) runtime JARs are shipped within the binary release tarball.

>> A solution to workaround both problems would be to avoid targeting our
>> built-from-source repositories for such Java software and pack them with
>> their binary dependencies (or as binary directly)...
>
> This has historically been acceptable for some packages in
> multiverse, but without using the built-from-source method, it is hard
> for us to comply with licenses that insist that source code be provided
> on request as we cannot know that the source provided generates the
> binary provided. This is further complicated in terms of defect
> management: without being able to ourselves collect a source tree we
> know to be capable of generating a given binary object, it becomes very
> difficult for us to address any user-reported problems. This is
> compounded if there are embedded libraries, as a patch to fix a bug in a
> library may need to be repeated for several packages embedding that
> library. Even if such patches are generated, but the use of our build
> tools to generate the package has not been tested, we cannot be
> confident of our ability to regenerate a suitable package to distribute
> to users.

There is the middle alternative, where we would still build from source
everything we *distribute*. There are a lot more Java build dependencies
than runtime dependencies: most of the Java build dependencies are just
used to check external method signatures during the bytecode
compilation, much like an API description. So we could still produce the
needed runtime JARs from their source by shipping those JAR build
dependencies as part of the source package. That way we solve the source
code providing hole, and we can have a clean defect management system,
for a much smaller packaging cost.

Taking numbers from a project I'm working on :
- binary distribution: 1 package
- build from source all runtime dependencies: 15 packages
- build from source all runtime/build deps: ~200 packages (just counting
one level deep. Deeper analysis under way, conservative estimates are
~500 packages needed)

So you can see that we are definitely still in the "doable" area with
the middle solution. If someone knew a way of turning off external
method checking during bytecode compilation, or a way of providing a
text-based API description to compile against... that would avoid the
"binary" thing altogether.

--
Thierry Carrez
Ubuntu server team

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel

Dustin Kirkland 01-27-2009 01:07 PM

Packaging large Java software stacks ?
 
Thierry-

Just curious, but have you (or anyone else) checked how JBoss is
packaged and built by and for Red Hat? Not that I'm suggesting JBoss
as a replacement for Glassfish/Geronimo/etc, but perhaps they have
made some break-throughs in the realm of packaging that we might
emulate?

:-Dustin

--
ubuntu-devel mailing list
ubuntu-devel@lists.ubuntu.com
Modify settings or unsubscribe at: https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel


All times are GMT. The time now is 12:57 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.