FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 10-23-2011, 03:13 PM
Raphael Hertzog
 
Default Dealing with embedded javascript libraries

Hello,

I would like to discuss our handling of embedded javascript libraries.
In theory, like any other embedded library, they are to be avoided and
we have a lintian warning catching many of the common cases:
http://lintian.debian.org/tags/embedded-javascript-library.html

Unfortunately, blindly replacing the file with a symlink can
create problems of its own. Upstream tested their application
with the version of the library that they ship. Using another
version might break things horribly, an example here:
https://bugs.launchpad.net/ubuntu/+source/wordpress/+bug/816962

Even if the Debian maintainer takes care to ensure the correct
version is used (which is difficult to do, you never know what the next
upstream version of the javascript will do), it's quite possible that he
will have to switch regularly between the embedded version and the
packaged one just because of differing schedules between the two projects.
If the package uses symlinks to directories, those switchs
quickly become painful (and forgetting about them even more, see
for example #639733).

And with javascript libraries, there's no failure at build time,
you only discover much later when something is not working...
(BTW, a large part of what I'm saying also applies to PHP libraries but
the resulting runtimes failures are usually easier to catch and diagnose)

So I'm wondering if we're not giving ourselves bad advice by recommending
to replace those files with symlinks. Most of the time the Debian
maintainer is not knowledgeable enough (or doesn't have the time required
to make the proper assessment) to safely decide to diverge from upstream.
It's good to reduce the workload of the security team but only if we
don't break stuff while doing it.

My current suggestion is to use what I would call opportunistic
replacement of those embedded copies, if the packaged library matches the
embedded one, then provide a symlink instead of a copy and ensure the
dependencies hardcode the current upstream version of the packaged
library (via ${miscepends} most likely).

And instead of creating symlinks to directories, we duplicate the
directory hierarchy and we only symlink files so that it's easy to
switch back and forth between the embedded copy and the packaged one.

I have some early prototype code in the wordpress package, see
debian/dh_linktree. It can replace identical files with symlinks
if you pass --replace-only --same-only. It doesn't implement the
dependency generation though. In any case it needs further thoughts
and cleanups.
http://anonscm.debian.org/gitweb/?p=collab-maint/wordpress.git;a=blob;f=debian/dh_linktree;hb=master

What are your thoughts on this topic?

One downside of this approach is that a package like wordpress
would have to be rebuilt when a new upstream version of one of its
javascript libraries is released. (And we don't have bin-nmu for
arch: all currently).

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/go/ulule-rh/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111023151317.GA26161@rivendell.home.ouaza.com">h ttp://lists.debian.org/20111023151317.GA26161@rivendell.home.ouaza.com
 
Old 10-23-2011, 03:29 PM
Roland Mas
 
Default Dealing with embedded javascript libraries

Raphael Hertzog, 2011-10-23 17:13:17 +0200 :

[...]

> Unfortunately, blindly replacing the file with a symlink can
> create problems of its own. Upstream tested their application
> with the version of the library that they ship. Using another
> version might break things horribly, an example here:
> https://bugs.launchpad.net/ubuntu/+source/wordpress/+bug/816962

I don't do much library packaging myself, but it was my understanding
that versions of libraries that break API/ABI are meant to go in
different binary packages, usually with a version number in the package
name. Javascript doesn't have an ABI, but libraries written in that
language are still libraries, aren't they?

Roland.
--
Roland Mas

That's one of the good fings about not existin'; they leave you alone most
of the time. -- in My Hero (Tom Holt)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87y5wbvi9s.fsf@mirexpress.internal.placard.fr.eu.o rg">http://lists.debian.org/87y5wbvi9s.fsf@mirexpress.internal.placard.fr.eu.o rg
 
Old 10-23-2011, 03:29 PM
Paul Wise
 
Default Dealing with embedded javascript libraries

On Sun, Oct 23, 2011 at 11:13 PM, Raphael Hertzog wrote:

> And with javascript libraries, there's no failure at build time,
> you only discover much later when something is not working...

This is the crux of the issue and it is not specific to JavaScript
libraries, anything that is interpreted has this issue. I'm pretty
sure I've seen API breakage in libraries implemented in Python for
example.

More automated and manual testing can help here I guess.

Also hopefully maintainers are using the packages they maintain and
will therefore notice when they are broken by newer JavaScript
libraries.

> What are your thoughts on this topic?

One of the other problems with embedded JavaScript libraries is that
often only the pre-compiled/obfuscated/minified version is
distributed, which would be a violation of DFSG item 2.

--
bye,
pabs

http://wiki.debian.org/PaulWise


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAKTje6FGAQUVeSjpVHwUOfgtuSo1Ta5GzpUaCFoxK=THvQ9wt w@mail.gmail.com">http://lists.debian.org/CAKTje6FGAQUVeSjpVHwUOfgtuSo1Ta5GzpUaCFoxK=THvQ9wt w@mail.gmail.com
 
Old 10-23-2011, 04:51 PM
Yaroslav Halchenko
 
Default Dealing with embedded javascript libraries

On Sun, 23 Oct 2011, Paul Wise wrote:
> > And with javascript libraries, there's no failure at build time,
> > you only discover much later when something is not working...
> This is the crux of the issue and it is not specific to JavaScript
> libraries, anything that is interpreted has this issue. I'm pretty
> sure I've seen API breakage in libraries implemented in Python for
> example.
> More automated and manual testing can help here I guess.

yeap -- in python world the situation is generally better because
unittesting is easy and popular, so for nearly all python packages I
maintain I introduce package build-time unittesting against all
supported versions of Python. That helps A LOT to assure correct
performance especially having heavy backporting in mind (we usually
build nearly for all recent releases of debian and ubuntu within
neuro.debian.net project).

Sure thing not all projects have 100% unittest coverage, also
problems some times go unnoticed if it is arch 'all' package while has
some arch specific operations (I/O, byteswapping) which do not get
picked up unless they get excercised by unittests of dependent modules
with packages of arch 'any'... but altogether there testing is
indispensable to verify compatibility of API and just general QA.

So, if only we could provide relevant automated basic testing to those
JS-dependent packages...

--
=------------------------------------------------------------------=
Keep in touch www.onerussian.com
Yaroslav Halchenko www.ohloh.net/accounts/yarikoptic


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111023165103.GC10325@onerussian.com">http://lists.debian.org/20111023165103.GC10325@onerussian.com
 
Old 10-23-2011, 05:39 PM
Pau Garcia i Quiles
 
Default Dealing with embedded javascript libraries

On Sun, Oct 23, 2011 at 5:29 PM, Paul Wise <pabs@debian.org> wrote:


On Sun, Oct 23, 2011 at 11:13 PM, Raphael Hertzog wrote:



> And with _javascript_ libraries, there's no failure at build time,

> you only discover much later when something is not working...



This is the crux of the issue and it is not specific to _javascript_

libraries, anything that is interpreted has this issue. I'm pretty

sure I've seen API breakage in libraries implemented in Python for

example.



I agree.

So far it seems I have been pretty lucky with my package witty, which depends on jquery but upstream is not really happy Debian and Ubuntu replace the jquery version they have tested with.




> What are your thoughts on this topic?




One of the other problems with embedded _javascript_ libraries is that

often only the pre-compiled/obfuscated/minified version is

distributed, which would be a violation of DFSG item 2.



IMHO that should not be a problem provided that:
- The _javascript_ library is open source with the proper license, etc
- Upstream is using the _javascript_ library "as is" (i. e. with no local modifications)



Maybe README.Debian should mention "this package embeds the _javascript_ library XXX which is available independently in package libjs-XXX (source package: libjs-XXX) :-?

--
Pau Garcia i Quiles


http://www.elpauer.org
(Due to my workload, I may need 10 days to answer)
 
Old 10-23-2011, 11:20 PM
Ben Finney
 
Default Dealing with embedded javascript libraries

Roland Mas <lolando@debian.org> writes:

> I don't do much library packaging myself, but it was my understanding
> that versions of libraries that break API/ABI are meant to go in
> different binary packages, usually with a version number in the package
> name. Javascript doesn't have an ABI, but libraries written in that
> language are still libraries, aren't they?

Yes, I think so. The problem arises IMO because the conventions are
different.

Generally, programmers using Javascript libraries expect and encourage
the direct inclusion of the library in the application, and there are
very few cultural norms to discourage this code duplication.

I would very much like that to change – that programmers should expect a
single instance of a Javascript library to be useable across the OS, and
that a Javascript library without a dependable ABI should be shunned by
most application writers, and for applications to declare the library
versions they expect in a standardised, automated way. But I don't know
what to do to get there.

--
“A ‘No’ uttered from deepest conviction is better and greater |
` than a ‘Yes’ merely uttered to please, or what is worse, to |
_o__) avoid trouble.” —Mohandas K. Gandhi |
Ben Finney


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87bot7l2hb.fsf@benfinney.id.au">http://lists.debian.org/87bot7l2hb.fsf@benfinney.id.au
 
Old 10-26-2011, 02:31 PM
Raphael Hertzog
 
Default Dealing with embedded javascript libraries

On Sun, 23 Oct 2011, Paul Wise wrote:
> More automated and manual testing can help here I guess.

Sure, but I don't expect Debian maintainers to write a test suite
when upstream hasn't created one. And testing an interactive web
application is a rather difficult problem.

> Also hopefully maintainers are using the packages they maintain and
> will therefore notice when they are broken by newer JavaScript
> libraries.

I do but I'm not using all the features all the time and I don't test
them for each upload.

For instance I just noticed that we can't install new widgets with the
current wordpress package due to some javascript related problem. I'm not
familiar enough with the codebase to investigate it easily. I can't
ask upstream about it because it works with the version of the libraries
that they are shipping.

While I would also love that all upstreams would be more aware of the
fact that they are maintaining libraries and that they must take
compatibility seriously, I can't really change that. Furthermore
many of them are not using a Linux system and have a (vastly) different
culture.

Giving all this I plan to put into practice the opportunistic replacement
idea that I suggested and to add lintian overrides for all the javascript files
where this opportunistic replacement is in place.

Does this sound reasonable enough ?

(I see quite a few overrides already for such files)

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/go/ulule-rh/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111026143143.GA26650@rivendell.home.ouaza.com">h ttp://lists.debian.org/20111026143143.GA26650@rivendell.home.ouaza.com
 
Old 10-26-2011, 04:11 PM
Jakub Wilk
 
Default Dealing with embedded javascript libraries

* Raphael Hertzog <hertzog@debian.org>, 2011-10-26, 16:31:
Also hopefully maintainers are using the packages they maintain and
will therefore notice when they are broken by newer JavaScript
libraries.


I do but I'm not using all the features all the time and I don't test
them for each upload.


For instance I just noticed that we can't install new widgets with the
current wordpress package due to some javascript related problem. I'm
not familiar enough with the codebase to investigate it easily. I can't
ask upstream about it because it works with the version of the
libraries that they are shipping.


Why you can't? Do they bite if you do?

--
Jakub Wilk


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111026161114.GA4398@jwilk.net">http://lists.debian.org/20111026161114.GA4398@jwilk.net
 
Old 10-26-2011, 04:47 PM
Raphael Hertzog
 
Default Dealing with embedded javascript libraries

Hi,

On Wed, 26 Oct 2011, Jakub Wilk wrote:
> * Raphael Hertzog <hertzog@debian.org>, 2011-10-26, 16:31:
> >For instance I just noticed that we can't install new widgets with
> >the current wordpress package due to some javascript related
> >problem. I'm not familiar enough with the codebase to investigate
> >it easily. I can't ask upstream about it because it works with the
> >version of the libraries that they are shipping.
>
> Why you can't? Do they bite if you do?

For the very same reason we don't like Ubuntu bugs that have not been
reproduced on Debian.

Obviously I can ask but it's unlikely to lead to any fruitful discussion
if I don't come up with a patch that supports both versions at the same
time.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/go/ulule-rh/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20111026164702.GC28315@rivendell.home.ouaza.com">h ttp://lists.debian.org/20111026164702.GC28315@rivendell.home.ouaza.com
 
Old 10-26-2011, 10:20 PM
Zygmunt Krynicki
 
Default Dealing with embedded javascript libraries

W dniu 24.10.2011 01:20, Ben Finney pisze:

I would very much like that to change – that programmers should expect a
single instance of a Javascript library to be useable across the OS, and
that a Javascript library without a dependable ABI should be shunned by
most application writers, and for applications to declare the library
versions they expect in a standardised, automated way. But I don't know
what to do to get there.


Hi.

I'd like to throw a few comments here. I speak both as an web
application developer, debian packager (well, packaging my application
for debian-based systems) and deployment/ops engineer.


Asking for this is *highly* unrealistic and will only further alienate
developers from Debian's way of doing things. Current best practice is
to use CDNs that distribute a particular or latest version of a library
(jQuery for example has several such networks available) or to bundle
the library directly.


The reason for this is simple: quality control and portability (across
hosting systems)


Another thing to point out is that unlike traditional applications where
linking is explicit and done before actual startup here the linking
happens at another machine, the one of the user of the web application.
The actual library can come from a yet another system. The key issue to
point out here is that this is not really linking the way we think about
it. Here it is not code but data.


If anything, having one version of a javascript library *hurts*
Debian-as-a-platform. I would encourage a different approach altogether:
explicit mutli-versioning (ideally for all upstream releases or for all
upstream releases that are known to introduce features and are not
emergency bug/security fixes) and explicit version dependency that
matches upstream. That later requirement may be ignored if uptream is
not responding or the application is known to work correctly with
another version of a library.


This approach will allow web application developers to use Debian in the
way they expect systems to work.


At the end of the day I cannot use Debian's jQuery because I must
control what goes out to ensure quality. Multiply this problem by
co-hosting many applications and you get the answer.


As long what I described above is the prevailing point of view or
motivation of web application developers then packaged javascript
libraries will only fill a niche where someone deploys a packaged
wiki/bug tracker/something else for their small team/co workers.


More complicated web applications will be deployed outside the packaging
system, without any reuse of packaged libraries, without any advantages
that Debian brings to the table.


Thanks
Zygmunt Krynicki


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4EA887A6.4060708@ubuntu.com">http://lists.debian.org/4EA887A6.4060708@ubuntu.com
 

Thread Tools




All times are GMT. The time now is 07:52 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org