Proposal for additional metadata in Debian archives (DEP-11)
 

Hello!
With more and more non-technical users using Linux and Debian, the
package database became much more application-centric instead of
package-centric, as it was before. This resulted e.g. in the Ubuntu
Software Center and GNOME-AppInstall as simple ways for endusers to
install software.
In order to provide these "Software-centers", additional metadata was
required, which is currently stored in the package "app-install-data",
which is a very inflexible way of storing this data. The data also has
to be rebuild for every distribution release, which is a problem e.g.
for the unstable or testing "release" of Debian, which is used by many
people.

In January, people from many distributions met in the "AppStream
Meeting"[1] to define how an application-centric Software Center
should look like and how distributions could collaborate and share
technology in this area. Sharing stuff would reduce work on all sides
and have some nice side-effects, e.g. better collaboration on patches.
It also improves upstream relations. For more information on
AppStream, you can check the wiki page at fd.o. ([1])

Also, Debian lacks some package-neutral metadata other applications
could use to request a specific "extension" to be installed. E.g. the
Anjuta IDE uses this metadata available on Fedora to install some
plugins. With additional metadata in the Debian archive, applications
using the distribution-neutral PackageKit API could e.g. request a
specific firmware, mimetype, USB-Device handler etc. to be installed.

To solve this issues and to provide a complete and smooth experience
for non-technical users, we created DEP-11, which describes how the
Debian archive could be extended to support these additional features.
Please do always keep in mind that this data will be optional and will
be downloaded on demand, so it will be present on desktop machines,
but not on servers, where it doesn't make much sense.
AppStream features XML to store metadata. Because we don't use XML
somewhere in Debian, DEP-11 features a well-known RFC822-style format.
The metadata to provide "applicataion-data", which is information
about which package contains which application and "component-data",
which is information about which component (shlib, python-module,
firmware, mimetype, driver, ...) a package provides can be created
automatically in one pass. It also covers the same use-case, therefore
it makes sense to store it in one file.
Implementing this should be trivial. Some people are already working
on Python scripts to extract the required data very, very fast and
Michael would implement some additional methods into apt to support
the new files. It is also possible that package-maintainers could
define custom "Component-Provides" for their packages - this would
make sense in some cases, if the provides entry is the same on all
distributions.
Tools like Apper for KDE already support installing Plasma-Dataengines
from the archive, if metadata is present.
So, there are many people who would like this feature, also many
upstream developers. (I talked to them at Desktop-Summit this year)
Implementing and using this new feature is not a problem :)
So, do you have any comments/complains/improvements for DEP-11? Would
you generally agree that adding this data is useful?
You can find the full text of DEP-11 at [2].
I CCed debian-devel, so more people can comment on this.
Thanks for the attention!

Kind regards
Matthias Klumpp

-------
[1]: http://distributions.freedesktop.org/wiki/Meetings/AppInstaller2011
[2]: http://wiki.debian.org/AppStreamDebianProposal


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAKNHny_biK3h72yC9D0=3qTuzV1yWXfrsv+a7yULBgE_d4HuY w@mail.gmail.com">http://lists.debian.org/CAKNHny_biK3h72yC9D0=3qTuzV1yWXfrsv+a7yULBgE_d4HuY w@mail.gmail.com

Proposal for additional metadata in Debian archives (DEP-11)
 

2011/10/31 Charles Plessy <plessy@debian.org>:
> Le Thu, Oct 27, 2011 at 05:49:12PM +0200, Matthias Klumpp a écrit :
>>
>> For us, it is necessary that APT can process this data (will be
>> implemented if DEP-11 can make it) and that parts of it can be written
>> into a Xapian-DB for fast searching. - Both would work perfectly well
>> with any format.
>>
>> It would be very nice, if ftpmasters could tell if they would accept a
>> new format in the archive or if we should stay with Â*RFC822 which is
>> used for nearly everything else already.
>
> Dear Matthias,
Hi! :)

> I am still not sure to understand how the data will be used. Â*Is it only to be
> used via Internet ?
No, it is used by applications running on Debian. The application
which will use this data the most is the Software-Center and related
apps for sure, but with the components-metadata, every application
needing to find extra-components (like plugins) in the archive using a
distro-agnostic way will use this data.
The data will be accessible via APT and PackageKit. (and via aptd too, I guess)

> In that case perhaps it is not needed to distribute it via
> the Debian archive. Â*What is the Debian-specific data ? Â*If it is the
> association between a FreeDesktop menu “.desktop� file and a package name,
> there is already a file in the Debian archive that provides this.
Not only. It also exposes some content-elements of the .desktop-file
to a new file which is easily-searchable. And it will provide extra
informations about which package contains which python module, shared
lib etc. This is basically a metadata <-> deb-package matching. It is
good to have this in a extra file, because the Contents.tar.gz does
not provide all of this information and because Contents.tar.gz is a
very, very huge file. We don't want to download, process and cache
this file everytime the archive updates. The Components.gz file would
be much smaller. So, two important reasons for the new approach :)

> Then, a
> repository of the contents of FreeDesktop menu entries would definitely be
> valuable, especially if served semantically, but as it would not contain data
> specific to Debian, wouldn't it be better to develop it with less ties to the
> Debian archive ? Â*That would be a great contribution from Debian to the the
> rest of the Free software ecosystem.
Others can fetch the data if they want, but the data is specific to
Debian's current archive state... There are some other things you
might be able to do with it, like matching it with Fedora's data and
produce messages like "Fedora has packaged library X version Y which
is not yet present in the archive." or sharing patches more easily
because finding the "related" RPM package to a DEB package in other
distros would be easier and faster.
But that's of course not the main goal of DEP-11, although it's a nice
possibility, IMHO.

Have a nice day too!
Cheers,
Matthias


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAKNHny_wdVjgfvdsozi+BYq-8wtKHyDvx_fVx2N9aHNQPg0-8Q@mail.gmail.com">http://lists.debian.org/CAKNHny_wdVjgfvdsozi+BYq-8wtKHyDvx_fVx2N9aHNQPg0-8Q@mail.gmail.com