FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Development

 
 
LinkBack Thread Tools
 
Old 06-04-2011, 08:56 PM
Zygmunt Krynicki
 
Default dh_splitpackage 0.2.2

W dniu 04.06.2011 21:59, Benjamin Drung pisze:

Am Samstag, den 04.06.2011, 20:46 +0200 schrieb Zygmunt Krynicki:

The new script can be called instead of dh_install (assuming all the
files you are interested in are already in debian/tmp/) or afterwards.


I looked at the man page of dh_splitpackage and compared it to
"dh_install --fail-missing". I found three differences. Please let me
know if there were more.


There are two more differences:

1) dh_install is much more generic, dh_splitpackage cannot install files
from source directory. Only from debian/tmp. This is by design to keep
the same experience for developers and to focus the tool on only one thing.


2) dh_install reads multiple configuration files and is also used
directly in custom deban/rules overrides. dh_splitpackage is usable in
only one way, via a configuration file. The intent for this is the same
as above, to simplify the developer experience. It also limits the
number of configuration files that you need to create to one per source
package.



1. dh_splitpackage complains if a file ends up in more than one binary
package. Having the same file in two binary package leads to a conflict
if you want to install both binary packages. IMO dh_install should do
this check too.

2. dh_splitpackage put files in the primary package if no other install
destination is specified. I would love to see a command line option for
dh_install that does the same. For example, I want to call "dh_install
--put-everything-else-in=vlc-nox" for the daily builds of vlc.


It only does this when explicitly requested by setting primary_package
in the configuration file. By default it just silently ignores those
files (silently as in not breaking the build, the build log indicates
this when printing pathname classification).


I can see it would be sensible to warn the packager that primary_package
was not specified and some pathnames were not classified. I'll add this
to next release.



3. dh_splitpackage supports more pattern than dh_install. To name them:
** for matching all directories, [] to match as specific set of
characters. I could live without that, but ** would be nice sometimes.

I think that dh_install could be improved to support all three points
above.


I would like to see that as well. I asked debhelper maintainer to have a
look at my helper and see what we can do.



The features of dh_splitpackage are great, but the configuration format
(JSON) seems to be too complex for this use case.


JSON was my initial attempt as I love it very much and is the best
general-purpose tree-like format that is still very easy to work with.
After looking at the examples I wrote I still like it but I see the
problem it could pose to packagers.


I could augment/replace the JSON format with a simple ini-style format.
Fortunately the configuration is very simple and could be mapped almost
directly.


$ cat debian/splitpackage
# Global configuration goes first.
# Here you can set the primary package
primary_package=libfoo

# Hypothetical new option that warns the packager
# when files are left unclassified. Possible options:
# ignore, fail.
leftover_files=ignore

# Classifiers for libfoo-dev. Pick up all the header files,
# libtool files, and static libraries
[libfoo-dev]
include=**/*.h
include=**/*.la
include=**/*.a
# Just for demonstration, never pick any shared objects
exclude=**/*.so

[libfoo]
# Everything is automatically
# assigned by default thanks to
# primary_package in the global
# section.

Best regards
Zygmunt Krynicki


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4DEA9BF8.6070702@canonical.com">http://lists.debian.org/4DEA9BF8.6070702@canonical.com
 
Old 06-05-2011, 06:03 AM
Vincent Bernat
 
Default dh_splitpackage 0.2.2

On Sat, 04 Jun 2011 22:56:24 +0200, Zygmunt Krynicki wrote:


[libfoo-dev]
include=**/*.h
include=**/*.la
include=**/*.a
# Just for demonstration, never pick any shared objects
exclude=**/*.so


I know this is just an exemple to show exclude but .so files have to be
put in -dev packages. Exclude should be **/*.so.*. Moreover, there is an
ongoing effort to drop .la files. Here is an updated example (with a
real use of exclude)


[libfoo-dev]
include=**/*.h
include=**/*.a
include=**/*.so
exclude=**/libnotfoo.*



--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 965a6126bed8bf55197a3bbb79387da2@luffy.cx">http://lists.debian.org/965a6126bed8bf55197a3bbb79387da2@luffy.cx
 
Old 06-05-2011, 04:22 PM
Olivier Berger
 
Default dh_splitpackage 0.2.2

Hi.

Le samedi 04 juin 2011 20:46 +0200, Zygmunt Krynicki a crit :

>
> I wrote dh_splitpackage, a helper script that unambiguously splits the
> files of a binary package into multiple packages based on a
> configuration file.
>

Uh... Maybe I missed the point of this tool which was obvious to many
others, but could you provide a typical use case for using it ?

As is, I didn't find the description you provided potentially applying
to the kind of tasks I'd be doing on my packages.

Thanks in advance.

Best regards,
--
Olivier BERGER (Debian developer)
(OpenPGP: 4096R/7C5BB6A5)
http://www.olivierberger.com/weblog/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1307290930.4252.2.camel@inf-8657.int-evry.fr">http://lists.debian.org/1307290930.4252.2.camel@inf-8657.int-evry.fr
 
Old 06-05-2011, 08:32 PM
David Bremner
 
Default dh_splitpackage 0.2.2

On Sun, 05 Jun 2011 18:22:10 +0200, Olivier Berger <obergix@debian.org> wrote:

> Uh... Maybe I missed the point of this tool which was obvious to many
> others, but could you provide a typical use case for using it ?

I assume it is a matter of not wanting to manually maintain $pkg.install
files.

Here is an except from d/rules for darktable.

PLUGIN_EXCLUDES=$(patsubst %,-X%,
$(notdir $(shell cat $(wildcard debian/darktable-plugins-*.install))))

override_dh_install:
dh_install -pdarktable $(PLUGIN_EXCLUDES)
dh_install --remaining-packages

The top level directory /usr/lib/darktable is listed in
darktable.install, but any files under that installed in other packages
will be excluded.

As usual with such "cleverness", one can question the tradeoff of
fragility versus ease of maintenance.

David


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87vcwk2f4k.fsf@zancas.localnet">http://lists.debian.org/87vcwk2f4k.fsf@zancas.localnet
 
Old 06-08-2011, 10:25 AM
Zygmunt Krynicki
 
Default dh_splitpackage 0.2.2

W dniu 05.06.2011 18:22, Olivier Berger pisze:

Hello Olivier, sorry for the late reply.

Uh... Maybe I missed the point of this tool which was obvious to many
others, but could you provide a typical use case for using it ?


I see two general use cases:

1) "Hard" packages for which dh_install requires you to use override
rules and manually call dh_install -X xxx


While technically it's possible to do it without causing conflicts (same
file in more than one package) it's arguably trivial to do it right with
dh_splitpackage and not so easy with dh_install. You also don't need to
maintain shell code to do so (declarative language is somewhat less
error prone).


2) "Easy" packages that can be handled with dh_install.

The advantage here is that you can use one file (not one per binary
package) and the intent is clearer.


In the future I can envision templates that say how to split a C-like
library, a program from documentation and other common things. Those
could be maintained as templates of good-practice/standard behaviour and
simply symlinked/inherited/whatever into the packaging tree.



As is, I didn't find the description you provided potentially applying
to the kind of tasks I'd be doing on my packages.


Please check out dh_splitpackage(1) for more details. For the sake of
conversation I'll reproduce it here (typos and small errors are fixed in
trunk):


DH_SPLITPACKAGE(1) User Commands
DH_SPLITPACKAGE(1)




NAME
dh_splitpackage - split monolithic installation
directory into

sub-packages

SYNOPSIS
dh_splitpackage [-h] [-v] [-c config] [--sourcedir dir] [-n] [-q]

DESCRIPTION
dh_splitpackage works by interpreting the file
debian/splitpackage ,
which must be a JSON file, describing the split operation to
perform.

The actual operation is performed in stages:

First the debian/splitpackage file is read to check what
packages
should be considered and what patterns should be associated
with each

package. The file is also checked for syntax errors and structure.

Then each file found under debian/tmp/ is classified to
determine which
package it belongs to. If a file matches patterns of more
than one
package an appropriate error is reported, with details of the
problem.
If a file does not match any pattern it is automatically
associated to

the primary package, if designated.

Finally the split operation is performed. dh_splitpackage
walks over
the list of files and directories and, for each file or
directory,
copies it (with the full prefix), to the appropriate package.
Directo‐
ries are not copied recursively, if you want to copy the
contents sim‐
ply specify appropriate pattern that would match all the
descendants of

that directory.

FILES
debian/splitpackage
The primary configuration file for dh_splitpacakge.

In the configuration defines any number of packages and
inclusion and
exclusion rules for each package. In addition one package can be
desig‐
nated as primary to make all unclassified files belong there
automati‐
cally. The inclusion/exclusion patterns may be defined as a
list of
strings or just a single string. Each string is using a
special glob-
like matching pattern documented below. It's best to see some
examples

to understand how it works.

Note that JSON is very picky about missing or excessive commas,
if pro‐
cessing your configuration file throws errors then it's most
likely

caused by this annoying property.

See CONFIG FILE SCHEMA and EXAMPLE CONFIGURATION FILES below
for more

information.

OPTIONS
-h, --help
Show help message and exit

-v, --version
Show program's version number and exit

-c config, --conf config
Use alternate configuration file.

--sourcedir dir
Use alternate source directory.

-n, --dry-run
Don't actually copy any files or directories.

-q, --quiet
Don't print the classification table.

CONFIG FILE SCHEMA
The configuration file format it described by the
following JSON
Schema. The schema is following the 2nd draft of the JSON Schema
speci‐
fication. You can find the
specification here:

http://tools.ietf.org/html/draft-zyp-json-schema-02

The schema of debian/splitpackage is defined below
{
"type": "object",
"properties": {
"format": {
"type": "string",
"enum": ["dh_splitpackage 0.1"],
},
"packages": {
"type": "object",
"additionalProperties": {
"type": "object",
"properties": {
"inclusion_patterns": {
"type": [
"string", {
"type": "array",
"items": {
"type": "string"
}
}
],
"optional": true
},
"exclusion_patterns": {
"type": [
"string", {
"type": "array",
"items": "string"
}
],
"optional": true,
}
},
"additionalProperties": false,
}
},
"primary_package": {
"type": "string",
"optional": true
}
},
"additionalProperties": false
}

PATHNAME PATTERNS
Patterns used by dh_splitpackage are similar to globs with some
impor‐
tant distinctions. Technically they are implemented with python
regular
expressions. The algorithm used to translate from patterns to
regular

expressions is defined below.

In general there are three rules:

Dot is a normal character, not a wildcard.

A single start may match a filename and a filename only.
It will
never match a directory name. Note, if you use '*/' it
_will_

match directories but a sole '*' will not.

A special extension for matching directories is provided
in the
form of '**/' (star, star, forward slash). This pattern
matches
directories (and directories only) of any depth
(including not

matching any directory at all).

Each glob-like pattern is transformed to a regular expression
that is
matched to each pathname. Each pathname that denotes a
directory is
always terminated with a forward slash. The glob pattern is
transformed

with the following rules:

The dot pattern looses match-single-character semantics
normally

found in regular expressions. Each '.' is replaced with '.´.

The single star pattern is rewritten to ensure it only
matches
filenames, never directories. This is achieved by
replacing each
occurrence of '*' with slash that is guaranteed to
terminate

each pathname pointing to a directory.

The double-star-forward-slash pattern is rewritten to
ensure it
matches any sequence directories but never files.
This is
achieved by replacing each occurrence of '**/' with
'(.+/|)'.
This regular expression matches a non-empty string
followed by a

forward slash _or_ an empty string.

Finally to pattern must match the whole pathname. To do
that the

pattern is extended with leading '^' and trailing '$'.

EXAMPLES
Example pathname patterns
foo - match a file called 'foo' in the root
directory

* - match all files in the root directory
**/ - match all directories
**/* - match all files and directories
*.txt - match all files with the extension
'.txt' in the

root directory
foo/**/* - match all files and directories
underneath foo/

foo/bar/froz.txt - match this path explicitly
**/man*/*.[0-9] - match all manual pages, this shows how
regular
expressions can still be used alongside
the new

pattern extensions.

Hypothetical library package
A library separated into library, development files and
documentation.
Since there is no primary package designated any files not
matched by

the patterns defined below would simply be left behind.

{
"format": "dh_splitpackage 0.1",
"packages": {
"libfoo-dev": {
"inclusion_patterns": [
"**/*.a",
"**/*.h",
"**/*.la",
"**/*.m4",
"**/*.pc",
"man/man**/*.[1-9]"
]
},
"libfoo": {
"inclusion_patterns": "**/*.so"
},
"libfoo-doc": {
"inclusion_patterns": "/usr/share/doc/**/*"
}
}
}

Hypothetical server package
Hypothetical "server" package, with two packages for foo and
bar mod‐
ules, special "server-module-others" package that grabs the
remaining

modules and a documentation package.

The "server-module-others" package is using exclusion patterns
to avoid

clashes between "server-module-foo" and "server-module-bar".

{
"format": "dh_splitpackage 0.1",
"packages": {
"server": {
"inclusion_patterns": [
"/usr/bin/*",
"/etc/server.d/*.conf"
]
},
"server-module-foo": {
"inclusion_patterns": "/usr/lib/server-module-foo.so"
},
"server-module-bar": {
"inclusion_patterns": "/usr/lib/server-module-bar.so"
},
"server-module-others": {
"inclusion_patterns": "/usr/lib/server-module-*.so",
"exclusion_patterns": [
"/usr/lib/server-module-foo.so",
"/usr/lib/server-module-bar.so"
]
},
"server-doc": {
"inclusion_patterns": "/usr/share/doc/**/*"
}
}
}

Python library with unit tests
Python library with separated tests. Tests are in a
sub-directory of
the actual library. The package relies on "primary_package"
setting to

associate leftover files with the "python-foo" package.

{
"format": "dh_splitpackage 0.1",
"primary_package": "python-foo",
"packages": {
"python-foo.tests": {
"inclusion_patterns": "**/tests/**/*.py"
}
}
}

AUTHOR
This manual page as well as dh_splitpackage itself was written
by Zyg‐
munt Krynicki. You can contact me using the email address given
below.


BUGS
Please report bugs to Zygmunt Krynicki
<zygmunt.krynicki@canonical.com>




dh_splitpackage 0.2 June 2011
DH_SPLITPACKAGE(1)


Best regards
ZK



--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4DEF4E06.3050509@canonical.com">http://lists.debian.org/4DEF4E06.3050509@canonical.com
 

Thread Tools




All times are GMT. The time now is 10:41 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org