FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian dpkg

 
 
LinkBack Thread Tools
 
Old 06-03-2011, 10:47 PM
Sam Dunne
 
Default Declarative Diversions - Report 1

The following is the design spec so far for this project. It has to be fleshed out a bit more but most of the basic structure is there. Anyone who cares to give input is more than welcome.A further update will be posted on Monday to my blog, planet debian, the wiki page and to the dpkg mailing list.

DECLARATIVE DIVERSIONS======================------------Introduction------------A diversion is when it is possible to have dpkg not overwrite a file when it*
reinstalls the package it belongs to, and to have it put the file from the*package somewhere else instead.
Declarative diversions involves a new control file with a declarative*
syntax which dpkg will parse and process directly as part of the package unpack*and removal phases, eliminating the problems resulting from non-atomic*handling of diversions.


------Topics------There are a number of topics involved in implementing this kind of project
** What syntax do we use for the new control file?
** What dpkg does with the control file** How do we handle diversions to a non-existant directory?** How do we order unpacking a new package which adds a diversion?** How do we order removing a package which had a diversion?
** How do we handle errors?** What happens to dpkg-divert?

-----------------------------Details - Control File Syntax-----------------------------
It will conform to RFC2822 style with the following format:** Divert-From:** Divert-To:** Blank lines and lines beginning with '#' will be comments*'Divert-To' will be optional and if it is ommitted then files being diverted*
will have their filename changed to 'file.distrib'
The above style has it's advantages, one of the main ones being that there isno need to escape whitespace within filenames.


-------------------------------Details - Control File Handling-------------------------------Within control.tar.gz the file should be named 'diversions'
This file is then copied to /var/lib/info/$package.diversionsThis is ensuring we have a copy of the control file just in case it is needed.Any input on this subject would be appreciated.


---------------------------------------------------------Details - Handling Diversions to non-existant directories---------------------------------------------------------
Diverting files to directories that don't exist can cause a number of problems.*If the package does not 'own' the directory it may be left orphaned onremoval of the package
The package is responsible for ensuring the availability of the target directory in the unpack phase.

-------------------------------Details - Ordering Requirements
-------------------------------*=>Unpacking a new package that adds a diversion* **1. Unpack folders* **2. Add diversions* **3. Unpack files* *==Notes==
* *Performing diversions this way avoids trouble with creating folders and** *accidentally overwrtiting the wrong version of files.** *If we don't use this method the problems that can occur include creating*
* *folders before the package itself would create them which may cause** *permission errors.** *Another major problem would be extracting a new version of the file and ** *overwriting the old one.*
* *This might cause the wrong version of the file to be diverted and the** *package to break.* **** **=>Removing a package which had a diversion* **1. Remove files
* **2. Remove diversions* **3. Remove folders* *==Notes==* *This ensures that all files, diversions and folders are removed correctly
* *------------------------
Details - Error handling------------------------Errors in diversions will have to handled with a great deal of care due tothe fact that if they are not the package could be broken.
This means that a great deal of checks must be done to ensure that all the filescan be diverted properly before any actual diverting takes place. If they can't*the package installation/update must be stopped and rolled back to avoid the*
package being installed incorrectly or broken.

-----------------------Details - 'dpkg-divert'-----------------------When we impliment the new diversion method we should keep the current*
dpkg-divert. This allows maintainers to catch up with the new method without*breaking their packages. It also allows maintainers to perform some operations*that aren't support by the new method.


----------------Example Usage #1----------------The file to be diverted is '/usr/share/foo'It needs to be moved to '/usr/share/bar'

The syntax of the control file would be:
#Start FileDivert-From: /usr/share/fooDivert-To: /usr/share/bar#End File


----------------Example Usage #2----------------In this example the maintainer doesn't want to move the file to any specificfolder
The syntax of the control file would be:

#Start FileDivert-From: /usr/share/foo#End File
This would divert the file to '/usr/share/foo.distrib'

---------
Footnotes---------RFC2822 Guide:*** http://www.faqs.org/rfcs/rfc2822.html
First Email Thread on Declarative Diversions (First Message in Thread):
** http://lists.debian.org/debian-dpkg/2011/05/msg00102.html*Declarative Diversions Wiki:** http://wiki.debian.org/SummerOfCode2011/DeclarativeDiversions
*My Blog for this Project:*** http://blog.sam-dunne.com
--
Sam Dunne
BSc Computer Science, UCD Dublin.
 
Old 06-03-2011, 11:01 PM
Sune Vuorela
 
Default Declarative Diversions - Report 1

On 2011-06-03, Sam Dunne <gmail@sam.33mail.com> wrote:
> -----------------------
> Details - 'dpkg-divert'
> -----------------------
> When we impliment the new diversion method we should keep the current
> dpkg-divert. This allows maintainers to catch up with the new method
> without
> breaking their packages. It also allows maintainers to perform some
> operations
> that aren't support by the new method.

And whattabout sysadmins usages of dpkg-divert ?

/Sune


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: slrniuiptr.p7v.nospam@sshway.ssh.pusling.com">http ://lists.debian.org/slrniuiptr.p7v.nospam@sshway.ssh.pusling.com
 
Old 06-04-2011, 05:29 AM
Luk Claes
 
Default Declarative Diversions - Report 1

On 06/04/2011 12:47 AM, Sam Dunne wrote:

> DECLARATIVE DIVERSIONS
> ======================
> ------------
> Introduction
> ------------
> A diversion is when it is possible to have dpkg not overwrite a file
> when it
> reinstalls the package it belongs to, and to have it put the file from the
> package somewhere else instead.
>
> Declarative diversions involves a new control file with a declarative
> syntax which dpkg will parse and process directly as part of the package
> unpack
> and removal phases, eliminating the problems resulting from non-atomic
> handling of diversions.

Will it also solve the problem that currently diversions only really
work when no more than 2 packages are involved?

> ------
> Topics
> ------
> There are a number of topics involved in implementing this kind of project
>
> * What syntax do we use for the new control file?
> * What dpkg does with the control file
> * How do we handle diversions to a non-existant directory?
> * How do we order unpacking a new package which adds a diversion?
> * How do we order removing a package which had a diversion?
> * How do we handle errors?
> * What happens to dpkg-divert?

* What happens when 3 or more packages divert the same file/directory?

> -----------------------------
> Details - Control File Syntax
> -----------------------------
> It will conform to RFC2822 style with the following format:
> * Divert-From:
> * Divert-To:
> * Blank lines and lines beginning with '#' will be comments
>
> 'Divert-To' will be optional and if it is ommitted then files being
> diverted
> will have their filename changed to 'file.distrib'

Would it not be better to have the filename changed to
'file.<package_name>' if 'Divert-To' is not specified, so it's possible
to support more packages diverting the same file?

Cheers

Luk


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4DE9C29E.4040108@debian.org">http://lists.debian.org/4DE9C29E.4040108@debian.org
 
Old 06-04-2011, 06:38 AM
Raphael Hertzog
 
Default Declarative Diversions - Report 1

Hi,

On Fri, 03 Jun 2011, Sam Dunne wrote:
> -----------------------------
> Details - Control File Syntax
> -----------------------------
> It will conform to RFC2822 style with the following format:
> * Divert-From:
> * Divert-To:
> * Blank lines and lines beginning with '#' will be comments
>
> 'Divert-To' will be optional and if it is ommitted then files being
> diverted
> will have their filename changed to 'file.distrib'
>
> The above style has it's advantages, one of the main ones being that there
> is no need to escape whitespace within filenames.

While RFC2822 is ok, I would like to point out that up to now, this format
is only used for the "control" file, all the other files (triggers,
shlibs, symbols) use very simple and dedicated file formats.

Going with RFC2822 will either require some refactoring to change the
current "control" parser (lib/dpkg/parse.c) or to add another similar
parser.

This will be much more complicated than having to deal with escaping
whitespaces.

> -------------------------------
> Details - Control File Handling
> -------------------------------
> Within control.tar.gz the file should be named 'diversions'
> This file is then copied to /var/lib/info/$package.diversions
> This is ensuring we have a copy of the control file just in case it is
> needed.
> Any input on this subject would be appreciated.

This is the default behaviour of dpkg with any file in control.tar.gz
(except control).

> ------------------------
> Details - Error handling
> ------------------------
> Errors in diversions will have to handled with a great deal of care due to
> the fact that if they are not the package could be broken.

Much like anything in dpkg... :-)

On Sat, 04 Jun 2011, Luk Claes wrote:
> On 06/04/2011 12:47 AM, Sam Dunne wrote:
>
> > DECLARATIVE DIVERSIONS
> > ======================
> > ------------
> > Introduction
> > ------------
> > A diversion is when it is possible to have dpkg not overwrite a file
> > when it
> > reinstalls the package it belongs to, and to have it put the file from the
> > package somewhere else instead.
> >
> > Declarative diversions involves a new control file with a declarative
> > syntax which dpkg will parse and process directly as part of the package
> > unpack
> > and removal phases, eliminating the problems resulting from non-atomic
> > handling of diversions.
>
> Will it also solve the problem that currently diversions only really
> work when no more than 2 packages are involved?

I don't see how it could solve a problem that boils down to "How can we
install 2 versions of a file under the same filename?".

It would require some sort of priority to know which of the diverting
package must take precedence. At which point you're close to having
reinvented the alternative mechanism.

In any case, I don't think it's in the scope of this project.

> >
> > 'Divert-To' will be optional and if it is ommitted then files being
> > diverted
> > will have their filename changed to 'file.distrib'
>
> Would it not be better to have the filename changed to
> 'file.<package_name>' if 'Divert-To' is not specified, so it's possible
> to support more packages diverting the same file?

This name would be misleading... in the current scheme the renamed file
is the one being diverted.

And there's only one package that is being diverted, so suggesting that it
would allow supporting more packages diverting the same file is
misleading.

Supporting multiple diverting packages would require to keep backups of
diverting files under a name different from the target file.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110604063853.GA23891@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110604063853.GA23891@rivendell.home.ouaza.com
 
Old 06-04-2011, 07:17 AM
Steve Langasek
 
Default Declarative Diversions - Report 1

On Sat, Jun 04, 2011 at 07:29:02AM +0200, Luk Claes wrote:
> > -----------------------------
> > Details - Control File Syntax
> > -----------------------------
> > It will conform to RFC2822 style with the following format:
> > * Divert-From:
> > * Divert-To:
> > * Blank lines and lines beginning with '#' will be comments
> >
> > 'Divert-To' will be optional and if it is ommitted then files being
> > diverted
> > will have their filename changed to 'file.distrib'

> Would it not be better to have the filename changed to
> 'file.<package_name>' if 'Divert-To' is not specified, so it's possible
> to support more packages diverting the same file?

If you do that, how do you keep track of which package's file was diverted
where, so that on *removal*, the files are put where they belong?

Why do you *want* to have parallel diversions of the same file by more than
one package? It may seem the answer is obvious, but if you think about it I
believe you'll find those semantics aren't actually useful. *Nested*
diversions can be useful (one package diverts foo to foo.distrib and wraps
it; another package diverts foo.distrib to foo.distrib.distrib and wraps it
again), but having two diversions happen in parallel, where the unpack order
determines which package ends up on top, isn't useful at all.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org
 
Old 06-04-2011, 07:51 AM
Luk Claes
 
Default Declarative Diversions - Report 1

On 06/04/2011 09:17 AM, Steve Langasek wrote:
> On Sat, Jun 04, 2011 at 07:29:02AM +0200, Luk Claes wrote:
>>> -----------------------------
>>> Details - Control File Syntax
>>> -----------------------------
>>> It will conform to RFC2822 style with the following format:
>>> * Divert-From:
>>> * Divert-To:
>>> * Blank lines and lines beginning with '#' will be comments
>>>
>>> 'Divert-To' will be optional and if it is ommitted then files being
>>> diverted
>>> will have their filename changed to 'file.distrib'
>
>> Would it not be better to have the filename changed to
>> 'file.<package_name>' if 'Divert-To' is not specified, so it's possible
>> to support more packages diverting the same file?
>
> If you do that, how do you keep track of which package's file was diverted
> where, so that on *removal*, the files are put where they belong?

Indeed.

> Why do you *want* to have parallel diversions of the same file by more than
> one package? It may seem the answer is obvious, but if you think about it I
> believe you'll find those semantics aren't actually useful. *Nested*
> diversions can be useful (one package diverts foo to foo.distrib and wraps
> it; another package diverts foo.distrib to foo.distrib.distrib and wraps it
> again), but having two diversions happen in parallel, where the unpack order
> determines which package ends up on top, isn't useful at all.

Because people want to have both atomic changes of their /bin/sh as well
as being able to choose between more than 2 options for their /bin/sh ...

Cheers

Luk


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4DE9E40B.7070806@debian.org">http://lists.debian.org/4DE9E40B.7070806@debian.org
 
Old 06-04-2011, 07:59 AM
Jonathan Nieder
 
Default Declarative Diversions - Report 1

Luk Claes wrote:

> Because people want to have both atomic changes of their /bin/sh as well
> as being able to choose between more than 2 options for their /bin/sh ...

See http://repo.or.cz/w/dash/debian/jrn.git/commitdiff/cea77edd1 for
an example where the ability to atomically "steal" a diversion would
be helpful. This is certainly not a candidate for declarative
diversions (and the tricky use of diversions is just meant as a means
of escape from a scenario where diversions were not the right tool for
the job imho).


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110604075950.GC9081@elie">http://lists.debian.org/20110604075950.GC9081@elie
 
Old 06-04-2011, 08:07 AM
Luk Claes
 
Default Declarative Diversions - Report 1

On 06/04/2011 09:59 AM, Jonathan Nieder wrote:
> Luk Claes wrote:
>
>> Because people want to have both atomic changes of their /bin/sh as well
>> as being able to choose between more than 2 options for their /bin/sh ...
>
> See http://repo.or.cz/w/dash/debian/jrn.git/commitdiff/cea77edd1 for
> an example where the ability to atomically "steal" a diversion would
> be helpful. This is certainly not a candidate for declarative
> diversions (and the tricky use of diversions is just meant as a means
> of escape from a scenario where diversions were not the right tool for
> the job imho).

It might not be the best way to do it, but at the time we discussed the
possible solutions and the diversions one was the one that looked the
safest and was possible in the short term.

I'm all for having a better solution, so please do continue your quest.

Cheers

Luk


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4DE9E7CF.4040406@debian.org">http://lists.debian.org/4DE9E7CF.4040406@debian.org
 
Old 06-11-2011, 07:03 AM
Goswin von Brederlow
 
Default Declarative Diversions - Report 1

Steve Langasek <vorlon@debian.org> writes:

> On Sat, Jun 04, 2011 at 07:29:02AM +0200, Luk Claes wrote:
>> > -----------------------------
>> > Details - Control File Syntax
>> > -----------------------------
>> > It will conform to RFC2822 style with the following format:
>> > * Divert-From:
>> > * Divert-To:
>> > * Blank lines and lines beginning with '#' will be comments
>> >
>> > 'Divert-To' will be optional and if it is ommitted then files being
>> > diverted
>> > will have their filename changed to 'file.distrib'
>
>> Would it not be better to have the filename changed to
>> 'file.<package_name>' if 'Divert-To' is not specified, so it's possible
>> to support more packages diverting the same file?
>
> If you do that, how do you keep track of which package's file was diverted
> where, so that on *removal*, the files are put where they belong?

Because dpkg would record the name in its database in the "Divert-To"
slot. It already needs to know the name in the case Divert-To was
specified so this doesn't make it more complex.

> Why do you *want* to have parallel diversions of the same file by more than
> one package? It may seem the answer is obvious, but if you think about it I
> believe you'll find those semantics aren't actually useful. *Nested*
> diversions can be useful (one package diverts foo to foo.distrib and wraps
> it; another package diverts foo.distrib to foo.distrib.distrib and wraps it
> again), but having two diversions happen in parallel, where the unpack order
> determines which package ends up on top, isn't useful at all.

One use for this would be to later merge in alternatives.

Think about it. What are alternatives? Some packages all provide the
same file and based on priority or users choice one of them wins out and
gets to use the name. One minor change that would probably be usefull
for alternatives would be to allways store the "file" as
"file.<package_name>" even for the package that gets picked to provide
"file".

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 87d3ikx32l.fsf@frosties.localnet">http://lists.debian.org/87d3ikx32l.fsf@frosties.localnet
 

Thread Tools




All times are GMT. The time now is 12:13 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org