FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian dpkg

 
 
LinkBack Thread Tools
 
Old 02-14-2012, 03:40 PM
Guillem Jover
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

On Tue, 2012-02-14 at 14:28:58 +0000, Ian Jackson wrote:
> Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> > On Mon, 2012-02-13 at 22:43:04 -0800, Russ Allbery wrote:
> > > * The binNMU process is changed to add the binNMU changelog entry to an
> > > arch-qualified file (changelog.Debian.arch, probably). We need to
> > > figure out what this means if the package being binNMU'd has a
> > > /usr/share/doc/<package> symlink to another package, though; it's not
> > > obvious what to do here.
> >
> > This requires IMO multitude of hacks when the simplest and obvious
> > arch-qualified pkgname solves this cleanly, and allows debhelper to
> > automatically deal with it. And for tools to just change where they
> > always look for those files in the M-A:same case regardless of the
> > package being binNMUed or not.
>
> I agree that it would be nice to always arch-qualify the changelog
> filename. But that would involve a lot of changes to
> changelog-reading tools which we perhaps don't want to do right now.

I've never proposed to arch-qualify the filename for the stuff under
/usr/share/doc/pkgname/, I've proposed to arch-qualify the pkgname in
the path (/usr/share/doc/pkgname:arch/), but only for M-A:same packages,
which are the only ones needing the disambiguation. This is how dpkg
handles pkgname output, or how it stores their data in the db too.

And it should be easy to ask a multiarch enabled dpkg-query for example
to normalize the pkgname output to be used on those paths, or otherwise
do it by hand:

if M-A == same
pkgname:arch
else
pkgname

> Note that even if we decide to always arch-qualify, we will still have
> lots of old packages so all changelog-reading tools will need to look
> in both places.

> For most changelog-reading tools it won't be very troublesome if they
> accidentally don't spot a binNMU entry. So Russ's proposal is a good
> step towards your proposal. And if we decide we don't need to go all
> the way then it's good enough for now.

How many tools are there that actually read the binary package changelog
file anyway? I only know of packages.d.o. Any other tool reading from
the installed path, cannot really rely on it being present at all
anyway, per policy.

And in addition, binNMU split changelogs are going to be there forever,
and as such their possible double locations. While the possible double
location for M-A:same packages using pkgname:arch qualified pathnames
would only be temporary and disappear once the packages have been rebuilt
with a new debhelper which automatically installs them in the correct
place.

> > So this is still pretty much unconvincing, and seems like clinging
> > into the refcnt'ing “solution” while it makes things overall more
> > complicated, will introduce inconsistency and incertainty to
> > maintainers, needs way more global changes to keep it going, etc.
>
> I think the refcounting approach is very worthwhile because it
> eliminates unnecessary work (by human maintainers) in many simple
> cases.

As I mentioned in Riku's reply, the amount of packages that would need
splitting that would otherwise not be needed should be even less than
before (which was predicted at around 700), also as I mentioned there
too, nothing prevents us from arch-qualifying paths (with Debian arch
or multiarch triplet depending on the case) if that's more convenient
or safer (as per your essential data example), and is what we've been
doing anyway for arch-indep data shipped in arch:any packages all along.
Given the amount of hacks or special casing piling up to make refcnt'ing
workable, when all that's really needed is a one time handling (or a
possible additional change for already converted packages, for things
that debhelper might not be able to handle) of moving qualifying paths
or splitting into new packages, it really does not seem worth it, no.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120214164015.GA27571@gaara.hadrons.org">http://lists.debian.org/20120214164015.GA27571@gaara.hadrons.org
 
Old 02-15-2012, 12:15 AM
Guillem Jover
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

On Tue, 2012-02-14 at 14:28:58 +0000, Ian Jackson wrote:
> I think the refcounting approach is very worthwhile because it
> eliminates unnecessary work (by human maintainers) in many simple
> cases.

Aside from what I said on my other reply, I just wanted to note that
this seems to be a recurring point of tension in the project when it
comes to archive wide source package changes, where supposed short
term convenience (with its usually long term harmful effects) appears
to initially seduce people over what seems to be the cleaner although
slightly a bit more laborious solution.

Other recent-ish incarnations of this tension could be the build-arch
build-indep targets, or the build flag settings; where the former got
recently resolved so that the right thing to do is for *all* packages
needing to eventually support those targets, or for the latter which
got switched from the seemingly more convenient to the more laborious
but correct solution, that is, *all* packages need to set those build
flags by themselves.

This is a fundamental issue with how our source packages are handled,
and the freedom and power it gives to experiment and implement them
whatever way the maintainer wants, has the price that doing some
archive wide changes is sometimes more costly, than changing something
centrally and be done with it. But trying to workaround this by coming
up with stacks of hacked up solutions will not solve that fundamental
issue, and this kind of tension will keep coming up again and again,
as long as the foundation is not reworked. Either that, or the project
needs to accept that fact and learn to live with this kind of changes,
with patience.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120215011510.GA15353@gaara.hadrons.org">http://lists.debian.org/20120215011510.GA15353@gaara.hadrons.org
 
Old 02-15-2012, 12:15 AM
Guillem Jover
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

On Tue, 2012-02-14 at 14:28:58 +0000, Ian Jackson wrote:
> I think the refcounting approach is very worthwhile because it
> eliminates unnecessary work (by human maintainers) in many simple
> cases.

Aside from what I said on my other reply, I just wanted to note that
this seems to be a recurring point of tension in the project when it
comes to archive wide source package changes, where supposed short
term convenience (with its usually long term harmful effects) appears
to initially seduce people over what seems to be the cleaner although
slightly a bit more laborious solution.

Other recent-ish incarnations of this tension could be the build-arch
build-indep targets, or the build flag settings; where the former got
recently resolved so that the right thing to do is for *all* packages
needing to eventually support those targets, or for the latter which
got switched from the seemingly more convenient to the more laborious
but correct solution, that is, *all* packages need to set those build
flags by themselves.

This is a fundamental issue with how our source packages are handled,
and the freedom and power it gives to experiment and implement them
whatever way the maintainer wants, has the price that doing some
archive wide changes is sometimes more costly, than changing something
centrally and be done with it. But trying to workaround this by coming
up with stacks of hacked up solutions will not solve that fundamental
issue, and this kind of tension will keep coming up again and again,
as long as the foundation is not reworked. Either that, or the project
needs to accept that fact and learn to live with this kind of changes,
with patience.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120215011510.GA15353@gaara.hadrons.org">http://lists.debian.org/20120215011510.GA15353@gaara.hadrons.org
 
Old 02-15-2012, 02:54 AM
Joey Hess
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

Guillem Jover wrote:
> Aside from what I said on my other reply, I just wanted to note that
> this seems to be a recurring point of tension in the project when it
> comes to archive wide source package changes, where supposed short
> term convenience (with its usually long term harmful effects) appears
> to initially seduce people over what seems to be the cleaner although
> slightly a bit more laborious solution.
>
> Other recent-ish incarnations of this tension could be the build-arch
> build-indep targets, or the build flag settings; where the former got
> recently resolved so that the right thing to do is for *all* packages
> needing to eventually support those targets, or for the latter which
> got switched from the seemingly more convenient to the more laborious
> but correct solution, that is, *all* packages need to set those build
> flags by themselves.
>
> This is a fundamental issue with how our source packages are handled,
> and the freedom and power it gives to experiment and implement them
> whatever way the maintainer wants, has the price that doing some
> archive wide changes is sometimes more costly, than changing something
> centrally and be done with it. But trying to workaround this by coming
> up with stacks of hacked up solutions will not solve that fundamental
> issue, and this kind of tension will keep coming up again and again,
> as long as the foundation is not reworked. Either that, or the project
> needs to accept that fact and learn to live with this kind of changes,
> with patience.

Very interesting mail. While I certianly agree with your examples, it's
worth remembering the counterexample of the /usr/doc transition which
took approximately 5 years to complete[1], and probably could have been
accomplished quickly and without pain with a simple hack to dpkg.

Anyway, my worry about the refcounting approach (or perhaps M-A: same in
general) is not the details of the implementation in dpkg, but the added
mental complexity of dpkg now being able to have multiple distinct
packages installed under the same name. I had a brief exposure to rpm,
which can install multiple versions of the same package, and that was
the main cause of much confusing behavior in rpm. While dpkg's invariant
that all co-installable package names be unique (and have unique files)
has certianly led to lots of ugly package names, it's kept the users'
and developers' mental models quite simple.

I worry that we have barely begun to scratch the surface of the added
complexity of losing this invariant.

--
see shy jo

[1] To the extent it was ever completed.. master.debian.org still has
a vestigial /usr/doc/
 
Old 02-15-2012, 02:54 AM
Joey Hess
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

Guillem Jover wrote:
> Aside from what I said on my other reply, I just wanted to note that
> this seems to be a recurring point of tension in the project when it
> comes to archive wide source package changes, where supposed short
> term convenience (with its usually long term harmful effects) appears
> to initially seduce people over what seems to be the cleaner although
> slightly a bit more laborious solution.
>
> Other recent-ish incarnations of this tension could be the build-arch
> build-indep targets, or the build flag settings; where the former got
> recently resolved so that the right thing to do is for *all* packages
> needing to eventually support those targets, or for the latter which
> got switched from the seemingly more convenient to the more laborious
> but correct solution, that is, *all* packages need to set those build
> flags by themselves.
>
> This is a fundamental issue with how our source packages are handled,
> and the freedom and power it gives to experiment and implement them
> whatever way the maintainer wants, has the price that doing some
> archive wide changes is sometimes more costly, than changing something
> centrally and be done with it. But trying to workaround this by coming
> up with stacks of hacked up solutions will not solve that fundamental
> issue, and this kind of tension will keep coming up again and again,
> as long as the foundation is not reworked. Either that, or the project
> needs to accept that fact and learn to live with this kind of changes,
> with patience.

Very interesting mail. While I certianly agree with your examples, it's
worth remembering the counterexample of the /usr/doc transition which
took approximately 5 years to complete[1], and probably could have been
accomplished quickly and without pain with a simple hack to dpkg.

Anyway, my worry about the refcounting approach (or perhaps M-A: same in
general) is not the details of the implementation in dpkg, but the added
mental complexity of dpkg now being able to have multiple distinct
packages installed under the same name. I had a brief exposure to rpm,
which can install multiple versions of the same package, and that was
the main cause of much confusing behavior in rpm. While dpkg's invariant
that all co-installable package names be unique (and have unique files)
has certianly led to lots of ugly package names, it's kept the users'
and developers' mental models quite simple.

I worry that we have barely begun to scratch the surface of the added
complexity of losing this invariant.

--
see shy jo

[1] To the extent it was ever completed.. master.debian.org still has
a vestigial /usr/doc/
 
Old 02-15-2012, 06:40 AM
Raphael Hertzog
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

On Tue, 14 Feb 2012, Guillem Jover wrote:
> I've never proposed to arch-qualify the filename for the stuff under
> /usr/share/doc/pkgname/, I've proposed to arch-qualify the pkgname in
> the path (/usr/share/doc/pkgname:arch/), but only for M-A:same packages,
> which are the only ones needing the disambiguation. This is how dpkg
> handles pkgname output, or how it stores their data in the db too.
[...]
> How many tools are there that actually read the binary package changelog
> file anyway?

There's apt-listchanges surely. And probably a bunch of other that are
less known.

I don't know if it's worth it, but if we go down that route, and if we
want to keep /usr/share/doc/pkgname on user's systems we could create
a new command in dpkg-maintscript-helper to manage that path as
a symlink to the native "M-A: same" package (if possible, otherwise
to any installed arch). That dpkg-maintscript-helper call could be
auto-enabled by debhelper for "M-A: same" packages.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120215074019.GC24340@rivendell.home.ouaza.com">h ttp://lists.debian.org/20120215074019.GC24340@rivendell.home.ouaza.com
 
Old 02-15-2012, 06:40 AM
Raphael Hertzog
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

On Tue, 14 Feb 2012, Guillem Jover wrote:
> I've never proposed to arch-qualify the filename for the stuff under
> /usr/share/doc/pkgname/, I've proposed to arch-qualify the pkgname in
> the path (/usr/share/doc/pkgname:arch/), but only for M-A:same packages,
> which are the only ones needing the disambiguation. This is how dpkg
> handles pkgname output, or how it stores their data in the db too.
[...]
> How many tools are there that actually read the binary package changelog
> file anyway?

There's apt-listchanges surely. And probably a bunch of other that are
less known.

I don't know if it's worth it, but if we go down that route, and if we
want to keep /usr/share/doc/pkgname on user's systems we could create
a new command in dpkg-maintscript-helper to manage that path as
a symlink to the native "M-A: same" package (if possible, otherwise
to any installed arch). That dpkg-maintscript-helper call could be
auto-enabled by debhelper for "M-A: same" packages.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Pre-order a copy of the Debian Administrator's Handbook and help
liberate it: http://debian-handbook.info/liberation/


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120215074019.GC24340@rivendell.home.ouaza.com">h ttp://lists.debian.org/20120215074019.GC24340@rivendell.home.ouaza.com
 
Old 02-15-2012, 03:41 PM
Ian Jackson
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> On Tue, 2012-02-14 at 14:28:58 +0000, Ian Jackson wrote:
> > I think the refcounting approach is very worthwhile because it
> > eliminates unnecessary work (by human maintainers) in many simple
> > cases.
>
> Aside from what I said on my other reply, I just wanted to note that
> this seems to be a recurring point of tension in the project when it
> comes to archive wide source package changes, where supposed short
> term convenience (with its usually long term harmful effects) appears
> to initially seduce people over what seems to be the cleaner although
> slightly a bit more laborious solution.

The refcnt doesn't just eliminate unnecessary multiarch
conversion work. It also eliminates unnecessary maintenance effort.
Maintaining a split package will be more work than without.

I think that over the lifetime of the multiarch deployment this extra
packaging work will far outweigh the extra maintenance and
documentation burden of the refcnt feature.

> [...] But trying to workaround this by coming
> up with stacks of hacked up solutions [...]

I disagree with your tendentious phrasing. The refcnt feature is not
a "hacked up solution" (nor a stack of them). It is entirely normal
in Debian core tools (as in any substantial piece of software serving
a lot of diverse needs) to have extra code to make it easier to deploy
or use in common cases simpler.

Ian.


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20283.57393.237949.649743@chiark.greenend.org.uk"> http://lists.debian.org/20283.57393.237949.649743@chiark.greenend.org.uk
 
Old 02-15-2012, 03:41 PM
Ian Jackson
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> On Tue, 2012-02-14 at 14:28:58 +0000, Ian Jackson wrote:
> > I think the refcounting approach is very worthwhile because it
> > eliminates unnecessary work (by human maintainers) in many simple
> > cases.
>
> Aside from what I said on my other reply, I just wanted to note that
> this seems to be a recurring point of tension in the project when it
> comes to archive wide source package changes, where supposed short
> term convenience (with its usually long term harmful effects) appears
> to initially seduce people over what seems to be the cleaner although
> slightly a bit more laborious solution.

The refcnt doesn't just eliminate unnecessary multiarch
conversion work. It also eliminates unnecessary maintenance effort.
Maintaining a split package will be more work than without.

I think that over the lifetime of the multiarch deployment this extra
packaging work will far outweigh the extra maintenance and
documentation burden of the refcnt feature.

> [...] But trying to workaround this by coming
> up with stacks of hacked up solutions [...]

I disagree with your tendentious phrasing. The refcnt feature is not
a "hacked up solution" (nor a stack of them). It is entirely normal
in Debian core tools (as in any substantial piece of software serving
a lot of diverse needs) to have extra code to make it easier to deploy
or use in common cases simpler.

Ian.


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20283.57393.237949.649743@chiark.greenend.org.uk"> http://lists.debian.org/20283.57393.237949.649743@chiark.greenend.org.uk
 
Old 02-29-2012, 06:51 PM
Guillem Jover
 
Default Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)

On Wed, 2012-02-15 at 16:41:21 +0000, Ian Jackson wrote:
> Guillem Jover writes ("Re: Multiarch file overlap summary and proposal (was: Summary: dpkg shared / reference counted files and version match)"):
> > [...] But trying to workaround this by coming
> > up with stacks of hacked up solutions [...]
>
> I disagree with your tendentious phrasing. The refcnt feature is not
> a "hacked up solution" (nor a stack of them). It is entirely normal
> in Debian core tools (as in any substantial piece of software serving
> a lot of diverse needs) to have extra code to make it easier to deploy
> or use in common cases simpler.

All along this thread, when referring to the additional complexity and
the additional hacks, I've not been talking about the refcnt'ing at
all, but to all the other "fixes" needed to make it a workable solution.

regards,
guillem


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20120229195152.GA4852@gaara.hadrons.org">http://lists.debian.org/20120229195152.GA4852@gaara.hadrons.org
 

Thread Tools




All times are GMT. The time now is 05:26 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org