Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian dpkg (http://www.linux-archive.org/debian-dpkg/)
-   -   Triggers on file patterns (similar to bug #553266) (http://www.linux-archive.org/debian-dpkg/561432-triggers-file-patterns-similar-bug-553266-a.html)

Sjors Gielen 08-07-2011 02:35 PM

Triggers on file patterns (similar to bug #553266)
 
Hello developers,

In current dpkg releases, it is possible to add triggers by expressing an interest in a directory. Whenever a file is installed or modified in that directory, the PostInst script is called and a trigger is able to run. This makes it possible, for example, to be triggered whenever a file is installed in /usr/lib, so a script can go through all the files there and, for example, run some actions on every *.la file found.

However, this is quite impractical if all you're interested in are those *.la files. You don't only go through all of /usr/lib (including subdirectories) for almost nothing, but also you have a problem whenever suddenly a .la file is installed in, say, /opt/foo/lib. Therefore, it would be cool if it were possible to add triggers on file name patterns, like "*.la". This is similar to what bug #553266[0] proposes; it doesn't want to look at everything in /usr/lib when it is just interested in files matching "*.so.*".

My questions: Have you thought about something like this? What do you think about it now? Would you accept patches implementing this functionality?

Thanks,
Sjors

[0] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=553266

Raphael Hertzog 08-07-2011 06:49 PM

Triggers on file patterns (similar to bug #553266)
 
On Sun, 07 Aug 2011, Sjors Gielen wrote:
> My questions: Have you thought about something like this? What do you
> think about it now?

I think that it's encouraging people to abuse triggers. In essence a
trigger is useful to configure something for the package that activates
the file trigger or at least something for the package that is
activated.

I don't see what you would "configure" by matching *.la files. So for now
I think it's a bad idea and you would have to come up with a good use case
and a good rationale before I accept such a patch.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110807184925.GG19159@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110807184925.GG19159@rivendell.home.ouaza.com

Sjors Gielen 08-10-2011 11:30 AM

Triggers on file patterns (similar to bug #553266)
 
Op 7 aug 2011, om 20:49 heeft Raphael Hertzog het volgende geschreven:

> I think that it's encouraging people to abuse triggers. In essence a
> trigger is useful to configure something for the package that activates
> the file trigger or at least something for the package that is
> activated.
>
> I don't see what you would "configure" by matching *.la files. So for now
> I think it's a bad idea and you would have to come up with a good use case
> and a good rationale before I accept such a patch.

Hi Raphael,

(disclaimer: I've been doing work for the Fink project, but in no way represent the project as a whole, just myself and my own personal opinions.)

The reason I'm asking is because of the Fink package manager for Mac OS X, which uses dpkg and apt extensively. We're in the process of upgrading from the ancient dpkg 1.10 to 1.15. One of the advantages of this version is that we are now blessed with triggers, which can help with some of the things we had implemented in Fink itself, earlier. For example, the TeXinfo system from Debian is a much handier, better approach than the "old way": every package maintainer had to specify what infodocs a package had, only calling install-info on these. And if the directory was corrupted because some external software installation had touched it, you're stuck with a broken directory, and on dpkg 1.15 it would even cause fatal package installation errors because of dpkg -> GNU install-info, leading to a broken package state. So I'm very happy the trigger system, combined with the install-info / update-info-dir system from Debian, helps us to fix those issues properly now.

But there's one piece missing: Mac OS X has its own solution for the linking of libraries, which conflicts with libtool's "dependency_libs" in all .la files (I don't completely know the internals). This has always been solved by modifying all .la files that packages install and clearing the dependency_libs statement before actually installing the .la files. We can't simply patch libtool either, because it's installed on the system, not by Fink itself.

The code for this is in Fink, and iirc it works something like this: for every package that's going to be installed, check every file it installs, and for every ".la" file, modify the file a bit. Thinking about how triggers work, it seems silly to have this in Fink itself: it matches everything 'filesystem-aware' triggers are supposed to do:

* triggered when there are filesystem changes matching a rule,
* runs a script that will 'configure' the installed package.

So if we could add a trigger on every file matching the "*.la" glob (or the /.la$/ regex, if you prefer that method), it sounds like a good feature for dpkg with valid use-cases, and it would allow us to remove this 'duplicate' code in Fink. An alternative would be to have a "dpkg-trigger" in every package installing .la files, but that kind of defeats the purpose.

What do you think about the feature with this background? :)

Thanks,
Sjors

Raphael Hertzog 08-10-2011 01:31 PM

Triggers on file patterns (similar to bug #553266)
 
Hi,

On Wed, 10 Aug 2011, Sjors Gielen wrote:
> But there's one piece missing: Mac OS X has its own solution for the
> linking of libraries, which conflicts with libtool's "dependency_libs"
> in all .la files (I don't completely know the internals). This has
> always been solved by modifying all .la files that packages install and
> clearing the dependency_libs statement before actually installing the
> .la files. We can't simply patch libtool either, because it's installed
> on the system, not by Fink itself.

Then you will be glad to learn that Debian is dropping .la files when
possible and emptying dependency_libs when not possible.

http://wiki.debian.org/ReleaseGoals/LAFileRemoval

In any case, I don't agree that the modification of those .la files
should happen at installation time with a trigger. It's something
that should be done at build time.

To avoid having to modify all packages, I would much prefer that
you provide a patch for http://bugs.debian.org/570934 and decide
to use that new feature to clear dependency_libs in the remaining .la
files.

That way we're not introducing a discrepancy between the installed file
and the checksum recorded in the md5sums file, and we're not pushing the
cost of doing the modification to all users.

Would this work for you? I guess so since you have to rebuild the packages
anyway, isn't it?

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110810133120.GD8026@rivendell.home.ouaza.com">ht tp://lists.debian.org/20110810133120.GD8026@rivendell.home.ouaza.com

Guillem Jover 08-11-2011 01:48 PM

Triggers on file patterns (similar to bug #553266)
 
Hi Sjors!

On Wed, 2011-08-10 at 13:30:56 +0200, Sjors Gielen wrote:
> The reason I'm asking is because of the Fink package manager for
> Mac OS X, which uses dpkg and apt extensively. We're in the process
> of upgrading from the ancient dpkg 1.10 to 1.15.

Ah nice, I've looked at the changes done to dpkg in Fink at some point
in the past, and some seemed a bit hackish, or stuff that should be
fixed somewhere else or in a different way. I'd be glad to try to
help reduce the delta you carry on dpkg.

> But there's one piece missing: Mac OS X has its own solution for the
> linking of libraries, which conflicts with libtool's "dependency_libs"
> in all .la files (I don't completely know the internals). This has
> always been solved by modifying all .la files that packages install
> and clearing the dependency_libs statement before actually installing
> the .la files. We can't simply patch libtool either, because it's
> installed on the system, not by Fink itself.

In Debian what's being done is clear out the dependency_libs variable
at package creation time. No need to delegate that step to all
installing systems, just doing it once at build time seems cleaner and
more efficient.

<http://wiki.debian.org/ReleaseGoals/LAFileRemoval>

> What do you think about the feature with this background? :)

I think this is the wrong approach, and you should reuse what Debian
packagers are doing instead.

thanks,
guillem


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110811134830.GA27564@gaara.hadrons.org">http://lists.debian.org/20110811134830.GA27564@gaara.hadrons.org

"Daniel Macks" 08-11-2011 09:46 PM

Triggers on file patterns (similar to bug #553266)
 
On Wed, 10 Aug 2011 15:31:20 0200, Raphael Hertzog wrote:
Hi,
>
> On Wed, 10 Aug 2011, Sjors Gielen wrote:
> > But there's one piece missing: Mac OS X has its own solution for the
> > linking of libraries, which conflicts with libtool's "dependency_libs"
> > in all .la files (I don't completely know the internals). This has
> > always been solved by modifying all .la files that packages install and
> > clearing the dependency_libs statement before actually installing the
> > .la files. We can't simply patch libtool either, because it's installed
> > on the system, not by Fink itself.
>
> Then you will be glad to learn that Debian is dropping .la files when
> possible and emptying dependency_libs when not possible.
>
> http://wiki.debian.org/ReleaseGoals/LAFileRemoval
>
> In any case, I don't agree that the modification of those .la files
> should happen at installation time with a trigger. It's something
> that should be done at build time.
>
> To avoid having to modify all packages, I would much prefer that
> you provide a patch for http://bugs.debian.org/570934 and decide
> to use that new feature to clear dependency_libs in the remaining .la
> files.
>
> That way we're not introducing a discrepancy between the installed file
> and the checksum recorded in the md5sums file, and we're not pushing the
> cost of doing the modification to all users.
>
> Would this work for you? I guess so since you have to rebuild the packages
> anyway, isn't it?

Bit of history/context for the current feature in fink (which I
wrote)--rationale for current implementation and why the Debian
approaches (file removal and build-system magic) aren't suitable.

First, fink doesn't have periodic system-wide release-levels the way
Debian does, so we don't have an opportunity to push out a whole new
set of packages (rebuild across-the-board) with a self-consistent new
policy or release a full binary collection built to a specific
point-distro policy. Rather, users update packages on their own
schedule and may be running (within the limits of Depends) specs a wide
mix of latest release of a library but year-old reverse-dependencies
still present). As a result, we use the current implementation to avoid
causing breakage for users and maintain control over the .deb contents
(at the expense
of installed files not matching the .deb). It's a fatal error if a .la
disappears but another still points to it via dependency_libs , so we
took the data-cleaning approach rather than deleting the files
themselves.

Doing it at build-time would entail bumping the Revision (fink takes a
strong policy stand on "same name-version-revision sound gives same
.deb binary" consistency). Altering the build system would be
technically easy but break that policy, and we couldn't be assured by
*any* technical means that a user would have the "improved" .deb
installed. The change would therefore have to be done on a per-package
("opt-in") basis by adding some explicit token to the source package
build recipe. The rev-up would cause all users to do lots of rebuilding
(fink is largely build-from-source by end-users not binary distro) for
no gain (from their perspective). The change had to happen for all (or
at least many hundreds of) packages "soon" because having all that .la
data was holding up a bunch of other development. So the automatic
PostInst (pushed out before dpkg had stabilized the actual "triggers"
feature IIRC, but we'd love to use an upstream feature instead of local
tricks!) handled it all cleanly: guaranteed system-wide removal of .la data
without requiring rebuilding.

We could easily add a build-recipe control, which maintainers could add
when packages are being updated for other reasons (i.e., users are
rebuilding anyway) and maybe even could hurry this out for the fresh
packages being added to the 10.7 collection, but we're still years away
from dropping older systems' collections that are relying on having .la
but having an easy automatic way to clear their data.

dan

--
Daniel Macks
dmacks@netspace.org


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110811174618.ka9y4najackk8occ@webmail.netspace.o rg">http://lists.debian.org/20110811174618.ka9y4najackk8occ@webmail.netspace.o rg

Sjors Gielen 08-19-2011 09:19 PM

Triggers on file patterns (similar to bug #553266)
 
Hi Raphael, Guillem, Daniel,

Even though I agree with the standpoint of the dpkg developers that using triggers to modify package-installed files is something to avoid, I must side with the practical point introduced by the Fink core developers in that this is something that is not lightly fixed otherwise. The fact that whatever we do in the Fink software, the produced .debs should stay the same and .debs from months or years ago might still be used on users' machines, and we don't want to rebuild everything, means any process like this is probably best to run after installation for now (alas debsums). This is a totally unsatisfying dirty solution, but I propose any discussion to change it will be on the thread I just started on the fink-core list for 10.7; let's take this as a starting point for the rest of this thread.

There are other use cases to come up with to defend the idea of a glob-, regexp- or even just file extension-based trigger ("I want to do a virus scan on every .so library file", "I want every .pl file using #!/usr/bin/perl to use #!/usr/local/bin/perl as the interpreter", "I want to build an index for every HTML file with 'foo' in its name"), but I must admit those are all not my use cases. Nevertheless, do you think the feature would be an addition to dpkg? I'm asking specifically because I'm trying to, as Guillem nicely put it, reduce the delta we carry on dpkg -- writing a patch only to have it rejected upstream either increases this delta or is a sad waste of my time. If you think it wouldn't be a valuable addition, I'd rather invest some time in seeking other parts of our changes to the Debian tools to upstream :)

Thanks,
Sjors

Raphael Hertzog 09-05-2011 08:36 AM

Triggers on file patterns (similar to bug #553266)
 
Hi Sjors,

On Fri, 19 Aug 2011, Sjors Gielen wrote:
> There are other use cases to come up with to defend the idea of a glob-,
> regexp- or even just file extension-based trigger ("I want to do a virus
> scan on every .so library file", "I want every .pl file using
> #!/usr/bin/perl to use #!/usr/local/bin/perl as the interpreter", "I
> want to build an index for every HTML file with 'foo' in its name"), but
> I must admit those are all not my use cases. Nevertheless, do you think
> the feature would be an addition to dpkg? I'm asking specifically
> because I'm trying to, as Guillem nicely put it, reduce the delta we
> carry on dpkg -- writing a patch only to have it rejected upstream
> either increases this delta or is a sad waste of my time. If you think
> it wouldn't be a valuable addition, I'd rather invest some time in
> seeking other parts of our changes to the Debian tools to upstream :)

I already stated my gut feeling but then I must say that it depends on
how intrusive the patch ends up being. I have the feeling that it should
be relatively easy to implement this feature as a variation of the current
file based triggers so it might be ok and could be valuable for some
use cases.

That said I can't tell with 100% certainty that we're going to accept the
patch. Currently we don't have clear rules on how to decide what new
features are acceptable or not, and thus I'm not comfortable taking a
decision alone. I usually hope for an ACK of Guillem but when he
doesn't respond we tend to fall in some grey area...

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-dpkg-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110905083651.GA19922@rivendell.home.ouaza.com">h ttp://lists.debian.org/20110905083651.GA19922@rivendell.home.ouaza.com


All times are GMT. The time now is 10:31 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.