Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian Development (http://www.linux-archive.org/debian-development/)
-   -   Bash-completion with triggers (http://www.linux-archive.org/debian-development/511118-bash-completion-triggers.html)

David Paleino 04-07-2011 09:09 AM

Bash-completion with triggers
 
Hello everybody,
I've implemented a new revision of bash-completion, which uses debtriggers(5)
to load only relevant completions, and symlink them when something
touches /usr/bin/, /usr/games/, /usr/sbin/, /sbin/, /bin/, and so on.

For this to work, the completions have been moved out from /etc/ -- they would
be in /usr/share/bash-completion/completions/, but /etc/bash_completion.d/ is
being kept not to break tons of other packages installing files there.

However, I'm writing to get comments about the triggers issue. When a package
installs an executable in one of the above directories, bash-completion's
postinst removed all symlinks in /etc/bash_completion.d/triggered/, and
re-creates them. The speed of such operation varies greatly depending on the
installed packages -- on my system, it takes about 10s -- but the shell loading
seems much faster too.

Is there any objection to bash-completion using triggers to "watch" the
aforementioned directories?

I have a package ready, and would upload it to experimental before going
to unstable.

Kindly,
David

--
. '`. Debian developer | http://wiki.debian.org/DavidPaleino
: :' : Linuxer #334216 --|-- http://www.hanskalabs.net/
`. `'` GPG: 1392B174 ----|---- http://deb.li/dapal
`- 2BAB C625 4E66 E7B8 450A C3E1 E6AA 9017 1392 B174

David Paleino 04-07-2011 10:36 AM

Bash-completion with triggers
 
Hello Goswin,
re-putting debian-devel in the loop, since I believe you forgot it.

On Thu, 07 Apr 2011 12:20:34 +0200, Goswin von Brederlow wrote:

> David Paleino <dapal@debian.org> writes:
>
> > Hello everybody,
> > I've implemented a new revision of bash-completion, which uses
> > debtriggers(5) to load only relevant completions, and symlink them when
> > something touches /usr/bin/, /usr/games/, /usr/sbin/, /sbin/, /bin/, and so
> > on.
> >
> > For this to work, the completions have been moved out from /etc/ -- they
> > would be in /usr/share/bash-completion/completions/,
> > but /etc/bash_completion.d/ is being kept not to break tons of other
> > packages installing files there.
>
> Shouldn't they be in /var/?

The symlinks created by the trigger?
I'd tend to agree -- but what sub-hierarchy would be best? /var/run/ isn't kept
across reboots, so it's not suitable; /var/cache/ smells a bit (one is supposed
to be able to purge the cache whenever she wants); maybe
/var/lib/bash-completion?

> Both /etc and /usr are writable during package installation so it won't fail.
> But is /usr really the right place for this?

They won't be written to /usr/ -- they will be written to /etc/.

This is mostly because of backwards-compatibility: bash-completion has been
in /etc/ since ages, and users are used to find completions there. I'd like
to avoid changing the layout too much in a single step :). But, at a second
thought, /var/lib/ is really the best choice to me -- so maybe I'll just
document it and do the relevant changes.

Thanks for your input,
David

--
. '`. Debian developer | http://wiki.debian.org/DavidPaleino
: :' : Linuxer #334216 --|-- http://www.hanskalabs.net/
`. `'` GPG: 1392B174 ----|---- http://deb.li/dapal
`- 2BAB C625 4E66 E7B8 450A C3E1 E6AA 9017 1392 B174

Raphael Hertzog 04-07-2011 01:16 PM

Bash-completion with triggers
 
Hi,

On Thu, 07 Apr 2011, David Paleino wrote:
> Hello everybody,
> I've implemented a new revision of bash-completion, which uses debtriggers(5)
> to load only relevant completions, and symlink them when something
> touches /usr/bin/, /usr/games/, /usr/sbin/, /sbin/, /bin/, and so on.
>
> For this to work, the completions have been moved out from /etc/ -- they would
> be in /usr/share/bash-completion/completions/, but /etc/bash_completion.d/ is
> being kept not to break tons of other packages installing files there.
>
> However, I'm writing to get comments about the triggers issue. When a package
> installs an executable in one of the above directories, bash-completion's
> postinst removed all symlinks in /etc/bash_completion.d/triggered/, and
> re-creates them. The speed of such operation varies greatly depending on the
> installed packages -- on my system, it takes about 10s -- but the shell loading
> seems much faster too.
>
> Is there any objection to bash-completion using triggers to "watch" the
> aforementioned directories?

I feel uneasy about this. It means the trigger is going to be activated
for any package installation and all packages are going to be put in
triggers-awaited state.

The trigger is going to happen very often (like the man-db one) and
with a 10s impact it's very noticable...

Even if we change apt to actually keep trigger processing to the end of
the APT run, I fear that the dependency resolution logic is going to
force the trigger to run sooner and more often because a package
in trigger-awaited status does not satisfy the dependencies.

If file triggers could also specify the --no-await option that
dpkg-triggers supports, things might be different.

But in the current state, I think it's a bad idea to use file triggers
in your case.

You'd better use some apt hook to do the task you envision. A file
trigger that is activated for a majority of package installation is
probably better dealt with such a solution.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110407131620.GD2767@rivendell.home.ouaza.com">ht tp://lists.debian.org/20110407131620.GD2767@rivendell.home.ouaza.com

David Paleino 04-07-2011 03:22 PM

Bash-completion with triggers
 
On Thu, 7 Apr 2011 15:16:20 +0200, Raphael Hertzog wrote:

> Hi,
>
> On Thu, 07 Apr 2011, David Paleino wrote:
> > Hello everybody,
> > I've implemented a new revision of bash-completion, which uses
> > debtriggers(5) to load only relevant completions, and symlink them when
> > something touches /usr/bin/, /usr/games/, /usr/sbin/, /sbin/, /bin/, and so
> > on.
> >
> > For this to work, the completions have been moved out from /etc/ -- they
> > would be in /usr/share/bash-completion/completions/,
> > but /etc/bash_completion.d/ is being kept not to break tons of other
> > packages installing files there.
> >
> > However, I'm writing to get comments about the triggers issue. When a
> > package installs an executable in one of the above directories,
> > bash-completion's postinst removed all symlinks
> > in /etc/bash_completion.d/triggered/, and re-creates them. The speed of
> > such operation varies greatly depending on the installed packages -- on my
> > system, it takes about 10s -- but the shell loading seems much faster too.
> >
> > Is there any objection to bash-completion using triggers to "watch" the
> > aforementioned directories?
>
> I feel uneasy about this. It means the trigger is going to be activated
> for any package installation and all packages are going to be put in
> triggers-awaited state.

"Only" those installing executables in these directories:

$ cat debian/triggers
interest /sbin
interest /usr/sbin
interest /usr/local/sbin
interest /bin
interest /usr/bin
interest /usr/local/bin
interest /usr/games
interest /usr/local/games
$

> The trigger is going to happen very often (like the man-db one) and
> with a 10s impact it's very noticable...

That's on my system, heh :)
(for comparison, it's faster than other triggers here. You might want to try
yourself, package at [1] -- beware, lots of messages regarding removed
conffiles)

[1]: http://people.debian.org/~dapal/debs/bash-completion_1.3-2_all.deb

Unfortunately, I have to remove and re-create all the symlinks upon trigger
activation: in fact, only the directory name is being passed (/usr/bin); if
my postinst had passed the file name activating the trigger (/usr/bin/foo); it
would really be much faster.

> Even if we change apt to actually keep trigger processing to the end of
> the APT run, I fear that the dependency resolution logic is going to
> force the trigger to run sooner and more often because a package
> in trigger-awaited status does not satisfy the dependencies.
>
> If file triggers could also specify the --no-await option that
> dpkg-triggers supports, things might be different.

That would be great; ideas how to do it?

> But in the current state, I think it's a bad idea to use file triggers
> in your case.

A named trigger isn't better in any case, I believe.

> You'd better use some apt hook to do the task you envision. A file
> trigger that is activated for a majority of package installation is
> probably better dealt with such a solution.

Which hook would you suggest?

Reading apt.conf(5), I find: Dpkg::Pre-Invoke, Dpkg::Post-Invoke and
Dpkg::Pre-Install-Pkgs. I think the first two are run at each unpacking (thus
are unsuitable to me, it would be much slower than now), and the latter is
_before_ installing packages.
Why there's no Dpkg::Post-Install-Pkgs? :)


I believe that passing --no-await for file triggers, or letting the maintainer
specify that in debian/triggers, would be the best solution.

Kindly,
David

--
. '`. Debian developer | http://wiki.debian.org/DavidPaleino
: :' : Linuxer #334216 --|-- http://www.hanskalabs.net/
`. `'` GPG: 1392B174 ----|---- http://deb.li/dapal
`- 2BAB C625 4E66 E7B8 450A C3E1 E6AA 9017 1392 B174

Goswin von Brederlow 04-07-2011 04:15 PM

Bash-completion with triggers
 
David Paleino <dapal@debian.org> writes:

> On Thu, 7 Apr 2011 15:16:20 +0200, Raphael Hertzog wrote:
>
>> You'd better use some apt hook to do the task you envision. A file
>> trigger that is activated for a majority of package installation is
>> probably better dealt with such a solution.
>
> Which hook would you suggest?
>
> Reading apt.conf(5), I find: Dpkg::Pre-Invoke, Dpkg::Post-Invoke and
> Dpkg::Pre-Install-Pkgs. I think the first two are run at each unpacking (thus
> are unsuitable to me, it would be much slower than now), and the latter is
> _before_ installing packages.
> Why there's no Dpkg::Post-Install-Pkgs? :)

Dpkg::Post-Invoke would be the right (best available) one. That would
call your trigger after every dpkg invocation, what Raphael fears will
hapen with our trigger anyway but without the trigger-waiting state.

But those hooks would only wotk for apt/aptitude. Not for invoking dpkg
or cupt or others, which I think rules them out anyway.

MfG
Goswin


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 8739luqbpl.fsf@frosties.localnet">http://lists.debian.org/8739luqbpl.fsf@frosties.localnet

"Eugene V. Lyubimkin" 04-07-2011 07:14 PM

Bash-completion with triggers
 
On 2011-04-07 18:15, Goswin von Brederlow wrote:
> Dpkg::Post-Invoke would be the right (best available) one. That would
> call your trigger after every dpkg invocation [...]

This is not true, 'Dpkg::*-Invoke' script chain are called once
before/after all dpkg invocations.

> But those hooks would only wotk for apt/aptitude. Not for [...] cupt

This is not true as well.

--
Eugene V. Lyubimkin aka JackYF, JID: jackyf.devel(maildog)gmail.com
C++/Perl developer, Debian Developer


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110407191431.GA5613@r500-debian">http://lists.debian.org/20110407191431.GA5613@r500-debian

Raphael Hertzog 04-07-2011 07:17 PM

Bash-completion with triggers
 
On Thu, 07 Apr 2011, David Paleino wrote:
> "Only" those installing executables in these directories:

I know, I do not have stats but I expect this to be a very important
percentage of packages.

> Unfortunately, I have to remove and re-create all the symlinks upon trigger
> activation: in fact, only the directory name is being passed (/usr/bin); if
> my postinst had passed the file name activating the trigger (/usr/bin/foo); it
> would really be much faster.

I know the limitations of triggers. :-)

Still, you can do better than recreating everything each time. Like
scanning the non-installed completion files, extract some metadata
listing which program's completion this file provide, and verify if
you can find the binary before deciding to install it.

Removing completion for uninstalled packages is not important to have in
real-time, you can do it in a weekly cron or something like this.

> > If file triggers could also specify the --no-await option that
> > dpkg-triggers supports, things might be different.
>
> That would be great; ideas how to do it?

Extend the syntax of the triggers file and patch dpkg to support it.

> > But in the current state, I think it's a bad idea to use file triggers
> > in your case.
>
> A named trigger isn't better in any case, I believe.

Indeed, they are of no use.

> > You'd better use some apt hook to do the task you envision. A file
> > trigger that is activated for a majority of package installation is
> > probably better dealt with such a solution.
>
> Which hook would you suggest?
>
> Reading apt.conf(5), I find: Dpkg::Pre-Invoke, Dpkg::Post-Invoke and
> Dpkg::Pre-Install-Pkgs. I think the first two are run at each unpacking (thus
> are unsuitable to me, it would be much slower than now), and the latter is
> _before_ installing packages.
> Why there's no Dpkg::Post-Install-Pkgs? :)

Pre/Post-Invoke are called (as their name suggest it) before and after
dpkg has been called no matter the operation requested (unpack,
configure).

I was indeed thinking there was some sort of hook that would run after apt
had completed its work but it's not the case (you could still ask APT
developers if they can do it).

Still I think using Dpkg::Post-Invoke with an update-bash-completion
program intelligent enough to not do useless work is preferable over
using a file trigger given the current limitations of triggers.

File triggers are useful to replace calls to update-foo in postinst
but in your case the data is already centralized in a package and you
have the somewhat oppposite problem of knowing which of those files are
useful.

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110407191733.GD6396@rivendell.home.ouaza.com">ht tp://lists.debian.org/20110407191733.GD6396@rivendell.home.ouaza.com

David Paleino 04-07-2011 07:19 PM

Bash-completion with triggers
 
On Thu, 7 Apr 2011 22:14:31 +0300, Eugene V. Lyubimkin wrote:

> On 2011-04-07 18:15, Goswin von Brederlow wrote:
> > Dpkg::Post-Invoke would be the right (best available) one. That would
> > call your trigger after every dpkg invocation [...]
>
> This is not true, 'Dpkg::*-Invoke' script chain are called once
> before/after all dpkg invocations.

So it's just like a trigger monitoring /? (without the implications of triggers
in terms of sequence of operations)

This is meaningless to me. This way, my postinst would be run at *every* package
installation, not only those installing files in /usr/bin/ and companions.

Kindly,
David

--
. '`. Debian developer | http://wiki.debian.org/DavidPaleino
: :' : Linuxer #334216 --|-- http://www.hanskalabs.net/
`. `'` GPG: 1392B174 ----|---- http://deb.li/dapal
`- 2BAB C625 4E66 E7B8 450A C3E1 E6AA 9017 1392 B174

"Eugene V. Lyubimkin" 04-07-2011 07:27 PM

Bash-completion with triggers
 
On 2011-04-07 21:19, David Paleino wrote:
> > This is not true, 'Dpkg::*-Invoke' script chain are called once
> > before/after all dpkg invocations.
>
> So it's just like a trigger monitoring /? (without the implications of triggers
> in terms of sequence of operations)

No, I phrased it badly probably. Let me try again:

Dpkg::Pre-Invoke are called once. Then all dpkg invocations are called.
The Dpkg::Post-Invoke is called.

--
Eugene V. Lyubimkin aka JackYF, JID: jackyf.devel(maildog)gmail.com
C++/Perl developer, Debian Developer


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110407192730.GC5613@r500-debian">http://lists.debian.org/20110407192730.GC5613@r500-debian

Raphael Hertzog 04-07-2011 07:28 PM

Bash-completion with triggers
 
On Thu, 07 Apr 2011, Eugene V. Lyubimkin wrote:
> On 2011-04-07 18:15, Goswin von Brederlow wrote:
> > Dpkg::Post-Invoke would be the right (best available) one. That would
> > call your trigger after every dpkg invocation [...]
>
> This is not true, 'Dpkg::*-Invoke' script chain are called once
> before/after all dpkg invocations.

Ah, good to know. I thought the same than Goswin and did not verify it.

I guess the existence of Dpkg::Pre-Install-Pkgs led me to believe that
the difference with Dpkg::Pre-Invoke would be the number of time it's run
when in fact it's only the data passed to the program that changes...
since both are called at the same place in the code.

Thanks for correcting our mistakes!

Cheers,
--
Raphaël Hertzog ◈ Debian Developer

Follow my Debian News ▶ http://RaphaelHertzog.com (English)
▶ http://RaphaelHertzog.fr (Français)


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110407192810.GE6396@rivendell.home.ouaza.com">ht tp://lists.debian.org/20110407192810.GE6396@rivendell.home.ouaza.com


All times are GMT. The time now is 06:26 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.