kernel notification of file system changes
 

Hi,

I'm looking for some kernel-based notification of changes to my file
system. I've been looking at inotify, but it's not exactly what I want.

Basically I want to know if _any_ write occurs anywhere. I don't want
to register a whole bunch of files to watch, I just want to watch an
entire mount. When a file is changed (ie. a write operation occurs), I
then want to add that file or fd to a list in RAM. That's all.

I know this may be a lot of data, considering streams and devices, but I
can filter out /dev, /proc, etc. and just focus on "real" files.

Is there anything that can do this?

thanks :)
--
Iain Buchanan <iaindb at netspace dot net dot au>

I hate it when my foot falls asleep during the day cause that means
it's going to be up all night.
-- Steven Wright

kernel notification of file system changes
 

Can the older dnotify do what you want? - it monitors files differently
to inotify. There is also gammin/fam.

BillK


On Wed, 2010-05-05 at 16:24 +0930, Iain Buchanan wrote:
> Hi,
>
> I'm looking for some kernel-based notification of changes to my file
> system. I've been looking at inotify, but it's not exactly what I want.
>
> Basically I want to know if _any_ write occurs anywhere. I don't want
> to register a whole bunch of files to watch, I just want to watch an
> entire mount. When a file is changed (ie. a write operation occurs), I
> then want to add that file or fd to a list in RAM. That's all.
>
> I know this may be a lot of data, considering streams and devices, but I
> can filter out /dev, /proc, etc. and just focus on "real" files.
>
> Is there anything that can do this?
>
> thanks :)

kernel notification of file system changes
 

On 5 May 2010, at 07:54, Iain Buchanan wrote:

...
I'm looking for some kernel-based notification of changes to my file
system. I've been looking at inotify, but it's not exactly what I
want.


Basically I want to know if _any_ write occurs anywhere. I don't want
to register a whole bunch of files to watch, I just want to watch an
entire mount.


man inotify(7):
... When a directory is monitored, inotify will return events for the
directory itself, and for files inside the directory.



Am I missing something?


This article was posted to a different froup recently:
http://www.ibm.com/developerworks/linux/library/l-inotify/index.html
It looks interesting.


Stroller.

kernel notification of file system changes
 

On 5 May, Iain Buchanan wrote:
> Hi,
>
> I'm looking for some kernel-based notification of changes to my file
> system. I've been looking at inotify, but it's not exactly what I want.
>
> Basically I want to know if _any_ write occurs anywhere. I don't want
> to register a whole bunch of files to watch, I just want to watch an
> entire mount. When a file is changed (ie. a write operation occurs), I
> then want to add that file or fd to a list in RAM. That's all.
>
> I know this may be a lot of data, considering streams and devices, but I
> can filter out /dev, /proc, etc. and just focus on "real" files.
>
> Is there anything that can do this?
>

Might be I've just asked a similar question on the ZSH mailing list.
Please have a look at inotifywatch from the sys-fs/inotify-tools
package. It can watch a directory tree recursively.

I hope this helps,
Helmut.

kernel notification of file system changes
 

Am 05.05.2010 15:34, schrieb Stroller:
>
> On 5 May 2010, at 07:54, Iain Buchanan wrote:
>> ...
>> I'm looking for some kernel-based notification of changes to my file
>> system. I've been looking at inotify, but it's not exactly what I want.
>>
>> Basically I want to know if _any_ write occurs anywhere. I don't want
>> to register a whole bunch of files to watch, I just want to watch an
>> entire mount.
>
> man inotify(7):
> ... When a directory is monitored, inotify will return events for the
> directory itself, and for files inside the directory.
>
>
> Am I missing something?
>
>
> This article was posted to a different froup recently:
> http://www.ibm.com/developerworks/linux/library/l-inotify/index.html
> It looks interesting.
>
>
> Stroller.
>

To repeat my comment on Iain's original "backup to a cold-swap drive"
thread, Inotify has two drawbacks which make it hard or even impossible
to use for Iain's use case:

a) It does not work recursively which means that you have to create a
new handle for each subdirectory. Of course, this only means more work
for the programmer but there is also the problem that

b) As far as I know, Inotify does not scale very good, at least not good
enough to monitor a whole system. /proc/sys/fs/inotify/max_user_watches
is 65535 on my system.

On the other hand, I've never tried to increase that limit and just let
it run on a deep directory structure. Who knows, maybe it actually works.

Hope this helps,
Florian Philipp

kernel notification of file system changes
 

On 5 May 2010, at 18:24, Florian Philipp wrote:

...
man inotify(7):
... When a directory is monitored, inotify will return events for the
directory itself, and for files inside the directory.
...


To repeat my comment on Iain's original "backup to a cold-swap drive"
thread ...


Sorry, I started ignoring that almost immediately it was posted. He
rejected too quickly too many workable solutions to basically
functional backup. Perhaps Iain is a perfectionist, but I did not wish
to follow the thread.



... Inotify has two drawbacks which make it hard or even impossible
to use for Iain's use case:

a) It does not work recursively which means that you have to create a
new handle for each subdirectory. Of course, this only means more work
for the programmer but there is also the problem that


Pardon me. I assumed that "files inside the directory" meant that foo
would be be changed when foo/bar changed, thus monitoring grunt would
reflect changes in grunt/foo/bar. I overlooked that a directory is not
a file.


Stroller.

kernel notification of file system changes
 

On Wed, 2010-05-05 at 17:02 +0200, Helmut Jarausch wrote:

> Might be I've just asked a similar question on the ZSH mailing list.
> Please have a look at inotifywatch from the sys-fs/inotify-tools
> package. It can watch a directory tree recursively.

it does look interesting, thanks. I would still run into the directory
limit if I wanted to watch something large like /

$ sudo find / -xdev -type d | wc -l
71168

"...The default maximum is 8192; it can be increased by writing
to /proc/sys/fs/inotify/max_user_watches."

but it's an angle to follow. I wonder how max_user_watches would handle
being 100k or more... no doubt you just need some RAM?!

thanks,
--
Iain Buchanan <iaindb at netspace dot net dot au>

<drobbins> we should send him a commemorative gentoo crack pipe for all
his contributions to this project

kernel notification of file system changes
 

On Donnerstag 06 Mai 2010, Iain Buchanan wrote:
> On Wed, 2010-05-05 at 15:12 +0800, Bill Kenworthy wrote:
> > Can the older dnotify do what you want? - it monitors files differently
> > to inotify. There is also gammin/fam.
>
> dnotify locks the files or directories you want to watch, so it would
> prevent external media from being unmounted. dnotify also uses a file
> descripter per watched item, which could get interesting for large
> amounts of watches!
>
> I'm not sure about FAM, but Gamin uses inotify or dnotify anyway (in
> Linux). I think they're all designed in a similar way: you have to
> register a whole bunch of files or directories to watch.
>
> thanks,

oh god... fam... that crap caused me so much pain over the years. This bug
ridden zombie is still around?