"Write once only but read many" filesystem
 

For reasons of auditability/accountability, I would like a filesystem
such that I can write to it only ONCE, subsequently not
modifiable/deletable, but always readable. Kind of a database journal
logs - it is continuously being written, sequentiall appending, but not
circular buffer based, so that upon running out of space, logging will
be paused in memory, and after new storage devices added to it, it will
continue to flush out whatever is outstanding in memory.


Can ext3 / ext4 or current jbd2 be easily configured to serve this purpose?

Thanks.



_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

"Write once only but read many" filesystem
 

Thank you for your reply :-).

On Sat, Mar 22, 2008 at 11:06 PM, Jörn Engel <joern@logfs.org> wrote:
> On Sat, 22 March 2008 22:52:12 +0800, Peter Teoh wrote:
> >
> > what are the difference in terms of final features provided by these
> > two different filesystem? what is this "garbage collection"? u
> > still have features like creating different directories, and creating
> > different files, and writing the files? How about setting the file
> > attributes...it should be set before writing right (so that after
> > writing and handle is closed it becomes permanently not
> > modifiable)..but creating a subdirectory below the current dir should
> > be possible right (even after closing the previous directory)?
>
> Your requirements aren't quite clear to me. Do you want the complete
> filesystem to be read-only after being written once?

YES....

> Or do you want individual files/directories to be immutable - chattr?

chattr is not good enough, as root can still modify it. So if
current feature is not there, then some small development may be
needed.

> And in either case, what problem do you want to solve with a read-only filesystem?

Simple: i want to record down everything that a user does, or a
database does, or any applications running - just record down its
state permanently securely into the filesystem, knowing that for sure,
there is not way to modify the data, short of recreating the
filesystem again. Sound logical? Or is there any loophole in this
concept?

In summary, are there any strong demand for such a concept/filesystem?
I may take the plunge to implementing it, if justfiable and
everybody is interested..:-)...

--
Regards,
Peter Teoh

_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

"Write once only but read many" filesystem
 

Jörn Engel wrote:

On Sat, 22 March 2008 23:55:53 +0800, Peter Teoh wrote:

Or do you want individual files/directories to be immutable - chattr?

chattr is not good enough, as root can still modify it. So if
current feature is not there, then some small development may be
needed.


And in either case, what problem do you want to solve with a read-only filesystem?

Simple: i want to record down everything that a user does, or a
database does, or any applications running - just record down its
state permanently securely into the filesystem, knowing that for sure,
there is not way to modify the data, short of recreating the
filesystem again. Sound logical? Or is there any loophole in this
concept?


The loophole is called root. In a normal setup, root can do anything,
including writing directly to the device your filesystem resides in,
writing to kernel memory, etc.

It may be rather inconvenient to change a filesystem by writing to the
block device, but far from impossible. If you want to make such changes
impossible, you are facing an uphill battle that I personally don't care
about. And if inconvenience is good enough, wouldn't chattr be
sufficiently inconvenient?

Jörn



How about mounting an isofs via loopback? This has the added benefit of
being ready to be exported to disc. You can make it with mkisofs on a
directory structure and mount it to the tree with a normal mount(1). If
it asks for fs type on mount, I think its 'iso9660'.


_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users

"Write once only but read many" filesystem
 

Scott Lovenberg wrote:
Jörn
Engel wrote:

On Sat, 22 March 2008 23:55:53 +0800, Peter
Teoh wrote:



* Or do you want individual
files/directories to be immutable - chattr?



chattr is not good enough, as root can still modify it.** So if


current feature is not there, then some small development may be


needed.




*And in either case, what problem do you
want to solve with a read-only filesystem?



Simple:** i want to record down everything that a user does, or a


database does, or any applications running - just record down its


state permanently securely into the filesystem, knowing that for sure,


there is not way to modify the data, short of recreating the


filesystem again.*** Sound logical?** Or is there any loophole in this


concept?





The loophole is called root.* In a normal setup, root can do anything,


including writing directly to the device your filesystem resides in,


writing to kernel memory, etc.




It may be rather inconvenient to change a filesystem by writing to the


block device, but far from impossible.* If you want to make such
changes


impossible, you are facing an uphill battle that I personally don't
care


about.* And if inconvenience is good enough, wouldn't chattr be


sufficiently inconvenient?




Jörn







How about mounting an isofs via loopback?* This has the added benefit
of being ready to be exported to disc.* You can make it with mkisofs on
a directory structure and mount it to the tree with a normal mount(1).*
If it asks for fs type on mount, I think its 'iso9660'.






Thanks for
the idea.** Based on this idea, I will start looking at the
implementation of isofs, and how is it made to be readonly.......my
ultimate aim is to make the filesystem readonly upon after being
written, and the file closed.** Not sure if it can be done, but I
envisaged a lot of audit journalling are of these nature.** Of course,
it is always possible to "dd" the filesystem to modify the content, but
then if given design into its protection mechanism (like incremental
checksum - current checksum based on previous checksum, generated and
stored together with the file, upon after every writing of data) we can
always protect its integrity.** Aim is to set it to
readonly......anyone can read....but not modifiable.** As a start I
will try out patching ext2, hoping that it is much simpler than
ext3/ext4.



Upon changes to its content (via dd) it will invalidate the immediate
future incremental
checksum.** Similarly, if u patch the current checksum (which depends
on a hash function of previous data, and previous checksum), it will
not be
valid, as the current checksum is also dependent on the history of
previous checksum.** So everytime u change the content via dd, u will
need to modify the next checksum, which is calculated based on this
data, and the current checksum, which again provide the seed for the
next checksum etc.** U will have to modify a lot of data, unless u are
near the tip of the latest modification.** The smaller the chunk of
data per checksum, the more difficult to keep up with the rate of
modification.** Tradeoff is more work for CPU.



The userspace tool part will then always validate the checksum with the
data being read, if modified, checksum will not be valid.** Since it is
a hash function, given the modified data,
and the previous checksum, it is not possible calculate the current
checksum.



Of course, if the intruder is root, then it is as good as not having
all these complex calculations, so our assumption is that the machine
is not compromised yet.



Then of course the "chattr" can be used as well - if it is not
compromised.** True, but the possibility that it can be modified
infinitely via chattr also exists - which is what write-once-read-many
is against - to provide the assurance that it has not been overwritten
the second time (possible technically, but very very difficult).



An equivalent requirements of such a filesystem will be:** a filesystem
such that upon every changes made, a log of dates of changes are
made.***** Overhead for these feature is high, so a lightweighted
version will be the date/history only it is first written.



On an existing ext2 filesystem, once the '"worm" kernel module is
loaded, the feature immediately take effect - content becomes read-only
but not modifiable, but modifiable for new contents.** And old contents
may or may not have a checksum to protect it, but if the checksum
exists, it will come from the last time "worm" was running and
generating checksum.



So u have the best of both world - ext2/3/4 with/without worm.*** If
worm-enabled, it may also be configured at the directory level - some
directory can be solely dedicated for worm-journalling.



What do u guys think - any conceptual errors?















_______________________________________________
Ext3-users mailing list
Ext3-users@redhat.com
https://www.redhat.com/mailman/listinfo/ext3-users