This isn't for a web server although we might apply the same approach to that if this speeds things up a lot.
I was going to store the cached copy on a RAM disk as many of the files
are larger than the 1MB limit of memlockd and I don't feel like coming
up with my own solution if I can avoid it.
Is there a way to know how much RAM is being used for file cache or to tell it to use more? If the server has 128GB of RAM and typically uses half of that for it's actual work will it use the rest as file cache? Likewise is there a way to track/test if file stats are being pushed out of cache a lot?
We've been considering switching to SSD or RAM drives but it seems they'd always be slower than system RAM and we haven't found a product that can affordably store sufficient data. I couldn't find a product that just sits between the disk and controller, or a controller that does this itself, and adds a large RAM-based file cache either.
On Tue, Mar 23, 2010 at 12:52 PM, David Schwartz <firstname.lastname@example.org> wrote:
Michael McGlothlin wrote:
> I've been asked to cache some high traffic files on one of our server.
> Is there an easy way to get ext3/ext4 filesystems to cache several GB
> of files in memory at once? I'd like writes to happen normally but reads
> to happen from RAM. (We have plenty of RAM so that isn't an issue.)
> If that isn't possible I can cache the files myself. Does the filesystem
> keep a cache in memory of the file attributes such as modification time?
> So if I check for a change will the disk actually have to physically move
> to check the mod time?
I would first investigate whether your web server has some specific way to
do this. Failing that, I strongly recommend just letting the disk cache do
its job. If they really are frequently-accessed, they will stay in cache if
sufficient RAM is available anyway. I would only suggest going further if
you have specific latency requirements.
If you do, I'd recommend simply using a separate program to map the files
and then lock the pages in memory. The 'memlockd' program can do this. I'm
not sure how well it handles file changes, but it shouldn't be difficult to
modify it to restart if any file changes.
The other possibility is to put the files on a ramdisk. You can use a
scheduled script to update them from an on-disk copy if needed.
Linux has good stat caching, so the need to move the disk to check the
modification time will only occur if that information was pushed out of
Ext3-users mailing list
Ext3-users mailing list
On Tue, Mar 23, 2010 at 01:16:57PM -0600, Michael McGlothlin wrote:
> I was going to store the cached copy on a RAM disk as many of the files are
Note that it probably won't help you that much, the kernel usually does
quite good job at caching reads (but, specific situations like extensive
writes can starve it - which ramdisk solution would avoid as it is
completely manually controlled. Only way to know it is to test it)
> larger than the 1MB limit of memlockd and I don't feel like coming up with
> my own solution if I can avoid it.
> Is there a way to know how much RAM is being used for file cache or to tell
free(1) will tell you (look at "cached" column). Note that programs you
"load" from disk are also actually executed directly from that page cache
(without making any separate copy). Also, the writes (unless being done with
O_DIRECT or such) will go to that same cache before they're flushed to disk
(which is usually what you want, as subsequent reads can then be satisfied
from cache, and the application issuing writes gets control much sooner than
if it would wait for writes to complete to disk).
> it to use more? If the server has 128GB of RAM and typically uses half of
> that for it's actual work will it use the rest as file cache? Likewise is
Yes, it will use (almost) ALL otherwise unused RAM (the "free" column in
free(1) means "unused" or "wasted" if you like) for cache. The "almost" is
because there is some very small amount reserved by /proc/sys/vm/min_free_kbytes
(but you don't want to touch it, it is too small to give you any benefit, and
your kernel might die if you set it too low).
> there a way to track/test if file stats are being pushed out of cache a lot?
Uh, dunno for something elegant. You can track /proc/meminfo and
/proc/slabinfo (for example by free(1) and slabtop(1) or manually of course)
and look how they change.
Note that there are other users of memory (see "slabtop -s c"). For example,
especially if you have lots of small files and/or directories, then dentry,
inode and related fs cache structures can eat significant amounts of RAM.
You can tune priority of expiration of those with /proc/sys/vm/vfs_cache_pressure
(you might also want to look in your kernel docs for rest of /proc/sys/vm)
There are also vmstat(8), iostat(1) and blktrace(8) which might help you
track actual I/O, and you can compare that with actual data read from your
program logs for example to see how well the cache performs.
Opinions above are GNU-copylefted.
Ext3-users mailing list
|All times are GMT. The time now is 02:21 PM.|
VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.