FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Fedora Directory

 
 
LinkBack Thread Tools
 
Old 04-01-2010, 02:31 PM
"jim@scusting.com"
 
Default Cache tuning errors

Hi - I had a read of the Redhat documentation on cache sizes and tuning
and tried tweaking one of my servers as it was setup with the default.
The server is a Fedora 10 box with 4GB of RAM so I decided to setup the
cache as:

nsslapd-cachememsize = 3145728000 (3GB)
nsslapd-dbcachesize = 3670016000 (3.5GB)

But when I restart I get the message:
[01/Apr/2010:13:56:30 +0100] - WARNING -- Possible CONFIGURATION ERROR
-- cachesize (2072199168) may be configured to use more than the
available physical memory.
[01/Apr/2010:13:56:30 +0100] - WARNING---Likely CONFIGURATION
ERROR---dbcachesize is configured to use more than the available
physical memory, decreased to the largest available size (2072199168 bytes).
[01/Apr/2010:13:56:30 +0100] - I'm resizing my cache now...cache was
2072203264 and is now 2072199168

Is there a limit of 2GB on the cache sizes? The server has 4GB memory..

I then put the settings down lower to:
nsslapd-cachememsize = 1153433600 (1.1 GB)
nsslapd-dbcachesize = 1677721600 (1.6 GB)

I then hit it with some big queries to load up the cache and now I get:
[01/Apr/2010:14:34:15 +0100] - libdb: txn_checkpoint: failed to flush
the buffer cache: Cannot allocate memory
[01/Apr/2010:14:34:15 +0100] - Serious Error---Failed to checkpoint
database, err=12 (Cannot allocate memory)
[01/Apr/2010:14:34:15 +0100] - libdb: malloc: 3145764: Cannot allocate
memory
[01/Apr/2010:14:34:15 +0100] - libdb: txn_checkpoint: failed to flush
the buffer cache: Cannot allocate memory
[01/Apr/2010:14:34:15 +0100] memory allocator - calloc of 8194 elems of
4 bytes failed; OS error 12 (Cannot allocate memory)
The server has probably allocated all available virtual memory.

When that happened top was reporting the slapd process only using %40
memory and there is nothing else running on this server.

I was trying to do some tuning as these servers are worked quite hard
and the "Entry Cache hits" was showing as 0 - I would of expected some
hits even with a smallish cache?

Thanks.

Jim.






--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 04-01-2010, 02:46 PM
Rich Megginson
 
Default Cache tuning errors

jim@scusting.com wrote:
> Hi - I had a read of the Redhat documentation on cache sizes and tuning
> and tried tweaking one of my servers as it was setup with the default.
> The server is a Fedora 10 box with 4GB of RAM so I decided to setup the
> cache as:
>
> nsslapd-cachememsize = 3145728000 (3GB)
> nsslapd-dbcachesize = 3670016000 (3.5GB)
>
> But when I restart I get the message:
> [01/Apr/2010:13:56:30 +0100] - WARNING -- Possible CONFIGURATION ERROR
> -- cachesize (2072199168) may be configured to use more than the
> available physical memory.
> [01/Apr/2010:13:56:30 +0100] - WARNING---Likely CONFIGURATION
> ERROR---dbcachesize is configured to use more than the available
> physical memory, decreased to the largest available size (2072199168 bytes).
> [01/Apr/2010:13:56:30 +0100] - I'm resizing my cache now...cache was
> 2072203264 and is now 2072199168
>
> Is there a limit of 2GB on the cache sizes? The server has 4GB memory..
>
See the documentation for your kernel. Even though you have 4GB of
memory, on a 32-bit system, the kernel cannot allocate all of it for
user processes.

How big is your database? What is the size of your id2entry.db4 file?

You usually want to have your nsslapd-cachememsize large enough to hold
your database in memory, if possible. That gives the most boost to
performance. You probably want to leave nsslapd-dbcachesize at the
default size - it does not impact performance nearly as much.
> I then put the settings down lower to:
> nsslapd-cachememsize = 1153433600 (1.1 GB)
> nsslapd-dbcachesize = 1677721600 (1.6 GB)
>
> I then hit it with some big queries to load up the cache and now I get:
> [01/Apr/2010:14:34:15 +0100] - libdb: txn_checkpoint: failed to flush
> the buffer cache: Cannot allocate memory
> [01/Apr/2010:14:34:15 +0100] - Serious Error---Failed to checkpoint
> database, err=12 (Cannot allocate memory)
> [01/Apr/2010:14:34:15 +0100] - libdb: malloc: 3145764: Cannot allocate
> memory
> [01/Apr/2010:14:34:15 +0100] - libdb: txn_checkpoint: failed to flush
> the buffer cache: Cannot allocate memory
> [01/Apr/2010:14:34:15 +0100] memory allocator - calloc of 8194 elems of
> 4 bytes failed; OS error 12 (Cannot allocate memory)
> The server has probably allocated all available virtual memory.
>
> When that happened top was reporting the slapd process only using %40
> memory and there is nothing else running on this server.
>
> I was trying to do some tuning as these servers are worked quite hard
> and the "Entry Cache hits" was showing as 0 - I would of expected some
> hits even with a smallish cache?
>
Let's first figure out how to size your cache appropriately.

The cache starts out empty, so until it becomes populated with entries,
your cache hits will be 0.
> Thanks.
>
> Jim.
>
>
>
>
>
>
> --
> 389 users mailing list
> 389-users@lists.fedoraproject.org
> https://admin.fedoraproject.org/mailman/listinfo/389-users
>

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 04-01-2010, 02:47 PM
Roberto Polli
 
Default Cache tuning errors

On Thursday 01 April 2010 16:31:17 jim@scusting.com wrote:
>... more than the available
> physical memory, decreased to the largest available size (2072199168
> bytes).
on 64bit or 32bit?

Peace,
R.
--

Roberto Polli
Babel S.r.l. - http://www.babel.it
Tel. +39.06.91801075 - fax +39.06.91612446
Tel. cel +39.340.6522736
P.zza S.Benedetto da Norcia, 33 - 00040 Pomezia (Roma)

"Il seguente messaggio contiene informazioni riservate. Qualora questo
messaggio fosse da Voi ricevuto per errore, Vogliate cortesemente darcene
notizia a mezzo e-mail. Vi sollecitiamo altresì a distruggere il messaggio
erroneamente ricevuto. Quanto precede Vi viene chiesto ai fini del rispetto
della legge in materia di protezione dei dati personali."
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 04-01-2010, 02:56 PM
"jim@scusting.com"
 
Default Cache tuning errors

> See the documentation for your kernel. Even though you have 4GB of
> memory, on a 32-bit system, the kernel cannot allocate all of it for
> user processes.
>
> How big is your database? What is the size of your id2entry.db4 file?
>
> You usually want to have your nsslapd-cachememsize large enough to hold
> your database in memory, if possible. That gives the most boost to
> performance. You probably want to leave nsslapd-dbcachesize at the
> default size - it does not impact performance nearly as much.
>
du -h id2entry.db4
2.3G id2entry.db4

OK - so I need the cachememsize to be > 2.3G - I'll have a read about
kernel limitations then - I had thought 32bit = 4Gb.

Thanks.

Jim.
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 04-01-2010, 02:58 PM
"jim@scusting.com"
 
Default Cache tuning errors

Roberto Polli wrote:
> On Thursday 01 April 2010 16:31:17 jim@scusting.com wrote:
>
>> ... more than the available
>> physical memory, decreased to the largest available size (2072199168
>> bytes).
>>
> on 64bit or 32bit?
>
> Peace,
> R.
>
Fedora 10 - 32bit: Linux 2.6.27.25-170.2.72.fc10.i686.PAE #1 SMP Sun
Jun 21 18:51:33 EDT 2009 i686 i686 i386 GNU/Linux

Jim.

--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 
Old 04-01-2010, 03:56 PM
Roberto Polli
 
Default Cache tuning errors

On Thursday 01 April 2010 16:58:18 jim@scusting.com wrote:
> > on 64bit or 32bit?
> Fedora 10 - 32bit: Linux 2.6.27.25-170.2.72.fc10.i686.PAE #1 SMP Sun
> Jun 21 18:51:33 EDT 2009 i686 i686 i386 GNU/Linux
so that's it.. see rich answer. on 64bit I got no issues.

Peace,
R.

--

Roberto Polli
Babel S.r.l. - http://www.babel.it
Tel. +39.06.91801075 - fax +39.06.91612446
Tel. cel +39.340.6522736
P.zza S.Benedetto da Norcia, 33 - 00040 Pomezia (Roma)

"Il seguente messaggio contiene informazioni riservate. Qualora questo
messaggio fosse da Voi ricevuto per errore, Vogliate cortesemente darcene
notizia a mezzo e-mail. Vi sollecitiamo altresì a distruggere il messaggio
erroneamente ricevuto. Quanto precede Vi viene chiesto ai fini del rispetto
della legge in materia di protezione dei dati personali."
--
389 users mailing list
389-users@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/389-users
 

Thread Tools




All times are GMT. The time now is 03:24 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org