Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian User (http://www.linux-archive.org/debian-user/)
-   -   to which package should I file against the allocation bug described below? (http://www.linux-archive.org/debian-user/498936-package-should-i-file-against-allocation-bug-described-below.html)

Enrico Bertocchi 03-08-2011 07:37 PM

to which package should I file against the allocation bug described below?
 
- Debian Squeeze, kernel 2.6.32-5-amd64, used in SOHO like environment
- huge amount of committed memory, e.g. from /proc/meminfo
* Committed_AS:** 369822496 kB
* see Appendix 1 below;
- huge amounts of memory allocated by many processes, e.g. firefox-bin @ 120GB, see Appendix 2 below

- confirmed by huge [anon] lines in pmap, eg.
* 00007fac3a1f7000 8388608K rw---*** [ anon ]
* see Appendix 3 below
- problem pretty widespread amongst the main open graphical applications (evince, gvfs-fuse-daemon, gnome-keyring...)


It's the machine I'm writing from, and everything seems to work unless I change the VM parameter overcommit_memory to 2, which causes any new process to fail its mallocs due to excessive overcommit.

It seems a "base" system library problem, but I'm not so sure...


Thank you for the kind attention

--
Enrico Bertocchi


Appendix 1: /proc/meminfo
"""
MemTotal:******* 8197856 kB
MemFree:********* 968536 kB
Buffers:********** 85280 kB

Cached:********* 6333036 kB
SwapCached:******** 6064 kB
Active:********* 5573300 kB
Inactive:******* 1475164 kB
Active(anon):**** 397408 kB
Inactive(anon):** 242152 kB
Active(file):*** 5175892 kB
Inactive(file):* 1233012 kB

Unevictable:********** 0 kB
Mlocked:************** 0 kB
SwapTotal:****** 7815612 kB
SwapFree:******* 7794940 kB
Dirty:*************** 32 kB
Writeback:************ 0 kB
AnonPages:******* 626572 kB
Mapped:*********** 89428 kB

Shmem:************* 9380 kB
Slab:************* 91504 kB
SReclaimable:***** 73528 kB
SUnreclaim:******* 17976 kB
KernelStack:******* 2352 kB
PageTables:******* 19736 kB
NFS_Unstable:******** 20 kB
Bounce:*************** 0 kB

WritebackTmp:********* 0 kB
CommitLimit:*** 11914540 kB
Committed_AS:** 369824888 kB
VmallocTotal:** 34359738367 kB
VmallocUsed:***** 326092 kB
VmallocChunk:** 34359409148 kB
HardwareCorrupted:**** 0 kB

HugePages_Total:****** 0
HugePages_Free:******* 0
HugePages_Rsvd:******* 0
HugePages_Surp:******* 0
Hugepagesize:****** 2048 kB
DirectMap4k:**** 4222400 kB
DirectMap2M:**** 4165632 kB
"""


Appendix 2: top output
"""
top - 21:20:37 up 6 days, 23:51,* 4 users,* load average: 0.00, 0.00, 0.00
Tasks: 184 total,** 1 running, 183 sleeping,** 0 stopped,** 0 zombie
Cpu(s):* 2.5%us,* 0.3%sy,* 0.0%ni, 97.1%id,* 0.0%wa,* 0.0%hi,* 0.0%si,* 0.0%st

Mem:** 8197856k total,* 7224432k used,** 973424k free,*** 85276k buffers
Swap:* 7815612k total,*** 20672k used,* 7794940k free,* 6332820k cached

* PID USER***** PR* NI* VIRT* RES* SHR S %CPU %MEM*** TIME+* COMMAND******************************************* ******************************

13061 ebertocc* 20** 0* 120g 229m* 35m S*** 5* 2.9* 24:27.88 firefox-bin*********************************************** **********************
12865 ebertocc* 20** 0 32.0g 2312 1668 S*** 0* 0.0** 0:06.27 gvfs-fuse-daemo********************************************* ********************

12748 ebertocc* 20** 0 24.0g 9528 2064 S*** 0* 0.1** 0:00.28 gnome-keyring-d************************************************* ****************
19046 ebertocc* 20** 0 16.4g* 72m* 15m S*** 0* 0.9** 0:08.25 evince******************************************** ******************************

18644 ebertocc* 20** 0 16.3g* 48m* 12m S*** 0* 0.6** 0:05.39 evince******************************************** ******************************
18312 ebertocc* 20** 0 16.3g* 85m* 13m S*** 0* 1.1** 0:11.36 evince******************************************** ******************************

14320 ebertocc* 30* 10 16.1g 2440 2032 S*** 0* 0.0** 0:00.00 hw_server***************************************** ******************************
12890 ebertocc* 20** 0 8804m* 54m* 23m S*** 0* 0.7** 3:09.11 nautilus****************************************** ******************************

19294 ebertocc* 20** 0 8582m* 28m* 15m S*** 0* 0.4** 0:00.90 gedit********************************************* ******************************
12917 ebertocc* 20** 0 8528m* 11m 7452 S*** 0* 0.1** 0:00.10 mixer_applet2************************************* ******************************

12920 ebertocc* 20** 0 8470m 6112 4280 S*** 0* 0.1** 0:00.15 e-calendar-fact********************************************** *******************
13703 ebertocc* 20** 0 8398m* 13m 8240 S*** 0* 0.2** 0:23.47 gnome-terminal****************************************** ************************

19554 ebertocc* 20** 0 8389m* 11m 1276 S*** 0* 0.1** 0:00.00 xchat********************************************* ******************************
19170 ebertocc* 20** 0 8383m* 22m* 12m S*** 1* 0.3** 0:49.65 xchat********************************************* ******************************

12892 ebertocc* 20** 0 8364m 3048 2188 S*** 0* 0.0** 0:00.02 bonobo-activati****************************************** ***********************
12772 ebertocc* 20** 0 8330m 5868 4304 S*** 0* 0.1** 0:00.37 x-session-manag********************************************* ********************

12734 root***** 20** 0 8316m 3428 2448 S*** 0* 0.0** 0:00.01 gdm-session-wor*********************************************** ******************
12884 ebertocc* 20** 0 8310m 3372 2624 S*** 0* 0.0** 0:00.22 gvfs-gdu-volume******************************************** *********************

12679 root***** 20** 0 8273m 2776 2388 S*** 0* 0.0** 0:00.02 gdm-simple-slav********************************************** *******************
12888 ebertocc* 20** 0 8251m 1900 1460 S*** 0* 0.0** 0:01.40 gvfs-afc-volume******************************************** *********************

12673 root***** 20** 0 8250m 2152 1960 S*** 0* 0.0** 0:00.01 gdm3********************************************** ******************************
12882 ebertocc* 20** 0* 359m* 23m* 12m S*** 0* 0.3** 1:30.92 gnome-panel********************************************* ************************

12906 ebertocc* 20** 0* 286m* 23m 8736 S*** 0* 0.3** 0:00.33 python******************************************** ******************************
12900 ebertocc* 20** 0* 267m 8100 5588 S*** 0* 0.1** 0:00.22 nm-applet******************************************** ***************************

12905 ebertocc* 20** 0* 249m 8232 6064 S*** 0* 0.1** 0:00.06 evolution-alarm*******
"""

Appendix 3
"""
13061:** /usr/lib/iceweasel/firefox-bin
0000000000400000**** 76K r-x--* /usr/lib/xulrunner-1.9.1/xulrunner-stub

0000000000613000***** 4K rw---* /usr/lib/xulrunner-1.9.1/xulrunner-stub
0000000000614000***** 4K rw---*** [ anon ]
00007fac3a1f6000***** 4K -----*** [ anon ]
00007fac3a1f7000 8388608K rw---*** [ anon ]
00007fae3a1f7000***** 4K -----*** [ anon ]

00007fae3a1f8000 8388608K rw---*** [ anon ]
00007fb03a1f8000***** 4K -----*** [ anon ]
00007fb03a1f9000 8388608K rw---*** [ anon ]
00007fb23a1f9000***** 4K -----*** [ anon ]
00007fb23a1fa000 8388608K rw---*** [ anon ]

00007fb43a1fa000***** 4K -----*** [ anon ]
00007fb43a1fb000 8388608K rw---*** [ anon ]
00007fb63a1fb000***** 4K -----*** [ anon ]
00007fb63a1fc000 8388608K rw---*** [ anon ]
00007fb83a1fc000***** 4K -----*** [ anon ]

00007fb83a1fd000 8388608K rw---*** [ anon ]
00007fba3a1fd000***** 4K -----*** [ anon ]
00007fba3a1fe000 8388608K rw---*** [ anon ]
00007fbc3a1fe000***** 4K -----*** [ anon ]
00007fbc3a1ff000 8388608K rw---*** [ anon ]

00007fbe3a1ff000***** 4K -----*** [ anon ]
00007fbe3a200000 8389632K rw---*** [ anon ]
00007fc03a3f2000***** 8K r-x--* /usr/lib/libXss.so.1.0.0
00007fc03a3f4000** 2048K -----* /usr/lib/libXss.so.1.0.0
00007fc03a5f4000***** 4K rw---* /usr/lib/libXss.so.1.0.0

00007fc03a5f5000**** 44K r-x--* /usr/lib/xulrunner-1.9.1/components/libnkgnomevfs.so
00007fc03a600000** 2044K -----* /usr/lib/xulrunner-1.9.1/components/libnkgnomevfs.so
00007fc03a7ff000***** 4K rw---* /usr/lib/xulrunner-1.9.1/components/libnkgnomevfs.so

00007fc03a800000** 1024K rw---*** [ anon ]
00007fc03a9fd000***** 4K -----*** [ anon ]
00007fc03a9fe000 8388608K rw---*** [ anon ]
00007fc23a9fe000***** 4K -----*** [ anon ]
00007fc23a9ff000 8388608K rw---*** [ anon ]

00007fc43a9ff000***** 4K -----*** [ anon ]
00007fc43aa00000 8391680K rw---*** [ anon ]
00007fc815c00000** 1024K rw---*** [ anon ]
00007fc815e00000** 1024K rw---*** [ anon ]
00007fc816200000** 1024K rw---*** [ anon ]

00007fc816600000** 1024K rw---*** [ anon ]
00007fc816a00000** 1024K rw---*** [ anon ]
00007fc816c00000** 1024K rw---*** [ anon ]
00007fc817200000** 1024K rw---*** [ anon ]
00007fc817600000** 1024K rw---*** [ anon ]

00007fc817800000** 1024K rw---*** [ anon ]
00007fc817a00000** 1024K rw---*** [ anon ]
00007fc817bfd000**** 12K r-x--* /usr/lib/pango/1.6.0/modules/pango-hangul-fc.so
00007fc817c00000** 2044K -----* /usr/lib/pango/1.6.0/modules/pango-hangul-fc.so

00007fc817dff000***** 4K rw---* /usr/lib/pango/1.6.0/modules/pango-hangul-fc.so
00007fc817e00000** 1024K rw---*** [ anon ]
00007fc817ff3000**** 12K r-x--* /usr/lib/pango/1.6.0/modules/pango-hebrew-fc.so
00007fc817ff6000** 2044K -----* /usr/lib/pango/1.6.0/modules/pango-hebrew-fc.so

00007fc8181f5000***** 4K rw---* /usr/lib/pango/1.6.0/modules/pango-hebrew-fc.so
00007fc8181f6000**** 28K r-x--* /usr/lib/pango/1.6.0/modules/pango-indic-fc.so
00007fc8181fd000** 2048K -----* /usr/lib/pango/1.6.0/modules/pango-indic-fc.so

00007fc8183fd000***** 4K rw---* /usr/lib/pango/1.6.0/modules/pango-indic-fc.so
00007fc8183fe000***** 8K r-x--* /usr/lib/pango/1.6.0/modules/pango-indic-lang.so
00007fc818400000** 2044K -----* /usr/lib/pango/1.6.0/modules/pango-indic-lang.so

00007fc8185ff000***** 4K rw---* /usr/lib/pango/1.6.0/modules/pango-indic-lang.so
00007fc818600000** 1024K rw---*** [ anon ]
00007fc818800000** 1024K rw---*** [ anon ]
00007fc818afd000**** 12K r-x--* /usr/lib/pango/1.6.0/modules/pango-arabic-fc.so

00007fc818b00000** 2044K -----* /usr/lib/pango/1.6.0/modules/pango-arabic-fc.so
00007fc818cff000***** 4K rw---* /usr/lib/pango/1.6.0/modules/pango-arabic-fc.so
00007fc818d00000** 1024K rw---*** [ anon ]
00007fc818f00000** 1024K rw---*** [ anon ]

00007fc819100000** 1024K rw---*** [ anon ]
00007fc819300000** 1024K rw---*** [ anon ]
00007fc819500000** 1024K rw---*** [ anon ]
00007fc8196ff000***** 4K r-x--* /usr/lib/pango/1.6.0/modules/pango-arabic-lang.so

00007fc819700000** 2044K -----* /usr/lib/pango/1.6.0/modules/pango-arabic-lang.so
00007fc8198ff000***** 4K rw---* /usr/lib/pango/1.6.0/modules/pango-arabic-lang.so
00007fc819900000** 1024K rw---*** [ anon ]
00007fc819b00000** 1024K rw---*** [ anon ]

[... many omitted ...]
00007fce4c72b000***** 4K rw---* /lib/ld-2.11.2.so
00007fce4c72c000***** 4K rw---*** [ anon ]
00007fff87a47000*** 232K rw---*** [ stack ]
00007fff87b2c000***** 4K r-x--*** [ anon ]

ffffffffff600000***** 4K r-x--*** [ anon ]
*total******* 134942848K
"""

"Boyd Stephen Smith Jr." 03-08-2011 09:23 PM

to which package should I file against the allocation bug described below?
 
On 2011-03-08 14:37:23 Enrico Bertocchi wrote:
>- Debian Squeeze, kernel 2.6.32-5-amd64, used in SOHO like environment
>- huge amount of committed memory, e.g. from /proc/meminfo
> Committed_AS: 369822496 kB
> see Appendix 1 below;
>- huge amounts of memory allocated by many processes, e.g. firefox-bin @
>120GB, see Appendix 2 below
>- confirmed by huge [anon] lines in pmap, eg.
> 00007fac3a1f7000 8388608K rw--- [ anon ]
> see Appendix 3 below
>- problem pretty widespread amongst the main open graphical applications
>(evince, gvfs-fuse-daemon, gnome-keyring...)

Based on the output you sent, I don't see a bug. Could you please provide
more detail into what you expect vs. what you observe?

Also, you should keep in mind that RAM not is use is RAM wasted. The kernel
will attempt to use all of it so you shouldn't expect much free RAM in a
running system.

Your output indicated a lot of 'Cached' memory. 'Cached' memory is
effectively free, if needed the kernel can quickly repurpose those pages of
memory since they are just cache.

You can't prevent the kernel from growing 'Buffers' and 'Cache'. (Buffers are
another type of almost-free memory.) However, you can encourage the kernel to
always get rid of buffers/cache before swapping by setting vm.swapiness to 0.
I run with it set at 1; there are a number of practical situations where
swapping out some application memory provides significant increase in
throughput over dropping buffer/cache so I don't want to completely forbid the
kernel from doing that.

I agree that the default vm.swapiness of 60 is rather high.

>It's the machine I'm writing from, and everything seems to work unless I
>change the VM parameter overcommit_memory to 2, which causes any new process
>to fail its mallocs due to excessive overcommit.

I run with overcommit_memory set to 2 all the time. I suggest you set it from
/etc/sysctl.conf; the system is less likely be happy with it changing after it
has satisfied some allocations.
--
Boyd Stephen Smith Jr. ,= ,-_-. =.
bss@iguanasuicide.net ((_/)o o(\_))
ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-'
http://iguanasuicide.net/ \_/


All times are GMT. The time now is 10:50 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.