FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 05-25-2011, 02:20 PM
Alex Schuster
 
Default Swap performance

Hi there!

I still wonder why my KDE4 system starts swapping so early. Until a week
ago, I had 6G of RAM, but after a day of being logged in, I usually had some
swap usage. Sometimes this goes up to 1.5G, this is when the system becomes
way too slow and I log out.

Normally I don't mind having swap, I always had. But to me it looks like
this is worse than it should be. From the point when swapping starts, it's
nearly permanent, like, when switching applications and desktops. As if
important stuff were swapped out that will be needed again soon. While
unimportant stuff stays in memory.

This was even worse when using the ati-drivers/fglrx, but at the moment
I'm using xf86-video-ati with xorg-server-1.10.1.901 and KMS.
echo 1 > /proc/sys/vm/drop_caches helps a little, but not much. And not
for long.

Sometimes I quit memory-hungry applications like Kontact, Amarok and TV
Browser and restart them immediately afterwards. Memory usage has
dropped, and desktop switching is fast again, once I went to every desktop
so that stuff will be swapped in. BTW, vm.swappiness is set to 20.

Or I do a 'swapoff -a && swapon -a', this empties the swap and also
frees memory. But this takes _quite_ _a_ _while_. Once I measured 37 minutes
to clear 1G of swap.
But I do not remember how much memory was being used then, so I tried
again, after closing memory-intensive applications:

weird ~ # free -m
total used free shared buffers cached
Mem: 5721 4184 1536 0 34 111
-/+ buffers/cache: 4039 1682
Swap: 4093 885 3208

weird ~ # time swapoff -a

real 27m8.757s
user 2m12.284s
sys 21m37.089s

weird ~ # free -m
total used free shared buffers cached
Mem: 5721 4785 936 0 53 409
-/+ buffers/cache: 4322 1398
Swap: 0 0 0

So, 27 minutes to put 885MB of swap back into RAM, with the double amount of
that being free RAM. I monitored with iotop, and the transfer rate started
around 60-100 K/s, later it went higher. But the average transfer rate is
550K/s. Shouldn't swap be, like, a little faster?

My swap is on an LUKS-encrypted LVM volume, but I tried with a fresh new
LVM volume, and it was a little better only. The SATA drive gives 80-100
MB/s according to hdparm -t. dd'ing /dev/zero directly to the swap volume
gives around 100 MB/s.

I'm running ck-sources 2.6.38, but I experience this for a long time
now, and changed from tuxonice-sources (another thing that just doesn't
work well for me) to ck-sources, to try if this would improve things. I
even cloned a .config from some live cd, in case I had some stupid option
activated that slowed things down.

Lowering swappiness helped, as did more memory. With 3.7G of RAM, KDE4
was barely usable. But two years ago, when this PC was new, I ran KDE 4.2 on
x86, compiled all except OpenOffice in tmpfs, often had a virtual
machine running in VMplayer, and it was fine.

I sure run lots of applications (Kontact, Amarok, TV Browser which uses an
incredible amount of RAM), some Dolphins, some Konsoles, a few Konquerors,
about 15-26 Chromium tabs, KMyMoney, some editors, sometimes Qt Creator, and
some Okulars. When I file some photos from my camera with Digikam, I
normally do not close it afterwards, hoping that it gets swapped out if
needed, and unless I start using it again this should not cost too much of
my RAM. This has always been my style of working, I don't like to close an
applications just after working with it. And it was fine in the past, with
much less memory, and with a virtual machine running all of the time.

Now I got another 2G of RAM, and things are better. But still, I have 800M
of swap right now, without using special applications or VMs. At this
moment, it's not problem, there is no paging going on. A little while ago,
it was quite noticeable. I was running rdiff-backup stuff at that moment,
but so I am doing now. Weird.
Oh, even weirder: The phone just rang, and five minutes later, swap has gone
to 860M. I was running rdiff-backup --list-increment-sizes, maybe this uses
much memory, and caches the stuff. Now the command has finished, and paging
has stopped. The rdiff-backup process itself does not use much memory.

BTW, does anyone else's kwin use 750M? That's pretty high, I think it used
to be more like 300M.

Wonko
 
Old 05-25-2011, 03:31 PM
Paul Hartman
 
Default Swap performance

On Wed, May 25, 2011 at 9:20 AM, Alex Schuster <wonko@wonkology.org> wrote:
> Hi there!
>
> I still wonder why my KDE4 system starts swapping so early. Until a week
> ago, I had 6G of RAM, but after a day of being logged in, I usually had some
> swap usage. Sometimes this goes up to 1.5G, this is when the system becomes
> way too slow and I log out.

I can't remember the last time my swap was used at all. I have 12G of
RAM, but in my prior system with 8G it was the same. Only in a rare
case when some program had run-away memory usage/memory leak did I
ever get to swap usage... I'm using vanilla kernel 2.6.39 with no
patches, no BFS. And I use proprietary nvidia-drivers. I normally
don't have so many programs running at once, but it happens sometimes.

Since 2.6.38 and enabling automatic process grouping, I don't use nice
or ionice at all anymore. I do parallel emerge with -j along with make
-j12 and never notice any slowdown or lag in UI at all.

With KDE4 logged in, and no GUI apps running (other than knutmon and
wicd), my RAM usage is slightly less than 900M (not counting
filesystem caches).

>BTW, does anyone else's kwin use 750M? That's pretty high, I think it used
>to be more like 300M.

My kwin (4.6.3-r1) has currently 507M VIRT, 54M RES, 37M SHR according to top.

My worst memory offenders, by resident memory:
clamd 124M
denyhosts 114M
X 75M
plasma-desktop 56M

By virtual memory:
krunner 964M
wicd-client (python) 681M
plasma-desktop 643M
kded4 581M

>Or I do a 'swapoff -a && swapon -a', this empties the swap and also
>frees memory. But this takes _quite_ _a_ _while_. Once I measured 37 minutes
>to clear 1G of swap.

My PC doesn't swap, but in my Nokia N900, it runs Linux and X, heavy
use of gtk and Qt4 libs, it has 256M of RAM and 768M of swap on eMMC
(transfer rate about 20MB/sec). It swaps like crazy. Usually there
is more swap in use than RAM, actually.

When I move the swap to a slow SD card instead (2MB/sec transfer
rate), even in that slow device, swapoff on the eMMC swap partition
with ~500M in-use takes about 2 or 3 minutes at most with the data
being swapped slowly into the SD card.

So I think in your case it should be much faster than that!
 
Old 05-25-2011, 04:49 PM
Alan McKinnon
 
Default Swap performance

Apparently, though unproven, at 16:20 on Wednesday 25 May 2011, Alex Schuster
did opine thusly:

> Oh, even weirder: The phone just rang, and five minutes later, swap has
> gone to 860M. I was running rdiff-backup --list-increment-sizes, maybe
> this uses much memory, and caches the stuff. Now the command has finished,
> and paging has stopped. The rdiff-backup process itself does not use much
> memory.
>
> BTW, does anyone else's kwin use 750M? That's pretty high, I think it used
> to be more like 300M.

I've been noticing this kind of thing too for a while now. In my case it's the
nepomuk/akonadi/virtuoso stack doing it - it seems to trigger full scans at
weird times and does other special things after a resume from suspend.

Virtuoso can sometimes get as high as 800M RES memory in top. Which is all
quite bizarre, I suspect a dodgy config on my part.

As for kwin - what column are you reading the value from? Here kwin uses more
like 60M


--
alan dot mckinnon at gmail dot com
 
Old 05-25-2011, 04:53 PM
Volker Armin Hemmann
 
Default Swap performance

On Wednesday 25 May 2011 16:20:58 Alex Schuster wrote:

> So, 27 minutes to put 885MB of swap back into RAM, with the double amount of
> that being free RAM. I monitored with iotop, and the transfer rate started
> around 60-100 K/s, later it went higher. But the average transfer rate is
> 550K/s. Shouldn't swap be, like, a little faster?

no, sounds about right. Swap in linux is brain damaging slow. If you think
about it your gall bladder might explode. That slow.
 
Old 05-25-2011, 04:58 PM
Volker Armin Hemmann
 
Default Swap performance

On Wednesday 25 May 2011 10:31:19 Paul Hartman wrote:

> When I move the swap to a slow SD card instead (2MB/sec transfer
> rate), even in that slow device, swapoff on the eMMC swap partition
> with ~500M in-use takes about 2 or 3 minutes at most with the data
> being swapped slowly into the SD card.
>
> So I think in your case it should be much faster than that!

you are comparing apples with oranges (harddisks with moving arms with solid
state devices).

Do yourself a favour. Look up how long a harddisk needs to position its head.

Now you can calculate how many times a second a harddisk can position its
head.

Now remember: swap is stupid, so lots and lots of head movement needed (and a
cash flush is running too - so even more movements to write all that crap to
disk),

The result: the whole mess is fscking slow.

You can have a nice fat raid with nice and fast harddisks - if you try to
stream to a 15 year old DLT drive with 5/10mb/sec speed the dlt drive will
constantly rewind - because harddisks suck when they have to seek. And swap
(just like a backup) = lots and lots and lots of seeks.
 
Old 05-25-2011, 07:34 PM
Paul Hartman
 
Default Swap performance

On Wed, May 25, 2011 at 11:58 AM, Volker Armin Hemmann
<volkerarmin@googlemail.com> wrote:
> On Wednesday 25 May 2011 10:31:19 Paul Hartman wrote:
>
>> When I move the swap to a slow SD card instead (2MB/sec transfer
>> rate), even in that slow device, swapoff on the eMMC swap partition
>> with ~500M in-use takes about 2 or 3 minutes at most with the data
>> being swapped slowly into the SD card.
>>
>> So I think in your case it should be much faster than that!
>
> you are comparing apples with oranges (harddisks with moving arms with solid
> state devices).
>
> Do yourself a favour. Look up how long a harddisk needs to position its head.

Yeah, measuring speed in microseconds versus milliseconds.

My SD read/write speed is about 0.9MB/sec in benchmark, and about 2
MB/sec in real-life swap usage. Based on Alex's timings it seems his
HDD is getting about 0.5 MB/sec in swap usage. HDD benchmark random
read/write is usually faster than that. But probably you're right and
the seek operations are really killing it. HDD is about 20 times
slower than SD card in seeking.

If he has a USB flash drive, or memory card reader, he can use it for
swap and see if it's any better or worse. I am curious how it will
compare.

Or, if he has multiple disks, he could create more swap partitions
with equal priority and kernel will stripe/load balance automatically,
hopefully improving his swap performance.

Or better yet, figure out why his system is swapping at all which is
what he was going for I think. With 8 GB I think he should be able to
disable swap entirely anyway.
 
Old 05-25-2011, 08:03 PM
Alan McKinnon
 
Default Swap performance

Apparently, though unproven, at 18:58 on Wednesday 25 May 2011, Volker Armin
Hemmann did opine thusly:

> On Wednesday 25 May 2011 10:31:19 Paul Hartman wrote:
> > When I move the swap to a slow SD card instead (2MB/sec transfer
> > rate), even in that slow device, swapoff on the eMMC swap partition
> > with ~500M in-use takes about 2 or 3 minutes at most with the data
> > being swapped slowly into the SD card.
> >
> > So I think in your case it should be much faster than that!
>
> you are comparing apples with oranges (harddisks with moving arms with
> solid state devices).
>
> Do yourself a favour. Look up how long a harddisk needs to position its
> head.
>
> Now you can calculate how many times a second a harddisk can position its
> head.
>
> Now remember: swap is stupid, so lots and lots of head movement needed (and
> a cash flush is running too - so even more movements to write all that
> crap to disk),
>
> The result: the whole mess is fscking slow.
>
> You can have a nice fat raid with nice and fast harddisks - if you try to
> stream to a 15 year old DLT drive with 5/10mb/sec speed the dlt drive will
> constantly rewind - because harddisks suck when they have to seek. And swap
> (just like a backup) = lots and lots and lots of seeks.

That reminds me of how SSDs ought to be much faster than hard disks.

But every time I use my Acer netbook (8G SSD) I curse and swear and commit
random acts of violence - that first gen SSD controller is the worst possible
thing to ever hit computers. I swear the 4G SDHC expansion card is leaps
faster....

(completely OT, I know. I'll keep quiet now.)

--
alan dot mckinnon at gmail dot com
 
Old 05-25-2011, 08:07 PM
Alan McKinnon
 
Default Swap performance

Apparently, though unproven, at 18:53 on Wednesday 25 May 2011, Volker Armin
Hemmann did opine thusly:

> On Wednesday 25 May 2011 16:20:58 Alex Schuster wrote:
> > So, 27 minutes to put 885MB of swap back into RAM, with the double amount
> > of that being free RAM. I monitored with iotop, and the transfer rate
> > started around 60-100 K/s, later it went higher. But the average
> > transfer rate is 550K/s. Shouldn't swap be, like, a little faster?
>
> no, sounds about right. Swap in linux is brain damaging slow. If you think
> about it your gall bladder might explode. That slow.

For years now I've considered only two possible uses for linux swap:

- a teeny small one just for wiggle room to try and hold that POS called the
oom killer at bay

- a bigger one the same size as total RAM, as a place to put the suspend image


Every other usage makes no sense at all (RAM being so cheap and all). In fact,
I've banned swap on all company servers except databases.

I'd be interested to hear any current use cases where swap delivers a provable
benefit.

--
alan dot mckinnon at gmail dot com
 
Old 05-25-2011, 08:38 PM
Alex Schuster
 
Default Swap performance

Alan McKinnon writes:

> That reminds me of how SSDs ought to be much faster than hard disks.
>
> But every time I use my Acer netbook (8G SSD) I curse and swear and commit
> random acts of violence - that first gen SSD controller is the worst possible
> thing to ever hit computers. I swear the 4G SDHC expansion card is leaps
> faster....
>
> (completely OT, I know. I'll keep quiet now.)

Ah, it's okay, we all need this from time to time

Wonko
 
Old 05-25-2011, 10:18 PM
Alex Schuster
 
Default Swap performance

Volker Armin Hemmann writes:

> On Wednesday 25 May 2011 16:20:58 Alex Schuster wrote:
>
>> So, 27 minutes to put 885MB of swap back into RAM, with the double amount of
>> that being free RAM. I monitored with iotop, and the transfer rate started
>> around 60-100 K/s, later it went higher. But the average transfer rate is
>> 550K/s. Shouldn't swap be, like, a little faster?
>
> no, sounds about right. Swap in linux is brain damaging slow. If you think
> about it your gall bladder might explode. That slow.

Oh. _That_ slow.

Wonko
 

Thread Tools




All times are GMT. The time now is 04:03 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org