FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 01-16-2011, 11:15 PM
Mark Knecht
 
Default Near freezes during large emerges

On Sun, Jan 16, 2011 at 2:41 PM, Grant <emailgrant@gmail.com> wrote:
>>> I've been running without swap for quite a while, but my system goes
>>> into a near freeze whenever I undertake a large emerge such as
>>> chromium or openoffice. *Is there anything I can do to prevent this
>>> besides turning swap back on? *I have 3GB RAM and MAKEOPTS="-j1".
>>>
>>> - Grant
>>>
>>>
>>
>> As Volker says, don't turn swap off. Make it small if you must, but
>> keep some around. It's just disk space.
>
> I thought swap was no longer necessary on a machine with sufficient
> memory. *I guess I took I some bad advice a while back.
>

I think the idea is never use swap if possible, but in a case where
you don't have swap space or run out of swap space I think it's still
possible to lose data. I no longer double memory in swap. In the old
days I did that. On this server I have 24GB or memory. It seems silly
to chew up 50GB of disk space for something that almost never gets
touched. If I see this machine swapping I turn something off, but I'm
the only user and here to watch what it's doing.


<SNIP>
>> You don't say what sort of processor this machine has, nor what sort
>> of hard drives. Even on a very high-end machine I saw the symptoms you
>> report when I tried a 4K sector WD Green drive with the partitions
>> misaligned. Once properly aligned the machine worked as expected.
>
> It's a laptop with a dual-core 2.2ghz CPU.
>

So -j1 is pretty safe. That's what I was wondering...

<SNIP>
>
> 'onice -c 3 emerge -DuN world' ended up working great.
>
> - Grant

Yeah, I saw the response that got you there. Good info and good to
know it works.

Cheers,
Mark
 
Old 01-17-2011, 12:08 AM
Alan McKinnon
 
Default Near freezes during large emerges

Apparently, though unproven, at 02:15 on Monday 17 January 2011, Mark Knecht
did opine thusly:

[snip]

> >> As Volker says, don't turn swap off. Make it small if you must, but
> >> keep some around. It's just disk space.
> >
> > I thought swap was no longer necessary on a machine with sufficient
> > memory. I guess I took I some bad advice a while back.
>
> I think the idea is never use swap if possible, but in a case where
> you don't have swap space or run out of swap space I think it's still
> possible to lose data. I no longer double memory in swap. In the old
> days I did that. On this server I have 24GB or memory. It seems silly
> to chew up 50GB of disk space for something that almost never gets
> touched. If I see this machine swapping I turn something off, but I'm
> the only user and here to watch what it's doing.

The 2 x RAM rule is an ancient artifact that hasn't been true for, well for
ages now. It came about because way back when you had to have swap to get
anything done. The question is how much? The answer sucked out of someone's
thumb was 2xRAM. This is a pretty useless generic value, but it was less
useless than any other default.

Picking swap amounts is like picking a wife - there's no sane default. A
modern desktop that swaps is unusable - enormous amounts of data has to be
pulled back in from the drive. A web server that swaps is already thrashing so
you always want to avoid that.

Besides, RAM is cheap and a server with 24G is common place. So is 4G on a
notebook. So your viewpoint is completely correct.

The kernel does need some swap though - it needs wiggle room for when you DO
run out of RAM, and a little bit of swap gives that. It also staves off that
bastard demon spawn progeny of satan called the dreaded oom killer....


--
alan dot mckinnon at gmail dot com
 
Old 01-17-2011, 12:13 AM
William Kenworthy
 
Default Near freezes during large emerges

On Sun, 2011-01-16 at 14:41 -0800, Grant wrote:
> >> I've been running without swap for quite a while, but my system goes
> >> into a near freeze whenever I undertake a large emerge such as
> >> chromium or openoffice. Is there anything I can do to prevent this
> >> besides turning swap back on? I have 3GB RAM and MAKEOPTS="-j1".
> >>
> >> - Grant
> >>
> >>
> >
> > As Volker says, don't turn swap off. Make it small if you must, but
> > keep some around. It's just disk space.
>
> I thought swap was no longer necessary on a machine with sufficient
> memory. I guess I took I some bad advice a while back.

The answer is that you have insufficient memory when emerging - hence
swap is necessary - turn it on! ionice will help, but it is alleviating
symptoms of lack of swap, not curing it. Downside in this case is
slower emerges and some will still be flaky.

You can use a temporary swapfile or swap over ndb for those special
cases if you have no swap partition. Also check the tunable
parameter /proc/sys/vm/swappiness to force memory to swap, or to get the
kernel to be very reluctant to use swap - can give the benefits of no
swap, but still have a safety margin with a small penalty when running
low on memory. google for "/proc/sys/vm/swappiness"

In short, swap is good, downside to swap is sometimes small pauses while
pages are swapped back in. But not having swap forces the kernel to
have a large overhead in trying to manage a low memory situation when it
gets low such as during an emerge. No swap is ok if you have a system
doing almost nothing, but with almost any normal use, 3G and no swap is
going to be trouble at times.

BillK
 
Old 01-17-2011, 12:21 AM
William Kenworthy
 
Default Near freezes during large emerges

On Mon, 2011-01-17 at 03:08 +0200, Alan McKinnon wrote:
> Apparently, though unproven, at 02:15 on Monday 17 January 2011, Mark Knecht
> did opine thusly:
>
> [snip]
>
> > >> As Volker says, don't turn swap off. Make it small if you must, but
> > >> keep some around. It's just disk space.
> > >
> > > I thought swap was no longer necessary on a machine with sufficient
> > > memory. I guess I took I some bad advice a while back.
> >
> > I think the idea is never use swap if possible, but in a case where
> > you don't have swap space or run out of swap space I think it's still
> > possible to lose data. I no longer double memory in swap. In the old
> > days I did that. On this server I have 24GB or memory. It seems silly
> > to chew up 50GB of disk space for something that almost never gets
> > touched. If I see this machine swapping I turn something off, but I'm
> > the only user and here to watch what it's doing.
>
> The 2 x RAM rule is an ancient artifact that hasn't been true for, well for
> ages now. It came about because way back when you had to have swap to get
> anything done. The question is how much? The answer sucked out of someone's
> thumb was 2xRAM. This is a pretty useless generic value, but it was less
> useless than any other default.
>

There was in ye old unix days a good technical reason for 2xmemory for
swap - google didnt confirm it for me but I think old solaris used to
coredump memory to swap on a crash.

> Picking swap amounts is like picking a wife - there's no sane default.

love the metaphor!

> A
> modern desktop that swaps is unusable - enormous amounts of data has to be
> pulled back in from the drive. A web server that swaps is already thrashing so
> you always want to avoid that.
>
> Besides, RAM is cheap and a server with 24G is common place. So is 4G on a
> notebook. So your viewpoint is completely correct.
>
> The kernel does need some swap though - it needs wiggle room for when you DO
> run out of RAM, and a little bit of swap gives that. It also staves off that
> bastard demon spawn progeny of satan called the dreaded oom killer....
>

There is one case where ~2xram is still a good idea - when hibernating
to swap using (in my case) tuxonice - 2xram gives a reasonable safety
margin for hibernation plus existing swap contents.

BillK


--
William Kenworthy <billk@iinet.net.au>
Home in Perth!
 
Old 01-17-2011, 12:26 AM
Mark Knecht
 
Default Near freezes during large emerges

On Sun, Jan 16, 2011 at 5:13 PM, William Kenworthy <billk@iinet.net.au> wrote:
> On Sun, 2011-01-16 at 14:41 -0800, Grant wrote:
>> >> I've been running without swap for quite a while, but my system goes
>> >> into a near freeze whenever I undertake a large emerge such as
>> >> chromium or openoffice. *Is there anything I can do to prevent this
>> >> besides turning swap back on? *I have 3GB RAM and MAKEOPTS="-j1".
>> >>
>> >> - Grant
>> >>
>> >>
>> >
>> > As Volker says, don't turn swap off. Make it small if you must, but
>> > keep some around. It's just disk space.
>>
>> I thought swap was no longer necessary on a machine with sufficient
>> memory. *I guess I took I some bad advice a while back.
>
> The answer is that you have insufficient memory when emerging - hence
> swap is necessary - turn it on! *ionice will help, but it is alleviating
> symptoms of lack of swap, not curing it. *Downside in this case is
> slower emerges and some will still be flaky.
>

I think that's well worded. He has insufficient memory when emerging.

If he's really running short of DRAM Then he might also do well to
boot to a console and do his emerges there. No memory given over to
other things like KDE or browsers, etc.

I am a bit surprised though that a -j1 type emerge would be running
out of memory on a 3GB machine. I just finished emerge updates on a
desktop with 4GB and only used 2.5GB which includes KDE, FIrefox and a
number of other things:

mark@firefly ~ $ top
top - 17:24:12 up 1:58, 3 users, load average: 1.01, 1.03, 0.91
Tasks: 121 total, 1 running, 120 sleeping, 0 stopped, 0 zombie
Cpu(s): 1.0%us, 0.6%sy, 0.0%ni, 98.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 3851008k total, 2466316k used, 1384692k free, 351200k buffers
Swap: 8393924k total, 0k used, 8393924k free, 1316212k cached


- Mark
 
Old 01-17-2011, 12:26 AM
Neil Bothwick
 
Default Near freezes during large emerges

On Sun, 16 Jan 2011 14:41:24 -0800, Grant wrote:

> 'onice -c 3 emerge -DuN world' ended up working great.

Or you can set PORTAGE_IONICE_COMMAND in make.conf to make it a default.

--
Neil Bothwick

Vuja De: the feeling that you've never been here before.
 
Old 01-17-2011, 12:39 AM
William Kenworthy
 
Default Near freezes during large emerges

On Sun, 2011-01-16 at 17:26 -0800, Mark Knecht wrote:
> On Sun, Jan 16, 2011 at 5:13 PM, William Kenworthy <billk@iinet.net.au> wrote:
> > On Sun, 2011-01-16 at 14:41 -0800, Grant wrote:
...
>
> I think that's well worded. He has insufficient memory when emerging.
>
> If he's really running short of DRAM Then he might also do well to
> boot to a console and do his emerges there. No memory given over to
> other things like KDE or browsers, etc.
>
> I am a bit surprised though that a -j1 type emerge would be running
> out of memory on a 3GB machine. I just finished emerge updates on a
> desktop with 4GB and only used 2.5GB which includes KDE, FIrefox and a
> number of other things:
>

I have a diskless 3GB ram atom system (mythtv frontend) and I have to
arrange swap over nbd for gcc and glibc emerges - others just get very
slow when getting to limits, or get flaky unless -j1 is used. Havent
tried OO on it yet

BillK
 
Old 01-17-2011, 07:05 AM
Alan McKinnon
 
Default Near freezes during large emerges

Apparently, though unproven, at 03:21 on Monday 17 January 2011, William
Kenworthy did opine thusly:


> > A
> > modern desktop that swaps is unusable - enormous amounts of data has to
> > be pulled back in from the drive. A web server that swaps is already
> > thrashing so you always want to avoid that.
> >
> > Besides, RAM is cheap and a server with 24G is common place. So is 4G on
> > a notebook. So your viewpoint is completely correct.
> >
> > The kernel does need some swap though - it needs wiggle room for when you
> > DO run out of RAM, and a little bit of swap gives that. It also staves
> > off that bastard demon spawn progeny of satan called the dreaded oom
> > killer....
>
> There is one case where ~2xram is still a good idea - when hibernating
> to swap using (in my case) tuxonice - 2xram gives a reasonable safety
> margin for hibernation plus existing swap contents.


Yes, that's true. But modern notebooks often have 4G of RAM so that's a big
partition for hibernating. Swap files probably function better for that case.
I stopped hibernating a long time ago for that reason and now just suspend.
Besides, the machine is seldom off for more than 4 hours


--
alan dot mckinnon at gmail dot com
 
Old 01-17-2011, 07:07 AM
Alan McKinnon
 
Default Near freezes during large emerges

Apparently, though unproven, at 03:39 on Monday 17 January 2011, William
Kenworthy did opine thusly:

> On Sun, 2011-01-16 at 17:26 -0800, Mark Knecht wrote:
> > On Sun, Jan 16, 2011 at 5:13 PM, William Kenworthy <billk@iinet.net.au>
wrote:
> > > On Sun, 2011-01-16 at 14:41 -0800, Grant wrote:
> ...
>
> > I think that's well worded. He has insufficient memory when emerging.
> >
> > If he's really running short of DRAM Then he might also do well to
> > boot to a console and do his emerges there. No memory given over to
> > other things like KDE or browsers, etc.
> >
> > I am a bit surprised though that a -j1 type emerge would be running
> > out of memory on a 3GB machine. I just finished emerge updates on a
> > desktop with 4GB and only used 2.5GB which includes KDE, FIrefox and a
>
> > number of other things:
> I have a diskless 3GB ram atom system (mythtv frontend) and I have to
> arrange swap over nbd for gcc and glibc emerges - others just get very
> slow when getting to limits, or get flaky unless -j1 is used. Havent
> tried OO on it yet

I'M flabbergasted. 3G is really a gigantic amount of memory and yet the
machine still runs out of the stuff?

Something is seriously wrong somewhere when code does this. I know memory is
cheap and all, but still ... that's just excessive


--
alan dot mckinnon at gmail dot com
 
Old 01-17-2011, 07:18 AM
William Kenworthy
 
Default Near freezes during large emerges

On Mon, 2011-01-17 at 10:07 +0200, Alan McKinnon wrote:
> Apparently, though unproven, at 03:39 on Monday 17 January 2011, William
> Kenworthy did opine thusly:
>
> > On Sun, 2011-01-16 at 17:26 -0800, Mark Knecht wrote:
> > > On Sun, Jan 16, 2011 at 5:13 PM, William Kenworthy <billk@iinet.net.au>
> wrote:
> > > > On Sun, 2011-01-16 at 14:41 -0800, Grant wrote:
> > ...
> >
> > > I think that's well worded. He has insufficient memory when emerging.
> > >
> > > If he's really running short of DRAM Then he might also do well to
> > > boot to a console and do his emerges there. No memory given over to
> > > other things like KDE or browsers, etc.
> > >
> > > I am a bit surprised though that a -j1 type emerge would be running
> > > out of memory on a 3GB machine. I just finished emerge updates on a
> > > desktop with 4GB and only used 2.5GB which includes KDE, FIrefox and a
> >
> > > number of other things:
> > I have a diskless 3GB ram atom system (mythtv frontend) and I have to
> > arrange swap over nbd for gcc and glibc emerges - others just get very
> > slow when getting to limits, or get flaky unless -j1 is used. Havent
> > tried OO on it yet
>
> I'M flabbergasted. 3G is really a gigantic amount of memory and yet the
> machine still runs out of the stuff?
>
> Something is seriously wrong somewhere when code does this. I know memory is
> cheap and all, but still ... that's just excessive

Your behind the times Alan - 3G was a huge amount 10 years ago ...

Ahh, progress ...

BillK
 

Thread Tools




All times are GMT. The time now is 02:11 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org