Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   CentOS (http://www.linux-archive.org/centos/)
-   -   storage servers crashing, hair being pulled out! (http://www.linux-archive.org/centos/298070-storage-servers-crashing-hair-being-pulled-out.html)

Les Mikesell 12-20-2009 05:28 AM

storage servers crashing, hair being pulled out!
 
Gordon McLellan wrote:
> I have a trio of servers that like to reboot during high disk /
> network IO operations. They don't appear to panic, as I have
> kernel.panic = 0 in sysctl.conf. The syslog just shows normal
> messages, like samba complaining about browse master and then just
> syslogd starting up.

Did this just start happening after the last update, or have they never been
reliable? I have one box that just started crashing after the last kernel
update but it may just be from old age instead. If they have never been
reliable I'd suspect bad RAM first. I've seen cases where you had to run the
memory test a few days to catch it.

--
Les Mikesell
lesmikesell@gmail.com



_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos

Matty 12-20-2009 05:34 PM

storage servers crashing, hair being pulled out!
 
On Sat, Dec 19, 2009 at 10:55 PM, Gordon McLellan <gordonthree@gmail.com> wrote:

> I have a trio of servers that like to reboot during high disk /
> network IO operations. *They don't appear to panic, as I have
> kernel.panic = 0 in sysctl.conf. *The syslog just shows normal
> messages, like samba complaining about browse master and then just
> syslogd starting up.

If the box is panicing under high load, you should definitely check
the memory / CPU / power supplies. You may also find it beneficial to
enable kdump, netdump and sysrq. If the box hangs, you can issue a
sysrq magic key sequence to force the box to panic. During the panic
process, you should get a core file that you can analyze to see what
is going on (crash has some useful options to dump thread stacks,
which you can use to search the LKML archives).

- Ryan
--
http://prefetch.net
_______________________________________________
CentOS mailing list
CentOS@centos.org
http://lists.centos.org/mailman/listinfo/centos


All times are GMT. The time now is 11:05 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.