FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Crash Utility

 
 
LinkBack Thread Tools
 
Old 01-26-2012, 01:15 AM
Brian Reichert
 
Default crash failing with CentOS 5 under VMware

I'm trying to explore crash dumps under these conditions:

- VMware workstation 7.1.5 build-491717
- CentOS 5.7 +updates, as of today
- kernel 2.6.18-274.17.1.el5 x86
- crash 6.0.2

I've sucessfully enabled kdump to generate crash dumps, but the
'crash' utility can neither find the vmlinux image for the live
system, nor match the crash dump with the vmlinux image I direct
it to.

Misc RPMs installed:

[root@172-20-1-25 modules]# uname -r; rpm -qa | grep kernel
2.6.18-274.17.1.el5
kernel-2.6.18-274.17.1.el5
kernel-debug-2.6.18-274.17.1.el5
kernel-debuginfo-2.6.18-274.17.1.el5.centos.plus
kernel-debuginfo-common-2.6.18-274.17.1.el5.centos.plus
kernel-devel-2.6.18-274.17.1.el5

Here are various invocations I've attempted; can anyone suggest what's
causing these failures, and how to work around them?

Please let me know if there are any details I can provide to assist.

Additional notes:

- I've tries both crashkernel=64M@16M and crashkernel=128M@16M as
kernel arguments, with the same results.

- I've attempted a symlink to match the debug modules directly to
the name of the kernel, but that didn't change the results:
2.6.18-274.17.1.el5.centos.plus -> 2.6.18-274.17.1.el5

-------------------------------
Against the live system:

[root@172-20-1-25 modules]# crash

crash 6.0.2
[...]

crash: cannot find booted kernel -- please enter namelist argument

-------------------------------
Against the vmlinux file supplied by the kernel-debuginfo RPM:

[root@172-20-1-25 modules]# crash /usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux

crash 6.0.2
[...]

WARNING: /usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux
and /proc/version do not match!

WARNING: /proc/version indicates kernel version: 2.6.18-274.17.1.el5

crash: please use the vmlinux file for that kernel version, or try using
the System.map for that kernel version as an additional argument.

-------------------------------
Against my crash dump:

[root@172-20-1-25 modules]# crash /usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux /home/crash/127.0.0.1-2012-01-26-19:19:19/vmcore

crash 6.0.2
[...]

crash: cannot determine base kernel version
WARNING: cannot read linux_banner string
crash: /usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux and /home/crash/127.0.0.1-2012-01-26-19:19:19/vmcore do not match!

--
Brian Reichert <reichert@numachi.com>
BSD admin/developer at large

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 
Old 01-26-2012, 08:21 PM
Brian Reichert
 
Default crash failing with CentOS 5 under VMware

On Thu, Jan 26, 2012 at 04:54:58PM -0500, Dave Anderson wrote:
> There is an inconsistency between the 2.6.18-274.17.1.el5.centos.plus kernel
> and the running kernel 2.6.18-274.17.1.el5. I don't know what the "centos.plus"
> vmlinux kernel is, but its symbols apparently do no match the 2.6.18-274.17.1.el5
> kernel that is running.
> Is the system running a bona fide RHEL5 2.6.18-274.17.1.el5
> kernel, and you're using a centos-generated kernel-debuginfo package?

No, this is a CentOS 5.7 installation. These RPMs came from the CentOS
official repos. I was concerned by the apparrent disrepancy in the
names of the files.

> Anyway, you could try what the error message suggests. There is probably a
> "/boot/System.map-2.6.18-274.17.1.el5" on that system, and presuming so,
> try this on the live system:

I'll go looking for such a file, and explore your other suggestions.
Thanks for the feedback!

> Dave
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility

--
Brian Reichert <reichert@numachi.com>
BSD admin/developer at large

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 
Old 01-26-2012, 08:54 PM
Dave Anderson
 
Default crash failing with CentOS 5 under VMware

----- Original Message -----
> I'm trying to explore crash dumps under these conditions:
>
> - VMware workstation 7.1.5 build-491717
> - CentOS 5.7 +updates, as of today
> - kernel 2.6.18-274.17.1.el5 x86
> - crash 6.0.2
>
> I've sucessfully enabled kdump to generate crash dumps, but the
> 'crash' utility can neither find the vmlinux image for the live
> system, nor match the crash dump with the vmlinux image I direct
> it to.
>
> Misc RPMs installed:
>
> [root@172-20-1-25 modules]# uname -r; rpm -qa | grep kernel
> 2.6.18-274.17.1.el5
> kernel-2.6.18-274.17.1.el5
> kernel-debug-2.6.18-274.17.1.el5
> kernel-debuginfo-2.6.18-274.17.1.el5.centos.plus
> kernel-debuginfo-common-2.6.18-274.17.1.el5.centos.plus
> kernel-devel-2.6.18-274.17.1.el5
>
> Here are various invocations I've attempted; can anyone suggest what's
> causing these failures, and how to work around them?
>
> Please let me know if there are any details I can provide to assist.
>
> Additional notes:
>
> - I've tries both crashkernel=64M@16M and crashkernel=128M@16M as
> kernel arguments, with the same results.
>
> - I've attempted a symlink to match the debug modules directly to
> the name of the kernel, but that didn't change the results:
> 2.6.18-274.17.1.el5.centos.plus -> 2.6.18-274.17.1.el5
>
> -------------------------------
> Against the live system:
>
> [root@172-20-1-25 modules]# crash
>
> crash 6.0.2
> [...]
>
> crash: cannot find booted kernel -- please enter namelist argument
>
> -------------------------------
> Against the vmlinux file supplied by the kernel-debuginfo RPM:
>
> [root@172-20-1-25 modules]# crash /usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux
>
> crash 6.0.2
> [...]
>
> WARNING: /usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux
> and /proc/version do not match!
>
> WARNING: /proc/version indicates kernel version: 2.6.18-274.17.1.el5
>
> crash: please use the vmlinux file for that kernel version, or try using
> the System.map for that kernel version as an additional
> argument.

There is an inconsistency between the 2.6.18-274.17.1.el5.centos.plus kernel
and the running kernel 2.6.18-274.17.1.el5. I don't know what the "centos.plus"
vmlinux kernel is, but its symbols apparently do no match the 2.6.18-274.17.1.el5
kernel that is running. Is the system running a bona fide RHEL5 2.6.18-274.17.1.el5
kernel, and you're using a centos-generated kernel-debuginfo package? In other words,
in a real RHEL5 environment there should be "kernel-debuginfo-2.6.18-274.17.1.el5" and
associated "kernel-debuginfo-common-2.6.18-274.17.1.el5" packages that would have
the vmlinux file for the running 2.6.18-274.17.1.el5 kernel. So again, I don't have a
clue about what those "centos.plus" packages are all about...

Anyway, you could try what the error message suggests. There is probably a
"/boot/System.map-2.6.18-274.17.1.el5" on that system, and presuming so,
try this on the live system:

$ crash usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux /boot/System.map-2.6.18-274.17.1.el5

>
> -------------------------------
> Against my crash dump:
>
> [root@172-20-1-25 modules]# crash /usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux /home/crash/127.0.0.1-2012-01-26-19:19:19/vmcore
>
> crash 6.0.2
> [...]
>
> crash: cannot determine base kernel version
> WARNING: cannot read linux_banner string
> crash: /usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux
> and /home/crash/127.0.0.1-2012-01-26-19:19:19/vmcore do not match!

Same thing is happening with the vmcore. In the case of a vmcore, it reads
the "linux_banner" string (what /proc/version shows on a live system) at the
address where the "centos.plus" vmlinux file says it is, but after doing so,
it doesn't match up. (so it's reading the wrong location)

If you do this:

$ strings /home/crash/127.0.0.1-2012-01-26-19:19:19/vmcore | grep "Linux version"

I'm guessing you won't see the "centos.plus" string?

As with the live system, you could try throwing the /boot/System.map-<version> file
on the command line.

In any case, if you add a debug count on the command line (the larger the number
the more debug data gets displayed), it may illuminate the failure, i.e., by
entering:

$ crash -d7 /path-to/vmlinux /path-to/vmcore

Dave

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 
Old 01-26-2012, 10:26 PM
Brian Reichert
 
Default crash failing with CentOS 5 under VMware

On Thu, Jan 26, 2012 at 04:54:58PM -0500, Dave Anderson wrote:
> As with the live system, you could try throwing the /boot/System.map-<version> file
> on the command line.

For the record, this suggestion did work, thanks.

Against the vmlinux file supplied by the kernel-debuginfo RPM:

crash -S /boot/System.map-2.6.18-274.17.1.el5
/usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux

I can analyze my crash thusly:

crash -S /boot/System.map-2.6.18-274.17.1.el5
/usr/lib/debug/lib/modules/2.6.18-274.17.1.el5.centos.plus/vmlinux
/home/crash/127.0.0.1-2012-01-27-12:22:05/vmcore

So, I'm off to the races. Hurray!

> Dave
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility

--
Brian Reichert <reichert@numachi.com>
BSD admin/developer at large

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 
Old 01-26-2012, 10:29 PM
Brian Reichert
 
Default crash failing with CentOS 5 under VMware

On Fri, Jan 27, 2012 at 12:03:44PM -0500, Dave Anderson wrote:
> I note that here:
>
> http://debuginfo.centos.org/5/x86_64/
>
> they have both the "centos.plus" version that you have installed, as well as
> what would appear to be the one that you actually want:
>
> kernel-debuginfo-2.6.18-274.17.1.el5.centos.plus.x86_64.rpm 11-Jan-2012 18:57 200M
> kernel-debuginfo-2.6.18-274.17.1.el5.x86_64.rpm 11-Jan-2012 01:59 179M
>
> I would install the kernel-debuginfo-2.6.18-274.17.1.el5.x86_64.rpm
> and its associated kernel-debuginfo-common-2.6.18-274.7.1.el5.x86_64.rpm
> packages.

Weird. All I did was this:

yum --enablerepo="debug" -y install kernel-debuginfo

And *.centos.plus is what came out of the repo. :/

> Dave
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility

--
Brian Reichert <reichert@numachi.com>
BSD admin/developer at large

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 
Old 01-27-2012, 04:03 PM
Dave Anderson
 
Default crash failing with CentOS 5 under VMware

----- Original Message -----
> On Thu, Jan 26, 2012 at 04:54:58PM -0500, Dave Anderson wrote:
> > There is an inconsistency between the
> > 2.6.18-274.17.1.el5.centos.plus kernel
> > and the running kernel 2.6.18-274.17.1.el5. I don't know what the
> > "centos.plus"
> > vmlinux kernel is, but its symbols apparently do no match the
> > 2.6.18-274.17.1.el5
> > kernel that is running.
> > Is the system running a bona fide RHEL5 2.6.18-274.17.1.el5
> > kernel, and you're using a centos-generated kernel-debuginfo
> > package?
>
> No, this is a CentOS 5.7 installation. These RPMs came from the CentOS
> official repos. I was concerned by the apparrent disrepancy in the
> names of the files.
>
> > Anyway, you could try what the error message suggests. There is probably a
> > "/boot/System.map-2.6.18-274.17.1.el5" on that system, and presuming so,
> > try this on the live system:
>
> I'll go looking for such a file, and explore your other suggestions.
> Thanks for the feedback!

I note that here:

http://debuginfo.centos.org/5/x86_64/

they have both the "centos.plus" version that you have installed, as well as
what would appear to be the one that you actually want:

kernel-debuginfo-2.6.18-274.17.1.el5.centos.plus.x86_64.rpm 11-Jan-2012 18:57 200M
kernel-debuginfo-2.6.18-274.17.1.el5.x86_64.rpm 11-Jan-2012 01:59 179M

I would install the kernel-debuginfo-2.6.18-274.17.1.el5.x86_64.rpm
and its associated kernel-debuginfo-common-2.6.18-274.7.1.el5.x86_64.rpm
packages.

Dave

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 

Thread Tools




All times are GMT. The time now is 07:04 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org