FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Crash Utility

 
 
LinkBack Thread Tools
 
Old 03-24-2012, 10:42 PM
"hujun_hotmail"
 
Default crash can live analysis but vmcore can't.

on another machine,I use all kinds of crash command
:
*
*
Vsles11sp1
/mnt/disk/crash/2012-03-24-18:57 #crash vmlinux-2.6.27.19-5-default.gz
System.map-2.6.27.19-5-default vmcore
*
crash 6.0.4
Copyright (C) 2002-2012* Red Hat,
Inc.
Copyright (C) 2004, 2005, 2006* IBM Corporation
Copyright (C)
1999-2006* Hewlett-Packard Co
Copyright (C) 2005, 2006* Fujitsu
Limited
Copyright (C) 2006, 2007* VA Linux Systems Japan
K.K.
Copyright (C) 2005* NEC Corporation
Copyright (C) 1999, 2002,
2007* Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002*
Mission Critical Linux, Inc.
This program is free software, covered by the
GNU General Public License,
and you are welcome to change it and/or
distribute copies of it under
certain conditions.* Enter "help copying"
to see the conditions.
This program has absolutely no warranty.* Enter
"help warranty" for details.
*
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free
Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This
is free software: you are free to change and redistribute it.
There is NO
WARRANTY, to the extent permitted by law.* Type "show copying"
and "show
warranty" for details.
This GDB was configured as
"x86_64-unknown-linux-gnu"...
*
crash: /var/tmp/vmlinux-2.6.27.19-5-default.gz_RYx9JC:
no debugging data available

Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #gunzip
vmlinux-2.6.27.19-5-default.gz

Vsles11sp1
/mnt/disk/crash/2012-03-24-18:57 #crash
vmlinux-2.6.27.19-5-default vmlinux-2.6.27.19-5-default.debug
vmcore
*
crash 6.0.4
Copyright (C) 2002-2012* Red Hat,
Inc.
Copyright (C) 2004, 2005, 2006* IBM Corporation
Copyright (C)
1999-2006* Hewlett-Packard Co
Copyright (C) 2005, 2006* Fujitsu
Limited
Copyright (C) 2006, 2007* VA Linux Systems Japan
K.K.
Copyright (C) 2005* NEC Corporation
Copyright (C) 1999, 2002,
2007* Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002*
Mission Critical Linux, Inc.
This program is free software, covered by the
GNU General Public License,
and you are welcome to change it and/or
distribute copies of it under
certain conditions.* Enter "help copying"
to see the conditions.
This program has absolutely no warranty.* Enter
"help warranty" for details.
*
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software
Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This
is free software: you are free to change and redistribute it.
There is NO
WARRANTY, to the extent permitted by law.* Type "show copying"
and "show
warranty" for details.
This GDB was configured as
"x86_64-unknown-linux-gnu"...
*
WARNING: could not find MAGIC_START!
crash:
vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
*
Usage:
*
* crash [OPTION]... NAMELIST MEMORY-IMAGE*
(dumpfile form)
* crash [OPTION]...
[NAMELIST]************
(live system form)
*
Enter "crash -h" for details.
*
Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #cat
README.txt
Kernel crashdump
----------------
*
Crash time**** : 2012-03-24
10:57 (+0000)
Kernel version :
2.6.27.19-5-default
Host**********
: sles11
Dump level**** : 0
Dump
format*** : compressed

but I use another command is OK, what's
wrong ?
*
Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57
#crash vmlinux-2.6.27.19-5-default System.map-2.6.27.19-5-default
vmcore
*
crash 6.0.4
Copyright (C) 2002-2012* Red Hat,
Inc.
Copyright (C) 2004, 2005, 2006* IBM Corporation
Copyright (C)
1999-2006* Hewlett-Packard Co
Copyright (C) 2005, 2006* Fujitsu
Limited
Copyright (C) 2006, 2007* VA Linux Systems Japan
K.K.
Copyright (C) 2005* NEC Corporation
Copyright (C) 1999, 2002,
2007* Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002*
Mission Critical Linux, Inc.
This program is free software, covered by the
GNU General Public License,
and you are welcome to change it and/or
distribute copies of it under
certain conditions.* Enter "help copying"
to see the conditions.
This program has absolutely no warranty.* Enter
"help warranty" for details.
*
GNU gdb (GDB) 7.3.1
Copyright (C) 2011 Free Software
Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This
is free software: you are free to change and redistribute it.
There is NO
WARRANTY, to the extent permitted by law.* Type "show copying"
and "show
warranty" for details.
This GDB was configured as
"x86_64-unknown-linux-gnu"...
*
* SYSTEM MAP: System.map-2.6.27.19-5-default
DEBUG
KERNEL: vmlinux-2.6.27.19-5-default (2.6.27.19-5-default)
***
DUMPFILE: vmcore
******* CPUS:
2
******* DATE: Sat Mar 24 18:57:31
2012
***** UPTIME: 00:13:30
LOAD AVERAGE: 0.27,
0.30, 0.28
****** TASKS:
190
*** NODENAME: sles11
**** RELEASE:
2.6.27.19-5-default
**** VERSION: #6 SMP Sat Nov 12
08:58:08 CST 2011
**** MACHINE: x86_64* (2393
Mhz)
***** MEMORY: 3.9
GB
****** PANIC: "SysRq : Trigger a
crashdump"
******** PID:
4700
**** COMMAND:
"bash"
******* TASK:
ffff88007f8f6080* [THREAD_INFO:
ffff88007f802000]
******** CPU:
0
****** STATE: TASK_RUNNING
(SYSRQ)
*
crash>

*
*




From: hujun_hotmail
Sent: Saturday, March 24, 2012 8:07 PM
To: crash mailist
Subject: [Crash-utility] crash can live analysis but vmcore
can't.


hello* experts:
*
My linux system : Suse SLES 11* , crash can live analysis, but can't
open vmcore generated by kdump.
*
why?
*
*
*
sles11:/mnt/sda12/crash/2012-03-24-18:57 #
crash

*
crash 6.0.4
Copyright (C) 2002-2012* Red Hat, Inc.
Copyright (C)
2004, 2005, 2006* IBM Corporation
Copyright (C) 1999-2006*
Hewlett-Packard Co
Copyright (C) 2005, 2006* Fujitsu
Limited
Copyright (C) 2006, 2007* VA Linux Systems Japan
K.K.
Copyright (C) 2005* NEC Corporation
Copyright (C) 1999, 2002,
2007* Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002*
Mission Critical Linux, Inc.
This program is free software, covered by the
GNU General Public License,
**** and you are welcome to
change it and/or distribute copies of it under
****
certain conditions.* Enter "help copying" to see the
conditions.
**** This program has absolutely no
warranty.* Enter "help warranty" for
details.
*****
**** GNU gdb
(GDB) 7.3.1
**** Copyright (C) 2011 Free Software
Foundation, Inc.
**** License GPLv3+: GNU GPL version 3
or later <http://gnu.org/licenses/gpl.html>
****
This is free software: you are free to change and redistribute
it.
**** There is NO WARRANTY, to the extent permitted by
law.* Type "show copying"
**** and "show warranty"
for details.
**** This GDB was configured as
"x86_64-unknown-linux-gnu"...
*
********** KERNEL:
/boot/vmlinuz-2.6.27.19-5-default.vmlinux-debug
**************
DUMPFILE:
/dev/mem
**********************
CPUS:
2
******************************
DATE: Sat Mar 24 19:10:45
2012
************************************
UPTIME:
00:07:17
************************************
LOAD AVERAGE: 0.16, 0.19,
0.12
*******************************************
TASKS:
189
***********************************************
NODENAME:
sles11
************************************************** **
RELEASE:
2.6.27.19-5-default
************************************************** *******
VERSION: #6 SMP Sat Nov 12 08:58:08 CST
2011
************************************************** ************
MACHINE: x86_64* (2393
Mhz)
********* MEMORY: 3.9
GB
******************
PID:
4590
***********************
COMMAND:
"crash"
*******************************
TASK: ffff8801230fa340* [THREAD_INFO:
ffff880131c86000]
****************************************
CPU:
0
***********************************************
STATE: TASK_RUNNING (ACTIVE)
*
*** crash>
***

*
*
*

sles11:/mnt/sda12/crash/2012-03-24-18:57 # crash
vmlinux-2.6.27.19-5-default.gz vmlinux-2.6.27.19-5-default.debug
vmcore
crash 6.0.4
Copyright (C) 2002-2012* Red Hat,
Inc.
Copyright (C) 2004, 2005, 2006* IBM Corporation
Copyright (C)
1999-2006* Hewlett-Packard Co
Copyright (C) 2005, 2006* Fujitsu
Limited
Copyright (C) 2006, 2007* VA Linux Systems Japan
K.K.
Copyright (C) 2005* NEC Corporation
Copyright (C) 1999, 2002,
2007* Silicon Graphics, Inc.
Copyright (C) 1999, 2000, 2001, 2002*
Mission Critical Linux, Inc.
This program is free software, covered by the
GNU General Public License,
and you are welcome to change it and/or
distribute copies of it under
certain conditions.* Enter "help copying"
to see the conditions.
This program has absolutely no warranty.* Enter
"help warranty" for details.
*
please wait... (uncompressing
vmlinux-2.6.27.19-5-default.gz)
************************************************** ***********

compressed kdump: header->utsname.machine:
compressed kdump: memory
bitmap offset: 2000
diskdump_data:

********* filename:
vmcore
************
flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED)

**************
dfd:
4
**************
ofp: 0
***** machine_type: 62 (EM_X86_64)
*
*********** header:
dee020
**********
signature: "KDUMP** "
*****
header_version:
1
************
utsname:
**************
sysname:

*************
nodename:

**************
release:

**************
version:

**************
machine:
***********
domainname:
**********
timestamp:
***************
tv_sec:
0
**************
tv_usec:
0
*************
status: 0 ()
*********
block_size: 4096
******* sub_hdr_size:
1
****** bitmap_blocks:
80
********** max_mapnr:
1294336
*** total_ram_blocks:
0
****** device_blocks:
0
***** written_blocks:
0
******** current_cpu:
0
************
nr_cpus: 1
***** tasks[nr_cpus]: 0
*
******* sub_header: 0 (n/a)
*
* sub_header_kdump: def030

********** phys_base:
0
********* dump_level: 0 (0x0)

*
****** data_offset:
52000
******* block_size:
4096
****** block_shift:
12
*********** bitmap:
7f812de69010
******* bitmap_len:
327680
** dumpable_bitmap:
7f812de18010
*************
byte:
0
**************
bit: 0
** compressed_page:
e00a40
******** curbufptr: 0
*
*page_cache_hdr[0]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df0a30
******* pg_hit_count:
0
*page_cache_hdr[1]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df1a30
******* pg_hit_count:
0
*page_cache_hdr[2]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df2a30
******* pg_hit_count:
0
*page_cache_hdr[3]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df3a30
******* pg_hit_count:
0
*page_cache_hdr[4]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df4a30
******* pg_hit_count:
0
*page_cache_hdr[5]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df5a30
******* pg_hit_count:
0
*page_cache_hdr[6]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df6a30
******* pg_hit_count:
0
*page_cache_hdr[7]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df7a30
******* pg_hit_count:
0
*page_cache_hdr[8]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df8a30
******* pg_hit_count:
0
*page_cache_hdr[9]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: df9a30
******* pg_hit_count:
0
page_cache_hdr[10]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: dfaa30
******* pg_hit_count:
0
page_cache_hdr[11]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: dfba30
******* pg_hit_count:
0
page_cache_hdr[12]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: dfca30
******* pg_hit_count:
0
page_cache_hdr[13]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: dfda30
******* pg_hit_count:
0
page_cache_hdr[14]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: dfea30
******* pg_hit_count:
0
page_cache_hdr[15]:
***********
pg_flags: 0
()
************
pg_addr: 0
**********
pg_bufptr: dffa30
******* pg_hit_count:
0
*
*** page_cache_buf:
df0a30
****** evict_index:
0
******** evictions:
0
********* accesses:
0
***** cached_reads: 0

****** valid_pages: df0040
readmem:
read_diskdump()
[?1034hcompressed kdump: phys_base: 0
gdb
vmlinux-2.6.27.19-5-default.debug
GNU gdb (GDB) 7.3.1
Copyright (C) 2011
Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://gnu.org/licenses/gpl.html>
This
is free software: you are free to change and redistribute it.
There is NO
WARRANTY, to the extent permitted by law.* Type "show copying"
and "show
warranty" for details.
This GDB was configured as
"x86_64-unknown-linux-gnu"...
GETBUF(248 -> 0)
* GETBUF(1500 ->
1)
*
please wait... (patching 35285 gdb minimal_symbol values)

************************************************** ********************

* FREEBUF(1)
FREEBUF(0)
<readmem: ffffffff804b8210, KVADDR,
"kernel_config_data", 32768, (ROE), 17e92b0>
<read_diskdump: addr:
ffffffff804b8210 paddr: 4b8210 cnt: 3568>
read_diskdump: paddr/pfn:
4b8210/4b8 -> cache physical page: 4b8000
<read_diskdump: addr:
ffffffff804b9000 paddr: 4b9000 cnt: 4096>
read_diskdump: paddr/pfn:
4b9000/4b9 -> cache physical page: 4b9000
<read_diskdump: addr:
ffffffff804ba000 paddr: 4ba000 cnt: 4096>
read_diskdump: paddr/pfn:
4ba000/4ba -> cache physical page: 4ba000
<read_diskdump: addr:
ffffffff804bb000 paddr: 4bb000 cnt: 4096>
read_diskdump: paddr/pfn:
4bb000/4bb -> cache physical page: 4bb000
<read_diskdump: addr:
ffffffff804bc000 paddr: 4bc000 cnt: 4096>
read_diskdump: paddr/pfn:
4bc000/4bc -> cache physical page: 4bc000
<read_diskdump: addr:
ffffffff804bd000 paddr: 4bd000 cnt: 4096>
read_diskdump: paddr/pfn:
4bd000/4bd -> cache physical page: 4bd000
<read_diskdump: addr:
ffffffff804be000 paddr: 4be000 cnt: 4096>
read_diskdump: paddr/pfn:
4be000/4be -> cache physical page: 4be000
<read_diskdump: addr:
ffffffff804bf000 paddr: 4bf000 cnt: 4096>
read_diskdump: paddr/pfn:
4bf000/4bf -> cache physical page: 4bf000
<read_diskdump: addr:
ffffffff804c0000 paddr: 4c0000 cnt: 528>
read_diskdump: paddr/pfn:
4c0000/4c0 -> cache physical page: 4c0000
WARNING: could not find
MAGIC_START!
GETBUF(248 -> 0)
FREEBUF(0)
GETBUF(64 ->
0)
<readmem: ffffffff80a59910, KVADDR, "possible", 64, (ROE),
da00e0>
<read_diskdump: addr: ffffffff80a59910 paddr: a59910 cnt:
64>
read_diskdump: paddr/pfn: a59910/a59 -> cache physical page:
a59000
cpu_possible_map:
<readmem: ffffffff8095d950, KVADDR,
"present", 64, (ROE), da00e0>
<read_diskdump: addr: ffffffff8095d950
paddr: 95d950 cnt: 64>
read_diskdump: paddr/pfn: 95d950/95d -> cache
physical page: 95d000
cpu_present_map: 2 3 6 7 10 11 14 15 18 19 22 23 26 27
30 31 34 35 38 39 42 43 46 47 50 51 54 55 58 59 62 63 66 67 70 71 74 75 78 79 82
83 86 87 90 91 94 95 98 99 102 103 106 107 110 111 114 115 118 119 122 123 126
127 130 131 134 135 138 139 142 143 146 147 150 151 154 155 158 159 162 163 166
167 170 171 174 175 178 179 182 183 186 187 190 191 194 195 198 199 202 203 206
207 210 211 214 215 218 219 222 223 226 227 230 231 234 235 238 239 242 243 246
247 250 251 254 255 258 259 262 263 266 267 270 271 274 275 278 279 282 283 286
287 290 291 294 295 298 299 302 303 306 307 310 311 314 315 318 319 322 323 326
327 330 331 334 335 338 339 342 343 346 347 350 351 354 355 358 359 362 363 366
367 370 371 374 375 378 379 382 383 386 387 390 391 394 395 398 399 402 403 406
407 410 411 414 415 418 419 422 423 426 427 430 431 434 435 438 439 442 443 446
447 450 451 454 455 458 459 462 463 466 467 470 471 474 475 478 479 482 483 486
487 490 491 494 495 498 499 502 503 506 507 510 511
<readmem:
ffffffff808cbe90, KVADDR, "online", 64, (ROE), da00e0>
<read_diskdump:
addr: ffffffff808cbe90 paddr: 8cbe90 cnt: 64>
read_diskdump: paddr/pfn:
8cbe90/8cb -> cache physical page: 8cb000
cpu_online_map:

FREEBUF(0)
<readmem: ffffffff80abaf00, KVADDR, "xtime", 16, (FOE),
c5e430>
<read_diskdump: addr: ffffffff80abaf00 paddr: abaf00 cnt:
16>
read_diskdump: paddr/pfn: abaf00/aba -> cache physical page:
aba000
<readmem: ffffffff806dc894, KVADDR, "init_uts_ns", 390, (ROE),
c5ea1c>
<read_diskdump: addr: ffffffff806dc894 paddr: 6dc894 cnt:
390>
read_diskdump: paddr/pfn: 6dc894/6dc -> cache physical page:
6dc000
base kernel version: 0.0.0
<readmem: ffffffff804a7000, KVADDR,
"accessible check", 8, (ROE|Q), 7fff35eeba88>
<read_diskdump: addr:
ffffffff804a7000 paddr: 4a7000 cnt: 8>
read_diskdump: paddr/pfn:
4a7000/4a7 -> cache physical page: 4a7000
<readmem: ffffffff804a7000,
KVADDR, "readstring characters", 1499, (ROE|Q),
7fff35eeaa70>
<read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt:
1499>
read_diskdump: paddr/pfn: 4a7000/4a7 -> physical page is cached:
4a7000
linux_banner:
?;??
crash: vmlinux-2.6.27.19-5-default.debug
and vmcore do not match!
*
Usage:
*
* crash [OPTION]... NAMELIST MEMORY-IMAGE* (dumpfile
form)
* crash [OPTION]...
[NAMELIST]************
(live system form)
*
Enter "crash -h" for details.
*
*
*






--
Crash-utility mailing
list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 
Old 03-26-2012, 02:06 PM
Dave Anderson
 
Default crash can live analysis but vmcore can't.

Whenever a crash session fails with a "do not match" message
such as this:

crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!

it is because the virtual addresses of the kernel symbols in the
vmlinux file do not match the virtual addresses of the kernel that
crashed.

When you run with "crash -d<number>" you will see the first few
attempts to gather kernel data (kernel_config_data, xtime, the
cpu_possible, cpu_present and cpu_online maps, the utsname structure,
and then the "linux_banner" string. The linux_banner string is what
you would see in /proc/version on the live system, for example:

# cat /proc/version
Linux version 2.6.32-220.el6.x86_64 (mockbuild@x86-004.build.bos.redhat.com) (gcc version 4.4.5 20110214 (Red Hat 4.4.5-6) (GCC) ) #1 SMP Wed Nov 9 08:03:13 EST 2011
#

If the crash utility does not find the correct string at the address
specified in the vmlinux file, then it makes no sense to continue,
and so it quits with the "do not match" message.

Accordingly, one of your examples ends like this, where the
vmlinux file indicates that the linux_banner string is located
at kernel virtual address ffffffff804a7000. That address was
translated to physical address 4a7000, and then read from the
compressed kdump vmcore. But the linux_banner string read from
that address in the vmcore contains garbage:

...
<readmem: ffffffff804a7000, KVADDR, "readstring characters", 1499, (ROE|Q), 7fff35eeaa70>
<read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 1499>
read_diskdump: paddr/pfn: 4a7000/4a7 -> physical page is cached: 4a7000
linux_banner:
�x;�����
crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
...

I cannot answer for SUSE, but if I am not mistaken, they build a
debug kernel (i.e, built with -g) for their distrobution kernels
after-the-fact, and when that is done, the kernel virtual addresses
are not exactly the same. To work around that, you can pass the
System.map file of the actual crashed kernel -- which has the *correct"
kernel virtual addresses -- on the crash command line. And when
you did that, things apparently worked correctly.

In other words, this failed:

# crash vmlinux-2.6.27.19-5-default vmlinux-2.6.27.19-5-default.debug vmcore

But this worked:

# crash vmlinux-2.6.27.19-5-default System.map-2.6.27.19-5-default vmcore

And it worked because the crash utility takes the (correct) symbol values
found in the System.map file, and "patches" the symbol values that were
initially read from the vmlinux file(s).

Complicating matters a bit is the usage of a vmlinux file and its associated
vmlinux.debug file. In old RHEL3 kernels, Red Hat would split the vmlinux
file into its stripped version and a .debug version, the same way as is
done with user-space programs. But from RHEL4 onwards, the that was no longer
done, and there is a single vmlinux file that contains all the debuginfo
data, and which is the same kernel as the distribution kernel. In SLES kernels,
I am not sure whether that was true in the example above, or whether the .debug
kernel is the kernel that they build after-the-fact?

In any case, I note that there a several novell.com addresses that are cc'd
with this email, and there are many SLES users and Novell/Suse employees
on this list who can give you the specifics of how they do things.

Dave





----- Original Message -----
>
>
> on another machine,I use all kinds of crash command :
>
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #crash
> vmlinux-2.6.27.19-5-default.gz System.map-2.6.27.19-5-default vmcore
>
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
>
> crash: /var/tmp/vmlinux-2.6.27.19-5-default.gz_RYx9JC: no debugging
> data available
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #gunzip vmlinux-2.
> 6.27.19-5-default.gz
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 # crash
> vmlinux-2.6.27.19-5-default vmlinux-2.6.27.19-5-default.debug vmcore
>
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
>
> WARNING: could not find MAGIC_START!
> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
>
> Usage:
>
> crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
> crash [OPTION]... [NAMELIST] (live system form)
>
> Enter "crash -h" for details.
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #cat README.txt
> Kernel crashdump
> ----------------
>
> Crash time : 2012-03-24 10:57 (+0000)
> Kernel version : 2.6.27.19-5-default
> Host : sles11
> Dump level : 0
> Dump format : compressed
>
> but I use another command is OK, what's wrong ?
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #crash
> vmlinux-2.6.27.19-5-default System.map-2.6.27.19-5-default vmcore
>
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
>
> SYSTEM MAP: System.map-2.6.27.19-5-default
> DEBUG KERNEL: vmlinux-2.6.27.19-5-default (2.6.27.19-5-default)
> DUMPFILE: vmcore
> CPUS: 2
> DATE: Sat Mar 24 18:57:31 2012
> UPTIME: 00:13:30
> LOAD AVERAGE: 0.27, 0.30, 0.28
> TASKS: 190
> NODENAME: sles11
> RELEASE: 2.6.27.19-5-default
> VERSION: #6 SMP Sat Nov 12 08:58:08 CST 2011
> MACHINE: x86_64 (2393 Mhz)
> MEMORY: 3.9 GB
> PANIC: "SysRq : Trigger a crashdump"
> PID: 4700
> COMMAND: "bash"
> TASK: ffff88007f8f6080 [THREAD_INFO: ffff88007f802000]
> CPU: 0
> STATE: TASK_RUNNING (SYSRQ)
>
> crash>
>
>
>
>
>
>
>
> From: hujun_hotmail
> Sent: Saturday, March 24, 2012 8:07 PM
> To: crash mailist
> Subject: [Crash-utility] crash can live analysis but vmcore can't.
>
>
> hello experts:
>
> My linux system : Suse SLES 11 , crash can live analysis, but can't
> open vmcore generated by kdump.
>
> why?
>
>
>
> sles11:/mnt/sda12/crash/2012-03-24-18:57 # crash
>
>
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
>
> KERNEL: /boot/vmlinuz-2.6.27.19-5-default.vmlinux-debug
> DUMPFILE: /dev/mem
> CPUS: 2
> DATE: Sat Mar 24 19:10:45 2012
> UPTIME: 00:07:17
> LOAD AVERAGE: 0.16, 0.19, 0.12
> TASKS: 189
> NODENAME: sles11
> RELEASE: 2.6.27.19-5-default
> VERSION: #6 SMP Sat Nov 12 08:58:08 CST 2011
> MACHINE: x86_64 (2393 Mhz)
> MEMORY: 3.9 GB
> PID: 4590
> COMMAND: "crash"
> TASK: ffff8801230fa340 [THREAD_INFO: ffff880131c86000]
> CPU: 0
> STATE: TASK_RUNNING (ACTIVE)
>
> crash>
>
>
>
>
>
>
> sles11:/mnt/sda12/crash/2012-03-24-18:57 # crash
> vmlinux-2.6.27.19-5-default.gz vmlinux-2.6.27.19-5-default.debug
> vmcore
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> please wait... (uncompressing vmlinux-2.6.27.19-5-default.gz)
>
> compressed kdump: header->utsname.machine:
> compressed kdump: memory bitmap offset: 2000
> diskdump_data:
> filename: vmcore
> flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED)
> dfd: 4
> ofp: 0
> machine_type: 62 (EM_X86_64)
>
> header: dee020
> signature: "KDUMP "
> header_version: 1
> utsname:
> sysname:
> nodename:
> release:
> version:
> machine:
> domainname:
> timestamp:
> tv_sec: 0
> tv_usec: 0
> status: 0 ()
> block_size: 4096
> sub_hdr_size: 1
> bitmap_blocks: 80
> max_mapnr: 1294336
> total_ram_blocks: 0
> device_blocks: 0
> written_blocks: 0
> current_cpu: 0
> nr_cpus: 1
> tasks[nr_cpus]: 0
>
> sub_header: 0 (n/a)
>
> sub_header_kdump: def030
> phys_base: 0
> dump_level: 0 (0x0)
>
> data_offset: 52000
> block_size: 4096
> block_shift: 12
> bitmap: 7f812de69010
> bitmap_len: 327680
> dumpable_bitmap: 7f812de18010
> byte: 0
> bit: 0
> compressed_page: e00a40
> curbufptr: 0
>
> page_cache_hdr[0]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df0a30
> pg_hit_count: 0
> page_cache_hdr[1]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df1a30
> pg_hit_count: 0
> page_cache_hdr[2]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df2a30
> pg_hit_count: 0
> page_cache_hdr[3]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df3a30
> pg_hit_count: 0
> page_cache_hdr[4]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df4a30
> pg_hit_count: 0
> page_cache_hdr[5]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df5a30
> pg_hit_count: 0
> page_cache_hdr[6]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df6a30
> pg_hit_count: 0
> page_cache_hdr[7]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df7a30
> pg_hit_count: 0
> page_cache_hdr[8]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df8a30
> pg_hit_count: 0
> page_cache_hdr[9]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df9a30
> pg_hit_count: 0
> page_cache_hdr[10]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfaa30
> pg_hit_count: 0
> page_cache_hdr[11]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfba30
> pg_hit_count: 0
> page_cache_hdr[12]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfca30
> pg_hit_count: 0
> page_cache_hdr[13]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfda30
> pg_hit_count: 0
> page_cache_hdr[14]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfea30
> pg_hit_count: 0
> page_cache_hdr[15]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dffa30
> pg_hit_count: 0
>
> page_cache_buf: df0a30
> evict_index: 0
> evictions: 0
> accesses: 0
> cached_reads: 0
> valid_pages: df0040
> readmem: read_diskdump()
> [?1034hcompressed kdump: phys_base: 0
> gdb vmlinux-2.6.27.19-5-default.debug
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
> GETBUF(248 -> 0)
> GETBUF(1500 -> 1)
>
> please wait... (patching 35285 gdb minimal_symbol values)
>
> FREEBUF(1)
> FREEBUF(0)
> <readmem: ffffffff804b8210, KVADDR, "kernel_config_data", 32768,
> (ROE), 17e92b0>
> <read_diskdump: addr: ffffffff804b8210 paddr: 4b8210 cnt: 3568>
> read_diskdump: paddr/pfn: 4b8210/4b8 -> cache physical page: 4b8000
> <read_diskdump: addr: ffffffff804b9000 paddr: 4b9000 cnt: 4096>
> read_diskdump: paddr/pfn: 4b9000/4b9 -> cache physical page: 4b9000
> <read_diskdump: addr: ffffffff804ba000 paddr: 4ba000 cnt: 4096>
> read_diskdump: paddr/pfn: 4ba000/4ba -> cache physical page: 4ba000
> <read_diskdump: addr: ffffffff804bb000 paddr: 4bb000 cnt: 4096>
> read_diskdump: paddr/pfn: 4bb000/4bb -> cache physical page: 4bb000
> <read_diskdump: addr: ffffffff804bc000 paddr: 4bc000 cnt: 4096>
> read_diskdump: paddr/pfn: 4bc000/4bc -> cache physical page: 4bc000
> <read_diskdump: addr: ffffffff804bd000 paddr: 4bd000 cnt: 4096>
> read_diskdump: paddr/pfn: 4bd000/4bd -> cache physical page: 4bd000
> <read_diskdump: addr: ffffffff804be000 paddr: 4be000 cnt: 4096>
> read_diskdump: paddr/pfn: 4be000/4be -> cache physical page: 4be000
> <read_diskdump: addr: ffffffff804bf000 paddr: 4bf000 cnt: 4096>
> read_diskdump: paddr/pfn: 4bf000/4bf -> cache physical page: 4bf000
> <read_diskdump: addr: ffffffff804c0000 paddr: 4c0000 cnt: 528>
> read_diskdump: paddr/pfn: 4c0000/4c0 -> cache physical page: 4c0000
> WARNING: could not find MAGIC_START!
> GETBUF(248 -> 0)
> FREEBUF(0)
> GETBUF(64 -> 0)
> <readmem: ffffffff80a59910, KVADDR, "possible", 64, (ROE), da00e0>
> <read_diskdump: addr: ffffffff80a59910 paddr: a59910 cnt: 64>
> read_diskdump: paddr/pfn: a59910/a59 -> cache physical page: a59000
> cpu_possible_map:
> <readmem: ffffffff8095d950, KVADDR, "present", 64, (ROE), da00e0>
> <read_diskdump: addr: ffffffff8095d950 paddr: 95d950 cnt: 64>
> read_diskdump: paddr/pfn: 95d950/95d -> cache physical page: 95d000
> cpu_present_map: 2 3 6 7 10 11 14 15 18 19 22 23 26 27 30 31 34 35 38
> 39 42 43 46 47 50 51 54 55 58 59 62 63 66 67 70 71 74 75 78 79 82 83
> 86 87 90 91 94 95 98 99 102 103 106 107 110 111 114 115 118 119 122
> 123 126 127 130 131 134 135 138 139 142 143 146 147 150 151 154 155
> 158 159 162 163 166 167 170 171 174 175 178 179 182 183 186 187 190
> 191 194 195 198 199 202 203 206 207 210 211 214 215 218 219 222 223
> 226 227 230 231 234 235 238 239 242 243 246 247 250 251 254 255 258
> 259 262 263 266 267 270 271 274 275 278 279 282 283 286 287 290 291
> 294 295 298 299 302 303 306 307 310 311 314 315 318 319 322 323 326
> 327 330 331 334 335 338 339 342 343 346 347 350 351 354 355 358 359
> 362 363 366 367 370 371 374 375 378 379 382 383 386 387 390 391 394
> 395 398 399 402 403 406 407 410 411 414 415 418 419 422 423 426 427
> 430 431 434 435 438 439 442 443 446 447 450 451 454 455 458 459 462
> 463 466 467 470 471 474 475 478 479 482 483 486 487 490 491 494 495
> 498 499 502 503 506 507 510 511
> <readmem: ffffffff808cbe90, KVADDR, "online", 64, (ROE), da00e0>
> <read_diskdump: addr: ffffffff808cbe90 paddr: 8cbe90 cnt: 64>
> read_diskdump: paddr/pfn: 8cbe90/8cb -> cache physical page: 8cb000
> cpu_online_map:
> FREEBUF(0)
> <readmem: ffffffff80abaf00, KVADDR, "xtime", 16, (FOE), c5e430>
> <read_diskdump: addr: ffffffff80abaf00 paddr: abaf00 cnt: 16>
> read_diskdump: paddr/pfn: abaf00/aba -> cache physical page: aba000
> <readmem: ffffffff806dc894, KVADDR, "init_uts_ns", 390, (ROE),
> c5ea1c>
> <read_diskdump: addr: ffffffff806dc894 paddr: 6dc894 cnt: 390>
> read_diskdump: paddr/pfn: 6dc894/6dc -> cache physical page: 6dc000
> base kernel version: 0.0.0
> <readmem: ffffffff804a7000, KVADDR, "accessible check", 8, (ROE|Q),
> 7fff35eeba88>
> <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 8>
> read_diskdump: paddr/pfn: 4a7000/4a7 -> cache physical page: 4a7000
> <readmem: ffffffff804a7000, KVADDR, "readstring characters", 1499,
> (ROE|Q), 7fff35eeaa70>
> <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 1499>
> read_diskdump: paddr/pfn: 4a7000/4a7 -> physical page is cached:
> 4a7000
> linux_banner:
> 紉;���
> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
>
> Usage:
>
> crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
> crash [OPTION]... [NAMELIST] (live system form)
>
> Enter "crash -h" for details.
>
>
>
>
>
>
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 
Old 03-26-2012, 04:26 PM
"Wei Kong"
 
Default crash can live analysis but vmcore can't.

Hi Dave,
**
Thank you for your detail explain.
It's very useful for us whom doesn't know what's in .debug file.
**
I have another question about crash, it's trouble a while.
**
Below call trace and register*is system call of poll.*
**
int poll(struct pollfd fds[], nfds_t nfds, int timeout);
****
We could know that RDI is fds, RSI is nfds=1, timeout=RDX=10hours
But before copy_from_user, fds is userspace address, how can we read it or*get the content from this address.*
*****
*#3 [ffff810028d51f60] sys_poll at ffffffff8019a64a
*** ffff810028d51f68: 0000000000002000 0000000000000000
*** ffff810028d51f78: 0000000002255100 ffffffff8010ae36
*#4 [ffff810028d51f80] system_call at ffffffff8010ae36
*** RIP: 00002b8543aa3dc6* RSP: 00007ffff6676478* RFLAGS: 00000246
*** RAX: 0000000000000007* RBX: ffffffff8010ae36* RCX: 0000000000000000
*** RDX: 0000000002255100* RSI: 0000000000000001* RDI: 00007ffff66760b0
*** RBP: 0000000002255100** R8: 0000000000000001** R9: 00002aaaaab35cf0
*** R10: 00002aaaab2ff9b0* R11: 0000000000000202* R12: 0000000000000000
*** R13: 0000000000002000* R14: ffffffff8019a64a* R15: 00002aaaaab36970
*** ORIG_RAX: 0000000000000007* CS: 0033* SS: 002b
*
It seems gcore could dump the task memory on latest crash., but on old crash and os,
how could we do it.
**
Thank you so much
Kong Wei
*
>>> Dave Anderson <anderson@redhat.com> 2012/3/26 22:06 >>>


Whenever a crash session fails with a "do not match" message
such as this:

* crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!

it is because the virtual addresses of the kernel symbols in the
vmlinux file do not match the virtual addresses of the kernel that
crashed.

When you run with "crash -d<number>" you will see the first few
attempts to gather kernel data (kernel_config_data, xtime, the
cpu_possible, cpu_present and cpu_online maps, the utsname structure,
and then the "linux_banner" string.* The linux_banner string is what
you would see in /proc/version on the live system, for example:

# cat /proc/version
Linux version 2.6.32-220.el6.x86_64 (mockbuild@x86-004.build.bos.redhat.com) (gcc version 4.4.5 20110214 (Red Hat 4.4.5-6) (GCC) ) #1 SMP Wed Nov 9 08:03:13 EST 2011
#

If the crash utility does not find the correct string at the address
specified in the vmlinux file, then it makes no sense to continue,
and so it quits with the "do not match" message.

Accordingly, one of your examples ends like this, where the
vmlinux file indicates that the linux_banner string is located
at kernel virtual address ffffffff804a7000.* That address was
translated to physical address 4a7000, and then read from the
compressed kdump vmcore.* But the linux_banner string read from
that address in the vmcore contains garbage:

...
<readmem: ffffffff804a7000, KVADDR, "readstring characters", 1499, (ROE|Q), 7fff35eeaa70>
<read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 1499>
read_diskdump: paddr/pfn: 4a7000/4a7 -> physical page is cached: 4a7000
linux_banner:
�x;�����
crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
...

I cannot answer for SUSE, but if I am not mistaken, they build a
debug kernel (i.e, built with -g) for their distrobution kernels
after-the-fact, and when that is done, the kernel virtual addresses
are not exactly the same.* To work around that, you can pass the
System.map file of the actual crashed kernel -- which has the *correct"
kernel virtual addresses -- on the crash command line.* And when
you did that, things apparently worked correctly.

In other words, this failed:

# crash vmlinux-2.6.27.19-5-default vmlinux-2.6.27.19-5-default.debug vmcore

But this worked:

# crash vmlinux-2.6.27.19-5-default System.map-2.6.27.19-5-default vmcore

And it worked because the crash utility takes the (correct) symbol values
found in the System.map file, and "patches" the symbol values that were
initially read from the vmlinux file(s).

Complicating matters a bit is the usage of a vmlinux file and its associated
vmlinux.debug file.* In old RHEL3 kernels, Red Hat would split the vmlinux
file into its stripped version and a .debug version, the same way as is
done with user-space programs.* But from RHEL4 onwards, the that was no longer
done, and there is a single vmlinux file that contains all the debuginfo
data, and which is the same kernel as the distribution kernel.* In SLES kernels,
I am not sure whether that was true in the example above, or whether the .debug
kernel is the kernel that they build after-the-fact?

In any case, I note that there a several novell.com addresses that are cc'd
with this email, and there are many SLES users and Novell/Suse employees
on this list who can give you the specifics of how they do things.

Dave





----- Original Message -----
>
>
> on another machine,I use all kinds of crash command :
>
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #crash
> vmlinux-2.6.27.19-5-default.gz System.map-2.6.27.19-5-default vmcore
>
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
>
> crash: /var/tmp/vmlinux-2.6.27.19-5-default.gz_RYx9JC: no debugging
> data available
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #gunzip vmlinux-2.
> 6.27.19-5-default.gz
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 # crash
> vmlinux-2.6.27.19-5-default vmlinux-2.6.27.19-5-default.debug vmcore
>
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
>
> WARNING: could not find MAGIC_START!
> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
>
> Usage:
>
> crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
> crash [OPTION]... [NAMELIST] (live system form)
>
> Enter "crash -h" for details.
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #cat README.txt
> Kernel crashdump
> ----------------
>
> Crash time : 2012-03-24 10:57 (+0000)
> Kernel version : 2.6.27.19-5-default
> Host : sles11
> Dump level : 0
> Dump format : compressed
>
> but I use another command is OK, what's wrong ?
>
> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #crash
> vmlinux-2.6.27.19-5-default System.map-2.6.27.19-5-default vmcore
>
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
>
> SYSTEM MAP: System.map-2.6.27.19-5-default
> DEBUG KERNEL: vmlinux-2.6.27.19-5-default (2.6.27.19-5-default)
> DUMPFILE: vmcore
> CPUS: 2
> DATE: Sat Mar 24 18:57:31 2012
> UPTIME: 00:13:30
> LOAD AVERAGE: 0.27, 0.30, 0.28
> TASKS: 190
> NODENAME: sles11
> RELEASE: 2.6.27.19-5-default
> VERSION: #6 SMP Sat Nov 12 08:58:08 CST 2011
> MACHINE: x86_64 (2393 Mhz)
> MEMORY: 3.9 GB
> PANIC: "SysRq : Trigger a crashdump"
> PID: 4700
> COMMAND: "bash"
> TASK: ffff88007f8f6080 [THREAD_INFO: ffff88007f802000]
> CPU: 0
> STATE: TASK_RUNNING (SYSRQ)
>
> crash>
>
>
>
>
>
>
>
> From: hujun_hotmail
> Sent: Saturday, March 24, 2012 8:07 PM
> To: crash mailist
> Subject: [Crash-utility] crash can live analysis but vmcore can't.
>
>
> hello experts:
>
> My linux system : Suse SLES 11 , crash can live analysis, but can't
> open vmcore generated by kdump.
>
> why?
>
>
>
> sles11:/mnt/sda12/crash/2012-03-24-18:57 # crash
>
>
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
>
> KERNEL: /boot/vmlinuz-2.6.27.19-5-default.vmlinux-debug
> DUMPFILE: /dev/mem
> CPUS: 2
> DATE: Sat Mar 24 19:10:45 2012
> UPTIME: 00:07:17
> LOAD AVERAGE: 0.16, 0.19, 0.12
> TASKS: 189
> NODENAME: sles11
> RELEASE: 2.6.27.19-5-default
> VERSION: #6 SMP Sat Nov 12 08:58:08 CST 2011
> MACHINE: x86_64 (2393 Mhz)
> MEMORY: 3.9 GB
> PID: 4590
> COMMAND: "crash"
> TASK: ffff8801230fa340 [THREAD_INFO: ffff880131c86000]
> CPU: 0
> STATE: TASK_RUNNING (ACTIVE)
>
> crash>
>
>
>
>
>
>
> sles11:/mnt/sda12/crash/2012-03-24-18:57 # crash
> vmlinux-2.6.27.19-5-default.gz vmlinux-2.6.27.19-5-default.debug
> vmcore
> crash 6.0.4
> Copyright (C) 2002-2012 Red Hat, Inc.
> Copyright (C) 2004, 2005, 2006 IBM Corporation
> Copyright (C) 1999-2006 Hewlett-Packard Co
> Copyright (C) 2005, 2006 Fujitsu Limited
> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> Copyright (C) 2005 NEC Corporation
> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> This program is free software, covered by the GNU General Public
> License,
> and you are welcome to change it and/or distribute copies of it under
> certain conditions. Enter "help copying" to see the conditions.
> This program has absolutely no warranty. Enter "help warranty" for
> details.
>
> please wait... (uncompressing vmlinux-2.6.27.19-5-default.gz)
>
> compressed kdump: header->utsname.machine:
> compressed kdump: memory bitmap offset: 2000
> diskdump_data:
> filename: vmcore
> flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED)
> dfd: 4
> ofp: 0
> machine_type: 62 (EM_X86_64)
>
> header: dee020
> signature: "KDUMP "
> header_version: 1
> utsname:
> sysname:
> nodename:
> release:
> version:
> machine:
> domainname:
> timestamp:
> tv_sec: 0
> tv_usec: 0
> status: 0 ()
> block_size: 4096
> sub_hdr_size: 1
> bitmap_blocks: 80
> max_mapnr: 1294336
> total_ram_blocks: 0
> device_blocks: 0
> written_blocks: 0
> current_cpu: 0
> nr_cpus: 1
> tasks[nr_cpus]: 0
>
> sub_header: 0 (n/a)
>
> sub_header_kdump: def030
> phys_base: 0
> dump_level: 0 (0x0)
>
> data_offset: 52000
> block_size: 4096
> block_shift: 12
> bitmap: 7f812de69010
> bitmap_len: 327680
> dumpable_bitmap: 7f812de18010
> byte: 0
> bit: 0
> compressed_page: e00a40
> curbufptr: 0
>
> page_cache_hdr[0]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df0a30
> pg_hit_count: 0
> page_cache_hdr[1]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df1a30
> pg_hit_count: 0
> page_cache_hdr[2]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df2a30
> pg_hit_count: 0
> page_cache_hdr[3]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df3a30
> pg_hit_count: 0
> page_cache_hdr[4]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df4a30
> pg_hit_count: 0
> page_cache_hdr[5]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df5a30
> pg_hit_count: 0
> page_cache_hdr[6]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df6a30
> pg_hit_count: 0
> page_cache_hdr[7]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df7a30
> pg_hit_count: 0
> page_cache_hdr[8]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df8a30
> pg_hit_count: 0
> page_cache_hdr[9]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: df9a30
> pg_hit_count: 0
> page_cache_hdr[10]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfaa30
> pg_hit_count: 0
> page_cache_hdr[11]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfba30
> pg_hit_count: 0
> page_cache_hdr[12]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfca30
> pg_hit_count: 0
> page_cache_hdr[13]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfda30
> pg_hit_count: 0
> page_cache_hdr[14]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dfea30
> pg_hit_count: 0
> page_cache_hdr[15]:
> pg_flags: 0 ()
> pg_addr: 0
> pg_bufptr: dffa30
> pg_hit_count: 0
>
> page_cache_buf: df0a30
> evict_index: 0
> evictions: 0
> accesses: 0
> cached_reads: 0
> valid_pages: df0040
> readmem: read_diskdump()
> [?1034hcompressed kdump: phys_base: 0
> gdb vmlinux-2.6.27.19-5-default.debug
> GNU gdb (GDB) 7.3.1
> Copyright (C) 2011 Free Software Foundation, Inc.
> License GPLv3+: GNU GPL version 3 or later <
> http://gnu.org/licenses/gpl.html >
> This is free software: you are free to change and redistribute it.
> There is NO WARRANTY, to the extent permitted by law. Type "show
> copying"
> and "show warranty" for details.
> This GDB was configured as "x86_64-unknown-linux-gnu"...
> GETBUF(248 -> 0)
> GETBUF(1500 -> 1)
>
> please wait... (patching 35285 gdb minimal_symbol values)
>
> FREEBUF(1)
> FREEBUF(0)
> <readmem: ffffffff804b8210, KVADDR, "kernel_config_data", 32768,
> (ROE), 17e92b0>
> <read_diskdump: addr: ffffffff804b8210 paddr: 4b8210 cnt: 3568>
> read_diskdump: paddr/pfn: 4b8210/4b8 -> cache physical page: 4b8000
> <read_diskdump: addr: ffffffff804b9000 paddr: 4b9000 cnt: 4096>
> read_diskdump: paddr/pfn: 4b9000/4b9 -> cache physical page: 4b9000
> <read_diskdump: addr: ffffffff804ba000 paddr: 4ba000 cnt: 4096>
> read_diskdump: paddr/pfn: 4ba000/4ba -> cache physical page: 4ba000
> <read_diskdump: addr: ffffffff804bb000 paddr: 4bb000 cnt: 4096>
> read_diskdump: paddr/pfn: 4bb000/4bb -> cache physical page: 4bb000
> <read_diskdump: addr: ffffffff804bc000 paddr: 4bc000 cnt: 4096>
> read_diskdump: paddr/pfn: 4bc000/4bc -> cache physical page: 4bc000
> <read_diskdump: addr: ffffffff804bd000 paddr: 4bd000 cnt: 4096>
> read_diskdump: paddr/pfn: 4bd000/4bd -> cache physical page: 4bd000
> <read_diskdump: addr: ffffffff804be000 paddr: 4be000 cnt: 4096>
> read_diskdump: paddr/pfn: 4be000/4be -> cache physical page: 4be000
> <read_diskdump: addr: ffffffff804bf000 paddr: 4bf000 cnt: 4096>
> read_diskdump: paddr/pfn: 4bf000/4bf -> cache physical page: 4bf000
> <read_diskdump: addr: ffffffff804c0000 paddr: 4c0000 cnt: 528>
> read_diskdump: paddr/pfn: 4c0000/4c0 -> cache physical page: 4c0000
> WARNING: could not find MAGIC_START!
> GETBUF(248 -> 0)
> FREEBUF(0)
> GETBUF(64 -> 0)
> <readmem: ffffffff80a59910, KVADDR, "possible", 64, (ROE), da00e0>
> <read_diskdump: addr: ffffffff80a59910 paddr: a59910 cnt: 64>
> read_diskdump: paddr/pfn: a59910/a59 -> cache physical page: a59000
> cpu_possible_map:
> <readmem: ffffffff8095d950, KVADDR, "present", 64, (ROE), da00e0>
> <read_diskdump: addr: ffffffff8095d950 paddr: 95d950 cnt: 64>
> read_diskdump: paddr/pfn: 95d950/95d -> cache physical page: 95d000
> cpu_present_map: 2 3 6 7 10 11 14 15 18 19 22 23 26 27 30 31 34 35 38
> 39 42 43 46 47 50 51 54 55 58 59 62 63 66 67 70 71 74 75 78 79 82 83
> 86 87 90 91 94 95 98 99 102 103 106 107 110 111 114 115 118 119 122
> 123 126 127 130 131 134 135 138 139 142 143 146 147 150 151 154 155
> 158 159 162 163 166 167 170 171 174 175 178 179 182 183 186 187 190
> 191 194 195 198 199 202 203 206 207 210 211 214 215 218 219 222 223
> 226 227 230 231 234 235 238 239 242 243 246 247 250 251 254 255 258
> 259 262 263 266 267 270 271 274 275 278 279 282 283 286 287 290 291
> 294 295 298 299 302 303 306 307 310 311 314 315 318 319 322 323 326
> 327 330 331 334 335 338 339 342 343 346 347 350 351 354 355 358 359
> 362 363 366 367 370 371 374 375 378 379 382 383 386 387 390 391 394
> 395 398 399 402 403 406 407 410 411 414 415 418 419 422 423 426 427
> 430 431 434 435 438 439 442 443 446 447 450 451 454 455 458 459 462
> 463 466 467 470 471 474 475 478 479 482 483 486 487 490 491 494 495
> 498 499 502 503 506 507 510 511
> <readmem: ffffffff808cbe90, KVADDR, "online", 64, (ROE), da00e0>
> <read_diskdump: addr: ffffffff808cbe90 paddr: 8cbe90 cnt: 64>
> read_diskdump: paddr/pfn: 8cbe90/8cb -> cache physical page: 8cb000
> cpu_online_map:
> FREEBUF(0)
> <readmem: ffffffff80abaf00, KVADDR, "xtime", 16, (FOE), c5e430>
> <read_diskdump: addr: ffffffff80abaf00 paddr: abaf00 cnt: 16>
> read_diskdump: paddr/pfn: abaf00/aba -> cache physical page: aba000
> <readmem: ffffffff806dc894, KVADDR, "init_uts_ns", 390, (ROE),
> c5ea1c>
> <read_diskdump: addr: ffffffff806dc894 paddr: 6dc894 cnt: 390>
> read_diskdump: paddr/pfn: 6dc894/6dc -> cache physical page: 6dc000
> base kernel version: 0.0.0
> <readmem: ffffffff804a7000, KVADDR, "accessible check", 8, (ROE|Q),
> 7fff35eeba88>
> <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 8>
> read_diskdump: paddr/pfn: 4a7000/4a7 -> cache physical page: 4a7000
> <readmem: ffffffff804a7000, KVADDR, "readstring characters", 1499,
> (ROE|Q), 7fff35eeaa70>
> <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 1499>
> read_diskdump: paddr/pfn: 4a7000/4a7 -> physical page is cached:
> 4a7000
> linux_banner:
> 紉;���
> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
>
> Usage:
>
> crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
> crash [OPTION]... [NAMELIST] (live system form)
>
> Enter "crash -h" for details.
>
>
>
>
>
>
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>


--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 
Old 03-26-2012, 06:48 PM
Dave Anderson
 
Default crash can live analysis but vmcore can't.

----- Original Message -----
>
> Hi Dave,
>
> Thank you for your detail explain.
> It's very useful for us whom doesn't know what's in .debug file.
>
> I have another question about crash, it's trouble a while.
>
> Below call trace and register is system call of poll.
>
> int poll(struct pollfd fds[], nfds_t nfds, int timeout);
>
> We could know that RDI is fds, RSI is nfds=1, timeout=RDX=10hours
> But before copy_from_user, fds is userspace address, how can we read
> it or get the content from this address.
>
> #3 [ffff810028d51f60] sys_poll at ffffffff8019a64a
> ffff810028d51f68: 0000000000002000 0000000000000000
> ffff810028d51f78: 0000000002255100 ffffffff8010ae36
> #4 [ffff810028d51f80] system_call at ffffffff8010ae36
> RIP: 00002b8543aa3dc6 RSP: 00007ffff6676478 RFLAGS: 00000246
> RAX: 0000000000000007 RBX: ffffffff8010ae36 RCX: 0000000000000000
> RDX: 0000000002255100 RSI: 0000000000000001 RDI: 00007ffff66760b0
> RBP: 0000000002255100 R8: 0000000000000001 R9: 00002aaaaab35cf0
> R10: 00002aaaab2ff9b0 R11: 0000000000000202 R12: 0000000000000000
> R13: 0000000000002000 R14: ffffffff8019a64a R15: 00002aaaaab36970
> ORIG_RAX: 0000000000000007 CS: 0033 SS: 002b
>
> It seems gcore could dump the task memory on latest crash., but on old crash and os,
> how could we do it.
>
> Thank you so much
> Kong Wei

If the vmcore has not excluded user-space memory, then you
can read it with the "struct" command by qualifying the
virtual address as a user-space address with the "-u" option.

So in your case, you could do this:

crash> struct pollfd -u 7ffff66760b0

For example, on my live system, I have this task:

crash> bt
PID: 4890 TASK: ffff810038c10080 CPU: 0 COMMAND: "automount"
#0 [ffff810027fbba68] schedule at ffffffff80063035
#1 [ffff810027fbbb40] schedule_timeout at ffffffff800638f0
#2 [ffff810027fbbb90] do_sys_poll at ffffffff8002f408
#3 [ffff810027fbbf60] sys_poll at ffffffff8004be58
#4 [ffff810027fbbf80] tracesys at ffffffff8005d28d (via system_call)
RIP: 00002b85faafa016 RSP: 0000000041f2ddb0 RFLAGS: 00000202
RAX: ffffffffffffffda RBX: ffffffff8005d28d RCX: ffffffffffffffff
RDX: ffffffffffffffff RSI: 0000000000000003 RDI: 0000000041f2ddd0
RBP: ffffffffffffffff R8: 0000000000000000 R9: 0000000004000001
R10: 0000000000000001 R11: 0000000000000202 R12: 0000000041f2ddd0
R13: 0000000000000003 R14: ffffffff8004be58 R15: 0000000000001000
ORIG_RAX: 0000000000000007 CS: 0033 SS: 002b
crash>

So if the fds[] array is at RDI/0000000041f2ddd0, and there
are RSI/0000000000000003 entries in the array, I can look
at them like this:

crash> struct pollfd -u 0000000041f2ddd0 3
struct pollfd {
fd = 10,
events = 1,
revents = 0
}

struct pollfd {
fd = 12,
events = 1,
revents = 1
}

struct pollfd {
fd = 14,
events = 1,
revents = 0
}
crash>

But if you have excluded user-space memory from the vmcore,
then you'll get "page excluded" error messages.

Dave



> >>> Dave Anderson <anderson@redhat.com> 2012/3/26 22:06 >>>
>
>
> Whenever a crash session fails with a "do not match" message
> such as this:
>
> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
>
> it is because the virtual addresses of the kernel symbols in the
> vmlinux file do not match the virtual addresses of the kernel that
> crashed.
>
> When you run with "crash -d<number>" you will see the first few
> attempts to gather kernel data (kernel_config_data, xtime, the
> cpu_possible, cpu_present and cpu_online maps, the utsname structure,
> and then the "linux_banner" string. The linux_banner string is what
> you would see in /proc/version on the live system, for example:
>
> # cat /proc/version
> Linux version 2.6.32-220.el6.x86_64
> (mockbuild@x86-004.build.bos.redhat.com) (gcc version 4.4.5 20110214
> (Red Hat 4.4.5-6) (GCC) ) #1 SMP Wed Nov 9 08:03:13 EST 2011
> #
>
> If the crash utility does not find the correct string at the address
> specified in the vmlinux file, then it makes no sense to continue,
> and so it quits with the "do not match" message.
>
> Accordingly, one of your examples ends like this, where the
> vmlinux file indicates that the linux_banner string is located
> at kernel virtual address ffffffff804a7000. That address was
> translated to physical address 4a7000, and then read from the
> compressed kdump vmcore. But the linux_banner string read from
> that address in the vmcore contains garbage:
>
> ...
> <readmem: ffffffff804a7000, KVADDR, "readstring characters", 1499,
> (ROE|Q), 7fff35eeaa70>
> <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 1499>
> read_diskdump: paddr/pfn: 4a7000/4a7 -> physical page is cached:
> 4a7000
> linux_banner:
> �x;�����
> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
> ...
>
> I cannot answer for SUSE, but if I am not mistaken, they build a
> debug kernel (i.e, built with -g) for their distrobution kernels
> after-the-fact, and when that is done, the kernel virtual addresses
> are not exactly the same. To work around that, you can pass the
> System.map file of the actual crashed kernel -- which has the
> *correct"
> kernel virtual addresses -- on the crash command line. And when
> you did that, things apparently worked correctly.
>
> In other words, this failed:
>
> # crash vmlinux-2.6.27.19-5-default vmlinux-2.6.27.19-5-default.debug
> vmcore
>
> But this worked:
>
> # crash vmlinux-2.6.27.19-5-default System.map-2.6.27.19-5-default
> vmcore
>
> And it worked because the crash utility takes the (correct) symbol
> values
> found in the System.map file, and "patches" the symbol values that
> were
> initially read from the vmlinux file(s).
>
> Complicating matters a bit is the usage of a vmlinux file and its
> associated
> vmlinux.debug file. In old RHEL3 kernels, Red Hat would split the
> vmlinux
> file into its stripped version and a .debug version, the same way as
> is
> done with user-space programs. But from RHEL4 onwards, the that was
> no longer
> done, and there is a single vmlinux file that contains all the
> debuginfo
> data, and which is the same kernel as the distribution kernel. In
> SLES kernels,
> I am not sure whether that was true in the example above, or whether
> the .debug
> kernel is the kernel that they build after-the-fact?
>
> In any case, I note that there a several novell.com addresses that
> are cc'd
> with this email, and there are many SLES users and Novell/Suse
> employees
> on this list who can give you the specifics of how they do things.
>
> Dave
>
>
>
>
>
> ----- Original Message -----
> >
> >
> > on another machine,I use all kinds of crash command :
> >
> >
> > Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #crash
> > vmlinux-2.6.27.19-5-default.gz System.map-2.6.27.19-5-default
> > vmcore
> >
> > crash 6.0.4
> > Copyright (C) 2002-2012 Red Hat, Inc.
> > Copyright (C) 2004, 2005, 2006 IBM Corporation
> > Copyright (C) 1999-2006 Hewlett-Packard Co
> > Copyright (C) 2005, 2006 Fujitsu Limited
> > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> > Copyright (C) 2005 NEC Corporation
> > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> > This program is free software, covered by the GNU General Public
> > License,
> > and you are welcome to change it and/or distribute copies of it
> > under
> > certain conditions. Enter "help copying" to see the conditions.
> > This program has absolutely no warranty. Enter "help warranty" for
> > details.
> >
> > GNU gdb (GDB) 7.3.1
> > Copyright (C) 2011 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later <
> > http://gnu.org/licenses/gpl.html >
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law. Type "show
> > copying"
> > and "show warranty" for details.
> > This GDB was configured as "x86_64-unknown-linux-gnu"...
> >
> > crash: /var/tmp/vmlinux-2.6.27.19-5-default.gz_RYx9JC: no debugging
> > data available
> >
> > Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #gunzip vmlinux-2.
> > 6.27.19-5-default.gz
> >
> > Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 # crash
> > vmlinux-2.6.27.19-5-default vmlinux-2.6.27.19-5-default.debug
> > vmcore
> >
> > crash 6.0.4
> > Copyright (C) 2002-2012 Red Hat, Inc.
> > Copyright (C) 2004, 2005, 2006 IBM Corporation
> > Copyright (C) 1999-2006 Hewlett-Packard Co
> > Copyright (C) 2005, 2006 Fujitsu Limited
> > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> > Copyright (C) 2005 NEC Corporation
> > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> > This program is free software, covered by the GNU General Public
> > License,
> > and you are welcome to change it and/or distribute copies of it
> > under
> > certain conditions. Enter "help copying" to see the conditions.
> > This program has absolutely no warranty. Enter "help warranty" for
> > details.
> >
> > GNU gdb (GDB) 7.3.1
> > Copyright (C) 2011 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later <
> > http://gnu.org/licenses/gpl.html >
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law. Type "show
> > copying"
> > and "show warranty" for details.
> > This GDB was configured as "x86_64-unknown-linux-gnu"...
> >
> > WARNING: could not find MAGIC_START!
> > crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
> >
> > Usage:
> >
> > crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
> > crash [OPTION]... [NAMELIST] (live system form)
> >
> > Enter "crash -h" for details.
> >
> > Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #cat README.txt
> > Kernel crashdump
> > ----------------
> >
> > Crash time : 2012-03-24 10:57 (+0000)
> > Kernel version : 2.6.27.19-5-default
> > Host : sles11
> > Dump level : 0
> > Dump format : compressed
> >
> > but I use another command is OK, what's wrong ?
> >
> > Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #crash
> > vmlinux-2.6.27.19-5-default System.map-2.6.27.19-5-default vmcore
> >
> > crash 6.0.4
> > Copyright (C) 2002-2012 Red Hat, Inc.
> > Copyright (C) 2004, 2005, 2006 IBM Corporation
> > Copyright (C) 1999-2006 Hewlett-Packard Co
> > Copyright (C) 2005, 2006 Fujitsu Limited
> > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> > Copyright (C) 2005 NEC Corporation
> > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> > This program is free software, covered by the GNU General Public
> > License,
> > and you are welcome to change it and/or distribute copies of it
> > under
> > certain conditions. Enter "help copying" to see the conditions.
> > This program has absolutely no warranty. Enter "help warranty" for
> > details.
> >
> > GNU gdb (GDB) 7.3.1
> > Copyright (C) 2011 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later <
> > http://gnu.org/licenses/gpl.html >
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law. Type "show
> > copying"
> > and "show warranty" for details.
> > This GDB was configured as "x86_64-unknown-linux-gnu"...
> >
> > SYSTEM MAP: System.map-2.6.27.19-5-default
> > DEBUG KERNEL: vmlinux-2.6.27.19-5-default (2.6.27.19-5-default)
> > DUMPFILE: vmcore
> > CPUS: 2
> > DATE: Sat Mar 24 18:57:31 2012
> > UPTIME: 00:13:30
> > LOAD AVERAGE: 0.27, 0.30, 0.28
> > TASKS: 190
> > NODENAME: sles11
> > RELEASE: 2.6.27.19-5-default
> > VERSION: #6 SMP Sat Nov 12 08:58:08 CST 2011
> > MACHINE: x86_64 (2393 Mhz)
> > MEMORY: 3.9 GB
> > PANIC: "SysRq : Trigger a crashdump"
> > PID: 4700
> > COMMAND: "bash"
> > TASK: ffff88007f8f6080 [THREAD_INFO: ffff88007f802000]
> > CPU: 0
> > STATE: TASK_RUNNING (SYSRQ)
> >
> > crash>
> >
> >
> >
> >
> >
> >
> >
> > From: hujun_hotmail
> > Sent: Saturday, March 24, 2012 8:07 PM
> > To: crash mailist
> > Subject: [Crash-utility] crash can live analysis but vmcore can't.
> >
> >
> > hello experts:
> >
> > My linux system : Suse SLES 11 , crash can live analysis, but can't
> > open vmcore generated by kdump.
> >
> > why?
> >
> >
> >
> > sles11:/mnt/sda12/crash/2012-03-24-18:57 # crash
> >
> >
> > crash 6.0.4
> > Copyright (C) 2002-2012 Red Hat, Inc.
> > Copyright (C) 2004, 2005, 2006 IBM Corporation
> > Copyright (C) 1999-2006 Hewlett-Packard Co
> > Copyright (C) 2005, 2006 Fujitsu Limited
> > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> > Copyright (C) 2005 NEC Corporation
> > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> > This program is free software, covered by the GNU General Public
> > License,
> > and you are welcome to change it and/or distribute copies of it
> > under
> > certain conditions. Enter "help copying" to see the conditions.
> > This program has absolutely no warranty. Enter "help warranty" for
> > details.
> >
> > GNU gdb (GDB) 7.3.1
> > Copyright (C) 2011 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later <
> > http://gnu.org/licenses/gpl.html >
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law. Type "show
> > copying"
> > and "show warranty" for details.
> > This GDB was configured as "x86_64-unknown-linux-gnu"...
> >
> > KERNEL: /boot/vmlinuz-2.6.27.19-5-default.vmlinux-debug
> > DUMPFILE: /dev/mem
> > CPUS: 2
> > DATE: Sat Mar 24 19:10:45 2012
> > UPTIME: 00:07:17
> > LOAD AVERAGE: 0.16, 0.19, 0.12
> > TASKS: 189
> > NODENAME: sles11
> > RELEASE: 2.6.27.19-5-default
> > VERSION: #6 SMP Sat Nov 12 08:58:08 CST 2011
> > MACHINE: x86_64 (2393 Mhz)
> > MEMORY: 3.9 GB
> > PID: 4590
> > COMMAND: "crash"
> > TASK: ffff8801230fa340 [THREAD_INFO: ffff880131c86000]
> > CPU: 0
> > STATE: TASK_RUNNING (ACTIVE)
> >
> > crash>
> >
> >
> >
> >
> >
> >
> > sles11:/mnt/sda12/crash/2012-03-24-18:57 # crash
> > vmlinux-2.6.27.19-5-default.gz vmlinux-2.6.27.19-5-default.debug
> > vmcore
> > crash 6.0.4
> > Copyright (C) 2002-2012 Red Hat, Inc.
> > Copyright (C) 2004, 2005, 2006 IBM Corporation
> > Copyright (C) 1999-2006 Hewlett-Packard Co
> > Copyright (C) 2005, 2006 Fujitsu Limited
> > Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
> > Copyright (C) 2005 NEC Corporation
> > Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
> > Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
> > This program is free software, covered by the GNU General Public
> > License,
> > and you are welcome to change it and/or distribute copies of it
> > under
> > certain conditions. Enter "help copying" to see the conditions.
> > This program has absolutely no warranty. Enter "help warranty" for
> > details.
> >
> > please wait... (uncompressing vmlinux-2.6.27.19-5-default.gz)
> >
> > compressed kdump: header-> utsname.machine:
> > compressed kdump: memory bitmap offset: 2000
> > diskdump_data:
> > filename: vmcore
> > flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED)
> > dfd: 4
> > ofp: 0
> > machine_type: 62 (EM_X86_64)
> >
> > header: dee020
> > signature: "KDUMP "
> > header_version: 1
> > utsname:
> > sysname:
> > nodename:
> > release:
> > version:
> > machine:
> > domainname:
> > timestamp:
> > tv_sec: 0
> > tv_usec: 0
> > status: 0 ()
> > block_size: 4096
> > sub_hdr_size: 1
> > bitmap_blocks: 80
> > max_mapnr: 1294336
> > total_ram_blocks: 0
> > device_blocks: 0
> > written_blocks: 0
> > current_cpu: 0
> > nr_cpus: 1
> > tasks[nr_cpus]: 0
> >
> > sub_header: 0 (n/a)
> >
> > sub_header_kdump: def030
> > phys_base: 0
> > dump_level: 0 (0x0)
> >
> > data_offset: 52000
> > block_size: 4096
> > block_shift: 12
> > bitmap: 7f812de69010
> > bitmap_len: 327680
> > dumpable_bitmap: 7f812de18010
> > byte: 0
> > bit: 0
> > compressed_page: e00a40
> > curbufptr: 0
> >
> > page_cache_hdr[0]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df0a30
> > pg_hit_count: 0
> > page_cache_hdr[1]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df1a30
> > pg_hit_count: 0
> > page_cache_hdr[2]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df2a30
> > pg_hit_count: 0
> > page_cache_hdr[3]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df3a30
> > pg_hit_count: 0
> > page_cache_hdr[4]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df4a30
> > pg_hit_count: 0
> > page_cache_hdr[5]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df5a30
> > pg_hit_count: 0
> > page_cache_hdr[6]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df6a30
> > pg_hit_count: 0
> > page_cache_hdr[7]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df7a30
> > pg_hit_count: 0
> > page_cache_hdr[8]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df8a30
> > pg_hit_count: 0
> > page_cache_hdr[9]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: df9a30
> > pg_hit_count: 0
> > page_cache_hdr[10]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: dfaa30
> > pg_hit_count: 0
> > page_cache_hdr[11]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: dfba30
> > pg_hit_count: 0
> > page_cache_hdr[12]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: dfca30
> > pg_hit_count: 0
> > page_cache_hdr[13]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: dfda30
> > pg_hit_count: 0
> > page_cache_hdr[14]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: dfea30
> > pg_hit_count: 0
> > page_cache_hdr[15]:
> > pg_flags: 0 ()
> > pg_addr: 0
> > pg_bufptr: dffa30
> > pg_hit_count: 0
> >
> > page_cache_buf: df0a30
> > evict_index: 0
> > evictions: 0
> > accesses: 0
> > cached_reads: 0
> > valid_pages: df0040
> > readmem: read_diskdump()
> > [?1034hcompressed kdump: phys_base: 0
> > gdb vmlinux-2.6.27.19-5-default.debug
> > GNU gdb (GDB) 7.3.1
> > Copyright (C) 2011 Free Software Foundation, Inc.
> > License GPLv3+: GNU GPL version 3 or later <
> > http://gnu.org/licenses/gpl.html >
> > This is free software: you are free to change and redistribute it.
> > There is NO WARRANTY, to the extent permitted by law. Type "show
> > copying"
> > and "show warranty" for details.
> > This GDB was configured as "x86_64-unknown-linux-gnu"...
> > GETBUF(248 -> 0)
> > GETBUF(1500 -> 1)
> >
> > please wait... (patching 35285 gdb minimal_symbol values)
> >
> > FREEBUF(1)
> > FREEBUF(0)
> > <readmem: ffffffff804b8210, KVADDR, "kernel_config_data", 32768,
> > (ROE), 17e92b0>
> > <read_diskdump: addr: ffffffff804b8210 paddr: 4b8210 cnt: 3568>
> > read_diskdump: paddr/pfn: 4b8210/4b8 -> cache physical page: 4b8000
> > <read_diskdump: addr: ffffffff804b9000 paddr: 4b9000 cnt: 4096>
> > read_diskdump: paddr/pfn: 4b9000/4b9 -> cache physical page: 4b9000
> > <read_diskdump: addr: ffffffff804ba000 paddr: 4ba000 cnt: 4096>
> > read_diskdump: paddr/pfn: 4ba000/4ba -> cache physical page: 4ba000
> > <read_diskdump: addr: ffffffff804bb000 paddr: 4bb000 cnt: 4096>
> > read_diskdump: paddr/pfn: 4bb000/4bb -> cache physical page: 4bb000
> > <read_diskdump: addr: ffffffff804bc000 paddr: 4bc000 cnt: 4096>
> > read_diskdump: paddr/pfn: 4bc000/4bc -> cache physical page: 4bc000
> > <read_diskdump: addr: ffffffff804bd000 paddr: 4bd000 cnt: 4096>
> > read_diskdump: paddr/pfn: 4bd000/4bd -> cache physical page: 4bd000
> > <read_diskdump: addr: ffffffff804be000 paddr: 4be000 cnt: 4096>
> > read_diskdump: paddr/pfn: 4be000/4be -> cache physical page: 4be000
> > <read_diskdump: addr: ffffffff804bf000 paddr: 4bf000 cnt: 4096>
> > read_diskdump: paddr/pfn: 4bf000/4bf -> cache physical page: 4bf000
> > <read_diskdump: addr: ffffffff804c0000 paddr: 4c0000 cnt: 528>
> > read_diskdump: paddr/pfn: 4c0000/4c0 -> cache physical page: 4c0000
> > WARNING: could not find MAGIC_START!
> > GETBUF(248 -> 0)
> > FREEBUF(0)
> > GETBUF(64 -> 0)
> > <readmem: ffffffff80a59910, KVADDR, "possible", 64, (ROE), da00e0>
> > <read_diskdump: addr: ffffffff80a59910 paddr: a59910 cnt: 64>
> > read_diskdump: paddr/pfn: a59910/a59 -> cache physical page: a59000
> > cpu_possible_map:
> > <readmem: ffffffff8095d950, KVADDR, "present", 64, (ROE), da00e0>
> > <read_diskdump: addr: ffffffff8095d950 paddr: 95d950 cnt: 64>
> > read_diskdump: paddr/pfn: 95d950/95d -> cache physical page: 95d000
> > cpu_present_map: 2 3 6 7 10 11 14 15 18 19 22 23 26 27 30 31 34 35
> > 38
> > 39 42 43 46 47 50 51 54 55 58 59 62 63 66 67 70 71 74 75 78 79 82
> > 83
> > 86 87 90 91 94 95 98 99 102 103 106 107 110 111 114 115 118 119 122
> > 123 126 127 130 131 134 135 138 139 142 143 146 147 150 151 154 155
> > 158 159 162 163 166 167 170 171 174 175 178 179 182 183 186 187 190
> > 191 194 195 198 199 202 203 206 207 210 211 214 215 218 219 222 223
> > 226 227 230 231 234 235 238 239 242 243 246 247 250 251 254 255 258
> > 259 262 263 266 267 270 271 274 275 278 279 282 283 286 287 290 291
> > 294 295 298 299 302 303 306 307 310 311 314 315 318 319 322 323 326
> > 327 330 331 334 335 338 339 342 343 346 347 350 351 354 355 358 359
> > 362 363 366 367 370 371 374 375 378 379 382 383 386 387 390 391 394
> > 395 398 399 402 403 406 407 410 411 414 415 418 419 422 423 426 427
> > 430 431 434 435 438 439 442 443 446 447 450 451 454 455 458 459 462
> > 463 466 467 470 471 474 475 478 479 482 483 486 487 490 491 494 495
> > 498 499 502 503 506 507 510 511
> > <readmem: ffffffff808cbe90, KVADDR, "online", 64, (ROE), da00e0>
> > <read_diskdump: addr: ffffffff808cbe90 paddr: 8cbe90 cnt: 64>
> > read_diskdump: paddr/pfn: 8cbe90/8cb -> cache physical page: 8cb000
> > cpu_online_map:
> > FREEBUF(0)
> > <readmem: ffffffff80abaf00, KVADDR, "xtime", 16, (FOE), c5e430>
> > <read_diskdump: addr: ffffffff80abaf00 paddr: abaf00 cnt: 16>
> > read_diskdump: paddr/pfn: abaf00/aba -> cache physical page: aba000
> > <readmem: ffffffff806dc894, KVADDR, "init_uts_ns", 390, (ROE),
> > c5ea1c>
> > <read_diskdump: addr: ffffffff806dc894 paddr: 6dc894 cnt: 390>
> > read_diskdump: paddr/pfn: 6dc894/6dc -> cache physical page: 6dc000
> > base kernel version: 0.0.0
> > <readmem: ffffffff804a7000, KVADDR, "accessible check", 8, (ROE|Q),
> > 7fff35eeba88>
> > <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 8>
> > read_diskdump: paddr/pfn: 4a7000/4a7 -> cache physical page: 4a7000
> > <readmem: ffffffff804a7000, KVADDR, "readstring characters", 1499,
> > (ROE|Q), 7fff35eeaa70>
> > <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 1499>
> > read_diskdump: paddr/pfn: 4a7000/4a7 -> physical page is cached:
> > 4a7000
> > linux_banner:
> > 紉;���
> > crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
> >
> > Usage:
> >
> > crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
> > crash [OPTION]... [NAMELIST] (live system form)
> >
> > Enter "crash -h" for details.
> >
> >
> >
> >
> >
> >
> >
> > --
> > Crash-utility mailing list
> > Crash-utility@redhat.com
> > https://www.redhat.com/mailman/listinfo/crash-utility
> >
> > --
> > Crash-utility mailing list
> > Crash-utility@redhat.com
> > https://www.redhat.com/mailman/listinfo/crash-utility
> >
>
>

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 
Old 03-27-2012, 12:25 AM
Kong Wei
 
Default crash can live analysis but vmcore can't.

Thank you very much!

On 2012-3-27, at 2:48, "Dave Anderson <anderson@redhat.com>" <anderson@redhat.com> wrote:

>
>
> ----- Original Message -----
>>
>> Hi Dave,
>>
>> Thank you for your detail explain.
>> It's very useful for us whom doesn't know what's in .debug file.
>>
>> I have another question about crash, it's trouble a while.
>>
>> Below call trace and register is system call of poll.
>>
>> int poll(struct pollfd fds[], nfds_t nfds, int timeout);
>>
>> We could know that RDI is fds, RSI is nfds=1, timeout=RDX=10hours
>> But before copy_from_user, fds is userspace address, how can we read
>> it or get the content from this address.
>>
>> #3 [ffff810028d51f60] sys_poll at ffffffff8019a64a
>> ffff810028d51f68: 0000000000002000 0000000000000000
>> ffff810028d51f78: 0000000002255100 ffffffff8010ae36
>> #4 [ffff810028d51f80] system_call at ffffffff8010ae36
>> RIP: 00002b8543aa3dc6 RSP: 00007ffff6676478 RFLAGS: 00000246
>> RAX: 0000000000000007 RBX: ffffffff8010ae36 RCX: 0000000000000000
>> RDX: 0000000002255100 RSI: 0000000000000001 RDI: 00007ffff66760b0
>> RBP: 0000000002255100 R8: 0000000000000001 R9: 00002aaaaab35cf0
>> R10: 00002aaaab2ff9b0 R11: 0000000000000202 R12: 0000000000000000
>> R13: 0000000000002000 R14: ffffffff8019a64a R15: 00002aaaaab36970
>> ORIG_RAX: 0000000000000007 CS: 0033 SS: 002b
>>
>> It seems gcore could dump the task memory on latest crash., but on old crash and os,
>> how could we do it.
>>
>> Thank you so much
>> Kong Wei
>
> If the vmcore has not excluded user-space memory, then you
> can read it with the "struct" command by qualifying the
> virtual address as a user-space address with the "-u" option.
>
> So in your case, you could do this:
>
> crash> struct pollfd -u 7ffff66760b0
>
> For example, on my live system, I have this task:
>
> crash> bt
> PID: 4890 TASK: ffff810038c10080 CPU: 0 COMMAND: "automount"
> #0 [ffff810027fbba68] schedule at ffffffff80063035
> #1 [ffff810027fbbb40] schedule_timeout at ffffffff800638f0
> #2 [ffff810027fbbb90] do_sys_poll at ffffffff8002f408
> #3 [ffff810027fbbf60] sys_poll at ffffffff8004be58
> #4 [ffff810027fbbf80] tracesys at ffffffff8005d28d (via system_call)
> RIP: 00002b85faafa016 RSP: 0000000041f2ddb0 RFLAGS: 00000202
> RAX: ffffffffffffffda RBX: ffffffff8005d28d RCX: ffffffffffffffff
> RDX: ffffffffffffffff RSI: 0000000000000003 RDI: 0000000041f2ddd0
> RBP: ffffffffffffffff R8: 0000000000000000 R9: 0000000004000001
> R10: 0000000000000001 R11: 0000000000000202 R12: 0000000041f2ddd0
> R13: 0000000000000003 R14: ffffffff8004be58 R15: 0000000000001000
> ORIG_RAX: 0000000000000007 CS: 0033 SS: 002b
> crash>
>
> So if the fds[] array is at RDI/0000000041f2ddd0, and there
> are RSI/0000000000000003 entries in the array, I can look
> at them like this:
>
> crash> struct pollfd -u 0000000041f2ddd0 3
> struct pollfd {
> fd = 10,
> events = 1,
> revents = 0
> }
>
> struct pollfd {
> fd = 12,
> events = 1,
> revents = 1
> }
>
> struct pollfd {
> fd = 14,
> events = 1,
> revents = 0
> }
> crash>
>
> But if you have excluded user-space memory from the vmcore,
> then you'll get "page excluded" error messages.
>
> Dave
>
>
>
>>>>> Dave Anderson <anderson@redhat.com> 2012/3/26 22:06 >>>
>>
>>
>> Whenever a crash session fails with a "do not match" message
>> such as this:
>>
>> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
>>
>> it is because the virtual addresses of the kernel symbols in the
>> vmlinux file do not match the virtual addresses of the kernel that
>> crashed.
>>
>> When you run with "crash -d<number>" you will see the first few
>> attempts to gather kernel data (kernel_config_data, xtime, the
>> cpu_possible, cpu_present and cpu_online maps, the utsname structure,
>> and then the "linux_banner" string. The linux_banner string is what
>> you would see in /proc/version on the live system, for example:
>>
>> # cat /proc/version
>> Linux version 2.6.32-220.el6.x86_64
>> (mockbuild@x86-004.build.bos.redhat.com) (gcc version 4.4.5 20110214
>> (Red Hat 4.4.5-6) (GCC) ) #1 SMP Wed Nov 9 08:03:13 EST 2011
>> #
>>
>> If the crash utility does not find the correct string at the address
>> specified in the vmlinux file, then it makes no sense to continue,
>> and so it quits with the "do not match" message.
>>
>> Accordingly, one of your examples ends like this, where the
>> vmlinux file indicates that the linux_banner string is located
>> at kernel virtual address ffffffff804a7000. That address was
>> translated to physical address 4a7000, and then read from the
>> compressed kdump vmcore. But the linux_banner string read from
>> that address in the vmcore contains garbage:
>>
>> ...
>> <readmem: ffffffff804a7000, KVADDR, "readstring characters", 1499,
>> (ROE|Q), 7fff35eeaa70>
>> <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 1499>
>> read_diskdump: paddr/pfn: 4a7000/4a7 -> physical page is cached:
>> 4a7000
>> linux_banner:
>> �x;�����
>> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
>> ...
>>
>> I cannot answer for SUSE, but if I am not mistaken, they build a
>> debug kernel (i.e, built with -g) for their distrobution kernels
>> after-the-fact, and when that is done, the kernel virtual addresses
>> are not exactly the same. To work around that, you can pass the
>> System.map file of the actual crashed kernel -- which has the
>> *correct"
>> kernel virtual addresses -- on the crash command line. And when
>> you did that, things apparently worked correctly.
>>
>> In other words, this failed:
>>
>> # crash vmlinux-2.6.27.19-5-default vmlinux-2.6.27.19-5-default.debug
>> vmcore
>>
>> But this worked:
>>
>> # crash vmlinux-2.6.27.19-5-default System.map-2.6.27.19-5-default
>> vmcore
>>
>> And it worked because the crash utility takes the (correct) symbol
>> values
>> found in the System.map file, and "patches" the symbol values that
>> were
>> initially read from the vmlinux file(s).
>>
>> Complicating matters a bit is the usage of a vmlinux file and its
>> associated
>> vmlinux.debug file. In old RHEL3 kernels, Red Hat would split the
>> vmlinux
>> file into its stripped version and a .debug version, the same way as
>> is
>> done with user-space programs. But from RHEL4 onwards, the that was
>> no longer
>> done, and there is a single vmlinux file that contains all the
>> debuginfo
>> data, and which is the same kernel as the distribution kernel. In
>> SLES kernels,
>> I am not sure whether that was true in the example above, or whether
>> the .debug
>> kernel is the kernel that they build after-the-fact?
>>
>> In any case, I note that there a several novell.com addresses that
>> are cc'd
>> with this email, and there are many SLES users and Novell/Suse
>> employees
>> on this list who can give you the specifics of how they do things.
>>
>> Dave
>>
>>
>>
>>
>>
>> ----- Original Message -----
>>>
>>>
>>> on another machine,I use all kinds of crash command :
>>>
>>>
>>> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #crash
>>> vmlinux-2.6.27.19-5-default.gz System.map-2.6.27.19-5-default
>>> vmcore
>>>
>>> crash 6.0.4
>>> Copyright (C) 2002-2012 Red Hat, Inc.
>>> Copyright (C) 2004, 2005, 2006 IBM Corporation
>>> Copyright (C) 1999-2006 Hewlett-Packard Co
>>> Copyright (C) 2005, 2006 Fujitsu Limited
>>> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
>>> Copyright (C) 2005 NEC Corporation
>>> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
>>> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
>>> This program is free software, covered by the GNU General Public
>>> License,
>>> and you are welcome to change it and/or distribute copies of it
>>> under
>>> certain conditions. Enter "help copying" to see the conditions.
>>> This program has absolutely no warranty. Enter "help warranty" for
>>> details.
>>>
>>> GNU gdb (GDB) 7.3.1
>>> Copyright (C) 2011 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later <
>>> http://gnu.org/licenses/gpl.html >
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law. Type "show
>>> copying"
>>> and "show warranty" for details.
>>> This GDB was configured as "x86_64-unknown-linux-gnu"...
>>>
>>> crash: /var/tmp/vmlinux-2.6.27.19-5-default.gz_RYx9JC: no debugging
>>> data available
>>>
>>> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #gunzip vmlinux-2.
>>> 6.27.19-5-default.gz
>>>
>>> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 # crash
>>> vmlinux-2.6.27.19-5-default vmlinux-2.6.27.19-5-default.debug
>>> vmcore
>>>
>>> crash 6.0.4
>>> Copyright (C) 2002-2012 Red Hat, Inc.
>>> Copyright (C) 2004, 2005, 2006 IBM Corporation
>>> Copyright (C) 1999-2006 Hewlett-Packard Co
>>> Copyright (C) 2005, 2006 Fujitsu Limited
>>> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
>>> Copyright (C) 2005 NEC Corporation
>>> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
>>> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
>>> This program is free software, covered by the GNU General Public
>>> License,
>>> and you are welcome to change it and/or distribute copies of it
>>> under
>>> certain conditions. Enter "help copying" to see the conditions.
>>> This program has absolutely no warranty. Enter "help warranty" for
>>> details.
>>>
>>> GNU gdb (GDB) 7.3.1
>>> Copyright (C) 2011 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later <
>>> http://gnu.org/licenses/gpl.html >
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law. Type "show
>>> copying"
>>> and "show warranty" for details.
>>> This GDB was configured as "x86_64-unknown-linux-gnu"...
>>>
>>> WARNING: could not find MAGIC_START!
>>> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
>>>
>>> Usage:
>>>
>>> crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
>>> crash [OPTION]... [NAMELIST] (live system form)
>>>
>>> Enter "crash -h" for details.
>>>
>>> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #cat README.txt
>>> Kernel crashdump
>>> ----------------
>>>
>>> Crash time : 2012-03-24 10:57 (+0000)
>>> Kernel version : 2.6.27.19-5-default
>>> Host : sles11
>>> Dump level : 0
>>> Dump format : compressed
>>>
>>> but I use another command is OK, what's wrong ?
>>>
>>> Vsles11sp1 /mnt/disk/crash/2012-03-24-18:57 #crash
>>> vmlinux-2.6.27.19-5-default System.map-2.6.27.19-5-default vmcore
>>>
>>> crash 6.0.4
>>> Copyright (C) 2002-2012 Red Hat, Inc.
>>> Copyright (C) 2004, 2005, 2006 IBM Corporation
>>> Copyright (C) 1999-2006 Hewlett-Packard Co
>>> Copyright (C) 2005, 2006 Fujitsu Limited
>>> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
>>> Copyright (C) 2005 NEC Corporation
>>> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
>>> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
>>> This program is free software, covered by the GNU General Public
>>> License,
>>> and you are welcome to change it and/or distribute copies of it
>>> under
>>> certain conditions. Enter "help copying" to see the conditions.
>>> This program has absolutely no warranty. Enter "help warranty" for
>>> details.
>>>
>>> GNU gdb (GDB) 7.3.1
>>> Copyright (C) 2011 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later <
>>> http://gnu.org/licenses/gpl.html >
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law. Type "show
>>> copying"
>>> and "show warranty" for details.
>>> This GDB was configured as "x86_64-unknown-linux-gnu"...
>>>
>>> SYSTEM MAP: System.map-2.6.27.19-5-default
>>> DEBUG KERNEL: vmlinux-2.6.27.19-5-default (2.6.27.19-5-default)
>>> DUMPFILE: vmcore
>>> CPUS: 2
>>> DATE: Sat Mar 24 18:57:31 2012
>>> UPTIME: 00:13:30
>>> LOAD AVERAGE: 0.27, 0.30, 0.28
>>> TASKS: 190
>>> NODENAME: sles11
>>> RELEASE: 2.6.27.19-5-default
>>> VERSION: #6 SMP Sat Nov 12 08:58:08 CST 2011
>>> MACHINE: x86_64 (2393 Mhz)
>>> MEMORY: 3.9 GB
>>> PANIC: "SysRq : Trigger a crashdump"
>>> PID: 4700
>>> COMMAND: "bash"
>>> TASK: ffff88007f8f6080 [THREAD_INFO: ffff88007f802000]
>>> CPU: 0
>>> STATE: TASK_RUNNING (SYSRQ)
>>>
>>> crash>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> From: hujun_hotmail
>>> Sent: Saturday, March 24, 2012 8:07 PM
>>> To: crash mailist
>>> Subject: [Crash-utility] crash can live analysis but vmcore can't.
>>>
>>>
>>> hello experts:
>>>
>>> My linux system : Suse SLES 11 , crash can live analysis, but can't
>>> open vmcore generated by kdump.
>>>
>>> why?
>>>
>>>
>>>
>>> sles11:/mnt/sda12/crash/2012-03-24-18:57 # crash
>>>
>>>
>>> crash 6.0.4
>>> Copyright (C) 2002-2012 Red Hat, Inc.
>>> Copyright (C) 2004, 2005, 2006 IBM Corporation
>>> Copyright (C) 1999-2006 Hewlett-Packard Co
>>> Copyright (C) 2005, 2006 Fujitsu Limited
>>> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
>>> Copyright (C) 2005 NEC Corporation
>>> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
>>> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
>>> This program is free software, covered by the GNU General Public
>>> License,
>>> and you are welcome to change it and/or distribute copies of it
>>> under
>>> certain conditions. Enter "help copying" to see the conditions.
>>> This program has absolutely no warranty. Enter "help warranty" for
>>> details.
>>>
>>> GNU gdb (GDB) 7.3.1
>>> Copyright (C) 2011 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later <
>>> http://gnu.org/licenses/gpl.html >
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law. Type "show
>>> copying"
>>> and "show warranty" for details.
>>> This GDB was configured as "x86_64-unknown-linux-gnu"...
>>>
>>> KERNEL: /boot/vmlinuz-2.6.27.19-5-default.vmlinux-debug
>>> DUMPFILE: /dev/mem
>>> CPUS: 2
>>> DATE: Sat Mar 24 19:10:45 2012
>>> UPTIME: 00:07:17
>>> LOAD AVERAGE: 0.16, 0.19, 0.12
>>> TASKS: 189
>>> NODENAME: sles11
>>> RELEASE: 2.6.27.19-5-default
>>> VERSION: #6 SMP Sat Nov 12 08:58:08 CST 2011
>>> MACHINE: x86_64 (2393 Mhz)
>>> MEMORY: 3.9 GB
>>> PID: 4590
>>> COMMAND: "crash"
>>> TASK: ffff8801230fa340 [THREAD_INFO: ffff880131c86000]
>>> CPU: 0
>>> STATE: TASK_RUNNING (ACTIVE)
>>>
>>> crash>
>>>
>>>
>>>
>>>
>>>
>>>
>>> sles11:/mnt/sda12/crash/2012-03-24-18:57 # crash
>>> vmlinux-2.6.27.19-5-default.gz vmlinux-2.6.27.19-5-default.debug
>>> vmcore
>>> crash 6.0.4
>>> Copyright (C) 2002-2012 Red Hat, Inc.
>>> Copyright (C) 2004, 2005, 2006 IBM Corporation
>>> Copyright (C) 1999-2006 Hewlett-Packard Co
>>> Copyright (C) 2005, 2006 Fujitsu Limited
>>> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.
>>> Copyright (C) 2005 NEC Corporation
>>> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.
>>> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.
>>> This program is free software, covered by the GNU General Public
>>> License,
>>> and you are welcome to change it and/or distribute copies of it
>>> under
>>> certain conditions. Enter "help copying" to see the conditions.
>>> This program has absolutely no warranty. Enter "help warranty" for
>>> details.
>>>
>>> please wait... (uncompressing vmlinux-2.6.27.19-5-default.gz)
>>>
>>> compressed kdump: header-> utsname.machine:
>>> compressed kdump: memory bitmap offset: 2000
>>> diskdump_data:
>>> filename: vmcore
>>> flags: 6 (KDUMP_CMPRS_LOCAL|ERROR_EXCLUDED)
>>> dfd: 4
>>> ofp: 0
>>> machine_type: 62 (EM_X86_64)
>>>
>>> header: dee020
>>> signature: "KDUMP "
>>> header_version: 1
>>> utsname:
>>> sysname:
>>> nodename:
>>> release:
>>> version:
>>> machine:
>>> domainname:
>>> timestamp:
>>> tv_sec: 0
>>> tv_usec: 0
>>> status: 0 ()
>>> block_size: 4096
>>> sub_hdr_size: 1
>>> bitmap_blocks: 80
>>> max_mapnr: 1294336
>>> total_ram_blocks: 0
>>> device_blocks: 0
>>> written_blocks: 0
>>> current_cpu: 0
>>> nr_cpus: 1
>>> tasks[nr_cpus]: 0
>>>
>>> sub_header: 0 (n/a)
>>>
>>> sub_header_kdump: def030
>>> phys_base: 0
>>> dump_level: 0 (0x0)
>>>
>>> data_offset: 52000
>>> block_size: 4096
>>> block_shift: 12
>>> bitmap: 7f812de69010
>>> bitmap_len: 327680
>>> dumpable_bitmap: 7f812de18010
>>> byte: 0
>>> bit: 0
>>> compressed_page: e00a40
>>> curbufptr: 0
>>>
>>> page_cache_hdr[0]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df0a30
>>> pg_hit_count: 0
>>> page_cache_hdr[1]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df1a30
>>> pg_hit_count: 0
>>> page_cache_hdr[2]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df2a30
>>> pg_hit_count: 0
>>> page_cache_hdr[3]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df3a30
>>> pg_hit_count: 0
>>> page_cache_hdr[4]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df4a30
>>> pg_hit_count: 0
>>> page_cache_hdr[5]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df5a30
>>> pg_hit_count: 0
>>> page_cache_hdr[6]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df6a30
>>> pg_hit_count: 0
>>> page_cache_hdr[7]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df7a30
>>> pg_hit_count: 0
>>> page_cache_hdr[8]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df8a30
>>> pg_hit_count: 0
>>> page_cache_hdr[9]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: df9a30
>>> pg_hit_count: 0
>>> page_cache_hdr[10]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: dfaa30
>>> pg_hit_count: 0
>>> page_cache_hdr[11]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: dfba30
>>> pg_hit_count: 0
>>> page_cache_hdr[12]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: dfca30
>>> pg_hit_count: 0
>>> page_cache_hdr[13]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: dfda30
>>> pg_hit_count: 0
>>> page_cache_hdr[14]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: dfea30
>>> pg_hit_count: 0
>>> page_cache_hdr[15]:
>>> pg_flags: 0 ()
>>> pg_addr: 0
>>> pg_bufptr: dffa30
>>> pg_hit_count: 0
>>>
>>> page_cache_buf: df0a30
>>> evict_index: 0
>>> evictions: 0
>>> accesses: 0
>>> cached_reads: 0
>>> valid_pages: df0040
>>> readmem: read_diskdump()
>>> [?1034hcompressed kdump: phys_base: 0
>>> gdb vmlinux-2.6.27.19-5-default.debug
>>> GNU gdb (GDB) 7.3.1
>>> Copyright (C) 2011 Free Software Foundation, Inc.
>>> License GPLv3+: GNU GPL version 3 or later <
>>> http://gnu.org/licenses/gpl.html >
>>> This is free software: you are free to change and redistribute it.
>>> There is NO WARRANTY, to the extent permitted by law. Type "show
>>> copying"
>>> and "show warranty" for details.
>>> This GDB was configured as "x86_64-unknown-linux-gnu"...
>>> GETBUF(248 -> 0)
>>> GETBUF(1500 -> 1)
>>>
>>> please wait... (patching 35285 gdb minimal_symbol values)
>>>
>>> FREEBUF(1)
>>> FREEBUF(0)
>>> <readmem: ffffffff804b8210, KVADDR, "kernel_config_data", 32768,
>>> (ROE), 17e92b0>
>>> <read_diskdump: addr: ffffffff804b8210 paddr: 4b8210 cnt: 3568>
>>> read_diskdump: paddr/pfn: 4b8210/4b8 -> cache physical page: 4b8000
>>> <read_diskdump: addr: ffffffff804b9000 paddr: 4b9000 cnt: 4096>
>>> read_diskdump: paddr/pfn: 4b9000/4b9 -> cache physical page: 4b9000
>>> <read_diskdump: addr: ffffffff804ba000 paddr: 4ba000 cnt: 4096>
>>> read_diskdump: paddr/pfn: 4ba000/4ba -> cache physical page: 4ba000
>>> <read_diskdump: addr: ffffffff804bb000 paddr: 4bb000 cnt: 4096>
>>> read_diskdump: paddr/pfn: 4bb000/4bb -> cache physical page: 4bb000
>>> <read_diskdump: addr: ffffffff804bc000 paddr: 4bc000 cnt: 4096>
>>> read_diskdump: paddr/pfn: 4bc000/4bc -> cache physical page: 4bc000
>>> <read_diskdump: addr: ffffffff804bd000 paddr: 4bd000 cnt: 4096>
>>> read_diskdump: paddr/pfn: 4bd000/4bd -> cache physical page: 4bd000
>>> <read_diskdump: addr: ffffffff804be000 paddr: 4be000 cnt: 4096>
>>> read_diskdump: paddr/pfn: 4be000/4be -> cache physical page: 4be000
>>> <read_diskdump: addr: ffffffff804bf000 paddr: 4bf000 cnt: 4096>
>>> read_diskdump: paddr/pfn: 4bf000/4bf -> cache physical page: 4bf000
>>> <read_diskdump: addr: ffffffff804c0000 paddr: 4c0000 cnt: 528>
>>> read_diskdump: paddr/pfn: 4c0000/4c0 -> cache physical page: 4c0000
>>> WARNING: could not find MAGIC_START!
>>> GETBUF(248 -> 0)
>>> FREEBUF(0)
>>> GETBUF(64 -> 0)
>>> <readmem: ffffffff80a59910, KVADDR, "possible", 64, (ROE), da00e0>
>>> <read_diskdump: addr: ffffffff80a59910 paddr: a59910 cnt: 64>
>>> read_diskdump: paddr/pfn: a59910/a59 -> cache physical page: a59000
>>> cpu_possible_map:
>>> <readmem: ffffffff8095d950, KVADDR, "present", 64, (ROE), da00e0>
>>> <read_diskdump: addr: ffffffff8095d950 paddr: 95d950 cnt: 64>
>>> read_diskdump: paddr/pfn: 95d950/95d -> cache physical page: 95d000
>>> cpu_present_map: 2 3 6 7 10 11 14 15 18 19 22 23 26 27 30 31 34 35
>>> 38
>>> 39 42 43 46 47 50 51 54 55 58 59 62 63 66 67 70 71 74 75 78 79 82
>>> 83
>>> 86 87 90 91 94 95 98 99 102 103 106 107 110 111 114 115 118 119 122
>>> 123 126 127 130 131 134 135 138 139 142 143 146 147 150 151 154 155
>>> 158 159 162 163 166 167 170 171 174 175 178 179 182 183 186 187 190
>>> 191 194 195 198 199 202 203 206 207 210 211 214 215 218 219 222 223
>>> 226 227 230 231 234 235 238 239 242 243 246 247 250 251 254 255 258
>>> 259 262 263 266 267 270 271 274 275 278 279 282 283 286 287 290 291
>>> 294 295 298 299 302 303 306 307 310 311 314 315 318 319 322 323 326
>>> 327 330 331 334 335 338 339 342 343 346 347 350 351 354 355 358 359
>>> 362 363 366 367 370 371 374 375 378 379 382 383 386 387 390 391 394
>>> 395 398 399 402 403 406 407 410 411 414 415 418 419 422 423 426 427
>>> 430 431 434 435 438 439 442 443 446 447 450 451 454 455 458 459 462
>>> 463 466 467 470 471 474 475 478 479 482 483 486 487 490 491 494 495
>>> 498 499 502 503 506 507 510 511
>>> <readmem: ffffffff808cbe90, KVADDR, "online", 64, (ROE), da00e0>
>>> <read_diskdump: addr: ffffffff808cbe90 paddr: 8cbe90 cnt: 64>
>>> read_diskdump: paddr/pfn: 8cbe90/8cb -> cache physical page: 8cb000
>>> cpu_online_map:
>>> FREEBUF(0)
>>> <readmem: ffffffff80abaf00, KVADDR, "xtime", 16, (FOE), c5e430>
>>> <read_diskdump: addr: ffffffff80abaf00 paddr: abaf00 cnt: 16>
>>> read_diskdump: paddr/pfn: abaf00/aba -> cache physical page: aba000
>>> <readmem: ffffffff806dc894, KVADDR, "init_uts_ns", 390, (ROE),
>>> c5ea1c>
>>> <read_diskdump: addr: ffffffff806dc894 paddr: 6dc894 cnt: 390>
>>> read_diskdump: paddr/pfn: 6dc894/6dc -> cache physical page: 6dc000
>>> base kernel version: 0.0.0
>>> <readmem: ffffffff804a7000, KVADDR, "accessible check", 8, (ROE|Q),
>>> 7fff35eeba88>
>>> <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 8>
>>> read_diskdump: paddr/pfn: 4a7000/4a7 -> cache physical page: 4a7000
>>> <readmem: ffffffff804a7000, KVADDR, "readstring characters", 1499,
>>> (ROE|Q), 7fff35eeaa70>
>>> <read_diskdump: addr: ffffffff804a7000 paddr: 4a7000 cnt: 1499>
>>> read_diskdump: paddr/pfn: 4a7000/4a7 -> physical page is cached:
>>> 4a7000
>>> linux_banner:
>>> 紉;���
>>> crash: vmlinux-2.6.27.19-5-default.debug and vmcore do not match!
>>>
>>> Usage:
>>>
>>> crash [OPTION]... NAMELIST MEMORY-IMAGE (dumpfile form)
>>> crash [OPTION]... [NAMELIST] (live system form)
>>>
>>> Enter "crash -h" for details.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Crash-utility mailing list
>>> Crash-utility@redhat.com
>>> https://www.redhat.com/mailman/listinfo/crash-utility
>>>
>>> --
>>> Crash-utility mailing list
>>> Crash-utility@redhat.com
>>> https://www.redhat.com/mailman/listinfo/crash-utility
>>>
>>
>>
>

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility
 

Thread Tools




All times are GMT. The time now is 01:01 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org