Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Crash Utility (http://www.linux-archive.org/crash-utility/)
-   -   Crash fails with the error: "crash: read error: kernel virtual address:..." (http://www.linux-archive.org/crash-utility/253632-crash-fails-error-crash-read-error-kernel-virtual-address.html)

Adhiraj Joshi 02-27-2009 08:56 AM

Crash fails with the error: "crash: read error: kernel virtual address:..."
 
Hi Dave,

Thanks for the quick reply! I followed approach 1 of disabling CONFIG_STRICT_DEVMEM and the problem disappeared. Now a new problem :-) On invoking crash, I get the following warning:
WARNING: cannot access vmalloc'd module memory


Because of this I am not able to run commands like "mod":
crash> mod
mod: cannot access vmalloc'd module memory
crash>

Should the approach 2 or 3 that you mentioned below solve this problem? I have upgraded crash to the latest version (4.0-7.7).


Thanks and regards,
Adhiraj.


On Wed, Feb 25, 2009 at 7:45 PM, Dave Anderson <anderson@redhat.com> wrote:



----- "Adhiraj Joshi" <adhiraj@linsyssoft.com> wrote:



> Hi All,

>

> On my x86 machine with fedora 10, I get an error which says: "crash:

> read error: kernel virtual address:...". The running kernel is not the

> Fedora 10 kernel, I have installed a latest vanilla kernel (2.6.27.12)

> on my machine.

>

> On googling, I found that someone has reported a similar problem but

> which was on Fedora kernel and x86_64 arch. Here is the link to that bug:

> https://bugzilla.redhat.com/show_bug.cgi?id=237383

>

> In my case, this is the output from the crash command:

>

> [root@maveric ~]# crash /lib/modules/`uname -r`/build/vmlinux

> crash 4.0-7

> Copyright (C) 2002, 2003, 2004, 2005, 2006, 2007, 2008 Red Hat, Inc.

> Copyright (C) 2004, 2005, 2006 IBM Corporation

> Copyright (C) 1999-2006 Hewlett-Packard Co

> Copyright (C) 2005, 2006 Fujitsu Limited

> Copyright (C) 2006, 2007 VA Linux Systems Japan K.K.

> Copyright (C) 2005 NEC Corporation

> Copyright (C) 1999, 2002, 2007 Silicon Graphics, Inc.

> Copyright (C) 1999, 2000, 2001, 2002 Mission Critical Linux, Inc.

> This program is free software, covered by the GNU General Public License,

> and you are welcome to change it and/or distribute copies of it under

> certain conditions. Enter "help copying" to see the conditions.

> This program has absolutely no warranty. Enter "help warranty" for

> details.

>

> GNU gdb 6.1

> Copyright 2004 Free Software Foundation, Inc.

> GDB is free software, covered by the GNU General Public License, and you are

> welcome to change it and/or distribute copies of it under certain conditions.

> Type "show copying" to see the conditions.

> There is absolutely no warranty for GDB. Type "show warranty" for details.

> This GDB was configured as "i686-pc-linux-gnu"...

>

> crash: read error: kernel virtual address: c08b0ae8 type: "cpu_possible_map"

> WARNING: cannot read cpu_possible_map

> crash: read error: kernel virtual address: c0801a94 type: "cpu_present_map"

> WARNING: cannot read cpu_present_map

> crash: read error: kernel virtual address: c080155c type: "cpu_online_map"

> WARNING: cannot read cpu_online_map

> crash: read error: kernel virtual address: c08dd700 type: "xtime"

> [root@maveric ~]#

>

> This is the "uname -a" output:

> "Linux maveric 2.6.27.12-adhi #1 SMP Fri Feb 20 22:13:54 IST 2009 i686

> i686 i386 GNU/Linux"

>

> Does anyone has any idea on this?



Your kernel is configured with CONFIG_STRICT_DEVMEM, which disallows

the reading of /dev/mem above 1MB physical. *That makes it useless for

the crash utility:



*https://www.redhat.com/archives/crash-utility/2009-February/msg00003.html



Fedora and RHEL kernels contain the /dev/crash driver, which gets automatically

installed upon crash invocation on the live system. *That driver, found in

the kernel file "drivers/char/crash.c", is a replacement for the (restricted)

/dev/mem driver.



For your kernel, you've got 3 options:



(1) Rebuild your kernel without the CONFIG_STRICT_DEVMEM restriction.

(2) Port the Fedora /dev/crash driver (./drivers/char/crash.c) to your kernel.

(3) Write a kretprobe module that tinkers with the return value of the

* *kernel's devmem_is_allowed() function such that it always returns 1.



And w/respect to each option:



(1) Obviously this is the course of least resistance.



(2) Porting the RHEL/Fedora /dev/crash driver is possible, but since page_is_ram()

* *is not EXPORT_GPL()'d in 2.6.27 upstream kernels, then its usage by the driver

* *would have to be removed. *And if you were to make it EXPORT_GPL(), then it

* *makes far more sense to just remove the CONFIG_STRICT_DEVMEM instead.



(3) I'll append some information re: writing a kretprobe module, but like

* *porting the /dev/crash driver, it may require rebuilding your kernel anyway.

* *And if that's the case, just remove the CONFIG_STRICT_DEVMEM restriction.



Also, you should upgrade from 4.0-7. *There have been several fixes

for 2.6.27 kernels since that version.



Dave



----------------------------------------------------------------------------



Writing a kretprobe for devmem_is_allowed():



There are are currently three types of probes: kprobes, jprobes, and

kretprobes (also called return probes), but for this purpose, a

kretprobe is required because it allows the return value of the

devmem_is_allowed() function to be modified such that it will

always return 1.



The kernel documentation contains both directions for building

a kretprobe module and example module files for each kprobe type.

Pre-2.6.25 kernel trees put sample cut-and-pastable module files

contained within the "./Documentation/kprobes.txt" file itself.

2.6.25 and later kernel trees locate them in the "./samples/kprobes"

directory.



Note that pre-2.6.25 kernels must have been configured with both

CONFIG_KPROBES, CONFIG_KALLSYMS and CONFIG_MODULES turned on.

Additionally, 2.6.25 and later kernels also need CONFIG_KRETPROBES,

In any case, if the target kernel configs preclude the installation

of the module, then it probably makes more sense to rebuild the

kernel with CONFIG_STRICT_DEVMEM turned off, and avoid this kprobe

approach entirely.



Take the "kretprobe-example.c" cut-and-pastable file from the

the "./Documentation/kprobes.txt" file, or the standalone

"./samples/kprobes/kretprobe_example.c" file, whichever is

appropriate, and modify the kretprobe_init() and ret_handler()

functions like so:



*static int __init kretprobe_init(void)

*{

* * * * int ret;



- * * * *my_kretprobe.kp.symbol_name = func_name;

+ * * * *my_kretprobe.kp.symbol_name = "devmem_is_allowed";

* * * * ret = register_kretprobe(&my_kretprobe);

* * * * if (ret < 0) {

* * * * * * * * printk(KERN_INFO "register_kretprobe failed, returned %d
",

* * * * * * * * * * * * * * * * ret);

* * * * * * * * return -1;

* * * * }

* * * * printk(KERN_INFO "Planted return probe at %s: %p
",

* * * * * * * * * * * * my_kretprobe.kp.symbol_name, my_kretprobe.kp.addr);

* * * * return 0;

*}



and make the ret_handler() function to simply do this:



*static int ret_handler(struct kretprobe_instance *ri, struct pt_regs *regs)

*{

* * * * regs->ax = 1;

* * * * return 0;

*}



Earlier kernel versions may have different pt_regs structure member

names for the return register, i.e., "regs->eax" or "regs->rax" for

the x86 and x86_64 architectures respectively.



Build the kretprobe module as directed in ./Documentation/kprobes.txt"

and insmod it, after which crash will run just fine with /dev/mem.













--

Crash-utility mailing list

Crash-utility@redhat.com

https://www.redhat.com/mailman/listinfo/crash-utility



--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility


All times are GMT. The time now is 02:43 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.