Fix relocation address
 

Hi!

I'm having problems getting kdumps from my relocatable kernel (2.6.31-8)
working with crash on a IA-32 board. I use makedumpfile to generate a
compressed dump, and when I try to load it with crash I get

./crash vmlinux vmcore --reloc=0x100000
crash: invalid kernel virtual address: 98 type: "present"
WARNING: cannot read cpu_present_map
crash: invalid kernel virtual address: 908bd975 type: "online"
WARNING: cannot read cpu_online_map
crash: cannot determine base kernel version
crash: vmlinux and vmcore do not match!

specifying --reloc also fails:

./crash vmlinux vmcore --reloc=0x100000
crash: seek error: kernel virtual address: c01a2108 type: "cpu_possible_mask"


So I started looking into the code and found something which looks like
a typo in relocate() (patch below). Changing this makes crash work for
me.

Great tool by the way, leaves you longing for the next kernel panic ;-)

// Simon

--- orig-crash-4.1.2/symbols.c 2009-12-09 21:37:40.000000000 +0100
+++ crash-4.1.2/symbols.c 2009-12-17 16:03:24.000000000 +0100
@@ -671,7 +671,7 @@ relocate(ulong symval, char *symname, in
break;
}

- return (symval - kt->relocate);
+ return (symval + kt->relocate);
}

/*

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Fix relocation address
 

----- "Simon Kagstrom" <simon.kagstrom@netinsight.net> wrote:

> Hi!
>
> I'm having problems getting kdumps from my relocatable kernel (2.6.31-8)
> working with crash on a IA-32 board. I use makedumpfile to generate a
> compressed dump, and when I try to load it with crash I get
>
> ./crash vmlinux vmcore --reloc=0x100000
> crash: invalid kernel virtual address: 98 type: "present"
> WARNING: cannot read cpu_present_map
> crash: invalid kernel virtual address: 908bd975 type: "online"
> WARNING: cannot read cpu_online_map
> crash: cannot determine base kernel version
> crash: vmlinux and vmcore do not match!
>
> specifying --reloc also fails:
>
> ./crash vmlinux vmcore --reloc=0x100000
> crash: seek error: kernel virtual address: c01a2108 type:
> "cpu_possible_mask"
>
>
> So I started looking into the code and found something which looks like
> a typo in relocate() (patch below). Changing this makes crash work for me.

Actually it's not a typo -- your patch would presumably break with all kernels
that have a CONFIG_PHYSICAL_START is greater than CONFIG_PHYSICAL_ALIGN, which
is what the patch was written to handle.

What are your kernel's CONFIG_PHYSICAL_START and CONFIG_PHYSICAL_ALIGN
values? Does crash work with your kernel on the live system?

Anyway, I believe that the fix would require support for supplying a
negative --reloc value.

>
> Great tool by the way, leaves you longing for the next kernel panic
> ;-)
>
> // Simon
>
> --- orig-crash-4.1.2/symbols.c 2009-12-09 21:37:40.000000000 +0100
> +++ crash-4.1.2/symbols.c 2009-12-17 16:03:24.000000000 +0100
> @@ -671,7 +671,7 @@ relocate(ulong symval, char *symname, in
> break;
> }
>
> - return (symval - kt->relocate);
> + return (symval + kt->relocate);
> }
>
> /*

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Fix relocation address
 

----- "Dave Anderson" <anderson@redhat.com> wrote:

> ----- "Simon Kagstrom" <simon.kagstrom@netinsight.net> wrote:
>
> > Hi!
> >
> > I'm having problems getting kdumps from my relocatable kernel (2.6.31-8)
> > working with crash on a IA-32 board. I use makedumpfile to generate a
> > compressed dump, and when I try to load it with crash I get
> >
> > ./crash vmlinux vmcore --reloc=0x100000
> > crash: invalid kernel virtual address: 98 type: "present"
> > WARNING: cannot read cpu_present_map
> > crash: invalid kernel virtual address: 908bd975 type: "online"
> > WARNING: cannot read cpu_online_map
> > crash: cannot determine base kernel version
> > crash: vmlinux and vmcore do not match!
> >
> > specifying --reloc also fails:
> >
> > ./crash vmlinux vmcore --reloc=0x100000
> > crash: seek error: kernel virtual address: c01a2108 type: "cpu_possible_mask"
> >
> >
> > So I started looking into the code and found something which looks like
> > a typo in relocate() (patch below). Changing this makes crash work for me.
>
> Actually it's not a typo -- your patch would presumably break with all kernels
> that have a CONFIG_PHYSICAL_START is greater than CONFIG_PHYSICAL_ALIGN, which
> is what the patch was written to handle.
>
> What are your kernel's CONFIG_PHYSICAL_START and CONFIG_PHYSICAL_ALIGN
> values? Does crash work with your kernel on the live system?
>
> Anyway, I believe that the fix would require support for supplying a
> negative --reloc value.

On the other hand, if the config values were the other way around, the
problem didn't use to show up -- at least according to list item "1)"
below in the changelog:

4.0-4.5 - Addresses FC7/upstream x86 kernels that have been configured such
that the vmlinux symbol values do not match their relocated values
when loaded. If CONFIG_PHYSICAL_START contains a value that is
greater then CONFIG_PHYSICAL_ALIGN, then this mismatch occurs.
Since the crash utility and its embedded gdb have always expected
that the compiled-in kernel symbol addresses are "real", the virtual
to physical translation fails, leading to an initialization-time
failure with the message: "crash: vmlinux and /dev/crash do not
match!" (/dev/mem or the dumpfile name may replace /dev/crash).
To deal with this issue, there are several alternatives:

1) Configure the kernel with CONFIG_PHYSICAL_START less than
or equal to CONFIG_PHYSICAL_ALIGN. Having done that, there
is no problem; the resultant vmlinux file will be loaded at
the address for which it was compiled, which has always
been the case.
2) Since /proc/kallsyms uses the same format as a System.map file,
and since it reflects the relocated symbol addresses, it
can be placed on the crash command line as if it were
a System.map file. (Note that the System.map file created
by these relocated kernels contains the same "wrong" symbol
values as the vmlinux file from which it was created.)
3) On a live system that has /proc/kallsyms (i.e., the kernel was
configured with CONFIG_KALLSYMS), this version of the crash
utility will replace/patch the vmlinux symbol values with those
seen in /proc/kallsyms. The relocation value will be displayed
as a WARNING message during initialization.
4) On a dumpfile, the relocation will not be performed automatically
as on a live system. It will require the addition of the
/proc/kallsyms on the command line, or if run on a different
host, a copy of the crashed system's /proc/kallsyms may be
used.
5) Alternatively on a dumpfile, a new command line option has been
created to specify the relocation amount. For example, if a
kernel was configured with a CONFIG_PHYSICAL_START value of 16MB
and a CONFIG_PHYSICAL_ALIGN of 4MB, that results in a relocation
of 12MB. To specify that, enter "crash --reloc=12m ..." on the
command line. (Recall that if crash is run on the live system,
a WARNING message will specify the relocation amount.)

Using /proc/kallsyms or a --reloc=[size] as a command line argument
is similar to using a System.map file, in that it results in the loss
of the use of line number debug data. (anderson@redhat.com)

I wonder if you can use the unpatched crash, but supply a --reloc value that
will cause a wrap-around to the correct value?

Dave


>
> >
> > Great tool by the way, leaves you longing for the next kernel panic
> > ;-)
> >
> > // Simon
> >
> > --- orig-crash-4.1.2/symbols.c 2009-12-09 21:37:40.000000000 +0100
> > +++ crash-4.1.2/symbols.c 2009-12-17 16:03:24.000000000 +0100
> > @@ -671,7 +671,7 @@ relocate(ulong symval, char *symname, in
> > break;
> > }
> >
> > - return (symval - kt->relocate);
> > + return (symval + kt->relocate);
> > }
> >
> > /*
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Fix relocation address
 

On Thu, 17 Dec 2009 11:17:56 -0500 (EST)
Dave Anderson <anderson@redhat.com> wrote:

> > > So I started looking into the code and found something which looks like
> > > a typo in relocate() (patch below). Changing this makes crash work for me.
> >
> > Actually it's not a typo -- your patch would presumably break with all kernels
> > that have a CONFIG_PHYSICAL_START is greater than CONFIG_PHYSICAL_ALIGN, which
> > is what the patch was written to handle.
> >
> > What are your kernel's CONFIG_PHYSICAL_START and CONFIG_PHYSICAL_ALIGN
> > values? Does crash work with your kernel on the live system?

You are right. I had problems with getting things working, so I've
played around with various settings. I had CONFIG_PHYSICAL_START set to
0 and CONFIG_PHYSICAL_ALIGN set to 0x100000. Setting these to e.g.,
0x100000 and 0x100000 unbreaks things again.

I don't need to supply --reloc either then, not sure what I did wrong
before. I'm sticking with sane settings from now on.

> > Anyway, I believe that the fix would require support for supplying a
> > negative --reloc value.
>
> On the other hand, if the config values were the other way around, the
> problem didn't use to show up -- at least according to list item "1)"
> below in the changelog:
>
> 1) Configure the kernel with CONFIG_PHYSICAL_START less than
> or equal to CONFIG_PHYSICAL_ALIGN. Having done that, there
> is no problem; the resultant vmlinux file will be loaded at
> the address for which it was compiled, which has always
> been the case.

> I wonder if you can use the unpatched crash, but supply a --reloc value that
> will cause a wrap-around to the correct value?

Well, I suppose that would work if it was possible to supply a negative
--reloc value, but I'm not sure it's really worth it. What would be
nice would be to get a more descriptive error message.

Thanks for the help, please ignore the patch.

// Simon

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Fix relocation address
 

----- "Simon Kagstrom" <simon.kagstrom@netinsight.net> wrote:

> On Thu, 17 Dec 2009 11:17:56 -0500 (EST)
> Dave Anderson <anderson@redhat.com> wrote:
>
> > > > So I started looking into the code and found something which looks like
> > > > a typo in relocate() (patch below). Changing this makes crash work for me.
> > >
> > > Actually it's not a typo -- your patch would presumably break with all kernels
> > > that have a CONFIG_PHYSICAL_START is greater than CONFIG_PHYSICAL_ALIGN, which
> > > is what the patch was written to handle.
> > >
> > > What are your kernel's CONFIG_PHYSICAL_START and CONFIG_PHYSICAL_ALIGN
> > > values? Does crash work with your kernel on the live system?
>
> You are right. I had problems with getting things working, so I've
> played around with various settings. I had CONFIG_PHYSICAL_START set to
> 0 and CONFIG_PHYSICAL_ALIGN set to 0x100000. Setting these to e.g.,
> 0x100000 and 0x100000 unbreaks things again.
>
> I don't need to supply --reloc either then, not sure what I did wrong
> before. I'm sticking with sane settings from now on.
>
> > > Anyway, I believe that the fix would require support for supplying a
> > > negative --reloc value.
> >
> > On the other hand, if the config values were the other way around, the
> > problem didn't use to show up -- at least according to list item "1)"
> > below in the changelog:
> >
> > 1) Configure the kernel with CONFIG_PHYSICAL_START less than
> > or equal to CONFIG_PHYSICAL_ALIGN. Having done that, there
> > is no problem; the resultant vmlinux file will be loaded at
> > the address for which it was compiled, which has always
> > been the case.
>
> > I wonder if you can use the unpatched crash, but supply a --reloc value that
> > will cause a wrap-around to the correct value?
>
> Well, I suppose that would work if it was possible to supply a negative
> --reloc value, but I'm not sure it's really worth it. What would be
> nice would be to get a more descriptive error message.

Yeah, the problem is that the "do not match" errors can result from
a multitude of error scenarios. Usually by entering a "-d <number>"
on the command line (the higher the debug number the more verbose),
the issue generating the failure typically is evident.

>
> Thanks for the help, please ignore the patch.

OK for now -- and thanks for posting. It's only a matter of time before
somebody else runs into the same thing.

Thanks,
Dave

>
> // Simon

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility