Question about ARM module address range
 

Hi guys,

While looking at the ARM per-cpu address range issue, I ran into
something re: the ARM module address range values that has me
confused.

In arm_init(), POST_VM, you've got:

machdep->machspec->modules_end = machdep->kvbase - 1;

But it never gets used, because here in arm_is_module_addr() the
local variable "module_end" is used instead -- although it is set
to the same value:

static int
arm_is_module_addr(ulong vaddr)
{
ulong modules_start;
ulong modules_end = machdep->kvbase - 1;

if (!MODULES_VADDR) {
/*
* In case we are still initializing, and vm_init() has not been
* called, we use defaults here which is 16MB below kernel start
* address.
*/
modules_start = machdep->kvbase - 16 * 1024 * 1024;
} else {
modules_start = MODULES_VADDR;
}

return (vaddr >= modules_start && vaddr <= modules_end);
}

and where MODULES_VADDR is #define'd as:

#define MODULES_VADDR (machdep->machspec->modules_vaddr)

So given that "modules_end" is set to "machdep->kvbase - 1", and given the
output from "help -m" on the 3 sample ARM dumps I have:

2.6.35:

crash> help -m
...
kvbase: c0000000
...
vmalloc_start_addr: d0000000
modules_vaddr: d0807000
modules_end: bfffffff
kernel_text_start: c0008000
...

2.6.36:

crash> help -m
...
kvbase: c0000000
...
vmalloc_start_addr: c5800000
modules_vaddr: c6024000
modules_end: bfffffff
kernel_text_start: c0008000

2.6.38:

crash> help -m
...
kvbase: c0000000
...
vmalloc_start_addr: d0000000
modules_vaddr: d0807000
modules_end: bfffffff
kernel_text_start: c0008000
...

So given that "modules_end" is always below the kvbase (bfffffff),
then arm_is_module_addr() will always return FALSE -- except maybe
during pre-vm_init() time when the range is temporarily set to
be the "default range" between bf000000 and bfffffff. (and would
there *ever* be a module in that range?)

Anyway, although I don't think it's a problem, given that the
only caller of arm_is_module_addr() is arm_is_vmalloc_addr(),
and that function will fall through and still return TRUE if
it is a module address.

But it just looks strange, and at least it would seem that:

(1) the machdep->machspec->modules_end version should be used, and
(2) it should be updated post-vm_init() so that the arm_is_module_addr()
function actually works?

Or am I missing something?

Dave

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Question about ARM module address range
 

Hi Dave

I have taken a short look at modules_vaddr and module_end and I have both seen relevant data:

crash> help -m
modules_vaddr: bf000000
modules_end: bfffffff

and data similar to what you see. What I also have seen is that when modules are loaded then modules_vaddr and modules_end seems correct and when no modules have been loaded then strange values are presented. I have looked at too few examples to be certain that this is "always" true.

I assume (not checked in source code) that no vmalloc area is allocated for modules if no modules are loaded. Then the function first_vmalloc_address() will return data which is stored in modules_vaddr but has nothing to do with this.

So I think that the question is what values should modules_vaddr and modules_end have if no modules are loaded. Does it matter, except that it might be confusing for a user? Looking at arm.c where modules_vaddr and modules_end are used, I think the code will behaves correctly (by luck?!), also in the case of no modules.

Jan

Jan Karlsson
Senior Software Engineer
MIB
*
Sony Mobile Communications
Tel: +46703062174
sonymobile.com
*


-----Original Message-----
From: Dave Anderson [mailto:anderson@redhat.com]
Sent: torsdag den 19 juli 2012 16:05
To: Discussion list for crash utility usage, maintenance and development
Cc: Karlsson, Jan; Fänge, Thomas; Mika Westerberg; per fransson ml
Subject: Question about ARM module address range


Hi guys,

While looking at the ARM per-cpu address range issue, I ran into something re: the ARM module address range values that has me confused.

In arm_init(), POST_VM, you've got:

machdep->machspec->modules_end = machdep->kvbase - 1;

But it never gets used, because here in arm_is_module_addr() the local variable "module_end" is used instead -- although it is set to the same value:

static int
arm_is_module_addr(ulong vaddr)
{
ulong modules_start;
ulong modules_end = machdep->kvbase - 1;

if (!MODULES_VADDR) {
/*
* In case we are still initializing, and vm_init() has not been
* called, we use defaults here which is 16MB below kernel start
* address.
*/
modules_start = machdep->kvbase - 16 * 1024 * 1024;
} else {
modules_start = MODULES_VADDR;
}

return (vaddr >= modules_start && vaddr <= modules_end);
}

and where MODULES_VADDR is #define'd as:

#define MODULES_VADDR (machdep->machspec->modules_vaddr)

So given that "modules_end" is set to "machdep->kvbase - 1", and given the output from "help -m" on the 3 sample ARM dumps I have:

2.6.35:

crash> help -m
...
kvbase: c0000000
...
vmalloc_start_addr: d0000000
modules_vaddr: d0807000
modules_end: bfffffff
kernel_text_start: c0008000
...

2.6.36:

crash> help -m
...
kvbase: c0000000
...
vmalloc_start_addr: c5800000
modules_vaddr: c6024000
modules_end: bfffffff
kernel_text_start: c0008000

2.6.38:

crash> help -m
...
kvbase: c0000000
...
vmalloc_start_addr: d0000000
modules_vaddr: d0807000
modules_end: bfffffff
kernel_text_start: c0008000
...

So given that "modules_end" is always below the kvbase (bfffffff), then arm_is_module_addr() will always return FALSE -- except maybe during pre-vm_init() time when the range is temporarily set to be the "default range" between bf000000 and bfffffff. (and would there *ever* be a module in that range?)

Anyway, although I don't think it's a problem, given that the only caller of arm_is_module_addr() is arm_is_vmalloc_addr(), and that function will fall through and still return TRUE if it is a module address.

But it just looks strange, and at least it would seem that:

(1) the machdep->machspec->modules_end version should be used, and
(2) it should be updated post-vm_init() so that the arm_is_module_addr()
function actually works?

Or am I missing something?

Dave


--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Question about ARM module address range
 

> Hi Dave
>
> I have taken a short look at modules_vaddr and module_end and I have both
> seen relevant data:
>
> crash> help -m
> modules_vaddr: bf000000
> modules_end: bfffffff
>
> and data similar to what you see. What I also have seen is that when
> modules are loaded then modules_vaddr and modules_end seems correct and
> when no modules have been loaded then strange values are presented. I have
> looked at too few examples to be certain that this is "always" true.
>
> I assume (not checked in source code) that no vmalloc area is allocated
> for modules if no modules are loaded. Then the function
> first_vmalloc_address() will return data which is stored in modules_vaddr
> but has nothing to do with this.
>
> So I think that the question is what values should modules_vaddr and
> modules_end have if no modules are loaded. Does it matter, except that it
> might be confusing for a user? Looking at arm.c where modules_vaddr and
> modules_end are used, I think the code will behaves correctly (by luck?!),
> also in the case of no modules.
>
> Jan
>
> Jan Karlsson
> Senior Software Engineer
> MIB

I don't have access to my sample ARM vmcores (on vacation), but none
of them have any modules loaded. So in those 3 cases, the vmalloc range
starts at either d0807000 or c6024000, and so the hardwired modules_end
is confusing. But it appears from your description that modules are put
in a their own virtual address region from bf00000 to bffffff, whereas
other vmalloc() calls generate virtual addresses above c000000?
(as shown by kmem -v). In that case, you're right, the code would
work as is. Anyway, it did confuse me a bit -- perhaps arm_cmd_mach()
should show different "KERNEL VMALLOC BASE" and "KERNEL MODULES BASE"
addresses, i.e., similar to x86_64?

Dave

Dave


--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Question about ARM module address range
 

This is what I see using kmem -v
crash> kmem -v
VM_STRUCT ADDRESS RANGE SIZE
c0042f80 bf000000 - bf022000 139264
de045b00 e1806000 - e1808000 8192
e09bbb40 e1808000 - e180a000 8192
e09f53c0 e180a000 - e180c000 8192
e09f5440 e180c000 - e180e000 8192
e09f58c0 e180e000 - e1810000 8192
e09f5940 e1810000 - e1812000 8192
e09f5f00 e1812000 - e1814000 8192
e09f5f80 e1814000 - e1816000 8192
......

This is the case with modules. In an example without modules the first range is not present, so your assumption how it works is correct. One thing that could be done to get the printout less confusing is to put 0xffffffff in modules_vaddr if no modules exist (and potentially put 0 in modules_end).

In arm_init in arm.c:
.....
case POST_VM:
machdep->machspec->vmalloc_start_addr = vt->high_memory;
/*
* Modules are placed in first vmalloc'd area. This is 16MB
* below PAGE_OFFSET.
*/
machdep->machspec->modules_vaddr = first_vmalloc_address();
machdep->machspec->modules_end = machdep->kvbase - 1;
if (machdep->machspec->modules_vaddr >= machdep->kvbase) {
/* no modules */
/* important that modules_end < modules_vaddr to get range test right */
machdep->machspec->modules_vaddr = 0xffffffff;
machdep->machspec->modules_end = 0;
}

To get some nicer code you can also make same changes in arm_is_module_addr.
Remove local variable modules_end and use the macro MODULES_END in the return statement.

Jan

Jan Karlsson
Senior Software Engineer
MIB
*
Sony Mobile Communications
Tel: +46703062174
sonymobile.com
*


-----Original Message-----
From: crash-utility-bounces@redhat.com [mailto:crash-utility-bounces@redhat.com] On Behalf Of anderson@prospeed.net
Sent: torsdag den 26 juli 2012 01:31
To: crash-utility@redhat.com
Subject: Re: [Crash-utility] Question about ARM module address range


> Hi Dave
>
> I have taken a short look at modules_vaddr and module_end and I have
> both seen relevant data:
>
> crash> help -m
> modules_vaddr: bf000000
> modules_end: bfffffff
>
> and data similar to what you see. What I also have seen is that when
> modules are loaded then modules_vaddr and modules_end seems correct
> and when no modules have been loaded then strange values are
> presented. I have looked at too few examples to be certain that this is "always" true.
>
> I assume (not checked in source code) that no vmalloc area is
> allocated for modules if no modules are loaded. Then the function
> first_vmalloc_address() will return data which is stored in
> modules_vaddr but has nothing to do with this.
>
> So I think that the question is what values should modules_vaddr and
> modules_end have if no modules are loaded. Does it matter, except that
> it might be confusing for a user? Looking at arm.c where modules_vaddr
> and modules_end are used, I think the code will behaves correctly (by
> luck?!), also in the case of no modules.
>
> Jan
>
> Jan Karlsson
> Senior Software Engineer
> MIB

I don't have access to my sample ARM vmcores (on vacation), but none of them have any modules loaded. So in those 3 cases, the vmalloc range starts at either d0807000 or c6024000, and so the hardwired modules_end is confusing. But it appears from your description that modules are put in a their own virtual address region from bf00000 to bffffff, whereas other vmalloc() calls generate virtual addresses above c000000?
(as shown by kmem -v). In that case, you're right, the code would work as is. Anyway, it did confuse me a bit -- perhaps arm_cmd_mach() should show different "KERNEL VMALLOC BASE" and "KERNEL MODULES BASE"
addresses, i.e., similar to x86_64?

Dave

Dave


--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Question about ARM module address range
 

On Thu, Jul 26, 2012 at 07:58:15AM +0200, Karlsson, Jan wrote:
>
> This is the case with modules. In an example without modules the first range
> is not present, so your assumption how it works is correct. One thing that
> could be done to get the printout less confusing is to put 0xffffffff in
> modules_vaddr if no modules exist (and potentially put 0 in modules_end).

Can we put there the default (0xbf000000) when there is no modules? This thing
is documented in kernel Documentation/arm/memory.txt - modules are always
placed in that area no matter what. And I think it works better with live
session as well.

Something like following patch? It still gets the first vmalloc address from
the vmcore but if it is larger than what we expect we use the default.

Jan, Thomas,

Can you check whether this still works with vmcores that have modules loaded?
My sample vmlinux/vmcores don't seem to have any modules loaded.

diff --git a/arm.c b/arm.c
index ea06e1a..7f63a10 100644
--- a/arm.c
+++ b/arm.c
@@ -305,14 +305,20 @@ arm_init(int when)
error(WARNING, "Couldn't retrieve crash_notes
");
break;

- case POST_VM:
+ case POST_VM: {
+ ulong vaddr;
+
machdep->machspec->vmalloc_start_addr = vt->high_memory;
/*
* Modules are placed in first vmalloc'd area. This is 16MB
* below PAGE_OFFSET.
*/
- machdep->machspec->modules_vaddr = first_vmalloc_address();
machdep->machspec->modules_end = machdep->kvbase - 1;
+ vaddr = first_vmalloc_address();
+ if (vaddr > machdep->machspec->modules_end)
+ machdep->machspec->modules_vaddr = DEFAULT_MODULES_VADDR;
+ else
+ machdep->machspec->modules_vaddr = vaddr;

if (init_unwind_tables()) {
if (CRASHDEBUG(1))
@@ -323,6 +329,7 @@ arm_init(int when)
}
break;
}
+ }
}

void
@@ -614,7 +621,7 @@ static int
arm_is_module_addr(ulong vaddr)
{
ulong modules_start;
- ulong modules_end = machdep->kvbase - 1;
+ ulong modules_end = machdep->machspec->modules_end;

if (!MODULES_VADDR) {
/*
@@ -622,7 +629,7 @@ arm_is_module_addr(ulong vaddr)
* called, we use defaults here which is 16MB below kernel start
* address.
*/
- modules_start = machdep->kvbase - 16 * 1024 * 1024;
+ modules_start = DEFAULT_MODULES_VADDR;
} else {
modules_start = MODULES_VADDR;
}
diff --git a/defs.h b/defs.h
index caa87c0..e775e24 100755
--- a/defs.h
+++ b/defs.h
@@ -2438,6 +2438,7 @@ struct load_module {

#define IS_VMALLOC_ADDR(X) arm_is_vmalloc_addr((ulong)(X))

+#define DEFAULT_MODULES_VADDR (machdep->kvbase - 16 * 1024 * 1024)
#define MODULES_VADDR (machdep->machspec->modules_vaddr)
#define MODULES_END (machdep->machspec->modules_end)
#define VMALLOC_START (machdep->machspec->vmalloc_start_addr)

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Question about ARM module address range
 

----- Original Message -----
> On Thu, Jul 26, 2012 at 07:58:15AM +0200, Karlsson, Jan wrote:
> >
> > This is the case with modules. In an example without modules the first range
> > is not present, so your assumption how it works is correct. One thing that
> > could be done to get the printout less confusing is to put 0xffffffff in
> > modules_vaddr if no modules exist (and potentially put 0 in modules_end).
>
> Can we put there the default (0xbf000000) when there is no modules? This thing
> is documented in kernel Documentation/arm/memory.txt - modules are always
> placed in that area no matter what. And I think it works better with live
> session as well.
>
> Something like following patch? It still gets the first vmalloc address from
> the vmcore but if it is larger than what we expect we use the default.
>
> Jan, Thomas,
>
> Can you check whether this still works with vmcores that have modules loaded?
> My sample vmlinux/vmcores don't seem to have any modules loaded.

Same here -- my sample vmcores don't have any modules installed -- but I cannot
see how this patch can possibly hurt, and it certainly makes sense. I'm going to
queue it for crash-6.0.9 unless it gets NAK'd before I release it.

Thanks,
Dave

>
> diff --git a/arm.c b/arm.c
> index ea06e1a..7f63a10 100644
> --- a/arm.c
> +++ b/arm.c
> @@ -305,14 +305,20 @@ arm_init(int when)
> error(WARNING, "Couldn't retrieve crash_notes
");
> break;
>
> - case POST_VM:
> + case POST_VM: {
> + ulong vaddr;
> +
> machdep->machspec->vmalloc_start_addr = vt->high_memory;
> /*
> * Modules are placed in first vmalloc'd area. This is 16MB
> * below PAGE_OFFSET.
> */
> - machdep->machspec->modules_vaddr = first_vmalloc_address();
> machdep->machspec->modules_end = machdep->kvbase - 1;
> + vaddr = first_vmalloc_address();
> + if (vaddr > machdep->machspec->modules_end)
> + machdep->machspec->modules_vaddr = DEFAULT_MODULES_VADDR;
> + else
> + machdep->machspec->modules_vaddr = vaddr;
>
> if (init_unwind_tables()) {
> if (CRASHDEBUG(1))
> @@ -323,6 +329,7 @@ arm_init(int when)
> }
> break;
> }
> + }
> }
>
> void
> @@ -614,7 +621,7 @@ static int
> arm_is_module_addr(ulong vaddr)
> {
> ulong modules_start;
> - ulong modules_end = machdep->kvbase - 1;
> + ulong modules_end = machdep->machspec->modules_end;
>
> if (!MODULES_VADDR) {
> /*
> @@ -622,7 +629,7 @@ arm_is_module_addr(ulong vaddr)
> * called, we use defaults here which is 16MB below kernel start
> * address.
> */
> - modules_start = machdep->kvbase - 16 * 1024 * 1024;
> + modules_start = DEFAULT_MODULES_VADDR;
> } else {
> modules_start = MODULES_VADDR;
> }
> diff --git a/defs.h b/defs.h
> index caa87c0..e775e24 100755
> --- a/defs.h
> +++ b/defs.h
> @@ -2438,6 +2438,7 @@ struct load_module {
>
> #define IS_VMALLOC_ADDR(X) arm_is_vmalloc_addr((ulong)(X))
>
> +#define DEFAULT_MODULES_VADDR (machdep->kvbase - 16 * 1024 * 1024)
> #define MODULES_VADDR (machdep->machspec->modules_vaddr)
> #define MODULES_END (machdep->machspec->modules_end)
> #define VMALLOC_START (machdep->machspec->vmalloc_start_addr)
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Question about ARM module address range
 

----- Original Message -----
> >
> > Jan, Thomas,
> >
> > Can you check whether this still works with vmcores that have modules loaded?
> > My sample vmlinux/vmcores don't seem to have any modules loaded.
>
> Same here -- my sample vmcores don't have any modules installed -- but I cannot
> see how this patch can possibly hurt, and it certainly makes sense. I'm going to
> queue it for crash-6.0.9 unless it gets NAK'd before I release it.
>
> Thanks,
> Dave

Also, as we discussed before, I'm also going to add
a "KERNEL MODULES BASE: <address>" line to the "mach"
command.

Thanks,
Dave

>
> >
> > diff --git a/arm.c b/arm.c
> > index ea06e1a..7f63a10 100644
> > --- a/arm.c
> > +++ b/arm.c
> > @@ -305,14 +305,20 @@ arm_init(int when)
> > error(WARNING, "Couldn't retrieve crash_notes
");
> > break;
> >
> > - case POST_VM:
> > + case POST_VM: {
> > + ulong vaddr;
> > +
> > machdep->machspec->vmalloc_start_addr = vt->high_memory;
> > /*
> > * Modules are placed in first vmalloc'd area. This is 16MB
> > * below PAGE_OFFSET.
> > */
> > - machdep->machspec->modules_vaddr = first_vmalloc_address();
> > machdep->machspec->modules_end = machdep->kvbase - 1;
> > + vaddr = first_vmalloc_address();
> > + if (vaddr > machdep->machspec->modules_end)
> > + machdep->machspec->modules_vaddr = DEFAULT_MODULES_VADDR;
> > + else
> > + machdep->machspec->modules_vaddr = vaddr;
> >
> > if (init_unwind_tables()) {
> > if (CRASHDEBUG(1))
> > @@ -323,6 +329,7 @@ arm_init(int when)
> > }
> > break;
> > }
> > + }
> > }
> >
> > void
> > @@ -614,7 +621,7 @@ static int
> > arm_is_module_addr(ulong vaddr)
> > {
> > ulong modules_start;
> > - ulong modules_end = machdep->kvbase - 1;
> > + ulong modules_end = machdep->machspec->modules_end;
> >
> > if (!MODULES_VADDR) {
> > /*
> > @@ -622,7 +629,7 @@ arm_is_module_addr(ulong vaddr)
> > * called, we use defaults here which is 16MB below kernel start
> > * address.
> > */
> > - modules_start = machdep->kvbase - 16 * 1024 * 1024;
> > + modules_start = DEFAULT_MODULES_VADDR;
> > } else {
> > modules_start = MODULES_VADDR;
> > }
> > diff --git a/defs.h b/defs.h
> > index caa87c0..e775e24 100755
> > --- a/defs.h
> > +++ b/defs.h
> > @@ -2438,6 +2438,7 @@ struct load_module {
> >
> > #define IS_VMALLOC_ADDR(X) arm_is_vmalloc_addr((ulong)(X))
> >
> > +#define DEFAULT_MODULES_VADDR (machdep->kvbase - 16 * 1024 * 1024)
> > #define MODULES_VADDR (machdep->machspec->modules_vaddr)
> > #define MODULES_END (machdep->machspec->modules_end)
> > #define VMALLOC_START (machdep->machspec->vmalloc_start_addr)
> >
> > --
> > Crash-utility mailing list
> > Crash-utility@redhat.com
> > https://www.redhat.com/mailman/listinfo/crash-utility
> >
>
> --
> Crash-utility mailing list
> Crash-utility@redhat.com
> https://www.redhat.com/mailman/listinfo/crash-utility
>

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility

Question about ARM module address range
 

On Tue, Aug 14, 2012 at 03:21:11PM -0400, Dave Anderson wrote:
>
> Also, as we discussed before, I'm also going to add
> a "KERNEL MODULES BASE: <address>" line to the "mach"
> command.
>

Sounds good, thanks.

--
Crash-utility mailing list
Crash-utility@redhat.com
https://www.redhat.com/mailman/listinfo/crash-utility