Bug#602273: linux-image-2.6.32-5-686-bigmem: domU hangs during dom0 reboot, recovers when dom0 uptime "caught up"

11-03-2010, 06:42 AM

Package: linux-image-2.6.32-5-686-bigmem
Version: 2.6.32-26
Severity: normal

Steps to reproduce:
1) use 64-bit squeeze as dom0
2) use 32-bit squeeze as domU
3) reboot dom0

Expected results:
3) domU's state is saved to disk and then restored

Actual results:
2) domU's state is saved to disk but it hangs after restore

More info:
1) Saving and restoring the domU with "xm save" and "xm restore"
worked at least twice. The problem occurs only if dom0 is rebooted.

=> Is some clock perhaps visible to the domU and which then gets
confused when it is reset and thus goes backwards in time?

2) hypervisor and dom0 information:

overlord2:~$ dpkg-query -W xen-hypervisor-4.0-amd64 linux-image-$(uname -r)
linux-image-2.6.32-5-xen-amd64 2.6.32-27
xen-hypervisor-4.0-amd64 4.0.1-1
overlord2:~$ sudo xm info
host : overlord2
release : 2.6.32-5-xen-amd64
version : #1 SMP Wed Oct 20 02:22:18 UTC 2010
machine : x86_64
nr_cpus : 4
nr_nodes : 1
cores_per_socket : 4
threads_per_core : 1
cpu_mhz : 2394
hw_caps : bfebfbff:20100800:00000000:00000940:0000e3bd:00000 000:00000001:00000000
virt_caps : hvm hvm_directio
total_memory : 8189
free_memory : 3
node_to_cpu : node0:0-3
node_to_memory : node0:3
node_to_dma32_mem : node0:3
max_node_id : 0
xen_major : 4
xen_minor : 0
xen_extra : .1
xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p hvm-3.0-x86_64
xen_scheduler : credit
xen_pagesize : 4096
platform_params : virt_start=0xffff800000000000
xen_changeset : unavailable
xen_commandline :
cc_compiler : gcc version 4.4.5 20100824 (prerelease) (Debian 4.4.4-11)
cc_compile_by : waldi
cc_compile_domain : debian.org
cc_compile_date : Fri Sep 3 15:38:12 UTC 2010
xend_config_format : 4
overlord2:~$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
stepping : 11
cpu MHz : 2394.082
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good aperfmperf pni est ssse3 cx16 hypervisor lahf_lm
bogomips : 4788.16
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 1
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
stepping : 11
cpu MHz : 2394.082
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good aperfmperf pni est ssse3 cx16 hypervisor lahf_lm
bogomips : 4788.16
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 2
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
stepping : 11
cpu MHz : 2394.082
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good aperfmperf pni est ssse3 cx16 hypervisor lahf_lm
bogomips : 4788.16
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

processor : 3
vendor_id : GenuineIntel
cpu family : 6
model : 15
model name : Intel(R) Core(TM)2 Quad CPU Q6600 @ 2.40GHz
stepping : 11
cpu MHz : 2394.082
cache size : 4096 KB
fpu : yes
fpu_exception : yes
cpuid level : 10
wp : yes
flags : fpu de tsc msr pae mce cx8 apic sep mtrr mca cmov pat clflush acpi mmx fxsr sse sse2 ss ht syscall nx lm constant_tsc rep_good aperfmperf pni est ssse3 cx16 hypervisor lahf_lm
bogomips : 4788.16
clflush size : 64
cache_alignment : 64
address sizes : 36 bits physical, 48 bits virtual
power management:

3) domU information:

$ dpkg-query -W linux-image-$(uname -r)
linux-image-2.6.32-5-686-bigmem 2.6.32-26

4) We used

sudo xm dump-core squeeze32 squeeze32.dump

to save the memory of the stuck domU to disk.

5) We used

crash -s --no_panic -e emacs vmlinux-2.6.32-5-686-bigmem squeeze32.dump

to analyze the memory dump:

crash> sys
KERNEL: vmlinux-2.6.32-5-686-bigmem
DUMPFILE: D32.2
CPUS: 4
DATE: Tue Nov 2 17:44:23 2010
UPTIME: 00:11:05
LOAD AVERAGE: 0.00, 0.00, 0.00
TASKS: 79
NODENAME: squeeze32
RELEASE: 2.6.32-5-686-bigmem
VERSION: #1 SMP Tue Oct 19 15:03:03 UTC 2010
MACHINE: i686 (2394 Mhz)
MEMORY: 511.6 MB
PANIC: ""
crash> ps | grep -v IN
PID PPID CPU TASK ST %MEM VSZ RSS COMM
> 0 0 0 c13a2ba0 RU 0.0 0 0 [swapper]
> 0 0 1 df830880 RU 0.0 0 0 [swapper]
> 0 0 2 df830cc0 RU 0.0 0 0 [swapper]
> 0 0 3 df831100 RU 0.0 0 0 [swapper]
17 2 2 df8350c0 UN 0.0 0 0 [events/2]
18 2 3 df835500 UN 0.0 0 0 [events/3]
24 2 3 df836e80 UN 0.0 0 0 [xenwatch]
crash> bt 17
PID: 17 TASK: df8350c0 CPU: 2 COMMAND: "events/2"
#0 [df87be4c] schedule at c127d79f
#1 [df87bebc] schedule_timeout at c127daf4
#2 [df87befc] wait_for_common at c127d9fd
#3 [df87bf30] synchronize_sched at c10486b1
#4 [df87bf50] dev_deactivate at c11f8946
#5 [df87bf64] __linkwatch_run_queue at c11f270e
#6 [df87bf7c] linkwatch_event at c11f2760
#7 [df87bf80] worker_thread at c10473c3
#8 [df87bfc0] kthread at c1049eca
#9 [df87bfe4] kernel_thread_helper at c1008d25
crash> bt 18
PID: 18 TASK: df835500 CPU: 3 COMMAND: "events/3"
#0 [df87dec4] schedule at c127d79f
#1 [df87df34] __mutex_lock_common at c127df3a
#2 [df87df68] __mutex_lock_slowpath at c127df9c
#3 [df87df70] mutex_lock at c127e02d
#4 [df87df7c] linkwatch_event at c11f274d
#5 [df87df80] worker_thread at c10473c3
#6 [df87dfc0] kthread at c1049eca
#7 [df87dfe4] kernel_thread_helper at c1008d25
crash> bt 24
PID: 24 TASK: df836e80 CPU: 3 COMMAND: "xenwatch"
#0 [df8b3e88] schedule at c127d79f
#1 [df8b3ef8] __mutex_lock_common at c127df3a
#2 [df8b3f2c] __mutex_lock_slowpath at c127df9c
#3 [df8b3f34] mutex_lock at c127e02d
#4 [df8b3f40] netif_notify_peers at c11f8c71
#5 [df8b3f48] backend_changed at e0822f15
#6 [df8b3f9c] xenwatch_thread at c1196dd6
#7 [df8b3fc0] kthread at c1049eca
#8 [df8b3fe4] kernel_thread_helper at c1008d25
crash>

=> pids 24 and 18 are waiting for a mutex, probably rtnlmutex

=> pid 17 has acquired the rtnlmutex.

=> What is supposed wake pid 17 up?

6) We took another memory dump just to obvserve that the value of jiffies is not changing:

crash> print jiffies
$8 = 91395
crash> quit
$ crash -s --no_panic -e emacs vmlinux-2.6.32-5-686-bigmem squeeze32.2.dump
crash> print jiffies
$8 = 91395

However, "xm list -l" shows that the cpu usage time is slowly increasing.

7) after about 3 hours of waiting the domU magically recovered:

Nov 2 14:17:32 squeeze32 kernel: imklog 4.6.4, log source = /proc/kmsg started.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Reserving virtual address space above 0xf5800000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Initializing cgroup subsys cpuset
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Initializing cgroup subsys cpu
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Linux version 2.6.32-5-686-bigmem (Debian 2.6.32-26) (dannf@debian.org) (gcc version 4.3.5 (Debian 4.3.5-4) ) #1 SMP Tue Oct 19 15:03:03 UTC 2010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] KERNEL supported cpus:
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Intel GenuineIntel
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] AMD AuthenticAMD
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] NSC Geode by NSC
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Cyrix CyrixInstead
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Centaur CentaurHauls
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Transmeta GenuineTMx86
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Transmeta TransmetaCPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] UMC UMC UMC UMC
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] ACPI in unprivileged domain disabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] BIOS-provided physical RAM map:
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: 0000000000000000 - 00000000000a0000 (usable)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: 00000000000a0000 - 0000000000100000 (reserved)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: 0000000000100000 - 0000000020000000 (usable)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMI not present or invalid.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] last_pfn = 0x20000 max_arch_pfn = 0x1000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] initial memory mapped : 0 - 033ff000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] init_memory_mapping: 0000000000000000-0000000020000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] NX (Execute Disable) protection: active
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 0000000000 - 0020000000 page 4k
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] kernel direct mapping tables up to 20000000 @ 100000-205000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] RAMDISK: 01620000 - 029f5000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 0MB HIGHMEM available.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 512MB LOWMEM available.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] mapped low ram: 0 - 20000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] low ram: 0 - 20000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] node 0 low ram: 00000000 - 20000000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] node 0 bootmap 00002000 - 00006000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] (9 early reservations) ==> bootmem [0000000000 - 0020000000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #1 [0002a78000 - 0002a90000] XEN PAGETABLES ==> [0002a78000 - 0002a90000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #2 [0000001000 - 0000002000] EX TRAMPOLINE ==> [0000001000 - 0000002000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #3 [0000006000 - 0000007000] TRAMPOLINE ==> [0000006000 - 0000007000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #4 [0001000000 - 00014fa3f4] TEXT DATA BSS ==> [0001000000 - 00014fa3f4]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #5 [0001620000 - 00029f5000] RAMDISK ==> [0001620000 - 00029f5000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #6 [00029f5000 - 0002a78000] XEN START INFO ==> [00029f5000 - 0002a78000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #7 [0000100000 - 00001e9000] PGTABLE ==> [0000100000 - 00001e9000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] #8 [0000002000 - 0000006000] BOOTMAP ==> [0000002000 - 0000006000]
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Zone PFN ranges:
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMA 0x00000000 -> 0x00001000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Normal 0x00001000 -> 0x00020000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] HighMem 0x00020000 -> 0x00020000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Movable zone start PFN for each node
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] early_node_map[2] active PFN ranges
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 0: 0x00000000 -> 0x000000a0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] 0: 0x00000100 -> 0x00020000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] On node 0 totalpages: 130976
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] free_area_init_node: node 0, pgdat c13cff40, node_mem_map c2a90000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMA zone: 32 pages used for memmap
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMA zone: 0 pages reserved
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] DMA zone: 3968 pages, LIFO batch:0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Normal zone: 992 pages used for memmap
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Normal zone: 125984 pages, LIFO batch:31
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Using APIC driver default
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] SFI: Simple Firmware Interface v0.7 http://simplefirmware.org
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Local APIC disabled by BIOS -- you can enable it with "lapic"
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] APIC: disable apic facility
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] nr_irqs_gsi: 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] PM: Registered nosave memory: 00000000000a0000 - 0000000000100000
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Allocating PCI resources starting at 20000000 (gap: 20000000:e0000000)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Booting paravirtualized kernel on Xen
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen version: 4.0.1 (preserve-AD)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:4 nr_node_ids:1
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] PERCPU: Embedded 15 pages/cpu @c2e94000 s37656 r0 d23784 u65536
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] pcpu-alloc: s37656 r0 d23784 u65536 alloc=16*4096
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] trying to map vcpu_info 0 at c2e95010, mfn 20d098, offset 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] cpu 0 using vcpu_info at c2e95010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] trying to map vcpu_info 1 at c2ea5010, mfn 20d088, offset 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] cpu 1 using vcpu_info at c2ea5010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] trying to map vcpu_info 2 at c2eb5010, mfn 20d078, offset 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] cpu 2 using vcpu_info at c2eb5010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] trying to map vcpu_info 3 at c2ec5010, mfn 20d068, offset 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] cpu 3 using vcpu_info at c2ec5010
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: using vcpu_info placement
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 129952
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Kernel command line: root=/dev/xvda ro console=hvc0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] PID hash table entries: 2048 (order: 1, 8192 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Enabling fast FPU save and restore... done.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Enabling unmasked SIMD FPU exception support... done.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Initializing CPU#0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Initializing HighMem for node 0 (00000000:00000000)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Memory: 491968k/524288k available (2569k kernel code, 31748k reserved, 1374k data, 388k init, 0k highmem)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] virtual kernel memory layout:
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] fixmap : 0xf5556000 - 0xf57ff000 (2724 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] pkmap : 0xf5000000 - 0xf5200000 (2048 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] vmalloc : 0xe0800000 - 0xf4ffe000 ( 327 MB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] lowmem : 0xc0000000 - 0xe0000000 ( 512 MB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] .init : 0xc13da000 - 0xc143b000 ( 388 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] .data : 0xc128244d - 0xc13d9cf0 (1374 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] .text : 0xc1000000 - 0xc128244d (2569 kB)
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] SLUB: Genslabs=13, HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Hierarchical RCU implementation.
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] NR_IRQS:1280
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Console: colour dummy device 80x25
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] console [tty0] enabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] console [hvc0] enabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Xen: using vcpuop timer interface
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] installing Xen timer for CPU 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.000000] Detected 2394.036 MHz processor.
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Calibrating delay loop (skipped), value calculated using timer frequency.. 4788.07 BogoMIPS (lpj=9576144)
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Security Framework initialized
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] SELinux: Disabled at boot.
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Mount-cache hash table entries: 512
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys ns
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys cpuacct
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys devices
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys freezer
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing cgroup subsys net_cls
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L2 cache: 4096K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Physical Processor ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Processor Core ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Performance Events: unsupported p6 CPU model 15 no PMU driver, software events only.
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] SMP alternatives: switching to UP code
Nov 2 14:17:32 squeeze32 kernel: [ 0.004399] installing Xen timer for CPU 1
Nov 2 14:17:32 squeeze32 kernel: [ 0.004431] SMP alternatives: switching to SMP code
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing CPU#1
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L2 cache: 4096K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Physical Processor ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Processor Core ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.008952] installing Xen timer for CPU 2
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing CPU#2
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L2 cache: 4096K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Physical Processor ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Processor Core ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.009241] installing Xen timer for CPU 3
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] Initializing CPU#3
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L1 I cache: 32K, L1 D cache: 32K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: L2 cache: 4096K
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Physical Processor ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.004000] CPU: Processor Core ID: 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.009393] Brought up 4 CPUs
Nov 2 14:17:32 squeeze32 kernel: [ 0.009447] CPU0 attaching sched-domain:
Nov 2 14:17:32 squeeze32 kernel: [ 0.009454] domain 0: span 0-3 level CPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.009463] groups: 0 1 2 3
Nov 2 14:17:32 squeeze32 kernel: [ 0.009487] CPU1 attaching sched-domain:
Nov 2 14:17:32 squeeze32 kernel: [ 0.009493] domain 0: span 0-3 level CPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.009501] groups: 1 2 3 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.009524] CPU2 attaching sched-domain:
Nov 2 14:17:32 squeeze32 kernel: [ 0.009530] domain 0: span 0-3 level CPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.009538] groups: 2 3 0 1
Nov 2 14:17:32 squeeze32 kernel: [ 0.009561] CPU3 attaching sched-domain:
Nov 2 14:17:32 squeeze32 kernel: [ 0.009566] domain 0: span 0-3 level CPU
Nov 2 14:17:32 squeeze32 kernel: [ 0.009575] groups: 3 0 1 2
Nov 2 14:17:32 squeeze32 kernel: [ 0.009738] devtmpfs: initialized
Nov 2 14:17:32 squeeze32 kernel: [ 0.009738] Grant table initialized
Nov 2 14:17:32 squeeze32 kernel: [ 0.009738] regulator: core version 0.5
Nov 2 14:17:32 squeeze32 kernel: [ 0.009738] NET: Registered protocol family 16
Nov 2 14:17:32 squeeze32 kernel: [ 0.012050] PCI: Fatal: No config space access function found
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] bio: create slab <bio-0> at 0
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] ACPI: Interpreter disabled.
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] xen_balloon: Initialising balloon driver.
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] vgaarb: loaded
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] PCI: System does not support PCI
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] PCI: System does not support PCI
Nov 2 14:17:32 squeeze32 kernel: [ 0.012875] Switching to clocksource xen
Nov 2 14:17:32 squeeze32 kernel: [ 0.019527] pnp: PnP ACPI: disabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.019540] PnPBIOS: Disabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.020189] NET: Registered protocol family 2
Nov 2 14:17:32 squeeze32 kernel: [ 0.020303] IP route cache hash table entries: 4096 (order: 2, 16384 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.119927] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.120005] TCP bind hash table entries: 16384 (order: 5, 131072 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.120072] TCP: Hash tables configured (established 16384 bind 16384)
Nov 2 14:17:32 squeeze32 kernel: [ 0.120086] TCP reno registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.120204] NET: Registered protocol family 1
Nov 2 14:17:32 squeeze32 kernel: [ 0.120318] Unpacking initramfs...
Nov 2 14:17:32 squeeze32 kernel: [ 0.153610] Freeing initrd memory: 20308k freed
Nov 2 14:17:32 squeeze32 kernel: [ 0.162590] platform rtc_cmos: registered platform RTC device (no PNP device found)
Nov 2 14:17:32 squeeze32 kernel: [ 0.163047] audit: initializing netlink socket (disabled)
Nov 2 14:17:32 squeeze32 kernel: [ 0.163071] type=2000 audit(1288700248.679:1): initialized
Nov 2 14:17:32 squeeze32 kernel: [ 0.166293] HugeTLB registered 2 MB page size, pre-allocated 0 pages
Nov 2 14:17:32 squeeze32 kernel: [ 0.168396] VFS: Disk quotas dquot_6.5.2
Nov 2 14:17:32 squeeze32 kernel: [ 0.168462] Dquot-cache hash table entries: 1024 (order 0, 4096 bytes)
Nov 2 14:17:32 squeeze32 kernel: [ 0.168598] msgmni has been set to 1000
Nov 2 14:17:32 squeeze32 kernel: [ 0.169229] alg: No test for stdrng (krng)
Nov 2 14:17:32 squeeze32 kernel: [ 0.169311] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
Nov 2 14:17:32 squeeze32 kernel: [ 0.169323] io scheduler noop registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.169330] io scheduler anticipatory registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.169338] io scheduler deadline registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.169389] io scheduler cfq registered (default)
Nov 2 14:17:32 squeeze32 kernel: [ 0.169586] isapnp: Write Data Register 0xa79 already used
Nov 2 14:17:32 squeeze32 kernel: [ 0.173466] Linux agpgart interface v0.103
Nov 2 14:17:32 squeeze32 kernel: [ 0.173710] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.174084] PNP: No PS/2 controller found. Probing ports directly.
Nov 2 14:17:32 squeeze32 kernel: [ 0.174919] i8042.c: No controller found.
Nov 2 14:17:32 squeeze32 kernel: [ 0.174994] mice: PS/2 mouse device common for all mice
Nov 2 14:17:32 squeeze32 kernel: [ 0.175058] rtc_cmos: probe of rtc_cmos failed with error -16
Nov 2 14:17:32 squeeze32 kernel: [ 0.175096] cpuidle: using governor ladder
Nov 2 14:17:32 squeeze32 kernel: [ 0.175103] cpuidle: using governor menu
Nov 2 14:17:32 squeeze32 kernel: [ 0.175113] No iBFT detected.
Nov 2 14:17:32 squeeze32 kernel: [ 0.175391] TCP cubic registered
Nov 2 14:17:32 squeeze32 kernel: [ 0.175574] NET: Registered protocol family 10
Nov 2 14:17:32 squeeze32 kernel: [ 0.175973] lo: Disabled Privacy Extensions
Nov 2 14:17:32 squeeze32 kernel: [ 0.176215] Mobile IPv6
Nov 2 14:17:32 squeeze32 kernel: [ 0.176223] NET: Registered protocol family 17
Nov 2 14:17:32 squeeze32 kernel: [ 0.176242] Using IPI No-Shortcut mode
Nov 2 14:17:32 squeeze32 kernel: [ 0.176326] PM: Resume from disk failed.
Nov 2 14:17:32 squeeze32 kernel: [ 0.176342] registered taskstats version 1
Nov 2 14:17:32 squeeze32 kernel: [ 0.176376] XENBUS: Device with no driver: device/vbd/51712
Nov 2 14:17:32 squeeze32 kernel: [ 0.176383] XENBUS: Device with no driver: device/vif/0
Nov 2 14:17:32 squeeze32 kernel: [ 0.176391] XENBUS: Device with no driver: device/console/0
Nov 2 14:17:32 squeeze32 kernel: [ 0.176412] /tmp/buildd/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/rtc/hctosys.c: unable to open rtc device (rtc0)
Nov 2 14:17:32 squeeze32 kernel: [ 0.176445] Initalizing network drop monitor service
Nov 2 14:17:32 squeeze32 kernel: [ 0.176491] Freeing unused kernel memory: 388k freed
Nov 2 14:17:32 squeeze32 kernel: [ 0.177527] Write protecting the kernel text: 2572k
Nov 2 14:17:32 squeeze32 kernel: [ 0.177839] Write protecting the kernel read-only data: 924k
Nov 2 14:17:32 squeeze32 kernel: [ 0.218177] udev[77]: starting version 163
Nov 2 14:17:32 squeeze32 kernel: [ 0.240302] Initialising Xen virtual ethernet driver.
Nov 2 14:17:32 squeeze32 kernel: [ 0.248886] blkfront: xvda: barriers enabled
Nov 2 14:17:32 squeeze32 kernel: [ 0.249192] xvda: unknown partition table
Nov 2 14:17:32 squeeze32 kernel: [ 0.431926] kjournald starting. Commit interval 5 seconds
Nov 2 14:17:32 squeeze32 kernel: [ 0.431944] EXT3-fs: mounted filesystem with ordered data mode.
Nov 2 14:17:32 squeeze32 kernel: [ 1.355631] udev[158]: starting version 163
Nov 2 14:17:32 squeeze32 kernel: [ 1.707941] input: PC Speaker as /devices/platform/pcspkr/input/input0
Nov 2 14:17:32 squeeze32 kernel: [ 1.849344] Error: Driver 'pcspkr' is already registered, aborting...
Nov 2 14:17:32 squeeze32 kernel: [ 2.153273] EXT3 FS on xvda, internal journal
Nov 2 14:17:32 squeeze32 kernel: [ 2.250745] loop: module loaded
Nov 2 14:17:42 squeeze32 kernel: [ 13.692056] eth0: no IPv6 routers present
Nov 2 14:18:58 squeeze32 kernel: [ 89.571723] suspending xenstore...
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] trying to map vcpu_info 0 at c2e95010, mfn 13188a, offset 16
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] cpu 0 using vcpu_info at c2e95010
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] trying to map vcpu_info 1 at c2ea5010, mfn 13187a, offset 16
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] cpu 1 using vcpu_info at c2ea5010
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] trying to map vcpu_info 2 at c2eb5010, mfn 13186a, offset 16
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] cpu 2 using vcpu_info at c2eb5010
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] trying to map vcpu_info 3 at c2ec5010, mfn 13185a, offset 16
Nov 2 14:19:39 squeeze32 kernel: [ 89.576142] cpu 3 using vcpu_info at c2ec5010
Nov 2 14:21:21 squeeze32 kernel: [ 191.777200] suspending xenstore...
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] trying to map vcpu_info 0 at c2e95010, mfn 13188a, offset 16
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] cpu 0 using vcpu_info at c2e95010
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] trying to map vcpu_info 1 at c2ea5010, mfn 13187a, offset 16
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] cpu 1 using vcpu_info at c2ea5010
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] trying to map vcpu_info 2 at c2eb5010, mfn 13186a, offset 16
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] cpu 2 using vcpu_info at c2eb5010
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] trying to map vcpu_info 3 at c2ec5010, mfn 13185a, offset 16
Nov 2 14:26:33 squeeze32 kernel: [ 191.784007] cpu 3 using vcpu_info at c2ec5010
Nov 2 14:34:27 squeeze32 kernel: [ 665.577176] suspending xenstore...
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] trying to map vcpu_info 0 at c2e95010, mfn 131889, offset 16
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] cpu 0 using vcpu_info at c2e95010
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] trying to map vcpu_info 1 at c2ea5010, mfn 131879, offset 16
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] cpu 1 using vcpu_info at c2ea5010
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] trying to map vcpu_info 2 at c2eb5010, mfn 131869, offset 16
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] cpu 2 using vcpu_info at c2eb5010
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] trying to map vcpu_info 3 at c2ec5010, mfn 131859, offset 16
Nov 2 17:44:23 squeeze32 kernel: [ 665.579316] cpu 3 using vcpu_info at c2ec5010
Nov 2 17:44:23 squeeze32 kernel: [ 665.597226] SysRq : HELP : loglevel(0-9) reBoot Crash terminate-all-tasks(E) memory-full-oom-kill(F) kill-all-tasks(I) thaw-filesystems(J) saK show-backtrace-all-active-cpus(L) show-memory-usage(M) nice-all-RT-tasks(N) powerOff show-registers(P) show-all-timers(Q) unRaw Sync show-task-states(T) Unmount show-blocked-tasks(W) dump-ftrace-buffer(Z)

=> This 3 hours and 10 minutes seems to correspond to the uptime that
dom0 had before its reboot:

overlord2:~$ last reboot
reboot system boot 2.6.32-5-xen-amd Tue Nov 2 14:36 - 09:34 (18:58)
reboot system boot 2.6.32-5-xen-amd Tue Nov 2 11:26 - 14:34 (03:07)
reboot system boot 2.6.32-5-xen-amd Tue Nov 2 09:37 - 11:24 (01:46)
reboot system boot 2.6.32-5-xen-amd Tue Nov 2 09:34 - 09:36 (00:02)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 16:44 - 09:32 (16:47)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 16:39 - 16:43 (00:03)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 16:25 - 16:29 (00:04)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 15:47 - 16:29 (00:42)
reboot system boot 2.6.32-5-amd64 Mon Nov 1 15:27 - 15:44 (00:17)

=> Theory: some counter is reset when dom0 is rebooted. domU becomes
very upset when the counter suddenly jumps backwards and recovers only
when the counter has finally reached its previous value?

--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 844oby7t0x.fsf@sauna.l.org">http://lists.debian.org/844oby7t0x.fsf@sauna.l.org

11-03-2010, 07:59 AM

package linux-image-2.6.32-5-686-bigmem
found 602273 2.6.32-27
thanks

This issue looks a lot like

"[Xen-devel] DomU clock jumps forward then freezes after Dom0 reboot"
http://lists.xensource.com/archives/html/xen-devel/2010-10/msg00498.html

and

"[Xen-devel] Migrated domUs hangs"
http://lists.xensource.com/archives/html/xen-devel/2010-10/msg00057.html

We are going to test the patch from

http://lists.xensource.com/archives/html/xen-devel/2010-10/msg01261.html

--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 84wrou6avn.fsf_-_@sauna.l.org">http://lists.debian.org/84wrou6avn.fsf_-_@sauna.l.org

11-03-2010, 10:23 AM

package linux-image-2.6.32-5-686-bigmem
tags 602273 patch
thanks

The patch from
http://lists.xensource.com/archives/html/xen-devel/2010-10/msg01261.html
seems to help! Please consider applying this to debian.

For completeness, here's the patch from the email archive (probably
messes up whitespace though):

From: Jeremy Fitzhardinge <jeremy.fitzhardinge@xxxxxxxxxx>
Date: Mon, 25 Oct 2010 16:53:46 -0700
Subject: [PATCH] x86/pvclock: zero last_value on resume

If the guest domain has been suspend/resumed or migrated, then the
system clock backing the pvclock clocksource may revert to a smaller
value (ie, can be non-monotonic across the migration/save-restore).
Make sure we zero last_value in that case so that the domain
continues to see clock updates.

Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@xxxxxxxxxx>

diff --git a/arch/x86/include/asm/pvclock.h b/arch/x86/include/asm/pvclock.h
index cd02f32..6226870 100644
--- a/arch/x86/include/asm/pvclock.h
+++ b/arch/x86/include/asm/pvclock.h
@@ -11,5 +11,6 @@ unsigned long pvclock_tsc_khz(struct pvclock_vcpu_time_info
*src);
void pvclock_read_wallclock(struct pvclock_wall_clock *wall,
struct pvclock_vcpu_time_info *vcpu,
struct timespec *ts);
+void pvclock_resume(void);

#endif /* _ASM_X86_PVCLOCK_H */
diff --git a/arch/x86/kernel/pvclock.c b/arch/x86/kernel/pvclock.c
index 239427c..a4f07c1 100644
--- a/arch/x86/kernel/pvclock.c
+++ b/arch/x86/kernel/pvclock.c
@@ -120,6 +120,11 @@ unsigned long pvclock_tsc_khz(struct
pvclock_vcpu_time_info *src)

static atomic64_t last_value = ATOMIC64_INIT(0);

+void pvclock_resume(void)
+{
+ atomic64_set(&last_value, 0);
+}
+
cycle_t pvclock_clocksource_read(struct pvclock_vcpu_time_info *src)
{
struct pvclock_shadow_time shadow;
diff --git a/arch/x86/xen/time.c b/arch/x86/xen/time.c
index b2bb5aa..5da5e53 100644
--- a/arch/x86/xen/time.c
+++ b/arch/x86/xen/time.c
@@ -426,6 +426,8 @@ void xen_timer_resume(void)
{
int cpu;

+ pvclock_resume();
+
if (xen_clockevent != &xen_vcpuop_clockevent)
return;

--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 84mxpq6480.fsf@sauna.l.org">http://lists.debian.org/84mxpq6480.fsf@sauna.l.org

11-04-2010, 11:25 AM

On Wed, 2010-11-03 at 13:23 +0200, Timo Juhani Lindfors wrote:
> package linux-image-2.6.32-5-686-bigmem
> tags 602273 patch
> thanks
>
> The patch from
> http://lists.xensource.com/archives/html/xen-devel/2010-10/msg01261.html
> seems to help! Please consider applying this to debian.

Thanks, I've pinged upstream to see if this patch will be applied and
also requested that it gets tagged to go into stable.

Ian.

--
Ian Campbell

Drinking is not a spectator sport.
-- Jim Brosnan

--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1288873529.2664.0.camel@cthulhu.hellion.org.uk">ht tp://lists.debian.org/1288873529.2664.0.camel@cthulhu.hellion.org.uk

12-07-2010, 09:14 AM

Ian Campbell <ijc@hellion.org.uk> writes:
> On Wed, 2010-11-03 at 13:23 +0200, Timo Juhani Lindfors wrote:
>> http://lists.xensource.com/archives/html/xen-devel/2010-10/msg01261.html
>
> Thanks, I've pinged upstream to see if this patch will be applied and
> also requested that it gets tagged to go into stable.

Any news on this? Did you ping them on the xen-devel mailing list? (I
tried to look but there are so many messages that I didn't spot your ping).

--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 8439q9dh40.fsf@sauna.l.org">http://lists.debian.org/8439q9dh40.fsf@sauna.l.org

12-07-2010, 10:51 AM

On Tue, 2010-12-07 at 12:14 +0200, Timo Juhani Lindfors wrote:
> Ian Campbell <ijc@hellion.org.uk> writes:
> > On Wed, 2010-11-03 at 13:23 +0200, Timo Juhani Lindfors wrote:
> >> http://lists.xensource.com/archives/html/xen-devel/2010-10/msg01261.html
> >
> > Thanks, I've pinged upstream to see if this patch will be applied and
> > also requested that it gets tagged to go into stable.
>
> Any news on this? Did you ping them on the xen-devel mailing list? (I
> tried to look but there are so many messages that I didn't spot your ping).

My ping at http://marc.info/?l=linux-kernel&m=128879080525214&w=2 went
unanswered but Jeremy picked it up again at
http://marc.info/?l=linux-kernel&m=129045702011984&w=2 and it seems to
have gone into the tip.git tree (in x86/urgent) but not to Linus yet.

Ian.

--
Ian Campbell
Current Noise: Mistress - Psychic One Inch Punch

Kleeneness is next to Godelness.

--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1291722710.13966.2851.camel@zakaz.uk.xensource.com ">http://lists.debian.org/1291722710.13966.2851.camel@zakaz.uk.xensource.com

12-07-2010, 11:02 AM

Ian Campbell <ijc@hellion.org.uk> writes:
> My ping at http://marc.info/?l=linux-kernel&m=128879080525214&w=2 went
> unanswered but Jeremy picked it up again at
> http://marc.info/?l=linux-kernel&m=129045702011984&w=2 and it seems to
> have gone into the tip.git tree (in x86/urgent) but not to Linus yet.

Ok thanks. I hope you are still planning to add this to squeeze when
it reaches Linus.

--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 84y681bxky.fsf@sauna.l.org">http://lists.debian.org/84y681bxky.fsf@sauna.l.org

12-07-2010, 12:09 PM

On Tue, 2010-12-07 at 14:02 +0200, Timo Juhani Lindfors wrote:
> Ian Campbell <ijc@hellion.org.uk> writes:
> > My ping at http://marc.info/?l=linux-kernel&m=128879080525214&w=2 went
> > unanswered but Jeremy picked it up again at
> > http://marc.info/?l=linux-kernel&m=129045702011984&w=2 and it seems to
> > have gone into the tip.git tree (in x86/urgent) but not to Linus yet.
>
> Ok thanks. I hope you are still planning to add this to squeeze when
> it reaches Linus.

Absolutely, if not before, I think it's far enough along to be
considered upstream at this point.

--
Ian Campbell
Current Noise: Testament - Nobody's Fault

I call them as I see them. If I can't see them, I make them up.
-- Biff Barf

--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1291727377.13966.3013.camel@zakaz.uk.xensource.com ">http://lists.debian.org/1291727377.13966.3013.camel@zakaz.uk.xensource.com

03-05-2012, 09:45 AM

Hi,

I seem to have hit the same bug again with 2.6.32-38 domU (amd64). After
I rebooted dom0 to 2.6.32-41 I see that the jiffies value of the domU
does not increase but cpu_time in xm list -l output does. Value of
"last_value" seems to increase (now around 1804512135271).

I looked at linux-source-2.6.32 2.6.32-38 and I see that it contains a
call to pvclock_resume in xen_timer_resume so the patch should still be
there. Is this a new bug that just has the same symptoms?

KERNEL: /usr/lib/debug/boot/vmlinux-2.6.32-5-amd64
DUMPFILE: /local/xen/lindi1/core
CPUS: 6
DATE: Mon Mar 5 10:02:26 2012
UPTIME: 54 days, 21:35:49
LOAD AVERAGE: 0.04, 0.01, 0.00
TASKS: 307
NODENAME: lindi1
RELEASE: 2.6.32-5-amd64
VERSION: #1 SMP Mon Oct 3 03:59:20 UTC 2011
MACHINE: x86_64 (3210 Mhz)
MEMORY: 2 GB
PANIC: ""
PID: 0
COMMAND: "swapper"
TASK: ffffffff814611f0 (1 of 6) [THREAD_INFO: ffffffff8142c000]
CPU: 0
STATE: TASK_RUNNING (ACTIVE)
WARNING: panic task not found

crash> disassemble xen_timer_resume
Dump of assembler code for function xen_timer_resume:
0xffffffff8100de32 <xen_timer_resume+0>: push %rbx
0xffffffff8100de33 <xen_timer_resume+1>: callq 0xffffffff8102cd7e <pvclock_resume>
^^^^^^^^^^^^^^
...

crash> disassemble pvclock_resume
Dump of assembler code for function pvclock_resume:
0xffffffff8102cd7e <pvclock_resume+0>: movq $0x0,0x5a42cf(%rip) # 0xffffffff815d1058
0xffffffff8102cd89 <pvclock_resume+11>: retq

crash> x/x 0xffffffff815d1058
0xffffffff815d1058: 0x000001a4254e0867

crash> p/x last_value
$17 = {
counter = 0x1a4254e0867
}

crash> ps | grep -v IN
PID PPID CPU TASK ST %MEM VSZ RSS COMM
> 0 0 0 ffffffff814611f0 RU 0.0 0 0 [swapper]
> 0 0 1 ffff88007ff50e20 RU 0.0 0 0 [swapper]
> 0 0 2 ffff88007ff51530 RU 0.0 0 0 [swapper]
> 0 0 3 ffff88007ff51c40 RU 0.0 0 0 [swapper]
> 0 0 4 ffff88007ff52350 RU 0.0 0 0 [swapper]
> 0 0 5 ffff88007ff52a60 RU 0.0 0 0 [swapper]
21 2 0 ffff88007ffab170 UN 0.0 0 0 [events/0]
26 2 5 ffff88007ffad4c0 UN 0.0 0 0 [events/5]
32 2 5 ffff88007f418000 UN 0.0 0 0 [xenwatch]
crash> bt 0 21 26 32
PID: 0 TASK: ffffffff814611f0 CPU: 0 COMMAND: "swapper"
#0 [ffffffff8142df70] xen_safe_halt at ffffffff8100dcbf
#1 [ffffffff8142df78] xen_idle at ffffffff8100be63
#2 [ffffffff8142df90] cpu_idle at ffffffff8100fe97

PID: 0 TASK: ffff88007ff50e20 CPU: 1 COMMAND: "swapper"
#0 [ffff88007ff5de50] schedule at ffffffff812fb2a7
#1 [ffff88007ff5de68] xen_force_evtchn_callback at ffffffff8100dc41
#2 [ffff88007ff5de70] check_events at ffffffff8100e252
#3 [ffff88007ff5dec8] tick_nohz_stop_sched_tick at ffffffff81070d4e
#4 [ffff88007ff5df28] cpu_idle at ffffffff8100fe97

PID: 0 TASK: ffff88007ff51530 CPU: 2 COMMAND: "swapper"
#0 [ffff88007ff5fe50] schedule at ffffffff812fb2a7
#1 [ffff88007ff5fe68] xen_force_evtchn_callback at ffffffff8100dc41
#2 [ffff88007ff5fe70] check_events at ffffffff8100e252
#3 [ffff88007ff5fec8] tick_nohz_stop_sched_tick at ffffffff81070d4e
#4 [ffff88007ff5ff28] cpu_idle at ffffffff8100fe97

PID: 0 TASK: ffff88007ff51c40 CPU: 3 COMMAND: "swapper"
#0 [ffff88007ff69e50] schedule at ffffffff812fb2a7
#1 [ffff88007ff69e68] xen_force_evtchn_callback at ffffffff8100dc41
#2 [ffff88007ff69e70] check_events at ffffffff8100e252
#3 [ffff88007ff69ec8] tick_nohz_stop_sched_tick at ffffffff81070d4e
#4 [ffff88007ff69f28] cpu_idle at ffffffff8100fe97

PID: 0 TASK: ffff88007ff52350 CPU: 4 COMMAND: "swapper"
#0 [ffff88007ff6be50] schedule at ffffffff812fb2a7
#1 [ffff88007ff6be68] xen_force_evtchn_callback at ffffffff8100dc41
#2 [ffff88007ff6be70] check_events at ffffffff8100e252
#3 [ffff88007ff6bec8] tick_nohz_stop_sched_tick at ffffffff81070d4e
#4 [ffff88007ff6bf28] cpu_idle at ffffffff8100fe97

PID: 0 TASK: ffff88007ff52a60 CPU: 5 COMMAND: "swapper"
#0 [ffff88007ff6de50] schedule at ffffffff812fb2a7
#1 [ffff88007ff6de68] xen_force_evtchn_callback at ffffffff8100dc41
#2 [ffff88007ff6de70] check_events at ffffffff8100e252
#3 [ffff88007ff6dec8] tick_nohz_stop_sched_tick at ffffffff81070d4e
#4 [ffff88007ff6df28] cpu_idle at ffffffff8100fe97

PID: 21 TASK: ffff88007ffab170 CPU: 0 COMMAND: "events/0"
#0 [ffff88007ffc3b90] schedule at ffffffff812fb2a7
#1 [ffff88007ffc3c68] schedule_timeout at ffffffff812fb6dd
#2 [ffff88007ffc3ce8] wait_for_common at ffffffff812fb594
#3 [ffff88007ffc3d78] synchronize_sched at ffffffff8106307b
#4 [ffff88007ffc3db8] dev_deactivate at ffffffff81262d5f
#5 [ffff88007ffc3de8] __linkwatch_run_queue at ffffffff8125a8ea
#6 [ffff88007ffc3e28] linkwatch_event at ffffffff8125a954
#7 [ffff88007ffc3e38] worker_thread at ffffffff8106195f
#8 [ffff88007ffc3ee8] kthread at ffffffff81064cc5
#9 [ffff88007ffc3f48] kernel_thread at ffffffff81011baa

PID: 26 TASK: ffff88007ffad4c0 CPU: 5 COMMAND: "events/5"
#0 [ffff88007ffd5cc0] schedule at ffffffff812fb2a7
#1 [ffff88007ffd5d98] __mutex_lock_common at ffffffff812fbb3b
#2 [ffff88007ffd5e08] mutex_lock at ffffffff812fbc63
#3 [ffff88007ffd5e28] linkwatch_event at ffffffff8125a93d
#4 [ffff88007ffd5e38] worker_thread at ffffffff8106195f
#5 [ffff88007ffd5ee8] kthread at ffffffff81064cc5
#6 [ffff88007ffd5f48] kernel_thread at ffffffff81011baa

PID: 32 TASK: ffff88007f418000 CPU: 5 COMMAND: "xenwatch"
#0 [ffff88007f413c60] schedule at ffffffff812fb2a7
#1 [ffff88007f413d38] __mutex_lock_common at ffffffff812fbb3b
#2 [ffff88007f413da8] mutex_lock at ffffffff812fbc63
#3 [ffff88007f413dc8] netif_notify_peers at ffffffff8126316d
#4 [ffff88007f413dd8] backend_changed at ffffffffa000a1b0
#5 [ffff88007f413e78] xenwatch_thread at ffffffff811f1628
#6 [ffff88007f413ee8] kthread at ffffffff81064cc5
#7 [ffff88007f413f48] kernel_thread at ffffffff81011baa

-Timo

--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 84linf8gxp.fsf@sauna.l.org">http://lists.debian.org/84linf8gxp.fsf@sauna.l.org