FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Kernel

 
 
LinkBack Thread Tools
 
Old 09-23-2010, 11:50 AM
Timo Juhani Lindfors
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

Package: linux-2.6
Version: 2.6.32-23
Severity: minor

[ Note that this bug is against a kernel that has been modified to add
debugging options. I report it to debian only because I was suggested
so on #debian-kernel.]

Steps to reproduce:
1) To debug #580889 I rebuilt the kernel with various debug options:

$ diff -u /boot/config-2.6.32-5-amd64 /boot/config-2.6.32-5+lindi.1-amd64 |grep -v "#"
--- /boot/config-2.6.32-5-amd64 2010-09-18 01:11:39.000000000 +0300
+++ /boot/config-2.6.32-5+lindi.1-amd64 2010-09-22 20:08:55.000000000 +0300
@@ -1,7 +1,7 @@
CONFIG_64BIT=y
@@ -133,7 +133,7 @@
CONFIG_UID16=y
CONFIG_SYSCTL_SYSCALL=y
CONFIG_KALLSYMS=y
+CONFIG_KALLSYMS_ALL=y
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
@@ -4367,15 +4367,19 @@
+CONFIG_DEBUG_RT_MUTEXES=y
+CONFIG_DEBUG_PI_LIST=y
+CONFIG_DEBUG_SPINLOCK=y
+CONFIG_DEBUG_MUTEXES=y
+CONFIG_DEBUG_LOCK_ALLOC=y
+CONFIG_PROVE_LOCKING=y
+CONFIG_LOCKDEP=y
+CONFIG_LOCK_STAT=y
+CONFIG_TRACE_IRQFLAGS=y
+CONFIG_DEBUG_SPINLOCK_SLEEP=y
+CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
CONFIG_STACKTRACE=y
CONFIG_DEBUG_BUGVERBOSE=y
@@ -4384,15 +4388,15 @@
CONFIG_DEBUG_MEMORY_INIT=y
+CONFIG_DEBUG_LIST=y
+CONFIG_DEBUG_NOTIFIERS=y
CONFIG_ARCH_WANT_FRAME_POINTERS=y
+CONFIG_FRAME_POINTER=y
+CONFIG_RCU_CPU_STALL_DETECTOR=y
@@ -4448,7 +4452,7 @@
CONFIG_X86_VERBOSE_BOOTUP=y
CONFIG_EARLY_PRINTK=y
+CONFIG_DEBUG_STACKOVERFLOW=y

2) I then booted the kernel and used "xm sysrq lindi1 t" to get a
backtrace from all threads but forgot to run "xm console lindi1". (The
bug does not occur if you are connected to the console!).

Expected results:
2) domU does not crash

Actual results:
2) domU prints a lot of backtraces and then panics.

More info:
1) I was able to capture the panic message by running

sudo gdbserver-xen 127.0.0.1:9999 --attach 4
sudo gdb /local/lindi/vmlinux-2.6.32-5-amd64
dump memory dump.txt __log_buf (__log_buf + 4096*100)

on dom0:

<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 00000000ffffffff 0 1440 1 0x00000080
<4>[ 71.816059] ffff88007d483c68 0000000000000246 ffff88007d483c38 ffff88007d483c34
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d483fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e554ec0 ffff88007ff52760
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000031 0 1441 1 0x00000080
<4>[ 71.816059] ffff88007d485c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d485fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e552760 ffff88007ddf3b10
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000000 0 1442 1 0x00000080
<4>[ 71.816059] ffff88007d487c68 0000000000000246 0000000000000000 ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d487fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e556270 ffff88007ff54ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000033 0 1443 1 0x00000080
<4>[ 71.816059] ffff88007d491c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d491fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d488000 ffff88007d4893b0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000034 0 1444 1 0x00000080
<4>[ 71.816059] ffff88007d493c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d493fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4893b0 ffff88007d4a6270
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000035 0 1445 1 0x00000080
<4>[ 71.816059] ffff88007d495c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d495fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48a760 ffff88007d4a6270
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000036 0 1446 1 0x00000080
<4>[ 71.816059] ffff88007d499c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d499fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48bb10 ffff88007db493b0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000037 0 1447 1 0x00000080
<4>[ 71.816059] ffff88007d49bc68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49bfd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48cec0 ffff88007d48e270
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000038 0 1448 1 0x00000080
<4>[ 71.816059] ffff88007d49dc68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49dfd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48e270 ffff88007ff54ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000039 0 1449 1 0x00000080
<4>[ 71.816059] ffff88007d49fc68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49ffd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a0000 ffff88007ff54ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 000000000000003a 0 1450 1 0x00000080
<4>[ 71.816059] ffff88007d4a9c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4a9fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a13b0 ffff88007ff54ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000000 0 1451 1 0x00000080
<4>[ 71.816059] ffff88007d4abc68 0000000000000246 0000000000000000 ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4abfd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a2760 ffff88007ff52760
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 000000000000003c 0 1452 1 0x00000080
<4>[ 71.816059] ffff88007d4adc68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4adfd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a3b10 ffff88007ff54ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 000000000000003d 0 1453 1 0x00000080
<4>[ 71.816059] ffff88007d4afc68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4affd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a4ec0 ffff88007ff54ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 000000000000003e 0 1454 1 0x00000080
<4>[ 71.816059] ffff88007d4b9c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4b9fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a6270 ffff88007ff54ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000000 0 1461 1 0x00000080
<4>[ 71.816059] ffff88007ebb9cc8 0000000000000246 0000000000000000 0000000000000000
<4>[ 71.816059] 0000000000000293 00000000000002b7 000000000000fa40 ffff88007ebb9fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7bb10 ffffffff8151be20
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff81339494>] ? __mutex_unlock_slowpath+0x132/0x13b
<4>[ 71.816059] [<ffffffff8110a7c3>] pipe_wait+0x5d/0x79
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81339a3b>] ? mutex_lock_nested+0x37/0x3c
<4>[ 71.816059] [<ffffffff8110b039>] pipe_read+0x361/0x3dc
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff81103685>] do_sync_read+0xe7/0x12d
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8116dba3>] ? security_file_permission+0x11/0x13
<4>[ 71.816059] [<ffffffff8110411d>] vfs_read+0xa8/0x102
<4>[ 71.816059] [<ffffffff8110423b>] sys_read+0x47/0x70
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] exim4 S 0000000000000000 0 1462 1 0x00000080
<4>[ 71.816059] ffff88007da7f918 0000000000000246 0000000000000000 0000000000000246
<4>[ 71.816059] 00000000000002d2 ffffffff81afd7e0 000000000000fa40 ffff88007da7ffd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d33a760 ffffffff8151be20
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
<4>[ 71.816059] [<ffffffff8106a41c>] ? add_wait_queue+0x3c/0x45
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8111285b>] ? __pollwait+0xc6/0xcf
<4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
<4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
<4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
<4>[ 71.816059] [<ffffffff810796fa>] ? register_lock_class+0x20/0x351
<4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
<4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8133a94c>] ? _spin_unlock_irq+0x2b/0x36
<4>[ 71.816059] [<ffffffff8107a98c>] ? trace_hardirqs_on_caller+0x110/0x134
<4>[ 71.816059] [<ffffffff8107a9bd>] ? trace_hardirqs_on+0xd/0xf
<4>[ 71.816059] [<ffffffff8133a953>] ? _spin_unlock_irq+0x32/0x36
<4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
<4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] getty S 0000000000000000 0 1490 1 0x00000080
<4>[ 71.816059] ffff88007da5dd08 0000000000000246 0000000000000000 0000000000000019
<4>[ 71.816059] ffff88007da5dc88 ffffffff8107a4b3 000000000000fa40 ffff88007da5dfd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7e270 ffffffff8151be20
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff81339060>] schedule_timeout+0x31/0xef
<4>[ 71.816059] [<ffffffff8105e704>] ? del_timer_sync+0x8e/0x9e
<4>[ 71.816059] [<ffffffff8105e676>] ? del_timer_sync+0x0/0x9e
<4>[ 71.816059] [<ffffffff810670c4>] ? flush_delayed_work+0x50/0x55
<4>[ 71.816059] [<ffffffff8122c57c>] n_tty_read+0x50b/0x7e1
<4>[ 71.816059] [<ffffffff8104cf3f>] ? default_wake_function+0x0/0xf
<4>[ 71.816059] [<ffffffff81227fa1>] tty_read+0x7d/0xba
<4>[ 71.816059] [<ffffffff8110411d>] vfs_read+0xa8/0x102
<4>[ 71.816059] [<ffffffff8110423b>] sys_read+0x47/0x70
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] sshd S 0000000000000000 0 1491 1211 0x00000080
<4>[ 71.816059] ffff88007d2fd918 0000000000000246 ffff8800079c0640 ffff8800079c0640
<4>[ 71.816059] ffff88007d2fd938 ffffffff81066ee2 000000000000fa40 ffff88007d2fdfd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7cec0 ffff88007d4c2760
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff81066ee2>] ? flush_work+0xcd/0xf2
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff8105e668>] ? try_to_del_timer_sync+0x66/0x74
<4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
<4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
<4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
<4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
<4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
<4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
<4>[ 71.816059] [<ffffffff81103558>] ? do_sync_write+0xe7/0x12d
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
<4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] bash S 0000000000000000 0 1492 1491 0x00000080
<4>[ 71.816059] ffff88007d6d7e68 0000000000000246 0000000000000000 00000000000001f0
<4>[ 71.816059] ffffffff8100de75 ffff88007d6d7e48 000000000000fa40 ffff88007d6d7fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dde2760 ffff88007ff54ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81053ef5>] do_wait+0x1b9/0x227
<4>[ 71.816059] [<ffffffff81054009>] sys_wait4+0xa6/0xc3
<4>[ 71.816059] [<ffffffff81052711>] ? child_wait_callback+0x0/0x53
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] screen S 0000000000000000 0 1508 1492 0x00000080
<4>[ 71.816059] ffff88007ebd1f68 0000000000000246 0000000000000000 ffffffff8107a98c
<4>[ 71.816059] ffff88007ddfd6c8 ffff88007ebd1f48 000000000000fa40 ffff88007ebd1fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e526270 ffffffff8151be20
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8107a98c>] ? trace_hardirqs_on_caller+0x110/0x134
<4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
<4>[ 71.816059] [<ffffffff8105f565>] sys_pause+0x19/0x22
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] screen S 0000000000000000 0 1509 1508 0x00000080
<4>[ 71.816059] ffff88007d80b918 0000000000000246 0000000000000000 ffff8800079c0640
<4>[ 71.816059] ffff88007d80b938 ffffffff81066ee2 000000000000fa40 ffff88007d80bfd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dde0000 ffffffff8151be20
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff81066ee2>] ? flush_work+0xcd/0xf2
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8105e668>] ? try_to_del_timer_sync+0x66/0x74
<4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
<4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
<4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
<4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
<4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
<4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
<4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] bash S 0000000000000000 0 1510 1509 0x00000080
<4>[ 71.816059] ffff88007da75e68 0000000000000246 0000000000000000 00000000000001f0
<4>[ 71.816059] ffffffff8100de75 ffff88007da75e48 000000000000fa40 ffff88007da75fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4c4ec0 ffffffff8151be20
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81053ef5>] do_wait+0x1b9/0x227
<4>[ 71.816059] [<ffffffff81054009>] sys_wait4+0xa6/0xc3
<4>[ 71.816059] [<ffffffff81052711>] ? child_wait_callback+0x0/0x53
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] sudo S 0000000000000000 0 1531 1510 0x00000080
<4>[ 71.816059] ffff88007d62f918 0000000000000246 0000000000000000 ffff88007d379900
<4>[ 71.816059] 0000000000000246 00000010739b297e 000000000000fa40 ffff88007d62ffd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7a760 ffff88007ff53b10
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8111285b>] ? __pollwait+0xc6/0xcf
<4>[ 71.816059] [<ffffffff812eb9ba>] ? unix_peer_get+0x19/0x3d
<4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
<4>[ 71.816059] [<ffffffff8133a9d2>] ? _spin_unlock+0x26/0x2a
<4>[ 71.816059] [<ffffffff812eb9d6>] ? unix_peer_get+0x35/0x3d
<4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
<4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
<4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
<4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
<4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
<4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
<4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] sh R running task 0 1532 1531 0x00000088
<4>[ 71.816059] ffff88007e56b838 ffff880007b9c580 ffff880007b9c590 0000000000000000
<4>[ 71.816059] 0000000000000005 0000000affffffff ffff88007e56b868 ffffffff811b2c0d
<4>[ 71.816059] ffff88007e56b918 ffffffff811b2de7 3178302b68746100 000062310000000a
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff811b2c0d>] ? put_dec+0x24/0xca
<4>[ 71.816059] [<ffffffff811b2de7>] ? number+0x134/0x246
<4>[ 71.816059] [<ffffffff8133d7e1>] ? __atomic_notifier_call_chain+0x78/0x87
<4>[ 71.816059] [<ffffffff8133a9d2>] ? _spin_unlock+0x26/0x2a
<4>[ 71.816059] [<ffffffff81237e24>] ? vt_console_print+0x28f/0x29e
<4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
<4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
<4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
<4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff81051a26>] ? vprintk+0x396/0x3d8
<4>[ 71.816059] [<ffffffff81237e24>] ? vt_console_print+0x28f/0x29e
<4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff81338252>] ? printk+0x67/0x6d
<4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
<4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
<4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
<4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff81051a26>] ? vprintk+0x396/0x3d8
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81085361>] ? __module_text_address+0xd/0x5b
<4>[ 71.816059] [<ffffffff810853b8>] ? is_module_text_address+0x9/0x11
<4>[ 71.816059] [<ffffffff81068128>] ? __kernel_text_address+0x65/0x6b
<4>[ 71.816059] [<ffffffff81068128>] ? __kernel_text_address+0x65/0x6b
<4>[ 71.816059] [<ffffffff81014828>] ? print_context_stack+0xa2/0xba
<4>[ 71.816059] [<ffffffff81013d3f>] ? dump_trace+0x260/0x271
<4>[ 71.816059] [<ffffffff810146df>] show_trace_log_lvl+0x4c/0x58
<4>[ 71.816059] [<ffffffff810138c7>] show_stack_log_lvl+0xff/0x10e
<4>[ 71.816059] [<ffffffff81014691>] show_stack+0x17/0x19
<4>[ 71.816059] [<ffffffff8104a227>] sched_show_task+0xae/0xb3
<4>[ 71.816059] [<ffffffff8104a27c>] show_state_filter+0x50/0xa9
<4>[ 71.816059] [<ffffffff8123d658>] sysrq_handle_showstate+0xb/0xd
<4>[ 71.816059] [<ffffffff8123d52f>] __handle_sysrq+0xbb/0x158
<4>[ 71.816059] [<ffffffff8123d5cc>] ? write_sysrq_trigger+0x0/0x44
<4>[ 71.816059] [<ffffffff8123d605>] write_sysrq_trigger+0x39/0x44
<4>[ 71.816059] [<ffffffff81149aad>] proc_reg_write+0x84/0xa1
<4>[ 71.816059] [<ffffffff81103f2f>] vfs_write+0xab/0x105
<4>[ 71.816059] [<ffffffff8110404d>] sys_write+0x47/0x6f
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<4>[ 71.816059] Sched Debug Version: v0.09, 2.6.32-5+lindi.1-amd64 #1
<4>[ 71.816059] now at 85287.968302 msecs
<4>[ 71.816059] .jiffies : 4294912466
<4>[ 71.816059] .sysctl_sched_latency : 15.000000
<4>[ 71.816059] .sysctl_sched_min_granularity : 3.000000
<4>[ 71.816059] .sysctl_sched_wakeup_granularity : 3.000000
<4>[ 71.816059] .sysctl_sched_child_runs_first : 0.000000
<4>[ 71.816059] .sysctl_sched_features : 15834235
<4>[ 71.816059]
<4>[ 71.816059] cpu#0, 2394.000 MHz
<4>[ 71.816059] .nr_running : 1
<4>[ 71.816059] .load : 1024
<4>[ 71.816059] .nr_switches : 11454
<4>[ 71.816059] .nr_load_updates : 5618
<4>[ 71.816059] .nr_uninterruptible : 1
<4>[ 71.816059] .next_balance : 4294.910575
<4>[ 71.816059] .curr->pid : 1046
<4>[ 71.816059] .clock : 75789.077421
<4>[ 71.816059] .cpu_load[0] : 0
<4>[ 71.816059] .cpu_load[1] : 0
<4>[ 71.816059] .cpu_load[2] : 0
<4>[ 71.816059] .cpu_load[3] : 0
<4>[ 71.816059] .cpu_load[4] : 0
<4>[ 71.816059]
<4>[ 71.816059] cfs_rq[0]:/
<4>[ 71.816059] .exec_clock : 0.000000
<4>[ 71.816059] .MIN_vruntime : 0.000001
<4>[ 71.816059] .min_vruntime : 9426.044216
<4>[ 71.816059] .max_vruntime : 0.000001
<4>[ 71.816059] .spread : 0.000000
<4>[ 71.816059] .spread0 : 0.000000
<4>[ 71.816059] .nr_running : 1
<4>[ 71.816059] .load : 1024
<4>[ 71.816059] .nr_spread_over : 0
<4>[ 71.816059] .shares : 0
<4>[ 71.816059]
<4>[ 71.816059] rt_rq[0]:
<4>[ 71.816059] .rt_nr_running : 0
<4>[ 71.816059] .rt_throttled : 0
<4>[ 71.816059] .rt_time : 0.000000
<4>[ 71.816059] .rt_runtime : 950.000000
<4>[ 71.816059]
<4>[ 71.816059] runnable tasks:
<4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
<4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
<4>[ 71.816059] R nsca 1046 9426.044216 51 120 0 0 0.000000 0.000000 0.000000 /
<4>[ 71.816059]
<4>[ 71.816059] cpu#1, 2394.000 MHz
<4>[ 71.816059] .nr_running : 1
<4>[ 71.816059] .load : 1024
<4>[ 71.816059] .nr_switches : 8912
<4>[ 71.816059] .nr_load_updates : 5067
<4>[ 71.816059] .nr_uninterruptible : 17
<4>[ 71.816059] .next_balance : 4294.910025
<4>[ 71.816059] .curr->pid : 1532
<4>[ 71.816059] .clock : 71816.059281
<4>[ 71.816059] .cpu_load[0] : 1024
<4>[ 71.816059] .cpu_load[1] : 512
<4>[ 71.816059] .cpu_load[2] : 256
<4>[ 71.816059] .cpu_load[3] : 128
<4>[ 71.816059] .cpu_load[4] : 64
<4>[ 71.816059]
<4>[ 71.816059] cfs_rq[1]:/
<4>[ 71.816059] .exec_clock : 0.000000
<4>[ 71.816059] .MIN_vruntime : 0.000001
<4>[ 71.816059] .min_vruntime : 12008.498567
<4>[ 71.816059] .max_vruntime : 0.000001
<4>[ 71.816059] .spread : 0.000000
<4>[ 71.816059] .spread0 : 2582.454351
<4>[ 71.816059] .nr_running : 1
<4>[ 71.816059] .load : 1024
<4>[ 71.816059] .nr_spread_over : 0
<4>[ 71.816059] .shares : 0
<4>[ 71.816059]
<4>[ 71.816059] rt_rq[1]:
<4>[ 71.816059] .rt_nr_running : 0
<4>[ 71.816059] .rt_throttled : 0
<4>[ 71.816059] .rt_time : 0.000000
<4>[ 71.816059] .rt_runtime : 950.000000
<4>[ 71.816059]
<4>[ 71.816059] runnable tasks:
<4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
<4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
<4>[ 71.816059] R sh 1532 12008.498567 3 120 0 0 0.000000 0.000000 0.000000 /
<4>[ 71.816059]
<4>[ 71.816059] cpu#2, 2394.000 MHz
<4>[ 71.816059] .nr_running : 0
<4>[ 71.816059] .load : 0
<4>[ 71.816059] .nr_switches : 5200
<4>[ 71.816059] .nr_load_updates : 6701
<4>[ 71.816059] .nr_uninterruptible : -8
<4>[ 71.816059] .next_balance : 4294.912466
<4>[ 71.816059] .curr->pid : 0
<4>[ 71.816059] .clock : 80673.012224
<4>[ 71.816059] .cpu_load[0] : 0
<4>[ 71.816059] .cpu_load[1] : 0
<4>[ 71.816059] .cpu_load[2] : 0
<4>[ 71.816059] .cpu_load[3] : 0
<4>[ 71.816059] .cpu_load[4] : 0
<4>[ 71.816059]
<4>[ 71.816059] cfs_rq[2]:/
<4>[ 71.816059] .exec_clock : 0.000000
<4>[ 71.816059] .MIN_vruntime : 0.000001
<4>[ 71.816059] .min_vruntime : 7154.290231
<4>[ 71.816059] .max_vruntime : 0.000001
<4>[ 71.816059] .spread : 0.000000
<4>[ 71.816059] .spread0 : -2271.753985
<4>[ 71.816059] .nr_running : 0
<4>[ 71.816059] .load : 0
<4>[ 71.816059] .nr_spread_over : 0
<4>[ 71.816059] .shares : 0
<4>[ 71.816059]
<4>[ 71.816059] rt_rq[2]:
<4>[ 71.816059] .rt_nr_running : 0
<4>[ 71.816059] .rt_throttled : 0
<4>[ 71.816059] .rt_time : 0.000000
<4>[ 71.816059] .rt_runtime : 950.000000
<4>[ 71.816059]
<4>[ 71.816059] runnable tasks:
<4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
<4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
<4>[ 71.816059]
<4>[ 71.816059] cpu#3, 2394.000 MHz
<4>[ 71.816059] .nr_running : 0
<4>[ 71.816059] .load : 0
<4>[ 71.816059] .nr_switches : 4398
<4>[ 71.816059] .nr_load_updates : 6055
<4>[ 71.816059] .nr_uninterruptible : -9
<4>[ 71.816059] .next_balance : 4294.912466
<4>[ 71.816059] .curr->pid : 0
<4>[ 71.816059] .clock : 80673.512781
<4>[ 71.816059] .cpu_load[0] : 0
<4>[ 71.816059] .cpu_load[1] : 0
<4>[ 71.816059] .cpu_load[2] : 0
<4>[ 71.816059] .cpu_load[3] : 0
<4>[ 71.816059] .cpu_load[4] : 0
<4>[ 71.816059]
<4>[ 71.816059] cfs_rq[3]:/
<4>[ 71.816059] .exec_clock : 0.000000
<4>[ 71.816059] .MIN_vruntime : 0.000001
<4>[ 71.816059] .min_vruntime : 5646.686691
<4>[ 71.816059] .max_vruntime : 0.000001
<4>[ 71.816059] .spread : 0.000000
<4>[ 71.816059] .spread0 : -3779.357525
<4>[ 71.816059] .nr_running : 0
<4>[ 71.816059] .load : 0
<4>[ 71.816059] .nr_spread_over : 0
<4>[ 71.816059] .shares : 0
<4>[ 71.816059]
<4>[ 71.816059] rt_rq[3]:
<4>[ 71.816059] .rt_nr_running : 0
<4>[ 71.816059] .rt_throttled : 0
<4>[ 71.816059] .rt_time : 0.000000
<4>[ 71.816059] .rt_runtime : 950.000000
<4>[ 71.816059]
<4>[ 71.816059] runnable tasks:
<4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
<4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
<4>[ 71.816059]

<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000004 0 1396 1 0x00000080
<4>[ 71.816059] ffff88007ddcfc68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007ddcffd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff880003560000 ffff88007dad4ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000005 0 1397 1 0x00000080
<4>[ 71.816059] ffff88007e387c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007e387fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d0d0000 ffff8800034ca760
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 00000000ffffffff 0 1398 1 0x00000080
<4>[ 71.816059] ffff88007e50dc68 0000000000000246 ffff88007e50dc38 ffff88007e50dc34
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007e50dfd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d0d13b0 ffff88007ff52760
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000007 0 1399 1 0x00000080
<4>[ 71.816059] ffff88007ddd1c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007ddd1fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dad4ec0 ffff88007d0d0000
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000008 0 1400 1 0x00000080
<4>[ 71.816059] ffff88007ebb1c68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007ebb1fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dad0000 ffff88007db24ec0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000009 0 1401 1 0x00000080
<4>[ 71.816059] ffff88007dd8bc68 0000000000000246 00000000000003da ffffffff81b1d760
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007dd8bfd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dad6270 ffff88007db293b0
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
<4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
<4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
<4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
<4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
<4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
<6>[ 71.816059] console-kit-d S 0000000000000001 0 1402 1 0x00000088
<4>[ 71.816059] ffff88007db05c68 0000000000000246 ffff88007db05c38 ffff88007db05c34
<4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007db05fd8
<4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e5213b0 ffff880003566270
<4>[ 71.816059] Call Trace:
<4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
<4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
<4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
<4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
<4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
<4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
<4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
<4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
<4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
<4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evt
 
Old 09-23-2010, 09:06 PM
Ian Campbell
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

Looks like SysRQ-t somehow causes an RCU stall, which may be a bug in
its own right (or maybe SysRQ-t is just slow?), but the eventual BUG is
because the RCU call causes arch_trigger_all_cpu_backtrace to be called
which, on x86, accesses the APIC directly to send an IPI to every CPU
which won't work under Xen.

I guess the right thing to do would be to using some existing paravirt
interface to send the IPI instead of tweaking the APIC directly, or if
necessary to add such an interface, Jeremy, what do you think? Is there
an existing interface we could use?

Timo, (this may not be as useful as I first though since I didn't twig
that you were using sysrq-t not sysrq-l, which causes the
arch_trigger_all_cpu_backtrace issue directly, until just now, I leave
it in since I've already typed it)... you may find the xenctx tool
useful as a workaround you can run it in domain 0 (usually it is
installed in /usr/lib/xen*/bin/xenctx) giving it the guest's System.map,
a domid and a vcpu id and it will give you a stack trace for that VCPU.
e.g. domain 5 vcpu 0:
# xenctx -s /boot/System.map-2.6.32.21-x86_32p-xenU-00969-gb297cda 5 0
cs:eip: 0061:c10013a7 _stext+0x3a7
flags: 00001246 i z p
ss:esp: 0069:c12d3f8c
eax: 00000000 ebx: deadbeef ecx: deadbeef edx: 00000004
esi: c12f15f0 edi: c12db074 ebp: c12d3f9c
ds: 007b es: 007b fs: 00d8 gs: 0000
Code (instr addr c10013a7)
cc cc cc cc cc cc cc cc cc cc cc cc cc cc b8 1d 00 00 00 cd 82 <c3> cc cc cc cc cc cc cc cc cc cc


Stack:
c1006528 00000000 c12f15f0 c12db074 c12d3fa4 c10034a9 c12d3fb4 c1007cae
c1319d88 c13191f0 c12d3fbc c11e71cf c12d3fcc c12f384c c1319d88 40000000
c12d3fd8 c12f30a2 40000000 c12d3ffc c12f6445 1f898975 80000201 01040800
000006fb 00000000 c19ca000 00000000 00000000

Call Trace:
[<c10013a7>] _stext+0x3a7 <--
[<c1006528>] xen_safe_halt+0x12
[<c10034a9>] xen_idle+0x5d
[<c1007cae>] cpu_idle+0x5b
[<c11e71cf>] rest_init+0x53
[<c12f384c>] start_kernel+0x309
[<c12f30a2>] i386_start_kernel+0x91
[<c12f6445>] xen_start_kernel+0x524


On Thu, 2010-09-23 at 14:50 +0300, Timo Juhani Lindfors wrote:
> Package: linux-2.6
> Version: 2.6.32-23
> Severity: minor
>
> [ Note that this bug is against a kernel that has been modified to add
> debugging options. I report it to debian only because I was suggested
> so on #debian-kernel.]
>
> Steps to reproduce:
> 1) To debug #580889 I rebuilt the kernel with various debug options:
>
> $ diff -u /boot/config-2.6.32-5-amd64 /boot/config-2.6.32-5+lindi.1-amd64 |grep -v "#"
> --- /boot/config-2.6.32-5-amd64 2010-09-18 01:11:39.000000000 +0300
> +++ /boot/config-2.6.32-5+lindi.1-amd64 2010-09-22 20:08:55.000000000 +0300
> @@ -1,7 +1,7 @@
> CONFIG_64BIT=y
> @@ -133,7 +133,7 @@
> CONFIG_UID16=y
> CONFIG_SYSCTL_SYSCALL=y
> CONFIG_KALLSYMS=y
> +CONFIG_KALLSYMS_ALL=y
> CONFIG_HOTPLUG=y
> CONFIG_PRINTK=y
> @@ -4367,15 +4367,19 @@
> +CONFIG_DEBUG_RT_MUTEXES=y
> +CONFIG_DEBUG_PI_LIST=y
> +CONFIG_DEBUG_SPINLOCK=y
> +CONFIG_DEBUG_MUTEXES=y
> +CONFIG_DEBUG_LOCK_ALLOC=y
> +CONFIG_PROVE_LOCKING=y
> +CONFIG_LOCKDEP=y
> +CONFIG_LOCK_STAT=y
> +CONFIG_TRACE_IRQFLAGS=y
> +CONFIG_DEBUG_SPINLOCK_SLEEP=y
> +CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
> CONFIG_STACKTRACE=y
> CONFIG_DEBUG_BUGVERBOSE=y
> @@ -4384,15 +4388,15 @@
> CONFIG_DEBUG_MEMORY_INIT=y
> +CONFIG_DEBUG_LIST=y
> +CONFIG_DEBUG_NOTIFIERS=y
> CONFIG_ARCH_WANT_FRAME_POINTERS=y
> +CONFIG_FRAME_POINTER=y
> +CONFIG_RCU_CPU_STALL_DETECTOR=y
> @@ -4448,7 +4452,7 @@
> CONFIG_X86_VERBOSE_BOOTUP=y
> CONFIG_EARLY_PRINTK=y
> +CONFIG_DEBUG_STACKOVERFLOW=y
>
> 2) I then booted the kernel and used "xm sysrq lindi1 t" to get a
> backtrace from all threads but forgot to run "xm console lindi1". (The
> bug does not occur if you are connected to the console!).
>
> Expected results:
> 2) domU does not crash
>
> Actual results:
> 2) domU prints a lot of backtraces and then panics.
>
> More info:
> 1) I was able to capture the panic message by running
>
> sudo gdbserver-xen 127.0.0.1:9999 --attach 4
> sudo gdb /local/lindi/vmlinux-2.6.32-5-amd64
> dump memory dump.txt __log_buf (__log_buf + 4096*100)
>
> on dom0:
>
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 00000000ffffffff 0 1440 1 0x00000080
> <4>[ 71.816059] ffff88007d483c68 0000000000000246 ffff88007d483c38 ffff88007d483c34
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d483fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e554ec0 ffff88007ff52760
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000031 0 1441 1 0x00000080
> <4>[ 71.816059] ffff88007d485c68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d485fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e552760 ffff88007ddf3b10
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000000 0 1442 1 0x00000080
> <4>[ 71.816059] ffff88007d487c68 0000000000000246 0000000000000000 ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d487fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e556270 ffff88007ff54ec0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000033 0 1443 1 0x00000080
> <4>[ 71.816059] ffff88007d491c68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d491fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d488000 ffff88007d4893b0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000034 0 1444 1 0x00000080
> <4>[ 71.816059] ffff88007d493c68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d493fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4893b0 ffff88007d4a6270
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000035 0 1445 1 0x00000080
> <4>[ 71.816059] ffff88007d495c68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d495fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48a760 ffff88007d4a6270
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000036 0 1446 1 0x00000080
> <4>[ 71.816059] ffff88007d499c68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d499fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48bb10 ffff88007db493b0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000037 0 1447 1 0x00000080
> <4>[ 71.816059] ffff88007d49bc68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49bfd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48cec0 ffff88007d48e270
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000038 0 1448 1 0x00000080
> <4>[ 71.816059] ffff88007d49dc68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49dfd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48e270 ffff88007ff54ec0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000039 0 1449 1 0x00000080
> <4>[ 71.816059] ffff88007d49fc68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49ffd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a0000 ffff88007ff54ec0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 000000000000003a 0 1450 1 0x00000080
> <4>[ 71.816059] ffff88007d4a9c68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4a9fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a13b0 ffff88007ff54ec0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000000 0 1451 1 0x00000080
> <4>[ 71.816059] ffff88007d4abc68 0000000000000246 0000000000000000 ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4abfd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a2760 ffff88007ff52760
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 000000000000003c 0 1452 1 0x00000080
> <4>[ 71.816059] ffff88007d4adc68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4adfd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a3b10 ffff88007ff54ec0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 000000000000003d 0 1453 1 0x00000080
> <4>[ 71.816059] ffff88007d4afc68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4affd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a4ec0 ffff88007ff54ec0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 000000000000003e 0 1454 1 0x00000080
> <4>[ 71.816059] ffff88007d4b9c68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4b9fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a6270 ffff88007ff54ec0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000000 0 1461 1 0x00000080
> <4>[ 71.816059] ffff88007ebb9cc8 0000000000000246 0000000000000000 0000000000000000
> <4>[ 71.816059] 0000000000000293 00000000000002b7 000000000000fa40 ffff88007ebb9fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7bb10 ffffffff8151be20
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff81339494>] ? __mutex_unlock_slowpath+0x132/0x13b
> <4>[ 71.816059] [<ffffffff8110a7c3>] pipe_wait+0x5d/0x79
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81339a3b>] ? mutex_lock_nested+0x37/0x3c
> <4>[ 71.816059] [<ffffffff8110b039>] pipe_read+0x361/0x3dc
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff81103685>] do_sync_read+0xe7/0x12d
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8116dba3>] ? security_file_permission+0x11/0x13
> <4>[ 71.816059] [<ffffffff8110411d>] vfs_read+0xa8/0x102
> <4>[ 71.816059] [<ffffffff8110423b>] sys_read+0x47/0x70
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] exim4 S 0000000000000000 0 1462 1 0x00000080
> <4>[ 71.816059] ffff88007da7f918 0000000000000246 0000000000000000 0000000000000246
> <4>[ 71.816059] 00000000000002d2 ffffffff81afd7e0 000000000000fa40 ffff88007da7ffd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d33a760 ffffffff8151be20
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
> <4>[ 71.816059] [<ffffffff8106a41c>] ? add_wait_queue+0x3c/0x45
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8111285b>] ? __pollwait+0xc6/0xcf
> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
> <4>[ 71.816059] [<ffffffff810796fa>] ? register_lock_class+0x20/0x351
> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8133a94c>] ? _spin_unlock_irq+0x2b/0x36
> <4>[ 71.816059] [<ffffffff8107a98c>] ? trace_hardirqs_on_caller+0x110/0x134
> <4>[ 71.816059] [<ffffffff8107a9bd>] ? trace_hardirqs_on+0xd/0xf
> <4>[ 71.816059] [<ffffffff8133a953>] ? _spin_unlock_irq+0x32/0x36
> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] getty S 0000000000000000 0 1490 1 0x00000080
> <4>[ 71.816059] ffff88007da5dd08 0000000000000246 0000000000000000 0000000000000019
> <4>[ 71.816059] ffff88007da5dc88 ffffffff8107a4b3 000000000000fa40 ffff88007da5dfd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7e270 ffffffff8151be20
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff81339060>] schedule_timeout+0x31/0xef
> <4>[ 71.816059] [<ffffffff8105e704>] ? del_timer_sync+0x8e/0x9e
> <4>[ 71.816059] [<ffffffff8105e676>] ? del_timer_sync+0x0/0x9e
> <4>[ 71.816059] [<ffffffff810670c4>] ? flush_delayed_work+0x50/0x55
> <4>[ 71.816059] [<ffffffff8122c57c>] n_tty_read+0x50b/0x7e1
> <4>[ 71.816059] [<ffffffff8104cf3f>] ? default_wake_function+0x0/0xf
> <4>[ 71.816059] [<ffffffff81227fa1>] tty_read+0x7d/0xba
> <4>[ 71.816059] [<ffffffff8110411d>] vfs_read+0xa8/0x102
> <4>[ 71.816059] [<ffffffff8110423b>] sys_read+0x47/0x70
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] sshd S 0000000000000000 0 1491 1211 0x00000080
> <4>[ 71.816059] ffff88007d2fd918 0000000000000246 ffff8800079c0640 ffff8800079c0640
> <4>[ 71.816059] ffff88007d2fd938 ffffffff81066ee2 000000000000fa40 ffff88007d2fdfd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7cec0 ffff88007d4c2760
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff81066ee2>] ? flush_work+0xcd/0xf2
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff8105e668>] ? try_to_del_timer_sync+0x66/0x74
> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
> <4>[ 71.816059] [<ffffffff81103558>] ? do_sync_write+0xe7/0x12d
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] bash S 0000000000000000 0 1492 1491 0x00000080
> <4>[ 71.816059] ffff88007d6d7e68 0000000000000246 0000000000000000 00000000000001f0
> <4>[ 71.816059] ffffffff8100de75 ffff88007d6d7e48 000000000000fa40 ffff88007d6d7fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dde2760 ffff88007ff54ec0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81053ef5>] do_wait+0x1b9/0x227
> <4>[ 71.816059] [<ffffffff81054009>] sys_wait4+0xa6/0xc3
> <4>[ 71.816059] [<ffffffff81052711>] ? child_wait_callback+0x0/0x53
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] screen S 0000000000000000 0 1508 1492 0x00000080
> <4>[ 71.816059] ffff88007ebd1f68 0000000000000246 0000000000000000 ffffffff8107a98c
> <4>[ 71.816059] ffff88007ddfd6c8 ffff88007ebd1f48 000000000000fa40 ffff88007ebd1fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e526270 ffffffff8151be20
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8107a98c>] ? trace_hardirqs_on_caller+0x110/0x134
> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> <4>[ 71.816059] [<ffffffff8105f565>] sys_pause+0x19/0x22
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] screen S 0000000000000000 0 1509 1508 0x00000080
> <4>[ 71.816059] ffff88007d80b918 0000000000000246 0000000000000000 ffff8800079c0640
> <4>[ 71.816059] ffff88007d80b938 ffffffff81066ee2 000000000000fa40 ffff88007d80bfd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dde0000 ffffffff8151be20
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff81066ee2>] ? flush_work+0xcd/0xf2
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8105e668>] ? try_to_del_timer_sync+0x66/0x74
> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] bash S 0000000000000000 0 1510 1509 0x00000080
> <4>[ 71.816059] ffff88007da75e68 0000000000000246 0000000000000000 00000000000001f0
> <4>[ 71.816059] ffffffff8100de75 ffff88007da75e48 000000000000fa40 ffff88007da75fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4c4ec0 ffffffff8151be20
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81053ef5>] do_wait+0x1b9/0x227
> <4>[ 71.816059] [<ffffffff81054009>] sys_wait4+0xa6/0xc3
> <4>[ 71.816059] [<ffffffff81052711>] ? child_wait_callback+0x0/0x53
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] sudo S 0000000000000000 0 1531 1510 0x00000080
> <4>[ 71.816059] ffff88007d62f918 0000000000000246 0000000000000000 ffff88007d379900
> <4>[ 71.816059] 0000000000000246 00000010739b297e 000000000000fa40 ffff88007d62ffd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7a760 ffff88007ff53b10
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8111285b>] ? __pollwait+0xc6/0xcf
> <4>[ 71.816059] [<ffffffff812eb9ba>] ? unix_peer_get+0x19/0x3d
> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
> <4>[ 71.816059] [<ffffffff8133a9d2>] ? _spin_unlock+0x26/0x2a
> <4>[ 71.816059] [<ffffffff812eb9d6>] ? unix_peer_get+0x35/0x3d
> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] sh R running task 0 1532 1531 0x00000088
> <4>[ 71.816059] ffff88007e56b838 ffff880007b9c580 ffff880007b9c590 0000000000000000
> <4>[ 71.816059] 0000000000000005 0000000affffffff ffff88007e56b868 ffffffff811b2c0d
> <4>[ 71.816059] ffff88007e56b918 ffffffff811b2de7 3178302b68746100 000062310000000a
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff811b2c0d>] ? put_dec+0x24/0xca
> <4>[ 71.816059] [<ffffffff811b2de7>] ? number+0x134/0x246
> <4>[ 71.816059] [<ffffffff8133d7e1>] ? __atomic_notifier_call_chain+0x78/0x87
> <4>[ 71.816059] [<ffffffff8133a9d2>] ? _spin_unlock+0x26/0x2a
> <4>[ 71.816059] [<ffffffff81237e24>] ? vt_console_print+0x28f/0x29e
> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff81051a26>] ? vprintk+0x396/0x3d8
> <4>[ 71.816059] [<ffffffff81237e24>] ? vt_console_print+0x28f/0x29e
> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff81338252>] ? printk+0x67/0x6d
> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff81051a26>] ? vprintk+0x396/0x3d8
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81085361>] ? __module_text_address+0xd/0x5b
> <4>[ 71.816059] [<ffffffff810853b8>] ? is_module_text_address+0x9/0x11
> <4>[ 71.816059] [<ffffffff81068128>] ? __kernel_text_address+0x65/0x6b
> <4>[ 71.816059] [<ffffffff81068128>] ? __kernel_text_address+0x65/0x6b
> <4>[ 71.816059] [<ffffffff81014828>] ? print_context_stack+0xa2/0xba
> <4>[ 71.816059] [<ffffffff81013d3f>] ? dump_trace+0x260/0x271
> <4>[ 71.816059] [<ffffffff810146df>] show_trace_log_lvl+0x4c/0x58
> <4>[ 71.816059] [<ffffffff810138c7>] show_stack_log_lvl+0xff/0x10e
> <4>[ 71.816059] [<ffffffff81014691>] show_stack+0x17/0x19
> <4>[ 71.816059] [<ffffffff8104a227>] sched_show_task+0xae/0xb3
> <4>[ 71.816059] [<ffffffff8104a27c>] show_state_filter+0x50/0xa9
> <4>[ 71.816059] [<ffffffff8123d658>] sysrq_handle_showstate+0xb/0xd
> <4>[ 71.816059] [<ffffffff8123d52f>] __handle_sysrq+0xbb/0x158
> <4>[ 71.816059] [<ffffffff8123d5cc>] ? write_sysrq_trigger+0x0/0x44
> <4>[ 71.816059] [<ffffffff8123d605>] write_sysrq_trigger+0x39/0x44
> <4>[ 71.816059] [<ffffffff81149aad>] proc_reg_write+0x84/0xa1
> <4>[ 71.816059] [<ffffffff81103f2f>] vfs_write+0xab/0x105
> <4>[ 71.816059] [<ffffffff8110404d>] sys_write+0x47/0x6f
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <4>[ 71.816059] Sched Debug Version: v0.09, 2.6.32-5+lindi.1-amd64 #1
> <4>[ 71.816059] now at 85287.968302 msecs
> <4>[ 71.816059] .jiffies : 4294912466
> <4>[ 71.816059] .sysctl_sched_latency : 15.000000
> <4>[ 71.816059] .sysctl_sched_min_granularity : 3.000000
> <4>[ 71.816059] .sysctl_sched_wakeup_granularity : 3.000000
> <4>[ 71.816059] .sysctl_sched_child_runs_first : 0.000000
> <4>[ 71.816059] .sysctl_sched_features : 15834235
> <4>[ 71.816059]
> <4>[ 71.816059] cpu#0, 2394.000 MHz
> <4>[ 71.816059] .nr_running : 1
> <4>[ 71.816059] .load : 1024
> <4>[ 71.816059] .nr_switches : 11454
> <4>[ 71.816059] .nr_load_updates : 5618
> <4>[ 71.816059] .nr_uninterruptible : 1
> <4>[ 71.816059] .next_balance : 4294.910575
> <4>[ 71.816059] .curr->pid : 1046
> <4>[ 71.816059] .clock : 75789.077421
> <4>[ 71.816059] .cpu_load[0] : 0
> <4>[ 71.816059] .cpu_load[1] : 0
> <4>[ 71.816059] .cpu_load[2] : 0
> <4>[ 71.816059] .cpu_load[3] : 0
> <4>[ 71.816059] .cpu_load[4] : 0
> <4>[ 71.816059]
> <4>[ 71.816059] cfs_rq[0]:/
> <4>[ 71.816059] .exec_clock : 0.000000
> <4>[ 71.816059] .MIN_vruntime : 0.000001
> <4>[ 71.816059] .min_vruntime : 9426.044216
> <4>[ 71.816059] .max_vruntime : 0.000001
> <4>[ 71.816059] .spread : 0.000000
> <4>[ 71.816059] .spread0 : 0.000000
> <4>[ 71.816059] .nr_running : 1
> <4>[ 71.816059] .load : 1024
> <4>[ 71.816059] .nr_spread_over : 0
> <4>[ 71.816059] .shares : 0
> <4>[ 71.816059]
> <4>[ 71.816059] rt_rq[0]:
> <4>[ 71.816059] .rt_nr_running : 0
> <4>[ 71.816059] .rt_throttled : 0
> <4>[ 71.816059] .rt_time : 0.000000
> <4>[ 71.816059] .rt_runtime : 950.000000
> <4>[ 71.816059]
> <4>[ 71.816059] runnable tasks:
> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
> <4>[ 71.816059] R nsca 1046 9426.044216 51 120 0 0 0.000000 0.000000 0.000000 /
> <4>[ 71.816059]
> <4>[ 71.816059] cpu#1, 2394.000 MHz
> <4>[ 71.816059] .nr_running : 1
> <4>[ 71.816059] .load : 1024
> <4>[ 71.816059] .nr_switches : 8912
> <4>[ 71.816059] .nr_load_updates : 5067
> <4>[ 71.816059] .nr_uninterruptible : 17
> <4>[ 71.816059] .next_balance : 4294.910025
> <4>[ 71.816059] .curr->pid : 1532
> <4>[ 71.816059] .clock : 71816.059281
> <4>[ 71.816059] .cpu_load[0] : 1024
> <4>[ 71.816059] .cpu_load[1] : 512
> <4>[ 71.816059] .cpu_load[2] : 256
> <4>[ 71.816059] .cpu_load[3] : 128
> <4>[ 71.816059] .cpu_load[4] : 64
> <4>[ 71.816059]
> <4>[ 71.816059] cfs_rq[1]:/
> <4>[ 71.816059] .exec_clock : 0.000000
> <4>[ 71.816059] .MIN_vruntime : 0.000001
> <4>[ 71.816059] .min_vruntime : 12008.498567
> <4>[ 71.816059] .max_vruntime : 0.000001
> <4>[ 71.816059] .spread : 0.000000
> <4>[ 71.816059] .spread0 : 2582.454351
> <4>[ 71.816059] .nr_running : 1
> <4>[ 71.816059] .load : 1024
> <4>[ 71.816059] .nr_spread_over : 0
> <4>[ 71.816059] .shares : 0
> <4>[ 71.816059]
> <4>[ 71.816059] rt_rq[1]:
> <4>[ 71.816059] .rt_nr_running : 0
> <4>[ 71.816059] .rt_throttled : 0
> <4>[ 71.816059] .rt_time : 0.000000
> <4>[ 71.816059] .rt_runtime : 950.000000
> <4>[ 71.816059]
> <4>[ 71.816059] runnable tasks:
> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
> <4>[ 71.816059] R sh 1532 12008.498567 3 120 0 0 0.000000 0.000000 0.000000 /
> <4>[ 71.816059]
> <4>[ 71.816059] cpu#2, 2394.000 MHz
> <4>[ 71.816059] .nr_running : 0
> <4>[ 71.816059] .load : 0
> <4>[ 71.816059] .nr_switches : 5200
> <4>[ 71.816059] .nr_load_updates : 6701
> <4>[ 71.816059] .nr_uninterruptible : -8
> <4>[ 71.816059] .next_balance : 4294.912466
> <4>[ 71.816059] .curr->pid : 0
> <4>[ 71.816059] .clock : 80673.012224
> <4>[ 71.816059] .cpu_load[0] : 0
> <4>[ 71.816059] .cpu_load[1] : 0
> <4>[ 71.816059] .cpu_load[2] : 0
> <4>[ 71.816059] .cpu_load[3] : 0
> <4>[ 71.816059] .cpu_load[4] : 0
> <4>[ 71.816059]
> <4>[ 71.816059] cfs_rq[2]:/
> <4>[ 71.816059] .exec_clock : 0.000000
> <4>[ 71.816059] .MIN_vruntime : 0.000001
> <4>[ 71.816059] .min_vruntime : 7154.290231
> <4>[ 71.816059] .max_vruntime : 0.000001
> <4>[ 71.816059] .spread : 0.000000
> <4>[ 71.816059] .spread0 : -2271.753985
> <4>[ 71.816059] .nr_running : 0
> <4>[ 71.816059] .load : 0
> <4>[ 71.816059] .nr_spread_over : 0
> <4>[ 71.816059] .shares : 0
> <4>[ 71.816059]
> <4>[ 71.816059] rt_rq[2]:
> <4>[ 71.816059] .rt_nr_running : 0
> <4>[ 71.816059] .rt_throttled : 0
> <4>[ 71.816059] .rt_time : 0.000000
> <4>[ 71.816059] .rt_runtime : 950.000000
> <4>[ 71.816059]
> <4>[ 71.816059] runnable tasks:
> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
> <4>[ 71.816059]
> <4>[ 71.816059] cpu#3, 2394.000 MHz
> <4>[ 71.816059] .nr_running : 0
> <4>[ 71.816059] .load : 0
> <4>[ 71.816059] .nr_switches : 4398
> <4>[ 71.816059] .nr_load_updates : 6055
> <4>[ 71.816059] .nr_uninterruptible : -9
> <4>[ 71.816059] .next_balance : 4294.912466
> <4>[ 71.816059] .curr->pid : 0
> <4>[ 71.816059] .clock : 80673.512781
> <4>[ 71.816059] .cpu_load[0] : 0
> <4>[ 71.816059] .cpu_load[1] : 0
> <4>[ 71.816059] .cpu_load[2] : 0
> <4>[ 71.816059] .cpu_load[3] : 0
> <4>[ 71.816059] .cpu_load[4] : 0
> <4>[ 71.816059]
> <4>[ 71.816059] cfs_rq[3]:/
> <4>[ 71.816059] .exec_clock : 0.000000
> <4>[ 71.816059] .MIN_vruntime : 0.000001
> <4>[ 71.816059] .min_vruntime : 5646.686691
> <4>[ 71.816059] .max_vruntime : 0.000001
> <4>[ 71.816059] .spread : 0.000000
> <4>[ 71.816059] .spread0 : -3779.357525
> <4>[ 71.816059] .nr_running : 0
> <4>[ 71.816059] .load : 0
> <4>[ 71.816059] .nr_spread_over : 0
> <4>[ 71.816059] .shares : 0
> <4>[ 71.816059]
> <4>[ 71.816059] rt_rq[3]:
> <4>[ 71.816059] .rt_nr_running : 0
> <4>[ 71.816059] .rt_throttled : 0
> <4>[ 71.816059] .rt_time : 0.000000
> <4>[ 71.816059] .rt_runtime : 950.000000
> <4>[ 71.816059]
> <4>[ 71.816059] runnable tasks:
> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
> <4>[ 71.816059]
>
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000004 0 1396 1 0x00000080
> <4>[ 71.816059] ffff88007ddcfc68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007ddcffd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff880003560000 ffff88007dad4ec0
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000005 0 1397 1 0x00000080
> <4>[ 71.816059] ffff88007e387c68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007e387fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d0d0000 ffff8800034ca760
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 00000000ffffffff 0 1398 1 0x00000080
> <4>[ 71.816059] ffff88007e50dc68 0000000000000246 ffff88007e50dc38 ffff88007e50dc34
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007e50dfd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d0d13b0 ffff88007ff52760
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000007 0 1399 1 0x00000080
> <4>[ 71.816059] ffff88007ddd1c68 0000000000000246 00000000000003da ffffffff81b1d760
> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007ddd1fd8
> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dad4ec0 ffff88007d0d0000
> <4>[ 71.816059] Call Trace:
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> <6>[ 71.816059] console-kit-d S 0000000000000008 0 1400 1 0x00000080
> <4>[ 71.816059]
 
Old 09-23-2010, 09:54 PM
Jeremy Fitzhardinge
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

On 09/23/2010 02:06 PM, Ian Campbell wrote:
> Looks like SysRQ-t somehow causes an RCU stall, which may be a bug in
> its own right (or maybe SysRQ-t is just slow?), but the eventual BUG is
> because the RCU call causes arch_trigger_all_cpu_backtrace to be called
> which, on x86, accesses the APIC directly to send an IPI to every CPU
> which won't work under Xen.
>
> I guess the right thing to do would be to using some existing paravirt
> interface to send the IPI instead of tweaking the APIC directly, or if
> necessary to add such an interface, Jeremy, what do you think? Is there
> an existing interface we could use?

It looks like a regular cross-cpu call to me. Don't know why it needs
special IPI magic. OIC, its an NMI. Well, we'd need to add something
to replicate that as-is (can you send an NMI in Xen?).

J


> Timo, (this may not be as useful as I first though since I didn't twig
> that you were using sysrq-t not sysrq-l, which causes the
> arch_trigger_all_cpu_backtrace issue directly, until just now, I leave
> it in since I've already typed it)... you may find the xenctx tool
> useful as a workaround you can run it in domain 0 (usually it is
> installed in /usr/lib/xen*/bin/xenctx) giving it the guest's System.map,
> a domid and a vcpu id and it will give you a stack trace for that VCPU.
> e.g. domain 5 vcpu 0:
> # xenctx -s /boot/System.map-2.6.32.21-x86_32p-xenU-00969-gb297cda 5 0
> cs:eip: 0061:c10013a7 _stext+0x3a7
> flags: 00001246 i z p
> ss:esp: 0069:c12d3f8c
> eax: 00000000 ebx: deadbeef ecx: deadbeef edx: 00000004
> esi: c12f15f0 edi: c12db074 ebp: c12d3f9c
> ds: 007b es: 007b fs: 00d8 gs: 0000
> Code (instr addr c10013a7)
> cc cc cc cc cc cc cc cc cc cc cc cc cc cc b8 1d 00 00 00 cd 82 <c3> cc cc cc cc cc cc cc cc cc cc
>
>
> Stack:
> c1006528 00000000 c12f15f0 c12db074 c12d3fa4 c10034a9 c12d3fb4 c1007cae
> c1319d88 c13191f0 c12d3fbc c11e71cf c12d3fcc c12f384c c1319d88 40000000
> c12d3fd8 c12f30a2 40000000 c12d3ffc c12f6445 1f898975 80000201 01040800
> 000006fb 00000000 c19ca000 00000000 00000000
>
> Call Trace:
> [<c10013a7>] _stext+0x3a7 <--
> [<c1006528>] xen_safe_halt+0x12
> [<c10034a9>] xen_idle+0x5d
> [<c1007cae>] cpu_idle+0x5b
> [<c11e71cf>] rest_init+0x53
> [<c12f384c>] start_kernel+0x309
> [<c12f30a2>] i386_start_kernel+0x91
> [<c12f6445>] xen_start_kernel+0x524
>
>
> On Thu, 2010-09-23 at 14:50 +0300, Timo Juhani Lindfors wrote:
>> Package: linux-2.6
>> Version: 2.6.32-23
>> Severity: minor
>>
>> [ Note that this bug is against a kernel that has been modified to add
>> debugging options. I report it to debian only because I was suggested
>> so on #debian-kernel.]
>>
>> Steps to reproduce:
>> 1) To debug #580889 I rebuilt the kernel with various debug options:
>>
>> $ diff -u /boot/config-2.6.32-5-amd64 /boot/config-2.6.32-5+lindi.1-amd64 |grep -v "#"
>> --- /boot/config-2.6.32-5-amd64 2010-09-18 01:11:39.000000000 +0300
>> +++ /boot/config-2.6.32-5+lindi.1-amd64 2010-09-22 20:08:55.000000000 +0300
>> @@ -1,7 +1,7 @@
>> CONFIG_64BIT=y
>> @@ -133,7 +133,7 @@
>> CONFIG_UID16=y
>> CONFIG_SYSCTL_SYSCALL=y
>> CONFIG_KALLSYMS=y
>> +CONFIG_KALLSYMS_ALL=y
>> CONFIG_HOTPLUG=y
>> CONFIG_PRINTK=y
>> @@ -4367,15 +4367,19 @@
>> +CONFIG_DEBUG_RT_MUTEXES=y
>> +CONFIG_DEBUG_PI_LIST=y
>> +CONFIG_DEBUG_SPINLOCK=y
>> +CONFIG_DEBUG_MUTEXES=y
>> +CONFIG_DEBUG_LOCK_ALLOC=y
>> +CONFIG_PROVE_LOCKING=y
>> +CONFIG_LOCKDEP=y
>> +CONFIG_LOCK_STAT=y
>> +CONFIG_TRACE_IRQFLAGS=y
>> +CONFIG_DEBUG_SPINLOCK_SLEEP=y
>> +CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
>> CONFIG_STACKTRACE=y
>> CONFIG_DEBUG_BUGVERBOSE=y
>> @@ -4384,15 +4388,15 @@
>> CONFIG_DEBUG_MEMORY_INIT=y
>> +CONFIG_DEBUG_LIST=y
>> +CONFIG_DEBUG_NOTIFIERS=y
>> CONFIG_ARCH_WANT_FRAME_POINTERS=y
>> +CONFIG_FRAME_POINTER=y
>> +CONFIG_RCU_CPU_STALL_DETECTOR=y
>> @@ -4448,7 +4452,7 @@
>> CONFIG_X86_VERBOSE_BOOTUP=y
>> CONFIG_EARLY_PRINTK=y
>> +CONFIG_DEBUG_STACKOVERFLOW=y
>>
>> 2) I then booted the kernel and used "xm sysrq lindi1 t" to get a
>> backtrace from all threads but forgot to run "xm console lindi1". (The
>> bug does not occur if you are connected to the console!).
>>
>> Expected results:
>> 2) domU does not crash
>>
>> Actual results:
>> 2) domU prints a lot of backtraces and then panics.
>>
>> More info:
>> 1) I was able to capture the panic message by running
>>
>> sudo gdbserver-xen 127.0.0.1:9999 --attach 4
>> sudo gdb /local/lindi/vmlinux-2.6.32-5-amd64
>> dump memory dump.txt __log_buf (__log_buf + 4096*100)
>>
>> on dom0:
>>
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 00000000ffffffff 0 1440 1 0x00000080
>> <4>[ 71.816059] ffff88007d483c68 0000000000000246 ffff88007d483c38 ffff88007d483c34
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d483fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e554ec0 ffff88007ff52760
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000031 0 1441 1 0x00000080
>> <4>[ 71.816059] ffff88007d485c68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d485fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e552760 ffff88007ddf3b10
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000000 0 1442 1 0x00000080
>> <4>[ 71.816059] ffff88007d487c68 0000000000000246 0000000000000000 ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d487fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e556270 ffff88007ff54ec0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000033 0 1443 1 0x00000080
>> <4>[ 71.816059] ffff88007d491c68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d491fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d488000 ffff88007d4893b0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000034 0 1444 1 0x00000080
>> <4>[ 71.816059] ffff88007d493c68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d493fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4893b0 ffff88007d4a6270
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000035 0 1445 1 0x00000080
>> <4>[ 71.816059] ffff88007d495c68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d495fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48a760 ffff88007d4a6270
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000036 0 1446 1 0x00000080
>> <4>[ 71.816059] ffff88007d499c68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d499fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48bb10 ffff88007db493b0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000037 0 1447 1 0x00000080
>> <4>[ 71.816059] ffff88007d49bc68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49bfd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48cec0 ffff88007d48e270
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000038 0 1448 1 0x00000080
>> <4>[ 71.816059] ffff88007d49dc68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49dfd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48e270 ffff88007ff54ec0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000039 0 1449 1 0x00000080
>> <4>[ 71.816059] ffff88007d49fc68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49ffd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a0000 ffff88007ff54ec0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 000000000000003a 0 1450 1 0x00000080
>> <4>[ 71.816059] ffff88007d4a9c68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4a9fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a13b0 ffff88007ff54ec0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000000 0 1451 1 0x00000080
>> <4>[ 71.816059] ffff88007d4abc68 0000000000000246 0000000000000000 ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4abfd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a2760 ffff88007ff52760
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 000000000000003c 0 1452 1 0x00000080
>> <4>[ 71.816059] ffff88007d4adc68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4adfd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a3b10 ffff88007ff54ec0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 000000000000003d 0 1453 1 0x00000080
>> <4>[ 71.816059] ffff88007d4afc68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4affd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a4ec0 ffff88007ff54ec0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 000000000000003e 0 1454 1 0x00000080
>> <4>[ 71.816059] ffff88007d4b9c68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4b9fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a6270 ffff88007ff54ec0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000000 0 1461 1 0x00000080
>> <4>[ 71.816059] ffff88007ebb9cc8 0000000000000246 0000000000000000 0000000000000000
>> <4>[ 71.816059] 0000000000000293 00000000000002b7 000000000000fa40 ffff88007ebb9fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7bb10 ffffffff8151be20
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff81339494>] ? __mutex_unlock_slowpath+0x132/0x13b
>> <4>[ 71.816059] [<ffffffff8110a7c3>] pipe_wait+0x5d/0x79
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81339a3b>] ? mutex_lock_nested+0x37/0x3c
>> <4>[ 71.816059] [<ffffffff8110b039>] pipe_read+0x361/0x3dc
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff81103685>] do_sync_read+0xe7/0x12d
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8116dba3>] ? security_file_permission+0x11/0x13
>> <4>[ 71.816059] [<ffffffff8110411d>] vfs_read+0xa8/0x102
>> <4>[ 71.816059] [<ffffffff8110423b>] sys_read+0x47/0x70
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] exim4 S 0000000000000000 0 1462 1 0x00000080
>> <4>[ 71.816059] ffff88007da7f918 0000000000000246 0000000000000000 0000000000000246
>> <4>[ 71.816059] 00000000000002d2 ffffffff81afd7e0 000000000000fa40 ffff88007da7ffd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d33a760 ffffffff8151be20
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
>> <4>[ 71.816059] [<ffffffff8106a41c>] ? add_wait_queue+0x3c/0x45
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8111285b>] ? __pollwait+0xc6/0xcf
>> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
>> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
>> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
>> <4>[ 71.816059] [<ffffffff810796fa>] ? register_lock_class+0x20/0x351
>> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
>> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8133a94c>] ? _spin_unlock_irq+0x2b/0x36
>> <4>[ 71.816059] [<ffffffff8107a98c>] ? trace_hardirqs_on_caller+0x110/0x134
>> <4>[ 71.816059] [<ffffffff8107a9bd>] ? trace_hardirqs_on+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8133a953>] ? _spin_unlock_irq+0x32/0x36
>> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
>> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] getty S 0000000000000000 0 1490 1 0x00000080
>> <4>[ 71.816059] ffff88007da5dd08 0000000000000246 0000000000000000 0000000000000019
>> <4>[ 71.816059] ffff88007da5dc88 ffffffff8107a4b3 000000000000fa40 ffff88007da5dfd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7e270 ffffffff8151be20
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff81339060>] schedule_timeout+0x31/0xef
>> <4>[ 71.816059] [<ffffffff8105e704>] ? del_timer_sync+0x8e/0x9e
>> <4>[ 71.816059] [<ffffffff8105e676>] ? del_timer_sync+0x0/0x9e
>> <4>[ 71.816059] [<ffffffff810670c4>] ? flush_delayed_work+0x50/0x55
>> <4>[ 71.816059] [<ffffffff8122c57c>] n_tty_read+0x50b/0x7e1
>> <4>[ 71.816059] [<ffffffff8104cf3f>] ? default_wake_function+0x0/0xf
>> <4>[ 71.816059] [<ffffffff81227fa1>] tty_read+0x7d/0xba
>> <4>[ 71.816059] [<ffffffff8110411d>] vfs_read+0xa8/0x102
>> <4>[ 71.816059] [<ffffffff8110423b>] sys_read+0x47/0x70
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] sshd S 0000000000000000 0 1491 1211 0x00000080
>> <4>[ 71.816059] ffff88007d2fd918 0000000000000246 ffff8800079c0640 ffff8800079c0640
>> <4>[ 71.816059] ffff88007d2fd938 ffffffff81066ee2 000000000000fa40 ffff88007d2fdfd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7cec0 ffff88007d4c2760
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff81066ee2>] ? flush_work+0xcd/0xf2
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff8105e668>] ? try_to_del_timer_sync+0x66/0x74
>> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
>> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
>> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
>> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
>> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
>> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
>> <4>[ 71.816059] [<ffffffff81103558>] ? do_sync_write+0xe7/0x12d
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
>> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] bash S 0000000000000000 0 1492 1491 0x00000080
>> <4>[ 71.816059] ffff88007d6d7e68 0000000000000246 0000000000000000 00000000000001f0
>> <4>[ 71.816059] ffffffff8100de75 ffff88007d6d7e48 000000000000fa40 ffff88007d6d7fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dde2760 ffff88007ff54ec0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81053ef5>] do_wait+0x1b9/0x227
>> <4>[ 71.816059] [<ffffffff81054009>] sys_wait4+0xa6/0xc3
>> <4>[ 71.816059] [<ffffffff81052711>] ? child_wait_callback+0x0/0x53
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] screen S 0000000000000000 0 1508 1492 0x00000080
>> <4>[ 71.816059] ffff88007ebd1f68 0000000000000246 0000000000000000 ffffffff8107a98c
>> <4>[ 71.816059] ffff88007ddfd6c8 ffff88007ebd1f48 000000000000fa40 ffff88007ebd1fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e526270 ffffffff8151be20
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8107a98c>] ? trace_hardirqs_on_caller+0x110/0x134
>> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
>> <4>[ 71.816059] [<ffffffff8105f565>] sys_pause+0x19/0x22
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] screen S 0000000000000000 0 1509 1508 0x00000080
>> <4>[ 71.816059] ffff88007d80b918 0000000000000246 0000000000000000 ffff8800079c0640
>> <4>[ 71.816059] ffff88007d80b938 ffffffff81066ee2 000000000000fa40 ffff88007d80bfd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dde0000 ffffffff8151be20
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff81066ee2>] ? flush_work+0xcd/0xf2
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8105e668>] ? try_to_del_timer_sync+0x66/0x74
>> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
>> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
>> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
>> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
>> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
>> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
>> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] bash S 0000000000000000 0 1510 1509 0x00000080
>> <4>[ 71.816059] ffff88007da75e68 0000000000000246 0000000000000000 00000000000001f0
>> <4>[ 71.816059] ffffffff8100de75 ffff88007da75e48 000000000000fa40 ffff88007da75fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4c4ec0 ffffffff8151be20
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81053ef5>] do_wait+0x1b9/0x227
>> <4>[ 71.816059] [<ffffffff81054009>] sys_wait4+0xa6/0xc3
>> <4>[ 71.816059] [<ffffffff81052711>] ? child_wait_callback+0x0/0x53
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] sudo S 0000000000000000 0 1531 1510 0x00000080
>> <4>[ 71.816059] ffff88007d62f918 0000000000000246 0000000000000000 ffff88007d379900
>> <4>[ 71.816059] 0000000000000246 00000010739b297e 000000000000fa40 ffff88007d62ffd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7a760 ffff88007ff53b10
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8111285b>] ? __pollwait+0xc6/0xcf
>> <4>[ 71.816059] [<ffffffff812eb9ba>] ? unix_peer_get+0x19/0x3d
>> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
>> <4>[ 71.816059] [<ffffffff8133a9d2>] ? _spin_unlock+0x26/0x2a
>> <4>[ 71.816059] [<ffffffff812eb9d6>] ? unix_peer_get+0x35/0x3d
>> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
>> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
>> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
>> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
>> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
>> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
>> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] sh R running task 0 1532 1531 0x00000088
>> <4>[ 71.816059] ffff88007e56b838 ffff880007b9c580 ffff880007b9c590 0000000000000000
>> <4>[ 71.816059] 0000000000000005 0000000affffffff ffff88007e56b868 ffffffff811b2c0d
>> <4>[ 71.816059] ffff88007e56b918 ffffffff811b2de7 3178302b68746100 000062310000000a
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff811b2c0d>] ? put_dec+0x24/0xca
>> <4>[ 71.816059] [<ffffffff811b2de7>] ? number+0x134/0x246
>> <4>[ 71.816059] [<ffffffff8133d7e1>] ? __atomic_notifier_call_chain+0x78/0x87
>> <4>[ 71.816059] [<ffffffff8133a9d2>] ? _spin_unlock+0x26/0x2a
>> <4>[ 71.816059] [<ffffffff81237e24>] ? vt_console_print+0x28f/0x29e
>> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
>> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
>> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff81051a26>] ? vprintk+0x396/0x3d8
>> <4>[ 71.816059] [<ffffffff81237e24>] ? vt_console_print+0x28f/0x29e
>> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff81338252>] ? printk+0x67/0x6d
>> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff81051a26>] ? vprintk+0x396/0x3d8
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81085361>] ? __module_text_address+0xd/0x5b
>> <4>[ 71.816059] [<ffffffff810853b8>] ? is_module_text_address+0x9/0x11
>> <4>[ 71.816059] [<ffffffff81068128>] ? __kernel_text_address+0x65/0x6b
>> <4>[ 71.816059] [<ffffffff81068128>] ? __kernel_text_address+0x65/0x6b
>> <4>[ 71.816059] [<ffffffff81014828>] ? print_context_stack+0xa2/0xba
>> <4>[ 71.816059] [<ffffffff81013d3f>] ? dump_trace+0x260/0x271
>> <4>[ 71.816059] [<ffffffff810146df>] show_trace_log_lvl+0x4c/0x58
>> <4>[ 71.816059] [<ffffffff810138c7>] show_stack_log_lvl+0xff/0x10e
>> <4>[ 71.816059] [<ffffffff81014691>] show_stack+0x17/0x19
>> <4>[ 71.816059] [<ffffffff8104a227>] sched_show_task+0xae/0xb3
>> <4>[ 71.816059] [<ffffffff8104a27c>] show_state_filter+0x50/0xa9
>> <4>[ 71.816059] [<ffffffff8123d658>] sysrq_handle_showstate+0xb/0xd
>> <4>[ 71.816059] [<ffffffff8123d52f>] __handle_sysrq+0xbb/0x158
>> <4>[ 71.816059] [<ffffffff8123d5cc>] ? write_sysrq_trigger+0x0/0x44
>> <4>[ 71.816059] [<ffffffff8123d605>] write_sysrq_trigger+0x39/0x44
>> <4>[ 71.816059] [<ffffffff81149aad>] proc_reg_write+0x84/0xa1
>> <4>[ 71.816059] [<ffffffff81103f2f>] vfs_write+0xab/0x105
>> <4>[ 71.816059] [<ffffffff8110404d>] sys_write+0x47/0x6f
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <4>[ 71.816059] Sched Debug Version: v0.09, 2.6.32-5+lindi.1-amd64 #1
>> <4>[ 71.816059] now at 85287.968302 msecs
>> <4>[ 71.816059] .jiffies : 4294912466
>> <4>[ 71.816059] .sysctl_sched_latency : 15.000000
>> <4>[ 71.816059] .sysctl_sched_min_granularity : 3.000000
>> <4>[ 71.816059] .sysctl_sched_wakeup_granularity : 3.000000
>> <4>[ 71.816059] .sysctl_sched_child_runs_first : 0.000000
>> <4>[ 71.816059] .sysctl_sched_features : 15834235
>> <4>[ 71.816059]
>> <4>[ 71.816059] cpu#0, 2394.000 MHz
>> <4>[ 71.816059] .nr_running : 1
>> <4>[ 71.816059] .load : 1024
>> <4>[ 71.816059] .nr_switches : 11454
>> <4>[ 71.816059] .nr_load_updates : 5618
>> <4>[ 71.816059] .nr_uninterruptible : 1
>> <4>[ 71.816059] .next_balance : 4294.910575
>> <4>[ 71.816059] .curr->pid : 1046
>> <4>[ 71.816059] .clock : 75789.077421
>> <4>[ 71.816059] .cpu_load[0] : 0
>> <4>[ 71.816059] .cpu_load[1] : 0
>> <4>[ 71.816059] .cpu_load[2] : 0
>> <4>[ 71.816059] .cpu_load[3] : 0
>> <4>[ 71.816059] .cpu_load[4] : 0
>> <4>[ 71.816059]
>> <4>[ 71.816059] cfs_rq[0]:/
>> <4>[ 71.816059] .exec_clock : 0.000000
>> <4>[ 71.816059] .MIN_vruntime : 0.000001
>> <4>[ 71.816059] .min_vruntime : 9426.044216
>> <4>[ 71.816059] .max_vruntime : 0.000001
>> <4>[ 71.816059] .spread : 0.000000
>> <4>[ 71.816059] .spread0 : 0.000000
>> <4>[ 71.816059] .nr_running : 1
>> <4>[ 71.816059] .load : 1024
>> <4>[ 71.816059] .nr_spread_over : 0
>> <4>[ 71.816059] .shares : 0
>> <4>[ 71.816059]
>> <4>[ 71.816059] rt_rq[0]:
>> <4>[ 71.816059] .rt_nr_running : 0
>> <4>[ 71.816059] .rt_throttled : 0
>> <4>[ 71.816059] .rt_time : 0.000000
>> <4>[ 71.816059] .rt_runtime : 950.000000
>> <4>[ 71.816059]
>> <4>[ 71.816059] runnable tasks:
>> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
>> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
>> <4>[ 71.816059] R nsca 1046 9426.044216 51 120 0 0 0.000000 0.000000 0.000000 /
>> <4>[ 71.816059]
>> <4>[ 71.816059] cpu#1, 2394.000 MHz
>> <4>[ 71.816059] .nr_running : 1
>> <4>[ 71.816059] .load : 1024
>> <4>[ 71.816059] .nr_switches : 8912
>> <4>[ 71.816059] .nr_load_updates : 5067
>> <4>[ 71.816059] .nr_uninterruptible : 17
>> <4>[ 71.816059] .next_balance : 4294.910025
>> <4>[ 71.816059] .curr->pid : 1532
>> <4>[ 71.816059] .clock : 71816.059281
>> <4>[ 71.816059] .cpu_load[0] : 1024
>> <4>[ 71.816059] .cpu_load[1] : 512
>> <4>[ 71.816059] .cpu_load[2] : 256
>> <4>[ 71.816059] .cpu_load[3] : 128
>> <4>[ 71.816059] .cpu_load[4] : 64
>> <4>[ 71.816059]
>> <4>[ 71.816059] cfs_rq[1]:/
>> <4>[ 71.816059] .exec_clock : 0.000000
>> <4>[ 71.816059] .MIN_vruntime : 0.000001
>> <4>[ 71.816059] .min_vruntime : 12008.498567
>> <4>[ 71.816059] .max_vruntime : 0.000001
>> <4>[ 71.816059] .spread : 0.000000
>> <4>[ 71.816059] .spread0 : 2582.454351
>> <4>[ 71.816059] .nr_running : 1
>> <4>[ 71.816059] .load : 1024
>> <4>[ 71.816059] .nr_spread_over : 0
>> <4>[ 71.816059] .shares : 0
>> <4>[ 71.816059]
>> <4>[ 71.816059] rt_rq[1]:
>> <4>[ 71.816059] .rt_nr_running : 0
>> <4>[ 71.816059] .rt_throttled : 0
>> <4>[ 71.816059] .rt_time : 0.000000
>> <4>[ 71.816059] .rt_runtime : 950.000000
>> <4>[ 71.816059]
>> <4>[ 71.816059] runnable tasks:
>> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
>> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
>> <4>[ 71.816059] R sh 1532 12008.498567 3 120 0 0 0.000000 0.000000 0.000000 /
>> <4>[ 71.816059]
>> <4>[ 71.816059] cpu#2, 2394.000 MHz
>> <4>[ 71.816059] .nr_running : 0
>> <4>[ 71.816059] .load : 0
>> <4>[ 71.816059] .nr_switches : 5200
>> <4>[ 71.816059] .nr_load_updates : 6701
>> <4>[ 71.816059] .nr_uninterruptible : -8
>> <4>[ 71.816059] .next_balance : 4294.912466
>> <4>[ 71.816059] .curr->pid : 0
>> <4>[ 71.816059] .clock : 80673.012224
>> <4>[ 71.816059] .cpu_load[0] : 0
>> <4>[ 71.816059] .cpu_load[1] : 0
>> <4>[ 71.816059] .cpu_load[2] : 0
>> <4>[ 71.816059] .cpu_load[3] : 0
>> <4>[ 71.816059] .cpu_load[4] : 0
>> <4>[ 71.816059]
>> <4>[ 71.816059] cfs_rq[2]:/
>> <4>[ 71.816059] .exec_clock : 0.000000
>> <4>[ 71.816059] .MIN_vruntime : 0.000001
>> <4>[ 71.816059] .min_vruntime : 7154.290231
>> <4>[ 71.816059] .max_vruntime : 0.000001
>> <4>[ 71.816059] .spread : 0.000000
>> <4>[ 71.816059] .spread0 : -2271.753985
>> <4>[ 71.816059] .nr_running : 0
>> <4>[ 71.816059] .load : 0
>> <4>[ 71.816059] .nr_spread_over : 0
>> <4>[ 71.816059] .shares : 0
>> <4>[ 71.816059]
>> <4>[ 71.816059] rt_rq[2]:
>> <4>[ 71.816059] .rt_nr_running : 0
>> <4>[ 71.816059] .rt_throttled : 0
>> <4>[ 71.816059] .rt_time : 0.000000
>> <4>[ 71.816059] .rt_runtime : 950.000000
>> <4>[ 71.816059]
>> <4>[ 71.816059] runnable tasks:
>> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
>> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
>> <4>[ 71.816059]
>> <4>[ 71.816059] cpu#3, 2394.000 MHz
>> <4>[ 71.816059] .nr_running : 0
>> <4>[ 71.816059] .load : 0
>> <4>[ 71.816059] .nr_switches : 4398
>> <4>[ 71.816059] .nr_load_updates : 6055
>> <4>[ 71.816059] .nr_uninterruptible : -9
>> <4>[ 71.816059] .next_balance : 4294.912466
>> <4>[ 71.816059] .curr->pid : 0
>> <4>[ 71.816059] .clock : 80673.512781
>> <4>[ 71.816059] .cpu_load[0] : 0
>> <4>[ 71.816059] .cpu_load[1] : 0
>> <4>[ 71.816059] .cpu_load[2] : 0
>> <4>[ 71.816059] .cpu_load[3] : 0
>> <4>[ 71.816059] .cpu_load[4] : 0
>> <4>[ 71.816059]
>> <4>[ 71.816059] cfs_rq[3]:/
>> <4>[ 71.816059] .exec_clock : 0.000000
>> <4>[ 71.816059] .MIN_vruntime : 0.000001
>> <4>[ 71.816059] .min_vruntime : 5646.686691
>> <4>[ 71.816059] .max_vruntime : 0.000001
>> <4>[ 71.816059] .spread : 0.000000
>> <4>[ 71.816059] .spread0 : -3779.357525
>> <4>[ 71.816059] .nr_running : 0
>> <4>[ 71.816059] .load : 0
>> <4>[ 71.816059] .nr_spread_over : 0
>> <4>[ 71.816059] .shares : 0
>> <4>[ 71.816059]
>> <4>[ 71.816059] rt_rq[3]:
>> <4>[ 71.816059] .rt_nr_running : 0
>> <4>[ 71.816059] .rt_throttled : 0
>> <4>[ 71.816059] .rt_time : 0.000000
>> <4>[ 71.816059] .rt_runtime : 950.000000
>> <4>[ 71.816059]
>> <4>[ 71.816059] runnable tasks:
>> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
>> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
>> <4>[ 71.816059]
>>
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000004 0 1396 1 0x00000080
>> <4>[ 71.816059] ffff88007ddcfc68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007ddcffd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff880003560000 ffff88007dad4ec0
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000005 0 1397 1 0x00000080
>> <4>[ 71.816059] ffff88007e387c68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007e387fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d0d0000 ffff8800034ca760
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 00000000ffffffff 0 1398 1 0x00000080
>> <4>[ 71.816059] ffff88007e50dc68 0000000000000246 ffff88007e50dc38 ffff88007e50dc34
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007e50dfd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d0d13b0 ffff88007ff52760
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
>> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
>> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
>> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
>> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
>> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
>> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
>> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
>> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
>> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
>> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
>> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
>> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
>> <6>[ 71.816059] console-kit-d S 0000000000000007 0 1399 1 0x00000080
>> <4>[ 71.816059] ffff88007ddd1c68 0000000000000246 00000000000003da ffffffff81b1d760
>> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007ddd1fd8
>> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dad4ec0 ffff88007d0d0000
>> <4>[ 71.816059] Call Trace:
>> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
>> <4>[ 71.816059] [<ffffffff8133a9a
 
Old 09-24-2010, 05:11 AM
Timo Juhani Lindfors
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

Ian Campbell <ijc@hellion.org.uk> writes:
> I guess the right thing to do would be to using some existing paravirt
> interface to send the IPI instead of tweaking the APIC directly, or if

Is

void xen_send_IPI_one(unsigned int cpu, enum ipi_vector vector);

in xen/events.h perhaps already this?





--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 8462xv20at.fsf@sauna.l.org">http://lists.debian.org/8462xv20at.fsf@sauna.l.org
 
Old 09-24-2010, 08:07 AM
Ian Campbell
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

On Thu, 2010-09-23 at 14:54 -0700, Jeremy Fitzhardinge wrote:
> On 09/23/2010 02:06 PM, Ian Campbell wrote:
> > Looks like SysRQ-t somehow causes an RCU stall, which may be a bug in
> > its own right (or maybe SysRQ-t is just slow?), but the eventual BUG is
> > because the RCU call causes arch_trigger_all_cpu_backtrace to be called
> > which, on x86, accesses the APIC directly to send an IPI to every CPU
> > which won't work under Xen.
> >
> > I guess the right thing to do would be to using some existing paravirt
> > interface to send the IPI instead of tweaking the APIC directly, or if
> > necessary to add such an interface, Jeremy, what do you think? Is there
> > an existing interface we could use?
>
> It looks like a regular cross-cpu call to me. Don't know why it needs
> special IPI magic. OIC, its an NMI. Well, we'd need to add something
> to replicate that as-is (can you send an NMI in Xen?).

NMI injection on 2.6.18 was one of the first Xen things I worked on (for
injecting h/w NMI button faults) so something worked once upon a time. I
can see CALLBACKTYPE_nmi and VCPUOP_send_nmi which seems promising that
it's still around (and extended since I didn't do VCPUOP_send_nmi), even
if its bit-rotted or not made it to pvops yet.

In 2.6.18-xen the guest side of receiving an NMI was pretty ugly IIRC
since we needed to force a return via the iret hypercall (to allow the
hypervisor to simulate the NMI interrupt window).

There is a small amount of residual support even in pvops
(XEN_EFLAGS_NMI is defined, and tested once on 32 bit, but never set).
It might be possible to do something nicer with all the pvops
infrastructure, I'm also not sure it isn't a premature optimisation to
save a compare by putting the NMI flag into a reserved bit of the saved
EFLAGS anyway.

Ian.

>
> J
>
>
> > Timo, (this may not be as useful as I first though since I didn't twig
> > that you were using sysrq-t not sysrq-l, which causes the
> > arch_trigger_all_cpu_backtrace issue directly, until just now, I leave
> > it in since I've already typed it)... you may find the xenctx tool
> > useful as a workaround you can run it in domain 0 (usually it is
> > installed in /usr/lib/xen*/bin/xenctx) giving it the guest's System.map,
> > a domid and a vcpu id and it will give you a stack trace for that VCPU.
> > e.g. domain 5 vcpu 0:
> > # xenctx -s /boot/System.map-2.6.32.21-x86_32p-xenU-00969-gb297cda 5 0
> > cs:eip: 0061:c10013a7 _stext+0x3a7
> > flags: 00001246 i z p
> > ss:esp: 0069:c12d3f8c
> > eax: 00000000 ebx: deadbeef ecx: deadbeef edx: 00000004
> > esi: c12f15f0 edi: c12db074 ebp: c12d3f9c
> > ds: 007b es: 007b fs: 00d8 gs: 0000
> > Code (instr addr c10013a7)
> > cc cc cc cc cc cc cc cc cc cc cc cc cc cc b8 1d 00 00 00 cd 82 <c3> cc cc cc cc cc cc cc cc cc cc
> >
> >
> > Stack:
> > c1006528 00000000 c12f15f0 c12db074 c12d3fa4 c10034a9 c12d3fb4 c1007cae
> > c1319d88 c13191f0 c12d3fbc c11e71cf c12d3fcc c12f384c c1319d88 40000000
> > c12d3fd8 c12f30a2 40000000 c12d3ffc c12f6445 1f898975 80000201 01040800
> > 000006fb 00000000 c19ca000 00000000 00000000
> >
> > Call Trace:
> > [<c10013a7>] _stext+0x3a7 <--
> > [<c1006528>] xen_safe_halt+0x12
> > [<c10034a9>] xen_idle+0x5d
> > [<c1007cae>] cpu_idle+0x5b
> > [<c11e71cf>] rest_init+0x53
> > [<c12f384c>] start_kernel+0x309
> > [<c12f30a2>] i386_start_kernel+0x91
> > [<c12f6445>] xen_start_kernel+0x524
> >
> >
> > On Thu, 2010-09-23 at 14:50 +0300, Timo Juhani Lindfors wrote:
> >> Package: linux-2.6
> >> Version: 2.6.32-23
> >> Severity: minor
> >>
> >> [ Note that this bug is against a kernel that has been modified to add
> >> debugging options. I report it to debian only because I was suggested
> >> so on #debian-kernel.]
> >>
> >> Steps to reproduce:
> >> 1) To debug #580889 I rebuilt the kernel with various debug options:
> >>
> >> $ diff -u /boot/config-2.6.32-5-amd64 /boot/config-2.6.32-5+lindi.1-amd64 |grep -v "#"
> >> --- /boot/config-2.6.32-5-amd64 2010-09-18 01:11:39.000000000 +0300
> >> +++ /boot/config-2.6.32-5+lindi.1-amd64 2010-09-22 20:08:55.000000000 +0300
> >> @@ -1,7 +1,7 @@
> >> CONFIG_64BIT=y
> >> @@ -133,7 +133,7 @@
> >> CONFIG_UID16=y
> >> CONFIG_SYSCTL_SYSCALL=y
> >> CONFIG_KALLSYMS=y
> >> +CONFIG_KALLSYMS_ALL=y
> >> CONFIG_HOTPLUG=y
> >> CONFIG_PRINTK=y
> >> @@ -4367,15 +4367,19 @@
> >> +CONFIG_DEBUG_RT_MUTEXES=y
> >> +CONFIG_DEBUG_PI_LIST=y
> >> +CONFIG_DEBUG_SPINLOCK=y
> >> +CONFIG_DEBUG_MUTEXES=y
> >> +CONFIG_DEBUG_LOCK_ALLOC=y
> >> +CONFIG_PROVE_LOCKING=y
> >> +CONFIG_LOCKDEP=y
> >> +CONFIG_LOCK_STAT=y
> >> +CONFIG_TRACE_IRQFLAGS=y
> >> +CONFIG_DEBUG_SPINLOCK_SLEEP=y
> >> +CONFIG_DEBUG_LOCKING_API_SELFTESTS=y
> >> CONFIG_STACKTRACE=y
> >> CONFIG_DEBUG_BUGVERBOSE=y
> >> @@ -4384,15 +4388,15 @@
> >> CONFIG_DEBUG_MEMORY_INIT=y
> >> +CONFIG_DEBUG_LIST=y
> >> +CONFIG_DEBUG_NOTIFIERS=y
> >> CONFIG_ARCH_WANT_FRAME_POINTERS=y
> >> +CONFIG_FRAME_POINTER=y
> >> +CONFIG_RCU_CPU_STALL_DETECTOR=y
> >> @@ -4448,7 +4452,7 @@
> >> CONFIG_X86_VERBOSE_BOOTUP=y
> >> CONFIG_EARLY_PRINTK=y
> >> +CONFIG_DEBUG_STACKOVERFLOW=y
> >>
> >> 2) I then booted the kernel and used "xm sysrq lindi1 t" to get a
> >> backtrace from all threads but forgot to run "xm console lindi1". (The
> >> bug does not occur if you are connected to the console!).
> >>
> >> Expected results:
> >> 2) domU does not crash
> >>
> >> Actual results:
> >> 2) domU prints a lot of backtraces and then panics.
> >>
> >> More info:
> >> 1) I was able to capture the panic message by running
> >>
> >> sudo gdbserver-xen 127.0.0.1:9999 --attach 4
> >> sudo gdb /local/lindi/vmlinux-2.6.32-5-amd64
> >> dump memory dump.txt __log_buf (__log_buf + 4096*100)
> >>
> >> on dom0:
> >>
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 00000000ffffffff 0 1440 1 0x00000080
> >> <4>[ 71.816059] ffff88007d483c68 0000000000000246 ffff88007d483c38 ffff88007d483c34
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d483fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e554ec0 ffff88007ff52760
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000031 0 1441 1 0x00000080
> >> <4>[ 71.816059] ffff88007d485c68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d485fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e552760 ffff88007ddf3b10
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000000 0 1442 1 0x00000080
> >> <4>[ 71.816059] ffff88007d487c68 0000000000000246 0000000000000000 ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d487fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e556270 ffff88007ff54ec0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000033 0 1443 1 0x00000080
> >> <4>[ 71.816059] ffff88007d491c68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d491fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d488000 ffff88007d4893b0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000034 0 1444 1 0x00000080
> >> <4>[ 71.816059] ffff88007d493c68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d493fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4893b0 ffff88007d4a6270
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000035 0 1445 1 0x00000080
> >> <4>[ 71.816059] ffff88007d495c68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d495fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48a760 ffff88007d4a6270
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000036 0 1446 1 0x00000080
> >> <4>[ 71.816059] ffff88007d499c68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d499fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48bb10 ffff88007db493b0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000037 0 1447 1 0x00000080
> >> <4>[ 71.816059] ffff88007d49bc68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49bfd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48cec0 ffff88007d48e270
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000038 0 1448 1 0x00000080
> >> <4>[ 71.816059] ffff88007d49dc68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49dfd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d48e270 ffff88007ff54ec0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000039 0 1449 1 0x00000080
> >> <4>[ 71.816059] ffff88007d49fc68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d49ffd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a0000 ffff88007ff54ec0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 000000000000003a 0 1450 1 0x00000080
> >> <4>[ 71.816059] ffff88007d4a9c68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4a9fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a13b0 ffff88007ff54ec0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000000 0 1451 1 0x00000080
> >> <4>[ 71.816059] ffff88007d4abc68 0000000000000246 0000000000000000 ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4abfd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a2760 ffff88007ff52760
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 000000000000003c 0 1452 1 0x00000080
> >> <4>[ 71.816059] ffff88007d4adc68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4adfd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a3b10 ffff88007ff54ec0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 000000000000003d 0 1453 1 0x00000080
> >> <4>[ 71.816059] ffff88007d4afc68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4affd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a4ec0 ffff88007ff54ec0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 000000000000003e 0 1454 1 0x00000080
> >> <4>[ 71.816059] ffff88007d4b9c68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007d4b9fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4a6270 ffff88007ff54ec0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000000 0 1461 1 0x00000080
> >> <4>[ 71.816059] ffff88007ebb9cc8 0000000000000246 0000000000000000 0000000000000000
> >> <4>[ 71.816059] 0000000000000293 00000000000002b7 000000000000fa40 ffff88007ebb9fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7bb10 ffffffff8151be20
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff81339494>] ? __mutex_unlock_slowpath+0x132/0x13b
> >> <4>[ 71.816059] [<ffffffff8110a7c3>] pipe_wait+0x5d/0x79
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81339a3b>] ? mutex_lock_nested+0x37/0x3c
> >> <4>[ 71.816059] [<ffffffff8110b039>] pipe_read+0x361/0x3dc
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff81103685>] do_sync_read+0xe7/0x12d
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8116dba3>] ? security_file_permission+0x11/0x13
> >> <4>[ 71.816059] [<ffffffff8110411d>] vfs_read+0xa8/0x102
> >> <4>[ 71.816059] [<ffffffff8110423b>] sys_read+0x47/0x70
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] exim4 S 0000000000000000 0 1462 1 0x00000080
> >> <4>[ 71.816059] ffff88007da7f918 0000000000000246 0000000000000000 0000000000000246
> >> <4>[ 71.816059] 00000000000002d2 ffffffff81afd7e0 000000000000fa40 ffff88007da7ffd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d33a760 ffffffff8151be20
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
> >> <4>[ 71.816059] [<ffffffff8106a41c>] ? add_wait_queue+0x3c/0x45
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8111285b>] ? __pollwait+0xc6/0xcf
> >> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
> >> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
> >> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
> >> <4>[ 71.816059] [<ffffffff810796fa>] ? register_lock_class+0x20/0x351
> >> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
> >> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8133a94c>] ? _spin_unlock_irq+0x2b/0x36
> >> <4>[ 71.816059] [<ffffffff8107a98c>] ? trace_hardirqs_on_caller+0x110/0x134
> >> <4>[ 71.816059] [<ffffffff8107a9bd>] ? trace_hardirqs_on+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8133a953>] ? _spin_unlock_irq+0x32/0x36
> >> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
> >> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] getty S 0000000000000000 0 1490 1 0x00000080
> >> <4>[ 71.816059] ffff88007da5dd08 0000000000000246 0000000000000000 0000000000000019
> >> <4>[ 71.816059] ffff88007da5dc88 ffffffff8107a4b3 000000000000fa40 ffff88007da5dfd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7e270 ffffffff8151be20
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff81339060>] schedule_timeout+0x31/0xef
> >> <4>[ 71.816059] [<ffffffff8105e704>] ? del_timer_sync+0x8e/0x9e
> >> <4>[ 71.816059] [<ffffffff8105e676>] ? del_timer_sync+0x0/0x9e
> >> <4>[ 71.816059] [<ffffffff810670c4>] ? flush_delayed_work+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff8122c57c>] n_tty_read+0x50b/0x7e1
> >> <4>[ 71.816059] [<ffffffff8104cf3f>] ? default_wake_function+0x0/0xf
> >> <4>[ 71.816059] [<ffffffff81227fa1>] tty_read+0x7d/0xba
> >> <4>[ 71.816059] [<ffffffff8110411d>] vfs_read+0xa8/0x102
> >> <4>[ 71.816059] [<ffffffff8110423b>] sys_read+0x47/0x70
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] sshd S 0000000000000000 0 1491 1211 0x00000080
> >> <4>[ 71.816059] ffff88007d2fd918 0000000000000246 ffff8800079c0640 ffff8800079c0640
> >> <4>[ 71.816059] ffff88007d2fd938 ffffffff81066ee2 000000000000fa40 ffff88007d2fdfd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7cec0 ffff88007d4c2760
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff81066ee2>] ? flush_work+0xcd/0xf2
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff8105e668>] ? try_to_del_timer_sync+0x66/0x74
> >> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
> >> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
> >> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
> >> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
> >> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
> >> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
> >> <4>[ 71.816059] [<ffffffff81103558>] ? do_sync_write+0xe7/0x12d
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
> >> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] bash S 0000000000000000 0 1492 1491 0x00000080
> >> <4>[ 71.816059] ffff88007d6d7e68 0000000000000246 0000000000000000 00000000000001f0
> >> <4>[ 71.816059] ffffffff8100de75 ffff88007d6d7e48 000000000000fa40 ffff88007d6d7fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dde2760 ffff88007ff54ec0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81053ef5>] do_wait+0x1b9/0x227
> >> <4>[ 71.816059] [<ffffffff81054009>] sys_wait4+0xa6/0xc3
> >> <4>[ 71.816059] [<ffffffff81052711>] ? child_wait_callback+0x0/0x53
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] screen S 0000000000000000 0 1508 1492 0x00000080
> >> <4>[ 71.816059] ffff88007ebd1f68 0000000000000246 0000000000000000 ffffffff8107a98c
> >> <4>[ 71.816059] ffff88007ddfd6c8 ffff88007ebd1f48 000000000000fa40 ffff88007ebd1fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007e526270 ffffffff8151be20
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8107a98c>] ? trace_hardirqs_on_caller+0x110/0x134
> >> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> >> <4>[ 71.816059] [<ffffffff8105f565>] sys_pause+0x19/0x22
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] screen S 0000000000000000 0 1509 1508 0x00000080
> >> <4>[ 71.816059] ffff88007d80b918 0000000000000246 0000000000000000 ffff8800079c0640
> >> <4>[ 71.816059] ffff88007d80b938 ffffffff81066ee2 000000000000fa40 ffff88007d80bfd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007dde0000 ffffffff8151be20
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff81066ee2>] ? flush_work+0xcd/0xf2
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8105e668>] ? try_to_del_timer_sync+0x66/0x74
> >> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
> >> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
> >> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
> >> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
> >> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8122efb1>] ? put_ldisc+0xb8/0xbd
> >> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
> >> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] bash S 0000000000000000 0 1510 1509 0x00000080
> >> <4>[ 71.816059] ffff88007da75e68 0000000000000246 0000000000000000 00000000000001f0
> >> <4>[ 71.816059] ffffffff8100de75 ffff88007da75e48 000000000000fa40 ffff88007da75fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d4c4ec0 ffffffff8151be20
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81053ef5>] do_wait+0x1b9/0x227
> >> <4>[ 71.816059] [<ffffffff81054009>] sys_wait4+0xa6/0xc3
> >> <4>[ 71.816059] [<ffffffff81052711>] ? child_wait_callback+0x0/0x53
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] sudo S 0000000000000000 0 1531 1510 0x00000080
> >> <4>[ 71.816059] ffff88007d62f918 0000000000000246 0000000000000000 ffff88007d379900
> >> <4>[ 71.816059] 0000000000000246 00000010739b297e 000000000000fa40 ffff88007d62ffd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007db7a760 ffff88007ff53b10
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8111285b>] ? __pollwait+0xc6/0xcf
> >> <4>[ 71.816059] [<ffffffff812eb9ba>] ? unix_peer_get+0x19/0x3d
> >> <4>[ 71.816059] [<ffffffff81339a8e>] schedule_hrtimeout_range+0x4e/0x11a
> >> <4>[ 71.816059] [<ffffffff8133a9d2>] ? _spin_unlock+0x26/0x2a
> >> <4>[ 71.816059] [<ffffffff812eb9d6>] ? unix_peer_get+0x35/0x3d
> >> <4>[ 71.816059] [<ffffffff811115d9>] poll_schedule_timeout+0x39/0x59
> >> <4>[ 71.816059] [<ffffffff8111229e>] do_select+0x5c1/0x605
> >> <4>[ 71.816059] [<ffffffff81111cdd>] ? do_select+0x0/0x605
> >> <4>[ 71.816059] [<ffffffff81112795>] ? __pollwait+0x0/0xcf
> >> <4>[ 71.816059] [<ffffffff81112864>] ? pollwake+0x0/0x5c
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8111249b>] core_sys_select+0x1b9/0x255
> >> <4>[ 71.816059] [<ffffffff81112323>] ? core_sys_select+0x41/0x255
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff8111276d>] sys_select+0x94/0xbc
> >> <4>[ 71.816059] [<ffffffff810a1697>] ? audit_syscall_entry+0x114/0x140
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] sh R running task 0 1532 1531 0x00000088
> >> <4>[ 71.816059] ffff88007e56b838 ffff880007b9c580 ffff880007b9c590 0000000000000000
> >> <4>[ 71.816059] 0000000000000005 0000000affffffff ffff88007e56b868 ffffffff811b2c0d
> >> <4>[ 71.816059] ffff88007e56b918 ffffffff811b2de7 3178302b68746100 000062310000000a
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff811b2c0d>] ? put_dec+0x24/0xca
> >> <4>[ 71.816059] [<ffffffff811b2de7>] ? number+0x134/0x246
> >> <4>[ 71.816059] [<ffffffff8133d7e1>] ? __atomic_notifier_call_chain+0x78/0x87
> >> <4>[ 71.816059] [<ffffffff8133a9d2>] ? _spin_unlock+0x26/0x2a
> >> <4>[ 71.816059] [<ffffffff81237e24>] ? vt_console_print+0x28f/0x29e
> >> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> >> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> >> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff81051a26>] ? vprintk+0x396/0x3d8
> >> <4>[ 71.816059] [<ffffffff81237e24>] ? vt_console_print+0x28f/0x29e
> >> <4>[ 71.816059] [<ffffffff81237bd8>] ? vt_console_print+0x43/0x29e
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff81338252>] ? printk+0x67/0x6d
> >> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8107924e>] ? trace_hardirqs_off+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff81051a26>] ? vprintk+0x396/0x3d8
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81085361>] ? __module_text_address+0xd/0x5b
> >> <4>[ 71.816059] [<ffffffff810853b8>] ? is_module_text_address+0x9/0x11
> >> <4>[ 71.816059] [<ffffffff81068128>] ? __kernel_text_address+0x65/0x6b
> >> <4>[ 71.816059] [<ffffffff81068128>] ? __kernel_text_address+0x65/0x6b
> >> <4>[ 71.816059] [<ffffffff81014828>] ? print_context_stack+0xa2/0xba
> >> <4>[ 71.816059] [<ffffffff81013d3f>] ? dump_trace+0x260/0x271
> >> <4>[ 71.816059] [<ffffffff810146df>] show_trace_log_lvl+0x4c/0x58
> >> <4>[ 71.816059] [<ffffffff810138c7>] show_stack_log_lvl+0xff/0x10e
> >> <4>[ 71.816059] [<ffffffff81014691>] show_stack+0x17/0x19
> >> <4>[ 71.816059] [<ffffffff8104a227>] sched_show_task+0xae/0xb3
> >> <4>[ 71.816059] [<ffffffff8104a27c>] show_state_filter+0x50/0xa9
> >> <4>[ 71.816059] [<ffffffff8123d658>] sysrq_handle_showstate+0xb/0xd
> >> <4>[ 71.816059] [<ffffffff8123d52f>] __handle_sysrq+0xbb/0x158
> >> <4>[ 71.816059] [<ffffffff8123d5cc>] ? write_sysrq_trigger+0x0/0x44
> >> <4>[ 71.816059] [<ffffffff8123d605>] write_sysrq_trigger+0x39/0x44
> >> <4>[ 71.816059] [<ffffffff81149aad>] proc_reg_write+0x84/0xa1
> >> <4>[ 71.816059] [<ffffffff81103f2f>] vfs_write+0xab/0x105
> >> <4>[ 71.816059] [<ffffffff8110404d>] sys_write+0x47/0x6f
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <4>[ 71.816059] Sched Debug Version: v0.09, 2.6.32-5+lindi.1-amd64 #1
> >> <4>[ 71.816059] now at 85287.968302 msecs
> >> <4>[ 71.816059] .jiffies : 4294912466
> >> <4>[ 71.816059] .sysctl_sched_latency : 15.000000
> >> <4>[ 71.816059] .sysctl_sched_min_granularity : 3.000000
> >> <4>[ 71.816059] .sysctl_sched_wakeup_granularity : 3.000000
> >> <4>[ 71.816059] .sysctl_sched_child_runs_first : 0.000000
> >> <4>[ 71.816059] .sysctl_sched_features : 15834235
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] cpu#0, 2394.000 MHz
> >> <4>[ 71.816059] .nr_running : 1
> >> <4>[ 71.816059] .load : 1024
> >> <4>[ 71.816059] .nr_switches : 11454
> >> <4>[ 71.816059] .nr_load_updates : 5618
> >> <4>[ 71.816059] .nr_uninterruptible : 1
> >> <4>[ 71.816059] .next_balance : 4294.910575
> >> <4>[ 71.816059] .curr->pid : 1046
> >> <4>[ 71.816059] .clock : 75789.077421
> >> <4>[ 71.816059] .cpu_load[0] : 0
> >> <4>[ 71.816059] .cpu_load[1] : 0
> >> <4>[ 71.816059] .cpu_load[2] : 0
> >> <4>[ 71.816059] .cpu_load[3] : 0
> >> <4>[ 71.816059] .cpu_load[4] : 0
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] cfs_rq[0]:/
> >> <4>[ 71.816059] .exec_clock : 0.000000
> >> <4>[ 71.816059] .MIN_vruntime : 0.000001
> >> <4>[ 71.816059] .min_vruntime : 9426.044216
> >> <4>[ 71.816059] .max_vruntime : 0.000001
> >> <4>[ 71.816059] .spread : 0.000000
> >> <4>[ 71.816059] .spread0 : 0.000000
> >> <4>[ 71.816059] .nr_running : 1
> >> <4>[ 71.816059] .load : 1024
> >> <4>[ 71.816059] .nr_spread_over : 0
> >> <4>[ 71.816059] .shares : 0
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] rt_rq[0]:
> >> <4>[ 71.816059] .rt_nr_running : 0
> >> <4>[ 71.816059] .rt_throttled : 0
> >> <4>[ 71.816059] .rt_time : 0.000000
> >> <4>[ 71.816059] .rt_runtime : 950.000000
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] runnable tasks:
> >> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
> >> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
> >> <4>[ 71.816059] R nsca 1046 9426.044216 51 120 0 0 0.000000 0.000000 0.000000 /
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] cpu#1, 2394.000 MHz
> >> <4>[ 71.816059] .nr_running : 1
> >> <4>[ 71.816059] .load : 1024
> >> <4>[ 71.816059] .nr_switches : 8912
> >> <4>[ 71.816059] .nr_load_updates : 5067
> >> <4>[ 71.816059] .nr_uninterruptible : 17
> >> <4>[ 71.816059] .next_balance : 4294.910025
> >> <4>[ 71.816059] .curr->pid : 1532
> >> <4>[ 71.816059] .clock : 71816.059281
> >> <4>[ 71.816059] .cpu_load[0] : 1024
> >> <4>[ 71.816059] .cpu_load[1] : 512
> >> <4>[ 71.816059] .cpu_load[2] : 256
> >> <4>[ 71.816059] .cpu_load[3] : 128
> >> <4>[ 71.816059] .cpu_load[4] : 64
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] cfs_rq[1]:/
> >> <4>[ 71.816059] .exec_clock : 0.000000
> >> <4>[ 71.816059] .MIN_vruntime : 0.000001
> >> <4>[ 71.816059] .min_vruntime : 12008.498567
> >> <4>[ 71.816059] .max_vruntime : 0.000001
> >> <4>[ 71.816059] .spread : 0.000000
> >> <4>[ 71.816059] .spread0 : 2582.454351
> >> <4>[ 71.816059] .nr_running : 1
> >> <4>[ 71.816059] .load : 1024
> >> <4>[ 71.816059] .nr_spread_over : 0
> >> <4>[ 71.816059] .shares : 0
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] rt_rq[1]:
> >> <4>[ 71.816059] .rt_nr_running : 0
> >> <4>[ 71.816059] .rt_throttled : 0
> >> <4>[ 71.816059] .rt_time : 0.000000
> >> <4>[ 71.816059] .rt_runtime : 950.000000
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] runnable tasks:
> >> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
> >> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
> >> <4>[ 71.816059] R sh 1532 12008.498567 3 120 0 0 0.000000 0.000000 0.000000 /
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] cpu#2, 2394.000 MHz
> >> <4>[ 71.816059] .nr_running : 0
> >> <4>[ 71.816059] .load : 0
> >> <4>[ 71.816059] .nr_switches : 5200
> >> <4>[ 71.816059] .nr_load_updates : 6701
> >> <4>[ 71.816059] .nr_uninterruptible : -8
> >> <4>[ 71.816059] .next_balance : 4294.912466
> >> <4>[ 71.816059] .curr->pid : 0
> >> <4>[ 71.816059] .clock : 80673.012224
> >> <4>[ 71.816059] .cpu_load[0] : 0
> >> <4>[ 71.816059] .cpu_load[1] : 0
> >> <4>[ 71.816059] .cpu_load[2] : 0
> >> <4>[ 71.816059] .cpu_load[3] : 0
> >> <4>[ 71.816059] .cpu_load[4] : 0
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] cfs_rq[2]:/
> >> <4>[ 71.816059] .exec_clock : 0.000000
> >> <4>[ 71.816059] .MIN_vruntime : 0.000001
> >> <4>[ 71.816059] .min_vruntime : 7154.290231
> >> <4>[ 71.816059] .max_vruntime : 0.000001
> >> <4>[ 71.816059] .spread : 0.000000
> >> <4>[ 71.816059] .spread0 : -2271.753985
> >> <4>[ 71.816059] .nr_running : 0
> >> <4>[ 71.816059] .load : 0
> >> <4>[ 71.816059] .nr_spread_over : 0
> >> <4>[ 71.816059] .shares : 0
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] rt_rq[2]:
> >> <4>[ 71.816059] .rt_nr_running : 0
> >> <4>[ 71.816059] .rt_throttled : 0
> >> <4>[ 71.816059] .rt_time : 0.000000
> >> <4>[ 71.816059] .rt_runtime : 950.000000
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] runnable tasks:
> >> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
> >> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] cpu#3, 2394.000 MHz
> >> <4>[ 71.816059] .nr_running : 0
> >> <4>[ 71.816059] .load : 0
> >> <4>[ 71.816059] .nr_switches : 4398
> >> <4>[ 71.816059] .nr_load_updates : 6055
> >> <4>[ 71.816059] .nr_uninterruptible : -9
> >> <4>[ 71.816059] .next_balance : 4294.912466
> >> <4>[ 71.816059] .curr->pid : 0
> >> <4>[ 71.816059] .clock : 80673.512781
> >> <4>[ 71.816059] .cpu_load[0] : 0
> >> <4>[ 71.816059] .cpu_load[1] : 0
> >> <4>[ 71.816059] .cpu_load[2] : 0
> >> <4>[ 71.816059] .cpu_load[3] : 0
> >> <4>[ 71.816059] .cpu_load[4] : 0
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] cfs_rq[3]:/
> >> <4>[ 71.816059] .exec_clock : 0.000000
> >> <4>[ 71.816059] .MIN_vruntime : 0.000001
> >> <4>[ 71.816059] .min_vruntime : 5646.686691
> >> <4>[ 71.816059] .max_vruntime : 0.000001
> >> <4>[ 71.816059] .spread : 0.000000
> >> <4>[ 71.816059] .spread0 : -3779.357525
> >> <4>[ 71.816059] .nr_running : 0
> >> <4>[ 71.816059] .load : 0
> >> <4>[ 71.816059] .nr_spread_over : 0
> >> <4>[ 71.816059] .shares : 0
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] rt_rq[3]:
> >> <4>[ 71.816059] .rt_nr_running : 0
> >> <4>[ 71.816059] .rt_throttled : 0
> >> <4>[ 71.816059] .rt_time : 0.000000
> >> <4>[ 71.816059] .rt_runtime : 950.000000
> >> <4>[ 71.816059]
> >> <4>[ 71.816059] runnable tasks:
> >> <4>[ 71.816059] task PID tree-key switches prio exec-runtime sum-exec sum-sleep
> >> <4>[ 71.816059] ----------------------------------------------------------------------------------------------------------
> >> <4>[ 71.816059]
> >>
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000004 0 1396 1 0x00000080
> >> <4>[ 71.816059] ffff88007ddcfc68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007ddcffd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff880003560000 ffff88007dad4ec0
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8107a4b3>] ? mark_lock+0x22/0x22d
> >> <4>[ 71.816059] [<ffffffff8102e140>] ? pvclock_clocksource_read+0x42/0x9c
> >> <4>[ 71.816059] [<ffffffff8100e4f2>] ? check_events+0x12/0x20
> >> <4>[ 71.816059] [<ffffffff8122a179>] tty_ioctl+0x7f4/0x85c
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff81110444>] vfs_ioctl+0x2a/0x78
> >> <4>[ 71.816059] [<ffffffff811109b7>] do_vfs_ioctl+0x4aa/0x4e7
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8107d438>] ? lock_release+0x199/0x1a6
> >> <4>[ 71.816059] [<ffffffff81110a49>] sys_ioctl+0x55/0x77
> >> <4>[ 71.816059] [<ffffffff81010c82>] system_call_fastpath+0x16/0x1b
> >> <6>[ 71.816059] console-kit-d S 0000000000000005 0 1397 1 0x00000080
> >> <4>[ 71.816059] ffff88007e387c68 0000000000000246 00000000000003da ffffffff81b1d760
> >> <4>[ 71.816059] 0000000000000246 00000000000003da 000000000000fa40 ffff88007e387fd8
> >> <4>[ 71.816059] 00000000001d5bc0 00000000001d5bc0 ffff88007d0d0000 ffff8800034ca760
> >> <4>[ 71.816059] Call Trace:
> >> <4>[ 71.816059] [<ffffffff8100e4df>] ? xen_restore_fl_direct_end+0x0/0x1
> >> <4>[ 71.816059] [<ffffffff8133a9a7>] ? _spin_unlock_irqrestore+0x50/0x55
> >> <4>[ 71.816059] [<ffffffff812315c6>] vt_event_wait+0xc2/0x105
> >> <4>[ 71.816059] [<ffffffff8106a190>] ? autoremove_wake_function+0x0/0x38
> >> <4>[ 71.816059] [<ffffffff81231635>] vt_waitactive+0x2c/0x49
> >> <4>[ 71.816059] [<ffffffff81232b09>] vt_ioctl+0x125d/0x19c9
> >> <4>[ 71.816059] [<ffffffff8100de75>] ? xen_force_evtchn_callback+0xd/0xf
> >> <4>[ 71.816059] [<ffffffff
 
Old 09-24-2010, 05:34 PM
Jeremy Fitzhardinge
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

On 09/23/2010 10:11 PM, Timo Juhani Lindfors wrote:
> Ian Campbell <ijc@hellion.org.uk> writes:
>> I guess the right thing to do would be to using some existing paravirt
>> interface to send the IPI instead of tweaking the APIC directly, or if
> Is
>
> void xen_send_IPI_one(unsigned int cpu, enum ipi_vector vector);
>
> in xen/events.h perhaps already this?
>
>

The crashing code is actually using the LAPIC to send a NMI to the other
CPUs, not a regular IPI. We don't have any equivalent to that at
present. We could probably do the higher-level "show all cpu's
stacktraces" operation some other way entirely (since we can grab the
other CPU's context via Xen without having to rely on the CPU itself to
do the job). But I haven't looked into what it would take to do that.

For now, it might just be easiest to put a "if (xen_pv_domain())
return;" to nop out the whole thing. We could also set up a dummy lapic
page to absorb the write, but that would just leave it spinning.

Hm, since we don't enable APIC in the CPU capabilities, that code
shouldn't be running anyway; that's probably the cleanest workaround for
now.

J



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4C9CE127.1010508@goop.org">http://lists.debian.org/4C9CE127.1010508@goop.org
 
Old 09-24-2010, 06:53 PM
Jeremy Fitzhardinge
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

On 09/24/2010 01:07 AM, Ian Campbell wrote:
> NMI injection on 2.6.18 was one of the first Xen things I worked on (for
> injecting h/w NMI button faults) so something worked once upon a time. I
> can see CALLBACKTYPE_nmi and VCPUOP_send_nmi which seems promising that
> it's still around (and extended since I didn't do VCPUOP_send_nmi), even
> if its bit-rotted or not made it to pvops yet.
>
> In 2.6.18-xen the guest side of receiving an NMI was pretty ugly IIRC
> since we needed to force a return via the iret hypercall (to allow the
> hypervisor to simulate the NMI interrupt window).
>
> There is a small amount of residual support even in pvops
> (XEN_EFLAGS_NMI is defined, and tested once on 32 bit, but never set).
> It might be possible to do something nicer with all the pvops
> infrastructure, I'm also not sure it isn't a premature optimisation to
> save a compare by putting the NMI flag into a reserved bit of the saved
> EFLAGS anyway.
>

I think the Intel MCE people have made NMI work in pvops, but I didn't
look closely.

But from a pvops perspective, I think the tricky part is sending an NMI
rather than receiving. As I said in my mail to Timo, the simplest
option is to disable that code for now.

J



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4C9CF394.90509@goop.org">http://lists.debian.org/4C9CF394.90509@goop.org
 
Old 09-24-2010, 08:28 PM
Ian Campbell
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

On Fri, 2010-09-24 at 11:53 -0700, Jeremy Fitzhardinge wrote:
> On 09/24/2010 01:07 AM, Ian Campbell wrote:
> > NMI injection on 2.6.18 was one of the first Xen things I worked on (for
> > injecting h/w NMI button faults) so something worked once upon a time. I
> > can see CALLBACKTYPE_nmi and VCPUOP_send_nmi which seems promising that
> > it's still around (and extended since I didn't do VCPUOP_send_nmi), even
> > if its bit-rotted or not made it to pvops yet.
> >
> > In 2.6.18-xen the guest side of receiving an NMI was pretty ugly IIRC
> > since we needed to force a return via the iret hypercall (to allow the
> > hypervisor to simulate the NMI interrupt window).
> >
> > There is a small amount of residual support even in pvops
> > (XEN_EFLAGS_NMI is defined, and tested once on 32 bit, but never set).
> > It might be possible to do something nicer with all the pvops
> > infrastructure, I'm also not sure it isn't a premature optimisation to
> > save a compare by putting the NMI flag into a reserved bit of the saved
> > EFLAGS anyway.
> >
>
> I think the Intel MCE people have made NMI work in pvops, but I didn't
> look closely.
>
> But from a pvops perspective, I think the tricky part is sending an NMI
> rather than receiving.

It's not just "HYPERVISOR_vcpu_op(VCPUOP_send_nmi, cpu, NULL)"?

> As I said in my mail to Timo, the simplest option is to disable that code for now.

Agreed.

--
Ian Campbell

Your business will assume vast proportions.
 
Old 09-24-2010, 09:02 PM
Jeremy Fitzhardinge
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

On 09/24/2010 01:28 PM, Ian Campbell wrote:
>> I think the Intel MCE people have made NMI work in pvops, but I didn't
>> look closely.
>>
>> But from a pvops perspective, I think the tricky part is sending an NMI
>> rather than receiving.
> It's not just "HYPERVISOR_vcpu_op(VCPUOP_send_nmi, cpu, NULL)"?

That part's simple, but how to expose it in pvops? A "send NMI" would
be the lowest-level version, but not very generic. We could do
something like "dump all CPU backtraces" as a high-level operation
without needing any NMIs/IPIs.

J



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 4C9D11E5.1000108@goop.org">http://lists.debian.org/4C9D11E5.1000108@goop.org
 
Old 09-29-2010, 10:37 AM
Ian Campbell
 
Default Bug#597828: linux-image-2.6.32-5-amd64: debug options + sysrq-t => sending NMI to all CPUs: BUG: unable to handle kernel paging request

On Fri, 2010-09-24 at 14:02 -0700, Jeremy Fitzhardinge wrote:
> On 09/24/2010 01:28 PM, Ian Campbell wrote:
> >> I think the Intel MCE people have made NMI work in pvops, but I didn't
> >> look closely.
> >>
> >> But from a pvops perspective, I think the tricky part is sending an NMI
> >> rather than receiving.
> > It's not just "HYPERVISOR_vcpu_op(VCPUOP_send_nmi, cpu, NULL)"?
>
> That part's simple, but how to expose it in pvops? A "send NMI" would
> be the lowest-level version, but not very generic.

It seems like generally useful functionality to me, wouldn't other
sub-archs want it too? e.g. UV appears to use a non-x86 APIC mechanism
for sending IPIs, although its not clear if the non-x86-ness of their
APIC extends to sending NMIs differently or not.

> We could do something like "dump all CPU backtraces" as a high-level operation
> without needing any NMIs/IPIs.

I think the only hypercalls which return VCPU state are domctl's and
hence not much use to us in the kernel.

Ian.
--
Ian Campbell
Current Noise: Judas Priest - Leather Rebel

Have you noticed that all you need to grow healthy, vigorous grass is a
crack in your sidewalk?




--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1285756676.16095.38058.camel@zakaz.uk.xensource.co m">http://lists.debian.org/1285756676.16095.38058.camel@zakaz.uk.xensource.co m
 

Thread Tools




All times are GMT. The time now is 07:27 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org