Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   Debian Kernel (http://www.linux-archive.org/debian-kernel/)
-   -   Bug#659169: (http://www.linux-archive.org/debian-kernel/630459-bug-659169-a.html)

Carlos Alberto Lopez Perez 02-08-2012 08:13 PM

Bug#659169:
 
Source: linux-2.6
Version: 2.6.32-5
Severity: normal


Hello,

Today one of my servers stopped responding to some webs, ssh'ing into it was impossible. The ping worked and the ssh connection was starting but the shell didn't showed up after waiting a long time. Finally a hard-reset was needed in order to bring it back

After the reboot I found this on kern.log

# tail /var/log/kern.log
Feb 1 09:41:00 server-i7_920 kernel: [17383037.287331] EXT4-fs (dm-29): mounted filesystem with ordered data mode
Feb 5 09:38:15 server-i7_920 kernel: [17727613.769052] NOHZ: local_softirq_pending 100
Feb 7 05:56:07 server-i7_920 kernel: [17886689.887577] e1000e: eth0 NIC Link is Down
Feb 7 05:59:05 server-i7_920 kernel: [17886867.166464] e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Feb 7 05:59:41 server-i7_920 kernel: [17886903.722727] e1000e: eth0 NIC Link is Down
Feb 7 06:00:00 server-i7_920 kernel: [17886922.309159] e1000e: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: None
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876326] BUG: soft lockup - CPU#7 stuck for 17163091979s! [init:9709]
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876371] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs ext4 jbd2 crc16 ext2 ipt_LOG sg xt_limit xt_tcpudp xt_state iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables x_tables dummy loop snd_pcm snd_timer snd soundcore snd_page_alloc ioatdma i2c_i801 i2c_core pcspkr dca psmouse evdev button serio_raw processor ext3 jbd mbcache dm_mod sd_mod crc_t10dif ahci libata uhci_hcd ehci_hcd scsi_mod usbcore nls_base e1000e thermal thermal_sys [last unloaded: scsi_wait_scan]
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876405] CPU 7:
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876406] Modules linked in: btrfs zlib_deflate crc32c libcrc32c ufs qnx4 hfsplus hfs minix ntfs vfat msdos fat jfs xfs exportfs reiserfs ext4 jbd2 crc16 ext2 ipt_LOG sg xt_limit xt_tcpudp xt_state iptable_mangle iptable_nat nf_nat nf_conntrack_ipv4 nf_conntrack nf_defrag_ipv4 iptable_filter ip_tables x_tables dummy loop snd_pcm snd_timer snd soundcore snd_page_alloc ioatdma i2c_i801 i2c_core pcspkr dca psmouse evdev button serio_raw processor ext3 jbd mbcache dm_mod sd_mod crc_t10dif ahci libata uhci_hcd ehci_hcd scsi_mod usbcore nls_base e1000e thermal thermal_sys [last unloaded: scsi_wait_scan]
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876433] Pid: 9709, comm: init Not tainted 2.6.32-5-vserver-amd64 #1 X8STi
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876435] RIP: 0023:[<00000000f76c0430>] [<00000000f76c0430>] 0xf76c0430
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876440] RSP: 002b:00000000ffa6673c EFLAGS: 00000296
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876442] RAX: fffffffffffffff6 RBX: 0000000000000000 RCX: 00000000ffa6673c
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876443] RDX: 00000000f76c0430 RSI: 000000000000000f RDI: 0000000000000000
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876445] RBP: ffffffff8101166e R08: 0000000000000000 R09: 0000000000000000
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876447] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876448] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876451] FS: 0000000000000000(0000) GS:ffff880016bc0000(0063) knlGS:00000000f74a96c0
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876453] CS: 0010 DS: 002b ES: 002b CR0: 000000008005003b
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876454] CR2: 00007fa1566220a0 CR3: 0000000514e5d000 CR4: 00000000000006e0
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876456] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876458] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Feb 8 17:28:57 server-i7_920 kernel: [18446744016.876460] Call Trace:
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130843] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130874] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130919] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130923] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130927] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130931] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130934] Call Trace:
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130942] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130947] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.130950] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131009] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131013] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131016] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131021] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131025] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:20:04 server-i7_920 kernel: [ 3003.131028] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832664] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832694] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832797] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832802] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832806] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832809] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832813] Call Trace:
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832820] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832825] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832829] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832833] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832837] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832840] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832845] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832849] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:22:04 server-i7_920 kernel: [ 3122.832852] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537240] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537271] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537316] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537320] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537324] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537328] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537331] Call Trace:
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537339] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537344] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537347] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537352] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537356] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537359] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537364] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537367] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:24:04 server-i7_920 kernel: [ 3242.537370] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236311] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236342] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236387] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236392] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236395] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236399] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236403] Call Trace:
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236410] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236415] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236418] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236423] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236427] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236430] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236435] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236439] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:26:04 server-i7_920 kernel: [ 3362.236442] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938131] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938162] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938207] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938211] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938215] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938219] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938223] Call Trace:
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938230] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938235] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938238] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938243] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938247] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938250] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938255] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938258] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:28:04 server-i7_920 kernel: [ 3481.938261] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.643990] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644021] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644066] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644070] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644074] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644078] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644081] Call Trace:
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644089] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644094] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644097] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644101] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644105] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644108] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644113] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644117] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:30:04 server-i7_920 kernel: [ 3601.644120] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346325] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346363] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346408] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346413] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346417] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346420] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346424] Call Trace:
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346431] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346437] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346440] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346444] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346449] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346452] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346457] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346460] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:32:04 server-i7_920 kernel: [ 3721.346463] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043601] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043632] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043677] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043682] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043685] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043689] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043693] Call Trace:
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043700] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043705] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043708] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043713] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043717] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043720] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043725] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043729] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:34:04 server-i7_920 kernel: [ 3841.043732] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745606] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745637] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745682] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745686] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745690] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745694] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745698] Call Trace:
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745705] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745710] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745713] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745718] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745722] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745725] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745730] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745734] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:36:04 server-i7_920 kernel: [ 3960.745737] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447611] INFO: task cron:8748 blocked for more than 120 seconds.
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447642] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447687] cron D 0000000000000000 0 8748 25441 0x00020000
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447692] ffff88063ccf3250 0000000000000086 0000000000000000 0000000000000000
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447696] ffff88055c60c7e0 00000000000000d0 000000000000f9e0 ffff88045d2bdfd8
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447699] 0000000000015780 0000000000015780 ffff88055c60c7e0 ffff88055c60cad8
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447703] Call Trace:
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447710] [<ffffffff81309fec>] ? schedule_timeout+0x2e/0xdd
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447715] [<ffffffff81070704>] ? vx_update_load+0x18/0x13c
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447718] [<ffffffff81309ea5>] ? wait_for_common+0xde/0x15b
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447723] [<ffffffff8104a4f0>] ? default_wake_function+0x0/0x9
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447727] [<ffffffff8103a4af>] ? activate_task+0x22/0x28
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447730] [<ffffffff8104de19>] ? do_fork+0x2d5/0x34e
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447735] [<ffffffff8101657d>] ? read_tsc+0xa/0x20
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447738] [<ffffffff810175f5>] ? sys_vfork+0x20/0x22
Feb 8 18:38:04 server-i7_920 kernel: [ 4080.447741] [<ffffffff810377d5>] ? ia32_ptregs_common+0x25/0x4c



Notice the weird thing between 17:28:57 and 18:20:04. The [timestamps] seems to go backwards (!!) How this can be even possible?... Also many of the modules listed on "Modules linked in" (btrfs,ufs,qnx4..) were *NOT* expected to be in use.



This is some useful information about this server (got after the reboot):

# lsmod
Module Size Used by
xt_limit 1782 1
xt_tcpudp 2319 9
xt_state 1303 2
iptable_mangle 2817 0
iptable_nat 4299 1
nf_nat 13388 1 iptable_nat
nf_conntrack_ipv4 9833 5 iptable_nat,nf_nat
nf_conntrack 46551 4 xt_state,iptable_nat,nf_nat,nf_conntrack_ipv4
nf_defrag_ipv4 1139 1 nf_conntrack_ipv4
iptable_filter 2258 1
ip_tables 13915 3 iptable_mangle,iptable_nat,iptable_filter
x_tables 12845 5 xt_limit,xt_tcpudp,xt_state,iptable_nat,ip_tables
ext4 291462 5
jbd2 67015 1 ext4
crc16 1319 1 ext4
dummy 1584 0
loop 11975 0
snd_pcm 60503 0
i2c_i801 7830 0
snd_timer 15582 1 snd_pcm
i2c_core 15819 1 i2c_i801
snd 46526 2 snd_pcm,snd_timer
soundcore 4598 1 snd
snd_page_alloc 6249 1 snd_pcm
pcspkr 1699 0
ioatdma 34892 24
dca 3761 1 ioatdma
psmouse 49937 0
evdev 7352 2
serio_raw 3752 0
processor 29951 8
button 4650 0
ext3 109110 25
jbd 37101 1 ext3
mbcache 5050 2 ext4,ext3
dm_mod 54320 91
sd_mod 29921 3
crc_t10dif 1276 1 sd_mod
uhci_hcd 18537 0
ahci 32534 2
libata 133776 1 ahci
ehci_hcd 32097 0
scsi_mod 126533 2 sd_mod,libata
usbcore 122514 3 uhci_hcd,ehci_hcd
nls_base 6377 1 usbcore
thermal 11674 0
e1000e 110079 0
thermal_sys 11942 2 processor,thermal


# parted /dev/sda print
Model: ATA ST32000641AS (scsi)
Disk /dev/sda: 2000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt

Number Start End Size File system Name Flags
1 17.4kB 512MB 512MB ext3 boot
2 512MB 2000GB 2000GB lvm



# pvs
PV VG Fmt Attr PSize PFree
/dev/sda2 vg lvm2 a- 1.82t 1.46t


# uname -r
2.6.32-5-vserver-amd64



Thanks!


--
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~
Carlos Alberto Lopez Perez http://neutrino.es
Igalia - Free Software Engineering http://www.igalia.com
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~~~


All times are GMT. The time now is 11:11 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.