FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Red Hat Linux

 
 
LinkBack Thread Tools
 
Old 06-03-2011, 07:59 PM
Bob Wickline
 
Default kernel: INFO: task xxx:yyy blocked for more than zzz seconds.

I have an HP DL385 running RHEL5.5 that's close to going into production. However, we've experienced a couple of lockups in the past couple weeks that has management concerned. I've been doing some research but I haven't found any answer to the issue.

I did see something about a possible issue with the cciss driver so I decided to run bonnie++ against one of the internal LUNs and sure enough I can duplocate the issue within minutes.

What is going on here? Is there a fix for this issue? Is anyone working on a solution???


>Jun 2 12:59:46 hidden kernel: INFO: task kswapd0:563 blocked for more
>than
>120 seconds.
>Jun 2 12:59:46 hidden kernel: "echo 0 >
>/proc/sys/kernel/hung_task_timeout_secs" disables this message.
>Jun 2 12:59:46 hidden kernel: kswapd0 D ffff810021004420 0
>563
>185 564 560 (L-TLB)
>Jun 2 12:59:46 hidden kernel: ffff81042efd9b00 0000000000000046
>0000000000000002 0000000000000010
>Jun 2 12:59:46 hidden kernel: 0000000000008f3f 000000000000000a
>ffff81042f5950c0 ffffffff80311b60
>Jun 2 12:59:46 hidden kernel: 0002c883e50c78ed 00000000002f2877
>ffff81042f5952a8 000000000000003d
>Jun 2 12:59:46 hidden kernel: Call Trace:
>Jun 2 12:59:46 hidden kernel: [<ffffffff88036d8a>]
>:jbd:log_wait_commit+0xa3/0xf5
>Jun 2 12:59:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 12:59:46 hidden kernel: [<ffffffff8002781a>]
>try_to_free_buffers+0x60/0xb9
>Jun 2 12:59:46 hidden kernel: [<ffffffff880332da>]
>:jbd:journal_try_to_free_buffers+0x19d/0x1c0
>Jun 2 12:59:46 hidden kernel: [<ffffffff800cd369>]
>shrink_inactive_list+0x511/0x8d8
>Jun 2 12:59:46 hidden kernel: [<ffffffff8004817a>]
>__pagevec_release+0x19/0x22
>Jun 2 12:59:48 hidden kernel: [<ffffffff800ccd37>]
>shrink_active_list+0x4b4/0x4c4
>Jun 2 12:59:49 hidden kernel: [<ffffffff8001327d>]
>shrink_zone+0x127/0x18d
>Jun 2 12:59:50 hidden kernel: [<ffffffff80057e54>] kswapd+0x33d/0x495
>Jun 2 12:59:50 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 12:59:51 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 12:59:51 hidden kernel: [<ffffffff80057b17>] kswapd+0x0/0x495
>Jun 2 12:59:51 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 12:59:51 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132
>Jun 2 12:59:52 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11
>Jun 2 12:59:53 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 12:59:54 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132
>Jun 2 12:59:54 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11
>Jun 2 12:59:55 hidden kernel:
>Jun 2 12:59:56 hidden kernel: INFO: task kjournald:6071 blocked for
>more
>than 120 seconds.
>Jun 2 12:59:56 hidden kernel: "echo 0 >
>/proc/sys/kernel/hung_task_timeout_secs" disables this message.
>Jun 2 12:59:57 hidden kernel: kjournald D ffff81002100caa0 0
>6071
>185 6077 5264 (L-TLB)
>Jun 2 12:59:58 hidden kernel: ffff810427f4bcf0 0000000000000046
>000000002dd9c000 ffff81042ff15950
>Jun 2 12:59:58 hidden kernel: 00000001ffffffff 0000000000000009
>ffff810420322080 ffff81042ff18100
>Jun 2 12:59:59 hidden kernel: 0002c889276dcdab 00000000071ac130
>ffff810420322268 0000000100000086
>Jun 2 13:00:00 hidden kernel: Call Trace:
>Jun 2 13:00:02 hidden kernel: [<ffffffff8006ec4e>]
>do_gettimeofday+0x40/0x90
>Jun 2 13:00:02 hidden kernel: [<ffffffff800155a2>]
>sync_buffer+0x0/0x3f
>Jun 2 13:00:03 hidden kernel: [<ffffffff800637ca>]
>io_schedule+0x3f/0x67
>Jun 2 13:00:04 hidden kernel: [<ffffffff800155dd>]
>sync_buffer+0x3b/0x3f
>Jun 2 13:00:05 hidden kernel: [<ffffffff800639f6>]
>__wait_on_bit+0x40/0x6e
>Jun 2 13:00:06 hidden kernel: [<ffffffff800155a2>]
>sync_buffer+0x0/0x3f
>Jun 2 13:00:06 hidden kernel: [<ffffffff80063a90>]
>out_of_line_wait_on_bit+0x6c/0x78
>Jun 2 13:00:07 hidden kernel: [<ffffffff800a2921>]
>wake_bit_function+0x0/0x23
>Jun 2 13:00:08 hidden kernel: [<ffffffff880339a5>]
>:jbd:journal_commit_transaction+0x543/0x1066
>Jun 2 13:00:10 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:00:12 hidden kernel: [<ffffffff8004b3fb>]
>try_to_del_timer_sync+0x7f/0x88
>Jun 2 13:00:13 hidden kernel: [<ffffffff880375d3>]
>:jbd:kjournald+0xc1/0x213
>Jun 2 13:00:13 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:00:13 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:00:15 hidden kernel: [<ffffffff88037512>]
>:jbd:kjournald+0x0/0x213
>Jun 2 13:00:15 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:00:17 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132
>Jun 2 13:00:18 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11
>Jun 2 13:00:19 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:00:19 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132
>Jun 2 13:00:20 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11
>Jun 2 13:00:22 hidden kernel:
>Jun 2 13:01:46 hidden kernel: INFO: task kswapd0:563 blocked for more
>than
>120 seconds.
>Jun 2 13:01:46 hidden kernel: "echo 0 >
>/proc/sys/kernel/hung_task_timeout_secs" disables this message.
>Jun 2 13:01:46 hidden kernel: kswapd0 D ffff810021004420 0
>563
>185 564 560 (L-TLB)
>Jun 2 13:01:46 hidden kernel: ffff81042efd9b00 0000000000000046
>0000000000000002 0000000000000010
>Jun 2 13:01:46 hidden kernel: 0000000000008f3f 000000000000000a
>ffff81042f5950c0 ffffffff80311b60
>Jun 2 13:01:46 hidden kernel: 0002c883e50c78ed 00000000002f2877
>ffff81042f5952a8 000000000000003d
>Jun 2 13:01:46 hidden kernel: Call Trace:
>Jun 2 13:01:46 hidden kernel: [<ffffffff88036d8a>]
>:jbd:log_wait_commit+0xa3/0xf5
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:01:46 hidden kernel: [<ffffffff8002781a>]
>try_to_free_buffers+0x60/0xb9
>Jun 2 13:01:46 hidden kernel: [<ffffffff880332da>]
>:jbd:journal_try_to_free_buffers+0x19d/0x1c0
>Jun 2 13:01:46 hidden kernel: [<ffffffff800cd369>]
>shrink_inactive_list+0x511/0x8d8
>Jun 2 13:01:46 hidden kernel: [<ffffffff8004817a>]
>__pagevec_release+0x19/0x22
>Jun 2 13:01:46 hidden kernel: [<ffffffff800ccd37>]
>shrink_active_list+0x4b4/0x4c4
>Jun 2 13:01:46 hidden kernel: [<ffffffff8001327d>]
>shrink_zone+0x127/0x18d
>Jun 2 13:01:46 hidden kernel: [<ffffffff80057e54>] kswapd+0x33d/0x495
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:01:46 hidden kernel: [<ffffffff80057b17>] kswapd+0x0/0x495
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:01:46 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132
>Jun 2 13:01:46 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:01:46 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132
>Jun 2 13:01:46 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11
>Jun 2 13:01:46 hidden kernel:
>Jun 2 13:01:46 hidden kernel: INFO: task kjournald:6071 blocked for
>more
>than 120 seconds.
>Jun 2 13:01:46 hidden kernel: "echo 0 >
>/proc/sys/kernel/hung_task_timeout_secs" disables this message.
>Jun 2 13:01:46 hidden kernel: kjournald D ffff81002100caa0 0
>6071
>185 6077 5264 (L-TLB)
>Jun 2 13:01:46 hidden kernel: ffff810427f4bcf0 0000000000000046
>000000002dd9c000 ffff81042ff15950
>Jun 2 13:01:46 hidden kernel: 00000001ffffffff 0000000000000009
>ffff810420322080 ffff81042ff18100
>Jun 2 13:01:46 hidden kernel: 0002c889276dcdab 00000000071ac130
>ffff810420322268 0000000100000086
>Jun 2 13:01:46 hidden kernel: Call Trace:
>Jun 2 13:01:46 hidden kernel: [<ffffffff8006ec4e>]
>do_gettimeofday+0x40/0x90
>Jun 2 13:01:46 hidden kernel: [<ffffffff800155a2>]
>sync_buffer+0x0/0x3f
>Jun 2 13:01:46 hidden kernel: [<ffffffff800637ca>]
>io_schedule+0x3f/0x67
>Jun 2 13:01:46 hidden kernel: [<ffffffff800155dd>]
>sync_buffer+0x3b/0x3f
>Jun 2 13:01:46 hidden kernel: [<ffffffff800639f6>]
>__wait_on_bit+0x40/0x6e
>Jun 2 13:01:46 hidden kernel: [<ffffffff800155a2>]
>sync_buffer+0x0/0x3f
>Jun 2 13:01:46 hidden kernel: [<ffffffff80063a90>]
>out_of_line_wait_on_bit+0x6c/0x78
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a2921>]
>wake_bit_function+0x0/0x23
>Jun 2 13:01:46 hidden kernel: [<ffffffff880339a5>]
>:jbd:journal_commit_transaction+0x543/0x1066
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:01:46 hidden kernel: [<ffffffff8004b3fb>]
>try_to_del_timer_sync+0x7f/0x88
>Jun 2 13:01:46 hidden kernel: [<ffffffff880375d3>]
>:jbd:kjournald+0xc1/0x213
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:01:46 hidden kernel: [<ffffffff88037512>]
>:jbd:kjournald+0x0/0x213
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:01:46 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132
>Jun 2 13:01:46 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11
>Jun 2 13:01:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:01:46 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132
>Jun 2 13:01:46 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11
>Jun 2 13:01:46 hidden kernel:
>Jun 2 13:03:46 hidden kernel: INFO: task kswapd0:563 blocked for more
>than
>120 seconds.
>Jun 2 13:03:46 hidden kernel: "echo 0 >
>/proc/sys/kernel/hung_task_timeout_secs" disables this message.
>Jun 2 13:03:46 hidden kernel: kswapd0 D ffff810021004420 0
>563
>185 564 560 (L-TLB)
>Jun 2 13:03:46 hidden kernel: ffff81042efd9b00 0000000000000046
>0000000000000002 0000000000000010
>Jun 2 13:03:46 hidden kernel: 0000000000008f3f 000000000000000a
>ffff81042f5950c0 ffffffff80311b60
>Jun 2 13:03:46 hidden kernel: 0002c883e50c78ed 00000000002f2877
>ffff81042f5952a8 000000000000003d
>Jun 2 13:03:46 hidden kernel: Call Trace:
>Jun 2 13:03:46 hidden kernel: [<ffffffff88036d8a>]
>:jbd:log_wait_commit+0xa3/0xf5
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:03:46 hidden kernel: [<ffffffff8002781a>]
>try_to_free_buffers+0x60/0xb9
>Jun 2 13:03:46 hidden kernel: [<ffffffff880332da>]
>:jbd:journal_try_to_free_buffers+0x19d/0x1c0
>Jun 2 13:03:46 hidden kernel: [<ffffffff800cd369>]
>shrink_inactive_list+0x511/0x8d8
>Jun 2 13:03:46 hidden kernel: [<ffffffff8004817a>]
>__pagevec_release+0x19/0x22
>Jun 2 13:03:46 hidden kernel: [<ffffffff800ccd37>]
>shrink_active_list+0x4b4/0x4c4
>Jun 2 13:03:46 hidden kernel: [<ffffffff8001327d>]
>shrink_zone+0x127/0x18d
>Jun 2 13:03:46 hidden kernel: [<ffffffff80057e54>] kswapd+0x33d/0x495
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:03:46 hidden kernel: [<ffffffff80057b17>] kswapd+0x0/0x495
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:03:46 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132
>Jun 2 13:03:46 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:03:46 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132
>Jun 2 13:03:46 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11
>Jun 2 13:03:46 hidden kernel:
>Jun 2 13:03:46 hidden kernel: INFO: task kjournald:6071 blocked for
>more
>than 120 seconds.
>Jun 2 13:03:46 hidden kernel: "echo 0 >
>/proc/sys/kernel/hung_task_timeout_secs" disables this message.
>Jun 2 13:03:46 hidden kernel: kjournald D ffff81002100caa0 0
>6071
>185 6077 5264 (L-TLB)
>Jun 2 13:03:46 hidden kernel: ffff810427f4bcf0 0000000000000046
>000000002dd9c000 ffff81042ff15950
>Jun 2 13:03:46 hidden kernel: 00000001ffffffff 0000000000000009
>ffff810420322080 ffff81042ff18100
>Jun 2 13:03:46 hidden kernel: 0002c889276dcdab 00000000071ac130
>ffff810420322268 0000000100000086
>Jun 2 13:03:46 hidden kernel: Call Trace:
>Jun 2 13:03:46 hidden kernel: [<ffffffff8006ec4e>]
>do_gettimeofday+0x40/0x90
>Jun 2 13:03:46 hidden kernel: [<ffffffff800155a2>]
>sync_buffer+0x0/0x3f
>Jun 2 13:03:46 hidden kernel: [<ffffffff800637ca>]
>io_schedule+0x3f/0x67
>Jun 2 13:03:46 hidden kernel: [<ffffffff800155dd>]
>sync_buffer+0x3b/0x3f
>Jun 2 13:03:46 hidden kernel: [<ffffffff800639f6>]
>__wait_on_bit+0x40/0x6e
>Jun 2 13:03:46 hidden kernel: [<ffffffff800155a2>]
>sync_buffer+0x0/0x3f
>Jun 2 13:03:46 hidden kernel: [<ffffffff80063a90>]
>out_of_line_wait_on_bit+0x6c/0x78
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a2921>]
>wake_bit_function+0x0/0x23
>Jun 2 13:03:46 hidden kernel: [<ffffffff880339a5>]
>:jbd:journal_commit_transaction+0x543/0x1066
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:03:46 hidden kernel: [<ffffffff8004b3fb>]
>try_to_del_timer_sync+0x7f/0x88
>Jun 2 13:03:46 hidden kernel: [<ffffffff880375d3>]
>:jbd:kjournald+0xc1/0x213
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:03:46 hidden kernel: [<ffffffff88037512>]
>:jbd:kjournald+0x0/0x213
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:03:46 hidden kernel: [<ffffffff80032afc>] kthread+0xfe/0x132
>Jun 2 13:03:46 hidden kernel: [<ffffffff8005dfb1>] child_rip+0xa/0x11
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a26db>]
>keventd_create_kthread+0x0/0xc4
>Jun 2 13:03:46 hidden kernel: [<ffffffff800329fe>] kthread+0x0/0x132
>Jun 2 13:03:46 hidden kernel: [<ffffffff8005dfa7>] child_rip+0x0/0x11
>Jun 2 13:03:46 hidden kernel:
>Jun 2 13:03:46 hidden kernel: INFO: task oracle:20274 blocked for more
>than
>120 seconds.
>Jun 2 13:03:46 hidden kernel: "echo 0 >
>/proc/sys/kernel/hung_task_timeout_secs" disables this message.
>Jun 2 13:03:46 hidden kernel: oracle D ffff810021025e20 0
>20274
>1 20276 20272 (NOTLB)
>Jun 2 13:03:46 hidden kernel: ffff810270fedcf8 0000000000000086
>ffff810270fedce0 ffff810270fedca8
>Jun 2 13:03:46 hidden kernel: 0000000000000000 0000000000000009
>ffff81037f726820 ffff81042fe2a080
>Jun 2 13:03:46 hidden kernel: 0002c8c7206b57d6 000000000003d4c1
>ffff81037f726a08 0000000470fedd18
>Jun 2 13:03:46 hidden kernel: Call Trace:
>Jun 2 13:03:46 hidden kernel: [<ffffffff88032002>]
>:jbd:start_this_handle+0x2e5/0x36c
>Jun 2 13:03:46 hidden kernel: [<ffffffff800a28f3>]
>autoremove_wake_function+0x0/0x2e
>Jun 2 13:03:46 hidden kernel: [<ffffffff88032152>]
>:jbd:journal_start+0xc9/0x100
>Jun 2 13:03:46 hidden kernel: [<ffffffff8805389c>]
>:ext3:ext3_create+0x49/0x102
>Jun 2 13:03:46 hidden kernel: [<ffffffff8003a86a>]
>vfs_create+0xe6/0x158
>Jun 2 13:03:46 hidden kernel: [<ffffffff8001b2d5>]
>open_namei+0x19d/0x712
>Jun 2 13:03:46 hidden kernel: [<ffffffff80027660>]
>do_filp_open+0x1c/0x38
>Jun 2 13:03:46 hidden kernel: [<ffffffff8001a061>]
>do_sys_open+0x44/0xbe
>Jun 2 13:03:46 hidden kernel: [<ffffffff8005d28d>] tracesys+0xd5/0xe0
>Jun 2 13:03:46 hidden kernel:

--
Sent from my Android phone with K-9 Mail. Please excuse my brevity.

--
redhat-list mailing list
unsubscribe mailto:redhat-list-request@redhat.com?subject=unsubscribe
https://www.redhat.com/mailman/listinfo/redhat-list
 

Thread Tools




All times are GMT. The time now is 02:42 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org