FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 04-03-2011, 05:20 PM
David Sastre
 
Default Kernel trace, how to debug?

Hello,

I got a kernel trace related to some I/O error.
I'd like to debug it further, but don't not how to proceed.
I have some basic gdb skills, but the pkg involved is the kernel, so
I can't just run 'gdb executable trace'...

The trace looks like this:

2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554092] ------------[ cut here ]------------
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554096] WARNING: at /build/buildd-linux-2.6_2.6.32-30-i386-UYhWt7/linux-2.6-2.6.32/debian/build/source_i386_none/fs/buffer.c:1160 mark_buffer_dirty+0x20/0x7a()
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554098] Hardware name: P55-US3L
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554099] Modules linked in: tun ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables dahdi_echocan_oslec echo wcfxo dahdi crc_ccitt ppdev lp kvm_intel kvm binfmt_misc fuse bridge stp loop saa7134_alsa tda10048 saa7134_dvb videobuf_dvb dvb_core snd_hda_codec_atihdmi snd_hda_codec_realtek tda18271 tda8290 joydev tuner usbhid hid snd_hda_intel saa7134 sg snd_hda_codec ir_common sr_mod v4l2_common videodev snd_hwdep snd_pcm_oss v4l1_compat videobuf_dma_sg snd_seq_midi snd_mixer_oss videobuf_core snd_rawmidi snd_pcm r8169 snd_seq_midi_event cdrom usb_storage snd_seq ata_generic uhci_hcd snd_timer ehci_hcd mii tulip tveeprom ata_piix usbcore snd_seq_device ahci fglrx(P) snd serio_raw pata_jmicron soundcore snd_page_alloc nls_base pcspkr i2c_i801 i2c_core evdev processor button parport_pc parport ext3 jbd mbcache dm_mod raid1 md_mod sd_mod crc_t10dif sata_via libata scsi_mod thermal thermal_sys
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554140] Pid: 18351, comm: amarok Tainted: P 2.6.32-5-686-bigmem #1
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554142] Call Trace:
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554146] [<c103652d>] ? warn_slowpath_common+0x5e/0x8a
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554148] [<c1036563>] ? warn_slowpath_null+0xa/0xc
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554150] [<c10d5528>] ? mark_buffer_dirty+0x20/0x7a
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554157] [<f8307fac>] ? ext3_commit_super+0x40/0x56 [ext3]
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554162] [<f8309516>] ? ext3_handle_error+0x6e/0x8e [ext3]
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554166] [<f83095df>] ? ext3_error+0x3a/0x40 [ext3]
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554172] [<f8303401>] ? ext3_free_branches+0xa3/0x1e9 [ext3]
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554178] [<f8303a33>] ? ext3_truncate+0x4ec/0x70c [ext3]
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554182] [<f82c8a97>] ? journal_stop+0x254/0x260 [jbd]
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554187] [<f83041b9>] ? ext3_write_begin+0x1ac/0x1d2 [ext3]
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554191] [<c108e657>] ? generic_file_buffered_write+0xd0/0x21d
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554194] [<c108eba1>] ? __generic_file_aio_write+0x265/0x28d
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554197] [<c10c92d6>] ? touch_atime+0x69/0xd9
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554199] [<c108ec1a>] ? generic_file_aio_write+0x51/0x93
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554202] [<c10ba776>] ? do_sync_write+0xc0/0x107
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554205] [<c10bd4eb>] ? cp_new_stat64+0xf7/0x109
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554208] [<c104a322>] ? autoremove_wake_function+0x0/0x2d
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554211] [<c10bda3f>] ? sys_fstat64+0x1e/0x23
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554214] [<c1109ce8>] ? security_file_permission+0xc/0xd
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554216] [<c10ba6b6>] ? do_sync_write+0x0/0x107
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554217] [<c10bb0a2>] ? vfs_write+0x7e/0xd6
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554219] [<c10bb192>] ? sys_write+0x3c/0x63
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554222] [<c100813b>] ? sysenter_do_call+0x12/0x28
2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554223] ---[ end trace 20f4b55a0560b754 ]---

FWIW, I have dbg symbols installed for this kernel:

ii linux-image-2.6.32-5-686-bigmem 2.6.32-30 Linux 2.6.32 for PCs with 4GB+ RAM
ii linux-image-2.6.32-5-686-bigmem-dbg 2.6.32-31 Debugging infos for Linux 2.6.32-5-686-bigmem

Any pointers highly appreciated.

--
Huella de clave primaria: 0FDA C36F F110 54F4 D42B D0EB 617D 396C 448B 31EB
 
Old 04-03-2011, 06:41 PM
David Sastre
 
Default Kernel trace, how to debug?

On Sun, Apr 03, 2011 at 07:20:08PM +0200, David Sastre wrote:
> Hello,
>
> I got a kernel trace related to some I/O error.
> I'd like to debug it further, but don't not how to proceed.
> I have some basic gdb skills, but the pkg involved is the kernel, so
> I can't just run 'gdb executable trace'...
>
> The trace looks like this:
>
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554092] ------------[ cut here ]------------
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554096] WARNING: at /build/buildd-linux-2.6_2.6.32-30-i386-UYhWt7/linux-2.6-2.6.32/debian/build/source_i386_none/fs/buffer.c:1160 mark_buffer_dirty+0x20/0x7a()
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554098] Hardware name: P55-US3L
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554099] Modules linked in: tun ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables dahdi_echocan_oslec echo wcfxo dahdi crc_ccitt ppdev lp kvm_intel kvm binfmt_misc fuse bridge stp loop saa7134_alsa tda10048 saa7134_dvb videobuf_dvb dvb_core snd_hda_codec_atihdmi snd_hda_codec_realtek tda18271 tda8290 joydev tuner usbhid hid snd_hda_intel saa7134 sg snd_hda_codec ir_common sr_mod v4l2_common videodev snd_hwdep snd_pcm_oss v4l1_compat videobuf_dma_sg snd_seq_midi snd_mixer_oss videobuf_core snd_rawmidi snd_pcm r8169 snd_seq_midi_event cdrom usb_storage snd_seq ata_generic uhci_hcd snd_timer ehci_hcd mii tulip tveeprom ata_piix usbcore snd_seq_device ahci fglrx(P) snd serio_raw pata_jmicron soundcore snd_page_alloc nls_base pcspkr i2c_i801 i2c_core evdev processor button parport_pc parport ext3 jbd mbcache dm_mod raid1 md_mod sd_mod crc_t10dif sata_via libata scsi_mod thermal thermal_sys
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554140] Pid: 18351, comm: amarok Tainted: P 2.6.32-5-686-bigmem #1
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554142] Call Trace:
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554146] [<c103652d>] ? warn_slowpath_common+0x5e/0x8a
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554148] [<c1036563>] ? warn_slowpath_null+0xa/0xc
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554150] [<c10d5528>] ? mark_buffer_dirty+0x20/0x7a
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554157] [<f8307fac>] ? ext3_commit_super+0x40/0x56 [ext3]
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554162] [<f8309516>] ? ext3_handle_error+0x6e/0x8e [ext3]
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554166] [<f83095df>] ? ext3_error+0x3a/0x40 [ext3]
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554172] [<f8303401>] ? ext3_free_branches+0xa3/0x1e9 [ext3]
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554178] [<f8303a33>] ? ext3_truncate+0x4ec/0x70c [ext3]
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554182] [<f82c8a97>] ? journal_stop+0x254/0x260 [jbd]
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554187] [<f83041b9>] ? ext3_write_begin+0x1ac/0x1d2 [ext3]
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554191] [<c108e657>] ? generic_file_buffered_write+0xd0/0x21d
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554194] [<c108eba1>] ? __generic_file_aio_write+0x265/0x28d
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554197] [<c10c92d6>] ? touch_atime+0x69/0xd9
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554199] [<c108ec1a>] ? generic_file_aio_write+0x51/0x93
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554202] [<c10ba776>] ? do_sync_write+0xc0/0x107
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554205] [<c10bd4eb>] ? cp_new_stat64+0xf7/0x109
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554208] [<c104a322>] ? autoremove_wake_function+0x0/0x2d
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554211] [<c10bda3f>] ? sys_fstat64+0x1e/0x23
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554214] [<c1109ce8>] ? security_file_permission+0xc/0xd
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554216] [<c10ba6b6>] ? do_sync_write+0x0/0x107
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554217] [<c10bb0a2>] ? vfs_write+0x7e/0xd6
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554219] [<c10bb192>] ? sys_write+0x3c/0x63
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554222] [<c100813b>] ? sysenter_do_call+0x12/0x28
> 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554223] ---[ end trace 20f4b55a0560b754 ]---
>
> FWIW, I have dbg symbols installed for this kernel:
>
> ii linux-image-2.6.32-5-686-bigmem 2.6.32-30 Linux 2.6.32 for PCs with 4GB+ RAM
> ii linux-image-2.6.32-5-686-bigmem-dbg 2.6.32-31 Debugging infos for Linux 2.6.32-5-686-bigmem

Can anybody tell me how to properly enable ddebug on a 2.6.32-5 kernel?

I've been able to find some info regarding 2.6.38¹, but that doesn't seem
to apply to 2.6.32, as there is no /sys/kernel/debug/dynamic_debug
directory after mounting the debufgs:

# mount -t debugfs none /sys/kernel/debug/
# ll /sys/kernel/debug/
total 0
drwxr-xr-x 40 root root 0 abr 3 19:58 bdi
drwxr-xr-x 4 root root 0 abr 3 11:47 hid
drwxr-xr-x 2 root root 0 abr 3 11:47 kprobes
drwxr-xr-x 2 root root 0 abr 3 12:07 kvm
drwxr-xr-x 2 root root 0 abr 3 11:47 mce
-rw-r--r-- 1 root root 0 abr 3 11:47 sched_features
drwxr-xr-x 5 root root 0 abr 3 11:47 tracing
drwxr-xr-x 2 root root 0 abr 3 11:47 usb
drwxr-xr-x 2 root root 0 abr 3 11:47 x86

and therefore I can't issue:

# echo 'file buffer.c line 1160 +p' > /sys/kernel/debug/dynamic_debug/control

¹http://www.mjmwired.net/kernel/Documentation/dynamic-debug-howto.txt

--
Huella de clave primaria: 0FDA C36F F110 54F4 D42B D0EB 617D 396C 448B 31EB
 
Old 04-03-2011, 08:13 PM
David Sastre
 
Default Kernel trace, how to debug?

On Sun, Apr 03, 2011 at 08:41:51PM +0200, David Sastre wrote:
> On Sun, Apr 03, 2011 at 07:20:08PM +0200, David Sastre wrote:
> > Hello,
> >
> > I got a kernel trace related to some I/O error.
> > I'd like to debug it further, but don't not how to proceed.
> > I have some basic gdb skills, but the pkg involved is the kernel, so
> > I can't just run 'gdb executable trace'...
> >
> > The trace looks like this:
> >
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554092] ------------[ cut here ]------------
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554096] WARNING: at /build/buildd-linux-2.6_2.6.32-30-i386-UYhWt7/linux-2.6-2.6.32/debian/build/source_i386_none/fs/buffer.c:1160 mark_buffer_dirty+0x20/0x7a()
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554098] Hardware name: P55-US3L
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554099] Modules linked in: tun ipt_REJECT xt_tcpudp iptable_filter ip_tables x_tables dahdi_echocan_oslec echo wcfxo dahdi crc_ccitt ppdev lp kvm_intel kvm binfmt_misc fuse bridge stp loop saa7134_alsa tda10048 saa7134_dvb videobuf_dvb dvb_core snd_hda_codec_atihdmi snd_hda_codec_realtek tda18271 tda8290 joydev tuner usbhid hid snd_hda_intel saa7134 sg snd_hda_codec ir_common sr_mod v4l2_common videodev snd_hwdep snd_pcm_oss v4l1_compat videobuf_dma_sg snd_seq_midi snd_mixer_oss videobuf_core snd_rawmidi snd_pcm r8169 snd_seq_midi_event cdrom usb_storage snd_seq ata_generic uhci_hcd snd_timer ehci_hcd mii tulip tveeprom ata_piix usbcore snd_seq_device ahci fglrx(P) snd serio_raw pata_jmicron soundcore snd_page_alloc nls_base pcspkr i2c_i801 i2c_core evdev processor button parport_pc parport ext3 jbd mbcache dm_mod raid1 md_mod sd_mod crc_t10dif sata_via libata scsi_mod thermal thermal_sys
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554140] Pid: 18351, comm: amarok Tainted: P 2.6.32-5-686-bigmem #1
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554142] Call Trace:
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554146] [<c103652d>] ? warn_slowpath_common+0x5e/0x8a
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554148] [<c1036563>] ? warn_slowpath_null+0xa/0xc
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554150] [<c10d5528>] ? mark_buffer_dirty+0x20/0x7a
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554157] [<f8307fac>] ? ext3_commit_super+0x40/0x56 [ext3]
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554162] [<f8309516>] ? ext3_handle_error+0x6e/0x8e [ext3]
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554166] [<f83095df>] ? ext3_error+0x3a/0x40 [ext3]
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554172] [<f8303401>] ? ext3_free_branches+0xa3/0x1e9 [ext3]
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554178] [<f8303a33>] ? ext3_truncate+0x4ec/0x70c [ext3]
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554182] [<f82c8a97>] ? journal_stop+0x254/0x260 [jbd]
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554187] [<f83041b9>] ? ext3_write_begin+0x1ac/0x1d2 [ext3]
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554191] [<c108e657>] ? generic_file_buffered_write+0xd0/0x21d
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554194] [<c108eba1>] ? __generic_file_aio_write+0x265/0x28d
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554197] [<c10c92d6>] ? touch_atime+0x69/0xd9
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554199] [<c108ec1a>] ? generic_file_aio_write+0x51/0x93
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554202] [<c10ba776>] ? do_sync_write+0xc0/0x107
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554205] [<c10bd4eb>] ? cp_new_stat64+0xf7/0x109
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554208] [<c104a322>] ? autoremove_wake_function+0x0/0x2d
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554211] [<c10bda3f>] ? sys_fstat64+0x1e/0x23
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554214] [<c1109ce8>] ? security_file_permission+0xc/0xd
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554216] [<c10ba6b6>] ? do_sync_write+0x0/0x107
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554217] [<c10bb0a2>] ? vfs_write+0x7e/0xd6
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554219] [<c10bb192>] ? sys_write+0x3c/0x63
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554222] [<c100813b>] ? sysenter_do_call+0x12/0x28
> > 2011-04-03T18:36:32+02:00 jethro kernel: : [24521.554223] ---[ end trace 20f4b55a0560b754 ]---
> >
> > FWIW, I have dbg symbols installed for this kernel:
> >
> > ii linux-image-2.6.32-5-686-bigmem 2.6.32-30 Linux 2.6.32 for PCs with 4GB+ RAM
> > ii linux-image-2.6.32-5-686-bigmem-dbg 2.6.32-31 Debugging infos for Linux 2.6.32-5-686-bigmem
>
> Can anybody tell me how to properly enable ddebug on a 2.6.32-5 kernel?
>
> I've been able to find some info regarding 2.6.38¹, but that doesn't seem
> to apply to 2.6.32, as there is no /sys/kernel/debug/dynamic_debug
> directory after mounting the debufgs:
>
> # mount -t debugfs none /sys/kernel/debug/
> # ll /sys/kernel/debug/
> total 0
> drwxr-xr-x 40 root root 0 abr 3 19:58 bdi
> drwxr-xr-x 4 root root 0 abr 3 11:47 hid
> drwxr-xr-x 2 root root 0 abr 3 11:47 kprobes
> drwxr-xr-x 2 root root 0 abr 3 12:07 kvm
> drwxr-xr-x 2 root root 0 abr 3 11:47 mce
> -rw-r--r-- 1 root root 0 abr 3 11:47 sched_features
> drwxr-xr-x 5 root root 0 abr 3 11:47 tracing
> drwxr-xr-x 2 root root 0 abr 3 11:47 usb
> drwxr-xr-x 2 root root 0 abr 3 11:47 x86
>
> and therefore I can't issue:
>
> # echo 'file buffer.c line 1160 +p' > /sys/kernel/debug/dynamic_debug/control
>
> ¹http://www.mjmwired.net/kernel/Documentation/dynamic-debug-howto.txt

OK. As stated in the kernel sources under
Documentation/dynamic-debug-howto.txt, CONFIG_DYNAMIC_DEBUG must be
set to be able to use ddebug. As it is not my case:

# grep CONFIG_DYNAMIC_DEBUG config-2.6.32-5-686-bigmem
# CONFIG_DYNAMIC_DEBUG is not set

...is easy to see why I don't have that dir under <debugfs>

Looks like I have to recompile to debug further.

--
Huella de clave primaria: 0FDA C36F F110 54F4 D42B D0EB 617D 396C 448B 31EB
 

Thread Tools




All times are GMT. The time now is 02:21 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org