FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Redhat > Device-mapper Development

 
 
LinkBack Thread Tools
 
Old 11-20-2007, 09:10 AM
"S. J. van Harmelen"
 
Default [dm-devel] Possible bug in multipathd (getting a segfault)

On Tue, 2007-11-20 at 10:49 +0100, Tore Anderson wrote:
> * S. J. van Harmelen
>
> > When I do use a 2.6.23.x kernel with the RDAC hardware handler I get the
> > following error: multipathd[6895]: segfault at
> > 000000000000000a rip 00002b251f24394f rsp 00007fff8bfaa730 error 4
>
> This is indicative of a bug in multipathd. It might or might not be
> serious, but I guess Christophe Varoqui would be interested in seeing a
> backtrace of the segfault in any case...
>
> Try running multipathd (make sure it's not stripped of debugging
> symbols) with core dumps enabled ("ulimit -c unlimited"), and get a
> backtrace ("gdb /sbin/multipathd core" - then run "bt full").

I just changed the subject, because this is realy another issue then
what we where talking about before.

Could you specify the steps I have to take to create the backtrace a bit
more as I have not done this before.

I run a debian etch server with a custom compiled 2.6.23.1 kernel when I
get the error.

Sander

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-20-2007, 09:21 AM
Tore Anderson
 
Default [dm-devel] Possible bug in multipathd (getting a segfault)

* S. J. van Harmelen

> Could you specify the steps I have to take to create the backtrace a
> bit more as I have not done this before.
>
> I run a debian etch server with a custom compiled 2.6.23.1 kernel
> when I get the error.

Debian strips all their binaries by default, so you need to recompile
multipath-tools and reinstall it:

$ export DEB_BUILD_OPTIONS=nostrip
$ apt-get build-dep multipath-tools
$ apt-get --compile source multipath-tools
$ dpkg -i multipath-tools*deb

Then start multipathd from a shell that has core dumps enabled (make
sure it's not already running):

$ ulimit -c unlimited
$ multipathd

Wait for a segfault to happen, and you should have gotten a file named
"core" in your current directory (or maybe in the root directory since
multipathd chdir()s there, I'm not sure) that contains the backtrace.
We need to extract it with GDB, though, so run:

$ gdb /sbin/multipathd /path/to/core
(gdb) bt full

...and post the resulting backtrace here. Hopefully a developer (not
me, unfortunately) will be able to make sense of it.

Another thing you might want to try is to use your 2.6.23.1 with the
same configuration as your 2.6.22.x (ie. don't use hwhandler rdac), and
see if the segfaults still occur.

Regards
--
Tore Anderson

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-20-2007, 09:34 AM
"S. J. van Harmelen"
 
Default [dm-devel] Possible bug in multipathd (getting a segfault)

On Tue, 2007-11-20 at 11:21 +0100, Tore Anderson wrote:
> * S. J. van Harmelen
>
> > Could you specify the steps I have to take to create the backtrace a
> > bit more as I have not done this before.
> >
> > I run a debian etch server with a custom compiled 2.6.23.1 kernel
> > when I get the error.
>
> Debian strips all their binaries by default, so you need to recompile
> multipath-tools and reinstall it:
>
> $ export DEB_BUILD_OPTIONS=nostrip
> $ apt-get build-dep multipath-tools
> $ apt-get --compile source multipath-tools
> $ dpkg -i multipath-tools*deb

I guess I don't have to do that as I compiled the multipath-tools
myself?

wget http://christophe.varoqui.free.fr/.../multipath-tools-0.4.8.tar.bz2
tar jxf multipath-tools-0.4.8.tar.bz2
cd multipath-tools-0.4.8/
make
make install

I did compile them when running the 2.6.22.2 kernel though. Should I
recompile them when running the 2.6.23.1 kernel, or doesn't that matter?

>
> Then start multipathd from a shell that has core dumps enabled (make
> sure it's not already running):
>
> $ ulimit -c unlimited
> $ multipathd
>
> Wait for a segfault to happen, and you should have gotten a file named
> "core" in your current directory (or maybe in the root directory since
> multipathd chdir()s there, I'm not sure) that contains the backtrace.
> We need to extract it with GDB, though, so run:
>
> $ gdb /sbin/multipathd /path/to/core
> (gdb) bt full

Oke, the segfault happens right at the start (and only at the start), so
that shouldn't be a problem. But I have to stop the running mulipathd
and that seems like a problem. This machine is running in production

So that will have to wait until I get a maintance window I guess...

>
> ...and post the resulting backtrace here. Hopefully a developer (not
> me, unfortunately) will be able to make sense of it.
>
> Another thing you might want to try is to use your 2.6.23.1 with the
> same configuration as your 2.6.22.x (ie. don't use hwhandler rdac), and
> see if the segfaults still occur.
>
> Regards

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-20-2007, 10:15 AM
Tore Anderson
 
Default [dm-devel] Possible bug in multipathd (getting a segfault)

* S. J. van Harmelen

> I guess I don't have to do that as I compiled the multipath-tools
> myself?

Then you should have the debugging symbols in place, yes.

> I did compile them when running the 2.6.22.2 kernel though. Should I
> recompile them when running the 2.6.23.1 kernel, or doesn't that
> matter?

Doesn't matter.

> Oke, the segfault happens right at the start (and only at the start),
> so that shouldn't be a problem. But I have to stop the running
> mulipathd and that seems like a problem. This machine is running in
> production
>
> So that will have to wait until I get a maintance window I guess...

Stopping multipathd isn't problematic. It is only responsible for
periodically testing paths to pre-emptively fail non-active paths or
re-instate paths that has been failed by the kernel's dm-multipath layer.

If everything is running stable with all paths alive and well,
restarting multipathd is nothing to worry about.

Regards
--
Tore Anderson

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-20-2007, 10:25 AM
"S. J. van Harmelen"
 
Default [dm-devel] Possible bug in multipathd (getting a segfault)

On Tue, 2007-11-20 at 12:15 +0100, Tore Anderson wrote:
> * S. J. van Harmelen
>
> > I guess I don't have to do that as I compiled the multipath-tools
> > myself?
>
> Then you should have the debugging symbols in place, yes.
>
> > I did compile them when running the 2.6.22.2 kernel though. Should I
> > recompile them when running the 2.6.23.1 kernel, or doesn't that
> > matter?
>
> Doesn't matter.
>
> > Oke, the segfault happens right at the start (and only at the start),
> > so that shouldn't be a problem. But I have to stop the running
> > mulipathd and that seems like a problem. This machine is running in
> > production
> >
> > So that will have to wait until I get a maintance window I guess...
>
> Stopping multipathd isn't problematic. It is only responsible for
> periodically testing paths to pre-emptively fail non-active paths or
> re-instate paths that has been failed by the kernel's dm-multipath layer.
>
> If everything is running stable with all paths alive and well,
> restarting multipathd is nothing to worry about.

Oke, then I will give this a try this evening and post the results here
for de developers...

Thanks!

>
> Regards

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 
Old 11-20-2007, 10:28 AM
"S. J. van Harmelen"
 
Default [dm-devel] Possible bug in multipathd (getting a segfault)

On Tue, 2007-11-20 at 12:15 +0100, Tore Anderson wrote:
> * S. J. van Harmelen
>
> > I guess I don't have to do that as I compiled the multipath-tools
> > myself?
>
> Then you should have the debugging symbols in place, yes.
>
> > I did compile them when running the 2.6.22.2 kernel though. Should I
> > recompile them when running the 2.6.23.1 kernel, or doesn't that
> > matter?
>
> Doesn't matter.
>
> > Oke, the segfault happens right at the start (and only at the start),
> > so that shouldn't be a problem. But I have to stop the running
> > mulipathd and that seems like a problem. This machine is running in
> > production
> >
> > So that will have to wait until I get a maintance window I guess...
>
> Stopping multipathd isn't problematic. It is only responsible for
> periodically testing paths to pre-emptively fail non-active paths or
> re-instate paths that has been failed by the kernel's dm-multipath layer.
>
> If everything is running stable with all paths alive and well,
> restarting multipathd is nothing to worry about.

Oeps... How stupid of me! I'm running the 2.6.22.2 kernel now, so I will
have to reboot to get kernel 2.6.23.1 running.

So I will have to wait a few days for this...

Thanks again

>
> Regards

--
dm-devel mailing list
dm-devel@redhat.com
https://www.redhat.com/mailman/listinfo/dm-devel
 

Thread Tools




All times are GMT. The time now is 11:51 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org