FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Gentoo > Gentoo User

 
 
LinkBack Thread Tools
 
Old 12-17-2009, 02:07 AM
Tom Bennet
 
Default Broken upgrade from udev troubles.

I spent the day recovering from a Gentoo upgrade, and thought I'd document the experience in case it helps someone else.

I'm running a custom kernel 2.6.25-gentoo-r7 on amd64, though I don't think the rarer hardware is relevant.


I tend to put off upgrading my Gentoo box because anytime I do, something breaks.* I'm afraid I haven't changed my opinion about that.* Anyway, I did "emerge --update --deep world" and plugged my ears. Some 600-odd packages (and a few simpler problems) later, the system seemed to be doing okay.* So I thought I'd see if it could survive a reboot.* No, it couldn't.


On boot it failed checking the root file system and dropped into the repair shell.* The reason the fsck failed is that the root pseudo device file /dev/md0, didn't exist.* The root file system was actually, fine, though.* Inside the repair shell, I could see all the files from my root, but there wasn't much in /dev.* (I have the md stuff compiled in to the kernel, and don't use an initrd, so it wasn't an initrd problem.)


Short Solution

The problem was with udev, the facility which automatically populates the /dev directory.* During the upgrade, emerge noted that my kernel version was a bit early, but acceptable.* What was missing, apparently, was the signalfd syscall, which that kernel version either doesn't have or I hadn't configured.* Apparently, udev has only started using signalfd recently, so the solution was to downgrade to an older version of udev (udev-141 to be precise).


What I Actually Did To Get There

Of course, I didn't know that at first.* Just had a fun unbootable system.* I might have been able to simply emerge the downgrade from the repair shell (the network did come up), but I didn't know to try that yet.* I figured I wanted to find some way to make the system boot.* Since the failing file check is done from /etc/init.d/checkroot, I added a mknod command to create the device node before trying to run the file check.* At the start of the start() method:


******* if [ ! -e /dev/md0 ] ; then
********** mknod -m 0660 /dev/md0 b 9 0
******* fi

It's a hack, not a solution, but it did make the system boot, to a rather crippled state.* Since there were a lot of devices missing, a lot of services wouldn't start.* (If you're using a more boring root partition, it might be something like "mknod -m 0660 /dev/sda1 b 8 1")


So I had managed by now to gather that udev wasn't working, but I didn't know why.* My first thought was to try "/etc/init.d/udev start", to see if it would start.* But it told me that the script is written for baselevel-2, and I shouldn't use it on baselevel-1.* Following a bit of googling about what the heck a baselevel is, I gathered that I was using baselevel-1, and so the service wasn't supposed to be started that way.** So it wasn't a bug that it wouldn't start that way.* Another page suggested trying to run it directly, with "/sbin/udevd --daemon", which gave the message "error getting signalfd".* That told my why it didn't start. This message was also in the logs, but for some reason I didn't look there until later.


So back to Google, and I found a message on a Debian board noting that udev had started using signalfd recently.* This suggested an old version might do the trick.* I tried one, and it did.
 
Old 12-17-2009, 04:23 AM
Joshua Murphy
 
Default Broken upgrade from udev troubles.

On Wed, Dec 16, 2009 at 10:07 PM, Tom Bennet <twbennet@gmail.com> wrote:
> I spent the day recovering from a Gentoo upgrade, and thought I'd document
> the experience in case it helps someone else.
>
> I'm running a custom kernel 2.6.25-gentoo-r7 on amd64, though I don't think
> the rarer hardware is relevant.
>
> I tend to put off upgrading my Gentoo box because anytime I do, something
> breaks.* I'm afraid I haven't changed my opinion about that.* Anyway, I did
> "emerge --update --deep world" and plugged my ears. Some 600-odd packages
> (and a few simpler problems) later, the system seemed to be doing okay.* So
> I thought I'd see if it could survive a reboot.* No, it couldn't.
>
> On boot it failed checking the root file system and dropped into the repair
> shell.* The reason the fsck failed is that the root pseudo device file
> /dev/md0, didn't exist.* The root file system was actually, fine, though.
> Inside the repair shell, I could see all the files from my root, but there
> wasn't much in /dev.* (I have the md stuff compiled in to the kernel, and
> don't use an initrd, so it wasn't an initrd problem.)
>
> Short Solution
>
> The problem was with udev, the facility which automatically populates the
> /dev directory.* During the upgrade, emerge noted that my kernel version was
> a bit early, but acceptable.* What was missing, apparently, was the signalfd
> syscall, which that kernel version either doesn't have or I hadn't
> configured.* Apparently, udev has only started using signalfd recently, so
> the solution was to downgrade to an older version of udev (udev-141 to be
> precise).
>
> What I Actually Did To Get There
>
> Of course, I didn't know that at first.* Just had a fun unbootable system.
> I might have been able to simply emerge the downgrade from the repair shell
> (the network did come up), but I didn't know to try that yet.* I figured I
> wanted to find some way to make the system boot.* Since the failing file
> check is done from /etc/init.d/checkroot, I added a mknod command to create
> the device node before trying to run the file check.* At the start of the
> start() method:
>
> ******* if [ ! -e /dev/md0 ] ; then
> ********** mknod -m 0660 /dev/md0 b 9 0
> ******* fi
>
> It's a hack, not a solution, but it did make the system boot, to a rather
> crippled state.* Since there were a lot of devices missing, a lot of
> services wouldn't start.* (If you're using a more boring root partition, it
> might be something like "mknod -m 0660 /dev/sda1 b 8 1")
>
> So I had managed by now to gather that udev wasn't working, but I didn't
> know why.* My first thought was to try "/etc/init.d/udev start", to see if
> it would start.* But it told me that the script is written for baselevel-2,
> and I shouldn't use it on baselevel-1.* Following a bit of googling about
> what the heck a baselevel is, I gathered that I was using baselevel-1, and
> so the service wasn't supposed to be started that way.** So it wasn't a bug
> that it wouldn't start that way.* Another page suggested trying to run it
> directly, with "/sbin/udevd --daemon", which gave the message "error getting
> signalfd".* That told my why it didn't start. This message was also in the
> logs, but for some reason I didn't look there until later.
>
> So back to Google, and I found a message on a Debian board noting that udev
> had started using signalfd recently.* This suggested an old version might do
> the trick.* I tried one, and it did.

I really only have two things to say, after reading this... First, and
this really does overshadow the second in weight, thank you for the
excellently presented writeup of problem *and* solution, as more often
than ever should be (less so here, but across the net as a whole),
problems are mentioned, solutions are offered, and rarely does a good,
clear, "this worked" follow. Secondly... it's been my experience, with
Gentoo, that things break far more often when I allow longer delays
between updating than when I keep up to date with everything, and it's
held true for me on both x86 and ~x86 systems (as has the headache
when I've put updates off).

And.. I reiterate a part of the "first"... Thank you for the writeup.

--
Poison [BLX]
Joshua M. Murphy
 

Thread Tools




All times are GMT. The time now is 08:22 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org