Linux Archive - System problems

Linux Archive (http://www.linux-archive.org/)

- Gentoo Desktop (http://www.linux-archive.org/gentoo-desktop/)

- - System problems (http://www.linux-archive.org/gentoo-desktop/503149-system-problems.html)

I'm caught between a rock and a hard place.

I've been running this desktop box using kernel 2.6.23-gentoo-r3 and
have come to the point at which there are too many dependencies and
reverse dependencies, so I _have_ to upgrade to kernel 2.6.29-gentoo-r5
and have been unable to bring the system up in the new kernel. Here's
what's happening.

The newer kernel requires a newer version of udev, which I emerged. The
system came up with a root device of some sort mounted, I think in
single-user mode, but couldn't mount other devices.

So I changed the main drive designations to UUID's in /etc/fstab,
re-emerged the newer udev, and tried again. This time I got a message
that the kernel needed a root parameter at boot time. It seems that all
my /dev/hda? drives have been renamed /dev/sda? so I set gave
"root=/dev/sda4" as a kernel parameter and got a little further. After
"Checking root filesystem" in the boot sequence, I got a message that
the UUID for the root filesystem wasn't understood in /etc/fstab.

So I set the root filesystem in /etc/fstab to /dev/sda4, and got the
same error - that "/dev/sda4" was not understood either, although the
kernel seemed to understand this just fine as a boot parameter, and once
again, I'm dumped into a very limited single-user mode.

So I'm stuck! I had to boot from a rescue disk, back-version to
udev-141 and revert to kernel 2.6.23-gentoo-r3 to get my desktop back.

What do I need to put into /etc/fstab to satisfy the kernel? I need to
move forward with this, but I need my desktop system to run my business.
Any _real_ suggestions will be welcome. Please be aware that I'm no
Linux novice, so don't give me novice advice. I've been building,
running, and getting paid to admin Linux systems since 1995.

--
Lindsay Haisley | "Everything works if you let it"
FMP Computer Services | (The Roadie)
512-259-1190 |
http://www.fmp.com |

Hi,
This is due to ATA/ATAPI (DEPRECATED) being disabled in newer kernels, replaced by Serial ATA and Parallel ATA drivers.
Make sure you enabled this support properly.

In my case that happened to me as well, on a remote computer, which was my mother's box...
Anyway, in fstab /dev/sdXX shoud work, at least I made this change on a couple of machines and that went fine.

Cheers,JM

On Sun, Mar 20, 2011 at 4:46 AM, Lindsay Haisley <fmouse-gentoo@fmp.com> wrote:

I'm caught between a rock and a hard place.

I've been running this desktop box using kernel 2.6.23-gentoo-r3 and

have come to the point at which there are too many dependencies and

reverse dependencies, so I _have_ to upgrade to kernel 2.6.29-gentoo-r5

and have been unable to bring the system up in the new kernel. *Here's

what's happening.

The newer kernel requires a newer version of udev, which I emerged. *The

system came up with a root device of some sort mounted, I think in

single-user mode, but couldn't mount other devices.

So I changed the main drive designations to UUID's in /etc/fstab,

re-emerged the newer udev, and tried again. *This time I got a message

that the kernel needed a root parameter at boot time. *It seems that all

my /dev/hda? drives have been renamed /dev/sda? so I set gave

"root=/dev/sda4" as a kernel parameter and got a little further. *After

"Checking root filesystem" in the boot sequence, I got a message that

the UUID for the root filesystem wasn't understood in /etc/fstab.

So I set the root filesystem in /etc/fstab to /dev/sda4, and got the

same error - that "/dev/sda4" was not understood either, although the

kernel seemed to understand this just fine as a boot parameter, and once

again, I'm dumped into a very limited single-user mode.

So I'm stuck! *I had to boot from a rescue disk, back-version to

udev-141 and revert to kernel 2.6.23-gentoo-r3 to get my desktop back.

What do I need to put into /etc/fstab to satisfy the kernel? *I need to

move forward with this, but I need my desktop system to run my business.

Any _real_ suggestions will be welcome. *Please be aware that I'm no

Linux novice, so don't give me novice advice. *I've been building,

running, and getting paid to admin Linux systems since 1995.

--

Lindsay Haisley * * * | "Everything works if you let it"

FMP Computer Services | * (The Roadie)

512-259-1190 * * * * *|

http://www.fmp.com * *|

--
Jean-Marc

Please send us your `emerge --info`, /etc/fstab, grub/lilo/smthng
config and kernel config for the 2.6.29.

By the way, why not gentoo-sources-2.6.36-r5? I hope you have a good
reason to run a system as ancient as that. Your system is swarming with
widely known security holes. I suppose it's theoretically possible to
have a desktop protected by some special means from all the real threats
an actively used desktop is facing, but I wonder if you use any such
means. Also, by upgrading to a little less ancient versions than 2.6.29
you won't have the same situation like now boomerang back at you in the
near future.

-rz

Lindsay Haisley (Sat, 19 Mar 2011 23:46:06 -0500):
> I'm caught between a rock and a hard place.
>
> I've been running this desktop box using kernel 2.6.23-gentoo-r3 and
> have come to the point at which there are too many dependencies and
> reverse dependencies, so I _have_ to upgrade to kernel 2.6.29-gentoo-r5
> and have been unable to bring the system up in the new kernel. Here's
> what's happening.
>
> The newer kernel requires a newer version of udev, which I emerged. The
> system came up with a root device of some sort mounted, I think in
> single-user mode, but couldn't mount other devices.
>
> So I changed the main drive designations to UUID's in /etc/fstab,
> re-emerged the newer udev, and tried again. This time I got a message
> that the kernel needed a root parameter at boot time. It seems that all
> my /dev/hda? drives have been renamed /dev/sda? so I set gave
> "root=/dev/sda4" as a kernel parameter and got a little further. After
> "Checking root filesystem" in the boot sequence, I got a message that
> the UUID for the root filesystem wasn't understood in /etc/fstab.
>
> So I set the root filesystem in /etc/fstab to /dev/sda4, and got the
> same error - that "/dev/sda4" was not understood either, although the
> kernel seemed to understand this just fine as a boot parameter, and once
> again, I'm dumped into a very limited single-user mode.
>
> So I'm stuck! I had to boot from a rescue disk, back-version to
> udev-141 and revert to kernel 2.6.23-gentoo-r3 to get my desktop back.
>
> What do I need to put into /etc/fstab to satisfy the kernel? I need to
> move forward with this, but I need my desktop system to run my business.
> Any _real_ suggestions will be welcome. Please be aware that I'm no
> Linux novice, so don't give me novice advice. I've been building,
> running, and getting paid to admin Linux systems since 1995.
>

Lindsay Haisley posted on Sat, 19 Mar 2011 23:46:06 -0500 as excerpted:

> I'm caught between a rock and a hard place.
>
> I've been running this desktop box using kernel 2.6.23-gentoo-r3 and
> have come to the point at which there are too many dependencies and
> reverse dependencies, so I _have_ to upgrade to kernel 2.6.29-gentoo-r5
> and have been unable to bring the system up in the new kernel. Here's
> what's happening.

You're still running 2.6.23 on a /desktop/? I can see how it could be
argued for a server that had /years/ of uptime, but for a desktop... I
/really/ think you should upgrade a bit more often (as it would seem
you're unfortunately finding out)...

> The newer kernel requires a newer version of udev, which I emerged. The
> system came up with a root device of some sort mounted, I think in
> single-user mode, but couldn't mount other devices.
>
> So I changed the main drive designations to UUID's in /etc/fstab,
> re-emerged the newer udev, and tried again. This time I got a message
> that the kernel needed a root parameter at boot time. It seems that all
> my /dev/hda? drives have been renamed /dev/sda? so I set gave
> "root=/dev/sda4" as a kernel parameter and got a little further.

Yes. The old IDE subsystem used hdX device names. The newer libata based
subsystem handles both PATA and SATA, using the traditionally SCSI device
names of sdX instead of the traditionally IDE names of hdX.

Of course, that's years'-old news by now, for anyone taking the "rolling"
in rolling update at anything close to face value... Seriously. If you
/are/ going to let things get that outdated, I'd recommend something like
CentOS. At least there, such big updates so far apart are semi-standard,
and thus, both the distribution itself and its community are better
equipped to handle them.

Gentoo, OTOH, is a /rolling/ /update/ distribution. They actually find it
necessary to warn people not to (routinely) sync more than once a day, and
updating daily to perhaps at the outside monthly, really is how it works
best. I believe you've seen this "sermon" before so I'll keep it short,
but honestly, I'm not just saying this, if you're not going to go rolling
update with it, and rolling update is NOT for everybody, you really
/should/ reevaluate whether Gentoo is the really the best matching choice
for your needs. An "impedance mismatch" of that magnitude may be possible
to work around, but just because it's possible doesn't mean it's the most
efficient choice available, for sure.

(The two following paragraphs are recent personal experience with an
8-month-out update. Skip if you wish...

FWIW, I just upgraded my netbook from about 8 months out, kde 4.4.5,
skipping 4.5 entirely, to 4.6.1. I was able to do it because I understand
reasonably well both the kernel and the Gentoo distribution as well as the
hardware, following developments quite closely even when I'm not actually
updating, AND because I kept the usual incremental rolling updates on my
workstation during the period, so I'd done all except the hardware-
specific updates before, just not all at once, but I'll tell you what, I'd
have a whole lot rather kept it updated at least every 2-3 months maximum.

... Further, while it's probably fair to say it was easier for me to do
the 8-month update than to backup /etc, /home and the kernel config, and
start from a clean stage-3 tarball, it wasn't so by much, and for people
who don't keep such close track of Gentoo and Linux developments in
general and/or who hadn't been regularly updating other systems so as to
have gone thru most of the updates already, just not all at once, starting
from that clean stage-3 probably would have been easier! And I can say
that as I recently helped a friend install from a stage-3, too, so I've
done both recently. Thus, I now have personal experience to confirm what
has been previously stated in other threads, that a six-month update is
approaching the outer limits of practicality for an ordinary Gentooer, and
that somewhere between that six months and a year, installing from a fresh
stage-3 does become the most practical option.)

> After
> "Checking root filesystem" in the boot sequence, I got a message that
> the UUID for the root filesystem wasn't understood in /etc/fstab.

> So I set the root filesystem in /etc/fstab to /dev/sda4, and got the
> same error - that "/dev/sda4" was not understood either, although the
> kernel seemed to understand this just fine as a boot parameter, and once
> again, I'm dumped into a very limited single-user mode.
>
> So I'm stuck! I had to boot from a rescue disk, back-version to
> udev-141 and revert to kernel 2.6.23-gentoo-r3 to get my desktop back.
>
> What do I need to put into /etc/fstab to satisfy the kernel? I need to
> move forward with this, but I need my desktop system to run my business.
> Any _real_ suggestions will be welcome. Please be aware that I'm no
> Linux novice, so don't give me novice advice. I've been building,
> running, and getting paid to admin Linux systems since 1995.

Do you run an initrd/initramfs? (AFAIK you do if you use genkernel, since
it pretty much compiles everything as modules, putting what it thinks
necessary to load the real-root in the initrd/initramfs.)

I'll skip the detail on init(rd|ramfs) both because I don't use it myself
and because as you mention, you /aren't/ a novice, so if you do use one,
you can probably tell me more about how it works than I can tell you, but..

Either...

1) it's loading the kernel and dropping you in the initrd/initramfs,
because it can't load real-root,

OR

2) It's loading root initially (either you don't have an initrd/initramfs
or it gets past that), which normally defaults to read-only mode, and is
failing when it tries to do the initial rootfs fsck and/or when it tries
to do the remount read/write, which occurs just after that.

In either case, since you specifically mention that you get a limited
single-user-mode, NOT a kernel panic as it tries to read the initial root
(whether that's an initrd/initramfs or the real root), the kernel
definitely DOES know how to read and mount that initial root and drop you
into /some/ sort of user mode, even if limited single-user.

(It's worth noting here that one of the issues I experienced with that
8-month update above, was that somehow, when I updated the kernel from
2.35-rc6-gitsomething, what I'd been running before, to 2.6.38, what I'm
running now on both workstation and netbook, the config option for my SATA
chipset vanished when I did make oldconfig, and my first kernel rebuild
panicked when it couldn't find root. I'm familiar enough with the
kernel's boot messages and the hardware that I realized that it couldn't
see sda at all, which meant either the hardware was dying and that didn't
seem to be the case, or it wasn't loading the driver, so I deduced the
problem almost immediately, and sure enough, when I checked the kernel
config, it wasn't building that driver (I don't use modules, only the
options I need, built-in), so change the option to build it, rebuild and
install the kernel, and try again. It worked once it had the driver for
the sata controller, but the point is, I use all built-ins and no init(rd|
ramfs), so it kernel panicked before loading any userspace at all.)

If you do NOT use an initr*, the fact that it's loading even the single-
user userspace indicates that it not only properly loads the PATA driver
and detects the hardware, but that it ALSO groks the rootfs and loads it
read-only. In that case, your problem really is one with fsck/mount/fstab
or the scripts that invoke them.

OTOH, if you DO use an initr*, the first thing to determine is whether
it's getting past that and dropping you into the real-root, in which case
you have the same as above fsck/mount/fstab or script issues to look at,
*OR*, whether you're getting dropped into that limited userspace while
it's still initr*-based-only, in which case your problem is with the
initr*, NOT with the real-root that the system later pivot-root-s to.

Given the info you posted, I'll predict that the most likely case, and I
state this again with the caveat that I don't use initr* personally so
don't know that much about it, is that you DO use an initr*, and are stuck
in it, as it either doesn't have the correct kernel PATA driver and thus
can't see the drive at all, or it has that but is missing the correct
filesystem module, so it sees the drive but can't understand the
filesystem you're pointing it at.

From what I've /read/ about initr*, back when initrds were the norm, the
most common problem here was that either the updated kernel entry line
still pointed at the old initrd, or that the initrd hadn't been updated at
all. In either case, the actually loaded initrd had the modules for the
old kernel, which the new kernel would refuse to load.

Initramfs made that problem much less common as the initramfs is now
appended to the end of the kernel file making it far more difficult to
have a kernel/initr* mismatch, but the same effect sometimes happens, if
the new kernel build doesn't build the appropriate modules or for whatever
reason they're not included in the initramfs.

But beyond that you'll need to find someone with more initr* knowledge to
help if it's an initr* issue, which I suspect it is. It's a reasonably
common problem, but one I have no experience with and don't know the
details of the fix on.

If you're actually getting your real-root loaded, either because you don't
use an initr* or because it's correct and you're successfully getting the
pivot-root, then as I said, it would appear to be a problem with fsck,
mount, fstab, or possibly the initscripts invoking them. However, those
are less common and there's not enough data in the original post to really
go further with it at this point.

But I bet (and hope) it's an initr* problem... simply because those are
most common, and normally easy to solve -- simply make sure you're loading
the one corresponding to the new kernel and that it has the correct
drivers and config...

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman

On Sun, 20 Mar 2011, Jean-Marc Beaune wrote:

This is due to ATA/ATAPI (DEPRECATED) being disabled in newer kernels,
replaced by Serial ATA and Parallel ATA drivers.

Make sure you enabled this support properly.

In my case that happened to me as well, on a remote computer, which was my
mother's box...

Anyway, in fstab /dev/sdXX shoud work, at least I made this change on a
couple of machines and that went fine.

Yes, I went through this also awhile back. All your drivers should now
come from the Sata section, both the Sata drivers for your hard drives,
plus the Pata drivers for any Pata DVD drives you may have. There are
now Pata support drivers living in the Sata section. You should turn
off the toggle for the entire old Pata driver section in menuconfig.
If you have only Pata drives, you should just use the Pata drivers from
the Sata section by themselves. No need for anything in the old Pata
section in any case anymore.

The newer Sata subsystem Pata drivers had been included before in the
kernel for a long time as deprecated drivers for people to beta test,
and the regular Pata drivers were recommended for use. Now it's
reversed -- the Pata drivers in the Sata section are recommended, and
the old Pata drivers from the Pata section are deprecated...expecially
because they don't work anymore with current versions of udev!

--
+ Brent A. Busby + "We've all heard that a million monkeys
+ UNIX Systems Admin + banging on a million typewriters will
+ University of Chicago + eventually reproduce the entire works of
+ Physical Sciences Div. + Shakespeare. Now, thanks to the Internet,
+ James Franck Institute + we know this is not true." -Robert Wilensky

On Sun, 2011-03-20 at 09:50 +0100, Roman Zilka wrote:
> By the way, why not gentoo-sources-2.6.36-r5? I hope you have a good
> reason to run a system as ancient as that. Your system is swarming with
> widely known security holes.

Yes, I have a good reason for this, and I'll be responsible for security
on the box, which I'm well able to do. Security, and the relative
antiquity of the portage tree are not issues for which I'm seeking
advice, so your observations are totally beside the point, and not
particularly welcome. I'm well aware of both issues.

The challenge here is to get the system to boot with the newer kernel
and version of udev which I cited in my previous post.

/etc/fstab has been edited several times, as I noted in my post. The
kernel, udev and /etc/fstab have been now been reverted, as I also noted, so
I could get the desktop working. Considering that, posting any of the
information you've asked for would probably be useless.

Roman, if you don't have any useful insights based on the information I
already posted, then please don't post on thread and leave it to others
who may.

Having said this, I'll ask you one question:

> Also, by upgrading to a little less ancient versions than 2.6.29
> you won't have the same situation like now boomerang back at you in the
> near future.

Can you cite a source or sources for this assertion? Is there a known
problem with kernel 2.6.29, or the portage tree which spec'd that
kernel? Is there any discussion, bug report, or anything that you can
cite noting that this was a known problem at one point and addressed at
a later date?

In almost every case, I've found that people who lecture me online about
my system admin practices don't really have a handle on the issue about
which I'm writing. Please prove me wrong :-)

--
Lindsay Haisley |"What if the Hokey Pokey really IS all it
FMP Computer Services | really is about?"
512-259-1190 |
http://www.fmp.com | -- Jimmy Buffett

--
Lindsay Haisley |"Windows .....
FMP Computer Services | life's too short!"
512-259-1190 |
http://www.fmp.com | - Brad Johnston

On Sun, 2011-03-20 at 08:24 +0000, Jean-Marc Beaune wrote:
> This is due to ATA/ATAPI (DEPRECATED) being disabled in newer kernels,
> replaced by Serial ATA and Parallel ATA drivers.
>
> Make sure you enabled this support properly.
>
> In my case that happened to me as well, on a remote computer, which
> was my mother's box...
>
> Anyway, in fstab /dev/sdXX shoud work, at least I made this change on
> a couple of machines and that went fine.

Jean-Marc, thanks. This gives me a bit of insight.

I've been compiling kernels for this box for some time, carrying over
the .config file between kernel versions and updating them. Last night
I built IDE functionality as a module (it was previously compiled into
the kernel, as per my carried-over .config files) thinking that I would
simply not load it at run-time. As I noted in my post, while the the
kernel recognized "root=/dev/sda4" as a kernel param, the boot process
crashed with a note that "/dev/sda4" wasn't recognized. It's entirely
possible that this module got auto-loaded, or that there's some other
anomaly in my kernel config that's throwing things off.

My next step is to completely rebuild the kernel, using the config built
into the distributed kernel source, making only the necessary mods for
the box's hardware needs. I'll find a time window during the next few
days to work on this.

--
Lindsay Haisley |"Windows .....
FMP Computer Services | life's too short!"
512-259-1190 |
http://www.fmp.com | - Brad Johnston

On Sun, 2011-03-20 at 09:32 +0000, Duncan wrote:
> Gentoo, OTOH, is a /rolling/ /update/ distribution. They actually find it
> necessary to warn people not to (routinely) sync more than once a day, and
> updating daily to perhaps at the outside monthly, really is how it works
> best. I believe you've seen this "sermon" before so I'll keep it short,
> but honestly, I'm not just saying this, if you're not going to go rolling
> update with it, and rolling update is NOT for everybody, you really
> /should/ reevaluate whether Gentoo is the really the best matching choice
> for your needs. An "impedance mismatch" of that magnitude may be possible
> to work around, but just because it's possible doesn't mean it's the most
> efficient choice available, for sure.

Duncan, thank you. I'm well aware of the advantages and disadvantages
of Gentoo's "rolling update" distribution design, and really appreciate
the advantages. My problem isn't knowledge, of which I have enough, but
time. Maintaining a Gentoo system is a relatively time consuming
activity, and I'm spread extremely thin these days. This problem box
has been running Gentoo since 2004, but my work and priorities on my
time have changed over the years and it's probably time to move on.
Gentoo has gotten a lot more complex since then - arguably better, but
certainly more time-consuming to maintain.

In the meantime, I need to address the kernel | fstab | udev issue here.
Insights and suggestions to address _this_ problem would be most
helpful. Lectures on the age of my system, my qualifications as an
admin for my own desktop, or my continued us of Gentoo as a desktop
distribution are not helpful.

--
Lindsay Haisley |"Windows .....
FMP Computer Services | life's too short!"
512-259-1190 |
http://www.fmp.com | - Brad Johnston

On Sun, 2011-03-20 at 11:50 -0500, Lindsay Haisley wrote:
> This problem box
> has been running Gentoo since 2004, but my work and priorities on my
> time have changed over the years and it's probably time to move on.

To be a bit more precise here, I'm actively working on replacing this
box, and will be running a Linux distribution on it other than Gentoo.
But in the meantime, I do need to solve the problem about which I
posted.

--
Lindsay Haisley |"Friends are like potatoes.
FMP Computer Services | If you eat them, they die"
512-259-1190 |
http://www.fmp.com | - Aaron Edmund

Lindsay Haisley posted on Sun, 20 Mar 2011 13:48:43 -0500 as excerpted:

> On Sun, 2011-03-20 at 11:50 -0500, Lindsay Haisley wrote:
>> This problem box has been running Gentoo since 2004, but my work and
>> priorities on my time have changed over the years and it's probably
>> time to move on.
>
> To be a bit more precise here, I'm actively working on replacing this
> box, and will be running a Linux distribution on it other than Gentoo.
> But in the meantime, I do need to solve the problem about which I
> posted.

I appreciate that you /have/ decided to ultimately switch, and obviously
believe it best or I'd have not recommend it.

I also appreciate that you need a solution for now. That's what I suspect
I may have provided (at least to a point) further down that post, which
you may not have read given the order.

I was hoping to read all these replies and have you tell me whether I was
on the right track or not, answering the question about initrd/initramfs
so we could go from there if need be. However, I don't see that answer,
which suggests you didn't read that far down my post, for which I can't
really blame you.

But you can always go back and do it now... =:^)

Briefly repeating so you can see if it's worth your trouble going back to
get the details.

If the kernel could not load its initial root, be that initr* or the real-
root, it'd fail to load any userspace at all as it couldn't get to it,
panicking instead when it couldn't get to a root at all. (I know this
from experience.)

Thus, that it's getting even a /limited/ userspace indicates it's getting
to it's initial root. That's a very significant point.

The question from there is whether that limited userspace is loading from
the initrd/initramfs if you have one, indicating it's not successfully
doing the initial real-root mount and pivot-root from the initr*, taking
you down one path toward a solution (missing or incorrect drivers in the
initr* or wrong initrd), or whether it's loading the real-root (as just
getting to userspace at all indicates if you don't have an initr*), thus
indicating that it has the necessary drivers (both pata and fs) to do so,
and the problem is later, with the fsck or remount, taking you down a
different path toward a solution (userspace issue).

If that sounds useful, check the earlier post for more. If it doesn't,
don't bother, as that's what I was detailing in the earlier post.

--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman