FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian Kernel

 
 
LinkBack Thread Tools
 
Old 03-19-2009, 07:13 PM
maximilian attems
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

[ adding jcm and lkml to Cc: ]



On Thu, Mar 19, 2009 at 07:40:46PM +0100, Frans Pop wrote:
>
> Hmmm. I wonder if it is the old m-i-t's modprobe that is the problem when
> you do:
> modprobe --set-version=2.6.26.3 --ignore-install --show-depends <module>
>
> Looks like that's it:
> # modprobe -V
> module-init-tools version 3.4
> # modprobe --set-version=2.6.26.3 --ignore-install --show-depends nfs
> WARNING: Could not open 'kernel/net/sunrpc/sunrpc.ko': No such file or
> directory
> WARNING: Could not open 'kernel/fs/nfs_common/nfs_acl.ko': No such file or
> directory
> WARNING: Could not open 'kernel/fs/lockd/lockd.ko': No such file or
> directory
> FATAL: Could not open 'kernel/fs/nfs/nfs.ko': No such file or directory
>
> # modprobe -V
> module-init-tools version 3.7-pre9
> # modprobe --set-version=2.6.26.3 --ignore-install --show-depends nfs
> WARNING: All config files need .conf: /etc/modprobe.d/pnp-hotplug, it will
> be ignored in a future release.
> WARNING: All config files need .conf: /etc/modprobe.d/display_class, it
> will be ignored in a future release.
> WARNING: All config files need .conf: /etc/modprobe.d/blacklist, it will
> be ignored in a future release.
> insmod /lib/modules/2.6.26.3/kernel/net/sunrpc/sunrpc.ko
> insmod /lib/modules/2.6.26.3/kernel/fs/nfs_common/nfs_acl.ko
> insmod /lib/modules/2.6.26.3/kernel/fs/lockd/lockd.ko
> insmod /lib/modules/2.6.26.3/kernel/fs/nfs/nfs.ko
>
> That would mean that m-i-t has created a backwards incompatibility problem
> _with itself_ and that the problem actually is "installing a kernel, that
> was built on a system with new m-i-t, on a system with old m-i-t".
> Or, installing a kernel, built on a system running unstable or testing, on
> a system running oldstable or stable. That sucks.

argh indeed.

arguably it is also a bug by initramfs-tools to suppress the error of above
modprobe calls see manual_add_modules() in hook-functions:
for mam_x in $(modprobe --set-version="${version}" --ignore-install
--show-depends "${1}" 2>/dev/null | awk '/^insmod/ { print $2 }'); do


git history tells that jbailey added it from the start on,
so it must have been because modprobe was/is too chatty.
suse mkinitrd seems also only to have lost that suppression lately.



thanks for finding out fjp.

kind regards

--
maks



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-19-2009, 07:30 PM
Jon Masters
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

On Thu, 2009-03-19 at 21:13 +0100, maximilian attems wrote:
> [ adding jcm and lkml to Cc: ]

[ You want linux-modules@vger.kernel.org rather than LKML. I've added
the former to the CC list, we can kill LKML off the CC shortly. ]

> > That would mean that m-i-t has created a backwards incompatibility problem
> > _with itself_ and that the problem actually is "installing a kernel, that
> > was built on a system with new m-i-t, on a system with old m-i-t".

Looks like the problem is actually that depmod was run under the newer
version and then you tried to use the generated files with an older
modprobe. I'm not sure that's actually an error - it was noted that the
slight format change was unideal for such unlikely cases and in fact we
won't do that again in the future. But if you were just moving forwards
from one release to the next you would have been fine - you're talking
lack of forward compatibility actually.

Unless I'm missing something, in which case please clarify. I would like
it also if you'd send me the generated files, distro info, etc.

Jon.





--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-19-2009, 07:57 PM
Frans Pop
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

(lkml dropped)

On Thursday 19 March 2009, Jon Masters wrote:
> On Thu, 2009-03-19 at 21:13 +0100, maximilian attems wrote:
> > > That would mean that m-i-t has created a backwards incompatibility
> > > problem _with itself_ and that the problem actually is "installing
> > > a kernel, that was built on a system with new m-i-t, on a system
> > > with old m-i-t".
>
> Looks like the problem is actually that depmod was run under the newer
> version and then you tried to use the generated files with an older
> modprobe. I'm not sure that's actually an error - it was noted that the
> slight format change was unideal for such unlikely cases and in fact we
> won't do that again in the future. But if you were just moving forwards
> from one release to the next you would have been fine - you're talking
> lack of forward compatibility actually.

The use case here, which I suspect is not all that uncommon, is that I
built a kernel from upstream source on a (Debian unstable) system with
the new version of depmod and then installed that kernel on a (Debian
stable) system that has an older version of modprobe [1].

The kernel Makefile of course does:
/sbin/depmod -ae -F System.map -b $(INSTALL_MOD_PATH) $(KERNELRELEASE)

Because the old modprobe does not understand the new relative (or rather
rootless) paths, aggravated by the fact that initramfs-tools does not
error out or display errors from modprobe (probably for good historic
reasons), I suddenly had an initramfs that contained no modules and thus
a system that failed to reboot with the new kernel.

It took me quite a lot of time to trace it back to the upgrade of
module-init-tools.

Needless to say that this had always worked without any problems before.



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-19-2009, 08:23 PM
Jon Masters
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

On Thu, 2009-03-19 at 21:57 +0100, Frans Pop wrote:

> Because the old modprobe does not understand the new relative (or rather
> rootless) paths, aggravated by the fact that initramfs-tools does not
> error out or display errors from modprobe (probably for good historic
> reasons), I suddenly had an initramfs that contained no modules and thus
> a system that failed to reboot with the new kernel.

Well, that lack of understanding the difference between relative/explict
paths was fixed but of course we can't go back in time and fix what's
already out there and in use. Yes, it was a bad idea of mine (perhaps)
to change the existing file format and I've learned something, but it
should only have affected for example that 3.4 release you're using.

Jon.





--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-19-2009, 09:09 PM
Frans Pop
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

On Thursday 19 March 2009, Jon Masters wrote:
> Yes, it was a bad idea of
> mine (perhaps) to change the existing file format and I've learned
> something, but it should only have affected for example that 3.4
> release you're using.

Do you mean that earlier versions are not affected? Hasn't depmod
generated full paths basically any version up to 3.6?

But even if it is only 3.4, that still makes it every Debian stable user
(and unknown other distros) who runs the risk of ending up with an
unbootable system for hard to trace reasons...
Potentially painful for example for NAS devices where the kernel and
initrd get installed in flash, replacing the previous version.

As the kernel Makefile does run depmod during a build, I don't think it's
strange to assume users rely on that modules.dep being valid, even for
older versions of modprobe.

What exactly are the resons behind the change in file format that're so
strong that depmod cannot continue to generate the old format for the
next 5 years or so as a transition period, until the risk is much lower
that users run into problems because their current version of modprobe
does not understand the new format? Are they really worth the potential
consequences?



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-19-2009, 09:22 PM
Jon Masters
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

On Thu, 2009-03-19 at 23:09 +0100, Frans Pop wrote:
> On Thursday 19 March 2009, Jon Masters wrote:
> > Yes, it was a bad idea of
> > mine (perhaps) to change the existing file format and I've learned
> > something, but it should only have affected for example that 3.4
> > release you're using.
>
> Do you mean that earlier versions are not affected? Hasn't depmod
> generated full paths basically any version up to 3.6?

Yes, it was 3.6 where it changed, not 3.4. However, there was a backward
compatibility issue during development that I was referring to. That
isn't the problem here anyway. The "problem" is that you want to use a
newly generated .dep file on an old install. I know that would be nice,
and I'm sorry that it bothers you, but at the same time you don't expect
newly built binaries to run against an older version of glibc they
weren't built against, etc.

> But even if it is only 3.4, that still makes it every Debian stable user
> (and unknown other distros) who runs the risk of ending up with an
> unbootable system for hard to trace reasons...

No, it really doesn't. If you upgrade to a new module-init-tools and run
depmod, then everything works. If you use the old .dep files then it
still works. If you force downgrade and don't rebuild your .dep files
then there certainly is the risk of being out of sync - but there are
other times where such things can happen too. I'm sure other utilities
have similar.

> Potentially painful for example for NAS devices where the kernel and
> initrd get installed in flash, replacing the previous version.

Yes, and in that case there's no problem. If you do a forward upgrade
everything works just fine. It's only if you try to mix versions and go
backwards that you get a problem. I can see this in a cross-build setup
being a slight annoyance, but even there most people would use the same
version of the tools or expect to have problems building on a newer box
for an older release.

> As the kernel Makefile does run depmod during a build, I don't think it's
> strange to assume users rely on that modules.dep being valid, even for
> older versions of modprobe.

It works fine as long as your box is self-consistent. What is the actual
problem other than that mixing versions and trying to go backward
without quickly re-running depmod might cause module load problems?

> What exactly are the resons behind the change in file format that're so
> strong that depmod cannot continue to generate the old format for the
> next 5 years or so as a transition period

I guess it's called progress Sarcasm aside, if you can give me an
example of an actual real life set of users who are adversely affected
then I'll try to do something to help out. But if you're asking for old
versions of software to be compatible with newer releases in every case
I think you're not being terribly realistic. The kernel changes to make
progress, and so do other utilities.

Jon.





--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-19-2009, 09:58 PM
Frans Pop
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

On Thursday 19 March 2009, Jon Masters wrote:
[...]

I understand how and why and when it works now. I can also easily avoid
the problem now that I know about it. The question here is if the
breakage is really necessary.

I ran into the problem within days of installing the new m-i-t. I don't
think I'm very special, so my guess is it's going to affect others too.

> I guess it's called progress Sarcasm aside, if you can give me an
> example of an actual real life set of users who are adversely affected
> then I'll try to do something to help out. But if you're asking for old
> versions of software to be compatible with newer releases in every case
> I think you're not being terribly realistic. The kernel changes to make
> progress, and so do other utilities.

No, sorry. That isn't, at least AIUI, how kernel (related) development is
supposed to be done: http://lkml.org/lkml/2005/12/29/173 and similar
discussions later.
Sure, if there are very strong reasons to break things, fine. But whenever
possible the kernel has ensured backwards compatibility, mostly only
_after_ someone "complained". Think of the i386 and x86_64 symlinks after
the x86 integration, think of the COMPAT_SYSFS flags, think of optional
support for old /proc files, think feature-removal-schedule.txt.

So far you seem to be avoiding to give the reasons for the change. What
would be so wrong with ensuring the compatibility for some transition
period and avoiding the problem?

P.S. Thanks a lot for your prompt replies. I do appreciate the discussion.



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-19-2009, 10:11 PM
Jon Masters
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

On Thu, 2009-03-19 at 23:58 +0100, Frans Pop wrote:

> > I guess it's called progress Sarcasm aside, if you can give me an
> > example of an actual real life set of users who are adversely affected
> > then I'll try to do something to help out. But if you're asking for old
> > versions of software to be compatible with newer releases in every case
> > I think you're not being terribly realistic. The kernel changes to make
> > progress, and so do other utilities.

> Sure, if there are very strong reasons to break things, fine. But whenever
> possible the kernel has ensured backwards compatibility, mostly only
> _after_ someone "complained". Think of the i386 and x86_64 symlinks after
> the x86 integration, think of the COMPAT_SYSFS flags, think of optional
> support for old /proc files, think feature-removal-schedule.txt.

All of these examples refer to the future. The *future*. Things that
will change, preserving backward compatibility, keeping symlinks around
for those who are used to the old location. *Backward* compatibility.
But the issue you raised was about *Forward* compatibility. This is nice
but isn't the same. That would be like guaranteeing that all future
kernel features will work with a kernel from 6 months ago, or that
modules will, or other similar stuff.

> So far you seem to be avoiding to give the reasons for the change. What
> would be so wrong with ensuring the compatibility for some transition
> period and avoiding the problem?

There is compatibility. From the past, to the present, into the future.
But you want to use *future* generated files with a *past* version. That
is not backward compatibility and it is not the kind of thing where you
can "preserve for 5 years", etc. How does the old version know something
is going to change? It doesn't and it can't. v3.4 will always be the
same, and it won't change. The files changed slightly in v3.6, but v3.4
can never know about that.

> P.S. Thanks a lot for your prompt replies. I do appreciate the discussion.

Sure. Hopefully you see that this is not a regular compatibility issue.

Jon.





--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-19-2009, 10:40 PM
Frans Pop
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

On Friday 20 March 2009, Jon Masters wrote:
> > Sure, if there are very strong reasons to break things, fine. But
> > whenever possible the kernel has ensured backwards compatibility,
> > mostly only _after_ someone "complained". Think of the i386 and
> > x86_64 symlinks after the x86 integration, think of the COMPAT_SYSFS
> > flags, think of optional support for old /proc files, think
> > feature-removal-schedule.txt.
>
> All of these examples refer to the future. The *future*. Things that
> will change, preserving backward compatibility, keeping symlinks around
> for those who are used to the old location. *Backward* compatibility.

Exactly: making sure that tools in "older" userspace environments (old
version of modprobe) continue to work with new kernels (or built in an
environment that happens to have the latest and greatest depmod).

> But the issue you raised was about *Forward* compatibility. This is
> nice but isn't the same. That would be like guaranteeing that all
> future kernel features will work with a kernel from 6 months ago, or
> that modules will, or other similar stuff.

I really don't see the huge difference. IMO you are comparing apples with
oranges. I'm really don't think I'm trying to use modules from a 2.4
kernel with a 2.6 kernel here.

m-i-t is _not_ an integral part of the kernel. It's just another tool, and
one that's used in two different environments: a kernel build environment
and a kernel user environment. And the format of the files generated by
one and read by the other is the glue that keeps things together.
Changing that format should IMO be done with due consideration for
relevant use cases, and only if there are very strong arguments to do so.

> > So far you seem to be avoiding to give the reasons for the change.
> > What would be so wrong with ensuring the compatibility for some
> > transition period and avoiding the problem?
[...]
> Hopefully you see that this is not a regular compatibility issue.

What I mainly see is that you seem to be avoiding answering this question
and are apparently unwilling to consider to repair the situation.

I think my 2 cents are played out by now, so I'll drop things here. Maybe
someone else will be willing to take up the batton. At least the issue is
somewhat documented now. I'll inform others in Debian that the issue
exists and fix things locally for my own use case.

Thanks again for your replies.

Cheers,
FJP



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 
Old 03-20-2009, 09:10 AM
Michal Marek
 
Default Bug#518412: linux-2.6 install errors due to incompatible depmod files

Frans Pop napsal(a):
> The use case here, which I suspect is not all that uncommon, is that I
> built a kernel from upstream source on a (Debian unstable) system with
> the new version of depmod and then installed that kernel on a (Debian
> stable) system that has an older version of modprobe [1].

Backward vs. forward compatibility discussion aside, there is a reason
why most (all?) distro kernel packages run depmod at *install time* on
the target machine. There might be modules installed by other packages,
the format might change, newer depmod has a configuration file and
finally the files are not exactly small and can be recreated rather
easily. It might be a good idea to do the same when compiling kernels
manually and copying them around.

Michal



--
To UNSUBSCRIBE, email to debian-kernel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
 

Thread Tools




All times are GMT. The time now is 01:39 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org