Linux Archive

Linux Archive (http://www.linux-archive.org/)
-   ArchLinux Development (http://www.linux-archive.org/archlinux-development/)
-   -   final leg of /lib removal (http://www.linux-archive.org/archlinux-development/679730-final-leg-lib-removal.html)

Dave Reisner 07-03-2012 03:41 PM

final leg of /lib removal
 
Hey all,

*** If you use a custom kernel, this will affect you. Please read the
big scary note at the end ***

I'm taking today to work on the last roadblock before Allan can move
glibc out of /lib. This basically consists of a rebuild of:

- kmod (to drop our local patch)
- linux, linux-lts (diff for linux here: http://paste.xinu.at/LLd/)
- all OOT kernel modules (for /usr/lib/modules/extramodules-*)
- bash-completion (temp patch until /lib is a symlink: http://paste.xinu.at/xEs/)

I'll be doing this all locally to avoid building against allan's new
toolchain in [staging]. This will hopefully all hit [testing] by the
end of the day. You know where to find me if you have any questions or
angry rants.

If you'd like to do some early testing, I'm leaving the rebuilt kmod and
kernel packages on gerolde:

http://dev.archlinux.org/~dreisner/linux-usrmove/

(i686 packages are lagging behind at the moment)

BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
tools for users with their own kernels. If you do not rebuild your
kernel after pulling in the new kmod, you're going to have a bad time.
See the paste link above for inspiration.

Cheers!
Dave

Thomas Bächler 07-03-2012 03:48 PM

final leg of /lib removal
 
Am 03.07.2012 17:41, schrieb Dave Reisner:
> BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
> tools for users with their own kernels. If you do not rebuild your
> kernel after pulling in the new kmod, you're going to have a bad time.
> See the paste link above for inspiration.

This worries me, a lot. Can't we get a smoother upgrade path?

Dave Reisner 07-03-2012 04:05 PM

final leg of /lib removal
 
On Tue, Jul 03, 2012 at 05:48:43PM +0200, Thomas Bächler wrote:
> Am 03.07.2012 17:41, schrieb Dave Reisner:
> > BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
> > tools for users with their own kernels. If you do not rebuild your
> > kernel after pulling in the new kmod, you're going to have a bad time.
> > See the paste link above for inspiration.
>
> This worries me, a lot. Can't we get a smoother upgrade path?

Not really.

We could patch all things using kmod (udev and kmod's tools) to look
in both /usr/lib as well as /lib, but that's ugly and doesn't really do
us any good.

We could make this rebuild coincide with the glibc rebuild to get rid of
/lib, but you can't install glibc with /lib as a symlink until
/lib/modules doesn't exist. The only way /lib/modules doesn't exist is
if people with custom kernels rebuild them into /usr/lib/modules.

d

Thomas Bächler 07-03-2012 04:11 PM

final leg of /lib removal
 
Am 03.07.2012 18:05, schrieb Dave Reisner:
> On Tue, Jul 03, 2012 at 05:48:43PM +0200, Thomas Bächler wrote:
>> Am 03.07.2012 17:41, schrieb Dave Reisner:
>>> BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
>>> tools for users with their own kernels. If you do not rebuild your
>>> kernel after pulling in the new kmod, you're going to have a bad time.
>>> See the paste link above for inspiration.
>>
>> This worries me, a lot. Can't we get a smoother upgrade path?
>
> Not really.
>
> We could patch all things using kmod (udev and kmod's tools) to look
> in both /usr/lib as well as /lib, but that's ugly and doesn't really do
> us any good.
>
> We could make this rebuild coincide with the glibc rebuild to get rid of
> /lib, but you can't install glibc with /lib as a symlink until
> /lib/modules doesn't exist. The only way /lib/modules doesn't exist is
> if people with custom kernels rebuild them into /usr/lib/modules.

Okay.

Why do we want /lib as a symlink to /usr/lib anyway? You could have the
directory /lib, only containing the symlinks for ld-linux.so.2 ->
/usr/lib/ld-linux.so.2 and ld-linux-x86_64.so.2 ->
/usr/lib/ld-linux-x86_64.so.2 (and, of course
/lib64/ld-linux-x86_64.so.2 for compatibility).

I don't see any advantage in having the symlink /lib -> /usr/lib, except
a harder upgrade path where so many things could go wrong.

Dave Reisner 07-03-2012 04:22 PM

final leg of /lib removal
 
On Tue, Jul 03, 2012 at 06:11:16PM +0200, Thomas Bächler wrote:
> Am 03.07.2012 18:05, schrieb Dave Reisner:
> > On Tue, Jul 03, 2012 at 05:48:43PM +0200, Thomas Bächler wrote:
> >> Am 03.07.2012 17:41, schrieb Dave Reisner:
> >>> BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
> >>> tools for users with their own kernels. If you do not rebuild your
> >>> kernel after pulling in the new kmod, you're going to have a bad time.
> >>> See the paste link above for inspiration.
> >>
> >> This worries me, a lot. Can't we get a smoother upgrade path?
> >
> > Not really.
> >
> > We could patch all things using kmod (udev and kmod's tools) to look
> > in both /usr/lib as well as /lib, but that's ugly and doesn't really do
> > us any good.
> >
> > We could make this rebuild coincide with the glibc rebuild to get rid of
> > /lib, but you can't install glibc with /lib as a symlink until
> > /lib/modules doesn't exist. The only way /lib/modules doesn't exist is
> > if people with custom kernels rebuild them into /usr/lib/modules.
>
> Okay.
>
> Why do we want /lib as a symlink to /usr/lib anyway? You could have the
> directory /lib, only containing the symlinks for ld-linux.so.2 ->
> /usr/lib/ld-linux.so.2 and ld-linux-x86_64.so.2 ->
> /usr/lib/ld-linux-x86_64.so.2 (and, of course
> /lib64/ld-linux-x86_64.so.2 for compatibility).
>
> I don't see any advantage in having the symlink /lib -> /usr/lib, except
> a harder upgrade path where so many things could go wrong.
>

No offense, but you're a bit late to be trying to shoot this down only
now. Perhaps you recall Tom's post (and ensuing discussion) from a few
months ago:

http://www.mail-archive.com/arch-dev-public@archlinux.org/msg19028.html

With a followup:

http://mailman.archlinux.org/pipermail/arch-dev-public/2012-March/022756.html

Maybe the wiki page that detailed the plan:

https://wiki.archlinux.org/index.php/DeveloperWiki:UsrMove

Or when we started on this with todolist 143:

https://www.archlinux.org/todo/143/

And did more of this with todolist 148:

https://www.archlinux.org/todo/148/

What's the point of doing 95% of this and then polishing it off with
some silly hack? /lib64 is already going away and turning into a symlink
with the glibc-2.16 package in [staging].

d

P.S. i686 rebuilds are done -- I've moved this all to brynhild to avoid
the bandwidth restrictions on gerolde:

http://pkgbuild.com/~dreisner/linux-usrmove/

Tom Gundersen 07-03-2012 05:36 PM

final leg of /lib removal
 
On Jul 3, 2012 5:41 PM, "Dave Reisner" <d@falconindy.com> wrote:
>
> Hey all,
>
> *** If you use a custom kernel, this will affect you. Please read the
> big scary note at the end ***
>
> I'm taking today to work on the last roadblock before Allan can move
> glibc out of /lib. This basically consists of a rebuild of:
>
> - kmod (to drop our local patch)
> - linux, linux-lts (diff for linux here: http://paste.xinu.at/LLd/)
> - all OOT kernel modules (for /usr/lib/modules/extramodules-*)
> - bash-completion (temp patch until /lib is a symlink:
http://paste.xinu.at/xEs/)
>
> I'll be doing this all locally to avoid building against allan's new
> toolchain in [staging]. This will hopefully all hit [testing] by the
> end of the day. You know where to find me if you have any questions or
> angry rants.
>
> If you'd like to do some early testing, I'm leaving the rebuilt kmod and
> kernel packages on gerolde:
>
> http://dev.archlinux.org/~dreisner/linux-usrmove/
>
> (i686 packages are lagging behind at the moment)
>
> BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
> tools for users with their own kernels. If you do not rebuild your
> kernel after pulling in the new kmod, you're going to have a bad time.
> See the paste link above for inspiration.

Is a rebuild really necessary? I thought all that was needed was to move
the modules from / to /usr?

Thanks for picking this up!

I'll help out with testing in a couple of hours.

Tom

Dave Reisner 07-03-2012 05:48 PM

final leg of /lib removal
 
On Tue, Jul 03, 2012 at 07:36:48PM +0200, Tom Gundersen wrote:
> On Jul 3, 2012 5:41 PM, "Dave Reisner" <d@falconindy.com> wrote:
> >
> > Hey all,
> >
> > *** If you use a custom kernel, this will affect you. Please read the
> > big scary note at the end ***
> >
> > I'm taking today to work on the last roadblock before Allan can move
> > glibc out of /lib. This basically consists of a rebuild of:
> >
> > - kmod (to drop our local patch)
> > - linux, linux-lts (diff for linux here: http://paste.xinu.at/LLd/)
> > - all OOT kernel modules (for /usr/lib/modules/extramodules-*)
> > - bash-completion (temp patch until /lib is a symlink:
> http://paste.xinu.at/xEs/)
> >
> > I'll be doing this all locally to avoid building against allan's new
> > toolchain in [staging]. This will hopefully all hit [testing] by the
> > end of the day. You know where to find me if you have any questions or
> > angry rants.
> >
> > If you'd like to do some early testing, I'm leaving the rebuilt kmod and
> > kernel packages on gerolde:
> >
> > http://dev.archlinux.org/~dreisner/linux-usrmove/
> >
> > (i686 packages are lagging behind at the moment)
> >
> > BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
> > tools for users with their own kernels. If you do not rebuild your
> > kernel after pulling in the new kmod, you're going to have a bad time.
> > See the paste link above for inspiration.
>
> Is a rebuild really necessary? I thought all that was needed was to move
> the modules from / to /usr?
>
> Thanks for picking this up!
>
> I'll help out with testing in a couple of hours.
>
> Tom

As a (lazy) stop-gap measure, yes you probably could just move the module
dir. I offer no warranty on this method, though.

Testing on my side is going well so far... I've upgraded a bunch of VMs
with both the -ARCH and -lts kernels and haven't run into any unforeseen
problems yet.

Working on modules in extra now...

d

Ike Devolder 07-03-2012 05:56 PM

final leg of /lib removal
 
Op dinsdag 3 juli 2012 11:41:08 schreef Dave Reisner:
> Hey all,
>
> *** If you use a custom kernel, this will affect you. Please read the
> big scary note at the end ***
>
> I'm taking today to work on the last roadblock before Allan can move
> glibc out of /lib. This basically consists of a rebuild of:
>
> - kmod (to drop our local patch)
> - linux, linux-lts (diff for linux here: http://paste.xinu.at/LLd/)
> - all OOT kernel modules (for /usr/lib/modules/extramodules-*)
> - bash-completion (temp patch until /lib is a symlink:
> http://paste.xinu.at/xEs/)
>
> I'll be doing this all locally to avoid building against allan's new
> toolchain in [staging]. This will hopefully all hit [testing] by the
> end of the day. You know where to find me if you have any questions or
> angry rants.
>
> If you'd like to do some early testing, I'm leaving the rebuilt kmod and
> kernel packages on gerolde:
>
> http://dev.archlinux.org/~dreisner/linux-usrmove/
>
> (i686 packages are lagging behind at the moment)
>
> BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
> tools for users with their own kernels. If you do not rebuild your
> kernel after pulling in the new kmod, you're going to have a bad time.
> See the paste link above for inspiration.
>
> Cheers!
> Dave

I'm replying on this in arch-general and aur-general because i cant post in
arch-dev.

So if i understand correctly we (the people running custom kernels) can't
prepare for this ?

There is no way of moving the modules already to /usr/lib ? I assume kmod now
only looks in /lib.

Another note, people with custom repositories should move their kernels in
sync with the official repositories ?

--Ike

Ike Devolder 07-03-2012 05:56 PM

final leg of /lib removal
 
Op dinsdag 3 juli 2012 11:41:08 schreef Dave Reisner:
> Hey all,
>
> *** If you use a custom kernel, this will affect you. Please read the
> big scary note at the end ***
>
> I'm taking today to work on the last roadblock before Allan can move
> glibc out of /lib. This basically consists of a rebuild of:
>
> - kmod (to drop our local patch)
> - linux, linux-lts (diff for linux here: http://paste.xinu.at/LLd/)
> - all OOT kernel modules (for /usr/lib/modules/extramodules-*)
> - bash-completion (temp patch until /lib is a symlink:
> http://paste.xinu.at/xEs/)
>
> I'll be doing this all locally to avoid building against allan's new
> toolchain in [staging]. This will hopefully all hit [testing] by the
> end of the day. You know where to find me if you have any questions or
> angry rants.
>
> If you'd like to do some early testing, I'm leaving the rebuilt kmod and
> kernel packages on gerolde:
>
> http://dev.archlinux.org/~dreisner/linux-usrmove/
>
> (i686 packages are lagging behind at the moment)
>
> BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
> tools for users with their own kernels. If you do not rebuild your
> kernel after pulling in the new kmod, you're going to have a bad time.
> See the paste link above for inspiration.
>
> Cheers!
> Dave

I'm replying on this in arch-general and aur-general because i cant post in
arch-dev.

So if i understand correctly we (the people running custom kernels) can't
prepare for this ?

There is no way of moving the modules already to /usr/lib ? I assume kmod now
only looks in /lib.

Another note, people with custom repositories should move their kernels in
sync with the official repositories ?

--Ike

Dave Reisner 07-03-2012 06:18 PM

final leg of /lib removal
 
On Tue, Jul 03, 2012 at 07:56:32PM +0200, Ike Devolder wrote:
> Op dinsdag 3 juli 2012 11:41:08 schreef Dave Reisner:
> > Hey all,
> >
> > *** If you use a custom kernel, this will affect you. Please read the
> > big scary note at the end ***
> >
> > I'm taking today to work on the last roadblock before Allan can move
> > glibc out of /lib. This basically consists of a rebuild of:
> >
> > - kmod (to drop our local patch)
> > - linux, linux-lts (diff for linux here: http://paste.xinu.at/LLd/)
> > - all OOT kernel modules (for /usr/lib/modules/extramodules-*)
> > - bash-completion (temp patch until /lib is a symlink:
> > http://paste.xinu.at/xEs/)
> >
> > I'll be doing this all locally to avoid building against allan's new
> > toolchain in [staging]. This will hopefully all hit [testing] by the
> > end of the day. You know where to find me if you have any questions or
> > angry rants.
> >
> > If you'd like to do some early testing, I'm leaving the rebuilt kmod and
> > kernel packages on gerolde:
> >
> > http://dev.archlinux.org/~dreisner/linux-usrmove/
> >
> > (i686 packages are lagging behind at the moment)
> >
> > BIG SCARY NOTE: Due to the kmod changes, this will BREAK all module
> > tools for users with their own kernels. If you do not rebuild your
> > kernel after pulling in the new kmod, you're going to have a bad time.
> > See the paste link above for inspiration.
> >
> > Cheers!
> > Dave
>
> I'm replying on this in arch-general and aur-general because i cant post in
> arch-dev.
>
> So if i understand correctly we (the people running custom kernels) can't
> prepare for this ?

I posted the new kmod package here explicitly so that users can get this
package in preparation... I'll post it again since I changed the server
I'm hosting these on:

updated URL: http://pkgbuild.com/~dreisner/linux-usrmove/

> There is no way of moving the modules already to /usr/lib ? I assume kmod now
> only looks in /lib.

Currently, kmod is patched to respect config dirs in /usr/lib, but
modules in /lib. After removing the patch, it uniformly searches
/usr/lib for everything (I'm intentionally ignoring /etc and /run here).

> Another note, people with custom repositories should move their kernels in
> sync with the official repositories ?
>
> --Ike

As with any large change, I'll mention on dev-public when this goes to
testing, and there will be an associated news item when it moves to
core. I realize this is a harsh change, but I don't really have many
options for doing this more smoothly. If you're using the stock kernel,
this should all just work. mkinitcpio has supported this setup for
months now, and I've had my own kernel in /usr/lib/modules for almost as
long.

Worst case scenario, users of custom kernels can:

- manually move /lib/modules/mycustomkernel to /usr/lib/modules/ until
they can do a proper rebuild.
- boot a stock -ARCH kernel (you DO have it listed as a fallback,
right?) until they can do a proper rebuild.

Emphasis on "until they can do a proper rebuild". It's important that
this all gets done before we introduce a new glibc package that wipes
out /lib entirely. If you have custom kernel bits lying around in
/lib/modules, it's going to block the eventual glibc upgrade that brings
this (no, it won't be immediately with 2.16).

dave


All times are GMT. The time now is 01:32 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.