On Sat, Feb 20, 2010 at 01:21:15PM -0800, Steve Langasek wrote:
> On Sat, Feb 20, 2010 at 12:02:24PM +0100, Goswin von Brederlow wrote:
> > The reason would be size. I don't see anything else there.
>
> > For network based boots, specifically high performance cluster, the size
> > can make a real difference. When you turn the cluster on it is not just
> > one system downloading an extra meg but 100+ nodes. That largely
> > increases the network collisions, errors and dropped packages. Something
> > that can even make systems fail to boot.
>
> Has anyone tried using library reduction to get a stripped down eglibc for
> inclusion in initramfs at runtime, using the same techniques as d-i to get a
> library that contains only those symbols needed by the included utilities?

Don't know, but I'm not sure that's a very good idea. The library
reduction as implemented by mklibs is a bit of a kludge IMO, which seems
to fail every so often for one of our architectures, because that
architecture's ABI uses some obscure ELF feature or other that the
mklibs authors didn't know about, or mklibs missed a symbol dependency,
or something else.

This is not really a big deal in the case of d-i, since first, when
things fail, they fail for everyone who uses the same image, and second,
if the installer fails, you just can't install using the broken image,
but you can still use the system that's already installed (if any), or
fall back to a different install image. Both are not true for initramfs
generation.

--
The biometric identification system at the gates of the CIA headquarters
works because there's a guard with a large gun making sure no one is
trying to fool the system.
http://www.schneier.com/blog/archives/2009/01/biometrics.html


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100228110503.GA31023@celtic.nixsys.be">http://lists.debian.org/20100228110503.GA31023@celtic.nixsys.be

On Sun, Feb 28, 2010 at 12:05:03PM +0100, Wouter Verhelst wrote:
> Don't know, but I'm not sure that's a very good idea. The library
> reduction as implemented by mklibs is a bit of a kludge IMO, which seems
> to fail every so often for one of our architectures, because that
> architecture's ABI uses some obscure ELF feature or other that the
> mklibs authors didn't know about, or mklibs missed a symbol dependency,
> or something else.

> This is not really a big deal in the case of d-i, since first, when
> things fail, they fail for everyone who uses the same image, and second,
> if the installer fails, you just can't install using the broken image,
> but you can still use the system that's already installed (if any), or
> fall back to a different install image. Both are not true for initramfs
> generation.

You could obviously just fall back to using the full .so in the case of
initramfs generation.

--
Steve Langasek Give me a lever long enough and a Free OS
Debian Developer to set it on, and I can move the world.
Ubuntu Developer http://www.debian.org/
slangasek@ubuntu.com vorlon@debian.org

On Sun, Feb 28, 2010 at 10:26:48AM -0800, Steve Langasek wrote:
> On Sun, Feb 28, 2010 at 12:05:03PM +0100, Wouter Verhelst wrote:
> > This is not really a big deal in the case of d-i, since first, when
> > things fail, they fail for everyone who uses the same image, and second,
> > if the installer fails, you just can't install using the broken image,
> > but you can still use the system that's already installed (if any), or
> > fall back to a different install image. Both are not true for initramfs
> > generation.
>
> You could obviously just fall back to using the full .so in the case of
> initramfs generation.

You could always do that, but it would rather defeat the purpose.

Also, changing back to a working initramfs may be rather hard, depending
on the bootloader in use.

--
The biometric identification system at the gates of the CIA headquarters
works because there's a guard with a large gun making sure no one is
trying to fool the system.
http://www.schneier.com/blog/archives/2009/01/biometrics.html


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100301010031.GC15003@celtic.nixsys.be">http://lists.debian.org/20100301010031.GC15003@celtic.nixsys.be

On Mon, Mar 01, 2010 at 02:00:31AM +0100, Wouter Verhelst wrote:
> On Sun, Feb 28, 2010 at 10:26:48AM -0800, Steve Langasek wrote:
> >
> > You could obviously just fall back to using the full .so in the case of
> > initramfs generation.

If we can detect that the libc generated is unsuitable, then this
fallback seems safe enough.

> Also, changing back to a working initramfs may be rather hard, depending
> on the bootloader in use.

This argument holds for just about any change (epecially ones that we
suspect might behave different on different architectures, with
different bootloaders, or with different underlying storage techniques),
and the answer is the same in all cases. Test, test, test.

I know it's nowhere near as true as it was a decade ago, but "people
running unstable should know how to fix something as simple as their
system failing to boot due to a broken bootloader/kernel/initrd".

Personally, I think the idea of cutting out the klibc duplication and
stripping eglibc during initramfs generation is a pretty reasonable
solution to this longstanding issue.

The extra upshot of this is that some of the weirder corner cases you
refer to that have bitten d-i in the past (A) have made the implementation
more robust, but more interestingly (B) new and similar issues will be
discovered and fixed more readily if this is being tested by more than
just a handful of installer builders/testers.

... Adam


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100307161356.GB1269@0c3.net">http://lists.debian.org/20100307161356.GB1269@0c3.net

Hi Adam,

(long time no see!)

On Sun, Mar 07, 2010 at 04:13:56PM +0000, Adam Conrad wrote:
> On Mon, Mar 01, 2010 at 02:00:31AM +0100, Wouter Verhelst wrote:
> > On Sun, Feb 28, 2010 at 10:26:48AM -0800, Steve Langasek wrote:
> > >
> > > You could obviously just fall back to using the full .so in the case of
> > > initramfs generation.
>
> If we can detect that the libc generated is unsuitable, then this
> fallback seems safe enough.

The main problem is that this is indeed not easy to detect (otherwise
this would be a non-issue).

[...]
> The extra upshot of this is that some of the weirder corner cases you
> refer to that have bitten d-i in the past (A) have made the implementation
> more robust, but more interestingly (B) new and similar issues will be
> discovered and fixed more readily if this is being tested by more than
> just a handful of installer builders/testers.

That much, certainly, is true. I guess it is the better argument to
indeed go for this option, though I would still suggest doing so in
experimental first.

--
The biometric identification system at the gates of the CIA headquarters
works because there's a guard with a large gun making sure no one is
trying to fool the system.
http://www.schneier.com/blog/archives/2009/01/biometrics.html


--
To UNSUBSCRIBE, email to debian-devel-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20100307231723.GA3364@celtic.nixsys.be">http://lists.debian.org/20100307231723.GA3364@celtic.nixsys.be