A stern warning of things to come
 

In trying to get rid of some of the horrible method selection complexity in
loader and yuminstall.py, I realized two truths:

(1) It's silly that we expose stage2= when few people know what that means and
almost no one cares. It's an implementation-specific detail that has no
purpose leaking out to the world.

(2) It's very difficult to reduce the complexity without first guaranteeing that
no matter how you boot, you always have stage2. Once that is assurred, you can
remove an amazing amount of code out of the loader.

How do we deal with this? By removing install.img entirely, of course! Sure
we've talked about it off and on for years, but I think we're to a point where
we can actually do it. And doing so has tons of advantages. We reduce loader
down to a point where we can realistically consider removing it, or at least
moving it to python. Speaking of which, we'd have python in the initrd. method=
goes away. stage2= goes away. The bulk of the STEP_METHOD state machine crud
goes away. We eliminate entire classes of bugs. Let your imagination run wild
with everything this will simplify.

This is not without problems, though:

(1) It's possible some PXE implementations will not like downloading a single
100+ MB image. I'm hoping the number of machines affected is decreasing every
year and we can rapidly stop caring.

(2) The kernel may freak out about storing such a large image in memory so
early on. Peter has some thoughts about this.

(3) Memory requirements may be increased.

(4) Doing a large number of PXE-based installs all at once could perhaps bring
a network to its knees. Transferring a large number of copies of a large file
all at once is a sure sign of impending doom.

(5) We may rapidly be writing ourselves out of a job.

Anyway it's unlikely these patches currently apply. Because they touch so much
stuff, every single change someone else makes to scripts/ causes git to freak
out. And I haven't even started to work on the loader patches yet. This is
presented more as a proof of concept. However, I have done a build and tested
enough to get to the method selection screen in loader.

- Chris

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list

A stern warning of things to come
 

> This is not without problems, though:
>
> (1) It's possible some PXE implementations will not like downloading a single
> 100+ MB image. I'm hoping the number of machines affected is decreasing every
> year and we can rapidly stop caring.

This is actually of some concern, primarily that downloading within PXE
(of any type) is quite slow. It's doable, but there are a few PXE
stacks that will barf on something that big. Some of those can be
"fixed" by chaining first to gPXE then to the install.

That said RIP, which I've got in BKO, has an initrd that's about 100M in
size, and it works. Just takes forever to load.

Case in point:

http://boot.kernel.org/bko/pxeknife/linux_boot_disks/rip/tftpboot/rootfs.cgz

is an initrd of roughly the size your proposing, and one I know works.

Various download times:

the above link with wget across the internet to my home:

69s - 1.37 MB/s

With tftp from a server in my house to my laptop (with a wifi link involved)

6:15.60 - 262 KB/s

Same tftp server only only wired (100mbps)

43.034s - 2.1 MB/s

NOTE: this is using the tftp client thusly:

time echo "get rootfs.cgz
quit" | tftp <server>

tftp implementations in PXE stacks are quite a bit slower than this.

> (2) The kernel may freak out about storing such a large image in memory so
> early on. Peter has some thoughts about this.

I haven't seen any problems with this specifically.

> (3) Memory requirements may be increased.

It would effectively kill off anything with less than 256M of ram from
doing an install. I wouldn't think this is common anymore, but I know
machines with 512M of ram are still in production today, and running Fedora.

> (4) Doing a large number of PXE-based installs all at once could perhaps bring
> a network to its knees. Transferring a large number of copies of a large file
> all at once is a sure sign of impending doom.

You won't hit this any more than you would normally. The bottleneck is
going to be the tftp server in this case, it won't be the network.

> (5) We may rapidly be writing ourselves out of a job.

Doubtful ;-)

Just some thoughts, I'm not sure if this kind of change would ultimately
be good or bad. The biggest problem I can see if the want for an ever
increasing initrd.

- John 'Warthog9' Hawley

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list

A stern warning of things to come
 

> > (1) It's possible some PXE implementations will not like downloading a single
> > 100+ MB image. I'm hoping the number of machines affected is decreasing every
> > year and we can rapidly stop caring.
>
> This is actually of some concern, primarily that downloading within PXE
> (of any type) is quite slow. It's doable, but there are a few PXE
> stacks that will barf on something that big. Some of those can be
> "fixed" by chaining first to gPXE then to the install.

Any ideas how common these implementations are, and on what kinds of
machines?

> http://boot.kernel.org/bko/pxeknife/linux_boot_disks/rip/tftpboot/rootfs.cgz
>
> is an initrd of roughly the size your proposing, and one I know works.
>
> Various download times:
>
> the above link with wget across the internet to my home:
>
> 69s - 1.37 MB/s
>
> With tftp from a server in my house to my laptop (with a wifi link involved)
>
> 6:15.60 - 262 KB/s
>
> Same tftp server only only wired (100mbps)
>
> 43.034s - 2.1 MB/s

The test installs I've done here haven't seemed to take this long (they
definitely don't take six minutes, they may take 43 seconds) across a
wired network. There's also a sizeable delay when the kernel gets to
unpacking the image.

> > (3) Memory requirements may be increased.
>
> It would effectively kill off anything with less than 256M of ram from
> doing an install. I wouldn't think this is common anymore, but I know
> machines with 512M of ram are still in production today, and running Fedora.

These are basically dead anyway, and getting moreso all the time:

#if defined(__powerpc64__) || defined(__sparc__)
#define MIN_RAM 1024*1024 // 1 GB
#define GUI_INSTALL_EXTRA_RAM 512*1024 // 512 MB
#else
#define MIN_RAM 256 * 1024 // 256 MB
#define GUI_INSTALL_EXTRA_RAM 128 * 1024 // 128 MB
#endif
#define URL_INSTALL_EXTRA_RAM 128 * 1024 // 128 MB
#define MIN_GUI_RAM MIN_RAM + GUI_INSTALL_EXTRA_RAM
#define EARLY_SWAP_RAM 512 * 1024 // 512 MB

I kind of think MIN_RAM in the normal case is a lie anyway.

> Just some thoughts, I'm not sure if this kind of change would ultimately
> be good or bad. The biggest problem I can see if the want for an ever
> increasing initrd.

It's such a massive improvement in maintainability and reliability that
I think it's definitely for the best. Just removing the state machines
in loader alone would drastically cut down on the number of weird method
selection bugs we've got. This is the kind of stuff that end users
aren't directly going to notice, but they'll experience fewer bugs and
we'll have more time to work on other stuff.

I understand the problem about the magic growing initrd. I'm not sure
how we can address that (especially given our desire to use system stuff
- see Martin's other thread on the list). We'll need to be careful.

- Chris

_______________________________________________
Anaconda-devel-list mailing list
Anaconda-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/anaconda-devel-list