FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor


 
 
LinkBack Thread Tools
 
Old 01-13-2008, 11:14 PM
Linus Walleij
 
Default Preloading

On Sun, 13 Jan 2008, Linus Walleij wrote:


* They use a LOT of preloading,


I have now looked a bit closer at this, and it is *indeed* looking like a
major user-perceiveable thing, they preload pretty much everything and
also add other hints to the kernel of what may be expected to be used
soonish during boot, early-{x|g|k}dm and so forth.


It is rather crude: the actual mechanisms used are:

preload, source can be found here:
http://download.opensuse.org/distribution/10.3/repo/src-oss/suse/src/preload-0.2-110.src.rpm
anyone knows wher SuSE keep their CVS/SVN/git of this?

Preload then uses readahead(2) or fadvice(2) to do the actual preloading.

Then preload is triggered from /etc/init.d scripts prepended with
ionice(1) like this:


/usr/bin/ionice -n2 /sbin/preload /etc/preload.d/gdm

Then as can be seen there are some app profiles in /etc/preload.d that
configure depending on used desktop what to preload, e.g. gdm/kdm,
Khelpcenter, OpenOffice, Firefox, Gimp (why?), etc.


All code and scripts are heavily SuSEified.

But has this really been under our radar for all this time? The entire
concept is like 2-3 years old, surely it must have been discussed before?
(Not that I heard it though...)


I don't have much time in my life but unless someone else starts looking
at this I will se if it can be Fedorified, target for F10 or so.


And thanks to Jakub for the hints given that allowed me to find this cool
thing...

Linus

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 01-13-2008, 11:34 PM
"Naheem Zaffar"
 
Default Preloading

I think Fedora did use some sort of preloading (well, atleast
Readahead), but it was decided that there was no tangible benefit from
it in the way it was being done.

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 01-14-2008, 12:25 AM
Andrew Farris
 
Default Preloading

Linus Walleij wrote:

Preload then uses readahead(2) or fadvice(2) to do the actual preloading.

Then preload is triggered from /etc/init.d scripts prepended with
ionice(1) like this:


/usr/bin/ionice -n2 /sbin/preload /etc/preload.d/gdm

Then as can be seen there are some app profiles in /etc/preload.d that
configure depending on used desktop what to preload, e.g. gdm/kdm,
Khelpcenter, OpenOffice, Firefox, Gimp (why?), etc.


Incidentally, I think one of the main reasons the idea hasn't had much
discussion is that question of what applications are so crucial to preload like
this (gimp? laughable). Also, once an application is open its shared libraries
are cached for later, so for machines that stay on all day this has almost no
real benefit.



All code and scripts are heavily SuSEified.

But has this really been under our radar for all this time? The entire
concept is like 2-3 years old, surely it must have been discussed
before? (Not that I heard it though...)


I've seen discussion of this concept at least, maybe not that particular
implementation strategy. The idea should provide some better performance from
the user perspective... but the fact is the real wall time of getting your
machine cold booted, logged in, and an application fired up and usable should
not be much different overall. The real differences would be seen if you are
not sitting at the machine logging in asap for instance (the machine has nothing
to do until you log in, whereas with that preloading it might start loading up
firefox for you).


If you put this in the distro, you need good high-level configuration tools so
the user can choose what to preload or not, so you're not slowing the machine
unnecessarily, or packing all the free memory with libraries the user doesn't
intend to use that day/boot/login just to later unload it.


I wouldn't even want something learning my most used applications, just
something to configure what will get loaded and what won't.


--
Andrew Farris <lordmorgul@gmail.com> <ajfarris@gmail.com>
gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B 1DF3
No one now has, and no one will ever again get, the big picture. - Daniel Geer
---- ----

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 01-14-2008, 01:06 AM
Douglas McClendon
 
Default Preloading

Linus Walleij wrote:

On Sun, 13 Jan 2008, Linus Walleij wrote:


* They use a LOT of preloading,


I have now looked a bit closer at this, and it is *indeed* looking like
a major user-perceiveable thing, they preload pretty much everything and
also add other hints to the kernel of what may be expected to be used
soonish during boot, early-{x|g|k}dm and so forth.


It is rather crude: the actual mechanisms used are:


...

FYI, I recently outlined a (livecd) preloading implementation completely
unrelated to all of this, which I'd bet money will speed up livecd boot
by at least 10% (perhaps much more).


My method involves a devicemapper trick, the primary benefit of which is
removing seeks. Theoretically this could help disk (non-cdrom) based
boots, but would require ext3 features for sorting that I don't think
currently exist, and the benefit would probably be <<5% anyway.


Still, something to look forward to. Now that my no-root-privs livecd
generator tool is looking pretty swank along with liveusb persistence, I
think I'm nearly ready to write the relatively few lines of bash code
required to implement "super device-mapper caching" (just plain d.m.c.
was already taken for something different that might help the disk boot
case as well)


http://www.redhat.com/archives/fedora-livecd-list/2008-January/msg00036.html

-dmc

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 01-14-2008, 01:47 AM
Eric Sandeen
 
Default Preloading

Naheem Zaffar wrote:
> I think Fedora did use some sort of preloading (well, atleast
> Readahead), but it was decided that there was no tangible benefit from
> it in the way it was being done.
>

Well, I looked into the readahead util for boot time, and it seemed to
hurt more than help. But ongoing preload that learns & adjusts to usage
patterns could help I'm sure, if done right

-Eric (while waiting for the one long boot of OS X 10.5 after a system
update, as it updates its caches for the *next* fast boot...)

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 01-14-2008, 05:44 AM
"Jeff Spaleta"
 
Default Preloading

On Jan 13, 2008 5:47 PM, Eric Sandeen <sandeen@redhat.com> wrote:
> Well, I looked into the readahead util for boot time, and it seemed to
> hurt more than help. But ongoing preload that learns & adjusts to usage
> patterns could help I'm sure, if done right

That wont help the critical first impression phase.. before
application usage patterns are established and people are feeling out
the release. And to be more blunt about it.. it won't help first
impression from the reviewers who run a distro just long enough to
figure what they like and don't like about it and want to get their
review in within the first week of a release.
How many people end up relying on the first impression of others to
form their own impressions? Adaptive methods that only help boot
speeds after multiple reboots with extended usage pattern trending
isn't really going to impact the impression people have one damn bit.

And it wont help livecds where state information is lost between reboots.

-jef

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 01-14-2008, 06:05 AM
Andrew Farris
 
Default Preloading

Jeff Spaleta wrote:

On Jan 13, 2008 5:47 PM, Eric Sandeen <sandeen@redhat.com> wrote:

Well, I looked into the readahead util for boot time, and it seemed to
hurt more than help. But ongoing preload that learns & adjusts to usage
patterns could help I'm sure, if done right


That wont help the critical first impression phase.. before
application usage patterns are established and people are feeling out
the release. And to be more blunt about it.. it won't help first
impression from the reviewers who run a distro just long enough to
figure what they like and don't like about it and want to get their
review in within the first week of a release.
How many people end up relying on the first impression of others to
form their own impressions? Adaptive methods that only help boot
speeds after multiple reboots with extended usage pattern trending
isn't really going to impact the impression people have one damn bit.


Uhm, yeah you're right. But it will help those who actually use the
distribution rather than toss it aside based off bad information they got from
someone else. There is certainly time and place to consider user first
impressions, but I hardly think its a good idea to filter an idea like the above
by ONLY that consideration.



And it wont help livecds where state information is lost between reboots.

-jef



--
Andrew Farris <lordmorgul@gmail.com> <ajfarris@gmail.com>
gpg 0xC99B1DF3 fingerprint CDEC 6FAD BA27 40DF 707E A2E0 F0F6 E622 C99B 1DF3
No one now has, and no one will ever again get, the big picture. - Daniel Geer
---- ----

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 01-14-2008, 06:39 AM
Callum Lerwick
 
Default Preloading

The whole idea of preloading is crackrock. Its just hiding the problem,
at best.

The problem is the relentless march of bloat.
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 01-14-2008, 08:03 AM
Linus Walleij
 
Default Preloading

On Sun, 13 Jan 2008, Jeff Spaleta wrote:


That wont help the critical first impression phase..


Agreed, what "other" OS:es (Vista, MacOS X) do is to come pre-installed,
i.e. it was booted once by a technician out of sight of the user. Then

they leverage on that.

They are of course doing much more agressive things than simple
preloading, more like suspend-to-disk, so the user actually more or less
resumes a pre-booted image customized for that machine at "boot" time.


I wonder if we could do that.

Linus

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 
Old 01-14-2008, 01:18 PM
Les Mikesell
 
Default Preloading

Callum Lerwick wrote:

The whole idea of preloading is crackrock. Its just hiding the problem,
at best.

The problem is the relentless march of bloat.



One person's bloat is someone else's most needed feature - and there's
not a big problem these days with disk space to store unused options.
The problem is that the programs, their shared libraries, and config
files are splattered all over the disk and seeking to access them is
much slower than any other computer operation. There has to be some
trick that could be used either to pre-arrange all the files used in the
boot process together or load something early to pull in all of the
shared libs in a certain order, or have them preloaded in a swapfile or
something. If you change the goal to reducing disk seeks instead of
specifically preloading files it might be easier to accomplish a speedup.


--
Les Mikesell
lesmikesell@gmail.com

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list
 

Thread Tools




All times are GMT. The time now is 04:35 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org