On Wed, Oct 1, 2008 at 10:20 AM, Kyle McMartin <kyle@mcmartin.ca> wrote:
>
> Ah, I'm glad to see others working on this! I'd not seen the Mandriva
> efforts, but I should really poke at them as I'd turned modprobe into
> a library and started integrating it into udev.

By coincidence, I'd been thinking about doing the same thing the past
few days and kicking around how it might be accomplished. Do you have
any patches around? I'd like to help out if your progress is stalled.
It would be nice to just do the configuration parsing once.

--
Dan

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Wed, Oct 01, 2008 at 01:20:38PM -0400, Kyle McMartin wrote:
> On Wed, Oct 01, 2008 at 06:52:15PM +0200, Jakub Jelinek wrote:
> > MAKEDEV sources almost 500KB of config files on every invocation, and the
> > inner loop is terribly inefficient. The second patch allows it to shrink
> > the files to 55KB while expressing the same info and makes the inner loop
> > more efficient. Depending on how many modprobe and MAKEDEV invocations
> > are done on your box during bootup, this can or might not make meassurable
> > difference.
> >
>
> Definitely. The biggest win by far for MAKEDEV is profiling the often
> hit devices, and prioritizing things. Dave Airlie moved a bunch of the
> cciss and other almost never-seen devices to be sourced last and ended
> up with a huge win.

If the ordering makes significant difference, then there is further work on
it. My patch was mainly about speeding up the read_configs part, although
the more compact config files mean several times less entries as well.

makedevices then does a linear search through the entries. What we could do
is put the entries into a hash table, hashed by the name_prefix_len bytes
long prefix, or skip the entries that don't match any command line
arguments. I guess for many command line arguments the hash table could be
a win, for very few the skipping. Even just separate chains for each
starting letter could cut the number of entries down a lot (only 3 letters
with more than 100 entries), though they aren't evenly distributed; just
computing a hash and using 256 buckets with chains would be probably enough
though.

> Have you tested things on a big endian machine?

I have not, can easily do that tomorrow on ppc64. That said, the file
format has an endian 4byte word in the header to verify depmod and modprobe
agree on endianity, if they don't, the *cache files just aren't used and the
slower (still somewhat faster wrt. vanilla modprobe) reading of the text
config files is used, similarly to the case where used hand modifies them.

> really good. Hopefully Jon will apply this upstream soon so we can get
> it in F10.

That's where I don't have such high hopes, as I've mailed the patch to Jon
already more than 10 months ago.

Jakub

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Kyle McMartin wrote:

Ah, I'm glad to see others working on this! I'd not seen the Mandriva
efforts, but I should really poke at them as I'd turned modprobe into
a library and started integrating it into udev.


What an un-unixish design!

One could achieve the same performance with a trivial change that would
keep modprobe and udev independent and simple:


cat | modprobe --stdin
radeon
drm
foo
FATAL: Module foo not found.
e1000
^D

Well, you get the idea... :-)

--
\___/ Bernie Innocenti - http://www.codewiz.org/
_| X | Sugar Labs Team - http://www.sugarlabs.org/
|_O_| "It's an education project, not a laptop project!"

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Thu, Oct 02, 2008 at 05:06:30AM +0200, Bernie Innocenti wrote:
> Kyle McMartin wrote:
>> Ah, I'm glad to see others working on this! I'd not seen the Mandriva
>> efforts, but I should really poke at them as I'd turned modprobe into
>> a library and started integrating it into udev.
>
> What an un-unixish design!
>
> One could achieve the same performance with a trivial change that would
> keep modprobe and udev independent and simple:
>
> cat | modprobe --stdin
> radeon
> drm
> foo
> FATAL: Module foo not found.
> e1000
> ^D
>
> Well, you get the idea... :-)

The way modprobe is written, just changing it to a library or handling
multiple requests from one command doesn't help much, as for every argument
modprobe reads all config files again anyway and for alias handling the most
costly is the search among the aliases anyway and searching the deps is
somewhat costly too. Reading the config files just once is of course fixable
(both in the library and --stdin models) but the searching would still be
an issue. Either modprobe can create the hash tables or search tree for
aliases on the fly, or, what I've done in my patch, let depmod create it
ahead of time and modprobe just use it.

Jakub

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Wed, 1 Oct 2008, Jakub Jelinek wrote:


I thought I'd share my modprobe and MAKEDEV speedup patches with
a wider community so folks can experiment with them, especially seeing
Mandriva folks playing with turning modprobe into a daemon because of its
slowness.


GOOD WORK.

A common approach in the embedded community is to let go of modprobe
altogether and only use compiled-in modules and the occasional insmod to
speed things up. (Which is quite easy when you know your hardware, but
also messy to maintain.)


With a sufficiently fast modprobe that need should go away. I hope we'll
soon be there with these patches and perhaps a few more jumping in.


Linus

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Wed, 2008-10-01 at 18:52 +0200, Jakub Jelinek wrote:
> Hi!
>

> MAKEDEV sources almost 500KB of config files on every invocation, and the
> inner loop is terribly inefficient. The second patch allows it to shrink
> the files to 55KB while expressing the same info and makes the inner loop
> more efficient. Depending on how many modprobe and MAKEDEV invocations
> are done on your box during bootup, this can or might not make meassurable
> difference.
>

I've just pushed MAKEDEV 3.23-7 into koji should be in rawhide soon.

MAKEDEV is now hosted on git.fedorahosted.org for anyone who cares.

Thanks Jakub.

Dave.

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

Dave Airlie wrote:

On Wed, 2008-10-01 at 18:52 +0200, Jakub Jelinek wrote:

Hi!




MAKEDEV sources almost 500KB of config files on every invocation, and the
inner loop is terribly inefficient. The second patch allows it to shrink
the files to 55KB while expressing the same info and makes the inner loop
more efficient. Depending on how many modprobe and MAKEDEV invocations
are done on your box during bootup, this can or might not make meassurable
difference.



I've just pushed MAKEDEV 3.23-7 into koji should be in rawhide soon.

MAKEDEV is now hosted on git.fedorahosted.org for anyone who cares.

Thanks Jakub.


It is time for us to drop the exception in

http://fedoraproject.org/wiki/Packaging/SourceURL#We_are_Upstream

Insist that we use fedorahosted.org or other project hosting websites
whenever we are upstream for a project? It certainly makes it easier for
others to consume our bits when they don't have to crawl through source
rpm packages. SRPM dumps are a horrible way to manage upstream source
code anyway.


Rahul

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Thu, Oct 09, 2008 at 10:28:02AM +0530, Rahul Sundaram wrote:
> Dave Airlie wrote:
>> On Wed, 2008-10-01 at 18:52 +0200, Jakub Jelinek wrote:
>>> Hi!
>>>
>>
>>> MAKEDEV sources almost 500KB of config files on every invocation, and the
>>> inner loop is terribly inefficient. The second patch allows it to shrink
>>> the files to 55KB while expressing the same info and makes the inner loop
>>> more efficient. Depending on how many modprobe and MAKEDEV invocations
>>> are done on your box during bootup, this can or might not make meassurable
>>> difference.
>>>
>>
>> I've just pushed MAKEDEV 3.23-7 into koji should be in rawhide soon.
>>
>> MAKEDEV is now hosted on git.fedorahosted.org for anyone who cares.
>>
>> Thanks Jakub.
>
> It is time for us to drop the exception in
>
> http://fedoraproject.org/wiki/Packaging/SourceURL#We_are_Upstream
>
> Insist that we use fedorahosted.org or other project hosting websites
> whenever we are upstream for a project? It certainly makes it easier for
> others to consume our bits when they don't have to crawl through source
> rpm packages. SRPM dumps are a horrible way to manage upstream source
> code anyway.

I remember an old thread where it was agreed by all participants in
general that using any kind of hosting beside srpm was a must in fedora
(be it in fedorahosted.org, a simple html page with downloads, a project
in savannah or wherever).

--
Pat

--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Thu October 9 2008, Rahul Sundaram wrote:

> It is time for us to drop the exception in
>
> http://fedoraproject.org/wiki/Packaging/SourceURL#We_are_Upstream
>
> Insist that we use fedorahosted.org or other project hosting websites
> whenever we are upstream for a project? It certainly makes it easier for
> others to consume our bits when they don't have to crawl through source
> rpm packages. SRPM dumps are a horrible way to manage upstream source
> code anyway.

I agree. I find it also useful as a potential patch provider to have a clear
upstream location where development can be watched and patches can be
tracked.

Regards,
Till
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list

On Thu, 2008-10-09 at 10:28 +0530, Rahul Sundaram wrote:
> Insist that we use fedorahosted.org or other project hosting websites
> whenever we are upstream for a project? It certainly makes it easier for
> others to consume our bits when they don't have to crawl through source
> rpm packages. SRPM dumps are a horrible way to manage upstream source
> code anyway.

Only if we allow source repo tags to be a suitable source distribution
method. Since fedorahosted still doesn't have an easy to use way of
distributing tarballs, and getting the code via a scm checkout is quite
easy, it should suffice.

--
Jesse Keating
Fedora -- Freedom² is a feature!
identi.ca: http://identi.ca/jkeating
--
fedora-devel-list mailing list
fedora-devel-list@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-devel-list