FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Kernel Team

 
 
LinkBack Thread Tools
 
Old 03-16-2010, 05:09 PM
Andy Whitcroft
 
Default Built-in modules review

On Tue, Mar 16, 2010 at 01:01:23PM -0400, Chase Douglas wrote:

> I'm trying to figure out the rationale for the decision to include
> only some of the SATA modules. If it's really size in the kernel, then
> it can make sense to only statically include those modules that are
> used often. But there's a tradeoff in speed I assume, so we're sort of
> telling one group of people that they have popular hardware so we are
> giving them the benefit of speed, but we tell the other group that
> they don't have popular hardware, so we aren't giving them the benefit
> of speed.
>
> This type of decision sort of gets us down a path of decisions that
> are not easily distinguishable. People will start to wonder why some
> vendor drivers are included and others are not. If there's a steep
> drop off where 99% of users use 3 drivers, and 1% use the other 10
> drivers (or however many there are), then this approach can be
> articulated easily. But if the gap is more like 33% use the top 3
> drivers and 66% use the other 10 drivers, then our stance will be less
> clear.
>
> It can also make maintenance harder. If all the modules are static or
> dynamic, then we will see the same issues across them. But if we move
> the lesser used modules to the initramfs to be loaded dynamically then
> we may not have thorough testing on them and we might not find
> regressions due to the loading process itself.
>
> What do we really lose by statically building all the SATA modules in?
> Is there a known value here, or are we just assuming that there's some
> large cost?

We are already in the position actually where some SATA and indeed some
PATA drivers are not builtin, I suspect this is because they are newer
drivers. What we have is an arbitrary subset built in, and we want to
move to a more selective set builtin, with a more coherant reason.

We have seen a number of issues from generally pulling things in. We are
in a less flexible position for any driver which is built-in. For one it
cannot be overridden in a backports module, options are more difficult
to change, and sometimes no longer changable. As a result we have a
preference for things to be modular where that is not too high a cost.
This is expecially true where we have a long term support release on our
hands as there will be more unknown issues coming down the pipe for longer.

As an example we have a number of issues which were only triggered
when we brought all of the PATA drivers into the kernel in Karmic.
Ordering was no longer controllable and in some cases the wrong driver
loaded due to link ordering. This led to the requirement to build new
kernels for these users, a very undesirable situation. Indeed one which
has triggered this review.

What I am trying to say is that there is a cost for not building them in,
but there is also a cost for building them in. The cost for 'not' is a
very small boot penalty, the cost 'for' is the flexibility for us to configure, update,
and replace drivers on an ongoing basis.

Yes we are saying some h/w is more popular and therefore gets slightly
different treatment, but that would not be a new precident. We enabled
KMS on i915 first because it worked there better. Expediency always
colours our choices.

-apw

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-16-2010, 05:38 PM
Chase Douglas
 
Default Built-in modules review

On Tue, Mar 16, 2010 at 2:09 PM, Andy Whitcroft <apw@canonical.com> wrote:
> What I am trying to say is that there is a cost for not building them in,
> but there is also a cost for building them in. *The cost for 'not' is a
> very small boot penalty, the cost 'for' is the flexibility for us to configure, update,
> and replace drivers on an ongoing basis.

So is this small boot penalty something we really need to avoid in the
most popular hardware? If it's very small, why not make them all
dynamic?

If this decision has been made with careful consideration of the
weighting of boot speed vs flexibility, then I'm satisfied. I just
want to be sure we don't encounter the exact same problem we are
having the the pata drivers in the name of a few milliseconds.

-- Chase

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-17-2010, 08:48 AM
Stefan Bader
 
Default Built-in modules review

Andy Whitcroft wrote:
> During the karmic cycle we built-in a number of modules. Some as part
> of the removal of udev random module loading, and some as a part of
> trying to speed up the boot. At that time modprobe was significantly
> slower than it is now and it is appropriate to review that decision.
>
> We have also had a number of problems reported wherein people can no
> longer parameterise their modules as they wish and/or replace them
> easily to test updated features. In one specific case the wrong PATA
> driver is binding to the device and as it is built-in there is no longer
> a remedy.
>
> I have prepared a summary of the sub-systems and drivers as built-in
> currently and added them to the blueprint kernel-lucid-config-review's
> spec, at the URL below:
>
> http://tinyurl.com/ygvwbks

I think at some time the definition was boot essential as to make the common
boot speed faster. AS the module loading is much faster I think the issue is
probably reduced to have a kernel which might be used without a initramfs under
the right circumstances. Wheren't server or virtual kernels used that way?
And if that is required in some cases keep the configuration common if it does
not hurt.

> Filesystems: we currently have all of the ext2,3,4 filesystems builtin,
> plus all of the diagnostics filesystems (procfs et al), and also fuse.
> Other than FUSE these are all boot essential. I believe FUSE is
> required for automounting which needs confirmation. No changes
> required.
>
> Subsystems: of those listed the only one which appears to be a
> candidate for modularising is netfilter, I cannot recall this being
> required builtin.
>
> Network Protocols: again these seem reasonable, the only candidates for
> review being DCB and XFRM, if anyone can confirm that we need those.
>
> ATA Drivers: we have the majority of the PATA and SATA drivers built in.
> We already have reported issues with the PATA drivers where blacklisting
> could be used as a work around if they were not built in. We are also
> exposing users to some level of risk including all these drivers which
> are not used. In testing on the reference platform I did see a minor
> but measurable performance hit to modularising the SATA driver there.
> As we are moving to SATA over time I would propose we pull out all of
> the PATA drivers and build in only the 2 or 3 most common SATA drivers.

I think none of the PATA drivers are required to boot without initramfs while
the most common SATA drivers might be. So It sounds reasonable to proceed that way.

> Input drivers: here most things are modules already. TOUCHSCREEN and
> TABLET are possible candidates for modularising.
>
> HID drivers: we already took action to move HID to modules and most of
> the actual drivers already appear to be modules. No changes required.
>
> If people are in agreement I will put together a patch set to make these
> changes.
>
> Comments?

I think there was a comment on bluetooth somewhere being inconsistent. We likely
want that as a module everywhere because compat-wireless includes it.

> -apw
>


--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-17-2010, 10:50 AM
Andy Whitcroft
 
Default Built-in modules review

On Tue, Mar 16, 2010 at 02:38:23PM -0400, Chase Douglas wrote:
> On Tue, Mar 16, 2010 at 2:09 PM, Andy Whitcroft <apw@canonical.com> wrote:
> > What I am trying to say is that there is a cost for not building them in,
> > but there is also a cost for building them in. *The cost for 'not' is a
> > very small boot penalty, the cost 'for' is the flexibility for us to configure, update,
> > and replace drivers on an ongoing basis.
>
> So is this small boot penalty something we really need to avoid in the
> most popular hardware? If it's very small, why not make them all
> dynamic?

Its small, but large enough to bust our boot budget, we were less than
.1s inside budget overall last I looked.

> If this decision has been made with careful consideration of the
> weighting of boot speed vs flexibility, then I'm satisfied. I just
> want to be sure we don't encounter the exact same problem we are
> having the the pata drivers in the name of a few milliseconds.

-apw

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-17-2010, 10:51 AM
Andy Whitcroft
 
Default Built-in modules review

On Tue, Mar 16, 2010 at 03:08:38PM +0000, Andy Whitcroft wrote:

> ATA Drivers: we have the majority of the PATA and SATA drivers built in.
> We already have reported issues with the PATA drivers where blacklisting
> could be used as a work around if they were not built in. We are also
> exposing users to some level of risk including all these drivers which
> are not used. In testing on the reference platform I did see a minor
> but measurable performance hit to modularising the SATA driver there.
> As we are moving to SATA over time I would propose we pull out all of
> the PATA drivers and build in only the 2 or 3 most common SATA drivers.

Brad, can the data collection you have been doing over our linux bugs
give us any insight into which drivers are most commonly used, or at
least which physical h/w is most common?

-apw

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-17-2010, 01:08 PM
Leann Ogasawara
 
Default Built-in modules review

On Wed, 2010-03-17 at 11:51 +0000, Andy Whitcroft wrote:
> On Tue, Mar 16, 2010 at 03:08:38PM +0000, Andy Whitcroft wrote:
>
> > ATA Drivers: we have the majority of the PATA and SATA drivers built in.
> > We already have reported issues with the PATA drivers where blacklisting
> > could be used as a work around if they were not built in. We are also
> > exposing users to some level of risk including all these drivers which
> > are not used. In testing on the reference platform I did see a minor
> > but measurable performance hit to modularising the SATA driver there.
> > As we are moving to SATA over time I would propose we pull out all of
> > the PATA drivers and build in only the 2 or 3 most common SATA drivers.
>
> Brad, can the data collection you have been doing over our linux bugs
> give us any insight into which drivers are most commonly used, or at
> least which physical h/w is most common?

I have data for drivers most commonly used [1], but I admit it's a bit
dated (last updated Dec 2009). The data is from all hw profiles
submitted to Launchpad. Just sort the "# Driver Owners" column by
clicking on the column header.

Leann

[1] http://qa.ubuntu.com/reports/ogasawara/hwdb/driver-stats.html


--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-17-2010, 02:02 PM
Brad Figg
 
Default Built-in modules review

On 03/17/2010 04:51 AM, Andy Whitcroft wrote:
> On Tue, Mar 16, 2010 at 03:08:38PM +0000, Andy Whitcroft wrote:
>
>> ATA Drivers: we have the majority of the PATA and SATA drivers built in.
>> We already have reported issues with the PATA drivers where blacklisting
>> could be used as a work around if they were not built in. We are also
>> exposing users to some level of risk including all these drivers which
>> are not used. In testing on the reference platform I did see a minor
>> but measurable performance hit to modularising the SATA driver there.
>> As we are moving to SATA over time I would propose we pull out all of
>> the PATA drivers and build in only the 2 or 3 most common SATA drivers.
>
> Brad, can the data collection you have been doing over our linux bugs
> give us any insight into which drivers are most commonly used, or at
> least which physical h/w is most common?
>
> -apw
>

It could. It would take a little more work on my part but not too much
I wouldn't think.

Brad
--
Brad Figg brad.figg@canonical.com http://www.canonical.com

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-17-2010, 05:31 PM
Scott James Remnant
 
Default Built-in modules review

On Tue, 2010-03-16 at 15:08 +0000, Andy Whitcroft wrote:

> Filesystems: we currently have all of the ext2,3,4 filesystems builtin,
> plus all of the diagnostics filesystems (procfs et al), and also fuse.
> Other than FUSE these are all boot essential. I believe FUSE is
> required for automounting which needs confirmation. No changes
> required.
>
FUSE is built-in because we have no way of loading it
otherwise; /dev/fuse would not exist without the module loaded, and fuse
is generally used by user processes which don't have permission to
modprobe.

> ATA Drivers: we have the majority of the PATA and SATA drivers built in.
> We already have reported issues with the PATA drivers where blacklisting
> could be used as a work around if they were not built in. We are also
> exposing users to some level of risk including all these drivers which
> are not used. In testing on the reference platform I did see a minor
> but measurable performance hit to modularising the SATA driver there.
> As we are moving to SATA over time I would propose we pull out all of
> the PATA drivers and build in only the 2 or 3 most common SATA drivers.
>
I think we did this given the idea we could skip the initramfs; we went
down a different path and optimised the initramfs instead.

I think only the most common SATA driver or two is needed here.

Scott
--
Scott James Remnant
scott@ubuntu.com
--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-17-2010, 05:48 PM
Chase Douglas
 
Default Built-in modules review

On Wed, Mar 17, 2010 at 2:31 PM, Scott James Remnant <scott@ubuntu.com> wrote:
> On Tue, 2010-03-16 at 15:08 +0000, Andy Whitcroft wrote:
>
>> Filesystems: we currently have all of the ext2,3,4 filesystems builtin,
>> plus all of the diagnostics filesystems (procfs et al), and also fuse.
>> Other than FUSE these are all boot essential. *I believe FUSE is
>> required for automounting which needs confirmation. *No changes
>> required.
>>
> FUSE is built-in because we have no way of loading it
> otherwise; /dev/fuse would not exist without the module loaded, and fuse
> is generally used by user processes which don't have permission to
> modprobe.

Is this because fuse is broken as a module, or because it just needs
to be loaded at boot because users can't do it without an escalation
of privileges? If it's the latter then why can't we load it
dynamically at boot time?

-- Chase

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 03-17-2010, 05:50 PM
Scott James Remnant
 
Default Built-in modules review

On Wed, 2010-03-17 at 14:48 -0400, Chase Douglas wrote:

> On Wed, Mar 17, 2010 at 2:31 PM, Scott James Remnant <scott@ubuntu.com> wrote:
> > On Tue, 2010-03-16 at 15:08 +0000, Andy Whitcroft wrote:
> >
> >> Filesystems: we currently have all of the ext2,3,4 filesystems builtin,
> >> plus all of the diagnostics filesystems (procfs et al), and also fuse.
> >> Other than FUSE these are all boot essential. I believe FUSE is
> >> required for automounting which needs confirmation. No changes
> >> required.
> >>
> > FUSE is built-in because we have no way of loading it
> > otherwise; /dev/fuse would not exist without the module loaded, and fuse
> > is generally used by user processes which don't have permission to
> > modprobe.
>
> Is this because fuse is broken as a module, or because it just needs
> to be loaded at boot because users can't do it without an escalation
> of privileges? If it's the latter then why can't we load it
> dynamically at boot time?
>
Load it dynamically by doing what?

We have two module loading facilities:

- load because a kernel uevent contains a MODALIAS string that expands
to the module name (ie. load on hardware)

- load because the user opens the device node (load on demand)

Since fuse has no hardware, and no device node until loaded, neither of
these methods works.

So we build it in.

Scott
--
Scott James Remnant
scott@ubuntu.com
--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 

Thread Tools




All times are GMT. The time now is 11:56 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org