FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Ubuntu > Ubuntu Kernel Team

 
 
LinkBack Thread Tools
 
Old 07-14-2012, 07:51 PM
 
Default Simplifying kernel configuration for distro issues

On Sat, 14 Jul 2012, Cyrill Gorcunov wrote:


On Sat, Jul 14, 2012 at 07:48:27PM +0200, Borislav Petkov wrote:

On Sat, Jul 14, 2012 at 04:43:32PM +0400, Cyrill Gorcunov wrote:

For example to enable "PCI driver for virtio devices" I need to go to
Device Drivers -> Virtio drivers, while I think it would be great to
have everything virt. related in Virtualization section.


Actually, we need something more generic than that: everything X-related
should be automatically selected when setting CONFIG_X. And X can be any
subset of configuration options which belong to one feature, be it KVM,
distro-specific stuff, or CPU-vendor specific stuff, or whatever.

I can imagine, for example, that when a user wants to have an
AMD-specific config, it automatically selects CPU_SUP_AMD, X86_MCE_AMD
(for MCE), MICROCODE_AMD (microcode support), AGP_AMD64, EDAC_AMD64 and
a bunch of other AMD-specific features.


sure, no doubts, it would be great! (I must admit I never looked up into
kconfig parser code so i don't know if this could be achieved easily).


think of this as a variation of the miniconfig problem.

Instead of the miniconfig being one file that specifies everything you
want, that you then use to generate a real .config file, you instead have
one _or_ _more_ config files that get combined and then used to generate
the real .config file.


Doing this as a set of miniconfig snippets instead of as menu options in
the main kconfig has the advantage that once the real .config is created,
it can then be edited freely, without worrying about things like SELINUX
being enabled when you want to do development on a different LSM.


The distro config would be the first of these, but then you could have
multiple hardware specific config files (a vmware config, a KVM config,
one or more config files for the systems that you own, a USB config to
enable a bunch of the various USB deices that you want to support, etc)


The distros may choose to produce several miniconfig files, one the
minimum infrastructure features they need, and then additional files for
the rest of their config. For example, I could see them having a set of
files


1. Core
2. architecture
3. architecture specific drivers
4. architecture neutral drivers (USB devices and similar)


starting off with the distro config, the people creating this are going to
be very familiar with kconfig and could create these by hand, but going
forward we are going to want to have some tools to help sysadmins to
create these files. Not knowing kconfig, it may be as simple as taking the
miniconfig snippets that you start with, creating a .config and then doing
a diff of that .config with the one the admin created, using the result as
the new miniconfig snippet.


David Lang

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 07-15-2012, 10:09 PM
 
Default Simplifying kernel configuration for distro issues

On Mon, 16 Jul 2012, Cyrill Gorcunov wrote:


Replying to David's message (sorry for delay) I fear having a bunch of
miniconfig files will end up in a mess. Maybe (maybe (!) I don't know since
I've no time at moment to read kconfig code and I'm not sure if this
is right direction at all) it would worth to add some new keyword to
kconfig language, say "profile", which would tag symbol to a category
if needed, and these categories included into profiles automatically.
On the other hands this might end up in a mess as well.


I have a couple problems with the approach of modifying the existing
kconfig files


1. how does it handle the case when a profile wants something one way and
the admin wants it another way


the example is the fedora default wanting SELINUX and I want some other
LSM


2. since it requires making changes in the upstream kernel source, the
number of people who can make these changes is small.


3. since all these changes go into the upstream kernel source, changes to
these profiles are going to be visible churn (think of the issues with the
defconfigs for ARM a couple of years ago)


4. the complexity of tagging all possible profiles is very high.

Even if you limit the profiles to "Linux Distros", how many different
distros are there? Do you really want to have to start arguing over which
distros are large enough to get their profile added to the upstream kernel
source?



If instead we go with something along the lines of the miniconf approach,
the picture looks very different


1. this approach only sets things one time, after that the person doing
the compile is free to change anything.


2. since the changes are a local file, separate from the upstream source,
issues like who can provide the config, what distros will accept it,
complexity in having many different options, etc all vanish


3. by simply combining miniconfig files, you can combine sets of
pre-defined options


David Lang

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 07-15-2012, 11:06 PM
 
Default Simplifying kernel configuration for distro issues

On Mon, 16 Jul 2012, Cyrill Gorcunov wrote:


On Sun, Jul 15, 2012 at 03:09:12PM -0700, david@lang.hm wrote:


3. by simply combining miniconfig files, you can combine sets of
pre-defined options


Wait, David, I'm lost. These miniconfigs should live somewhere on
my home directory (if they are out of mainline tree)?


the distro provided miniconfig files should live in a "known location"
(like the distro specific scripts for "make install", but I'm saying that
we should be able to use this same mechanism to include locally generated
files.


combining external files doesn't need to be the job of the kernel kconfig
infrastructure either. There could be external scripts that take one or
more distro provided files and one or more local files and combine them
for the kconfig infrastructure.


For a distro, I could see their build farm using this as well, with a file
organization along the lines of


base config
arch specific options (one per arch)
generic drivers (USB devices, etc)

This sort of thing would help reduce the chances of there being unexpected
differences between the different architecture builds.


this can be done without having to modify the existing kconfig stuff at
all (by using make oldconfig), but it's likely that a little bit of
support from kconfig could make creating these miniconifg files much
easier.


Linus started off wanting the core requirements, other people want to
duplicate the full distro kernel, others want everything but the drivers,
others want to substatute a list of driver options that their local
systems need, etc. by using separate files that are external to the
kconfig files in the kernel.org source tree, I think that all these people
can be satisfied without an explosion of complexity in the source tree.


David Lang

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 07-16-2012, 04:43 PM
 
Default Simplifying kernel configuration for distro issues

On Mon, 16 Jul 2012, Borislav Petkov wrote:


On Sun, Jul 15, 2012 at 03:09:12PM -0700, david@lang.hm wrote:

On Mon, 16 Jul 2012, Cyrill Gorcunov wrote:


Replying to David's message (sorry for delay) I fear having a bunch of
miniconfig files will end up in a mess. Maybe (maybe (!) I don't know since
I've no time at moment to read kconfig code and I'm not sure if this
is right direction at all) it would worth to add some new keyword to
kconfig language, say "profile", which would tag symbol to a category
if needed, and these categories included into profiles automatically.
On the other hands this might end up in a mess as well.


I have a couple problems with the approach of modifying the existing
kconfig files

1. how does it handle the case when a profile wants something one
way and the admin wants it another way


Select the profile and then fixup the config the normal way.

If what the admin wants is incompatible with the profile, admin doesn't
select the profile.


the example is the fedora default wanting SELINUX and I want some
other LSM


Currently, if you run fedora and want something that's not enabled, you
recompile your kernel too, right?


The problem is that you can't select the Fedora profile and then unselect
SELINUX, so the profile will do you no good.


It's not a matter of needing to recompile or not, it's a mattter that when
you want to recompile, you want to be able to easily select the
infrastructure pieces that the distro needs to operate, without also
getting their huge selection of 'other' things.



2. since it requires making changes in the upstream kernel source, the
number of people who can make these changes is small.


I think that's moot since you either select the profile or you don't.


3. since all these changes go into the upstream kernel source, changes
to these profiles are going to be visible churn (think of the issues
with the defconfigs for ARM a couple of years ago)


AFAICT, those changes will be needed only for a new distro release and
that happens twice a year, tops.


4. the complexity of tagging all possible profiles is very high.


That's why we start simple.


Even if you limit the profiles to "Linux Distros", how many different
distros are there? Do you really want to have to start arguing over
which distros are large enough to get their profile added to the
upstream kernel source?


That's a valid question; its answer could be defined arbitrarily.

<joshing>
Let's say all distros which make money - more than a certain large
amount - are allowed. :-)
</joshing>


So that would eliminate all linux distros except for Red Hat Enterprise
and Suse (including eliminating opensuse and fedora), somehow I don't
think that would produce the benefit that Linus was looking for.



If instead we go with something along the lines of the miniconf
approach, the picture looks very different

1. this approach only sets things one time, after that the person
doing the compile is free to change anything.


...

Sorry, I don't see the simplification: you need to rebuild your kernel
anyway and before you rebuild it, you can do all the changes you want.
So either you select a profile or you load a miniconfig, it doesn't
really matter how you do it?


Yes, you have to recompile in any case.

The difference is that with a profile, the profile must be in the kernel
source, and you will have problems if you want to start with the profile
and then tweak something.


with miniconfig, you can have locally defined configs, you can combine
multiple configs, and the configs define a starting point, but you can
then override them if you want to.


David Lang

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 07-16-2012, 05:01 PM
Alan Cox
 
Default Simplifying kernel configuration for distro issues

> Select the profile and then fixup the config the normal way.
>
> If what the admin wants is incompatible with the profile, admin doesn't
> select the profile.

Thats ugly - "distro except..." is a standard thing you ask users to do
for debugging.

However providing you separate the initial profile from the later tools
it simply becomes

make distroconfig
[cp /etc/system-kconfig(.$ARCH?) .config
make oldconfig]

make menuconfig (if you want to customise)

In addition the make oldconfig means you can ship a deliberately
incomplete distroconfig and get the user asked some bits.

make



--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 07-16-2012, 05:05 PM
 
Default Simplifying kernel configuration for distro issues

On Mon, 16 Jul 2012, Alan Cox wrote:


Select the profile and then fixup the config the normal way.

If what the admin wants is incompatible with the profile, admin doesn't
select the profile.


Thats ugly - "distro except..." is a standard thing you ask users to do
for debugging.

However providing you separate the initial profile from the later tools
it simply becomes

make distroconfig
[cp /etc/system-kconfig(.$ARCH?) .config
make oldconfig]

make menuconfig (if you want to customise)

In addition the make oldconfig means you can ship a deliberately
incomplete distroconfig and get the user asked some bits.


exactly what I was fumbling to describe, thanks.

David Lang

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 07-16-2012, 07:26 PM
 
Default Simplifying kernel configuration for distro issues

On Mon, 16 Jul 2012, Linus Torvalds wrote:


On Mon, Jul 16, 2012 at 9:43 AM, <david@lang.hm> wrote:


The problem is that you can't select the Fedora profile and then unselect
SELINUX, so the profile will do you no good.


Guys, stop it now.

Your "problem" isn't what any sane person cares about, and isn't what
I started the RFC for.

Seriously. NOBODY CARES.

You can do what you want to do *today*. Just edit the config file, or
use any of the millions of config tools. Stop whining.

The thing I'm asking for is for normal people. Make it easy for people
who DO NOT CARE about the config file to just build a kernel for their
machine.

Don't complicate the issue by bringing up some totally unrelated
question. Don't derail a useful feature for the 99% because you're not
in it.


Some of the proposed ways to implement the minimum distro kernel would not
allow you to override the distro defaults because they would be
implemented by setting dependancies, not by selecting options that you as
the user could then unselect.


If you have to edit the .config file, and then have your next make *config
override your edits (due to dependancy resolution) and so have to edit the
.config file again this is a really ugly way of working.


David Lang

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 07-16-2012, 10:21 PM
 
Default Simplifying kernel configuration for distro issues

On Mon, 16 Jul 2012, Linus Torvalds wrote:


On Mon, Jul 16, 2012 at 12:26 PM, <david@lang.hm> wrote:


Some of the proposed ways to implement the minimum distro kernel would not
allow you to override the distro defaults because they would be implemented
by setting dependancies, not by selecting options that you as the user could
then unselect.


The sanest thing to do is just a list of "select" statements. And in
any case it would have to depend on the "distro config" entry, so EVEN
THEN you could just create the Kconfig file, then edit out the distro
config thing, and then do whatever you want.

So I don't see why you're arguing. Even if it is very much a "if you
select the FEDORA_X kconfig option, that will automatically force all
those other options on", I don't see why you are so upset. Just set
it, generate your kconfig, and then edit to file to NOT set the distro
config. Then you can do whatever the hell you want.

What I'm upset about is simply the fact that EVEN IF your arguments
were valid (and I don't think they are: see above for trivial fix) I
think your arguments are bogus, for the simple reason that what you
ask for isn't even what the whole "distro minconfig" is at all about.
It's about normal users who absolutely DO NOT want to edit the config.
People who want to say "I know I have intel/amd/nvidia graphics,
standard USB, and no CD-ROM drive". They couldn't care *less* about
SELINUX, and in fact that very issue is why we should have this.

Anybody who says "I want to run Fedora without SELINUX because I do my
own security development" is by *definition* not relevant to the whole
feature.


Don't mistake the example for the feature. the SELINUX thing is just an
example. As Alan Cox commented, taking a distro config and disabling one
thing is a common troubleshooting request from kernel developers.


I am not that familiar with the internals of kconfig, but my understanding
was that if you select feature X that then turns on A, B, and C, if you
then edit the .config file to remove feature X features A, B, and C will
be turned off the next time you do a make *config. If this is not the
case, then it's not as horrible as I was thinking.


However, there's still the question of how many different variations of
how many different distros are you going to accept into the upstream
kernel. How many different versions of Fedora are you going to be willing
to have in the menu? This seems to me to be the defconfig problem
multiplied by many distros and many versions of each distro. If we can
eliminate this combinational explosion by referencing something outside of
the kernel tree, it seems to me that it's a big win.


As Alan Cox put it in code


make distroconfig
[cp /etc/system-kconfig(.$ARCH?) .config
make oldconfig]

make menuconfig (if you want to customise)


with the distros providing /etc/system-kconfig files seems far more
scalable as well as far more flexible than either setting up a lot of
dependancies for new tags or adding a field to every option to make it
part of some profile.


This approach works both for the users who just want to say "I know I have
intel/amd/nvidia graphics, standard USB, and no CD-ROM drive", for the
users who know a bit more, but want to start with a known point, and for
the production sysadmins who want to use the same mechanism for other
purposes (standardizing builds for systems as kernel versions move
forward)


David Lang

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 07-18-2012, 08:42 AM
 
Default Simplifying kernel configuration for distro issues

On Wed, 18 Jul 2012, Ingo Molnar wrote:


* david@lang.hm <david@lang.hm> wrote:


Anybody who says "I want to run Fedora without SELINUX
because I do my own security development" is by *definition*
not relevant to the whole feature.


Don't mistake the example for the feature. the SELINUX thing
is just an example. As Alan Cox commented, taking a distro
config and disabling one thing is a common troubleshooting
request from kernel developers.


It's still irrelevant:

- if a user chooses a distro config it means that he is using
that distro. Disabling an essential component of the distro
config, even if a kernel developer asks for it, will likely
break that distro and is thus a dumb thing to do. (the
typical user will also be unlikely to be *able* to edit a
.config and make sure it works.)


that's assuming that everything listed really is essential.

The history of features defaulting to 'Y' in the existing kernel config
doesn't give me great confidence that reality will be anywhere close to
this ideal.



- Furthermore, there's *already* over ten thousand select's in
our Kconfig's, and it's already hard at times to disable
dependent options.

- I've been using what Linus suggested for many years via
private patches to do bootable randconfig testing and the
concept works just fine - enabling a distro specific
minconfig is absolutely useful, I'm glad it's being pursued
upstream as well...

So what you are arguing about is IMO irrelevant, it is
immaterial to the problem at hand and the concept works just
fine in practice.


Shrug, you guys have to maintain the result, I'm just a user.

But I don't see why the same logic that kept the kernel installation
outside of the makefiles and created /sbin/installkernel wouldn't also
apply here.


using a separate miniconfig in a known place would seem to be less code,
distribute the work better (as every distro can use it without having to
patch the same files in the kernel source), and be more flexible.


Flexibility has a way of being leveraged in ways never imagined initially,
so if it can be gained without complicating the code (and it's
maintinance), and the initial use case, I always tend to push for the more
flexible option.


David Lang

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 
Old 07-19-2012, 04:01 PM
Michal Marek
 
Default Simplifying kernel configuration for distro issues

On 17.7.2012 10:03, Geert Uytterhoeven wrote:
> On Mon, Jul 16, 2012 at 10:56 PM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>> On Mon, Jul 16, 2012 at 12:26 PM, <david@lang.hm> wrote:
>>> Some of the proposed ways to implement the minimum distro kernel would not
>>> allow you to override the distro defaults because they would be implemented
>>> by setting dependancies, not by selecting options that you as the user could
>>> then unselect.
>>
>> The sanest thing to do is just a list of "select" statements. And in
>> any case it would have to depend on the "distro config" entry, so EVEN
>> THEN you could just create the Kconfig file, then edit out the distro
>> config thing, and then do whatever you want.
>
> Except that "select" is one of the ugliest things in Kconfig, as it
> blindly sets a symbol
> without checking if its dependencies are fulfilled.

But for the few options Linus proposed (TMPFS, TMPFS_POSIX_*,
DEVTMPFS(_MOUNT)), the amount of additional dependencies is reasonable.
For something more advanced like 'build me a kernel for a laptop with
$VENDOR hardware', we would need a better dependency solver, indeed.

Michal

--
kernel-team mailing list
kernel-team@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/kernel-team
 

Thread Tools




All times are GMT. The time now is 11:46 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright 2007 - 2008, www.linux-archive.org