FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > Debian > Debian User

 
 
LinkBack Thread Tools
 
Old 09-24-2011, 09:07 PM
Ralf Mardorf
 
Default Reducing kernel compilation time

On Sat, 2011-09-24 at 22:44 +0200, Ralf Mardorf wrote:
> On Sat, 2011-09-24 at 20:20 +0000,
> debian-user-digest-request@lists.debian.org wrote:
> > CONCURRENCY_LEVEL=2
>
> I didn't read the thread, just one mail.
>
> For my 2.1 GHz dual-core Athlon CONCURRENCY_LEVEL does minimal reduce
> compiling time. I suspect that consequently reducing unneeded stuff
> would be much more time reducing, unfortunately it takes a lot of time
> to edit such a basic config. I never did and still suffer when compiling
> a kernel and regarding to my audio needs I need to build kernels very
> often.
>
> Fortunately I'm able to use my computer, while building a kernel,
> unfortunately it might take an hour, before I know, that something fishy
> does stop building the kernel and this could happen 3 or 4 times again,
> before a kernel will be build.
>
> Yep, it's a PITA, OTOH, for other OSs there's no possibility to build a
> kernel that fits to personal needs.
>
> Live is a PITA

An example

spinymouse@debian:/boot$ cat config-2.6.39.1 | grep TABLET
CONFIG_INPUT_TABLET=y
CONFIG_TABLET_USB_ACECAD=m
CONFIG_TABLET_USB_AIPTEK=m
CONFIG_TABLET_USB_GTCO=m
# CONFIG_TABLET_USB_HANWANG is not set
CONFIG_TABLET_USB_KBTAB=m
CONFIG_TABLET_USB_WACOM=m

I would like to have a tablet, but I don't have got a tablet. When
building this kernel, one time something unneeded was build as part of
the kernel and 5 times an unneeded module was build. Just one unneded
thingy wasn't build.

You can grep anything else and you will see, that you build tons of
unneeded stuff.



--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 1316898460.5414.2.camel@debian">http://lists.debian.org/1316898460.5414.2.camel@debian
 
Old 09-24-2011, 09:15 PM
Stan Hoeppner
 
Default Reducing kernel compilation time

On 9/24/2011 9:22 AM, Camaleón wrote:

Hello,

I had to compile the latest upstream kernel sources to make some
debugging with my wifi drivers (from staging) and discovered that
compilation took ~5 hours.

That's much for testing purposes.

Compilation takes place in a netbook governed by Intel's Atom N455 with 2
GiB of RAM and I would like to reduce the compilation time.

I'm using the same ".config" file I have for the current Debian stock
kernel (to avoid missing some modules I may need) and just added
"CONFIG_MATOM=y" but it takes almost the same time.

I don't need nothing special, just to be able to boot the system, test
the staging drivers and then remove/compile a new kernel again so wasting
the less time in the process would be great :-)

Any trick?


5 hours? Did you say 5 hours?

I have a 10 year old dual Mendocino 550 machine with only 384MB of PC100
that takes about 30 minutes to compile my custom kernels using make -j2.


I'd guess you're including the kitchen sink. Don't build the hundreds
of driver modules your machines won't ever use. That is the key to
reducing build time.


--
Stan


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4E7E4887.3090901@hardwarefreak.com">http://lists.debian.org/4E7E4887.3090901@hardwarefreak.com
 
Old 09-24-2011, 09:30 PM
Stan Hoeppner
 
Default Reducing kernel compilation time

On 9/24/2011 11:01 AM, Stephen Powell wrote:

On Sat, 24 Sep 2011 10:22:33 -0400 (EDT), Camaleón wrote:


Intel's Atom N455



I'm not familiar with the capabilities of your hardware, but if you have
multiple CPUs ("cores")


The Atom 455 is a single core 64/32 bit CPU with HyperThreading,
1.66GHz, 512KB L2 cache, single channel DDR2/3 interface.


http://ark.intel.com/products/49491

I would guess that 64 bit gcc would probably run a bit faster than 32
bit gcc on this chip, with twice as many GPRs available. Give that a
shot if you're not doing so already.


--
Stan


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4E7E4BFD.6020906@hardwarefreak.com">http://lists.debian.org/4E7E4BFD.6020906@hardwarefreak.com
 
Old 09-24-2011, 11:55 PM
Stephen Powell
 
Default Reducing kernel compilation time

On Sat, 24 Sep 2011 17:15:51 -0400 (EDT), Stan Hoeppner wrote:
>
> 5 hours? Did you say 5 hours?
>
> I have a 10 year old dual Mendocino 550 machine with only 384MB of PC100
> that takes about 30 minutes to compile my custom kernels using make -j2.
>
> I'd guess you're including the kitchen sink. Don't build the hundreds
> of driver modules your machines won't ever use. That is the key to
> reducing build time.

+1

Absolutely, Stan. Building a "lean and mean" kernel, one which only
contains what the machine needs, is where the big savings are. But
configuring the kernel to do that takes time. And it is easy to make
a mistake. For example, I've learned from experience that I need
SCSI support in my kernel, even though I have no SCSI adapter in my
machine. So much stuff emulates SCSI now, or uses SCSI protocols
in communication.

--
.'`. Stephen Powell
: :' :
`. `'`
`-


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 868903406.2094701.1316908513981.JavaMail.root@md01 .wow.synacor.com">http://lists.debian.org/868903406.2094701.1316908513981.JavaMail.root@md01 .wow.synacor.com
 
Old 09-25-2011, 07:42 AM
Elimar Riesebieter
 
Default Reducing kernel compilation time

* Stephen Powell [110924 14:13 -0400]:

> On Sat, 24 Sep 2011 12:18:47 -0400 (EDT), Camaleón wrote:
> >
> > What do you mean by "kernel-package"? Debian's vanilla kernel?
>
> "kernel-package" is the name of a Debian package, as in
>
> aptitude install kernel-package

Isn't [1] the proper way to build Debian kernels?

[1] http://kernel-handbook.alioth.debian.org/ch-common-tasks.html

Elimar
--
Alles was viel bedacht wird ist bedenklich!;-)
Friedrich Nietzsche


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 20110925074224.GA2162@samweis.home.lxtec.de">http://lists.debian.org/20110925074224.GA2162@samweis.home.lxtec.de
 
Old 09-25-2011, 07:53 AM
Stan Hoeppner
 
Default Reducing kernel compilation time

On 9/24/2011 6:55 PM, Stephen Powell wrote:

On Sat, 24 Sep 2011 17:15:51 -0400 (EDT), Stan Hoeppner wrote:


5 hours? Did you say 5 hours?

I have a 10 year old dual Mendocino 550 machine with only 384MB of PC100
that takes about 30 minutes to compile my custom kernels using make -j2.

I'd guess you're including the kitchen sink. Don't build the hundreds
of driver modules your machines won't ever use. That is the key to
reducing build time.


+1

Absolutely, Stan. Building a "lean and mean" kernel, one which only
contains what the machine needs, is where the big savings are. But
configuring the kernel to do that takes time. And it is easy to make
a mistake. For example, I've learned from experience that I need
SCSI support in my kernel, even though I have no SCSI adapter in my
machine. So much stuff emulates SCSI now, or uses SCSI protocols
in communication.


I'd bet that omitting the SCSI layer hamstrung many folks who tried to
use libata for the first time. It got me too. It forced me to go
and read up on how libata works, so it wasn't all bad.


--
Stan


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4E7EDE11.5060908@hardwarefreak.com">http://lists.debian.org/4E7EDE11.5060908@hardwarefreak.com
 
Old 09-25-2011, 08:09 AM
Stan Hoeppner
 
Default Reducing kernel compilation time

On 9/25/2011 2:42 AM, Elimar Riesebieter wrote:

* Stephen Powell [110924 14:13 -0400]:


On Sat, 24 Sep 2011 12:18:47 -0400 (EDT), Camaleón wrote:


What do you mean by "kernel-package"? Debian's vanilla kernel?


"kernel-package" is the name of a Debian package, as in

aptitude install kernel-package


Isn't [1] the proper way to build Debian kernels?

[1] http://kernel-handbook.alioth.debian.org/ch-common-tasks.html


As has already been discussed, this is the "new" recommended way to
build a Debian kernel. The instructions on that page 'silently' changed
not terribly that long ago. The build instructions on that page used to
tell us to use 'make-kpkg'. Note that this new method


make KDEB_PKGVERSION=custom.1.0 deb-pkg

did not exist in the days of 2.6.18 to 2.6.21, which is when these
command line examples were written. IIRC this deb-pkg method was
introduced in kernel 2.6.32 or 2.6.34. It is included in the pristine
kernel kernel source tarball. Thus you don't have to install any Debian
kernel building packages on your system in order to build a custom
kernel. This is likely one of the reasons this is the recommended
method now, since most everything you need is in the vanilla tarball.


--
Stan


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org

Archive: 4E7EE1CA.10603@hardwarefreak.com">http://lists.debian.org/4E7EE1CA.10603@hardwarefreak.com
 
Old 09-25-2011, 11:35 AM
Tom H
 
Default Reducing kernel compilation time

On Sat, Sep 24, 2011 at 2:20 PM, Stephen Powell <zlinuxman@wowway.com> wrote:
> On Sat, 24 Sep 2011 13:56:35 -0400 (EDT), Tom H wrote:
>>
>> CONCURRENCY_LEVEL=$(getconf _NPROCESSORS_ONLN)
>> although "getconf _NPROCESSORS_ONLN" on an atom's probably "1"; but
>> you never know...
>>
>> You can also pass "INSTALL_MOD_STRIP=1" to make-kpkg so that the "make
>> modules_install" step strips out debugging information (if this isn't
>> done by default).
>
> Thanks for the tips, Tom! *I believe I'll incorporate one or both of
> those tips during the next revision of my kernel-building web page.

You're very welcome. Should kernel-package ever be EOLd you could look
into the "fakeroot debian/rules..." way of compiling a kernel that
both the Debian and Ubuntu kernel teams seem to be promoting if you
use their sources. From what little I've read about this method, it
doesn't seem to have the equivalent of make-kpg's "kernel_image"
target so it might have the same deficiency as "make deb-pkg" from
your perspective...

I was re-reading the make-kpkg to see whether there were targets with
or without debugging symbols and found "kernel_debug":

kernel_debug
This target produces a Debian package containing the debugging
symbols for the modules contained in the corresponding image
package. The basic idea here is to keep the space in /lib/mod-
ules/<kver> under control, since this could be on a root parti-
tion with space restrictions.

So, if you create a linux-image package with debug info stripped out,
you can later create a package with just the unstripped modules rather
than recompile a complete package. The man page doesn't specify where
these modules would be installed.


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: CAOdo=SwyO0XV8qoSs_RZaSYtauHO064v0Mf93p8m+p_AXum2R Q@mail.gmail.com">http://lists.debian.org/CAOdo=SwyO0XV8qoSs_RZaSYtauHO064v0Mf93p8m+p_AXum2R Q@mail.gmail.com
 
Old 09-25-2011, 11:53 AM
Camaleón
 
Default Reducing kernel compilation time

On Sat, 24 Sep 2011 14:13:57 -0400, Stephen Powell wrote:

> On Sat, 24 Sep 2011 12:18:47 -0400 (EDT), Camaleón wrote:
>>
>> What do you mean by "kernel-package"? Debian's vanilla kernel?
>
> "kernel-package" is the name of a Debian package, as in
>
> aptitude install kernel-package
>
> It is not a kernel. It is a collection of scripts, configuration files,
> etc. that are intended to aid in the process of compiling a kernel and
> building a Debian binary kernel package. The make-kpkg command is one
> of the scripts in the kernel-package package.

Ah, then yes :-)

In fact, I already have a set of tools installed (along with "fakeroot",
that "kernel-package") because I'm following Debian Installation Guide
instructions:

http://d-i.alioth.debian.org/manual/en.i386/ch08s06.html

(there is a little bug there. "--revision=" string has to start with a
number or it won't compile...)

>> No, sources have to be from upstream.
>
> kernel-package can be used with official Debian kernel source packages
> or with pristine kernel sources acquired directly from upstream. When
> using upstream kernel sources directly, one cannot use the Debian
> package version in the "--revision" option, since there is no Debian
> package version. You have to make one up. Other than that, it works
> just fine for upstream kernel sources. I've done it before, and for
> similar reasons as you.

Yes, yes... that's what I have been doing. Then I can try your
suggestion, in addition with the new "make localmodconfig" that is
running right now.

>> Yes... I've already¹ "walked" through that link, and it's fantastic for
>> Debian starters, very well explained with detailed steps (have you
>> considered adding it into Debian's wiki, or at least -if not the full
>> article- a link to it?)
>>
>> ¹I noticed you pointed to it in another thread where someone asked for
>> something similar ;-)
>
> The Debian kernel team seems to think that kernel-package should be
> considered deprecated. (Although, as far as I know, the author of
> kernel-package does not share that opinion.) Therefore, I'm not sure
> that my kernel-building stuff would be welcome in the official wiki. The
> kernel team encourages the use of "make deb-pkg". But I personally
> don't like "make deb-pkg" because of its "one size fits all" build
> philosophy. For example, it always produces a headers package, and I
> often don't need a headers package. With kernel-package, I only get the
> packages that I ask for. And since your goal is to reduce kernel
> compilation time, I would think that you would not want extra packages
> produced that you don't need. That takes additional time.

I also found very easy to compile a new kernel from whatever source by
using the mentioned package, and in fact, it is suggested in the install
manual so they better remove the current instructions if want people use
another method but until then, I will keep using it :-P

> If you decide to try kernel-package, make sure that you apply the patch
> file listed in the web page. It won't work properly with a version 3
> kernel unless you do.

Hum... you mean the one for EDID? I've compiled 3 kernels just fine and
had to do nothing special :-?

Greetings,

--
Camaleón


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: pan.2011.09.25.11.53.59@gmail.com">http://lists.debian.org/pan.2011.09.25.11.53.59@gmail.com
 
Old 09-25-2011, 12:21 PM
Stephen Powell
 
Default Reducing kernel compilation time

On Sun, 25 Sep 2011 03:42:24 -0400 (EDT), Elimar Riesebieter wrote:
>
> Isn't [1] the proper way to build Debian kernels?
>
> [1] http://kernel-handbook.alioth.debian.org/ch-common-tasks.html

That depends on your definition of "proper". The kernel handbook,
not surprisingly, is maintained by the Debian kernel team. The
Debian kernel team is responsible for producing the official Debian
stock binary kernels as well as making any Debian customization to
the pristine kernel sources that they deem necessary or desirable.
If your definition of "proper" means the way the kernel team
currently recommends, then yes. However, with apologies to cat
lovers, there's an old American saying: "There's more than one way
to skin a cat." The method documented on my web page is the
historic method, and was also the method used by the Debian kernel
team to create official Debian stock kernel images up through and
including Sarge. The kernel team departed from the historic method
of producing stock kernels with Etch, it seems.

As I said in an earlier post, I still prefer the historic method because
I find it more flexible. For example, when using "make deb-pkg",
three binary packages are produced: a linux-image-* package, a
linux-headers-* package, and a linux-libc6-dev package. In most
cases, I only need and want the linux-image-* package. With "make
deb-pkg", it doesn't appear that there is any way to suppress the
creation of the unneeded packages. With make-kpkg, I only get the
packages that I ask for.

Another benefit of the historic method is that header packages are
rarely needed. If the entire kernel source tree is already installed,
header packages should not be needed, since the kernel headers are
included as part of the kernel source. The historic method of creating
kernel module binary packages takes advantage of this fact and therefore
does not require a separate headers package. The new way of creating
kernel modules requires a headers package, even if the entire kernel
source tree is already installed. This is all explained in my web page,
which, for reference, is

http://users.wowway.com/~zlinuxman/Kernel.htm

But, to each his own. Whatever floats your boat, man.

--
.'`. Stephen Powell
: :' :
`. `'`
`-


--
To UNSUBSCRIBE, email to debian-user-REQUEST@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmaster@lists.debian.org
Archive: 980275088.2101929.1316953277845.JavaMail.root@md01 .wow.synacor.com">http://lists.debian.org/980275088.2101929.1316953277845.JavaMail.root@md01 .wow.synacor.com
 

Thread Tools




All times are GMT. The time now is 09:46 PM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org