On Thu, Jul 26, 2012 at 12:01:22AM +0200, Tom Gundersen wrote:
> On Wed, Jul 25, 2012 at 11:43 PM, Dave Reisner <d@falconindy.com> wrote:
> > On Wed, Jul 25, 2012 at 11:21:10PM +0200, Tom Gundersen wrote:
> >> Dave, Tobias, Thomas,
> >>
> >> I just managed to hose one of my machines. It was mainly due to an
> >> unfortunate series of events, but I have a suggestion which would have
> >> avoided this situation and might beneficial in general.
> >
> > Oops. ;P
> >
> > What did you break?
>
> Had a hard crash during a pacman. Trying to fix things from a rescue
> shell I installed the kernel and as I didn't have udev running,
> mkinitcpio claimed /dev was not mounted (at least I think that's why
> that happened) and refused to run. I then started udevd manually, and
> the machine hung again.
>
> Result: on next restart I had a 3.5 kernel, but the modules in both my
> initramfs' were 3.2 (or something like that). So without an ext4 I was
> sad.
>

Fun!

> > You've mentioned this idea before, and more recently, so has Jan. I've
> > slowly and quietly been adding support to mkinitcpio to make this less
> > painful to do. An xz-compressed image with the usb and fw hooks still
> > weighs in at under 10MiB, so I have no real concerns about this being
> > problematic for users with separate /boot partitions.
>
> Nice!
>
> > That said, I'm wondering how it would be possible to generate this at
> > compile time. I'm not really sure how this would be possible. Is there
> > any objection to just adding another preset (maybe related to how you
> > fubar'd your install this time?).
>
> The underlying problem is that in principle you might (as I did)
> upgrade your kernel without running mkinitcpio. We have seen pacman
> issues causing this, simple crashes as what I had might cause it, or a
> broken mkinitcpio.conf might do it (I guess the last one could be
> worked around by adding a special preset that does not care about user
> config).
>

Well that answers the why, but that was the easy question =P. I suppose
I could re-add a flag that would point mkinitcpio at a different module
root (making no attempt to fully reimplement --basedir), which could be
simply passed onto modinfo/modprobe ...but that would require patching
modinfo first.

> > As an alternative/addition, which has also been brought up before, why
> > don't we build in the most basic of modules? I'll bet we can cover at
> > least 50% of the use cases by picking some choice pata/sata modules
> > (e.g. ahci, ata_piix, pata_jmicron, sd_mod, ext4) and compiling them in
> > staticly. It, of course, doesn't cover folks with non-trivial setups,
> > but it provides a bulletproof bootstrap for a lot of people.
>
> I really think this would be a good idea. I wanted to make some
> additions to pierres pkgstats stuff so we could have an idea of how
> large percentage of our users would be covered by the modules you
> propose. I expect the vast majority would.

Sure. I'd love to see the running kernel version and the first column of
/proc/modules submitted with pkgstats. If we were to reset the global
stats (or just reset the epoch) and make a concerted effort to have
people submit (news post, social media, allan's blog, etc) I'll bet we
could gather some good usage stats from -ARCH kernel consumers in a
fairly short timeframe.

> Cheers,
>
> Tom

On Thu, Jul 26, 2012 at 12:17 AM, Dave Reisner <d@falconindy.com> wrote:
> Well that answers the why, but that was the easy question =P.

Ah, my brain skipped over the hard part of the question ;-)

> I suppose
> I could re-add a flag that would point mkinitcpio at a different module
> root (making no attempt to fully reimplement --basedir), which could be
> simply passed onto modinfo/modprobe ...but that would require patching
> modinfo first.

Hm, I didn't realise that this was not currently possible. Good point.
If worst comes to worse, I guess we could try using some chroot magic?

-t

On Thu, Jul 26, 2012 at 12:17 AM, Dave Reisner <d@falconindy.com> wrote:
>> > As an alternative/addition, which has also been brought up before, why
>> > don't we build in the most basic of modules? I'll bet we can cover at
>> > least 50% of the use cases by picking some choice pata/sata modules
>> > (e.g. ahci, ata_piix, pata_jmicron, sd_mod, ext4) and compiling them in
>> > staticly. It, of course, doesn't cover folks with non-trivial setups,
>> > but it provides a bulletproof bootstrap for a lot of people.
>>
>> I really think this would be a good idea. I wanted to make some
>> additions to pierres pkgstats stuff so we could have an idea of how
>> large percentage of our users would be covered by the modules you
>> propose. I expect the vast majority would.
>
> Sure. I'd love to see the running kernel version and the first column of
> /proc/modules submitted with pkgstats. If we were to reset the global
> stats (or just reset the epoch) and make a concerted effort to have
> people submit (news post, social media, allan's blog, etc) I'll bet we
> could gather some good usage stats from -ARCH kernel consumers in a
> fairly short timeframe.

Pierre: would something like the attached patch make sense for pkgstats?

Cheers,

Tom

On Thu, Jul 26, 2012 at 12:45:39AM +0200, Tom Gundersen wrote:
> On Thu, Jul 26, 2012 at 12:17 AM, Dave Reisner <d@falconindy.com> wrote:
> > Well that answers the why, but that was the easy question =P.
>
> Ah, my brain skipped over the hard part of the question ;-)
>
> > I suppose
> > I could re-add a flag that would point mkinitcpio at a different module
> > root (making no attempt to fully reimplement --basedir), which could be
> > simply passed onto modinfo/modprobe ...but that would require patching
> > modinfo first.
>
> Hm, I didn't realise that this was not currently possible. Good point.
> If worst comes to worse, I guess we could try using some chroot magic?
>
> -t

Just to complete the illusion, there _is_ a --basedir flag for modinfo
(hell, it used to be part of mkinitcpio), but much like
module-init-tools, it isn't documented in the manpage. That's fixed
upstream, and I'll add a --moduleroot flag to mkinitcpio for next
release. That _should_ make image creation possible from makepkg. The
only caveat is that we'll need to ship our own one line mkinitcpio.conf
to specify HOOKS="", or hack it into the PKGBUILD and source that (would
be effectively benign as far as mkinitcpio is concerned).

d


Thu Jul 26 15:30:01 2012
Return-Path: <advisory-board-bounces@lists.fedoraproject.org>
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on
eagle542.startdedicated.com
X-Spam-Level:
X-Spam-Status: No, score=-2.2 required=5.0 tests=DKIM_ADSP_CUSTOM_MED,
DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_DNSWL_MED,SPF_PA SS,T_DKIM_INVALID,
T_RP_MATCHES_RCVD autolearn=ham version=3.3.2
X-Original-To: tom@linux-archive.org
Delivered-To: tom-linux-archive.org@eagle542.startdedicated.com
Received: from bastion.fedoraproject.org (bastion01.fedoraproject.org [209.132.181.2])
by eagle542.startdedicated.com (Postfix) with ESMTP id 6EA3520E0024
for <tom@linux-archive.org>; Thu, 26 Jul 2012 14:49:59 +0200 (CEST)
Received: from lists.fedoraproject.org (collab03.vpn.fedoraproject.org [192.168.1.70])
by bastion01.phx2.fedoraproject.org (Postfix) with ESMTP id D3ADE20E28;
Thu, 26 Jul 2012 12:49:57 +0000 (UTC)
Received: from collab03.fedoraproject.org (localhost [127.0.0.1])
by lists.fedoraproject.org (Postfix) with ESMTP id 7218940767;
Thu, 26 Jul 2012 12:49:57 +0000 (UTC)
X-Original-To: advisory-board@lists.fedoraproject.org
Delivered-To: advisory-board@lists.fedoraproject.org
Received: from smtp-mm03.fedoraproject.org (vm4.fedora.ibiblio.org
[152.19.134.143])
by lists.fedoraproject.org (Postfix) with ESMTP id 18A8E401DB
for <advisory-board@lists.fedoraproject.org>;
Thu, 26 Jul 2012 12:49:56 +0000 (UTC)
Received: from mail-ey0-f173.google.com (mail-ey0-f173.google.com
[209.85.215.173])
by smtp-mm03.fedoraproject.org (Postfix) with ESMTP id A3BC840116
for <advisory-board@lists.fedoraproject.org>;
Thu, 26 Jul 2012 12:49:55 +0000 (UTC)
Received: by eaah1 with SMTP id h1so228881eaa.32
for <advisory-board@lists.fedoraproject.org>;
Thu, 26 Jul 2012 05:49:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=message-id:date:from:user-agent:mime-version:to:subject:references
:in-reply-to:content-type:content-transfer-encoding;
bh=sdhXW5BiuAA24q9toZrPxJB/bQJxdPrRVxVrukl8wd4=;
b=nDml9DoJu7zLpwgfVxj8miZw/ysC7duR8ww7wraqcLdYtxCDUkbAcd8a0jkOZZ0zCd
FYyMGCLTCoOzASYV8Op3V0YPwuW+9GOOY/vU5iVbC0B2cmrHwDx+3Y9EDg/ByK/wrNlT
oe4y5dv65Y60WbKH9UaDz2MG6TiCLz3uVZMy3fnDMoSNoj09cE nHMXdj7Q+EYNErtF73
6vpYSyKyanrPKBcjYhM4coPyiymyeigIHr/qsTHYPdEHN3N8J7vorIQ0bk8RWjhfwydg
W1ED4XJjjbvC1APOT4Aw+7aAkuPWXaS0xL2DA1E0Dvf2+IaWtu aMqTlHBsXCOgx7CFWj
reEw==
Received: by 10.14.173.71 with SMTP id u47mr3946854eel.22.1343306995768;
Thu, 26 Jul 2012 05:49:55 -0700 (PDT)
Received: from localhost.localdomain ([217.28.190.130])
by mx.google.com with ESMTPS id c1sm8485118eeo.5.2012.07.26.05.49.54
(version=TLSv1/SSLv3 cipher=OTHER);
Thu, 26 Jul 2012 05:49:54 -0700 (PDT)
Message-ID: <50113CC6.6050707@gmail.com>
Date: Thu, 26 Jul 2012 12:49:10 +0000
From: =?UTF-8?B?IkrDs2hhbm4gQi4gR3XDsG11bmRzc29uIg==?=
<johannbg@gmail.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
rv:14.0) Gecko/20120717 Thunderbird/14.0
MIME-Version: 1.0
To: advisory-board@lists.fedoraproject.org
Subject: Re: Fedora Board Recap - 2012-07-25
References: <CAKprHVZrkKKBrnL_gfPjUogXdb_3ew3A89R6A9U90XNWrriL UQ@mail.gmail.com>
<50105C33.8000505@redhat.com> <20120725213015.GI11652@unaka.lan>
<CALY6xng+bfGtrMMnam2yk0q65sOt4eb6gTv9Jz5ZcTXL1nts Pg@mail.gmail.com>
<20120726013147.GJ11652@unaka.lan>
<5010A8DA.4010607@fedoraproject.org>
<20120726023116.GK11652@unaka.lan> <5011006B.9090104@gmail.com>
<5011142E.7000503@gmail.com> <50112C35.70107@gmail.com>
<50113857.2030907@redhat.com>
In-Reply-To: <50113857.2030907@redhat.com>
X-BeenThere: advisory-board@lists.fedoraproject.org
X-Mailman-Version: 2.1.12
Precedence: list
Reply-To: Fedora community advisory board
<advisory-board@lists.fedoraproject.org>
List-Id: Fedora community advisory board
<advisory-board.lists.fedoraproject.org>
List-Unsubscribe: <https://admin.fedoraproject.org/mailman/options/advisory-board>,
<mailto:advisory-board-request@lists.fedoraproject.org?subject=unsubscrib e>
List-Archive: <http://lists.fedoraproject.org/pipermail/advisory-board/>
List-Post: <mailto:advisory-board@lists.fedoraproject.org>
List-Help: <mailto:advisory-board-request@lists.fedoraproject.org?subject=help>
List-Subscribe: <https://admin.fedoraproject.org/mailman/listinfo/advisory-board>,
<mailto:advisory-board-request@lists.fedoraproject.org?subject=subscribe>
Content-Transfer-Encoding: base64
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Sender: advisory-board-bounces@lists.fedoraproject.org
Errors-To: advisory-board-bounces@lists.fedoraproject.org

T24gMDcvMjYvMjAxMiAxMjozMCBQTSwgUm9ieW4gQmVyZ2Vyb2 4gd3JvdGU6Cj4gKiBUaGUgdGlj
a2V0cyBmaWxlZCB3aXRoIHRoZSBCb2FyZCBhcmUsIGF0IGxlYX N0IGEgcmVhc29uYWJsZSAKPiBw
ZXJjZW50YWdlIG9mIHRoZSB0aW1lLCBvZiBhIG5hdHVyZSB0aG F0IHRoZXkgc2hvdWxkIGJlIGtl
cHQgcHJpdmF0ZSAKPiAobGVnYWwgc3R1ZmYsIHBlcnNvbm5lbC 9zZW5zaXRpdmUvY29uZmxpY3Qs
IGV0Yy4pLiAKCkkgd291bGQgYXJndWUgaGVyZSB0aGF0IHdlIH Nob3VsZCBjcmVhdGUgYW4gc2Vw
YXJhdGUgdHJhYyBpbnN0YW5jZSB0byAKaGFuZGxlIGFsbCB0aG UgbGVnYWwgaXNzdWVzIHdoaWNo
IHdvdWxkIGJlIGhhbmRsZWQgYnkgdGhlIGxlZ2FsIHRlYW0gZm 9yIAp0aGUgbGVnYWwgc3R1ZmYg
YW5kIGFub3RoZXIgb25lIGZvciBjb21tdW5pdHkgY29uZmxpY3 RzIGJlIGl0IAppbmRpdmlkdWFs
cyBvciBncm91cHMgaW4gb3VyIGNvbW11bml0eSB3aGljaCB3b3 VsZCBiZSBoYW5kbGVkIGJ5IHRo
ZSAKY29tbXVuaXR5IHdvcmtpbmcgZ3JvdXAgYW5kIHRoZSByZW xldmFudCB0aWNrZXRzIGJlIG1p
Z3JhdGVkIHRvIHRob3NlIAppbnN0YW5jZXMgKCBpZiBkb2FibG UgKS4KCkNvdWxkIHlvdSBwcm92
aWRlIGV4YW1wbGUgb24gd2hhdCB5b3UgbWVhbiBieSAicGVyc2 9ubmVsIGFuZCAic2Vuc2l0aXZl
IiAKaXNzdWVzIHRoYXQgdGhlIGJvYXJkIG1pZ2h0IGJlIGRlYW xpbmcgd2l0aCB3aGljaCBtaWdo
dCBub3QgYmVsb25nIGluIAplaXRoZXIgb2YgdGhvc2UgdHdvIH ByZXZpb3VzbHkgbWVudGlvbmVk
IGluc3RhbmNlcz8KCkpCRwpfX19fX19fX19fX19fX19fX19fX1 9fX19fX19fX19fX19fX19fX19f
X19fX19fXwphZHZpc29yeS1ib2FyZCBtYWlsaW5nIGxpc3QKYW R2aXNvcnktYm9hcmRAbGlzdHMu
ZmVkb3JhcHJvamVjdC5vcmcKaHR0cHM6Ly9hZG1pbi5mZWRvcm Fwcm9qZWN0Lm9yZy9tYWlsbWFu
L2xpc3RpbmZvL2Fkdmlzb3J5LWJvYXJk

Am 26.07.2012 00:17, schrieb Dave Reisner:
> On Thu, Jul 26, 2012 at 12:01:22AM +0200, Tom Gundersen wrote:
>> On Wed, Jul 25, 2012 at 11:43 PM, Dave Reisner <d@falconindy.com> wrote:
>> > As an alternative/addition, which has also been brought up before, why
>> > don't we build in the most basic of modules? I'll bet we can cover at
>> > least 50% of the use cases by picking some choice pata/sata modules
>> > (e.g. ahci, ata_piix, pata_jmicron, sd_mod, ext4) and compiling them in
>> > staticly. It, of course, doesn't cover folks with non-trivial setups,
>> > but it provides a bulletproof bootstrap for a lot of people.
>>
>> I really think this would be a good idea. I wanted to make some
>> additions to pierres pkgstats stuff so we could have an idea of how
>> large percentage of our users would be covered by the modules you
>> propose. I expect the vast majority would.
>
> Sure. I'd love to see the running kernel version and the first column of
> /proc/modules submitted with pkgstats. If we were to reset the global
> stats (or just reset the epoch) and make a concerted effort to have
> people submit (news post, social media, allan's blog, etc) I'll bet we
> could gather some good usage stats from -ARCH kernel consumers in a
> fairly short timeframe.

I have added pkgstats 2.3 to testing. It now sends the list of loaded
kernel modules and also the cpu architecture.

--
Pierre Schmitz, https://pierre-schmitz.com

On Wed, Jul 25, 2012 at 11:21 PM, Tom Gundersen <teg@jklm.no> wrote:
> It seems to me that a more useful fallback image, would be one that is
> generated at compile-time, rather than at install-time, and shipped as
> a part of the kernel package. This would avoid user errors, and
> mkinitcpio run-time problems.
>
> This fallback image should contain the widest sets of hooks and
> modules so that it should work on any hardware and any setup (at least
> to the extent possible with our current hooks).

I put a PKGBUILD up on AUR[0]. Rather than making it part of the
kernel package, I made it separate (at Thomas' request). At least this
should make it easy to test without having to rebuild kernels.

At the moment it simply includes all the hooks I could think of. If,
in the future, we would like to add more features to it (e.g. pacman,
arch-install-scripts or better networking tools), we should add
mkinitcpio hooks to the relevant pacakges.

Any thoughts?

Cheers,

Tom

[0]: <https://aur.archlinux.org/packages.php?ID=61313>

Am 30.07.2012 00:18, schrieb Tom Gundersen:
> On Wed, Jul 25, 2012 at 11:21 PM, Tom Gundersen <teg@jklm.no> wrote:
>> It seems to me that a more useful fallback image, would be one that is
>> generated at compile-time, rather than at install-time, and shipped as
>> a part of the kernel package. This would avoid user errors, and
>> mkinitcpio run-time problems.
>>
>> This fallback image should contain the widest sets of hooks and
>> modules so that it should work on any hardware and any setup (at least
>> to the extent possible with our current hooks).
>
> I put a PKGBUILD up on AUR[0]. Rather than making it part of the
> kernel package, I made it separate (at Thomas' request). At least this
> should make it easy to test without having to rebuild kernels.
>
> At the moment it simply includes all the hooks I could think of. If,
> in the future, we would like to add more features to it (e.g. pacman,
> arch-install-scripts or better networking tools), we should add
> mkinitcpio hooks to the relevant pacakges.
>
> Any thoughts?
>
> Cheers,
>
> Tom
>
> [0]: <https://aur.archlinux.org/packages.php?ID=61313>

It'll be better to make it a split of the linux package to make sure
modules are compatible.

--
Pierre Schmitz, https://pierre-schmitz.com

Am 30.07.2012 11:55, schrieb Pierre Schmitz:
> Am 30.07.2012 00:18, schrieb Tom Gundersen:
>> On Wed, Jul 25, 2012 at 11:21 PM, Tom Gundersen <teg@jklm.no> wrote:
>>> It seems to me that a more useful fallback image, would be one that is
>>> generated at compile-time, rather than at install-time, and shipped as
>>> a part of the kernel package. This would avoid user errors, and
>>> mkinitcpio run-time problems.
>>>
>>> This fallback image should contain the widest sets of hooks and
>>> modules so that it should work on any hardware and any setup (at least
>>> to the extent possible with our current hooks).
>> I put a PKGBUILD up on AUR[0]. Rather than making it part of the
>> kernel package, I made it separate (at Thomas' request). At least this
>> should make it easy to test without having to rebuild kernels.
>>
>> At the moment it simply includes all the hooks I could think of. If,
>> in the future, we would like to add more features to it (e.g. pacman,
>> arch-install-scripts or better networking tools), we should add
>> mkinitcpio hooks to the relevant pacakges.
>>
>> Any thoughts?
>>
>> Cheers,
>>
>> Tom
>>
>> [0]: <https://aur.archlinux.org/packages.php?ID=61313>
> It'll be better to make it a split of the linux package to make sure
> modules are compatible.
>
As far as I understand covers it everything so no need to be compatible
with the installed kernel.
greetingst
tpowa

--
Tobias Powalowski
Archlinux Developer & Package Maintainer (tpowa)
http://www.archlinux.org
tpowa@archlinux.org

Am 30.07.2012 12:03, schrieb Tobias Powalowski:
> Am 30.07.2012 11:55, schrieb Pierre Schmitz:
>> Am 30.07.2012 00:18, schrieb Tom Gundersen:
>>> On Wed, Jul 25, 2012 at 11:21 PM, Tom Gundersen <teg@jklm.no> wrote:
>>>> It seems to me that a more useful fallback image, would be one that is
>>>> generated at compile-time, rather than at install-time, and shipped as
>>>> a part of the kernel package. This would avoid user errors, and
>>>> mkinitcpio run-time problems.
>>>>
>>>> This fallback image should contain the widest sets of hooks and
>>>> modules so that it should work on any hardware and any setup (at least
>>>> to the extent possible with our current hooks).
>>> I put a PKGBUILD up on AUR[0]. Rather than making it part of the
>>> kernel package, I made it separate (at Thomas' request). At least this
>>> should make it easy to test without having to rebuild kernels.
>>>
>>> At the moment it simply includes all the hooks I could think of. If,
>>> in the future, we would like to add more features to it (e.g. pacman,
>>> arch-install-scripts or better networking tools), we should add
>>> mkinitcpio hooks to the relevant pacakges.
>>>
>>> Any thoughts?
>>>
>>> Cheers,
>>>
>>> Tom
>>>
>>> [0]: <https://aur.archlinux.org/packages.php?ID=61313>
>> It'll be better to make it a split of the linux package to make sure
>> modules are compatible.
>>
> As far as I understand covers it everything so no need to be compatible
> with the installed kernel.
> greetingst
> tpowa

Except it does not include the kernel itself. Modules without a
matching kernel are not that useful.

--
Pierre Schmitz, https://pierre-schmitz.com

Am 30.07.2012 12:09, schrieb Pierre Schmitz:
> Am 30.07.2012 12:03, schrieb Tobias Powalowski:
>> Am 30.07.2012 11:55, schrieb Pierre Schmitz:
>>> Am 30.07.2012 00:18, schrieb Tom Gundersen:
>>>> On Wed, Jul 25, 2012 at 11:21 PM, Tom Gundersen <teg@jklm.no> wrote:
>>>>> It seems to me that a more useful fallback image, would be one that is
>>>>> generated at compile-time, rather than at install-time, and shipped as
>>>>> a part of the kernel package. This would avoid user errors, and
>>>>> mkinitcpio run-time problems.
>>>>>
>>>>> This fallback image should contain the widest sets of hooks and
>>>>> modules so that it should work on any hardware and any setup (at least
>>>>> to the extent possible with our current hooks).
>>>> I put a PKGBUILD up on AUR[0]. Rather than making it part of the
>>>> kernel package, I made it separate (at Thomas' request). At least this
>>>> should make it easy to test without having to rebuild kernels.
>>>>
>>>> At the moment it simply includes all the hooks I could think of. If,
>>>> in the future, we would like to add more features to it (e.g. pacman,
>>>> arch-install-scripts or better networking tools), we should add
>>>> mkinitcpio hooks to the relevant pacakges.
>>>>
>>>> Any thoughts?
>>>>
>>>> Cheers,
>>>>
>>>> Tom
>>>>
>>>> [0]: <https://aur.archlinux.org/packages.php?ID=61313>
>>> It'll be better to make it a split of the linux package to make sure
>>> modules are compatible.
>>>
>> As far as I understand covers it everything so no need to be compatible
>> with the installed kernel.
>> greetingst
>> tpowa
> Except it does not include the kernel itself. Modules without a
> matching kernel are not that useful.
>
Well something similar is provided by me since ages here:
https://downloads.archlinux.de/iso/archboot/2012.06/boot
kernel+initramfs
greetings
tpowa

--
Tobias Powalowski
Archlinux Developer & Package Maintainer (tpowa)
http://www.archlinux.org
tpowa@archlinux.org