On Mon, Nov 28, 2011 at 2:16 PM, Nikos Chantziaras <firstname.lastname@example.org> wrote:
> On 11/28/2011 06:59 PM, Florian Philipp wrote:
>> Am 28.11.2011 17:15, schrieb Nikos Chantziaras:
>>> On 11/28/2011 02:29 PM, Albert W. Hopkins wrote:
>>>> On Sun, 2011-11-27 at 20:28 +0100, Andrea Conti wrote:
>>>>> With 100% repeatability, mind you, which does raise same questions on
>>>>> the amount of testing done before release. Yes, it's ~arch and
>>>>> rc_parallel is explicitly marked "experimental", but it's not expected
>>>>> to be completely and consistently broken, either.
>>>>> If that sounds like I'm ranting, it's because I just spent about an
>>>>> getting three machines affected by this problem back into working
>>>>> If anyone still has it installed, it's time to sync and downgrade
>>>> Sorry to add more to the whining but...
>>>> Yes, you are in the testing tree. *Yes, as a member of testing, *you*
>>>> expect things will occasionally break, and it is *your* job to test
>>>> things, break them, and report bugs.
>>> Generally true, but not when something is obviously broken. *That means
>>> not even its upstream dev bothered to test it.
>>> ~arch is for "we think this works, but please give it a go in case there
>>> are problems". *It's *not* for "we have no idea if this works because we
>>> didn't even try it once".
>> Do you have any idea how much time you can spend with the kind of system
>> testing you propose?
> About 2 minutes? *Enabling the parallel startup thingy and rebooting the
> machine. *There you go :-/
That's a facetious answer, and you're purposely only examining a tiny
piece of the testing surface. Hindsight is 20/20, though only if
Perhaps they've never seen this type of failure before, and they could
add a single test to whatever unit test suite they may be using.
Perhaps that's an improvement they can make going forward.
To fully test OpenRC, you'd want a two-stage testing harness. The
outer stage would generate Gentoo VMs with every plausibly-relevant
USE flag permutation crossed against as many automatically-generated
permutations of OpenRC configuration as could be considered plausibly
For each generated VM, spin it up. Watch for some kind of watchdog
"hey, I booted successfully!" indicator. Then spin up a testing
harness *inside* the VM to ensure all services started and behave
correctly. Dump a report to the vmhost detailing that everything went
well (or didn't), and hibernate the VM. vmhost looks at the report and
decides whether or not to keep the saved VM state.
That's an extraordinary amount of testing to do. And that's what I see
argued as what ~arch is for; instead of having a script whip up and
test hundreds of virtual machines, people running ~arch do that
testing. Gentoo devs get reports for the features and combinations
that people actually *use*, and can spend less time fixing features
nobody is using. (And it's obvious none of the OpenRC devs are using
parallel boot themselves, or they would have caught this. Perhaps
that's why it's experimental; nobody who actively uses that feature is
keeping up with HEAD and offering patches.)