FAQ Search Today's Posts Mark Forums Read
» Video Reviews

» Linux Archive

Linux-archive is a website aiming to archive linux email lists and to make them easily accessible for linux users/developers.


» Sponsor

» Partners

» Sponsor

Go Back   Linux Archive > ArchLinux > ArchLinux Development

 
 
LinkBack Thread Tools
 
Old 10-30-2009, 11:40 PM
Dan McGee
 
Default Xen domU kernel - what to do?

On Fri, Oct 30, 2009 at 7:22 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:
> On Fri, Oct 30, 2009 at 6:51 PM, Dan McGee <dpmcgee@gmail.com> wrote:
>> On Thu, Oct 29, 2009 at 7:59 PM, Thomas Bächler <thomas@archlinux.org> wrote:
>>> Okay, it is obvious that we need a kernel which can run on Xen domU for our
>>> servers. It would also be very convenient for those running Arch on hosting
>>> services that have pv-grub available.
>>>
>>> Until now, I compiled a modified version of the kernel26 package with
>>> pv_ops/Xern enabled which - in theory - should also run on a real machine.
>>>
>>> If all the external modules compile properly (which they probably do), we
>>> could add these options to our normal stock kernel. I haven't tried running
>>> it on a real machine though and tpowa said in the past the nvidia module
>>> failed. This has to be tested.
>>>
>>> However, compiling Xen on a 32 bit kernel requires enabling PAE which we
>>> refused to do in the past for performance reasons (no idea if these are
>>> valid concerns) and due to the fact that we want to encourage people to run
>>> 64 bit if they have large amounts of memory.
>>>
>>> Assuming all of untested things above work, what would you prefer:
>>>
>>> 1) Switch to PAE on the 32 bit kernel, ship Xen domU support with kernel26
>>> on both architectures.
>>>
>>> 2) Ship Xen domU support on the 64 bit kernel, and don't on the 32 bit
>>> kernel (my favorite so far).
>>>
>>> 3) Leave kernel26 unchanged and create a separate kernel26-xen package with
>>> domU support.
>>>
>>> The advantage of 3) is that the Xen kernel could be very slim, as almost all
>>> hardware drivers could be disabled. The advantage of 1) and 2) is the lower
>>> maintenance cost.
>>>
>>>
>>> On a different note, I'd like to try to switch over to pv-grub on our
>>> servers. Right now, upgrading the kernel requires copying the kernel to the
>>> dom0 manually, thus dom0 access is required to do a (potentially security
>>> relevant) kernel update. With pv-grub, upgrading the kernel and rebooting
>>> would be handled just like with a normal machine, as the dom0 only loads the
>>> pv-grub image which then runs inside the domU and can load the right kernel
>>> from there.
>>> The pv-grub is only available from Xen 3.3 and later, but we have Xen 3.2 -
>>> I compiled pv-grub from 3.4 though and guess it should work with the older
>>> hypervisor as well, but I haven't tested that yet.
>>>
>>> Anyway, please post your opinion on the issues above.
>>
>> I'm -1 on the PAE, leaving options 2 and 3 as viable. 2 is the easiest
>> for maintenance; 3 sounds like the best solution long term as it would
>> be a very lightweight and well-suited kernel package.
>>
>> If not for the maintenance issues, 3 seems by far the best. Is there
>> anyway we can automate the creation of this package to the point where
>> it doesn't take much extra effort? e.g. try to share resources with
>> the primary kernel package.
>
> Can you illustrate why you're -1 on PAE? I can't find exact info on
> the performance hit caused by enabling that, and with machines having
> more and more RAM, I'm sure we'll eventually need to enable PAE. I
> _know_ we're trying to push people to 64bit, but that shouldn't mean
> we leave the i686 arch in the dust.
>
> I'm just sure we're EVENTUALLY going to need to enable this. Why not now?

Completely disagree, PAE is stupid. If you want to stick with 32 bit
and have more memory available, at least run a 64 bit kernel and stick
with a 32 bit userspace:
http://lkml.indiana.edu/hypermail/linux/kernel/0906.1/01128.html
http://lkml.indiana.edu/hypermail/linux/kernel/0906.1/01131.html

And those numbers for performance problems? 12%:
http://lkml.indiana.edu/hypermail/linux/kernel/0906.1/00959.html

Leaving i686 in the dust is hardly the concern here; why add
unnecessary overhead to those machines that sit well under the 4GB
mark that need that extra 12% of performance the most?

-Dan
 
Old 10-31-2009, 12:03 AM
Allan McRae
 
Default Xen domU kernel - what to do?

Dan McGee wrote:

On Fri, Oct 30, 2009 at 7:22 PM, Aaron Griffin <aaronmgriffin@gmail.com> wrote:


On Fri, Oct 30, 2009 at 6:51 PM, Dan McGee <dpmcgee@gmail.com> wrote:


On Thu, Oct 29, 2009 at 7:59 PM, Thomas Bächler <thomas@archlinux.org> wrote:


Okay, it is obvious that we need a kernel which can run on Xen domU for our
servers. It would also be very convenient for those running Arch on hosting
services that have pv-grub available.

Until now, I compiled a modified version of the kernel26 package with
pv_ops/Xern enabled which - in theory - should also run on a real machine.

If all the external modules compile properly (which they probably do), we
could add these options to our normal stock kernel. I haven't tried running
it on a real machine though and tpowa said in the past the nvidia module
failed. This has to be tested.

However, compiling Xen on a 32 bit kernel requires enabling PAE which we
refused to do in the past for performance reasons (no idea if these are
valid concerns) and due to the fact that we want to encourage people to run
64 bit if they have large amounts of memory.

Assuming all of untested things above work, what would you prefer:

1) Switch to PAE on the 32 bit kernel, ship Xen domU support with kernel26
on both architectures.

2) Ship Xen domU support on the 64 bit kernel, and don't on the 32 bit
kernel (my favorite so far).

3) Leave kernel26 unchanged and create a separate kernel26-xen package with
domU support.

The advantage of 3) is that the Xen kernel could be very slim, as almost all
hardware drivers could be disabled. The advantage of 1) and 2) is the lower
maintenance cost.


On a different note, I'd like to try to switch over to pv-grub on our
servers. Right now, upgrading the kernel requires copying the kernel to the
dom0 manually, thus dom0 access is required to do a (potentially security
relevant) kernel update. With pv-grub, upgrading the kernel and rebooting
would be handled just like with a normal machine, as the dom0 only loads the
pv-grub image which then runs inside the domU and can load the right kernel
from there.
The pv-grub is only available from Xen 3.3 and later, but we have Xen 3.2 -
I compiled pv-grub from 3.4 though and guess it should work with the older
hypervisor as well, but I haven't tested that yet.

Anyway, please post your opinion on the issues above.


I'm -1 on the PAE, leaving options 2 and 3 as viable. 2 is the easiest
for maintenance; 3 sounds like the best solution long term as it would
be a very lightweight and well-suited kernel package.

If not for the maintenance issues, 3 seems by far the best. Is there
anyway we can automate the creation of this package to the point where
it doesn't take much extra effort? e.g. try to share resources with
the primary kernel package.


Can you illustrate why you're -1 on PAE? I can't find exact info on
the performance hit caused by enabling that, and with machines having
more and more RAM, I'm sure we'll eventually need to enable PAE. I
_know_ we're trying to push people to 64bit, but that shouldn't mean
we leave the i686 arch in the dust.

I'm just sure we're EVENTUALLY going to need to enable this. Why not now?



Completely disagree, PAE is stupid. If you want to stick with 32 bit
and have more memory available, at least run a 64 bit kernel and stick
with a 32 bit userspace:
http://lkml.indiana.edu/hypermail/linux/kernel/0906.1/01128.html
http://lkml.indiana.edu/hypermail/linux/kernel/0906.1/01131.html



I do that, and the are some usability issues. Nothing insurmountable.
Like needing to use "linux32" before ./configure's, makepkg's, etc.
Nothing a graphical user would notice, but would be more noticeable in
Arch where building from the AUR is common.

From memory, installing a 64 bit kernel when appropriate was a release
goal for Fedora 11 but got dropped. I do not know why, but I guess the
small annoyances I find are probably the reason.



And those numbers for performance problems? 12%:
http://lkml.indiana.edu/hypermail/linux/kernel/0906.1/00959.html

Leaving i686 in the dust is hardly the concern here; why add
unnecessary overhead to those machines that sit well under the 4GB
mark that need that extra 12% of performance the most?



Red Hat did a more comprehensive survey of PAE (a few years ago now).
They found about the same thing: ~0% for some operations, 10% for the
worst (which reflects the test case Dan linked). It averaged out to
~1%. See http://people.redhat.com/nmurray/RHEL-2.1-VM-whitepaper.pdf
(top page 10).


Allan
 
Old 10-31-2009, 01:15 AM
Thomas Bächler
 
Default Xen domU kernel - what to do?

RedShift schrieb:

Thomas Bächler wrote:
Okay, it is obvious that we need a kernel which can run on Xen domU
for our servers. It would also be very convenient for those running
Arch on hosting services that have pv-grub available.




What about dumping Xen in favor of KVM and move libvirt and virt-manager
to extra?


KVM needs hardware support, while Xen is a para-virtualization
environment and doesn't need any special hardware. I don't know about
libvirt and what is is. However, Xen works just great (and is deployed
by many commercial hosters) and the main Arch servers have relied on it
for over 2 months. Xen is very stable and well-tested, I doubt you can
say the last on other para-virtualization environments.
 

Thread Tools




All times are GMT. The time now is 01:20 AM.

VBulletin, Copyright ©2000 - 2014, Jelsoft Enterprises Ltd.
Content Relevant URLs by vBSEO ©2007, Crawlability, Inc.
Copyright ©2007 - 2008, www.linux-archive.org