1. Virtualization and Hardening

The hardening of virtualized environments is growing in popularity.
Virtualization has the advantages of isolating services on various slim guests
running on a larger server, while hardening provides for enhanced security for
both the guests and host.  In practice, however, getting the two to work
together is not always an easy task as the technologies employed by one often
interfer with the other.  This is complicated by the fact that there many
implementations of virtualization and many degrees of hardening.  This guide
aims to provide some clarity to the issues and outline some best practices.


2. Types of virtualization and degrees of hardening.

This guide looks at virtualization using kvm, xen and vmware under hardening
by GRSEC/PaX.  For each type of virtualization, we discuss what hardening
features work for the host and guests without either degrading performance
horribly or breaking completely.  This is not a howto on setting up
virtualization since that is covered elsewhere; rather, we limit our
discussion to just what hardening features ought to be enabled or disable when
configuring the kernel of the host or guest operating systems.


3. Hardening kvm.

KVM (Kernel-base Virtual Machine) provides virtualization on x86 and x86_64
hosts that have the required hardware support (Intel-VT or AMD-V).  The host
uses a general kernel module (kvm.ko), a processor specific module
(kvm-intel.ko or kvm-amd.ko), and a userland utility (qemu-kvm), to run the
guests.  The guests can be configured to use emulated hardware (full
virtualization) or virtio (para virtualization).  Paravirt has the advantage
of increasing performance and providing a common I/O interface between host
and guest.  Setting up kvm on gentoo is discussed in [1].  Configuring the
guest's kernel to use virtio is discussed in [2].

As of this writing, there are no known restrictions on hardening for the
guest.  Test of both x86 and x86_64 guests using either emulated hardware or
virtio, with all hardening features, including CONFIG_PAX_KERNEXEC and
CONFIG_PAX_MEMORY_UDEREF, have been successfull.

For the host, however, one must disable both CONFIG_PAX_KERNEXEC and
CONFIG_PAX_MEMORY_UDEREF.  Either of these will set an invisible kernel
option, CONFIG_PAX_PER_CPU_PGD, which is know to break kvm.  What is actually
happening is that the guest's performance is degraded to the point where it is
unusable, but doesn't crash, and the host is left with qemu-kvm in
uninterruptible sleep (state D when doing ps aux).  Only rebooting the host
clears the issue.

These tests were done using the 2.6.32 and 2.6.34 branches of the kernel with
GRSEC/PaX patch version 2.1.14 and 2.2.0.  See [3].  However, it unlikely that
this problem will be solved anytime soon, which is unfortunate because both
KERNEXEC and UDEREF are excellent hardening features.


4. Hardening xen.

Xen is an older virtualization technology than kvm, but similar in many
regards.  It employs a hypervisor which boots a specialize host's kernel
(dom0).  Once the host is up, it in turn runs guests (domU) 

TODO


5. VMWare Workstation

TODO





Ref.

[1] http://en.gentoo-wiki.com/wiki/KVM

[2] http://www.linux-kvm.org/page/Virtio

[3] http://bugs.gentoo.org/328623