[[ch-xen]]
== Using Xen with DRBD

indexterm:[Xen]This chapter outlines the use of DRBD as a Virtual
Block Device (VBD) for virtualization environments using the Xen
hypervisor.

[[s-xen-primer]]
=== Xen primer

Xen is a virtualization framework originally developed at the
University of Cambridge (UK), and later being maintained by XenSource,
Inc. (now a part of Citrix). It is included in reasonably recent
releases of most Linux distributions, such as Debian GNU/Linux (since
version 4.0), SUSE Linux Enterprise Server (since release 10), Red Hat
Enterprise Linux (since release 5), and many others.

Xen uses indexterm:[Xen]paravirtualization — a virtualization method
involving a high degree of cooperation between the virtualization host
and guest virtual machines — with selected guest operating systems for
improved performance in comparison to conventional virtualization
solutions (which are typically based on hardware
emulation). indexterm:[Xen]Xen also supports full hardware emulation
on CPUs that support the appropriate virtualization extensions, in Xen
parlance, this is known as HVM ( "hardware-assisted virtual machine").

NOTE: At the time of writing, CPU extensions supported by Xen for HVM
are Intel's Virtualization Technology (VT, formerly codenamed
"Vanderpool"), and AMD's Secure Virtual Machine (SVM, formerly known
as "Pacifica").

Xen supports indexterm:[Xen]_live migration_, which refers to the
capability of transferring a running guest operating system from one
physical host to another, without interruption.

When a DRBD resource is used as a replicated Virtual Block Device
(VBD) for Xen, it serves to make the entire contents of a domU's
virtual disk available on two servers, which can then be configured
for automatic fail-over. That way, DRBD does not only provide
redundancy for Linux servers (as in non-virtualized DRBD deployment
scenarios), but also for any other operating system that can be
virtualized under Xen — which, in essence, includes any operating
system available on 32- or 64-bit Intel compatible architectures.

[[s-xen-drbd-mod-params]]
=== Setting DRBD module parameters for use with Xen

For Xen Domain-0 kernels, it is recommended to load the DRBD module
with the option `disable_sendpage` set to `1`. To do so, create (or open) the file
`/etc/modprobe.d/drbd.conf` and enter the following line:

[source,drbd]
----------------------------
options drbd disable_sendpage=1
----------------------------

[[s-xen-create-resource]]
=== Creating a DRBD resource suitable to act as a Xen VBD

Configuring a DRBD resource that is to be used as a Virtual Block
Device for Xen is fairly straightforward -- in essence, the typical
configuration matches that of a DRBD resource being used for any other
purpose. However, if you want to enable indexterm:[Xen]live migration
for your guest instance, you need to enable indexterm:[dual-primary
mode]<<s-dual-primary-mode,dual-primary mode>>for this resource:

[source,drbd]
----------------------------
resource <resource> {
  net {
    allow-two-primaries yes;
    ...
  }
  ...
}
----------------------------

Enabling dual-primary mode is necessary because Xen, before initiating
live migration, checks for write access on all VBDs a resource is
configured to use on both the source and the destination host for the
migration.

[[s-xen-configure-domu]]
=== Using DRBD VBDs

In order to use a DRBD resource as the virtual block device, you must
add a line like the following to your Xen domU configuration:
indexterm:[Xen]

[source,drbd]
----------------------------
disk = [ 'drbd:<resource>,xvda,w' ]
----------------------------

This example configuration makes the DRBD resource named _resource_
available to the domU as `/dev/xvda` in read/write mode ( `w`).

Of course, you may use multiple DRBD resources with a single domU. In
that case, simply add more entries like the one provided in the
example to the `disk` option, separated by commas.


NOTE: There are three sets of circumstances under which you cannot use
this approach:

* You are configuring a fully virtualized (HVM) domU.

* You are installing your domU using a graphical installation utility,
  _and_ that graphical installer does not support the `drbd:` syntax.

* You are configuring a domU without the `kernel`, `initrd`, and
  `extra` options, relying instead on `bootloader` and
  `bootloader_args` to use a Xen pseudo-bootloader, _and_ that
  pseudo-bootloader does not support the `drbd:` syntax.

** pygrub` (prior to Xen 3.3) and `domUloader.py` (shipped with Xen on
SUSE Linux Enterprise Server 10) are two examples of
pseudo-bootloaders that do not support the `drbd:` virtual block device
configuration syntax.

** `pygrub` from Xen 3.3 forward, and the `domUloader.py` version that
   ships with SLES 11 _do_ support this syntax.

Under these circumstances, you must use the traditional `phy:` device
syntax and the DRBD device name that is associated with your resource,
not the resource name. That, however, requires that you manage DRBD
state transitions outside Xen, which is a less flexible approach than
that provided by the `drbd` resource type.

[[s-manage-domu]]
=== Starting, stopping, and migrating DRBD-backed domU's

.Starting the domU
Once you have configured your DRBD-backed domU, you may start it as
you would any other domU:
----------------------------
xm create <domU>
Using config file "+/etc/xen/<domU>+".
Started domain <domU>
----------------------------

In the process, the DRBD resource you configured as the VBD will be
promoted to the primary role, and made accessible to Xen as expected.


.Stopping the domU
This is equally straightforward:
----------------------------
xm shutdown -w <domU>
Domain <domU> terminated.
----------------------------

Again, as you would expect, the DRBD resource is returned to the
secondary role after the domU is successfully shut down.


.Migrating the domU
This, too, is done using the usual Xen tools:
----------------------------
xm migrate --live <domU> <destination-host>
----------------------------

In this case, several administrative steps are automatically taken in
rapid succession:

* The resource is promoted to the primary role on _destination-host_.

* Live migration of _domU_ is initiated on the local host.

* When migration to the destination host has completed, the resource
  is demoted to the secondary role locally.

The fact that both resources must briefly run in the primary role on
both hosts is the reason for having to configure the resource in
dual-primary mode in the first place.


[[s-xen-internal]]
=== Internals of DRBD/Xen integration

Xen supports two Virtual Block Device types natively:

.`phy`
This device type is used to hand "physical" block devices, available
in the host environment, off to a guest domU in an essentially
transparent fashion.

.`file`
This device type is used to make file-based block device images
available to the guest domU. It works by creating a loop block device
from the original image file, and then handing that block device off
to the domU in much the same fashion as the `phy` device type does.

If a Virtual Block Device configured in the `disk` option of a domU
configuration uses any prefix other than `phy:`, `file:`, or no prefix
at all (in which case Xen defaults to using the `phy` device type),
Xen expects to find a helper script named ``block``-__prefix__ in the Xen
scripts directory, commonly `/etc/xen/scripts`.

The DRBD distribution provides such a script for the `drbd` device
type, named `/etc/xen/scripts/block-drbd`. This script handles the
necessary DRBD resource state transitions as described earlier in this
chapter.

[[s-xen-pacemaker]]
=== Integrating Xen with Pacemaker

indexterm:[Xen]indexterm:[Heartbeat]In order to fully capitalize on
the benefits provided by having a DRBD-backed Xen VBD's, it is
recommended to have Heartbeat manage the associated domU's as
Heartbeat resources.

You may configure a Xen domU as a Pacemaker resource, and automate
resource failover. To do so, use the `Xen` OCF resource agent. If you
are using the `drbd` Xen device type described in this chapter, you
will _not_ need to configure any separate `drbd` resource for use by
the Xen cluster resource. Instead, the `block-drbd` helper script will
do all the necessary resource transitions for you.
