<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE refentry PUBLIC "-//OASIS//DTD DocBook XML V4.5//EN"
"http://www.oasis-open.org/docbook/xml/4.5/docbookx.dtd" [
  <!ENTITY drbdsetup_options SYSTEM "./drbdsetup-options.xml">
]>
<refentry id="re-drbdsetup">
  &drbdsetup_options;

  <refentryinfo>
    <date>17 January 2018</date>

    <productname>DRBD</productname>

    <productnumber>9.0.x</productnumber>
  </refentryinfo>

  <refmeta>
    <refentrytitle>drbdsetup</refentrytitle>

    <manvolnum>8</manvolnum>

    <refmiscinfo class="manual">System Administration</refmiscinfo>
  </refmeta>

  <refnamediv>
    <refname>drbdsetup</refname>

    <refpurpose>Configure the DRBD kernel module<indexterm significance="normal">
        <primary>drbdsetup</primary>
      </indexterm></refpurpose>
  </refnamediv>

  <refsynopsisdiv>
    <cmdsynopsis sepchar=" ">
      <command moreinfo="none">drbdsetup</command>
      <arg choice="plain" rep="norepeat">command</arg>
      <arg choice="req" rep="repeat">argument</arg>
      <arg choice="opt" rep="repeat">option</arg>
    </cmdsynopsis>
  </refsynopsisdiv>

  <refsect1>
    <title>Description</title>

    <para>The <command moreinfo="none">drbdsetup</command> utility serves to
      configure the DRBD kernel module and to show its current configuration.
      Users usually interact with the <command
	moreinfo="none">drbdadm</command> utility, which provides a more
      high-level interface to DRBD than <command
	moreinfo="none">drbdsetup</command>.  (See <option>drbdadm</option>'s
      <option>--dry-run</option> option to see how <option>drbdadm</option>
      uses <option>drbdsetup</option>.)</para>

    <para>Some option arguments have a default scale which applies when a plain
    number is specified (for example Kilo, or 1024 times the numeric value).
    Such default scales can be overridden by using a suffix (for example, M
    for Mega). The common suffixes K = 2^10 = 1024, M = 1024 K, and G = 1024
    M are supported.</para>
  </refsect1>

  <refsect1>
    <title>Commands</title>

    <variablelist>
      <varlistentry>
	<xi:include href="drbdsetup_attach.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Attach a lower-level device to an existing replicated device. -->
	<xi:include href="drbdsetup_disk-options.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Change the disk options of an attached lower-level device. -->

        <listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>disk</secondary>
	  </indexterm>

	  <para>The <option>attach</option> command attaches a lower-level
	    device to an existing replicated device.  The
	    <option>disk-options</option> command changes the disk options of
	    an attached lower-level device.  In either case, the replicated
	    device must have been created with <command
	    moreinfo="none">drbdsetup new-minor</command>.</para>

	    <para>Both commands refer to the replicated device by its
	    <replaceable>minor</replaceable> number.
	    <replaceable>lower_dev</replaceable> is the name of the lower-level
	    device.  <replaceable>meta_data_dev</replaceable> is the name of
	    the device containing the metadata, and may be the same as
	    <replaceable>lower_dev</replaceable>.
	    <replaceable>meta_data_index</replaceable> is either a numeric
	    metadata index, or the keyword <option>internal</option> for
	    internal metadata, or the keyword <option>flexible</option> for
	    variable-size external metadata. Available
	    options:</para>


	  <variablelist>
	    <pick_drbdsetup_option name="al-extents"/>
	    <pick_drbdsetup_option name="al-updates"/>
	    <pick_drbdsetup_option name="disk-barrier"/>
	    <pick_drbdsetup_option name="disk-timeout"/>
	    <pick_drbdsetup_option name="md-flushes"/>
	    <pick_drbdsetup_option name="on-io-error"/>
	    <pick_drbdsetup_option name="read-balancing"/>
	    <pick_drbdsetup_option name="resync-after"/>
	    <pick_drbdsetup_option name="size"/> <!-- only for attach, not for disk-options? -->
	    <pick_drbdsetup_option name="discard-zeroes-if-aligned"/>
	    <pick_drbdsetup_option name="disable-write-same"/>
	    <pick_drbdsetup_option name="rs-discard-granularity"/>
          </variablelist>
        </listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_peer-device-options.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Remember the current size of a lower-level device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>peer-device-options</secondary>
	  </indexterm>

	  <para>These are options that affect the <replaceable>peer</replaceable>'s device.
	  </para>


	  <variablelist>
	    <pick_drbdsetup_option name="c-delay-target"/>
	    <pick_drbdsetup_option name="c-fill-target"/>
	    <pick_drbdsetup_option name="c-max-rate"/>
	    <pick_drbdsetup_option name="c-min-rate"/>
	    <pick_drbdsetup_option name="c-plan-ahead"/>
	    <pick_drbdsetup_option name="resync-rate"/>
	 </variablelist>

	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_check-resize.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Remember the current size of a lower-level device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>check-resize</secondary>
	  </indexterm>

	  <para>Remember the current size of the lower-level device of the
	  specified replicated device.  Used by drbdadm.  The size information
	  is stored in file
	  /var/lib/drbd/drbd-minor-<replaceable>minor</replaceable>.lkbd.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_new-peer.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Create a connection/peer within a resource. -->
	<xi:include href="drbdsetup_net-options.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Change the network options of a connection. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>net</secondary>
	  </indexterm>

	  <para>The <option>new-peer</option> command creates a connection within
          a <replaceable>resource</replaceable>. The resource
	  must have been created with <command moreinfo="none">drbdsetup
	  new-resource</command>.  The <option>net-options</option> command
	  changes the network options of an existing connection. Before a
	  connection can be activated with the <option>connect</option>
	  command, at least one path need to added with the <option>new-path</option>
	  command. Available options:</para>

	  <variablelist>

	    <pick_drbdsetup_option name="after-sb-0pri"/>
	    <pick_drbdsetup_option name="after-sb-1pri"/>
	    <pick_drbdsetup_option name="after-sb-2pri"/>
	    <pick_drbdsetup_option name="allow-two-primaries"/>
	    <pick_drbdsetup_option name="always-asbp"/>
	    <pick_drbdsetup_option name="connect-int"/>
	    <pick_drbdsetup_option name="cram-hmac-alg"/>
	    <pick_drbdsetup_option name="csums-alg"/>
	    <pick_drbdsetup_option name="csums-after-crash-only"/>
	    <pick_drbdsetup_option name="data-integrity-alg"/>
	    <pick_drbdsetup_option name="fencing"/>
	    <pick_drbdsetup_option name="ko-count"/>
	    <pick_drbdsetup_option name="max-buffers"/>
	    <pick_drbdsetup_option name="max-epoch-size"/>
	    <pick_drbdsetup_option name="on-congestion"/>
	    <pick_drbdsetup_option name="ping-int"/>
	    <pick_drbdsetup_option name="ping-timeout"/>
	    <pick_drbdsetup_option name="socket-check-timeout"/>
	    <pick_drbdsetup_option name="protocol"/>
	    <pick_drbdsetup_option name="rcvbuf-size"/>
	    <pick_drbdsetup_option name="rr-conflict"/>
	    <pick_drbdsetup_option name="shared-secret"/>
	    <pick_drbdsetup_option name="sndbuf-size"/>
	    <pick_drbdsetup_option name="tcp-cork"/>
	    <pick_drbdsetup_option name="timeout"/>
	    <pick_drbdsetup_option name="use-rle"/>
	    <pick_drbdsetup_option name="verify-alg"/>
	  </variablelist>
	</listitem>
      </varlistentry>


      <varlistentry>
	<xi:include href="drbdsetup_new-path.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Create a path to a peer (within a connection). -->
	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>net</secondary>
	  </indexterm>

	  <para>The <option>new-path</option> command creates a path within
          a <replaceable>connection</replaceable>. The connection
	  must have been created with <command moreinfo="none">drbdsetup
	  new-peer</command>. <replaceable>Local_addr</replaceable> and
	  <replaceable>remote_addr</replaceable> refer to the local and remote
	  protocol, network address, and port in the format
	  <optional><replaceable>address-family</replaceable>:</optional><replaceable>address</replaceable><optional>:<replaceable>port</replaceable></optional>.
	  The address families <option>ipv4</option>,
	  <option>ipv6</option>, <option>ssocks</option> (Dolphin Interconnect
	  Solutions' "super sockets"), <option>sdp</option> (Infiniband Sockets
	  Direct Protocol), and <option>sci</option> are supported
	  (<option>sci</option> is an alias for <option>ssocks</option>).  If
	  no address family is specified, <option>ipv4</option> is assumed. For
	  all address families except <option>ipv6</option>, the
	  <replaceable>address</replaceable> uses IPv4 address notation (for
	  example, 1.2.3.4). For <option>ipv6</option>, the address is enclosed
	  in brackets and uses IPv6 address notation (for example,
	  [fd01:2345:6789:abcd::1]).  The <replaceable>port</replaceable>
	  defaults to 7788.</para>

	</listitem>
      </varlistentry>


      <varlistentry>
	<xi:include href="drbdsetup_connect.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Activate a connection. -->
	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>net</secondary>
	  </indexterm>

	  <para>The <option>connect</option> command activates a connection.
	  That means that the DRBD driver will bind and listen on all local
	  addresses of the connection-'s paths. It will begin to try to establish
	  one or more paths of the connection.  Available options: </para>

	  <variablelist>
	    <pick_drbdsetup_option name="tentative"/>
	    <pick_drbdsetup_option name="discard-my-data"/>
	  </variablelist>

	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_del-peer.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Remove a connection/peer within a resource. -->
	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>net</secondary>
	  </indexterm>

	  <para>The <option>del-peer</option> command removes a connection from
          a <replaceable>resource</replaceable>.</para>

	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_del-path.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Remove a connection/peer within a resource. -->
	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>net</secondary>
	  </indexterm>

	  <para>The <option>del-path</option> command removes a path from
          a <replaceable>connection</replaceable>. Please note that it fails if
	  the path is necessary to keep a connected connection in tact. In order
	  to remove all paths, disconnect the connection first.</para>

	</listitem>
      </varlistentry>


      <varlistentry>
	<xi:include href="drbdsetup_cstate.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Show the current state of a connection. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>cstate</secondary>
	  </indexterm>

	  <para>Show the current state of a connection.  The connection is
	  identified by the node-id of the peer; see the <command
	  moreinfo="none">drbdsetup connect</command> command.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_del-minor.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Remove a replicated device. -->

	<listitem>
	  <para>Remove a replicated device.  No lower-level device may be
	  attached; see <command moreinfo="none">drbdsetup detach</command>.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_del-resource.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Remove a resource. -->

	<listitem>
	  <para>Remove a resource.  All volumes and connections must be removed
	  first (<command moreinfo="none">drbdsetup del-minor</command>,
	  <command moreinfo="none">drbdsetup disconnect</command>).
	  Alternatively, <command moreinfo="none">drbdsetup down</command> can
	  be used to remove a resource together with all its volumes and
	  connections.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_detach.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Detach the lower-level device of a replicated device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>detach</secondary>
	  </indexterm>
	  <para>Detach the lower-level device of a replicated device.  Available options:
	  <variablelist>
	    <varlistentry>
	      <term><option>--force</option></term>
	      <listitem>
		<para>Force the detach and return immediately.  This puts the
		lower-level device into failed state until all pending I/O
		has completed, and then detaches the device.  Any I/O not yet
		submitted to the lower-level device (for example, because I/O
		on the device was suspended) is assumed to have failed.</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>
	  </para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_disconnect.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Disconnect a resource from a peer host. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>disconnect</secondary>
	  </indexterm>

	  <para>Remove a connection to a peer host.  The connection is
	  identified by the node-id of the peer; see the <command
	  moreinfo="none">drbdsetup connect</command> command.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_down.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Take a resource down. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>down</secondary>
	  </indexterm>

	  <para>Take a resource down by removing all volumes, connections, and
	    the resource itself.
	  </para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_dstate.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Show the current disk state of a lower-level device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>dstate</secondary>
	  </indexterm>

	  <para>Show the current disk state of a lower-level device.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_events2.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Show the current state and all state changes of all resources. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>events2</secondary>
	  </indexterm>

	  <para>Show the current state of all configured DRBD objects, followed
	    by all changes to the state.</para>

	  <para>The output format is meant to be human as well as machine
	    readable. The line starts with a word that indicates the kind of event:
	    <option>exists</option> for an existing object;
	    <option>create</option>, <option>destroy</option>, and
	    <option>change</option> if an object is created, destroyed, or
	    changed; <option>call</option> or <option>response</option> if
	    an event handler is called or it returns; or <option>rename</option>
	    when the name of an object is changed.  The second word indicates
	    the object the event applies to: <option>resource</option>,
	    <option>device</option>, <option>connection</option>,
	    <option>peer-device</option>, <option>path</option>,
	    <option>helper</option>, or a dash (<option>-</option>) to indicate
	    that the current state has been dumped completely.</para>

	  <para>The remaining words identify the object and describe the state
	    that the object is in. Some special keys are worth mentioning:

	    <variablelist>
	      <varlistentry>
		<term>resource <option>may_promote:{yes|no}</option></term>

		<listitem>
		  <para>Whether promoting to primary is expected to succeed.
		    When <option>quorum</option> is enabled, this can be used
		    to trigger failover. When <option>may_promote:yes</option>
		    is reported on this node, then no writes are possible on
		    any other node, which generally means that the application
		    can be started on this node, even when it has been running
		    on another.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term>resource <option>promotion_score:<replaceable>score</replaceable></option></term>

		<listitem>
		  <para>An integer heuristic indicating the relative preference
		    for promoting this resource. A higher score is better in
		    terms of having local disks and having access to up-to-date
		    data. The score may be positive even when some node is
		    primary. It will be zero when promotion is impossible due
		    to quorum or lack of any access to up-to-date data.</para>
		</listitem>
	      </varlistentry>
	    </variablelist>
	  </para>

	  <para>Available options:
	    <variablelist>
	      <varlistentry>
		<term><option>--now</option></term>

		<listitem>
		  <para>Terminate after reporting the current state.  The
		    default is to continuously listen and report state
		    changes.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term><option>--poll</option></term>

		<listitem>
		  <para>Read from stdin and update when <option>n</option> is
		    read. Newlines are ignored. Every other input terminates
		    the command.</para>

		  <para>Without <option>--now</option>, changes are printed as
		    usual. On each <option>n</option> the current state is
		    fetched, but only changed objects are printed. This is
		    useful with <option>--statistics</option> or
		    <option>--full</option> because DRBD does not otherwise
		    send updates when only the statistics change.</para>

		  <para>In combination with <option>--now</option> the full
		    state is printed on each <option>n</option>. No other
		    changes are printed.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term><option>--statistics</option></term>

		<listitem>
		  <para>Include statistics in the output.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term><option>--diff</option></term>

		<listitem>
		  <para>Write information in form of a diff between old and new state.
		   This helps simple tools to avoid (old) state tracking on their own.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term><option>--full</option></term>

		<listitem>
		  <para>Write complete state information, especially on change events.
		   This enables <option>--statistics</option> and <option>--verbose</option>.</para>
		</listitem>
	      </varlistentry>

	    </variablelist>
	  </para>
	</listitem>
    </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_get-gi.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Show the data generation identifiers for a device on a particular connection. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>get-gi</secondary>
	  </indexterm>

	  <para>Show the data generation identifiers for a device on a
	  particular connection.  The device is identified by its volume
	  number.  The connection is identified by its endpoints; see the
	  <command moreinfo="none">drbdsetup connect</command> command.</para>

	  <para>The output consists of the current UUID, bitmap UUID, and the
	  first two history UUIDS, folowed by a set of flags.  The current UUID
	  and history UUIDs are device specific; the bitmap UUID and flags are
	  peer device specific.  This command only shows the first two history
	  UUIDs.  Internally, DRBD maintains one history UUID for each possible
	  peer device.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_invalidate.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Replace the local data of a resource with that of a peer. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>invalidate</secondary>
	  </indexterm>

	  <para>Replace the local data of a device with that of a peer.  All
	    the local data will be marked out-of-sync, and a resync with the
	    specified peer device will be initialted.</para>
	  <para>Available options:
	    <variablelist>
	      <varlistentry>
		<term><option>--reset-bitmap=no</option></term>

		<listitem>
                  <para>Usually an invalidate operation sets all bits in the bitmap
                  to out-of-sync before beginning the resync from the peer. By giving
                  <option>--reset-bitmap=no</option> DRBD will use the bitmap as it is.
                  Usually this is used after an online verify operation found differences
                  in the backing devices.</para>
		  <para>The <option>--reset-bitmap</option> option is available since DRBD
		  kernel driver 9.0.29 and drbd-utils 9.17.</para>
                </listitem>
               </varlistentry>
	       <varlistentry>
		<term><option>--sync-from-peer-node-id</option></term>
		<listitem>
		  <para>This option allows the caller to select the node to resync from.
		    if it is not gives, DRBD selects a suitable source node itself.</para>
		</listitem>
               </varlistentry>
             </variablelist>
           </para>
	  <!-- "This command will fail if the device is not part of a connected device pair." -->
	  <!-- FIXME: What if the peer is not connected? -->
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_invalidate-remote.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Replace a peer's data of a resource with the local data. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>invalidate-remote</secondary>
	  </indexterm>

	  <para>Replace a peer device's data of a resource with the local data.
	    The peer device's data will be marked out-of-sync, and a resync
	    from the local node to the specified peer will be initiated.</para>

	  <para>Available options:
	    <variablelist>
	      <varlistentry>
		<term><option>--reset-bitmap=no</option></term>

		<listitem>
                  <para>Usually an invalidate remote operation sets all bits in the bitmap
                  to out-of-sync before beginning the resync to the peer. By giving
                  <option>--reset-bitmap=no</option> DRBD will use the bitmap as it is.
                  Usually this is used after an online verify operation found differences
                  in the backing devices.</para>
		  <para>The <option>--reset-bitmap</option> option is available since DRBD
		  kernel driver 9.0.29 and drbd-utils 9.17.</para>
                </listitem>
               </varlistentry>
	    </variablelist>
	  </para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_new-current-uuid.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Generate a new current UUID. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>new-current-uuid</secondary>
	  </indexterm>

	  <!-- FIXME: The description does not really explain what is going on
	  and why; this needs to be completely analyzed and revised.  Also, we
	  should have separate commands for creating a new current uuid and for
	  clearing unused bitmap slots. -->

	  <para>Generate a new current UUID and rotates all other UUID values. This has three
	  use cases: start the initial resync; skip the initial resync; bootstrap a single node cluster.</para>

	  <para>Available options: <variablelist>
	      <varlistentry>
		<term><option>--force-resync</option></term>

		<listitem>
		  <para>Start an initial resync. A precondition is that the volume is in disk state
		  <option>Inconsistent</option> on all nodes. This command updates the disk state
		  on the current node to <option>UpToDate</option> and makes it source of the
		  resync operations to the peers.</para>
		</listitem>
	      </varlistentry>
	      <varlistentry>
		<term><option>--clear-bitmap</option></term>

		<listitem>
		  <para>Clears the sync bitmap in addition to generating a new current UUID. This
		  skips the initial resync. As a consqeuence this volume's disk state changes to
		  <option>UpToDate</option> on all nodes in this resource.</para>
		</listitem>
	      </varlistentry>
	    </variablelist></para>

	    <para>Both operations require a "Just Created" meta data. Here is the complete
	    sequence step by step how to skip the initial resync: <orderedlist
	      continuation="restarts" inheritnum="ignore" numeration="arabic">
	      <listitem>
		<simpara>On <emphasis>both</emphasis> nodes, initialize meta data and configure the
		device.</simpara>

		<simpara><command moreinfo="none">drbdadm create-md --force
		<replaceable>res/volume-number</replaceable></command></simpara>
	      </listitem>

	      <listitem>
		<simpara>They need to do the initial handshake, so they know their sizes.</simpara>

		<simpara><command moreinfo="none">drbdadm up
		<replaceable>res</replaceable></command></simpara>
	      </listitem>

	      <listitem>
		<simpara>They are now Connected Secondary/Secondary Inconsistent/Inconsistent.
		Generate a new current-uuid and clear the dirty bitmap.</simpara>

		<simpara><command moreinfo="none">drbdadm --clear-bitmap new-current-uuid
		<replaceable>res</replaceable></command></simpara>
	      </listitem>

	      <listitem>
		<simpara>They are now Connected Secondary/Secondary UpToDate/UpToDate. Make one side
		primary and create a file system.</simpara>

		<simpara><command moreinfo="none">drbdadm primary
		<replaceable>res</replaceable></command></simpara>

		<simpara><command moreinfo="none">mkfs -t <replaceable>fs-type</replaceable> $(drbdadm
		sh-dev <replaceable>res/vol</replaceable>)</command></simpara>
	      </listitem>
	    </orderedlist></para>

	  <para>One obvious side-effect is that the replica is full of old garbage (unless you made
	  them identical using other means), so any online-verify is expected to find any number of
	  out-of-sync blocks.</para>

	  <para><emphasis>You must not use this on pre-existing data!</emphasis> Even though it may
	  appear to work at first glance, once you switch to the other node, your data is toast, as it
	  never got replicated. So <emphasis>do not leave out the mkfs</emphasis> (or
	  equivalent).</para>

	  <variablelist>
	    <varlistentry>
	      <term><emphasis role="strong">Bootstraping a single node cluster</emphasis></term><listitem>
	  <para>This can also be used to shorten the initial resync of a cluster where the second node
	  is added after the first node is gone into production, by means of disk shipping. This
	  use-case works on disconnected devices only, the device may be in primary or secondary
	  role.</para>

	  <para>The necessary steps on the current active server are: <orderedlist
	      continuation="restarts" inheritnum="ignore" numeration="arabic">
	      <listitem>
		<simpara><command moreinfo="none">drbdsetup new-current-uuid --clear-bitmap <replaceable>minor</replaceable>
		</command></simpara>
	      </listitem>

	      <listitem>
		<simpara>Take the copy of the current active server. E.g. by pulling a disk out of the
		RAID1 controller, or by copying with dd. You need to copy the actual data, and the
		meta data.</simpara>
	      </listitem>

	      <listitem>
		<simpara><command moreinfo="none">drbdsetup new-current-uuid <replaceable>minor</replaceable>
		</command></simpara>
	      </listitem>
	    </orderedlist> Now add the disk to the new secondary node, and join it to the cluster. You
	  will get a resync of that parts that were changed since the first call to <command
	  moreinfo="none">drbdsetup</command> in step 1.</para></listitem></varlistentry></variablelist>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_new-minor.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Create a new replicated device within a resource. -->

	<listitem>
	  <para>Create a new replicated device within a resource.  The command
	    creates a block device inode for the replicated device (by default,
	    /dev/drbd<replaceable>minor</replaceable>).  The
	    <replaceable>volume</replaceable> number identifies the device
	    within the <replaceable>resource</replaceable>.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_new-resource.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Create a new resource. -->
	<xi:include href="drbdsetup_resource-options.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Change the resource options of an existing resource. -->

	<listitem>
	  <para>The <option>new-resource</option> command creates a new
	    resource.  The <option>resource-options</option> command changes
	    the resource options of an existing resource.  Available options:

	  <variablelist>
	    <pick_drbdsetup_option name="auto-promote"/>
	    <pick_drbdsetup_option name="cpu-mask"/>
	    <pick_drbdsetup_option name="on-no-data-accessible"/>
	    <pick_drbdsetup_option name="peer-ack-window"/>
	    <pick_drbdsetup_option name="peer-ack-delay"/>
	    <pick_drbdsetup_option name="quorum"/>
	    <pick_drbdsetup_option name="quorum-minimum-redundancy"/>
	    <pick_drbdsetup_option name="on-no-quorum"/>
	    <pick_drbdsetup_option name="on-suspended-primary-outdated"/>
	  </variablelist>
	  </para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_outdate.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Mark the data on a lower-level device as outdated. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>outdate</secondary>
	  </indexterm>

	  <para>Mark the data on a lower-level device as outdated.  This
	    is used for fencing, and prevents the resource the device is part
	    of from becoming primary in the future.  See the
	    <option>--fencing</option> disk option.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_pause-sync.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Stop resynchronizing between a local and a peer device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>pause-sync</secondary>
	  </indexterm>

	  <para>Stop resynchronizing between a local and a peer device by
	    setting the local pause flag.  The resync can only resume if the
	    pause flags on both sides of a connection are cleared.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_primary.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Change the role of a node in a resource to primary. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>primary</secondary>
	  </indexterm>

	  <para>Change the role of a node in a resource to primary.  This
	  allows the replicated devices in this resource to be mounted or
	  opened for writing.  Available options:</para>

	  <variablelist>
	    <varlistentry>
	      <term><option>--overwrite-data-of-peer</option></term>

	      <listitem>
		<para>This option is an alias for the <option>--force</option> option.</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>

	  <variablelist>
	    <varlistentry>
	      <term><option>--force</option></term>

	      <listitem>
		<para>Force the resource to become primary even if some devices
		  are not guaranteed to have up-to-date data.  This option is
		  used to turn one of the nodes in a newly created cluster into
		  the primary node, or when manually recovering from a
		  disaster.</para>
		<para>Note that this can lead to split-brain scenarios.  Also,
		  when forcefully turning an inconsistent device into an
		  up-to-date device, it is highly recommended to use any
		  integrity checks available (such as a filesystem check) to
		  make sure that the device can at least be used without
		  crashing the system.</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>

	  <para>Note that DRBD usually only allows one node in a cluster to be
	    in primary role at any time; this allows DRBD to coordinate access
	    to the devices in a resource across nodes.  The
	    <option>--allow-two-primaries</option> network option changes this;
	    in that case, a mechanism outside of DRBD needs to coordinate
	    device access.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_resize.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Reexamine the lower-level device sizes to resize a replicated device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>resize</secondary>
	  </indexterm>

	  <para>Reexamine the size of the lower-level devices of a replicated
	  device on all nodes.  This command is called after the lower-level
	  devices on all nodes have been grown to adjust the size of the
	  replicated device.  Available options:

	    <variablelist>
	      <varlistentry>
		<term><option>--assume-peer-has-space</option></term>

		<listitem>
		  <para>Resize the device even if some of the peer devices are
		  not connected at the moment.  DRBD will try to resize the peer
		  devices when they next connect.  It will refuse to connect to a
		  peer device which is too small.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term><option>--assume-clean</option></term>
		<listitem>
		  <para>Do not resynchronize the added disk space; instead,
		  assume that it is identical on all nodes.  This option can be
		  used when the disk space is uninitialized and differences do
		  not matter, or when it is known to be identical on all nodes.
		  See the <command moreinfo="none">drbdsetup verify</command>
		  command.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term><option>--size <replaceable>val</replaceable></option></term>
		<listitem>
		  <para>This option can be used to online shrink the usable
		  size of a drbd device. It's the users responsibility to
		  make sure that a file system on the device is not
		  truncated by that operation.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term><option>--al-stripes <replaceable>val</replaceable></option>
		<option>--al-stripes <replaceable>val</replaceable></option></term>
		<listitem>
		  <para>These options may be used to change the layout of
		  the activity log online. In case of internal meta data
		  this may invovle shrinking the user visible size at the
		  same time (unsing the <option>--size</option>) or
		  increasing the avalable space on the backing
		  devices.</para>
		</listitem>
	      </varlistentry>

	    </variablelist>
	  </para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_resume-io.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Resume I/O on a replicated device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>resume-io</secondary>
	  </indexterm>

	  <para>Resume I/O on a replicated device.  See the
	  <option>--fencing</option> net option.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_resume-sync.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Resume the resynchronization between a local and a peer device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>resume-sync</secondary>
	  </indexterm>

	  <para>Allow resynchronization to resume by clearing the local sync pause flag.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_role.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Show the current role of a resource. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>role</secondary>
	  </indexterm>

	  <para>Show the current role of a resource.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_secondary.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Change the role of a node in a resource to secondary. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>secondary</secondary>
	  </indexterm>

	  <para>Change the role of a node in a resource to secondary.  This
	  command fails if the replicated device is in use.</para>

	  <variablelist>
	    <varlistentry>
	      <term><option>--force</option></term>

	      <listitem>
		<para>A forced demotion to secondary causes all pending and new IO requests
		to terminate with IO errors.</para>
		<para>Please note that a forced demotion returns immediately.
		The user should unmount any filesystem that might
		be mounted on the DRBD device. The device can be used again when
		<option>force-io-failures</option> has a value of <option>no</option>.
		(See <command moreinfo="none">drbdsetup status</command> and
		<command moreinfo="none">drbdsetup events2</command>).</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>

	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_show.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Show the current configuration of a resource, or of all resources. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>show</secondary>
	  </indexterm>

	  <para>Show the current configuration of a resource, or of all
	  resources.  Available options:

	    <variablelist>
	      <varlistentry>
		<term><option>--show-defaults</option></term>

		<listitem>
		  <para>Show all configuration parameters, even the ones with
		  default values.  Normally, parameters with default values are
		  not shown.</para>
		</listitem>
	      </varlistentry>
	    </variablelist>
	  </para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_show-gi.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Show data generation identifiers for a device and peer device, with explanations. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>show-gi</secondary>
	  </indexterm>

	  <para>Show the data generation identifiers for a device on a
	  particular connection.  In addition, explain the output.  The output
	  otherwise is the same as in the <command moreinfo="none">drbdsetup
	  get-gi</command> command.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<term><command moreinfo="none">drbdsetup</command><arg choice="plain" rep="norepeat">state</arg></term>

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>state</secondary>
	  </indexterm>

	  <para>This is an alias for <command moreinfo="none">drbdsetup
	  role</command>. Deprecated.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_status.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Show the status of a resource, or of all resources. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>status</secondary>
	  </indexterm>

	  <para>Show the status of a resource, or of all resources.  The
	  output consists of one paragraph for each configured resource.  Each
	  paragraph contains one line for each resource, followed by one line
	  for each device, and one line for each connection.  The device and
	  connection lines are indented.  The connection lines are followed by
	  one line for each peer device; these lines are indented against the
	  connection line.</para>

	  <para>Long lines are wrapped around at terminal width, and indented
	  to indicate how the lines belongs together.  Available options:

	    <variablelist>
	      <varlistentry>
		<term><option>--verbose</option></term>

		<listitem>
		  <para>Include more information in the output even when it is
		  likely redundant or irrelevant.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term><option>--statistics</option></term>

		<listitem>
		  <para>Include data transfer statistics in the output.</para>
		</listitem>
	      </varlistentry>

	      <varlistentry>
		<term>
		  <option>--color=<group choice="req" rep="norepeat">
		      <arg choice="plain" rep="norepeat">always</arg>
		      <arg choice="plain" rep="norepeat">auto</arg>
		      <arg choice="plain" rep="norepeat">never</arg>
		    </group>
		  </option>
		</term>

		<listitem>
		  <para>Colorize the output.  With
		    <option>--color=auto</option>, <option>drbdsetup</option>
		    emits color codes only when standard output is connected to
		    a terminal.
		  </para>
		</listitem>
	      </varlistentry>
	    </variablelist>
	  </para>

	  <para>For example, the non-verbose output for a resource with only
	  one connection and only one volume could look like this:
	      <programlisting format="linespecific">
drbd0 role:Primary
  disk:UpToDate
  host2.example.com role:Secondary
    disk:UpToDate
	      </programlisting>
	    </para>

	    <para>With the <option>--verbose</option> option, the same resource
	    could be reported as:
	      <programlisting format="linespecific">
drbd0 node-id:1 role:Primary suspended:no
  volume:0 minor:1 disk:UpToDate blocked:no
  host2.example.com local:ipv4:192.168.123.4:7788
      peer:ipv4:192.168.123.2:7788 node-id:0 connection:WFReportParams
      role:Secondary congested:no
    volume:0 replication:Connected disk:UpToDate resync-suspended:no
	      </programlisting>
	    </para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_suspend-io.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Suspend I/O on a replicated device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>suspend-io</secondary>
	  </indexterm>

	  <para>Suspend I/O on a replicated device.  It is not usually
	  necessary to use this command.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_verify.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Verify the data on a lower-level device against a peer device. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>verify</secondary>
	  </indexterm>

	  <para>Start online verification, change which part of the device will
	  be verified, or stop online verification.  The command requires the
	  specified peer to be connected.</para>

	  <para>Online verification compares each disk block on the local and
	  peer node.  Blocks which differ between the nodes are marked
	  as out-of-sync, but they are <emphasis>not</emphasis> automatically
	  brought back into sync.  To bring them into sync, the
	  <command>drbdsetup invalidate</command> or <command>drbdsetup invalidate-remote</command>
	  with the <option>--reset-bitmap=no</option> option can be used.
	  Progress can be monitored in the output of <command moreinfo="none">drbdsetup status
	    --statistics</command>.  Available options:</para>

	  <variablelist>
	    <varlistentry>
	      <term><option>--start <replaceable>position</replaceable></option></term>

	      <listitem>
		<para>Define where online verification should start.  This
		parameter is ignored if online verification is already in
		progress.  If the start parameter is not specified, online
		verification will continue where it was interrupted (if the
		connection to the peer was lost while verifying), after the
		previous stop sector (if the previous online verification has
		finished), or at the beginning of the device (if the end of the
		device was reached, or online verify has not run
		before).</para>

		<para>The position on disk is specified in disk sectors (512
		bytes) by default.</para>
	      </listitem>
	    </varlistentry>
	    <varlistentry>
	      <term><option>--stop <replaceable>position</replaceable></option></term>
	      <listitem>
		<para>Define where online verification should stop.  If online
		verification is already in progress, the stop position of the
		active online verification process is changed.  Use this to
		stop online verification.</para>

		<para>The position on disk is specified in disk sectors (512
		bytes) by default.</para>
	      </listitem>
	    </varlistentry>
	  </variablelist>

	  <para>Also see the notes on data integrity in the <citerefentry>
	  <refentrytitle>drbd.conf</refentrytitle> <manvolnum>5</manvolnum>
	  </citerefentry> manual page.</para>
	</listitem>
      </varlistentry>

      <varlistentry>

	<xi:include href="drbdsetup_wait-connect-volume.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Wait until a device on a peer is visible. -->
	<xi:include href="drbdsetup_wait-connect-connection.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Wait until all peer volumes of connection are visible. -->
	<xi:include href="drbdsetup_wait-connect-resource.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Wait until all connections are establised. -->

	<xi:include href="drbdsetup_wait-sync-volume.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Wait until a device on a peer is up to date. -->
	<xi:include href="drbdsetup_wait-sync-connection.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Wait until resync finished on all volumes of a connection. -->
	<xi:include href="drbdsetup_wait-sync-resource.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Wait until resync finished on all volumes. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>wait-connect-volume</secondary>
	    <secondary>wait-connect-connection</secondary>
	    <secondary>wait-connect-resource</secondary>
	  </indexterm>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>wait-sync-volume</secondary>
	    <secondary>wait-sync-connection</secondary>
	    <secondary>wait-sync-resource</secondary>
	  </indexterm>

	  <para>The <option>wait-connect-*</option> commands waits until a device
	  on a peer is visible.  The <option>wait-sync-*</option> commands waits
	  until a device on a peer is up to date.  Available options for both
	  commands:

	    <variablelist>
	      <pick_drbdsetup_option name="degr-wfc-timeout"/>
	      <pick_drbdsetup_option name="outdated-wfc-timeout"/>
	      <pick_drbdsetup_option name="wait-after-sb"/>
	      <pick_drbdsetup_option name="wfc-timeout"/>
	    </variablelist>
	  </para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_forget-peer.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Completely remove any reference to a unconnected peer from meta-data. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>forget-peer</secondary>
	  </indexterm>

	  <para>The <option>forget-peer</option> command removes all traces of
	  a peer node from the meta-data. It frees a bitmap slot in the meta-data
	  and make it avalable for futher bitmap slot allocation in case a
	  so-far never seen node connects.</para>
	  <para>The connection must be taken down before this command may be used.
	  In case the peer re-connects at a later point a bit-map based resync
	  will be turned into a full-sync.</para>
	</listitem>
      </varlistentry>

      <varlistentry>
	<xi:include href="drbdsetup_rename-resource.xml" xmlns:xi="http://www.w3.org/2001/XInclude" />
	<!-- Rename a resource. -->

	<listitem>
	  <indexterm significance="normal">
	    <primary>drbdsetup</primary>

	    <secondary>rename-resource</secondary>
	  </indexterm>

	  <para>Change the name of <option>resource</option> to
	  <option>new_name</option> on the local node. Note that, since there
	  is no concept of resource names in DRBD's network protocol, it is
	  technically possible to have different names for a resource on
	  different nodes. However, it is strongly recommended to issue the
	  same <option>rename-resource</option> command on all nodes to have
	  consistent naming across the cluster.</para>
	  <para>A <option>rename</option> event will be issued on the
	  <option>events2</option> stream to notify users of the new name.</para>
	</listitem>

      </varlistentry>
    </variablelist>
  </refsect1>

  <refsect1>
    <title>Examples</title>

    <para>Please see the <ulink
	url="http://www.drbd.org/users-guide/"><citetitle>DRBD User's Guide</citetitle></ulink>
      for examples.</para>

  </refsect1>

  <refsect1>
    <title>Version</title>

    <simpara>This document was revised for version 9.0.0 of the DRBD distribution.</simpara>
  </refsect1>

  <refsect1>
    <title>Author</title>

    <simpara>Written by Philipp Reisner <email>philipp.reisner@linbit.com</email> and Lars
    Ellenberg <email>lars.ellenberg@linbit.com</email>.</simpara>
  </refsect1>

  <refsect1>
    <title>Reporting Bugs</title>

    <simpara>Report bugs to <email>drbd-user@lists.linbit.com</email>.</simpara>
  </refsect1>

  <refsect1>
    <title>Copyright</title>

    <simpara>Copyright 2001-2018 LINBIT Information Technologies, Philipp Reisner, Lars Ellenberg.
    This is free software; see the source for copying conditions. There is NO warranty; not even
    for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.</simpara>
  </refsect1>

  <refsect1>
    <title>See Also</title>

    <para><citerefentry>
        <refentrytitle>drbd.conf</refentrytitle>

        <manvolnum>5</manvolnum>
      </citerefentry>, <citerefentry>
        <refentrytitle>drbd</refentrytitle>

        <manvolnum>8</manvolnum>
      </citerefentry>, <citerefentry>
        <refentrytitle>drbdadm</refentrytitle>

        <manvolnum>8</manvolnum>
      </citerefentry>,
      <ulink url="http://www.drbd.org/users-guide/"><citetitle>DRBD User's Guide</citetitle></ulink>,
       <ulink url="http://www.drbd.org/"><citetitle>DRBD Web Site</citetitle></ulink></para>

  </refsect1>
</refentry>
