Debian - Data replication on 2 servers with clustering and service balancing

When you host data on a server and/or offering services hosted on one or more servers, it's important that these data and/or services be available even in the event of a server crashing. To prevent the service or important data from becoming inaccessible, high availability is often set up.

In the case of a Linux server, we can provide high availability for almost any service (web, FTP, NFS, iSCSI, ...) thanks to DRBD and Heartbeat.

Note that Heartbeat is limited to 2 servers. To create a cluster with more than 2 servers, you will need to use Pacemaker. But keep in mind that Heartbeat is widely used for its performance, flexibility and stability.

1. What is DRBD

DRBD means Distributed Replicated Block Device and allows you to create a sort of RAID 1 over the network. It's DRBD that will allow us to create a real-time copy of a hard drive, a partition or a LVM system, on another server. This synchronization will be carried out in real time over the network and it is therefore recommended (but not mandatory) to use a dedicated network between the 2 concerned servers (the main server and the backup server).

Thanks to DRBD, our data will be accessible from 2 servers. If a server fails, data will always be accessible from the other server. Nevertheless, something is missing so that the service is always accessible by your users. This is what we will see in the next step. (Heartbeat)

2. What is Heartbeat

Heartbeat is a program that monitors the availability of a server and when one of them fails, services are automatically switched to the other server. To perform this check, the 2 servers will contact each other over the network at regular intervals.

In summary, if server 1 is faulty, the user can still access the same services on server 2. In addition, to make this transparent to your users, you will need to define a virtual IP address with Heartbeat.

Finally, be aware that it is highly recommended to use at least 2 different paths to verify that another server is still available. This ensures that the server is failing. Indeed, if a network card fails, heartbeat will always know the state of the server via the other network card. Otherwise, Heartbeat would have mistakenly considered that the other server was failing.

3. Configuration used

2 servers under Debian 7.7.0 x64 with : - 2 hard drives for each server (1 for Debian and the other for the data to be replicated) - 2 network cards for each server (1 for access from the outside and the other dedicated to data replication)

As mentioned earlier, the use of 2 network cards will allow you to have : - a network card for external access by your users - a dedicated network for data replication. Replication will therefore be faster and will not affect network performance for user access. To connect your two servers to each other, you can use a router, a switch or even a crossover cable (connected directly to the two eth1 network cards (see below) of each server).

In our case, these 2 servers will have 2 network interfaces : - eth0 : 10.0.0.31 for server 1 and 10.0.0.32 for server 2. This is the network card connected to the router through which users will access our services (webserver, ...) - eth1 : 192.168.1.31 for server 1 and 192.168.1.32 for server 2. This is the network adapter connected to another router or switch (since the servers will have static IPs, the DHCP server will be unnecessary in this case)

In our case, we have a dedicated network between our 2 servers through the switch and the eth1 interfaces of our 2 servers.

4. DRBD (Data replication)

IMPORTANT: the indicated commands must always be executed on your 2 servers (unless stated otherwise), at the same time (otherwise you may get error messages).

4.1. Installation

Before starting, we will create a partition on the 2nd hard disk (sdb) of each server. To do this, type this on the 2 servers :

Bash

fdisk /dev/sdb

To create a partition on the entire hard disk, type "n" (new partition), then "p" (primary), then leave the default values for the rest. When prompting you for a command again, type "w" (write the partition table).

4.2. Resource configuration

In our case, we will create a "drbd0.res" file. Note : this file must be identical on both servers.

Bash

vi /etc/drbd.d/drbd0.res

In this new file, we will paste this. Note that : - iw0 is the name of the resource you will need later to manage this resource. - server1 and server2 are the hostnames of our two Debian servers. - 192.168.1.31 and 192.168.1.32 correspond to the IP addresses of our 2 Debian servers on the dedicated network (the switch). - "device /dev/drbd0" is the "disk" provided by DRBD - "disk /dev/sdb1;" is the partition 1 (sdb1) of our "sdb" hard disk. - "syncer" allows you to limit the synchronization speed between the 2 servers. (optional)

On both servers, check and modify the name of your server if necessary by modifying these files : - "/etc/hostname" : contains only the name of your Linux server - "/etc/hosts" : contains the correspondences between the name of your Linux server and its IP address..

Bash

vi /etc/hostname

Note : if you change the name of your Linux server, it's best to restart it completely.

To avoid problems, add this to the "/etc/hosts" file of your 2 servers.

Bash

vi /etc/hosts

Plain Text

192.168.1.31 server1
192.168.1.32 server2

4.3. Enabling data replication

To enable the data replication, use these commands. Note : iw0 is the name of the resource we just defined in the file "drbd0.res".

Bash

drbdadm create-md iw0
drbdadm up iw0

If you use the "drbd-overview" command, you should see that your 2 servers are currently connected and that they are all 2 in "secondary". So, the replication doesn't occur because DRBD doesn't know in which direction it should synchronize your data.

Now, DRBD knows that server1 is the primary server and the other is the secondary server (the one that will be used in the event of failure of the main server). Note : if you need to iron the master server in secondary, here is the command to use :

Bash

drbdadm secondary r0

If you type the "drbd-overview" command, you will see that DRBD begins data replication from the primary server to the secondary server. This may take time depending on the use of your network and the size of the "disk" to replicate.

Now, our data will always be available from 2 servers. Finally, we need to create a cluster with these 2 servers so that our services are accessible with a single IP and that the services are always available, even if one of the 2 servers breaks down.