HA Linux Cluster On RackSpace Cloud Servers

Our goal is to setup a pair of RackSpace Cloud Servers in a redundant cluster using a shared IP address. We’ll use the “heartbeat” package from Linux-HA (http://www.linux-ha.org) for the cluster messaging layer and “pacemaker” package from ClusterLabs (http://clusterlabs.org) for the cluster resource manager.

Before starting this procedure you’ll need to:

a. Create the two cloud servers. These instructions are specific to CentOS for the operating system.

b. Open a ticket with RackSpace Cloud support and request a public IP address to be shared between the servers.

You can use the instructions for other situations but you’ll need to make the appropriate adjustments.

1. Setup hosts file entries. On each server, edit /etc/hosts and add entries for each servers public and private interfaces. You’ll also find it convenient to setup ssh keys between the servers for easy access.

Note: Several of these packages are not available on the standard RHEL yum channels. If you’re working on something other then a RackSpace Cloud server then you might need to install the EPEL channel. Just go to:

Where [SHARED_IP] is the IP address to be shared between the servers and [HOST1] is the hostname of the primary server.

Once this is done then you should be able to monitor the status of the cluster from either node using the crm_mon command. You’ll get output like this:

1

2

3

4

5

6

7

8

9

10

11

12

============

Last updated:Sun Feb614:00:422011

Stack:Heartbeat

Current DC:node01(cad6f81e-f772-4add-b5e2-c9a78b4ae430)-partition with quorum

Version:1.0.10-da7075976b5ff0bee71074385f8fd02f296ec8a3

2Nodes configured,2expected votes

1Resources configured.

============

Online:[node02 node01]

shared_ip_one(ocf::heartbeat:IPaddr):Started node01

8. Next step is to test failover on the servers.

a. Run crm_mon on the second server.

b. Reboot the first server:

1

/sbin/reboot

c. Monitor the second server and notice that when the first goes offline, the “shared_one_ip” is switched to the second server. After the first server finishes rebooting then you should see it come back online and “shared_one_ip” return to it’s original location on the first server.

d. Repeat this test but reboot the second server and monitor the first.

And that completes the setup process. You now have an HA Linux cluster on the cloud!

Won’t rackspace try to sell the load balancing service instead? You suggest opening a ticket for ‘public IP address to be shared between the servers” . Would they know what to do?

Anonymous

RackSpace just recently released the new Load Balancer service. We’ve been using it for several months in private beta for some of our clients. It’s a great service and can be used in place of building your own load balancer. For most simple cases the load balancer service will be a perfect solution. But for cases with special requirements a custom built load balancer may still be appropriate.

RackSpace Cloud support definitely knows how to setup a shared IP address. We’ve made the request dozens of times with out a problem.

Yes, RackSpace can allocate extra public IP addresses that can be shared between multiple servers.

codex73

Isn’t a load balancer different in nature than a HA? Won’t we need both to maintain proper load distribution (dns) and/or HA? I’m studying how to build a high traffic architecture (initial) with low small effort. So far some kind of Load Balancer(distributes requests), HA Web Servers (file system) and MYSQL cluster. What order goes to what it’s still the challenge. One more thing I’ve researched is using Nginx which caches better than others aparently.

http://reliablepenguin.com Lee Blakely

Yes load balancer and high availability are different goals. In a cluster of webheads, the load balancer monitors the webheads and stops sending request to servers that are down or overloaded. So in this case the LB and HA are handled by the same device. In a fileserver or mysql cluster LB and HA are separate functions. This set of articles is focused on HA/LB configurations for MySQL. The HA is provided by HA Linux. The LB is provided by HA Proxy.

We use both HA Proxy and Nginx. Nginx is a great reverse proxy and cache. The ability to define custom rules is really powerful. We use HA Proxy for cases where we don’t need a cache or we’re proxying non-HTTP protocols like MySQL. In a couple of cases we’ve used both – HA Proxy for load balancing and Nginx for content caching.

Seveman

In my experience, you should NOT use rackspacecloud for anything more than small workhorse-type servers that will require little setup work and keep no valuable information. I was running a server and somehow they lost my server images and were not able to recover it from the backup file in their own cloud files storage after 5 days. No way to upload a backup copy of your server images and techically unknowledgeable tech support so in case of what they called a “bug” you’re out of anything you had in your servers

Tarik Jabri

Interesting article… is there any way to do the same except share a private IP address? I’d like to setup HA servers for private infrastructure that will not be accessible via the Internet (only used be other nodes within the cloud). Is this possible?

http://reliablepenguin.com Lee Blakely

As of a few weeks ago, RS was not willing to provision private shared addresses. This will probably change in the near future. Best you can do right now I think is just configure IPTable to only accept traffic from known addresses.