Main sites

Editing Help

Please note that all the SIEpedia's articles address specific issues or questions raised by IAC users, so they do not attempt to be rigorous or exhaustive, and may or may not be useful or applicable in different or more general contexts.

Step by step guide for the installation and configuration of a cluster to run ParaView

Introduction

As part of a student project that I was mentoring, I built a small ParaView server (4 nodes, 16 CPUs). Since the ParaView documentation didn't seem to have a step-by-step guide on how to do it, I put together the notes of what I did. I'm sure many things could be done better, and I would appreciate any comments on how to improve things. Some of the decisions taken for the installation of our cluster reflect the type of cluster that we wanted to have, and therefore the guide is certainly not comprehensive, but I put it here hoping that it can be useful to someone else. Certainly, some issues (like X connections) seem to be a common source of errors, and the way we solved it can probably be useful to others.

Our test cluster is made up of four x86_64 nodes (each with four CPUs) connected via a Gigabit switch, and the frontend has two network cards. eth1 is connected to the outside world, while eth0 (as per the other nodes) is connected to the switch. In order to make the installation of the cluster easier, I decided to use the Rockscluster kit, in order to have the basic cluster up and running as quickly as possible (although whether you use other cluster kit or even build the basic cluster manually should not matter much for the installation of ParaView).

Installation of the basic cluster, with Rocks 5.1

The latest version of Rocks is 5.2, but we needed to have the PVFS2 parallel file system installed, and this version does not have (at the time I did the installation) the PVFS2 roll (a pre-packaged and ready-to-install bundle), so we went for Rocks 5.1 instead. The installation of Rocks to create the basic cluster is very easy, and only a few steps are needed.

You can find the Rocks 5.1 documentation here, and first you should install the frontend, following this. For the ParaView cluster, when asked, we select the following rolls to install: base, ganglia, hpc, kernel, os, web-server, and pvfs2 (for this you will need to swap DVDs). This is very similar to a standard distribution install, so you should decide on which partitioning you want (I selected autopartition), network details, keyboard layout, etc.

Once the installation of the frontend is finished, when you open for the first time a terminal, you will see that the ssh keys are created automatically. Verify that the network works OK, and before proceeding further, let's make a small change to make sure that the compute nodes will have an X server running (not usual in a computational cluster, but necessary to run ParaView). For this, we follow these instructions.

With this, the compute nodes will have all the necessary stuff to run ParaView, so we can proceed with their installation. For this, we follow these instructions. We run insert-ethers as root in the frontend, and proceed as per the mentioned instructions, although when asked about appliance type, we select Compute-PVFS2 I/O, which means that our nodes will be both computing nodes, and part of the PVFS2 file system (in a production cluster, these roles would probably belong to different nodes, but for our test cluster, this is OK). Installing everything does take a while, and the command rocks-console is useful, but the VNC connections are not always available, so you should also verify the status of the installations with ping, ssh, etc.

When the compute nodes have been installed, it is time to configure PVFS2, and for this we just need to do the step mentioned here, which basically is just running rocks sync pvfs. With this in place, we can now verify that PVFS2 works fine:

The Rocks 5.1 installation comes with a web server, with Ganglia and some documentation, but the iptables rules as configured by Rocks are too restrictive for our setting. The file /etc/sysconfig/iptables had lines for https and for www with a --source option, which we comment out, and copy without it (our server is in an internal network behind a firewall, so we can allow unrestricted access). With this, we can monitor our server.

Next step is create regular users, which is very easy. We just create a normal user in the frontend with the command useradd and then synchronize to all nodes with the command rocks sync users (the first time I run it, it complained about clock skews, which was solved by waiting some time, in the order of one or two hours. The second time I installed the cluster I didn't have this problem).

We are basically done with the basic cluster installation. We just need to verify that we can run (as a regular user) MPI programs. In the regular user account created above, we create a file named machines and try a basic MPI program.

Running ParaView

Depending on how you want to run ParaView, different settings could be possible. Our goal was to have the cluster only accessible to users via ssh. Some of the steps below could be easier if we left an X session open by one user, but we didn't like this idea very much, so the following will work assuming that no user is logged-in in any of the nodes in the cluster (which is convenient, since this is just the state you would get after rebooting. If you have a root session open in the frontend, you could log out and work remotely from now on).

The ParaView wiki tells you that configuring properly X connections is one of the most common problems with setting up a ParaView server. The way we solved it was with four basic scripts in /share/apps:

grant.X.access will copy the appropriate magic cookies to the given user, so that he/she will be able to open an X application in the local monitor of one of the nodes. remove.X.access will just remove the magic cookie for that monitor. rocks-grant will get all the magic cookies for all the nodes and rocks-remove will delete them. Ideally you would like to write a script that will run in all the nodes of your cluster (without hardcoding their names as in the examples above, but this was just convenient for our small cluster). In any case, you should run the scripts grant.X.access or remove.X.access sequentially in each node and not in parallel, to avoid problems when locking the .Xauthority file.

Now it is very easy to add or remove permissions (to open X applications in the local monitors) for a given user (and we could easily integrate the granting and removing of permissions with a queuing system):

We are almost there, and now we just need to do the final test, in which we will have a ParaView client connected to the ParaView server in the cluster. In the server we run ParaView in parallel (make sure that the name of the frontend is the first one in the machines file):

We now have to define the server to connect (File -> Connect), and we specify a new connection, with Host being the name of your server, and Startup Type set (for the moment) to Manual. We click to connect, and then in the ParaView server we see the last line appearing, showing a partial success.

We try rendering a sphere in the ParaView client GUI (Sources -> Sphere, and we select Theta Resolution and Phi Resolution to 1024), but after clicking Apply we only see a quarter of the sphere and some error messages appear in the server terminal:

With this in place, we can rocks-grant permission to our regular user again, restart the server and the client, and now when rendering the sphere as for the steps above, there are no errors, and everything works as expected!!