Striping Across Four Storage Nodes With GlusterFS 3.2.x On CentOS 6.3

This tutorial shows how to do data striping (segmentation of logically sequential data, such as a single file, so that segments can be assigned to multiple physical devices in a round-robin fashion and thus written concurrently) across four single storage servers (running CentOS 6.3) with GlusterFS. The client system (CentOS 6.3 as well) will be able to access the storage as if it was a local filesystem.
GlusterFS is a clustered file-system capable of scaling to several peta-bytes. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. Storage bricks can be made of any commodity hardware such as x86_64 servers with SATA-II RAID and Infiniband HBA.

Please note that this kind of storage doesn't provide any high-availability/fault tolerance features, as would be the case with replicated storage.

I do not issue any guarantee that this will work for you!

1 Preliminary Note

In this tutorial I use five systems, four servers and a client:

server1.example.com: IP address 192.168.0.100 (server)

server2.example.com: IP address 192.168.0.101 (server)

server3.example.com: IP address 192.168.0.102 (server)

server4.example.com: IP address 192.168.0.103 (server)

client1.example.com: IP address 192.168.0.104 (client)

All five systems should be able to resolve the other systems' hostnames. If this cannot be done through DNS, you should edit the /etc/hosts file so that it looks as follows on all five systems:

If you use a firewall, ensure that TCP ports 111, 24007, 24008, 24009-(24009 + number of bricks across all volumes) are open on server1.example.com, server2.example.com, server3.example.com, and server4.example.com.

Next we must add server2.example.com, server3.example.com, and server4.example.com to the trusted storage pool (please note that I'm running all GlusterFS configuration commands from server1.example.com, but you can as well run them from server2.example.com or server3.example.com or server4.example.com because the configuration is repliacted between the GlusterFS nodes - just make sure you use the correct hostnames or IP addresses):

Next we create the striped share named testvol (please note that the number of stripes is equal to the number of servers in this case) on server1.example.com, server2.example.com, server3.example.com, and server4.example.com in the /data directory (this will be created if it doesn't exist):