Ultimate Linux Guide to Transparent RAID

Welcome to Transparent RAID for the Linux Platform

Transparent RAID on Linux follows the same approach as Transparent RAID on Windows. The engine, service brokerage, and task facilities are effectively the same. In fact, more than 80% of the code base is shared between the implementations with less than 20% being platform specific code. The core differences are in the system configurations and kernel integration. From an end-user perspective though, the differences will be minimal if not absolutely trivial.
Since Transparent RAID on Windows is fairly well documented, it is a better strategy to have an ultimate guide to detail of the specific of the Linux port.

*Support for additional distributions might be added later based on the number of requests for a given distro.
*The Web UI can be installed on a 32 bit or 64 bit system. Only the host needs to be installed on a 64 bit system.
**Ubuntu 15.04 with its kernel 3.19.0-25-generic as well as older releases updated to kernel 3.19.0-25-generic will also work, but not tested.
For those wishing to boot from a USB thumb drive, the following guides might be of use:

The installation manager simplifies the installation, service configuration, and update of your installation of Transparent RAID and its dependencies.
Both the Web UI and Host packages will be installed as services, which will auto start on system reboot.
Further, the services can be started, stopped, and restarted by running (*):

sudo service traid [start|starthost|startclient]

sudo service traid [stop|stophost|stopclient]

sudo service traid [restart|restarthost|restartclient]

*The kernel tRAID engine is unaffected by these operations, which means that it is okay to update/uninstall while the array is online.
Furthermore, you must reboot after updating the host for the changes to take effect in the kernel.

If you do not wish to use the installation manager, follow this section to do a custom installation of the Web UI client package.
Although the Web UI is compiled and available to be installed on each platform (Windows and Linux), only one install is needed to manage any number of hosts and regardless of the platform the host is running on. That is, the Web UI installed on either Windows or Linux can manage both Windows and Linux hosts. As such, the host can be running on one platform and the Web UI can be running on another platform. Obviously, for users with a single host, the Web UI service would be installed along side of the host services for simplicity.
Installing the Web UI on Linux:
The Web UI package for Linux is a 32 bit installation. On a 64 bit system, some dependencies need to be installed before the installation of the Web UI as shown below.
Ubuntu:
sudo dpkg --add-architecture i386
sudo apt-get update
sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386
chmod a+x NZFS-TRAID-CLIENT-*.bin
./NZFS-TRAID-CLIENT-*.bin

Notes to Ubuntu users:
If running into issues installing the Web Client with an error that reads "NZFS-TRAID-CLIENT-1.0_final_XXX.bin not found", this means that you have unresolved dependencies that only aptitude can resolve.
Please follow the steps below to resolve that issue.
Run:
1. sudo apt-get install aptitude
2. sudo aptitude install libc6:i386 libncurses5:i386 libstdc++6:i386
3. When prompted to leave the installations unchanged, select no by typing "n".
4. When then prompted to downgrade the presented modules, select yes by typing "y"
5. Run the Transparent RAID installation manager and choose to uninstall the Web UI
6. Run the Trasparent RAID installation manager again, but this time choose to install the Web UI

If you do not wish to use the installation manager, follow this section to do a custom installation of the host package.
Transparent RAID for Linux does not require installation or uninstallation as it comes in a portable package. Installing is as easy as extracting the zip package to any directory and starting the service broker from there. Similarly, uninstalling is as easy as deleting the extracted folder. There are no configuration files, and the service broker takes care of all system configurations, which are all dynamic. The service broker also loads and unloads the drivers on demand, and the drivers clean up after themselves when unloaded such as to leave no trace of their presence.
This portability means that the host package can be added to a Live CD or placed on a thumb drive or integrated in a truly flexible fashion.
Extracted host package (two dynamic drivers and a service broker executable):Ensure that the service broker is executable:

Preparing all disks to be used as DRUs (data disks)

Do not use your OS's default Disk Manager!

It is highly recommended that the Linux OS's default Disk Manager NOT be used for formatting and partitioning disks. Instead use gparted. This is because on most Linux distributions, the default Disk Manager uses fdisk for the tasks, and fdisk does not properly support GPT disks. The GPT partition format is required for disks larger than 2TB. Even if your disks are MBR formatted, it is simply better to use gparted as it is the better tool.
If gparted is not installed on your system, please install it by issuing at a shell prompt: sudo apt-get install gparted.
Using a wrongly formatted GPT disk is not just an issue for tRAID usage, it is an issue period and even when tRAID is not involved.

Formatting or verifying the formatting of an existing disk

Transparent RAID supports disks with existing data on them. There is no data migration required. However, the proper partition alignment for the existing disks must be verified to conform.
In all cases:

The first partition on the disk must be at the 2048 sectors (1 MB) or greater offset

There must be at least 1 MB left free at the end of the disk (8 MB recommended)

Using the GPT patitioning format for all disks larger than 2TB (GPT and MBR are both fine for disks 2TB or under)

If any of your data disks does not conform to the above requirements, you will need to backup the data off that disk, erase and properly partition the disk as required above, and finally copy back the data on the disk.
All DRUs (data disks) must be formatted with at least one partition on them. Also, do label your partitions for easy identification later.
All PPUs (parity disks) must be unformatted an must NOT have any partition on them, but should be initialized with a partition table (GPT or MBR).
All this is technically not required as you could later format the disks through the transparent disks tRAID presents you, but the Web UI enforces this requirement in order to help users differentiate between their data disks and disks to be used as parity disks.
As previously mentioned, you can verify the partition alignment of existing disks in gparted.
You should also format new disks using gparted as shown below.
Formatting and partitioning are two separate actions. You can view the formatting of your disk (whether it is GPT or MBR) by selecting View -> Device Information in gparted.
To view the information on an existing partition, go to Partition -> Information.
In particular, ensure that your partitions are labeled!
If the disk is unformatted, you can format it by going through Device -> Create partition table.
To create a new partition, select Partition -> New.
Please label all partitions for easy reference later!
After, you have verified or formatted and partitioned all of your DRUs and deleted all partitions off your PPUs, you should verify that your DRUs partitions are labeled, mountable, and accessible as desired. Verify that you can read/write from/to each volume.
Important: Do not mount your partitions in fstab! All mounts should be temporary.
After verifying the mounts, please unmount all mounted volumes.

The service broker manages everything on the host. It dynamically manages the kernel drivers and takes care of all system configurations.
So, starting the service broker starts up everything else as required.
The service broker must be started as root user! It requires elevated privileges to perform many of its tasks.
If you used the installation manager, both the service broker and UI service will automatically start on boot.
To manually start them, run: sudo service traid start. This single command will start them both as needed.
If you did a custom installation, you can manually start the service broker through the shell by changing to the executable's directory and issuing: ./NZFSB &
Similarly, you can manually start a custom Web UI installation by changing to its installation directory and issuing: ./nzfsui &

Changes to your system

The thing you will notice when you create and initialize your array is that all of the data partitions for the disks included in the RAID will go missing from your system. This is normal. These partitions will be added under the tRAID disks to be created when the array is brought online.

Revision Differences

26 August, 2015 @ 10:11

Current Revision

Content

<h1>Welcome to Transparent RAID for the Linux Platform</h1>

<h1>Welcome to Transparent RAID for the Linux Platform</h1>

Transparent RAID on Linux follows the same approach as Transparent RAID on Windows. The engine, service brokerage, and task facilities are effectively the same. In fact, more than 80% of the code base is shared between the implementations with less than 20% being platform specific code. The core differences are in the system configurations and kernel integration. From an end-user perspective though, the differences will be minimal if not absolutely trivial.

Transparent RAID on Linux follows the same approach as Transparent RAID on Windows. The engine, service brokerage, and task facilities are effectively the same. In fact, more than 80% of the code base is shared between the implementations with less than 20% being platform specific code. The core differences are in the system configurations and kernel integration. From an end-user perspective though, the differences will be minimal if not absolutely trivial.

Since Transparent RAID on Windows is fairly well documented, it is a better strategy to have an ultimate guide to detail of the specific of the Linux port.

Since Transparent RAID on Windows is fairly well documented, it is a better strategy to have an ultimate guide to detail of the specific of the Linux port.

If you do not wish to use the installation manager, follow this section to do a custom installation of the Web UI client package.

If you do not wish to use the installation manager, follow this section to do a custom installation of the Web UI client package.

Although the Web UI is compiled and available to be installed on each platform (Windows and Linux), only one install is needed to manage any number of hosts and regardless of the platform the host is running on. That is, the Web UI installed on either Windows or Linux can manage both Windows and Linux hosts. As such, the host can be running on one platform and the Web UI can be running on another platform. Obviously, for users with a single host, the Web UI service would be installed along side of the host services for simplicity.

Although the Web UI is compiled and available to be installed on each platform (Windows and Linux), only one install is needed to manage any number of hosts and regardless of the platform the host is running on. That is, the Web UI installed on either Windows or Linux can manage both Windows and Linux hosts. As such, the host can be running on one platform and the Web UI can be running on another platform. Obviously, for users with a single host, the Web UI service would be installed along side of the host services for simplicity.

<strong><u>Installing the Web UI on Linux:</u></strong>

<strong><u>Installing the Web UI on Linux:</u></strong>

The Web UI package for Linux is a 32 bit installation. On a 64 bit system, some dependencies need to be installed before the installation of the Web UI as shown below.

The Web UI package for Linux is a 32 bit installation. On a 64 bit system, some dependencies need to be installed before the installation of the Web UI as shown below.

<strong>Ubuntu:</strong>

<strong>Ubuntu:</strong>

<code>

<code>

sudo dpkg --add-architecture i386

sudo dpkg --add-architecture i386

sudo apt-get update

sudo apt-get update

sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386

sudo apt-get install libc6:i386 libncurses5:i386 libstdc++6:i386

chmod a+x NZFS-TRAID-CLIENT-*.bin

chmod a+x NZFS-TRAID-CLIENT-*.bin

./NZFS-TRAID- CLIENT-*.bin

./NZFS-TRAID- CLIENT-*.bin

</code>

</code>

+

<blockquote>

+

<strong><u>Notes to Ubuntu users:</u></strong>

+

If running into issues installing the Web Client with an error that reads "NZFS-TRAID-CLIENT- 1.0_final_XXX.bin not found", this means that you have unresolved dependencies that only aptitude can resolve.

+

Please follow the steps below to resolve that issue.

+

Run:

+

1. sudo apt-get install aptitude

+

2. sudo aptitude install libc6:i386 libncurses5:i386 libstdc++6:i386

+

3. When prompted to leave the installations unchanged, select no by typing "n".

+

4. When then prompted to downgrade the presented modules, select yes by typing "y"

+

5. Run the Transparent RAID installation manager and choose to uninstall the Web UI

+

6. Run the Trasparent RAID installation manager again, but this time choose to install the Web UI

+

</blockquote>

<strong>RHEL/ Centos:</strong>

<strong>RHEL/ Centos:</strong>

<code>

<code>

sudo yum -y groupinstall "Compatibility libraries"

sudo yum -y groupinstall "Compatibility libraries"

sudo yum -y install glibc.i686

sudo yum -y install glibc.i686

sudo yum -y install libgcc.i686

sudo yum -y install libgcc.i686

sudo yum -y install libstdc++.i686

sudo yum -y install libstdc++.i686

chmod a+x NZFS-TRAID-CLIENT-*.bin

chmod a+x NZFS-TRAID-CLIENT-*.bin

./NZFS-TRAID- CLIENT-*.bin

./NZFS-TRAID- CLIENT-*.bin

</code>

</code>

<h1><a id="host">Transparent RAID Host - Custom Installation</a></h1>

<h1><a id="host">Transparent RAID Host - Custom Installation</a></h1>

If you do not wish to use the installation manager, follow this section to do a custom installation of the host package.

If you do not wish to use the installation manager, follow this section to do a custom installation of the host package.

Transparent RAID for Linux does not require installation or uninstallation as it comes in a portable package. Installing is as easy as extracting the zip package to any directory and starting the service broker from there. Similarly, uninstalling is as easy as deleting the extracted folder. There are no configuration files, and the service broker takes care of all system configurations, which are all dynamic. The service broker also loads and unloads the drivers on demand, and the drivers clean up after themselves when unloaded such as to leave no trace of their presence.

Transparent RAID for Linux does not require installation or uninstallation as it comes in a portable package. Installing is as easy as extracting the zip package to any directory and starting the service broker from there. Similarly, uninstalling is as easy as deleting the extracted folder. There are no configuration files, and the service broker takes care of all system configurations, which are all dynamic. The service broker also loads and unloads the drivers on demand, and the drivers clean up after themselves when unloaded such as to leave no trace of their presence.

This portability means that the host package can be added to a Live CD or placed on a thumb drive or integrated in a truly flexible fashion.

This portability means that the host package can be added to a Live CD or placed on a thumb drive or integrated in a truly flexible fashion.

It is highly recommended that the Linux OS's default Disk Manager <strong>NOT</strong> be used for formatting and partitioning disks. Instead use gparted. This is because on most Linux distributions, the default Disk Manager uses fdisk for the tasks, and fdisk does not properly support GPT disks. The GPT partition format is required for disks larger than 2TB. Even if your disks are MBR formatted, it is simply better to use gparted as it is the better tool.

It is highly recommended that the Linux OS's default Disk Manager <strong>NOT</strong> be used for formatting and partitioning disks. Instead use gparted. This is because on most Linux distributions, the default Disk Manager uses fdisk for the tasks, and fdisk does not properly support GPT disks. The GPT partition format is required for disks larger than 2TB. Even if your disks are MBR formatted, it is simply better to use gparted as it is the better tool.

If gparted is not installed on your system, please install it by issuing at a shell prompt: <strong>sudo apt-get install gparted</strong>.

If gparted is not installed on your system, please install it by issuing at a shell prompt: <strong>sudo apt-get install gparted</strong>.

Using a wrongly formatted GPT disk is not just an issue for tRAID usage, it is an issue period and even when tRAID is not involved.

Using a wrongly formatted GPT disk is not just an issue for tRAID usage, it is an issue period and even when tRAID is not involved.

<h3><strong>Formatting or verifying the formatting of an existing disk</strong></h3>

<h3><strong>Formatting or verifying the formatting of an existing disk</strong></h3>

Transparent RAID supports disks with existing data on them. There is no data migration required. However, the proper partition alignment for the existing disks must be verified to conform.

Transparent RAID supports disks with existing data on them. There is no data migration required. However, the proper partition alignment for the existing disks must be verified to conform.

In all cases:

In all cases:

<ul>

<ul>

<li>The first partition on the disk must be at the 2048 sectors (1 MB) or greater offset</li>

<li>The first partition on the disk must be at the 2048 sectors (1 MB) or greater offset</li>

<li>There must be at least 1 MB left free at the end of the disk (8 MB recommended)</li>

<li>There must be at least 1 MB left free at the end of the disk (8 MB recommended)</li>

<li>Using the GPT patitioning format for all disks larger than 2TB (GPT and MBR are both fine for disks 2TB or under)</li>

<li>Using the GPT patitioning format for all disks larger than 2TB (GPT and MBR are both fine for disks 2TB or under)</li>

</ul>

</ul>

If any of your data disks does not conform to the above requirements, you will need to backup the data off that disk, erase and properly partition the disk as required above, and finally copy back the data on the disk.

If any of your data disks does not conform to the above requirements, you will need to backup the data off that disk, erase and properly partition the disk as required above, and finally copy back the data on the disk.

All DRUs (data disks) must be formatted with at least one partition on them. <strong><u>Also, do label your partitions for easy identification later.</u></strong>

All DRUs (data disks) must be formatted with at least one partition on them. <strong><u>Also, do label your partitions for easy identification later.</u></strong>

All PPUs (parity disks) must be unformatted an must NOT have any partition on them, <u>but should be initialized with a partition table (GPT or MBR)</u>.

All PPUs (parity disks) must be unformatted an must NOT have any partition on them, <u>but should be initialized with a partition table (GPT or MBR)</u>.

All this is technically not required as you could later format the disks through the transparent disks tRAID presents you, but the Web UI enforces this requirement in order to help users differentiate between their data disks and disks to be used as parity disks.

All this is technically not required as you could later format the disks through the transparent disks tRAID presents you, but the Web UI enforces this requirement in order to help users differentiate between their data disks and disks to be used as parity disks.

As previously mentioned, you can verify the partition alignment of existing disks in gparted.

As previously mentioned, you can verify the partition alignment of existing disks in gparted.

You should also format new disks using gparted as shown below.

You should also format new disks using gparted as shown below.

Formatting and partitioning are two separate actions. You can view the formatting of your disk (whether it is GPT or MBR) by selecting <strong>View -> Device Information</strong> in gparted.

Formatting and partitioning are two separate actions. You can view the formatting of your disk (whether it is GPT or MBR) by selecting <strong>View -> Device Information</strong> in gparted.

To view the information on an existing partition, go to <strong>Partition -> Information</strong>.

To view the information on an existing partition, go to <strong>Partition -> Information</strong>.

After, you have verified or formatted and partitioned all of your DRUs and deleted all partitions off your PPUs, you should verify that your DRUs partitions are mountable and accessible as desired. Verify that you can read/write from/to each volume.

+

After, you have verified or formatted and partitioned all of your DRUs and deleted all partitions off your PPUs, you should verify that your DRUs partitions are labeled, mountable, and accessible as desired. Verify that you can read/write from/to each volume.

When not using the Express Wizard, disk registration is done essentially the same as documented for the Windows platform: <a href="http:// wiki.flexraid.com/2013/06/ 27/preparing- registering- disks-for-transparent-raid/ #disk-registration">http: //wiki.flexraid.com/2013/ 06/27/preparing- registering- disks-for-transparent-raid/ #disk-registration</a>

When not using the Express Wizard, disk registration is done essentially the same as documented for the Windows platform: <a href="http:// wiki.flexraid.com/2013/06/ 27/preparing- registering- disks-for-transparent-raid/ #disk-registration">http: //wiki.flexraid.com/2013/ 06/27/preparing- registering- disks-for-transparent-raid/ #disk-registration</a>

The thing you will notice when you create and initialize your array is that all of the data partitions for the disks included in the RAID will go missing from your system. This is normal. These partitions will be added under the tRAID disks to be created when the array is brought online.

The thing you will notice when you create and initialize your array is that all of the data partitions for the disks included in the RAID will go missing from your system. This is normal. These partitions will be added under the tRAID disks to be created when the array is brought online.

6 Responses to “Ultimate Linux Guide to Transparent RAID”

So I’d like try out T-RAID, but my server is running Arch Linux (Manjaro), and I don’t really want to switch. When I try running the Installation Manager, I get “Unable to detect OS!”. I would try installing manually, but I don’t know where to get the host package in order to do the manual install. What are my options?

Managed to grab all the files for a manual install (on my Arch server). Ran NZFSB and it aborted with “insmod: ERROR: could not insert module frouter.ko: Invalid module format” for both .ko files. Is there anything I can do to get this running?

Followed the auto installer and when I loaded the WebUI, it just saying Invalid login? This is a fresh install so no login was set. I am attempting to login with nothing in both User & Password at http://localhost:8080/#.