What is unRAID?

unRAID® is an embedded operating system that is designed to provide you with the ultimate control over your hardware. In addition to performing the duties of a robust NAS (network-attached storage), unRAID is also capable of acting as an application server and virtual machine host. unRAID installs to and boots from a USB flash device and loads into a root RAM file system. By using a modern Linux kernel with up-to-date hardware drivers, unRAID can operate on nearly any 64 bit system (x86_64) with minimal consumption of system memory. All configuration data relating to the operating system is stored on the flash device and loaded at the same time as the operating system itself. Management of your unRAID system is accomplished through an intuitive web interface that offers basic controls for common tasks as well as advanced tuning options for the more savvy user. unRAID automatically chooses default settings that should work for most people’s needs, but also allows you to tweak settings to your liking. This makes unRAID intuitive where you want it, and tunable where you need it. By combining the benefits of both hardware and software agnosticism into a single OS, unRAID provides a wide variety of ways to store, protect, serve, and play the content you download or create.

The capabilities of unRAID are separated into three core parts: software-defined NAS, application server, and localized virtualization

Network Attached Storage

At its core, unRAID is a hardware-agnostic solution that can turn almost any 64-bit capable system into a NAS. unRAID can manage an array of drives (connected via IDE, SATA, or SAS) that vary in size, speed, brand, and filesystem. In addition, by eliminating the use of traditional RAID-based technologies, we can scale on-demand by adding more drives and without needing to rebalance existing data. unRAID's NAS functionality consists of a parity-protected array, user shares, and an optional cache pool.

Parity-Protected Array

The primary purpose of an unRAID array is to manage and protect the data of any group of drives (JBOD) by adding a dedicated parity drive. A parity drive provides a way to reconstruct all of the data from a failed drive onto a replacement. Amazing as it seems, a single parity drive can add protection for all of the others! The contents of a hard drive can be thought of as a very long stream of bits, each of which can only be a zero or a one. If you sum the values of the nth bit on every drive and determine whether that sum is even or odd, then you can force the corresponding nth parity bit to also be even or odd (zero or one). If a data drive fails, that parity information can now be used to deduce the exact bit values of the failed drive, and perfectly rebuild it on a replacement drive. Here's an example:

Figure 1. Showing bit settings on a set of disks without a parity device.

In the picture above, we have three drives and each has a stream of bits that vary in count based on the device size. By themselves, these devices are unprotected and should any of them fail, data will be lost. To protect ourselves from failure, we must add a fourth disk to serve as parity. The parity disk must be of equal or greater size than the largest data disk. To calculate the value of each bit on the parity disk, we only need to know the sum total for each column. If the sum of a column is an even number, the parity bit should be a 0. If the sum of a column is an odd number, the parity bit should be a 1. Here's the same image as before, but with parity calculated per frame:

Figure 2. Showing bit settings on a set of disks with a parity device.

Now let's pretend that drive 2 in our example has suffered a failure and a new drive has been purchased to replace it:

Figure 3. Solve for the missing bits using parity.

To rebuild the data on the newly replaced disk, we use the same method as before, but instead of solving for the parity bit, we solve for the missing bit. For column 1, the sum would be 0, an even number, so the missing bit must be a 0 as well. For column 6, the sum would be 1, an odd number, so therefore the missing bit must also be a 1.

The ability to rebuild a disk using parity provides protection from data loss. Parity protection also provides fault-tolerance by allowing full usage of the system while keeping all data accessible, even when a drive has failed.

User Shares

Unlike most RAID systems, unRAID saves data to individual drives. To simplify manageability, users can create shares that allow files written to them to be spread across multiple drives. Each share can be thought of as a top-level folder on a drive. When browsing through a share, all data from all drives that participate in that share will be displayed together. Users do not need to know which disk a file is on in order to access it under a share. Shares can be tuned to include/exclude specific disks and to use various methods for determining how files are allocated across those disks. In addition to controlling how data is distributed across drives, users can also control what network protocols the share is visible through as well as define user-level security policy. When accessing your unRAID server over a network protocol, all shares exported through that protocol will be visible, but you can toggle protocols for both individual shares as well as at a global setting level. Should you have private data on your system that you wish to protect from anonymous access, user accounts can be created and policies defined to limit access to only trusted individuals.

Figure 1. Distribution policies define the disks to use when data is written to a share.

Figure 2. Access policies define the protocols and user-level security to use for a share.

Cache

The cache-drive feature of unRAID provides faster data capture. Generally speaking, by using a cache alongside an array of three or more drives, you can achieve up to 3x write performance. When data is written to a user share that has been configured to use the cache drive, all of that data is initially written directly to the dedicated cache drive. Because this drive is not a part of the array, the write speed is unimpeded by parity updates. Then an unRAID process called “the mover” copies the data from the cache to the array at a time and frequency of your choosing (typically in the middle of the night). Once the mover completes, the space consumed previously on the cache drive is freed up.

With a single cache drive, data captured there is at risk, as a parity drive only protects the array, not the cache. However, you can build a cache with multiple drives both to increase your cache capacity as well as to add protection for that data. The grouping of multiple drives in a cache is referred to as building a cache pool. The unRAID cache pool is created through a unique twist on traditional RAID 1, using a BTRFS feature that provides both the data redundancy of RAID 1 plus the capacity expansion of RAID 0.

Application Server

Traditional NAS solutions to application support come with three primary limitations:

They cannot support applications written for other operating systems.

They can be cumbersome to install and even more difficult to remove.

They don’t always “play nice” with other applications in the same OS.

Docker addresses these problems in a number of key ways:

It allows for the use of any Linux operating system to empower a given application (no longer limited by the operating system of the host itself).

It removes the “installation” process that applications have to go through by providing pre-installed images that ensure a consistent run-time experience for the user and making them easier to remove when the user is done with them.

It enables applications that would normally have issues with coexistence to live in harmony in the same operating environment.

Docker is made up of three primary components: the Engine, the Hub, and Containers.

The Engine

The Docker Engine represents the management component that is built into unRAID 6. Using the engine, we can control application access to vital system resources, interact with the Docker Hub, and isolate applications from conflicting with each other or our operating system.

From a storage perspective, the engine leverages the copy-on-write capabilities of the BTRFS filesystem combined with Docker images provided through the hub. The images are essentially tar files with a hierarchy so that other images which depend upon a common layer don’t need to replicate storage for the layer they share. The shared layers are put in a read-only state, while changes made to them are reflected only in the instance for the application that changed it. In simpler terms, this means that applications can be efficient in their use of both system performance and storage capacity.

Figure 1. Visualizing how different apps can share read-only access to a common base image, storing modifications to it in a copy-on-write data store.

The Hub

One of biggest advantages Docker provides over both traditional Linux containers (LXCs) and virtual machines (VMs) is in its application repository: the Docker Hub. Many traditional Linux operating systems nowadays come with a component in their framework known as a package manager. The job of the package manager is to let people easily install applications written for a particular operating system from catalogs that are known as repositories. While package managers do their job fairly well, they come with all the limitations mentioned earlier. Linux containers and virtual machines, while competent at providing a way to control resources allocated to an application, still rely on traditional package managers for software retrieval and installation into their run-time environments.

In contrast, the Docker Hub provides all the benefits without the limitations of a traditional package manager. Using the Docker engine, pre-built applications can be downloaded automatically and, thanks to the copy-on-write benefits we’ve already covered, the only data that is actually downloaded is data not already present on your system. The hub contains over 14,000 Dockerized apps, so finding what you’re looking for shouldn’t be a problem. In addition, thanks to some of our loyal community members, users can quickly add many of the most popular containers through the use of templates in unRAID 6. These forum members have taken it upon themselves to build and maintain these templates and the list of available templates continues to grow.

Containers

The cornerstone of Docker is in its ability to use Linux control groups, namespace isolation, and images to create isolated execution environments in the form of Docker containers. Docker controls the resources allocated to the Containers and isolates them from conflicting with other applications on the same system. This provides all the benefits of traditional virtual machines, but with none of the overhead associated with emulating hardware, making containers ridiculously efficient and in some studies, barely distinguishable from bare-metal equivalents.

Docker works by allowing applications access to the system resources of the host operating system, such as CPU, memory, disk, and network, but isolates them into their own run-time environments. Unlike virtual machines, containers do not require hardware emulation, which eliminates overhead, hardware requirements, and provides near bare-metal performance.

Figure 2. Containers can be assigned common system resources and remain isolated from negatively impacting each other on the same system.

Virtualization Host

Virtualization technology has advanced greatly since it was first introduced and provides a wealth of benefits to users. By supporting the use of virtual machines on unRAID 6, we can run an even wider array of applications in isolated environments. While Docker containers are the preferred method for running Linux-based headless applications, virtual machines offer these unique benefits:

Run non-Linux operating systems (e.g. Windows).

Support drivers for physical devices independently of unRAID OS.

Customize and tune the guest operating systems.

unRAID Server OS is designed to run as a virtualization host, leveraging a hypervisor to partition resources to virtualized guests in a secure and isolated manner. To simplify, virtual machines can be assigned a wider array of resources than Docker containers but still offer the same benefits of isolated access to those resources. This enables unRAID servers to handle a variety of other tasks, more than just network-attached storage.

Assignable Devices

Our implementation of KVM includes modern versions of QEMU, libvirt, VFIO*, VirtIO, and VirtFS. We also support Open Virtual Machine Firmware (OVMF) which enables UEFI support for virtual machines (adding SecureBoot support as well as simplified GPU pass through support). This allows for a wide array of resources to be assigned to virtual machines ranging from the basics (storage, compute, network, and memory) to the advanced (full PCI / USB devices). We can emulate multiple machine types (i440fx and Q35), support CPU pinning, optimize for SSDs, and much more. Best of all, these virtualization technologies prevent their use from impacting the reliability of the host operating system.

Simplified Management

Management of your unRAID system is accomplished through an intuitive web interface that offers basic controls for common tasks as well as advanced tuning options for the more savvy user. unRAID automatically chooses default settings that should work for most people’s needs, but also allows you to tweak settings to your liking. This makes unRAID intuitive where you want it, and tunable where you need it.

System Requirements

In general, if it’s supported by Linux, it’s supported by unRAID. CPU usage will be minimal, so even something as low end as a Celeron or Atom processor would be fine to use. However, should you wish to run more performance-demanding applications through Docker containers or virtual machines, a processor with extra clock speed and support for hyper-threading can be very beneficial. And as mentioned before, if using virtual machines, virtualization support on your CPU will be required (Intel VT-x / AMD-V), and if passing through PCI devices, IOMMU support will also be required (Intel VT-d / AMD-Vi). For running localized virtual desktops, you will need to be able to assign a graphics device (GPU) to a VM, which will require more specific component selection for compatibility.

Boot Device

unRAID installs to and boots from a quality USB flash storage device[1]. The device must be at least 512MB, no larger than 32GB, and contain a unique GUID (serial number).

Network Attached Storage

If the sole purpose of your unRAID system is to act as a traditional NAS, system requirements are minimal:

1GB of RAM

64-bit capable processor

Linux hardware driver support

At least 1 SATA/IDE/SAS HDD

Application Server

If you intend to use your system as an application server with Docker Containers, you will need to ensure you have enough memory to support the amount of concurrency you intend for your system. Most users will find it difficult to utilize more than 8GB of RAM on Docker alone, but usage may vary from application to application. General recommendations for running an application server are as follows:

Virtualization Host

To create virtual machines on unRAID, you will need HVM hardware support (Intel VT-x or AMD-V). To assign host-based PCI devices to those VMs, your hardware must also support IOMMU (Intel VT-d or AMD-Vi). Lastly, all virtualization features must be enabled in your motherboard BIOS (typically found in the CPU or System Agent sections). NOTE: Not all hardware that claims support for this has been proven to work effectively, so see the "tested hardware" section for known working component combinations. Virtual machines can also drive a need for much more RAM/CPU cores depending on the type. Here are some general recommendations on how much RAM should be allocated per virtual machine:

Virtual servers (Windows, Arch, etc.): 256MB - 1GB, 1-2 CPU cores

Virtual desktops (Windows, Ubuntu, etc.): 512MB - 8GB, 2-4 CPU cores

Hybrid VMs (GPU assignment, gaming, etc.): 1GB - 12GB, 2-6 CPU cores

Keep in mind that memory usage for virtual machines only occurs when they are running, so it's just important to think about these requirements in terms of peak concurrent usage on your system.

Determining HVM/IOMMU Hardware Support

To determine if hardware has support for HVM or IOMMU, there are two primary methods available:

Online Research

To check if your Intel processor has support for VT-x or VT-d, visit http://ark.intel.com/Search/Advanced. On the left hand filter panel, you can filter by processors that have support for VT-x, VT-d, or both.

For guidance with AMD processors, there is not an equivalent to the ARK site, but this Wikipedia article may assist you.

Motherboard support for virtualization is usually available as part of the product documentation or user manual.

Through the unRAID webGui

When accessing your unRAID system through the web interface, you can determine if your system is virtualization compatible by clicking the Info button on the right side of the top menu bar.

HVM Support refers to Intel VT-x or AMD-V

Not Available means that your hardware is not HVM capable.

Disabled means that your hardware is HVM capable, but the settings in your motherboard BIOS are not enabled.

Enabled means that your hardware is both HVM capable and the appropriate settings in your motherboard BIOS are also enabled.

IOMMU Support refers to Intel VT-d or AMD-Vi

Not Available only displays if your system is not HVM capable.

Disabled means that either your hardware is not capable of IOMMU or the appropriate settings in your motherboard BIOS are not enabled.

Enabled means that your hardware is both IOMMU capable and the appropriate settings in your motherboard BIOS are also enabled.

Assigning Graphics Devices

Unlike other PCI devices, graphics devices can be more difficult to pass through to a VM for control. With unRAID 6, we've implemented a number of tweaks to maximize success with graphics pass through for our users. Here are the currently known limitations associated with GPU pass through on unRAID 6:

NVIDIA GTX-series GPUs should work fine as of the 600 series or newer, but not all models have been tested.

AMD cards have had some issues depending on the make or model and which guest operating system is attached.

Some devices may work better for pass through to specific guest operating systems.

With OVMF-based virtual machines, if your GPU has UEFI support, it should work fine, but some users still report card-specific issues.

More information on assigning graphics devices to VMs can be found here.

Lime Technology Tested Components

For those looking to purchase a new system for unRAID, the following components are used in Lime Technology's lab for testing features and capabilities for unRAID 6. Note that these systems do not represent the only hardware that can be used with unRAID 6, but this list does represent the limit to what the Lime Technology R&D team has access to for testing in the lab.

Must pass through the ROM file for GPU pass through to work and survive reboots.

This is NOT a recommended GPU for the issue mentioned above and limited support for OVMF.

Getting Started

So are you ready to get started? Great! You’ve come to the right place! In this guide we will be covering how to prepare your flash device, boot the system, and configure your first array. The entire process should take less than 15 minutes.

Prerequisites

Before we begin, you should have your server assembled, connected via power and Ethernet, and you should have a monitor and keyboard attached for the initial configuration (to be ready to alter configuration settings in your BIOS). Once the initial setup is complete, you can disconnect your monitor and keyboard to run unRAID in a “headless” state if you so desire. You will also need a quality USB flash device that is 512MB or larger. If you haven’t purchased your hardware yet, we have lots of recommendations.

Booting Up

You’re now ready to remove the Flash from your PC or Mac, plug it into your server, and power up. If unRAID Server OS immediately boots (with some motherboards it will), you can skip ahead to assigning devices. If it doesn’t boot, reset your server, enter the BIOS, set the system to boot from USB flash, save your BIOS settings, and try to booting again. If you are still having difficulty getting your server to boot from the flash, ensure that the Flash is the only device plugged into any of the USB ports. Also avoid using front panel USB ports in favor of ports available directly on the motherboard I/O panel. If you’ve followed these guidelines and still can’t boot, try the following adjustments in your BIOS settings:

Set the boot order to as follows: Forced-FDD, USB-HDD, USB-ZIP

Try disabling USB 2.0 support (this will default to USB 1.1).

Try switching on or off any Fast Boot feature.

Try Switching on or off USB keyboard support.

NOTE: Many motherboards support only up 12 hard drives for purposes of boot selection. This is normally not an issue for unRAID® Server OS; however, if your Flash device is recognized by the bios as a hard drive, you may not be able to boot from the Flash after installing your 12th “real” hard drive. To avoid this, if possible set up your bios so that the Flash is treated as a removable device.

Assigning Devices to the Array and Cache

Now that you’ve booted up your unRAID Server, you are ready to begin setting up your first array. The boot process shouldn’t take more than a few minutes and when completed, open a web browser from your Mac or PC and navigate to http://tower (or http://tower.local if using a Mac). The first page you will be brought to is the unRAID Main tab, where you will select the devices to assign to slots for parity, data, and cache disks. Assigning devices to unRAID is easy! Just remember these guidelines:

Always pick the largest storage device available to act as your parity device. When expanding your array in the future (adding more devices to data disk slots), you cannot assign a data disk that is larger than your parity device. For this reason, it is highly recommended to purchase the largest HDD available for use as your initial parity device, so future expansions aren’t limited to small device sizes.

Do not assign an SSD as a data/parity device. While unRAID won’t stop you from doing this, SSDs are only supported for use as cache devices due TRIM/discard and how it impacts parity protection. Using SSDs as data/parity devices is unsupported and may result in data loss at this time.

Using a cache will improve array performance. It does this by redirecting write operations to a dedicated disk (or pool of disks in unRAID 6) and moves that data to the array on a schedule that you define (by default, once per day at 3:40AM). Data written to the cache is still presented through your user shares, making use of this function completely transparent.

Creating a cache-pool adds protection for cached data. If you only assign one cache device to the system, data residing their before being moved to the array on a schedule is not protected from data loss. To ensure data remains protected at all times (both on data and cache disks), you must assign more than one device to the cache function, creating what is called a cache-pool. Cache pools can be expanded on demand, similar to the array.

SSD-based cache devices are ideal for applications and virtual machines. Apps and VMs benefit from SSDs as they can leverage their raw IO potential to perform faster when interacting with them. Use SSDs in a cache pool for the ultimate combination of functionality, performance, and protection.

NOTE: Your array will not start if you assign more devices than your license key allows.

Starting the Array and Formatting Your Devices

Once you have all your devices assigned, you can click the Start button under Array Operation. This will mount your devices and start the array. New devices added to disk or cache device slots will appear as 'Unformatted' and will be unusable until you format them. unRAID 6 defaults to using the XFS filesystem for all devices, but if you define a cache pool, BTRFS will automatically be used for those devices. To format your devices for use, you must click the check box under ‘Array Operation’ that says Format and then click the Format button.

Even before the devices are formatted, a parity sync will be performing in the background to initialize the protection of the array. Until the sync is completed, the array will operate but in an unprotected state. It is recommended to wait until the initial parity sync completes before adding data to the array.

Additional Settings

While unRAID is configured to work automatically, you may wish to further refine your setup by customizing your IP address, hostname, disk tunables, and other settings. This section goes over the various settings you can configure from the unRAID webGui. All settings controls can be found under the Settings tab on the unRAID task bar unless otherwise specified.

Date & Time

From this page you can set your time zone and toggle use of up to 3 NTP servers. It is recommended that you adjust unRAID to your time zone for accurate timekeeping.

Disk Settings

You can configure additional settings for your disk devices from this page. Enable your array to auto-start on boot, adjust disk spin down timers, and even adjust advanced driver settings such as SMART polling frequency.

Identification

unRAID automatically uses the hostname of tower, but you can adjust that from this page. You can also give your system a description / model number (useful for system builders).

Network Settings

By default, unRAID will attempt to get an IP address from a DHCP server present on your local network (typically by your router). From this page you can configure a static IP address, set up bonding / bridging, or other options. Setting a static IP is recommended, but not required to use unRAID.

Global Share Settings

As described earlier, user shares can vastly simplify how content can be organized and accessed across multiple disks in the array. You can specify what disks are allowed to participate in user shares (global inclusion/exclusion) and if a cache device/pool is present, you can configure it's use with user shares from here.

UPS Settings

unRAID can be connected to an APC UPS (uninterruptable power supply) so that in the event of a power loss, the system can be commanded to shut down while being supplied power through a battery. From this page you can configure connection to your specific UPS and define policies for when the shutdown command should be issued. For a complete manual, visit: http://apcupsd.org/manual/manual.html

AFP (Apple File Protocol)

From this page you can enable user shares for use with the Apple File Protocol, allowing them to be used as valid Time Machine backup targets for your Mac OS X devices.

NFS (Network File System)

NFSv4 support has been included in unRAID 6. You can enable or disable it's use with user shares from this page, as well as adjust the fuse_remember tunable which can help with resolving NFS Stale File Handles error messages.

SMB (Server Message Block)

The SMB protocol is the standard used by Microsoft Windows-based clients. From this page, you can enable its use, define a Windows workgroup, or even join an active directory domain.

FTP (File Transfer Protocol)

Users can connect via FTP if they are added to the FTP user(s) field on this page. If no users are added, the FTP service will not be started.

Confirmations

From here, you can disable the need for confirmations to perform various tasks.

Display Settings

Customize the appearance of the unRAID webGui from this page. This includes adjusting the date and time format, number format, toggles for tabbed/non-tabbed view modes, temperature unit, and much more.

Notifications Settings

Browser and e-mail-based system notifications can be configured from this page. You can subscribe to different types of notifications for each method and even add custom alerts for SMART values attribute monitoring.

Scheduler

The scheduler settings page allows you configure the frequency for two types of automated system tasks: parity checks and the cache mover.

Using Docker

With unRAID 6, we can now run any Linux application on unRAID, regardless of the distribution format. That means whether an app was written for Ubuntu, CentOS, Arch, Red Hat, or any other variant, unRAID can run it. This is accomplished through the use of Docker Containers, which allow us to provide each application with it’s own isolated operating environment in which it cannot create software compatibility or coexistance conflicts with other applications. This guide will show you how to get started with Docker on unRAID 6 to install media servers, file sharing software, backup solutions, gaming servers, and much more.

If you want more information on docker and its underlying technology than is provided in this guide then you should visit the docker home page.

Prerequisites

A system up and running with unRAID 6.0 and are connected via a web browser to the unRAID webGui (e.g., “http://tower” or “http://tower.local” from Mac by default).

A share created called “appdata” that will be used to store application metadata.

NOTE: Applications are made available and supported by both the Docker and unRAID user communities respectively.

Creating Your Docker Virtual Disk

The first step on your Docker journey will be to create your Docker virtual disk image where the service and all the application images will live.

Open a web browser on your Mac or PC and navigate to the unRAID webGui.

Click on the Docker tab at the top of the screen.

Set Enable Docker to Yes.

Specify an initial virtual disk image size (it is recommended that beginners start with at least a 10GB image size). This can be enlarged later, but can never be reduced in size once set.

Pick a location for your Docker virtual disk.

The path must be device-specific (you cannot specify a path through the user share file system; e.g., “/mnt/user/docker.img” is not a valid path).

It is recommended to store the virtual disk on the root of the cache disk or on the root of a data disk if no cache disk is available.

Click Apply to create the virtual disk and start the Docker service (this may take some time).

Adding Template Repositories

Once the service has started, the web page will refresh and a new “Docker Containers” section will appear. The easiest way to add Dockerized applications to unRAID is through the use of template repositories which act as a catalog for installing and configuring applications with ease through the unRAID web interface. These templates and their respective applications are maintained by the unRAID user community.

Check out the complete list of available applications and repositories in our community forums.

For each repository you want to add, copy the link of the repository and paste it into the “Template repositories” field on the Docker Settings page.

Separate multiple entries in the list by pressing Enter on your keyboard.

When you’re done adding repositories, click the Save button.

Adding Your First Container

With your template repositories added, you can now begin creating application “Containers” using Docker. Containers prevent software from causing conflicts with other applications and services running on unRAID.

Now click the Template drop down to select an application from one of the repositories we added previously.

After selecting, the page will refresh and new fields will be presented for configuring the container’s network and storage access.

Be sure to read the Description section for any special instructions.

Network Type

If the Bridge type is selected, the application’s network access will be restricted to only communicating on the ports specified in the port mappings section. If the Host type is selected, the application will be given access to communicate using any port on the host that isn't already mapped to another in-use application/service. Generally speaking, it is recommended to leave this setting to its default value as specified per application template.

Volume Mappings

Applications can be given read and write access to your data by mapping a directory path from the container to a directory path on the host. When looking at the volume mappings section, the Container volume represents the path from the container that will be mapped. The Host path represents the path the Container volume will map to on your unRAID system. All applications should require at least one volume mapping to store application metadata (e.g., media libraries, application settings, user profile data, etc.). Clicking inside these fields provides a “picker” that will let you navigate to where the mapping should point. Additional mappings can be manually created by clicking the Add Path button. Most applications will need you to specify additional mappings in order for the application to interact with other data on the system (e.g., with Plex Media Server, you should specify an additional mapping to give it access to your media files). It is important that when naming Container volumes that you specify a path that won’t conflict with already existing folders present in the container. If unfamiliar with Linux, using a prefix such as “unraid_” for the volume name is a safe bet (e.g., “/unraid_media” is a valid Container volume name).

Port Mappings

When the network type is set to Bridge, you will be given the option of customizing what ports the container will use. While applications may be configured to talk to a specific port by default, we can remap those to different ports on our host with Docker. This means that while three different apps may all want to use port 8000, we can map each app to a unique port on the host (e.g., 8000, 8001, and 8002). When the network type is set to Host, the container will be allowed to use any available port on your system. Additional port mappings can be created, similar to Volumes, although this is not typically necessary when working with templates as port mappings should already be specified.

IMPORTANT NOTE: If adjusting port mappings, do not modify the settings for the Container port as only the Host port can be adjusted.

Container Creation Process

With your volume and port mappings configured, you are now ready to create your first Docker container. Click the Create button and the download process will begin. A few things worth noting while the image is downloading:

After clicking Create, do not close your browser window or attempt to navigate to other tabs using the browser until the download is complete.

Initial downloads per template repository may take longer than subsequent downloads per repository.

When the download process completes, you can click the Done button to return to the Docker page and continue adding applications.

Controlling Your Application

Once the download is complete, the application is started automatically. To interact with your application, we begin by clicking on the icon visible on the Docker page of the unRAID web interface. Doing so will make a context menu appear with multiple options:

WebUI

Most apps added through Docker will have a web interface that you can access to configure and use them, but not all.

Clicking this option will launch a new browser tab/window directly to the applications web interface.

For apps that do NOT have a web interface, read the description when adding the container for instructions on how to make use of the app once it’s running.

Update

This option only appears after clicking Check for Updates (if available).

Start/Stop

This will toggle the active state of the container.

Logs

If you are having difficulties with your application, useful information may be present the application’s log.

Logs for applications are stored separately from unRAID’s system log itself.

Edit

Container settings such as port and volume mappings can be changed by clicking this option.

Once changes are applied, the container will start automatically, even if it is stopped initially.

Enable/Disable autostart

Toggling this will change the default behavior of the application when the Docker service is started.

Remove

Allows you to remove either the entire application, or just the container.

Removing a container without it’s “image” will make adding the application again later a much faster process (as it will not need to be redownloaded).

Accessing a Volume Mapping Inside a Container

One of the first things you will do after your application is running will be to configure it. Configuration typically will involve specifying storage locations from within the applications web interface. When doing so, remember to look for the volume mappings you defined when adding your container. For example, if I needed to specify a folder path in the BT Sync app that would point to my Media share, I would specify the path of “/unraid_media” in the applications interface, as depicted below.

Other Tips and Tricks

Using Docker containers to run applications on unRAID is incredibly easy and very powerful. Here are some additional tips to improve your experience:

Using Virtual Machines

While Docker Containers are the preferred mechanism for running Linux-based applications such as media servers, backup software, and file sharing solutions, virtual machines add support for non-Linux workloads and the ability to provide driver support for assigned PCI devices. Localized Virtualization is our method of supporting VMs where all resources assigned to the guest are local to the host.

NOTE: This guide applies to KVM boot mode only.

Technology Stack

unRAID 6 features a number of key technologies to simplify creation and management of localized VMs:

KVM

A hypervisor is responsible for monitoring and managing the resources allocated to virtual machines.

Unlike other hypervisors, KVM is the only one that is built directly into and supported by the Linux kernel itself.

All other type-1 hypervisors out there will load before Linux does, and then Linux runs in an underprivileged state to that hypervisor.

By leveraging a hypervisor that is part of the Linux kernel itself, it means better support, less complexity, and more room for optimization improvements.

QEMU

KVM is the component in the kernel that manages / monitors resources allocated to virtual machines.

QEMU is responsible for the emulation of hardware components such as a motherboard, CPU, and various controllers that make up a virtual machine.

KVM can't work without QEMU, so you'll often times see KVM referred to as KVM/QEMU.

VirtIO

A virtualization standard for network and disk device drivers where just the guest's device driver "knows" it is running in a virtual environment, and cooperates with the hypervisor.

This enables guests to get high performance network and disk operations, and gives most of the performance benefits of paravirtualization[2].

VirtFS

Also referred to as the 9p filesystem, VirtFS allows us to easily pass through file system access from a virtualization host to a guest.

VirtFS is the equivalent of Docker Volumes for KVM, but requires a mount command to be issued from within the guest[3]. VirtFS works with Linux-based virtual machines only.

VFIO

Virtual function IO allows us to assign a physical device, such as a graphics card, directly to a virtual machine that in turn will provide driver support for the device directly.

VFIO prevents assigned devices from accessing spaces in memory that are outside of the VM to which they are assigned.

This limits the impact of issues pertaining to device drivers and memory space, shielding unRAID OS from being exposed to unnecessary risk.

Libvirt is collection of software that provides a convenient way to manage virtual machines and other virtualization functionality, such as storage and network interface management.

These software pieces include an API library, a daemon (libvirtd), and a command line utility (virsh)[5].

System Preparation

Before you can get started with creating virtual machines, there are a few preparatory tasks that must be completed.

Adjust BIOS Settings

In order to utilize all the virtualization features of unRAID 6, you must ensure your BIOS is configured properly for hardware-assisted virtualization as well as IO memory mapping (HVM / IOMMU support). In your BIOS settings, look for anything marked with Virtualization, Intel VT-x, Intel VT-d, AMD-V, or AMD-Vi and set it to Enabled.

examples of where virtualization settings can be found from various motherboard BIOS screens.

Configure a Network Bridge

There are two methods by which your virtual machines can get access to host-based networking: through a private NAT bridge managed by libvirt or through a public bridge managed by unRAID directly. The private bridge (virbr0) is automatically configured when libvirt starts. The public bridge can be created through the Network Settings page on the unRAID webGui.

The private bridge generates an internal DHCP server/address pool to create IPs for VMs automatically, but the VMs will be on a subnet that cannot be accessed by other devices or even other services on unRAID. This type of bridge is ideal if you want your VM to be completely isolated from all other network services accept internet access and the host's network file sharing protocols. VM management can be performed through a VNC session provided by the browser.

The public bridge provides VMs with an IP address from your router, but internally bridges communications between VMs and each other, as well the host. This type of bridge is ideal if you want your VMs to act just like another device on your network, where you manage it's network access at the LAN-router instead of inside the VM. We persist MAC address settings for the virtual interfaces you create, ensuring the VMs should get the same IP address each time they connect, as long as your router-managed DHCP pool doesn't run out of addresses. So if you want to connect to your VM from another PC, laptop, tablet, or other type of device, you should use the public bridge.

Whichever bridge you prefer can be defined as the Default Network Bridge on the VM Settings page.

Create User Shares for Virtualization

At the minimum, you should create two user shares for use with virtualization on unRAID. One share to store your installation media files (ISOs) and another to store your virtual machines themselves. If you don't already have a share you use for backups, you might consider adding one as well to use for backing up your virtual machines.

Recommendations for Share Configuration

Virtual machines will perform best when their primary vDisk is stored on a cache-only share.

While SSDs are not required for virtual machines to function, performance gains are substantial with their use.

Warning: Passing through a GPU to a SeaBIOS-based VM will disable console VGA access

If you rely upon a locally-attached monitor and keyboard to interact with the unRAID terminal directly, you will lose this ability once you create a SeaBIOS VM with a GPU assigned. This is due to a bug with VGA arbitration and cannot be solved. This does NOT affect your ability to access the console using a telnet or SSH session, but local console access directly will appear to be frozen (blinking cursor, but no visible response to keyboard input). It does not matter if you are using on-board graphics for the console compared to a discrete GPU for the pass through to a VM or not. With OVMF, however, VGA isn't utilized, therefore arbitration isn't needed and therefore your console graphics will remain intact. Note that not all GPUs support OVMF as OVMF requires UEFI support on your GPU.

Help! I can start my VM with a GPU assigned by all I get is a black screen on my monitor!

If you aren't receiving an error message, but the display doesn't "light up" when your VM is started, it means that while the device is being assigned properly, you may have an issue with your motherboard or GPU preventing proper VGA arbitration from occurring. There are a few things you can attempt to fix this:

Ensure your motherboard BIOS and video card BIOS are up to date.

Try adjusting the BIOS under Advanced View when adding a new VM from SeaBIOS to OVMF (existing VMs cannot have this setting changed once created).

Try adjusting the Machine Type from i440fx to Q35 under Advanced View when editing or adding a VM.

As a last resort, you can attempt to manually provide the ROM file for your video card by editing the XML for your VM (see below procedure).

Change the path after /mnt/user/ to the actual user share / sub-folder path to your romfile.

Once done editing the XML, click Update and try starting your VM again to see if GPU assignment works properly.

Installing a Windows VM

With Windows-based guests, the installation process is slightly different than others, as you will need to load the virtual drivers for I/O (disk, network, etc). To do this, you will need to perform the following steps:

Obtaining Your Installation Media (ISO)

Depending on which version of Windows you wish to install, the process for obtaining the installation media will differ slightly. To obtain installation media for Windows 7 or later, you will need to enter a valid Microsoft Windows product key for whichever version of the software you are attempting to download. Product keys can be obtained from either the Microsoft Store or authorized resellers. If you have a product key, but do not have the installation media, please see the following links:

It is important that you do not select to install the OS to a USB flash device. If prompted, select Install by creating media and select ISO file (this will let you save the media to an ISO file). Once you've obtained your ISO file, copy it to a share on your unRAID server.

Obtaining Virtual Hardware Drivers (VirtIO) for Windows

In order to maximize VM performance, unRAID utilizes VirtIO, which eliminates much of the overhead associated with I/O related to virtualization. These virtual devices will need to have their drivers loaded during the Windows installation process, or the process will not complete.

Copy the ISO file for the drivers to the ISO Library Share you created earlier

On the VM Settings page, use the file picker for VirtIO Windows Drivers ISO to select the ISO file you copied, then click Apply on that page.

You can override the default driver ISO on a per-VM basis (under Advanced View).

Creating Your Windows VM

Follow the documented procedure for creating a VM, but alter the following settings:

Select the appropriate version of Windows from the Operating System field.

Select the Windows ISO you downloaded and copied to unRAID for the OS Install ISO.

Be sure to select a minimum of 1GB of Initial Memory and specify at least 20GB for the Primary vDisk Size (as required by Windows 7, 8, and 8.1).

For Windows 7, make sure the BIOS setting is left at SeaBIOS.

For Windows 8/8.1, you can select either SeaBIOS or OVMF, but to assign a graphics device to OVMF, it must support UEFI.

Loading the VirtIO Drivers During Installation

During the Windows installation process, you will reach a point where "no disks are found", this is expected behavior.

Click Browse on this screen, then navigate to the virtio-win CD-ROM.

You will need to load the following drivers in the following order:

Balloon

NetKVM

vioserial

viostor (be sure to load this one last)

For each driver that needs to be loaded, you will navigate to the driver folder, then the OS version, then the amd64 subfolder (never click to load the x86 folder)

After each driver is loaded, you will need to click the Browse button again to load the next driver.

After loading the viostor driver, your virtual disk will then appear to select for installation and you can continue installing Windows as normal.

After Windows is completely installed, you can install the guest agent, which improves host to guest management

Open Windows File Explorer

Browse to the virtual CD-ROM for virtio-win again, and then open the guest-agent folder

Double-click to launch the qemu-ga-x64.msi installer (this process will be rather quick)

And that's all there is to it! If you have questions on this procedure, please post in the Lime Technology forums.

Converting VMs from Xen to KVM

Virtual machines that were running in Xen to KVM will require different procedures depending on whether they were created as paravirtualized or hardware-virtualized guests. Regardless of your conversion scenario, it is highly-recommended that you create a copy of your existing Xen virtual disk before proceeding. Use the copy to test your conversion process and if successful, you can remove your own Xen-based virtual disk should you so desire. In addition, you should ensure your hardware has support for hardware-assisted virtualization (Intel VT-x / AMD-V) as this is a requirement for use with KVM. Xen PV guests do not leverage hardware-virtualization extensions, which makes their process for converting slightly more involved than Xen HVM guests to KVM (it is not documented at the time of this writing).

Windows 7 Conversion Procedure

To convert a Windows 7 virtual machine from Xen to KVM, the process is fairly simple and takes about 10 minutes to perform. Remove any PCI device pass through that you are doing via your Xen domain cfg file before you begin. These devices can be re-added after the conversion process is complete.

General Notes

Changing the machine type between i440fx and Q35 under advanced mode will prompt Windows for reactivation of the license.

Windows 7 and earlier OS variants may not work with host-based graphics assignment correctly. Use Windows 8.1 or newer for the best experience.

If using OVMF, you must use Windows 8 or newer. UEFI is not directly supported by Windows 7 and therefore, OVMF will not work.

Enable MSI for Interrupts to Fix HDMI Audio Support

If you are assigning a graphics device to your Windows guest that uses an HDMI connection and you wish to push audio through that connection, you will need to perform a registry modification in Windows to ensure the audio driver remains working properly. For a comprehensive explanation of MSI and VFIO interrupts, you can visit Alex Williamson's blog[8]. Here's the procedure for doing this:

Shut down your VM and make a copy of your virtual disk before proceeding (as a backup).

Start your VM with the GPU device assigned.

Access your server using SSH or telnet.

For the device you wish to assign, locate it's PCI address identifier (this can be found when selecting the device from within the VM creation tool)

Look for a line that looks like this: Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+

If the Enable setting is set to +, that means your device claims it is MSI capable and it is enabled by the guest VM that is using it. If you cannot find a line that mentions MSI as a capability, it means your device does not support this. If the Enable setting is set to -, this means your device claims it is MSI capable, but that the guest VM is NOT using it. The procedure for enabling MSI support from Windows is documented here: http://forums.guru3d.com/showthread.php?t=378044

Physical to Virtual Machine Conversion Process

If you have an existing physical PC or server that you wish to convert to a virtual machine for use on unRAID 6, the process is fairly simple. Steps 1-3 apply for almost any modern Linux-based guest. Steps 4-6 apply for Windows-based guests.

Prerequisites

Your system must meet the hardware requirements and complete these preparation steps before utilizing virtual machines on unRAID Server OS 6.

You must have enough disk space available on a single storage device in your array (total free space in the cache pool) that is equal to or greater in size than the physical disk you wish to convert.

It is highly encouraged to make a complete backup of your most important files before attempting a conversion.

Step 1: Identify the disk to be converted using the unRAID webGui

With the array stopped, attach the physical disk you wish to convert to your server (SATA and power)

Replace sdX with the device letter handle you noted in step 1, replace vdisk_share with the share you created to store your virtual disks, and replace vmname with the name you gave it when you created it in step 2.

The -p tag will output progress in the form of a percentage while the conversion is occurring.

Step 4: Edit the XML for your virtual machine (Windows Guests Only)

From the VMs tab, click the VM icon and select Edit XML from the context menu.

Scroll down the XML and locate the <target> tag for the <disk> with a <source> file set to vdisk1.img, which will look like such:

Using Windows File Explorer, navigate to the VirtIO virtual cd-rom to browse its contents.

Navigate inside the Balloon folder.

Navigate to the subfolder named after your Windows OS version (e.g. w8.1)

Navigate to the amd64 subfolder

Right-click on the balloon.inf file inside and click Install from the context menu (you may need to enable viewing of file extensions to do this)

Repeat the above process for each of the following folders:

NetKVM

vioserial

viostor

When done installing drivers, navigate inside the virtual cd-rom one more time and open the guest-agent folder.

Double-click on qemu-ga-x64.msi to install the QEMU/KVM guest agent.

Step 6: Remove the secondary vdisk from your VM (Windows Guests Only)

Shutdown your VM if it isn’t already.

From the VMs tab, click the VM icon and select Edit from the context menu.

Remove the vdisk2.img virtual disk by clicking the red minus symbol.

Click Update to update the VM.

Start your newly converted virtual machine!

Extra: HELP! Stuck at SeaBIOS with "Booting from Hard Disk"

If your OS was installed using UEFI (as opposed to traditional VGA BIOS), start over from step 3, but select OVMF as the BIOS type instead of SeaBIOS. Most OS installations install using a traditional VGA BIOS, but it is possible to have a UEFI installation, in which case SeaBIOS will not work. The remainder of the conversion procedure is identical.

Using the unRAID webGui

To take control of and manage your unRAID system, you will need to connect to the unRAID webGui (also referred to as Dynamix webGui).

To connect to the webGui, simply type the name of your server (or it's IP address) into your browser's address bar (by default this would be http://tower or http://tower.local if using a Mac OS X device).

To adjust the default server name or IP address prior to booting your system, you can insert the USB flash device into your Mac or PC and edit the config/network.cfg and config/ident.cfg files respectively.

Dashboard

The first tab on the unRAID webGui is the dashboard. The dashboard is designed to provide you a summary view of your system and allows you to quickly jump to common tasks that are relevant to the running state of the system.

Main

The main tab is used to assign storage devices when the array is stopped, and provides an summary view of disk usage when running. Parity checks can also be manually invoked from this page. The page itself is broken into 5 sections:

Array devices show all disks assigned to the parity or data function in the array.

Cache devices show all disks assigned to the cache function.

Unassigned devices show disks that are not assigned for use with unRAID but are physically attached to the system.

Devices

Devices are organized into tables per section with columns to represent various relevant information.

Colored Status Indicator

The significance of the color indicator at the beginning of each line is as follows:

Green: the hard drive status is Normal

Yellow: the data contents of the actual hard drive are invalid. The parity disk has this status when Parity-Sync is taking place. A data disk has this status during Reconstruction.

Red: the disk is disabled.

Blue: a new disk not currently part of the array.

Grey: indicates the corresponding disk has been spun-down.

Identification

This data is read directly from the hard drive and contains the device serial number / model number as well as the drive letter assigned under /dev.

Temperature

This is the temperature reported by the hard drive via S.M.A.R.T. When the disk is spun down, there will be an asterisk (*) displayed here instead. This is because sending the command to a hard drive to obtain S.M.A.R.T. information would cause it to spin up.

Size

This is the raw capacity of the disk expressed as the number of 1024-byte blocks.

Used

This represents the amount of capacity used on the disk. Parity / Unassigned Devices will not display a value here.

Free

This is the amount of free space in the disk's file system, expressed as the number of 1024-byte blocks. The free space of a freshly formatted disk will always be less than the disk's raw size because of file system overhead. Parity / Unassigned Devices will not display a value here.

Reads, Writes, Errors

The Read and Write statistics display the number of 4096-byte read and write operations that have been performed by the disk.

The Error statistic displays the number of read and write operations which have failed. In a protected array, any single-disk read error will be corrected on-the-fly (using parity reconstruction). The Error counter will increment for every such occurrence. Any single-disk write error will result in the Error counter being incremented and that disk being disabled.

Upon system boot, the statistics start out cleared; and they may be manually cleared at any time; refer to the Settings page.

Filesystem

The filesystem assigned to the device will be listed here.

Array Operation

Starting and Stopping the Array

Normally following system boot up the array (complete set of disks) is automatically started (brought on-line and exported as a set of shares). But if there's been a change in disk configuration, such as a new disk added, the array is left stopped so that you can confirm the configuration is correct. This means that any time you have made a disk configuration change you must log into the unRAID webGui and manually start the array.

Disk Configuration Changes

Here are the normal configuration changes you can make:

add one or more new disks.

replace a single disk with a bigger one.

replace a failed disk.

remove one or more data disks

reset the array to an unconfigured state.

Add One or More New Disks

This is the normal case of expanding the capacity of the system by adding one or more new hard drives:

Stop the array.

Power down the server.

Install your new hard drive(s).

Power up the unit.

Start the array.

When you Start the array, the system will first format the new disk(s). When this operation finishes, all the data disks, including the new one(s), will be exported and be available for use.

The format operation consists of two phases. First, the the entire contents of the new disk(s) is cleared (written with zeros), and then it's marked active in the array. Next, a file system is created (either ReiserFS, XFS, or BTRFS). By default, unRAID prefers to use XFS for new array devices and BTRFS for new cache devices. These settings can be altered on the Disk Settings page.

The clearing phase is necessary to preserve the fault tolerance characteristic of the array. If at any time while the new disk(s) is being cleared, one of the other disks fails, you will still be able to recover the data of the failed disk. Unfortunately, the clearing phase can take several hours depending on the size of the new disks(s).

The capacity of any new disk(s) added must be the same size or smaller than your parity disk. If you wish to add a new disk which is larger than your parity disk, then you must instead first replace your parity disk. (You could use your new disk to replace parity, and then use your old parity disk as a new data disk.)

Replace a Single Disk with a Bigger One

This is the case where you are replacing a single small disk with a bigger one:

Stop the array.

Power down the unit.

Replace smaller disk with new bigger disk.

Power up the unit.

Start the array.

When you start the array, the system will reconstruct the contents of the original smaller disk onto the new disk. Upon completion, the disk's file system will be expanded to reflect the new size. You can only expand one disk at a time.

If you are replacing your existing Parity disk with a bigger one, then when you Start the array, the system will simply start a parity sync onto the new Parity disk.

A special case exists when the new bigger disk is also bigger than the existing parity disk. In this case you must use your new disk to first replace parity, and then replace your small disk with your old parity disk:

Stop the array.

Power down the unit.

Replace smaller parity disk with new bigger disk.

Power up the unit.

Start the array.

Wait for Parity-Sync to complete.

Stop the array.

Power down the unit.

Replace smaller data disk with your old parity disk.

Power up the unit.

Start the array.

Replace a Failed Disk

This is the case where you have replaced a failed disk with a new disk:

Stop the array.

Power down the unit.

Replace the failed hard disk with a new one.

Power up the unit.

Start the array.

When you Start the array after replacing a failed disk, the system will reconstruct the contents of the failed disk onto the new disk; and, if the new disk is bigger, expand the file system.

You must replace a failed disk with a disk which is as big or bigger than the original and not bigger than the parity disk. If the replacement disk is larger than your parity disk, then the system permits a special configuration change called swap-disable.

For swap-disable, you use your existing parity disk to replace the failed disk, and you install your new big disk as the parity disk:

Stop the array.

Power down the unit.

Replace the parity hard disk with a new bigger one.

Replace the failed hard disk with you old parity disk.

Power up the unit.

Start the array.

When you start the array, the system will first copy the parity information to the new parity disk, and then reconstruct the contents of the failed disk.

Remove One or More Data Disks

In this case the missing disk(s) will be identified. If there is only one missing disk when you start the array it will be marked as failed. All data disks will be exported (including the missing one), but the system will be running unprotected; that is, if a disk fails you will lose data.

If there are two or more missing disks, you can not start the array. In this case you must either put the disks back, or reset the array to an unconfigured state.

Reset Array to an Unconfigured State

When the array is Stopped, you can navigate to the Tools tab at the top of the webGui and click on New Config. This function will restore the array configuration data so that the system thinks it's brand new with all new hard drives. When you Start the array, the system will start a background process to generate the parity information.

In the special case where all the hard drives are new, the format operation will not clear the data areas; it simply generates parity. This can be used when you've added new disk(s) and you don't want to wait around for the clear phase to complete. In this case you could first Reset the array configuration, and then simply Start the array, and the system will re-sync parity, incorporating the new disk(s).

CAUTION: if a disk fails during the operation, you will not be able to rebuild it.

The array configuration data is stored in the file config/super.dat on the Flash. For this reason, you must always have the Flash installed in your server.

Check Parity

When the array is Started and parity is already valid, there is a button in the Array Operation section labeled Check, which will initiate a background Parity-Check function. Parity-Check will march through all data disks in parallel, computing parity and checking it against stored parity on the parity disk. If a mismatch occurs, the parity disk will be updated (written) with the computed data and the Sync Errors counter will be incremented.

The most common cause of Sync Errors is power-loss which prevents buffered write data being written to disk. Anytime the array is Started, if the system detects that a previous unsafe shutdown occurred, then it automatically initiates a Parity-Check.

Shares

The Shares page is used to configure shares and share access.

User Shares

This section lists all of the configured User shares.

Note: if User shares are not enabled, then this section is not present.

User shares is a feature of unRAID OS which provides a unified name space across multiple data disks. User shares simplify storage management by presenting a view of all unRAID storage as if it were one large file system.

When 'User Shares' are enabled, unRAID OS will automatically create a set of shares named after the top-level directories found on each data disk. If the same top-level directory exists on more than one disk, then the exported share will contain all directories/files under that top-level directory on all the disks.

For example, suppose each disk has the following structure:

disk1

Movies

Alien

folder.jpg

VIDEO_TS

VIDEO_TS.IFO

VTS_01_1.VOB

VTS_01_2.VOB

Basic

folder.jpg

VIDEO_TS

VIDEO_TS.IFO

VTS_01_1.VOB

VTS_01_2.VOB

disk2

Movies

Cars

folder.jpg

VIDEO_TS

VIDEO_TS.IFO

VTS_01_1.VOB

VTS_01_2.VOB

disk3

Movies

Dejavu

folder.jpg

VIDEO_TS

VIDEO_TS.IFO

VTS_01_1.VOB

VTS_01_2.VOB

With User Shares enabled, for the above tree we would see this share under 'My Network Places':

//tower/Movies

And it would have the following structure:

Movies

Alien

folder.jpg

VIDEO_TS

VIDEO_TS.IFO

VTS_01_1.VOB

VTS_01_2.VOB

Basic

folder.jpg

VIDEO_TS

VIDEO_TS.IFO

VTS_01_1.VOB

VTS_01_2.VOB

Cars

folder.jpg

VIDEO_TS

VIDEO_TS.IFO

VTS_01_1.VOB

VTS_01_2.VOB

Dejavu

folder.jpg

VIDEO_TS

VIDEO_TS.IFO

VTS_01_1.VOB

VTS_01_2.VOB

In the case where the same object (directory or file) exists at the same hierarchy on multiple disks, the User Share will reference the object on the lowest numbered disk. For example, if Movies/Cars existed on both disk1 and disk2, then Cars under the Movies User Share would refer to the version on disk1.

Each time the array is Started, if User Shares are enabled, unRAID OS will regenerate and re-export each top-level directory as a network share.

Allocation Method

When a new User share is created, or when any object (file or directory) is created within a User share, the system must determine which data disk the User share or object will be created on. In general, a new User share, or object within a User share, will be created on the data disk with the most free space. However there are a set of share configuration parameters available to fine tune disk allocation.

The basic allocation strategy for a share is defined by the Allocation method configuration parameter. You may select one of two Allocation methods for the system to use:

Most-Free

In this method, the system will simply pick the disk which currently has the most free space.

High-Water

In this method, the system will pick the disk which currently has the least free space that is still above a certain minimum (called the "high water" mark). Suppose in our example above, we have this situation:

disk

size

free

disk1

80GB

75GB

disk2

120GB

110GB

disk3

80GB

70GB

The initial high water mark is set to the 1/2 the size of the largest disk; in this case, it will be set to 60GB. In this state, disk1 has 15GB of free space above the "high water" mark; disk2 has 50GB, and disk3 has 10GB.

As new objects are created, the system will choose disk3 until the amount of free space on disk3 falls under 60GB. Subsequently, the system will start allocating from disk1 until it's free space falls under 60GB. Then it will allocate from disk2 until it's free space also falls under 60GB. Once the amount of free space on all disks is below 60GB, a new high water mark is established by dividing the old high water mark by 2.

The advantage of High-water method is that when writing a series of files, most of the time only one data disk will need to be spun up.

Split Level

Often media data will consolidated under a single directory, or directory tree. Then during playback the files will be accessed one after another. This is the case with the set of VOB files which make up a DVD movie. In this situation we want all the associated media files to be stored on the same physical disk if at all possible. This is because we don't want media playback to pause while the disk containing the next file spins up. unRAID OS solves this problem by introducing a configurable allocation parameter called "Split level".

Split level defines the highest level in the share directory hierarchy which can be split among multiple disks. In the Movie share example above, setting Split level to 1 only permits any object created directly under the Movie directory to be allocated to any disk according to the Allocation method. Thus, when we create the Alien subdirectory, it may reside on any of the data disks; however, when we create a file or another directory within the Movies/Alien directory, this object is at level 2, and will be created on whatever disk the Movies/Alien directory actually resides on.

If the share were organized differently, for example according to genre:

Movies

SciFi

Alien

Action

Basic

Dejavu

Kids

Cars

Then you would set 'Split Level' to 2. This will let the genres expand among all disks, but still ensure that the contents of the actual movie directories stay within the same disk.

If you set the 'Split Level' to 0 for a share, then all directories/files created under that share will be on the same disk where the share was originally created.

If you set the 'Split Level' high, e.g., 999 for a share, then every directory/file created under that share will get placed on a disk according to Allocation method.

Included and Excluded disk(s)

The last way to control which disks are used by a share is through the Included disks(s) and Excluded disk(s) configuration parameters.

The Included disks(s) parameter defines the set of disks which are candiates for allocation to that share. If Included disk(s) is blank, then all present data disks are candiates. For example, to restrict a share to using only disk1, disk2, and disk3, you would set Included disk(s) to disk1,disk2,disk3.

The Excluded disk(s) parameter defines the set of disks which are exluded from consideration for allocation. If Excluded disk(s) is blank, then no disks are excluded.

When considering which disk to allocate space for a new object, unRAID OS first checks if it's in the Included disks(s) set, and the checks if it's in the Excluded disk(s) set.

Creating User Shares

To create a new User share:

Click the Add Share button at the bottom of the User shares list.

Enter the new Share name and other configuration and click the Add Share button.

Once a share is created you can set Export and User Level Security parameters under SMB Security Settings.

unRAID OS will select the disk to create the initial top-level share directory according to the configured Allocation method.

User Level Security

User level security is a feature that lets you restrict access to shares according to user name.

If a share security level is set to Public (not enabled), then you do not need to create additional Users. Any user that attempts to connect to a share on your unRAID server is granted access, subject to the Export mode setting on the share.

If a share security level is set to Private (enabled), you will need to enter the list of users who may access the share. When a user attempts to connect to a share on your unRAID server, a dialog box will appear asking them to enter their user name and password before being granted access to shares. In addition, you can specify which users may access each share, as well as restrict access to read-only.

Examples

Suppose we have a share called Movies for which we want everyone on the network to be able to read, but only larry can read/write:

Export: YesSecurity: Secure

User

Access

larry

Read/Write

Suppose we have a share called Finances which only mom and dad can access:

Export: Yes (hidden)Security: Secure

User

Access

mom

Read/Write

dad

Read/Write

Further, suppose only mom should be able to change the files:

Export: Yes (hidden)Security: Secure

User

Access

mom

Read/Write

dad

Read-only

Deleting User Shares

To delete a User Share:

Move or delete the contents of the user share.

Check the 'Delete' box next to the Apply button under Share Settings.

Click the Delete button.

Note: Some operating systems add hidden files to the user share which will prevent you from deleting it. You can identify these files by executing the command below from a console (replace <sharename> with your user share name):

ls -a /mnt/user/<sharename>

On version 4.7 of unRAID to delete a User Share, just clear the Share name field and click Apply. Only entirely empty User Shares may be deleted.

Renaming User Shares

To rename a User share:

Click in the Share name field of the share.

Type it's new name, and then click Apply.

Technical Notes:

A user share configuration file called config/shares/.cfg is stored on the Flash for each User Share (where the Share name is). If this file does not exist, then a set of default values are used for the User Share. Whenever a User Share parameter is changed, it's configuration file is also updated, or created if it does not exist.

Adding a new User Share or changing the configuration parameters of an existing User Share will not break any current connections on other shares. Renaming or deleting a User Share will break all outstanding connections, however. This is because Samba must be stopped in order to rename or delete the top-level directory which is associated with the share.

User Shares are implemented using proprietary code which builds a composite directory hierarchy of all the data disks. This is created on a tmpfs file system mounted on /mnt/tmp. User Shares are exported using a proprietary FUSE pseudo-file system called 'shfs' which is mounted on /mnt/users.

When an object needs to be created on a selected disk, first the directory hierarchy is created on the disk (if it isn't already in place). When the last file of a particular directory on a disk is removed, the unused part of the directory hierarchy on that disk remains in place.

With User Shares enabled, files may still be accessed via the individual disk shares. However, depending on the disk directory hierarchy and user share settings, some operations on a disk share may not be reflected in the user share which includes this disk.

Parameters

Share Name

This is the name of the share. Use only the characters: a-z, A-Z, 0-9, - (dash), and . (dot).

Comments

Optional descriptive text that will appear in the Comments column under My Network Places.

Allocation Method

The method by which the system will select the disk to use for creating a User share, directory, or disk. See Allocation below.

Split Level

The maximum depth in the directory tree which may be split across multiple disks. See Split level below.

Included Disk(s)

The set of disks which will be considered for allocation. Blank means all disks. Disks are specified using the identifier disk1,disk2, etc. Separate each identifier with a comma.

Excluded Disks(s)

The set of disk which will be excluded from consideration for allocation. Blank means no disks.

Export Mode

Specifies the basic export mode of the share. See Export Mode above.

Exceptions

A list of users who are exceptions to the basic export mode of the share: If the export mode of the share is read/write, then this lists users who will have read-only access. If the export mode of the share is read-only, then this lists users who will have read/write access. Separate multiple user names with commas.

Note: this parameter is present only when User level security is enabled.

Valid Users

A list of users who can exclusively access the share. Blank means all users.

Note: this parameter is present only when User level security is enabled.

Invalid users

A list of users who may not access the share at all. Blank means no users.

Note: this parameter is present only when User level security is enabled.

Users

The Users page is used to set a password for the root user, and adding/removing Users for your server. User level security is a feature that lets you restrict access to shares according to user name.

If User level security is not enabled, then you do not need to enter a list of users. Any user that attempts to connect to a share on your unRAID server is granted access, subject to the Export mode setting on the share.

When User level security is enabled for a share, you will need to enter the list of users who may access the share. When a user attempts to connect to a share on your unRAID server, a dialog box will appear asking them to enter their user name and password before being granted access to shares. In addition, you can specify which users may access each share, as well as restrict access to read-only.

Users

This section lists each configured user name.

Regardless of whether 'User level security' is enabled, the built-in user name root always appears atop the Users list. If you enter a non-blank password for the root user, then your browser will also prompt you for the password when you attempt to open the webGui. In addition, you will be prompted for a password to log into the console or telnet session.

Add User

To create a new user, scroll to the end of the Users list, enter the new User name and Password (and Retype password), and then click Add User.

Change Password

To change the password of an existing user, just type the new Password (and Retype password) for the user and click Apply.

Remove User

To delete a user, change the User name to blank and click Apply. Note that you can not delete the root user.

Technical Notes

All user access restrictions on shares is defined for each share on the Shares page.

Each new user is automatically given a unique uid, and unique gid (group name same as the user name). However, all objects (files/directories) created in shares will be owned by root.

Only root can access the unRAID webGui, and log in to the system console or telnet session. The configured users do not have actual home accounts on the server.

The following files are maintained in the config directory on the Flash when User level security is enabled:

User Name

The name should only consist of the characters a-z, 0-9, - (dash), _ (underscore), and . (dot). Please do not use any uppercase letters.

Password

Type anything you want here. Blank is also ok.

Retype Password

Must be the same as what you typed for Password.

Common Procedures

The following section represents common tasks that you will need to perform throughout the life of your unRAID system. This includes expanding your array, replacing failed devices, and adding/removing devices from your cache pool.

Pool Operations

Creating a Cache Pool

Without a pool, data living in the cache is in an unprotected state until moved to the array. Pooling multiple storage devices together ensures that data protection is maintained at all times, whether data is in the cache or the array.

Before we begin

You must have at least two storage devices in order to create a cache pool. Most people find that SSDs are ideal for a cache pool.

A multi-device cache pool is implemented with BTRFS only.

You must have as many device slots available for assignment to the cache as you wish to use with the pool.

Creating your pool

Stop the array, if it is not already stopped.

Increase the number of cache slots to as many as you have disk devices you wish to assign to your pool (you may have to lower your array slot count to be able to increase your cache slots)

Select each device you wish to participate in the pool and assign it to a slot in the cache.

With all devices assigned appropriately, start the array.

A Format option will appear after the array is started; click the checkbox and button to approve the procedure and initialize your new cache pool.

When completed, the pool will be identified on the Main tab along with total available space in it.

Removing a Device from a Cache Pool

Stop the array if it is started (go to the Main tab, click the checkbox and button to stop the array).

Unassign the device you wish to remove from the slot it's assigned to in the cache pool.

Physically unplug the device from the system (detach SATA and power).

Start the array.

A balance operation will be automatically executed; when this operation completes, an entry will be printed to the log stating disk deleted missing

You can once again stop the array, physically reattach the previously removed device, and assign it for another purpose, such as to the array.