and other brilliant error messages

Tag Archives: NAS

Background

It all started last year with a humble single drive DS111 which I bought for home, speculating that I could probably run Serviio on it to stream my movies to my DLNA enabled Bluray player. I managed to compile FFmpeg and its dependencies, install Java and get Serviio running and it has been golden ever since. Since then I’ve learned a lot more about Linux, packaged many other pieces of software and reverse-engineered my own Synology package repository, which at the last count over 5,000 NASes had checked into. The value of the Synology products lies in their great DSM software.

A few months later at work I decided to buy an RS411 to replace an appallingly slow FireWire Drobo (sub 10MB/sec!) that one of my colleagues was using to store video production rushes. Using the same four 2TB drives that had been inside the Drobo it has behaved impeccably – and it is currently in the process of mirroring 3TB to CrashPlan courtesy of my own app package. Having passed this test, I decided that Synology was a worthy candidate for a more serious purchase. However I noticed that there wasn’t much information about their SME products online so I’m sharing my research here.

The need for 2nd tier storage

I have used a 15K SAS EqualLogic iSCSI storage array for VMware workloads since 2009 but this is quite full. It can’t accommodate the data which I need to migrate from older arrays which are at end of life. This data (most of it flat files) is very much 2nd tier – I need lots of space, but I don’t really care too much about latency or throughput. It’s also predominantly static data. I do need at least some additional VMware storage, so I can use vMotion to decant and re-arrange VMs on other storage arrays. A larger Synology rack mount NAS therefore presents itself as a very good value way of addressing this need, while keeping the risk of failure acceptably low.

Which model?

The choice is actually pretty easy. A redundant power supply is fairly mandatory in business since this is the most likely thing to fail after a drive. The unit is under warranty for three years, but can you really survive for the day or two of downtime it would most likely take to get a replacement on site? Once this requirement is considered, there are actually very few models to choose from – just three in fact: the RS812RP+ (1U 4 bay), the RS2212RP+ (2U 10 bay) and the RS3412RPxs (2U 10 bay). The 4 bay model isn’t big enough for my needs so that narrows the field. The RS3412RPxs is quite a price hike over the RS2212RP+, but you do get 4 x 1GbE ports with the option of an add-in card with 2 x 10GbE ports. Considering that the EqualLogic PS4000XV unit I’m using for 1st tier storage manages fine with 2 x 1GbE on each controller I think this is a little overkill for my needs and besides, I don’t have any 10GbE switches (more on bandwidth later). Other physical enhancements are a faster CPU, ECC server RAM, and the ability to add two InfiniBand-connected RS1211 (2U 12 bay) expansion units rather than one.

Synology now offer VMware certified VAAI support from DSM 4.1 on the xs models (currently in beta). They actually support more VAAI primitives than EqualLogic, including freeing up space on a thin provisioned LUN when blocks are deleted. Dell EqualLogic brazenly advertised this as a feature in a vSphere 4.0 marketing PDF back in 2009 and I had to chase tech support for weeks to discover that it was “coming soon”. To date this functionality is still missing. The latest ETA is that it will ship with EqualLogic firmware 6.00 whenever that arrives. Though this is a software feature, Synology are using it to differentiate the more expensive products. More CPU is required during these VAAI operations, though blogger John Nash suggests that it isn’t much of an overhead.

If you need high performance during cloning and copying operations, or are considering using a Synology NAS as your 1st tier storage then perhaps you should consider the xs range.

Which drives?

The choices are bewildering at first. Many of the cheapest drives are the ‘green’ ones which typically spin at 5,400RPM. Their performance won’t be as good as 7,200RPM models, and they also tend to have more aggressive head parking and spindown timer settings in their firmwares. Western Digital ones are notorious for this to the extent that these drives have dramatically shorter lifespans if this is not disabled. To do this is tedious and requires a PC and a DOS boot disk. Making a bootable MS-DOS USB key can try the patience of even the calmest person!

UPDATE – it seems from the Synology Forum that DSM build 3.2-1922 and later will automatically disable the idle timer on these drives. You can check the status by running this one-liner while logged into SSH as root:

The next choice is between Enterprise class and Desktop class drives. This is quite subjective, because for years we have been taught that only SCSI/SAS drives were meant to be sufficiently reliable for continuous use. Typically the Enterprise class drives will have a 5 year manufacturer warranty and the Desktop ones will have 3 years. Often it takes a call to the manufacturer’s customer service helpline to determine the true warranty cover for a particular drive since retailers often seem to misreport this detail on their website. The Enterprise ones are significantly more expensive (£160 Vs £90 for a 2TB drive).

There is one additional feature on Enterprise class drives – TLER (Time Limited Error Recovery). There are a few articles about how this relates to RAID arrays and it proved quite a distraction while researching this NAS purchase. The concept is this: when a drive encounters a failed block during a write operation there is a delay while that drive remaps the physical block to the spare blocks on the drive. On a desktop PC your OS would hang for a moment until the drive responds that the write was successful. A typical hardware RAID controller is intolerant of a delay here and will potentially fail the entire drive if this happens, even though only a single block is faulty. TLER allows to drive to inform the controller that the write is delayed, but not failed. The side effect of not having TLER support would be frequent drive rebuilds from parity, which can be very slow when you’re dealing with 2TB disks – not to mention impairing performance. The good news though, is that the Synology products use a Linux software RAID implementation, so TLER support becomes irrelevant.

Given what’s at stake it’s highly advisable to select drives which are in the Synology HCL. The NAS may be able to overcome some particular drive firmware quirks in software (like idle timers on some models etc.), and their presence on the list does also mean that Synology have tested them thoroughly. I decided to purchase 11 drives so I would have one spare on site ready for insertion directly after a failure. RAID parity can take a long time to rebuild so you don’t want to be waiting for a replacement. Bear in mind that returning a drive under a manufacturer warranty could take a week or two.

Apparently one of the value-added things with enterprise grade SAN storage is that individual drives will be selected from different production batches to minimize the chances of simultaneous failures. This does remain a risk for NAS storage, and all the RAID levels in the world cannot help you in that scenario.

My Order

The unit has 1GB of DDR3 RAM soldered on the mainboard, with an empty SODIMM slot which I would advise populating with a 2GB Kingston RAM module, part number KVR1066D3S8S7/2G (a mere £10), just in case you want to install additional software packages later.

Synology 2U sliding rack rail kit, part number 13-082URS010 (£80). The static 1U rail kit for the RS411 was pretty poorly designed but this one is an improvement. It is still a bit time consuming to set up compared to modern snap-to-fix rails from the likes of Dell and HP.

Setup – How to serve the data

A Synology NAS offers several ways to actually store the data:

Using the NAS as a file server in its own right, using SMB, AFP, or NFS

iSCSI at block level (dedicated partitions)

iSCSI at file level (more flexible, but a small performance hit)

For my non-critical RS411, using it as an Active Directory integrated file server has proved to be very reliable. However, for this new NAS I needed LUNs for VMware. I could have perhaps defined a Disk Group and dedicated some storage to iSCSI, and some to a normal ext4 volume. I had experimented with iSCSI, but there are several problems:

Online research does reveal that there have been some significant iSCSI reliability issues on Synology, though admittedly these issues could possibly date from when DSM first introduced iSCSI functionality.

To use iSCSI multipathing on Synology the two NAS network interfaces must be on separate subnets. This is at odds with the same-subnet approach of Dell EqualLogic storage, which the rest of my VMware infrastructure uses. This would mean that hosts using iSCSI storage would need additional iSCSI initiators, significantly increasing complexity.

It is customary to isolate iSCSI traffic onto a separate storage network infrastructure, but the Synology NAS does not possess a separate management NIC. So if it is placed on a storage LAN it will not be easily managed/monitored/updated, nor even be able to send email alerts when error conditions arise. This was a show-stopper for me. I think Synology ought to consider at least allowing management traffic to use a different VLAN even if it must use the same physical NICs. However, VLANing iSCSI traffic is something most storage vendors advise against.

All of which naturally lead us onto NFS which is very easy to configure and well supported by VMware. Multipathing isn’t possible for a single NFS share, so the best strategy is to bond the NAS network interfaces into a link aggregation group (‘Route Based on IP Hash’). This does mean however, that no hypervisor’s connection to the NFS storage IP can use more than 1Gbps of bandwidth. This gives a theoretical peak throughput of 1024/8 = 128MB/sec. Considering that each individual SATA hard disk in the array is capable of providing roughly this same sustained transfer rate, this figure is somewhat disappointing. The NAS can deliver much faster speeds than this but is restricted by its 1GbE interfaces. Some NFS storage appliances help to mitigate this limitation to a degree by allowing you to configure multiple storage IP addresses. You could then split your VMs between several NFS shares, each with a different destination IP which could be routed down a different physical link. In this way a single hypervisor could saturate both links. Not so for Synology NAS unfortunately.

If raw performance is important to you, perhaps you should reconsider the xs series’ 2 x 10GbE optional add-in card. Remember though that the stock xs config (4 x GbE) will still suffer from this NFS performance capping of a single NFS connection at 1GbE. It should be noted however that multiple hypervisors accessing this storage will each be able to achieve this transfer rate, up to the maximum performance of the RAID array (around 200MB/sec for an RS2212RP+ according to the official performance figures, rising to around 10 times that figure for the xs series – presumably with the 10GbE add-in card).

As per this blog post, VMware will preferentially route NFS traffic down the first kernel port that is on the same subnet as the target NFS share if one exists, if not it will connect using the management interface via the default gateway. So adding more kernel ports won’t help. My VMware hypervisor servers use 2 x GbE for management traffic, 2 x GbE for VM network traffic, and 2 x GbE for iSCSI. Though I had enough spare NICs, connecting another pair of interfaces solely for NFS was a little overkill, especially since I know that the IOPS requirement for this storage is low. I was also running out of ports on the network patch panel in that cabinet. I did test the performance using dedicated interfaces but unsurprisingly I found it no better. In theory it’s a bad idea to use management network pNICs for anything else since that could slow vMotion operations or in extreme scenarios even prevent remote management. However, vMotion traffic is also constrained by the same limitations of ‘Route Based on IP Hash’ link aggregation policy – i.e. no single connection can saturate more than one physical link (1GbE). In my environment I’m unlikely to be migrating multiple VMs by vMotion concurrently so I have decided to use the management connections for NFS traffic too.

Benchmarking and RAID level

I found the simplest way to benchmark the transfer rates was to perform vMotion operations while keeping the Resource Monitor app open in DSM, and then referring to Cacti graphs of my switch ports to sanity check the results. The network switch is a Cisco 3750 two unit stack, with the MTU temporarily configured to a max value of 9000 bytes.

Single NFS share transfer rates reading and writing were both around 120MB/sec at the stock MTU setting of 1500 (around 30% CPU load). That’s almost full GbE line speed.

The same transfers occurred using only 15% CPU load with jumbo frames enabled, though the actual transfer rates worsened to around 60-70MB/sec. Consequently I think jumbo frames are pointless here.

The CPU use did not significantly increase between RAID5 and RAID6.

I decided therefore to keep an MTU of 1500 and to use RAID6 since this buys a lot of additional resilience. The usable capacity of this VMware ready NAS is now 14TB. It has redundant power fed from two different UPS units on different power circuits, and it has aggregated network uplinks into separate switch stack members. All in all that’s pretty darn good for £2,800 + VAT.

Running the Serviio DLNA server on a NAS is a very efficient solution. It allows all your media to be constantly available to all your devices, all without the need for a dedicated media PC. Most TV and Bluray players are already capable media renderers – so indexing the library, retrieving metadata, and streaming the content are where Serviio fits in. Device support is continually improved by a community of contributors, and this crowd-sourced aspect was what initially drew me to the project, leading me to create the Sony Bluray device profile back in 2010. Serviio can also connect to online sources of content such as catch-up TV portals and allow fuss free content playback without advertising, even on devices which lack native support. These online plugins are also community maintained.

This new version of the Synology package updates Serviio to version 1.9.2 which mostly features MediaBrowser fixes (release notes). Version 1.9 replaced the Flash Player dependency in MediaBrowser with an HTML5 player. Serviio 1.7 brought audio track selection preferences, and automatic subtitle downloading. The most significant enhancement from version 1.6 onwards is the new web management console which removes the need to install the separate Serviio Console application on a computer. The web console uses a modern adaptive design which works well on mobile devices, and it is now integrated with DSM so you will see its icon in the application menu when the package is running:

The package supports hardware transcoding on the DS214play and DS415play with Intel Evansport media processor. This support required substantial personal effort and was made possible by the very kind donation of a new DS214play unit from MrK/Boretom a fellow package maintainer for ASUStor devices.

Limitations

If you use a TV, Bluray or media player bought in the last few years it is likely to have very good format support. In this case even a low specification NAS will be adequate for streaming most video content. There are some limitations however, in particular with video transcoding.

Serviio’s MediaBrowser is a Pro version only feature, and is designed to allow direct playback of media on tablet devices without the need for dedicated apps. Please note that Apple iOS devices require video in H.264 and audio in stereo AAC format when streaming. With the exception of the “play” models already mentioned, most Synology NAS models lack the CPU power to transcode video into H.264 in realtime. Even if the source video is already H.264, ARMv5 family CPUs (which lack an FPU) in the older models lack the power even to transcode AAC audio in realtime without libfdk_aac. Unfortunately I am unable to distribute an FFmpeg binary compiled against this library under the terms of its software license.

Ever since Serviio version 1.2 renderers that do not natively support subtitles via DLNA can have them burned into the video during transcoding, but as already explained a NAS usually lacks the CPU power to do this (more info on subs here). If you absolutely need hard subs for a particular film, you could start playback on your device then stop it and Serviio will continue transcoding. Then you will be able to try again in a few hours since it will play from the transcode cache folder. Newer TVs tend to support inline SRT subs, but for those with older models I can recommend the PlayStation 4 Media Player as a competent DLNA renderer with inline subs support.

I was able to compile FFmpeg for all eight supported architectures (ARMv5, ARMv7, ARMv7 with NEON, ARMv8, Intel i686, Intel x64, Intel Evansport media processor, and PowerPC e500v2) with the additional libraries Serviio requires while using the libs included within DSM as far as possible. I have published my compilation methods to make the process easier for others in future since parts of it were difficult to figure out. Only the Intel CPU or higher end ARM CPUs are likely to be of any practical use transcoding to H.264, since the other CPU types lack vector units (no AltiVec in QorIQ/PowerQUICC, and no NEON in Armada 370/XP).

Synology Package Installation

The repository will push its certificate automatically to the NAS, which is used to validate package integrity. Set the Trust Level to Synology Inc. and trusted publishers:

Serviio now requires Java 8 for OS-native filesystem watching, so you will need to install my Java SE Embedded package first if you have not already done so. Read the instructions on that page carefully too.

Now browse the Community section in Package Center to install Serviio:
Community packages are not listed in the All category for some reason. The repository only displays packages which are compatible with your specific model of NAS. If you don’t see Serviio in the list, then either your NAS model or your DSM version are not supported at this time. DSM 6.0 is the minimum supported version for this package.

When Serviio is started for the first time, a plugins folder will be created in the public shared folder (usually /volume1/public/serviio/plugins/). Download any Serviio online content plugins that you require and place the .groovy files in that location.

If you need subtitle support for glyphs not present in the default DejaVu fonts, add additional fonts to the following folder inside your NAS’s public shared folder (usually /volume1/public/serviio/fonts/).

Notes

You should reserve an IP address for your NAS (most home broadband routers will let you do this in the DHCP options), or use a static IP address. This will prevent Serviio from appearing multiple times in the menus of your renderer devices as its IP changes.

In the console disable ‘Generate thumbnails for local videos’ in the Metadata tab. Failure to do this can make adding videos to your library very slow on older NAS systems. With this disabled Serviio will still fetch thumbnail images automatically from the online databases.

Don’t alter the transcoded files location on the Transcoding tab, it’s already set to a sensible location for Synology systems. This particular folder is deleted each time your NAS restarts, and Serviio cleans up its temp files automatically in any case.

When adding folders to the media library use the Browse button. This will show the root of the entire NAS Linux filesystem, so you will find your DSM shared folders inside /volume1.

If you’re upgrading from a previous Serviio version you should check for updates of your online content plugins.

MediaBrowser (Pro only feature) can be opened using the link in the side bar of the web console (http://IPofYourNAS:23424/mediabrowser). If you want to use the MediaBrowser from a remote location you would need to either port forward 23424 on your router, or use SSH tunnelling. You can also manage Serviio remotely by SSH tunnelling port 23423 which the console uses. Both of these service definitions are added to the DSM Firewall.

Advanced users who wish to edit device profiles or enable debug logging can find the Serviio files in /var/packages/Serviio/target. You will need to use an SSH session to access this folder – it cannot be navigated to using DSM’s File Station. You will need to restart the Serviio package for profile changes to take effect.

Some users may wish to restrict running Serviio to specific times of day in order to free up RAM for a scheduled backup window for instance. This can be achieved by creating tasks to start and stop Serviio using the Task Scheduler in DSM Control Panel:

The package uses an FFmpeg wrapper script to modify the usual transcoding behaviour of Serviio to use the Intel Evansport hardware decoding and encoding available in the DS214play and DS415play models. I have left the FFmpeg wrapper scripts in the package even when not in use (in the Serviio/bin folder), because they are still useful for people trying to test a particular workaround or encoder setting. The wrapper was a tricky script to get working due to variable expansion precedence issues and quote handling. If you need it, just change the ffmpeg.location system property in /var/packages/Serviio/target/bin/serviio.sh to point to your modified wrapper script.

If additional fonts are not working as expected, some troubleshooting is available. You can re-create the fontconfig cache by connecting to the NAS via an SSH session as admin, and then running:sudo -i
(enter admin password to become root)
rm -rf ~/.fontconfig.cache
FC_DEBUG=128 /var/packages/Serviio/target/bin/fc-cache --verbose

Hardware transcoding support

DS214play and DS415play with Intel Evansport SoC are the only models supported for hardware transcoding. Serviio uses the multimedia tool FFmpeg for manipulating media files, and I am only able to build a hardware-assisted FFmpeg for DS214play and DS215play. Subsequent to those products, Synology has marketed a number of other systems with hardware transcoding features but these are not supported by Serviio.
The DS216play with STiH412 Monaco SoC uses a specialised build of Gstreamer for the Synology transcoding solution rather than FFmpeg, so its hardware features cannot be used by Serviio.
It seems likely that Synology is also using Gstreamer for the Intel Braswell or newer generation CPUs with QuikSync, since DSM ships with an older FFmpeg (2.7.1) than the version which introduced QuikSync support (2.8.0). To implement FFmpeg QuikSync hardware transcoding support requires Linux kernel patches for libmfx support which is not currently included in DSM 6.x, so Serviio support for Intel QuikSync on Synology is unlikely unless Synology in future switches to using FFmpeg for its own hardware transcoding solution.
The DS218play with Realtek RTD1296 SoC uses Gstreamer with OpenMAX drivers for the Synology transcoding solution. Although I was able to compile the ARMv8 build of FFmpeg with libomx support, unfortunately the Belaggio OpenMAX IL will not register these Realtek drivers which are distributed in the VideoStation package, therefore the hardware transcoding features of this model cannot be used by Serviio.

For the supported systems FFmpeg will use hardware decoding wherever possible.

Although multiple hardware assisted decodes can take place simultaneously, only a single hardware encode is available. The package intelligently avoids race conditions for this.

Any FFmpeg commandline to encode using libx264 will automatically be sent to the hardware encoder when available.

profiles.xml has been modified so that the preferred transcode format for most devices for non-natively supported videos is H.264.

In order to add hardware H.264 encoding to a device profile, specify h264 as the desired codec and the FFmpeg wrapper script will make the necessary substitution automatically.

When the hardware encoder is used, the low quality settings in Serviio’s default libx264 options to favour encoding speed are ignored. The hardware encoder performance is not altered by quality settings.

If a media stream is being both hardware decoded and hardware encoded, it will use pipelined mode. Based on responses from Synology support during development this means that both jobs occur on the media processor without exposing the intermediate bitstream to FFmpeg. Consequently it is not possible to burn subtitles in pipelined mode. As a result my FFmpeg wrapper script will run the decode part of the job in software in order to do this. The unfortunate result is that the encoder cannot quite achieve real time transcoding during subtitle compositing. For full 1080p HD material it achieves around 20fps, whereas most movies are 24fps. This means that in order to watch a film with hard subs you would need to start it off, pause or stop the renderer and return a bit later to view. An alternative would be to mux the content into an MKV container with the desired subtitles before adding it to the Serviio library, assuming that your device supports SRTs in MKV.

The hardware encoded H.264 streams are not 100% compliant with the specification and fail validation using the tool DGAVCDec. For most tested renderers this does not affect playback (Bravia, Sony Bluray, iOS). FlowPlayer which had been the embedded player in MediaBrowser did not play these files satisfactorily and dropped many frames resulting in jerky playback. However since Serviio version 1.9 MediaBrowser uses an HTML5 player which is not affected by this issue.

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology 3rd Party Developer Guide.

0029 28/Jan/16 – Updated to Serviio 1.6, integrated the new built-in web console into DSM, fixed stupid wrapper bug which had broken evansport hardware transcoding on DS214play and DS415play, fixed online content when using evansport build. You must have already upgraded to version 1.5.2-0028 or else Package Center will report a TCP port conflict and will fail to upgrade the package. If you missed the 0028 update, install it manually in Package Center using the .spk file: (DS214play/DS415play), (all other models).

0027 12/Nov/15 – Substantial package re-write hence the long delay since the last release
Completely refreshed documentation, please re-read
Updated to Serviio version 1.5.2
Java 8 is now required – this is a Serviio imposed requirement
DSM 5.0 or newer is now required
Updated to FFmpeg 2.7.1 and many libs e.g. x264 at latest versions from late July 2015
Added optional hardware transcoding support for Intel Evansport systems (DS214play, DS415play) built using Synology-modified sources for FFmpeg 2.0.2
FFmpeg is now compiled with https support via GNUTLS library (useful for online streams via plugins)
Switched to using root account which was necessary for Evansport hardware encoding – no more adding account permissions, package upgrades will no longer break this
Added support for Mindspeed Comcerto 2000 CPU (comcerto2k – ARM Cortex-A9 with NEON) in DS414j
Added support for Intel Atom C2538 (avoton) CPU in DS415+
Fontconfig fc-cache utility now included to allow troubleshooting issues with adding new fonts for additional subtitle language support
Fontconfig variables checked on each package startup so DSM upgrades no longer break hard subs support
Updated DejaVu subtitle font to version 2.35
DSM Firewall application definitions added
Tested with DSM Task Scheduler to allow package to start/stop at certain times of day, saving RAM when not needed
Daemon init script now uses a proper PID file instead of the unreliable method of using grep on the output of ps
Daemon init script can be run from the command line
Improved accuracy of temp folder and plugins folder detection
Switched to .tar.xz compression for native binaries to reduce web hosting storage footprint
Package is now signed with repository private key
User Agent customization while downloading Serviio binaries from serviio.org to allow download stats gathering

014 28/Oct/12 – Added support for Freescale QorIQ PowerPC CPUs used in some Synology x13 series products, switched to shared library compile of FFmpeg with some minor changes, ARM build now uses libshine encoder which allows realtime audio transcoding to MP3, edited FlowPlayer config to play AAC audio natively to make MediaBrowser more usable with online feed items.

013 Fixed a bug in the plugins folder creation, fixed deletion of home directories after user deletion (DSM 4.1)

012 Reduced Java prefs checking interval to once every 24 hours to allow the NAS to hibernate (was 30 seconds by default – remember to increase the time between library refreshes if you want hibernation)

007 Updated Serviio to 0.6.2, kairoh’s WebUI to 0.6.2c, changed package to download Serviio from the official website during installation, temp path is set by default to /volume1/@tmp on first run, and finally some CSS improvements

006 Updated to Serviio 0.6.1, switched to kairoh’s Java WebUI, added timezone support, removed DSM icon when Serviio is not running, adjusted Java max heap size for systems with low RAM, specified FFmpeg path directly rather than creating a symlink in /bin so as not to interfere with other packages which may use different versions of FFmpeg

005 Added Web Station dependency, EULA dialog, and links for Web UI and user forum in More Info

004 Test for package repo to allow update notification

0.6.0.1 v3 Hopefully fixed an issue with the Serviio DSM icon in DSM 3.2 on NAS units without Optware installed

0.6.0.1 v2 Inclusion of a modified version of the PHP Web UI

0.6.0.1 v1 New minor Serviio release 0.6.0.1 with some updated international translations – will upgrade 0.6 preserving the media library

0.6 v1 23/Sep/11 – First public release – Serviio 0.6

0.6b4 v2 Fixed permissions on transcode temporary folder following upgrade, avoiding the need for a reboot

0.6b4 v1 Updated to Serviio 0.6 beta 4, added upgrade scripts to allow media database migration (install future packages over the top of this one – database is preserved)

0.6b3c v2 Fixed HOME env var for serviio user (to fix librtmp issues with BBC iPlayer) and removed the need to edit the passwd file to change the shell (safer)

When I originally created Synology packages for Java in 2011 I noticed that they are being referred to by some diverse projects, and one that caught my eye was OpenRemote. Since a NAS is a low power always-on system, it’s a perfect candidate to run home automation control software. OpenRemote aims to be the integration and control layer for a huge range of devices and sensors which use a substantial variety of protocols – as the project’s strapline summarizes:OpenRemote is the Open Source Middleware for the Internet of Things.

This new version of the package is a significant update to version of 2.6.2 of the OpenRemote Controller. I selected this version because the drools component has been updated so that OpenRemote will run on Java 8. I have integrated the application with DSM’s Package Center and the software icons will be shown in main DSM applications menu when the package is running (button in the top left corner):

To use the software you will need to register for an OpenRemote Designer 2.5 account here. You will have to purchase this via the order process but notice the discount code for personal use which brings the price back to zero. If you had already created designs on OpenRemote versions prior to 2.5 there is a migration process to transfer this to the new version, though it looks like the OpenRemote documentation is overdue a tidy – much of it is clearly outdated.

Synology Package Installation

In the Advanced tab of the DSM User control panel enable the User Home service.

The repository will push its certificate automatically to the NAS, which is used to validate package integrity. Set the Trust Level to Synology Inc. and trusted publishers:

OpenRemote appears to need the unlimited cryptography policy files for the Java Cryptography Extension which are not included by default. If you want to use the Synology Java package you will need to manually install these policy files yourself, however I would recommend using my own Java SE Embedded package which does this automatically. Read the instructions on that page carefully too. Ensure you select Unlimited Strength Cryptography in the package installation menu:

Browse the Community section in Package Center to install OpenRemote:
The repository only displays packages which are compatible with your specific model of NAS. The OpenRemote package contains no native binaries at all, so it will work on any NAS that can run Java. This package does now require at least DSM 5.0.

Notes

You will only be able to sync the Controller with your Designer account once you have designed something. Until then, the sync will fail. This is normal.

Since this package does not include native binaries, if you need drivers for additional devices such as USB serial dongles, USB Z-Wave adapters etc., you will need to find these elsewhere. Rather than duplicating effort I will defer to the expertise of fellow Synology package developer jadahl who seems to have this part covered already.

Previous versions of this package required a port change to TCP18581 for the controller which required a matching config change in OpenRemote Designer. This is no longer necessary since the Controller now uses TCP8688 by default which does not clash with any Synology services.

OpenRemote saves your designs in the cloud with your online account, and the controller syncs with those designs. This means that you can uninstall the package without losing your designs.

You can see the Controller’s log in Package Center

DSM Package Center installs the application to /var/packages/OpenRemote/target. If you need to edit config files, or look at other logs in detail via SSH – that’s where you’ll find them.

Package scripts

For information, here are the package scripts so you can see what it’s going to do. You can get more information about how packages work by reading the Synology 3rd Party Developer Guide.

2.6.2-0010 25 Jul 2016 – Substantial re-brand and overhaul for DSM 6.0, updated to OpenRemote Controller 2.6.2 beta which adds Java 8 compatibility, Controller is now the Pro version which is free for personal non-commercial use – registration for an OpenRemote Designer account is required, added many enhancements developed for other packages

The Serviio Web UI, styled to look like the regular DSM control panel widgets

UPDATE – This is no longer the most recent package. I have released a new version and created a new blog post to keep the comments on topic.

My guide for installing Petr Nejedly’s excellent Serviio 0.5.2 on Synology has been a very popular post, and many people have got it running. However, the process is still complex enough to deter a lot of people and, as I learned more about my Synology NAS, I realised that I could automate the whole thing. The biggest obstacle was creating a package for Java, but adding locale support to the syno Linux also proved difficult.

The PHP Web UI is now included which you can use to manage Serviio once it’s installed. I have tried to fix most of the localization issues with it, and have edited the CSS to give it the look and feel of DSM as much as I could. You could also use Serviio Remote Console from another computer if you prefer. Any online content plugins (ending in .groovy) that are found in your NAS ‘public’ share will also be installed into Serviio’s plugins folder automatically during installation.

The package will allow future upgrades while preserving the media database and the online plugins, as long as Serviio itself is ok with that. You can see Serviio’s log by clicking More in the Package Centre. DSM Package Center installs the application to /volume1/@appstore/Serviio though from what I understand, on multi-volume systems the user is prompted for a destination volume. If you need to edit device profiles, add online content plugins, or look at the log in detail that’s where you’ll find the files. Bear in mind that a NAS, even an Intel Atom powered one, is unlikely to have sufficient CPU power to manage realtime video transcoding (e.g. from DivX to MPEG2), though it will manage remuxing MKV to M2TS and realtime audio transcoding.

Download

If you have DSM 3.2, you can install directly from Package Center. In Settings -> Package Sources add my package repository URL which is http://packages.pcloadletter.co.uk

If you’re still on DSM 3.1, download one of the following packages. Right-click, Save As:

Serviio uses some open source libraries without which it would not have been possible.
By downloading the software on this page you are agreeing to the terms of the original Serviio End User Licence Agreement.
I accept no liability for any damage that may occur to your data or your cat’s health as a result of using this software :) The scripts are shown in full below, so you are free to assess any risks yourself.

Instructions

This package can be installed on an unmodified NAS – no hacking is required. It has been tested on DSM 3.1 and DSM 3.2. Here’s what you need to do:

Give it some time to start (click More Info in Package Center to see the Log tab – check it has loaded all the renderer profiles), then you can use the Serviio icon in the DSM drop down menu in the top-right to launch the Web UI. This Serviio icon will only be visible to admin users in DSM

To use the Serviio Remote Console from a Windows PC edit ServiioConsole.exe.vmoptions in the Serviio folder and add the line: -Dserviio.remoteHost=192.168.x.x (replace with your NAS’s IP address). It can take a minute or so to start – this is normal. For details on how to configure the Remote Console on other operating systems refer to the Serviio FAQ. It seems that the first time Serviio starts with a blank database the Remote Console will get stuck showing the message “Serviio is updating its Media Library”. If that happens wait a couple of minutes, stop and restart Serviio via the Package Center. Give it another couple of minutes then the remote console should work fine.

Once the console is up, disable ‘Generate thumbnails for local videos’, and set the ‘Transcoded files location’ to /volume1/@tmp. Don’t forget to use the Users control panel in DSM to grant permission for the Serviio user to read your files (if they’re not in the public share)

When you add folders to your library you need to use the ‘Add path…’ button (not ‘Add local…’) and give the full Linux path (e.g. /volume1/share-name/Movies). There are issues with the Add Local ajax file browser which mean that it doesn’t work properly even if you give PHP access to the filesystem root

The DSM main drop-down menu, showing the Serviio Web UI icon

Upgrading from 0.5.2

If you want to upgrade from a manually installed 0.5.2, you will need to follow the steps below. I was considering automating it, but it gets complicated trying to remove people’s edits from /etc/profile which may not be entirely consistent.

Serviio is an excellent free Java DLNA media server by Petr Nejedly which focuses on minimizing the amount of unnecessary media transcoding, and maximizing the use of renderer devices’ supported features. Version 0.6 adds support for streaming content from online RSS feeds. Since the installation process has changed a lot, I decided to create a new post rather than edit the old one. The guide is accurate for Synology DSM 3.2 beta, and the one tricky section with wget-ssl also applies for DSM 3.1. Older DSM versions may vary.

In the Synology DSM go to Control Panel > Terminal > enable SSH.

Read this Synology wiki document about modifying your NAS carefully and install the bootstrap for your model. If you have previously been messing around with your unit and want to set it back to defaults, you can run the bootstrap again. It will prompt you to delete a couple of folders and reboot, whereupon you can start afresh.

Download the PuTTY SSH client if you’re using Windows – other operating systems will already allow you to SSH from the terminal. Give the session a name, set the remote character set to UTF-8 as shown and save it before you connect (to save time in future). This will ensure that all characters display properly. The Mac OS SSH client defaults to UTF-8.

Connect to your NAS’s IP address using SSH. Use the root account (same password as admin). For Mac OS for instance, use ssh root@x.x.x.x. I suggest enabling a more descriptive shell prompt which should reduce the chance of accidentally being in the wrong directory:

sed (stream editor) is a powerful tool which uses regular expressions. The first command opens the file ~./profile searches for the line beginning with PS1= and duplicates it, commenting out the first copy (so the change can be undone). Then second command sets the value PS1=”\w\$ “ which changes the prompt to be the current directory followed by dollar sign and space characters.

To install the development tools. Type:

ipkg update
ipkg install optware-devel

It will halt and complain that package wget-ssl clashes with wget. Continue with:

Next we need to install Lame MP3 encoder, providing libmp3lame which will be compiled into FFmpeg, and the Nano text editor (much easier to use than vi), and fix up some other issues (thanks to Nicolas Jolet for these):

Download Java SE for Embedded 7 from Oracle, selecting the Linux build for the appropriate CPU. The ARM v5 version is required for the Marvell Kirkwood CPU Synology Products. Note that there is a PowerPC e500v2 version – the CPU core in Synology products which use the Freescale mpc85x3). Unfortunately for PowerPC Synology owners, this depends on a higher version of glibc than the Synology DSM provides for this architecture. Until JamVM supports Java 1.6, or Synology update to glibc 2.4 you won’t be able to follow this guide on PowerPC models. This situation may have changed since DSM 3.2 beta was released.

You will need to sign up to receive the Java download link by email. It’s free to use for non-commercial evaluation use. Use your computer to save it into the top level shared folder of your NAS, which will probably be /volume1/public on the NAS filesystem. Then:

Synology’s Linux build has no localization support built in, though it does use UTF-8 character encoding for the filesystem. That’s no problem for storage, however the Java VM inherits the locale setting of the host OS. Since this is undefined, Serviio and all other Java software will default to US-ASCII which is a big problem if you have filenames with non-US characters. The solution is to obtain the missing files to add locale support from the Synology toolchain, which is distributed under the GPL. Many thanks to IWarez at the Subsonic forum for this fix, though it took me a good while to realise the required files are indeed included in the ARM toolchain:

At this point ou will see the error “cp: missing destination file operand after `/opt/lib'” because the libz.so shared library wasn’t compiled and the install script tries to copy it nonetheless. libz.a which we need was built ok, so ignore and continue.

In the DSM User Control Panel create a new user called serviio and set a password. Give that user access to the paths that contain the media you want to serve. Click the User Home button and enable the User Home Service. Go back to your SSH session and type:

nano /etc/passwd

Be very careful editing this file. A wrong move here could trash your system. Notice that the serviio user has a shell of /sbin/nologin. Change this to /bin/sh like the admin user has. Nano may try to line wrap this line as you type if you added an long account description. If it does, delete the carriage return before the line break and pull it back onto one line. Save and exit.

Now we’ll create the Serviio daemon start and stop script:

nano /volume1/@tmp/S99serviio.sh

Paste in the following text (mouseover and use the view source button to copy):

Check the web UI or the process list again and make sure it did indeed stop. If it’s all ok, we need to move the daemon launcher script so it starts automatically on boot (it will survive a DSM upgrade):

mv /volume1/@tmp/S99serviio.sh /opt/etc/init.d

Shutdown the NAS and restart, checking that Serviio starts by itself.

From now on you can start and stop Serviio manually using this same script (say if you were experimenting with changes to profiles.xml for instance):

Remote console

AcidumIrae’s PHP Web UI is only for Serviio up to version 0.5.2 at present, and at the time of writing the Java Restful Web UI by kairoh does not yet integrate with Serviio 0.6. The only option for now is to use the proper Serviio Console from another computer. Since 0.6 there is no longer a requirement to change anything at the server end, so you just need to pass the parameter -Dserviio.remoteHost=x.x.x.x to the Serviio Console to indicate the IP of your Serviio server. This is covered in the Serviio FAQ. There’s quite a lag for it to start up (at least on Windows), but eventually you see the familiar icon.

Serviio settings

Navigate to the Transcoding tab. Set the transcoded files location to /volume1/@tmp. Failure to do this will result in temporary files being written to /tmp which will fill up the partition it’s on the moment you remux a 4GB movie, which would prevent you from logging into the NAS. If your Synology has an Intel CPU set the number of transcoding threads to 2. The dual cores should take advantage of FFmpeg’s pthreads support for a decent performance boost.

I recommend de-selecting Generate thumbnails for local videos in the Metadata tab. Note that you have to click Save on each tab or any change will not take effect. As you’ll see if you have a DLNA renderer that supports thumbnails, Serviio retrieves good ones from the online databases it checks so they’re not really needed. Often FFmpeg will get stuck trying to generate a thumbnail for a video and will lock the CPU at 100% for long periods of time. This issue was raised in this Serviio forum thread.

You can use the Library tab to add the media folders you want to share. Use Add Path not Add Local (we’re using a remote console remember) and express the paths in Linux syntax, e.g. /volume1/public/videos. Be sure that the serviio user has been granted read privileges over the folders you add (User Control Panel in DSM).

Performance

The Synology seems perfectly able to transcode DTS audio in a hi-def Matroska file down to 2 channel AC3 in MPEG-TS while copying the H.264 stream. The CPU use leaps up to 100% but I guess that’s because it’s running ahead transcoding down to the end of the file (and is CPU-bound).

If I play something that’s only remuxing the container and copying both audio and video streams then the CPU stays at around 40% (because it’s I/O-bound) then 5 minutes into a film it falls away to pretty much idle – I guess it has finished remuxing to the temporary file.

Memory use even while running Serviio is around 20% at idle, 30% during a remux with audio transcode, though I have noticed that with a larger library this creeps up to around 70%. I have a 2011 product so it has 256MB of RAM. This seems to indicate that the value line of products (with the j suffix) like the DS211j should run Serviio on their 128MB. They have a 1.2GHz CPU though versus 1.6GHz on mine (and the DS110j is only 800MHz), so it would need testing by someone.

All in all it would seem that the Synology products are very capable Serviio appliances!

Uninstalling

Log in as root and stop Serviio, then in the DSM User Control Panel delete the user called serviio. Undo the changes that you made to /opt/etc/profile and /etc/profile (the LANG, JAVA_HOME and java path modifications marked in bold earlier in the guide).

Finally, re-download the bootstrap for your model of NAS and run it again. It will tell you to delete a few folders and restart. This will completely trash all optware ipkg packages (i.e. everything in /opt) and undo everything else you did in this guide without affecting your data partitions. If you had any other ipkg packages installed since installing Serviio, these would also be lost.

Serviio is an excellent free Java DLNA media server by Petr Nejedly which focuses on minimizing the amount of unnecessary media transcoding, and maximizing the use of renderer devices’ supported features. Some of the more main-stream servers like Windows Media Player just brute-force everything to MPEG2 video and MP3 audio, which degrades quality and wastes power. Though some servers like Mezzmo are better and will play Matroska files, even they tend to transcode all audio to AC-3 regardless of source type. As a Java app Serviio will run on anything that has a JVM, and the media tool it relies on is the open source and therefore highly portable FFmpeg. All these design priorities make Serviio an ideal choice to run on a NAS device since, when paired with a renderer with good format support like a Sony Bluray Player, the NAS will barely ever be transcoding.

I was about to buy a new large external hard disk, but once I realised that Serviio could probably run on a NAS I started looking at one of these instead. Synology seemed to offer a lot of value and seemed to have the sort of user-community enjoyed by my old Linksys NSLU2, which I promptly sold on eBay for almost what I had paid for it in 2007. I considered the value DS110j model but I decided to go for the more expensive DS111 on the basis that the double CPU speed and RAM would probably be a wise move.

This guide outlines how to get Serviio 0.5.2 running on the Marvell Kirkwood ARM CPUs found in most of the 2011 product line-up, but Synology devices also exist with Freescale PowerPC and Intel Atom processors. The key problem is finding a Java virtual machine, but FFmpeg also needs compiling from source. This is because although there is an FFmpeg binary bundled with DSM 3.0, it’s too old and lacks support for features critical to Serviio. This guide could be used for other CPU architectures, but the compilation options for FFmpeg need adapting.

Connect to your NAS’s IP address using SSH. Use the root account (same password as admin). I suggest that you perform the mod at the bottom of this post to enable colour directory listings and a more descriptive shell prompt which should reduce the chance of accidentally being in the wrong directory.

We need to install the development tools. Type:

ipkg install optware-devel

It will halt and complain that package wget-ssl clashes with wget. Continue with:

Update – It seems that there is a serious problem with running ipkg on a clean install of DSM 3.1. This guide was written before it was released, and though I have since upgraded my Synology I haven’t encountered that issue, but there have been many comments about it. User mayk on the Synology forum seems to have the solution here. Use the extra wget verbosity switch to find out the exact package URL for the following two packages, then manually download them with wget and install:

Next we need to install Lame MP3 encoder, providing libmp3lame which FFmpeg will be compiled to depend on, and the Nano text editor (much easier to use than vi):

ipkg install lame
ipkg install nano

JamVM is a JVM that gets mentioned a lot in connection with NAS systems, but it’s only Java 1.5, and Serviio needs version 1.6. Download the Java SE Embedded Runtime from Oracle, selecting the ARM v5 Linux version (note that there is a PowerPC e500v2 version – the CPU core in Synology products which use the Freescale mpc85x3). Unfortunately for PowerPC Synology owners, this depends on a higher version of glibc than the Synology DSM provides for this architecture. Until JamVM supports Java 1.6, or Synology update to glibc 2.4 you won’t be able to follow this guide on PowerPC models. This may have changed since DSM 3.1 was released.

You will need to sign up to receive the download link by email. It’s free to use for non-commercial self-educational use. Use your computer to save it into the top level shared folder of your NAS, which will probably be /volume1/public on the NAS filesystem. Then:

Synology’s Linux build has no localisation support built in, though it does use UTF-8 character encoding for the filesystem. That’s no problem for storage, however the Java VM inherits the locale setting of the host OS. Since this is undefined Serviio, and all other Java software, will default to US-ASCII which is a big problem if you have filenames with non-US characters. The solution is to obtain the missing files to add locale support from the Synology toolchain, which is distributed under the GPL:

In the DSM Control Panel got to Web Services > Web Applications tab > Enable Web Station.
Install AcidumIrae’s PHP web UI for Serviio. You will need to have enabled Web Station for the directory /volume1/web to exist.

You should already be able to browse to http://your_NAS_IP/serviio and see the user interface, though it will complain with a big red X that Serviio is not running.

FFmpeg depends on the libbz2 and zlib libraries, and although both are installed along with the optware-devel package, FFmpeg will only look for them in /lib rather than in their actual location in /opt/lib. Copies of the existing symbolic links will be fine:

cp /opt/lib/libbz2.so.1.0 /lib
cp /opt/lib/libz.so.1 /lib

Compile the patched version 26303 of FFmpeg from the Serviio download page. Running cat /proc/cpuinfo it is clear that the DS111 is an ARM 5TE platform so I enabled those specific optimizations:

Notice that the ./configure command is line wrapped – it’s all one command. The make command takes approximately 25 minutes on the 1.6GHz CPU and will show many warnings during compilation, but this is expected.

Update – Thanks to bakman for pointing out that for Intel Atom CPUs you will need to install the assembler YASM and also use the following ./configure parameters:

In the DSM User Control Panel create a new user called serviio and set a password. Give that user access to the paths that contain the media you want to serve. Click the User Home button and enable the User Home Service. Go back to your SSH session and type:

nano /etc/passwd

Be very careful editing this file. A wrong move here could trash your system. Notice that the serviio user has a shell of /sbin/nologin. Change this to /bin/sh like the admin user has. Nano may try to line wrap this line as you type if you added an long account description. If it does, delete the carriage return before the line break and pull it back onto one line. Save and exit.

Now we’ll create the Serviio daemon start and stop script:

nano /volume1/@tmp/S99serviio.sh

Paste in the following text (mouseover and use the icon in the top right to copy):

Check the web UI or the process list again and make sure it did indeed stop. If it’s all ok, we need to move the daemon launcher script so it starts automatically on boot:

mv /volume1/@tmp/S99serviio.sh /opt/etc/init.d

Shutdown the NAS and restart.

Update – previously I had used the directory /usr/syno/etc/rc.d, but this is destroyed when the DSM software is updated. /usr/local/etc/rc.d is the official Synology location for 3rd party daemon init scripts but I found that it doesn’t work, so I used /opt/etc/init.d instead which also survives a DSM upgrade. I just tested this by upgrading to DSM 3.1-1605 and Serviio remained intact. I only had to copy the sym links for the libraries FFmpeg needs (“cp /opt/lib/libbz2.so.1.0 /lib” and “cp /opt/lib/libz.so.1 /lib”).

The file loaded into nano will look like this, and you need to edit the IP address to match the IP of your Synology (either put your syno on a static IP, or set a reservation in your broadband router’s DHCP server options).

I can’t find a way to avoid manually specifying the IP address like this – relative paths can’t be used because the Synology runs two different webservers: the DSM one on port 5000 without PHP, and the PHP-enabled Web Station one on port 80.

Log out of DSM and log back in. You will see the Serviio icon in the pull down menu in the top left, which you can drag to the desktop if you prefer:

Serviio settings

Navigate to the Transcoding tab. Set the transcoded files location to /volume1/@tmp. Failure to do this will result in temporary files being written to /tmp which will fill up the partition it’s on the moment you remux a 4GB movie, which would prevent you from logging into the NAS.

I recommend de-selecting Generate thumbnails for local videos in the Metadata tab. Note that you have to click Save on each tab or any change will not take effect. As you’ll see if you have a DLNA renderer that supports thumbnails, Serviio retrieves good ones from the online databases it checks so they’re not really needed. Often FFmpeg will get stuck trying to generate a thumbnail for a video and will lock the CPU at 100% for long periods of time. This issue was raised in this Serviio forum thread.

You can use the Library tab to add the media folders you want to share. Note that the Add Local… button will fail because the web service user does not have access to the root of the filesystem (probably a good thing security-wise). Use Add Path… instead and express the paths as I have done in the first screenshot above. Be sure that the serviio user has been granted read privileges over the folders you add (User Control Panel in DSM).

Update – If you really want to use the Ajax file browser UI, then go to DSM Control Panel > Web Services > PHP Settings tab > open_basedir and append /volume1: to the start of that list. Then on the Web Applications tab, disable Web Station, Ok, enable Web Station. Go back to your SSH session and run:

nano /volume1/web/serviio/afb/config.php

Change the value of $path from / to /volume1/public (or the top level shared folder where your media resides). You’ll notice however that when you use Add local button in the Serviio Library tab it’s very buggy. It will only list a maximum of 5 child folders from each node.

Performance

The Synology seems perfectly able to transcode DTS audio in a hi-def Matroska file down to 2 channel AC3 in MPEG-TS while copying the H.264 stream. The CPU use leaps up to 100% but I guess that’s because it’s running ahead transcoding down to the end of the file (and is CPU-bound).

If I play something that’s only remuxing the container and copying both audio and video streams then the CPU stays at around 40% (because it’s I/O-bound) then 5 minutes into a film it falls away to pretty much idle – I guess it has finished remuxing to the temporary file.

Memory use even while running Serviio is around 20% at idle, 30% during a remux with audio transcode, though I have noticed that with a larger library this creeps up to around 70%. I have a 2011 product so it has 256MB of RAM. This seems to indicate that the value line of products (with the j suffix) like the DS211j should run Serviio on their 128MB. They have a 1.2GHz CPU though versus 1.6GHz on mine (and the DS110j is only 800MHz), so it would need testing by someone.

All in all it would seem that the Synology products are very capable Serviio appliances!

Uninstalling

Log in as root and stop Serviio, then delete a couple of libraries which had been copied to /lib:

In the DSM User Control Panel delete the user called serviio. Undo the changes that you made to /opt/etc/profile and /etc/profile (the JAVA_HOME and java path modifications marked in bold earlier in the guide).

Finally, re-download the bootstrap for your model of NAS and run it again. It will tell you to delete a few folders and restart. This will completely trash all optware ipkg packages (i.e. everything in /opt) and undo everything else you did in this guide without affecting your data partitions. If you had any other ipkg packages installed since installing Serviio, these would also be lost.