Running Nextcloud in a jail on FreeBSD

Jun 5, 2017

I configured Nextcloud inside a FreeBSD jail in order to allow me access to files i might need while at University. I figured this would be a optimal solution for files that I might need access to unexpectedly, on computers where I am not in complete control. My Nextcloud instance is externally accessible, and yet if someone were to get inside my Jail, I could rest easy knowing they still didn’t have access to the rest of my host server. I chronicled the setup process including jail setup using iocage, https with Lets Encrypt, and full setup of the web stack.

Updated on Febuary, 23rd, 2018

Login screen for Nextcloud.

Nextcloud has a variety of features such as calendar synchronization, email, collaborative editing, and even video conferencing. I haven’t had time to play with all these different offerings and have only utilized the file synchronization, but even if file sync is not needed, Nextcloud has many offerings that make it worth setting up.

On my latest install I am not making use of the file synchronization at all, I’ve attached external storage over NFS and am synchronizing the files using Syncthing. This lets me use my preferred sync client, but I’m still able to access all my files from the web interface or Nextcloud client. So far this has been the configuration I am most satisfied with.

Security

Given that I wanted my Nextcloud install to be accessible from the internet, security was paramount as a consideration when planning the set up of my ‘cloud’. In order to maintain network security, and at the same time allow my Nextcloud to be open to the internet, I utilized a combination of tools.

My server that houses the install is in my DMZ, where very limited access is given to the rest of my network. I also put Nextloud inside a FreeBSD jail which is incredibly convenient. My Nextcloud instance is externally accessible, and yet if someone were to get inside my Jail, I could rest easy knowing they still didn’t have access to the rest of my host server. While I obviously wouldn’t rely on this fact alone, it does increase security greatly. It also means I can give the jail even less access to my network than my hosting server has.

Configuration

I would have liked to use Nginx and Postgres, but the majority of the documentation for setting up Nextcloud is for Apache - with only a small mention of Nginx, and no mention of Postgres. Not being an expert in either of the areas of database or webserver administration I went with the recommended settings.

jail Management

To manage my jails I’m using iocage. In terms of jail managers it’s a fairly new player in the game of jail management and is being very actively developed. It just had a full rewrite in Python, and while the code in the background might be different, the actual user interface has stayed very similar.

Iocage makes use of ZFS clones in order to create “base jails”, which allow for sharing of one set of system packages between multiple jails, reducing the amount of resources necessary. Alternatively, jails can be completely independent from each other; however, using a base jail makes it easier to update multiple jails as well.

A fork of the old version written in shell script, iocell can be installed from the FreeBSD ports tree as sysutils/iocell, or as a package with pkg install iocell.

The new version written in Python can be installed from the ports tree as either the Python2.7 version sysutils/py-iocage, or the Python3 version with sysutils/py3-iocage. Binary packages only exist for the Python2.7 version so to install the package run pkg install py27-iocage. As of this moment the Python 2.7 version is fairly old, so if using ports is a possibility, or you’re able to compile your own binary packages, the Python 3.6 version is more up-to-date.

NOTE Regarding FreeNAS:

Due to using a different location for the pool mountpoint, i.e./mnt/<zpool> - zfs datasets can’t be delegated into an iocage jail on FreeNAS. I too ran into this issue while trying to use ZFS mounts on FreeNAS with iocage.

The solution I use, which is in fact how FreeNAS mounts datasets into the old style jails, is to use nullfs mounts. They’re fairly simple to use. While I didn’t intend this article for FreeNAS, and I can’t promise it will work since I haven’t tried it myself, it seems like a lot of people are trying to use Nextcloud on it, since it’s been mentioned to me a few times now.

I have added a note at the bottom regarding how this is done, and if you’re using FreeNAS you should probably take a look at it before continuing.

Setup iocage

Enable iocage to autostart at boot:

[root]# sysrc iocage_enable=YES

Fetch the freebsd jail template.This will create the following datasets and then fetch and extract the requested release.

--name stratus - Set a name for the jail to make it easier to refer to.

-r 11.0-RELEASE - Specify a release.

jail_zfs=on - Allow any assigned datasets to be controlled by the jail.

vnet=off - Do not use VIMAGE, a shared IP will be used instead.

ip4_addr="sge0|172.20.0.100/32" - Assign an interface and IP address.

boot=on - Start jail at boot

The properties can be either assigned one by one or all at once during the creation phase They will be explained in the following sections.

To view the Current properties run iocage get all stratus.

Shared ip

VIMAGE allows a virtualized NIC to be used inside of a jail, but it isn’t quite stable. Since having a virtualized interface isn’t a necessity with Nextcloud, I have turned VIMAGE off and used a shared IP instead.

[root]# iocage set vnet=off stratus

In order to use a shared ip, add it with ifconfig.

As the man page mentions, a different netmask then what is in use for the existing IP must be used.

alias Establish an additional network address for this interface. This
is sometimes useful when changing network numbers, and one wishes
to accept packets addressed to the old interface. If the address
is on the same subnet as the first network address for this
interface, a non-conflicting netmask must be given. Usually
0xffffffff is most appropriate.

So if the original network is /24, a netmask of 255.255.255.255 should be used.

For an interface sge0, with a target ip 172.20.0.100 run:

[root]# ifconfig sge0 172.20.0.100 netmask 255.255.255.255 alias

Don’t forget to add it to rc.conf so that after reboot the IP is re-enabled.

ifconfig_sge0_alias0="inet 172.20.0.100 netmask 255.255.255.255"

iocage and ZFS

iocage integrates well with ZFS. You may have noticed the command syntax is very similar to the syntax used by ZFS.

It’s possible to ‘jail’ a dataset which gives the jail control of any jailed datasets.

[root]# iocage set jail_zfs=on stratus

Start the jail

With the setup out of the way the jail can be used.

Start the jail:

[root]# iocage start stratus

Tell iocage if you want the jail to start at boot:

[root]# iocage set boot=on stratus

Drop down to the jail’s console:

[root]# iocage console stratus

Inside the jail the process should now be similar to the setup on a regular server.

Storage

I have chosen to provide storage to the Nextcloud Jail by mounting a dataset over NFS on my host box. This means my server can focus on serving Nextcloud and my storage box can focus on housing the data. The Nextcloud Jail is not even aware of this since the NFS Mount is simply mounted by the host server into the jail. The other benefit of this is the Nextcloud jail doesn’t need to be able to see my storage server, nor the ability to mount the NFS share itself.

Using a seperate server for storage isn’t neccesary and if the storage for my Nextcloud server was being stored on the same server I would have created a ZFS dataset on the host and mounted it into the jail. I show how to do this in the next section.

Database

Next I set up a dataset for the database and delegated it into the jail. Using a separate dataset allows me to specify certain properties that are better for a database, it also makes migration easier in case I ever need to move or backup the database.

Make sure the jail has access to the jailed dataset.

[root]# iocage set jail_zfs_dataset=data/jails/stratus/data stratus

Set the mountpoint to none, this way it can just be manipulated in the jail.

[root]# iocage set jail_zfs_mountpoint='none' stratus

Create the dataset that will be used in the jail. I keep my different jail’s datasets separate from iocage organized in vault/data/jails/<jail name> so I put my database in vault/data/jails/stratus/data/db.

[root]# zfs create vault/data/jails/stratus/data/db

Since MySQL uses it’s own cache, it isn’t necessary to cache both metadata and data in the ARC. Set the dataset to only cache metadata.

Separate Datasets

If you need more data sets they can be created in the jail under the jailed dataset vault/data/jails/stratus/data.

If you wanted to use a separate dataset for your files in nextcloud, and you’re not storing your data on a separate server like the example above of using the NFS server, you would simply create a new dataset.

While my preferred setup would be PostgreSQL, and Nginx, the majority of the documentation is based on the recommended setup. It would be nice to see these options better supported in the future. As of now there is no ‘official’ documentation for Nginx.

Confgure Challenge

The common way to acquire a certificate involves creating a challenge in the webroot of a server.

This can be done by specifying the webroot and domain to issue a certificate for.

[acme@stratus]$ acme.sh --issue-d <domain> -w <webroot>

At the time of setting up my server, it was not exposed to the internet. I had recently configured certificates for unexposed services using dns validation. Due to my familiarity, and the convenience of not needing to have a server accessible from the web, I issued my challenge this way.

To use DNS validation I used cloudflare - which along with a few other DNS providers has an api which supports a DNS challenge. I found my API key on my web dashboard.

Redis

Note: in order to use pecl-redis with php71, pecl-redis needs to be built with php71. It defaults to using php56 which is how it is built in the package. I built a package using poudriere with my required compile options.

Nextcloud Server Configuration

Create a file for nextcloud under /usr/local/etc/apache24/Includes/. I put mine in /usr/local/etc/apache24/Includes/<domain name>.conf

If you set https earlier you can use the following virtual host. I used HSTS here which means the server will always expect to use an https certificate, and removing it in the future will be difficult. If you are considering removing one at a later date, make sure you know how HSTS works.

Add the following virtual host replacing the ServerAdmin email, and ServerName domain name.

Post Setup

That should be the end of the set up. Some things you might want to do now are look through the administrator settings and enable some defaults depending on your use case. You’ll probably also want to create some users and setup email so that the Nextcloud server can contact you. I recommend using SMTP for simplicity.

Nextcloud file browser.

SMTP Email

Install msmtp to send email via SMTP. Since SSMTP was depreciated i’ve been using msmtp.

pkg install msmtp

Create a config file for the system in /usr/local/etc/msmtprc, or user in $HOME/.msmtprc.

Manual Upgrade

I’ve had problems with the WebUI updater and it has always failed on me. I do the upgrades manually from the commandline, but it doesn’t hurt to try the WebUI updater since it can be completed from the commandline if it fails.

This is assuming the name of your jail is tank, the path you want to mount is located at /mnt/tank/data/jails/stratus/data/db (it must be mounted here on the host), and the location you want to mount it to inside the jail is /var/db/mysql. You would then not be using any ZFS commands in the jail at all, you would do all the manipulation of datasets on the host, including setting of any properties you want.

Before doing this you’re also going to want to unset jailing of datasets, iocage set jail_zfs=off stratus.

You can repeat the command for any other datasets you want to mount. Use the full path from the host, and the full path you want it to be mounted to inside the jail.

If you get a message about the datasets being jailed if you try to manipulate them, you might just want to destroy all the datasets and then recreate them - assuming you don’t have anything in them.