Setting Up a Home Server with Ubuntu 11.10 – Part 3

Introduction

I am continuing a series of blog entries documenting how I set up my home server. Part 1 contains a description of what I am trying to accomplish and instructions for doing the initial install. Part 2 explains how to set up MythTV. In this part, I set up some miscellaneous components.

Automatic Updates

I like to set up the server so that security updates are automatically installed. And I get notifications of non-security updates, which I manually install.

First, let’s setup the unattended-upgrades package which will do our automatic updates:

sudo apt-get install unattended-upgrades

Set up the actions that are performed automatically, and set how often they are performed:

sudo $EDITOR /etc/apt/apt.conf.d/10periodic

Set the APT::Periodic::Download-Upgradeable-Packages line to "1". This will download updateable packages daily. Then, when you do manual updates, you will not have to wait for them to download.

Set the APT::Periodic::AutocleanInterval line to "7". This will do an auto-clean every week. This cleans up packages that are no longer being used.

Add a line:

APT::Periodic::Unattended-Upgrade "1";

This means automatic updates will be performed daily.

Configure the automatic updates:

sudo $EDITOR /etc/apt/apt.conf.d/50unattended-upgrades

The Unattended-Upgrade::Allowed-Originsblock enables specific types of updates. Updates that are commented out will not be automatically run (this doesn’t affect manual updates, though). I make sure that this line is the only one enabled:

"${distro_id} ${distro_codename}-security";

This enables security updates. If you are adventurous, you can uncomment other lines to enable other updates. These other updates are commented out because they are more likely to break something – they are best done manually so you can batch them together and plan them appropriately.

Uncomment this line so that unattended-upgradescan email you:

Unattended-Upgrade::Mail "root@localhost";

Now let’s set up apticron to send notifciations of pending updates. This is how I know what non-security updates I need to manually apply:

sudo apt-get install apticron

sudo $EDITOR /etc/apticron/apticron.conf

By default, apticron uses the results of /bin/hostname --all-fqdns to get the host name. On my server, this returns nothing (I’m guessing because it is trying to do a reverse lookup and I don’t have a valid domain). So I need to uncomment this line:

SYSTEM="foobar.example.com"

Instead of hardcoding a hostname, though, you can use the hostname command by changing the line like this:

SYSTEM=`hostname`

By default, you will get daily notifications. I don’t want notifications that frequently. As far as I can tell, this interval is hardcoded in the script itself, so I change the script:

And change -mmin -1381 to -mtime -X, where X is the number of days to wait between emails. I set mine to -mtime -7, which means weekly emails.

The apt-listchanges package gets installed with apticron. This causes the apt-get dist-ugprade command to display all updates, with descriptions. It makes you hit a key after each screen, which can take a while. Let’s change it:

sudo $EDITOR /etc/apt/listchanges.conf

Comment out this line (put a “#” in front of it):

frontend=pager

And add this line:

frontend=text

This means that apt-get will still print out the changes, but won’t pause after each screen full. apt-listchanges will still email you the changes that were applied, so you will have a record. More options are listed here.

BOINC

BOINC is a program that allows you to take on slices of distributed computing projects. There are many projects that run on BOINC – I use World Community Grid. I also used to run GIMPS, which has its own client program to install. Keep in mind that these programs will heat up your CPU, drive up it’s power usage, and possibly shorten it’s life. It will also act like a heater. Although during cold weather, this can be a good thing – if your going to run a heater you might as well run one that has additional benefits.

Run the setup utility. Note this pops up a graphical window, which means you need to SSH into your server with X11 forwarding enabled (see Part 1). Note that on my server, I get some garbled windows. I’m not sure if this is just my computer, or if everyone has the same problem. But I am able to get through.

Start the manager.

sudo boincmgr

I select “Add project”.

I select World Community Grid.

I login using my World Community Grid account.

Finish.

I still have a window open, so I close it out (it is garbled, I can’t tell what it says). I have a bunch of errors on the command line about not being able to load images, so I have to Ctrl-C to kill it. But when I start boincmgr back up, the window is no longer garbled,a nd all seems well.

When you go into the manager, you can monitor progress and change preferences. Even without the manager running, BOINC stays in the background. You can see by running this:

top

But, don’t worry, it is supposed to throttle back when you are using the CPU for other tasks and isn’t supposed to interfere.

After a while, I can see results by logging on to the World Community Grid webpage.

Hardware Monitoring

I like to be able to check the CPU temperature, especially since I know it will be running hot from BOINC. So, I set up the lm-sensors package, which not only gives CPU information, but gives other hardware information as well. You will be accessing hardware and loading kernel modules, which can be a bit risky. Skip this if you don’t want to take the risk:

sudo apt-get install lm-sensors

To view the current sensor data:

sensors

This will only show sensors for which drivers are already installed.

To probe the hardware and find out what additional drivers should be installed (be careful since this probes hardware – it will prompt you before performing tests, and give you an idea of how risky it is):

sudo sensors-detect

I answered YES to all questions, except for the one asking if you would like to automatically update /etc/modules. I prefer to do that myself.

You will be given you a section between the “cut here” lines that you can copy and paste into your /etc/modules file (reboot once you are done for them to take effect). You can also use modprobe to immediately load modules temporarily for testing (replace modulewith the module name):

sudo modprobe module

BitTorrent

Transmission

I use Transmission as my BitTorrent client. Transmission has a web client with which it can be managed. This makes it ideal for running on a server. Let’s set it up:

sudo apt-get install transmission-daemon

Add your admin user to the debian-transmission group:

sudo adduser adminuser debian-transmission

Log out and back in for the group change to take effect.

Stop transmission. You always need stop it before changing the settings.json configuration file – when Transmission shuts down, it rewrites the file with the original settings it loaded when starting up.

sudo /etc/init.d/transmission-daemon stop

Edit the configuration file:

sudo $EDITOR /var/lib/transmission-daemon/info/settings.json

Changing the rpc-whitelist-enabled entry to false will allow you to access the web client from any PC. This will allow you to access the website from any computer. Alternatively you can grant access to individual PCs by adding them to the rpc-whitelist entry.

The rpc-username entry contains the userid you use to log into the Transmission web client. It defaults to transmission.

The rpc-password entry contains the encrypted password. It defaults to transmission also. To change the password, type a plaintext password in between the quotes – when transmission starts it will automically encrypt it for you.

The rpc-port entry sets the port on which the web client listens. It defaults to 9091.

Start Transmission back up:

sudo /etc/init.d/transmission-daemon start

Go to the transmission web client at:

http://serverip:9091

Log in.

You can set some of the preferences from within the web client, which will automatically be applied in the configuration file for you. I suggest that you at least put some limits on the upload and download speeds.

I like to set up a separate partition for the torrent downloads. I assume you haven’t download any torrents into the downloads folder yet:

Create an LVM partition as described in Part 1. I call the partition torrentdownloads.

Make a note of the permissions on the existing Transmission download folder:

ls -dl /var/lib/transmission-daemon/downloads

sudo $EDITOR /etc/fstab

Add this line to mount the new partition in Transmission’s default location (replace the <tab>’s with actual tabs):

SSL with stunnel

Transmission does not support SSL internally (i.e. you need to use an http URL and cannot use https). So, if you want SSL, you need to use a reverse proxy server. Ultimately, it sounds like using the Apache web server would be a good idea. But for now, I will use stunnel. This will accepts https requests on a port of your choosing, and forwards the requests to transmission on its http port (the traffic is internal to the server, so it is supposedly secure).

Create a certificate – you need to generate a .pemfile. I generate a self-signed certificate (the guides I mentioned earlier explain how to do this). Note that with a self-signed certificate, you will need to set up your web browser to accept the certificate. Here is the command I use to generate the self-signed certificate:

With https, transmission is finicky about the URL, so it needs to be exact (see this thread).

Now you can make the 9091 port inaccessible from outside the server:

sudo /etc/init.d/transmission-daemon stop

sudo $EDITOR /var/lib/transmission-daemon/info/settings.json

Change rpc-whitelist-enabled back to true.

rpc-whitelist should be set to 127.0.0.1

sudo /etc/init.d/transmission-daemon start

Try the

http://serverip:9091

link again – it should give you a “Forbidden” error.

Web Server

I like to set up my web server so that it has two ports:

Default http port 80 – I do port forwardng on my router to expose this externally. And then I access it using a domain name from DynDNS Remote Access – this gives me a domain name I can use to access my router/server. My router is set up to automatically update the DNS entry with the latest IP address. I don’t use the public web server often, but I like to have it ready for when I do need it.

A second private http port – only accessible within my network.

Let’s install the web server (if you installed MythTV earlier in Part 2, then this is already installed:

sudo apt-get install apache2

Check it out at this URL:

http://serverip

You should get Apache’s “It Worked!” webpage.

But, if you have MythWeb installed, instead it will redirect to the MythWeb web client. Let’s disable it and restore the default site – this will give us a baseline to start from:

Disable the MythWeb site:

sudo a2dissite default-mythbuntu
sudo a2dissite mythweb.conf

Enable the defautlt site:

sudo a2ensite default

Reload the settings:

sudo service apache2 reload

Check the site again, it should be the default again. You may need to delete your temporary internet files, though, before it will work.

Now let’s disable the default sites and re-arrange things.

sudo a2dissite default

Make sure no other sites are enabled – /etc/apache2/sites-enabled should not contain anything:

ls /etc/apache2/sites-enabled

Let’s re-arrange the /var/www folder. I make a different folder underneath /var/www for each port. I got this idea from VirtualHost examples where a directory is created for each virtual host.

Create the public folder:

sudo mkdir /var/www/public

Put whatever you want in the public folder, or just put the default index.html there like so:

You may get an error from Apache: “apache2: Could not reliably determine the server’s fully qualified domain name, using 127.0.0.1 for ServerName”. To fix this problem, do the following (as described here):

sudo $EDITOR /etc/apache2/httpd.conf

Add this line (replace xxx with your server name):

ServerName xxx

sudo service apache2 restart

You shouldn’t get the error anymore.

Git

I use Git as my source control software. This is probably only something that software developers would be interested in. I set up several pieces of server software used to interact with Git.

Gitosis

Gitosis not only allows you to clone repositories, but also allows push your changes back to the repositories. This wiki has a pretty good summary of what it does. I used the Ubuntu Git Community Documentation to set it up.

Let’s install it:

sudo apt-get install git-core gitosis

I like to create a spearate partition for the git repositories. I originally tried creating the partition for the the /srv/gitosis/repositories folder. But, Gitosis was getting confused with the lost+found folder that is created inside a partition. So, I create the partition for the /srv/gitosis folder instead.

Create the LVM partition as described in Part 1. I name the partition gitosis.

Mount the new partition in a temporary location (we need to copy the Gitosis folder structure in the new partition):

Let’s install it. I assume you already have Gitosis installed and are using the /srv/gitosis/git folder to store your repositories.

sudo apt-get install git-daemon-run

Edit the script that starts the daemon:

sudo $EDITOR /etc/sv/git-daemon/run

Change base-path from /var/cache to /srv/gitosis/git.

Remove the /var/cache/git argument at the end

If you want to make all repositories available, add --export-all option to the git-daemon command line.Otherwise, you need to create a git-daemon-export-okfile in each repository you want to make available:

sudo touch /srv/gitosis/git/myrepository.git/git-daemon-export-ok

Now kill the daemon and it should respawn with the new settings:

sudo ps -A | grep git

Run this with xxx replaced by the pid of the daemon:

sudo kill xxx

Now you can test it out by cloning a repository.

ViewGit

ViewGit is a web client that allows you to browse your repositories with your web browser. The Ubuntu Git Community Documentation has a section about installing it. My instructions differ a somewhat, but they are pretty similar. Let’s install it:

Download and extract the viewgit tar file into your home directory. ViewGit is at version 0.0.6 as of this writing.

I’m still learning about this highlighting option, so I’m not sure what additional configuration should be done (.e.g. the $conf['geshi_line_numbers'] setting).

Move ViewGit into your web folder and set permissions. I assume here that you have a private web folder set up like I did under the Web Server section.

sudo mv viewgit /var/www/private

sudo chown -R root:root /var/www/private/viewgit

Add the www-data user to the gitosisgroup – this ensures apache has access to the gitosis folders:

sudo adduser www-data gitosis

Now test ViewGit by going to the following URL:

http://serverip:9000/viewgit

Disk Usage Alerts

I like to set up a job to monitor disk usage and email me if it gets low. I use a Perl script based on the one in this blog. I posted my version in the comments section of the blog. I assume you will be using this script.

Place your script in a file named /etc/cron.daily/diskspacecheck.

In order to use the Perl dfcommand, you need to install the Perl diskspace package:

sudo apt-get install libfilesys-diskspace-perl

Make the script executable:

sudo chmod +x /etc/cron.daily/diskspacecheck

Keep in mind that this Perl script can only check filesystem sizes (i.e. those shown when you run the df command). One way you can test the script is by adjusting the thresholds such that they will trigger an email.

What’s Next

I’m done for now, but there are still some things I want to set up:

Web search capability of my file shares. I have been looking into using Regain.

2 Responses to Setting Up a Home Server with Ubuntu 11.10 – Part 3

Thanks a lot! I’m glad you like it – it took a long time to put these together.
I see that you were writing a similar blog entry at the same time. Some interesting things on there that I wasn’t aware of. And I like your OpenSSH post.