Introduction

Why Use It?

LXC is in between a VM and a pure application container. It will take up more resources than a pure container, but unlike a VM use as much memory as needed. It will provide a full kernel and OS (albeit it must be Linux). Unlike a VM, LXC container will have access to the resources and speed of the hardware without needing to emulate hardware. LXC is wrapped by Warden and the foundation for Cloud Foundry. Even if you are just interested in Docker or Cloud and not yet ready to give up your ssh console, start here. Learning LXC, you'll also really understand how Cloud Foundry, Docker and generally Cloud concepts actually works. There is also value in then using LXD to learn how to build your own bare metal, orchestrated Cloud infrastructure using technologies like Kubernetes.

What about LXD? LXD is another higher level layer on top of LXC that is more orchestration friendly.

The key enablement for orchestration is remote API calls and higher level functions. LXD still uses LXC under the hoods. As of Oct 2016, LXD 2.0 is now available and hooking up with orchestration via Juju.

This tutorial was originally written before LXD was ready for production and evolved to be the precursor to the LXD tutorial.

When an container is created (last updated with Ubuntu 16.04.1) it is tiny,

Attribute

Size

Comment

Disk Space

337 MB

That's the entire operating system because it can mount shared core kernel files from the Host. Better yet as Read-Only

Create Regular Privileged Container

The simplest to implement container is a privileged container. In other words, the user and process running the container is root. This is not as bad as it seems from a security perspective and easiest to manage (so far). Tin to talk to Dickson to find out if Solaris (which is more mature with container technology) started using unprivileged and if so will look into that next for Linux.

Container Derived from Image File Downloaded to Base OS

This draws the files from the existing image matching downloaded from the Base OS to make your container. Note the image file must be compatible (for now) with the Base OS. I observed it may also be looking at details of your Host OS as the process creates accounts in the image that matched Host OS. The image folder was created/downloaded during the lxc setup and resides in /var/cache/lxc/trusty/rootfs-amd64. The template file to create the container resides in /usr/share/lxc/templates/,

Sparse Container from Host OS

Not sure if this functionality exists yet. Solaris has a concept of a sparse root zone which I would call here sparse container. The idea to reduce scope overhead so that aside from basics (home, opt, temp) the rest of the file system is actually mounted as read-only from the Host OS. In this model we have tighter security with minimal packages.

File Structure

You will need to use sudo or root to view the directories. I use root,

In other words, containers exist within a private network, may see each other, pull network data to whatever the host can access, but nothing outside aside from the host itself will see these servers. A good analogy is your home network behind 1 public IP. Your desktops can see each other, pull data from the Internet, but the Internet cannot see your systems.

Enabling LXC DNS Resolution

By default, the containers can only see each other by IP addresses. DNS resolutions by default is not enabled.

I'm not going to show how to fix this because we will get this out of the box when graduating to using LXD.

Static LXC Assigned IP

When working with servers, without orchestration, it is best to use static IPs. This should be your first step if seriously using containers for production use. By default, LXC assigns an internal set of IPs within the range of 10.0.3.2 to 10.0.3.254 which is defined in /etc/default/lxc-net.

Change the range and enable reservations in /etc/default/lxc-net. Start by backing up per Bonsai Framework convention,

sudo cp /etc/default/lxc-net /etc/default/lxc-net.ori

First change the range (LXC_DHCP_RANGE) and the maximum lease # (LXC_DHCP_MAX) to make available 10.0.3.2 to 10.0.3.99 for static assignment,

# Uncomment the next line if you'd like to use a conf-file for the lxcbr0
# dnsmasq. For instance, you can use 'dhcp-host=mail1,10.0.3.100' to have
# container 'mail1' always get ip address 10.0.3.100.
LXC_DHCP_CONFILE=/etc/lxc/dnsmasq.conf

Create and edit the /etc/lxc/dnsmasq.conf,

# Specify static IPs per container name or mac address.
# Addresses should be outside of range in /etc/default/lxc-net (10.0.3.100 to 10.0.3.254)
# Once setup these servers will be able to see each other by hostname assigned here.
dhcp-host=web,10.0.3.10
dhcp-host=app,10.0.3.20
dhcp-host=database,10.0.3.30

Specify static IPs per container name (you can also use the container's mac address in situations of multiple network interfaces) (I have not tried or tested yet). To avoid conflicts, make sure the static IP addresses are outside of the range specified in /etc/default/lxc-net. In case of multiple ip addresses look at LXC Advanced Networking.

You need to shutdown your containers and then restart the Host OS or flush the DNS Mask,

When editing /etc/lxc/dnsmasq.conf, changes do not take effect right away.

# Tried this and it did noot work http://seminar.io/2014/07/27/dns-resolution-for-lxc-in-ubuntu-trusty/
sudo service lxc-net stop
sudo service lxc-net start

The other option is to shutdown all containers and restart the host operating system.

Make sure everything restarted fine without errors. I had run into forgetting to change LXC_DHCP_MAX which resulted in lxc-dhcp not starting,

# Search boot log for errors,
... to put here ...

The nice thing about this approach is that the servers within the private network can see each other by assigned hostname. Try pinging by hostname and it will respond correctly. In the case two network cards though, there is some routing work that I have yet to figure out.

For troubleshooting, you may omit --daemon which will run the containers as an actual program where you can see the container booting up.

Logging In

Console

Replicate loading console,

sudo lxc-console -n my-container

Defaults for the template image are,

user = ubuntupass = ubuntu

To detach, CTRL+a, followed by ‘q’.

Make to change the password for the default user or delete the default user when you create your own accounts.

Simulate Terminal

There is another way of getting in (good for initial setup without default accounts or emergencies),

sudo lxc-attach -n my-container # Takes you right to root after a fresh install.

lxc-attach is used to execute an arbitrary command inside a container that is already running from outside the container. Because nothing was specified, you end up running bash inside of the container.

If you chose to log in using ssh, you will want to log out and update your fingerprint file otherwise you will receive a "REMOTE HOST IDENTIFICATION HAS CHANGED!" error and not be able to ssh in. Different ways of doing this. On a Unix, Linux or Mac OS X operating system,

ssh-keygen -R remote-server-name-here

Mounting a Network Folder

You can't at this moment (March 2016) and I understand this is because it is a shared kernel issue. Solaris does not allow this either if I recall.

Setting Up Zero Footprint 32-bit Java on LXC 64-bit

By default, the apt-get packages inside of the the containers only use 64-bit architecture,

sudo dpkg --add-architecture i386
sudo apt-get update

Then follow-up with installing the proper dependencies, but not sure if I need to install in the host... need to try this out in a VM.

Setup Name Resolution on Host

The containers know each other by name. However, the host has no awareness. This should be resolvable by adjusting the DNS resolution. However, first check if this is an issue that's already resolved in later implementations of Ubuntu (I think 17 has it solved) and also if LXD resolves this automatically.