Introduction

Open-Xchange is happy to release a first set of infrastructure cookbooks for the Chef automation platform. These cookbooks will help you to automate the configuration, deployment and management of your OX components across your network.

Install Requirements

For this tutorial we are assuming that you are running a Debian based Linux distribution. If your Operation System isn't Debian based the installation instructions will differ slightly depending on the package manager of your Operating System but you should still be able to follow and understand the overall approach.

The demo deployment covered in this tutorial neither reflects our deployment recommendations nor does it result in a supported installation of OX App Suite but its merely a convenient vehicle to show you how to use the provided infrastructure cookbooks for different deployment scenarios.

Before we can see the new cookbooks in action we have to provide some required tooling to fetch and use the cookbooks to automatically set up a new demo OX App Suite in a virtualized environment.

VirtualBox

VirtualBox is a hypervisor that allows the creation and management of guest virtual machines running Linux, Windows or other operating systems. We will use it to quickly generate virtual machines based on Debian Jessie or CentOS 7 which are supported platforms of OX App Suite.

+marens@openxchange:~$ sudo apt-get install virtualbox

Vagrant

Vagrant is a kind of wrapper around virtualization solutions like VirtualBox or VMware that presents a uniform interface for managing virtual machines. In our setup it will be used to create, configure and destroy the machines that are needed to set up the OX App Suite showcases or in general to work with chef/kitchen to test our cookbooks.

+marens@openxchange:~$ sudo apt-get install vagrant

To guarantee that upcoming versions of our default distro boxes are always usable with vagrant we have to make sure the virtualbox guestfs is usable. As this isn't always given after updates in the future we install a helpful vagrant plugin via:

+marens@openxchange:~$ vagrant plugin install vagrant-vbguest

ChefDK

The Chef Development Kit is a handy collection of tools that we'll need to make use of the cookbooks provided by Open-Xchange and set up a demo App Suite. Later on you can use it to develop your own specialized deployment cookbooks which are tailored to your needs.

To install it please go to www.chef.io and download/install the appropriate chef-dk for your operating system.

You see that we installed several new components with the chef-dk. Some of them will be used in the upcoming steps of this tutorial: test-kitchen, kitchen-vagrant and berkshelf so let's make the new tools known to our running shell.

eval "$(chef shell-init bash)"

If you don't want to do this manually every time you want to work with chef/kitchen you can add this to the startup scripts of your shell, see the chef docs for more infos.

Deployment showcases as cookbooks

Get from git

Update from git

When future changes from our side modify cookbook dependencies you should ensure that you are using the latest cookbooks (and dependencies) specified in the showcase deployment after pulling from git. The later point Kitchen details will explain what berks does and where exactly it comes into effect.

The platform images are automatically downloaded from atlas.hashicorp.com to ~/.vagrant.d/boxes/ on your workstation. At atlas.hashicorp.com you can search for other images in case one of those mentioned in this showcase is too outdated or no longer available.

This file is part of .gitignore so that passwords won't be added to your git when start developing your own deployments based on our infrastructure cookbooks.

Community showcase

Use this if you don't have a valid license key for Open-Xchange products and simply want to install the free components of OX App Suite. Compared to the customer showcase you'll be missing a working documentconverter so there won't be any document previews, thumbnails or conversion from proprietary document formats but otherwise the same stack of components will be installed and configured.

Deploy

for now we'll focus on the debian platform but feel free to retry the steps outlined here with centos. Depending on your choice you'll have to use either showcase-community-debian-jessie or showcase-customer-debian-jessie as last parameter of the kitchen command line tool.

Create showcase VM

This will take a while for the first run as vagrant will have to download the debian-jessie image which will be used as base for the virtual machine we want to install our showcase in.

After the kitchen converge run finished successfully we have a working OX App Suite setup as show below.

The chef client:

installed the required backend services

installed the ox middleware including ox documents

set up the ox guard server against the middleware

set up the apache http server/proxy in front of our middleware components

Login

SSH

You can easily login via ssh to take a look at the configured system.

+marens@openxchange:singlenode$ kitchen login showcase-community-debian-jessie
The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
You have mail.
Last login: Mon Nov 30 16:03:34 2015 from 10.0.2.2
vagrant@singlenode:~$ sudo su
root@singlenode:/home/vagrant#

HTTP

As we forwarded the ports 80 and 443 from the VM to ports 8080 and 8443 on our host you can easily login to your new OX App Suite with your browser via http or https. For https you'll have to manually trust the certificate we automatically created during the kitchen converge phase.

Some demo logins are:

user1@context1.example.com

user2@context1.example.com

user3@context1.example.com

user4@context1.example.com

They all share the password: secret

Destroy

Kitchen details

We already saw that Kitchen offers a nice level of abstraction that lets you easily create environments to execute and automatically test your cookbooks without too much manual preparation but what happened behind the scenes during the kitchen converge run?

A dna.json file gets created. This file is based on the entries of the matching showcase entry in .kitchen.yml. It acts a starting point for the chef client as the run list entry contains all needed infos to configure a machine into our desired state. Additionally this file may contain further entries like e.g. the ldb credentials for our customer showcase.

Berkshelf takes a look at the cookbook dependencies specified in Berksfile and metadata.rb, builds a dependency graph and downloads the missing dependencies. Those can be other open-xchange cookbooks from git or other third party cookbooks.

Data Bags needed for our deployment are collected. Data Bags are general purpose data containers that are indexed for fast lookups.

Environments specified to represent different workflows are collected. In the directory environments you'll find separate environments for our customer and community showcases.

The Roles we have defined for our deployments are collected. Roles are a way to assign attributes and features to a machine in our deploymnet environment. When you look into 'roles/singlenode_showcase.rb' you can inspect the role that we are using for our showcases. These define e.g. that the target machine will be running ox documents and guard in addition tothe default groupware components.

A file called solo.rb is created. This is the main configuration file for the chef client. It tells him where to find the cookbooks dependencies that Berkshelf computed, the Data Bags, Environments and Roles we defined.

Cookbooks

Deployment cookbooks

singlenode

The singlenode deployment cookbook used in this tutorial describes our simple deployment showcases that should help you understand how deployment and infrastructure cookbooks work together and give you an example how our infrastructure cookbooks are used in detail.

I'll briefly explain the deployment cookbook before i let you dive into the code itself to start exploring and experimenting.

As already explained above the first thing that the chef client needs to get started is the run_list entry from dna.json. This run_list instructs the client that the machine should be assigned the role role[singlenode_showcase]

Looking into roles/singlenode_showcase.rb you find yet another run_list containing several recipes that should be executed. After they are executed successfully the machine will fulfill the associated role. The format of those recipe entries is cookbook_name::recipe_name

As example recipe[singlenode::documents] instructs the client to run the recipe located at cookbooks/singlenode/recipes/documents.rb which then causes the OX Documents feature to be installed by including further recipes open-xchange-documents::default and singlenode::default

Let's have a quick look at the cookbooks/singlenode/recipes/default.rb

You'll notice a pattern where configuration attributes for this machine or node are read

if node['open-xchange-apache']['ssl']['enable']

or set

node.default['open-xchange-apache']['ssl']['cert'] = cert.cert_path

These attributes can be set before the chef client run depending on your needs and how your infrastructure is set up. We did this for our showcases via environments. If you have a look at e.g. environments/community.rb you'll find

Infrastructure cookbooks

Our infrastructure cookbooks are freely available via https://code.open-xchange.com at /infrastructure/cookbook/ox. Below you'll find a quick overview of the infrastructure cookbooks used in this demo deployment with direct links to the configuration attributes.

open-xchange-apache

This cookbook provides a complete Apache configuration as webserver for Open-Xchange. Its definitions and recipes reduce the necessary deployment wrapper cookbook code to its minimum.

Local cookbooks

If you want to start developing your own infrastructure cookbooks or want to modify our existing cookbooks you can do so by e.g. checking out one of our infrastructure cookbooks from code.open-xchange.com and making it know to Berkshelf. As explained above Berkshelf will automatically pull our infrastructure cookbooks from code.open-xchange.com for you. This is configured in Berksfile

If you you wanted to modify e.g. open-xchange-backend-core you'd have to:

Clone it from git (best into the relative path already defined in config/default.rb so you don't have to do additional modifications)

Tell Berkshelf about it by overriding the location from git to path in config/local.rb

$oxconfig.cookbook_locations["open-xchange-core"] = "path"

and updating your Berkshelf

berks update

From then on the modified cookbook will be uploaded to your vagrant virtual machine and used during your next kitchen converge run. The same approach can be used for additional cookbooks you might want to create for your deployment scenarios.