In every century people have thought they understood the universe at last, and in every century they were proved to be wrong.
It follows that the one thing we can say about our modern "knowledge" is that it is wrong.

- Isaac Asimov

I don’t assume I know everything. Not even that I know enough.
And no more than you ;-)
I will share some experience and facts from real life that can help us understand IT and Cloud better.
Comments welcome.

Terraform is one of the best open source tools to manage your Infrastructure as Code: it’s easy to install, learn and use (one hour).

You could start from tutorials and free examples available on Internet.

Here is an example of full automation (we'll try to get a little better result):

As a first step, to make the usage of Openstack easier on a large scale, we discussed the value of a managed service.

If the IT organization could just focus their effort on the development and operations of the business applications, instead of running the infrastructure, they would create more value for the internal customers (company's lines of business).

So I proposed the adoption of Cisco Metapod, that is Openstack as a managed service (delegation of all the tough administrative and operational work to a specialized 3rd party, while you just use the Openstack user interface and API enjoying a SLA of 99.99% uptime).

We created a lab where Openstack abstracts the resources from the physical and virtual infrastructure (etherogeneous servers, network and storage) and the configuration of different environments is managed by Terraform, so that you can create, destroy, restore and update a complex system in few minutes.

With Terraform you can describe the architecture in a declarative form (in a manifest file).

You simply describe what you need (the desired state), not how the different components (devices and software) must be configured with all their parameters and their specific syntax.

The goal of Terraform is to match the current state of the system with the desired state.

Desired State vs Current State

Terraform is used to create, manage, and manipulate infrastructure resources. Examples of resources include physical machines, VMs, network switches, containers, etc. Almost any infrastructure noun can be represented as a resource in Terraform. Terraform is agnostic to the underlying platforms by supporting providers. A provider is responsible for understanding API interactions and exposing resources. Providers generally are an IaaS (e.g. AWS, GCP, Microsoft Azure, OpenStack), PaaS (e.g. Heroku), or SaaS services (e.g. Atlas, DNSimple, CloudFlare).

Infrastructure as Code

Desired State

It contains all the resources you need to deploy a new Devstack instance (a all-in-one instance of Openstack, useful for developers) including the needed networks, public addresses, firewall rules on a target cloud platform. That, incidentally, is a Openstack instance (so we are deploying Openstack on Openstack).

Here is the content of the main.tf file used by Terraform: it references variables with the format ${variable_name}, including the output from actions on other resources. Dependencies between resources are managed automatically by Terraform. A separate file can contain the predefined values for your variables (like the references to the Openstack lab in my example).

If you are not interested in the content of this file (I guess it applies to 70% of my readers) you can skip it and go to next picture... there is also a good recorded demo down there :-)

main.tf (the manifest file where Terraform ready the desired state of all the resources):

To make it simple, for this blog post I replaced the part that deploys Devstack with a simpler setup of a web server (Apache).

deploy.sh (Terraform will copy and execute it on the remote instance as soon as it is created):

#!/bin/bash

# author: Joe Topjian (@jtopjian)

# source: https://gist.github.com/jtopjian/4ffc82bfcbbcc78d07e4

sudo apt-get update

sudo apt-get install -y -f apache2

The goal is to demonstrate how easy it is to create a new software environment on a Cisco Metapod Openstack target from scratch and run it.

The following pictures show the Metapod console before and after running the "terraform apply” command on my computer.

This is before I run the command:

The Openstack console from Cisco Metapod

And this is the expected result (network and server infrastructure created, apache installed):

Resources created in Openstack

Next video (the most important part of this post) is a recorded demonstration of the creation of the new Apache server: you can see the launch of the “terraform apply” command that, after reading the manifest file, creates a network, a subnet, a router with two interfaces, a floating ip and a instance on Openstack. Then the Apache web server is downloaded and installed in the new instance.

The Metapod console is left in the background and you see the Openstack objects pop up as they are created.

Finally the home page of the new web server is tested.

Conclusion

It is very easy to get rid of the delays, the misunderstandings and the inefficiency of many current IT organizations.

If you standardize the process that developers follow to obtain the environment for a new project - in all the phases of the life cycle - you can enable a faster go to market for new business initiatives making your customers happy.

It would be a first step towards DevOps (more is required, mostly in changing the culture of both developers and people in operations).

Infrastructure as code is a brilliant way to create the needed infrastructure on demand (and release it when no longer needed), to maintain it based on blueprints and manage the definition of the infrastructure with the same tools you use for the application source code: a text editor (or your preferred IDE), a version control system, an automation tool.

If you have a IaaS platform like Openstack, provisioning of both virtual and physical resources is made easy.

If you do a further step forward with a managed service, someone will grant that your Openstack is correctly configured for production, up to date and in perfect health. You enjoy all the benefits, without the hassle of setting it up and operating it daily.

2 comments:

As usual, you take reletively complex subject Luca and explain it in 'clear text' for all of us to understand better.

One thing I am starting to see more frequently is the use of Ansible/Chef/Puppet etc to complement the Terraform deployed virtual infrastructure so as to layer the App/SW Config on top and also manage it in a declaritive way.A great combination!

One question I have is...Q: Does it make sense for Terraforms to deploy the actual h/w if an appropriate API is available (Cisco UCS?). I do not see anyone writing a Provider for this use case today (which makes sense as VMs still rule currently if we are honest) but Containers are certainly on the way and the Terrforms deployment directly onto Bare Metal could have a relevance in the near future.

Sure, ansible is much more powerful when it comes to the provisioning of the software stack.In fact, taking mantl.io as a brilliant example, both Ansible and Terraform are used in conjunction by the mantl installer (for those that don't know mantl, it's a open source project to manage a microservices infrastructure and it makes your life much easier than dealing with Docker, Terraform, Mesos, Kubernetes, ELK, etc. directly).

With reference to the UCS servers and their API, Container Solutions have already created a Terraform provider for UCS, see http://container-solutions.com/tag/mantl/ and they incorporated it in the mantl installer. It's been used in our lab in Amsterdam.

Thank you for following and contributing to the share of experience on my blog :)