In every century people have thought they understood the universe at last, and in every century they were proved to be wrong.
It follows that the one thing we can say about our modern "knowledge" is that it is wrong.

- Isaac Asimov

I don’t assume I know everything. Not even that I know enough.
And no more than you ;-)
I will share some experience and facts from real life that can help us understand IT and Cloud better.
Comments welcome.

Pages

October 12, 2016

As described in my previous post about Terraform, the deployment environment for a new application can be created "on demand" by configuring physical and virtual resources.

Good open source products allow to describe the desired state and to automate the setup of a target infrastructure.

They can also deploy your software application and configure it properly.

But is some use cases this is not enough.

You might want to offer your users - depending on their needs and their skills - a visual catalog in a web portal.

You might want to apply a governance model based on policies, use different clouds as possible targets for the deployment, offer a easy way to manage the life cycle of the deployment (start, stop, scale up/down, terminate) and get reports on usage of the resources.

modeling the deployment of a software stack (creating a template or blueprint for common deployments) and

brokering cloud services (different resource pools available from a single catalog).

A easy to consume (and manage) self service catalog

Open Source or commercial products?

In the same
project where I used Terraform to deploy Apache on Openstack, I also
used Cisco Cloud Center to deploy a portal application on Openstack.

But at
the same time, I offered the possibility to target the same deployment to a
public cloud (AWS in this particular case) or to the private cloud
(choosing between Openstack and vmware in this particular case). No duplication of the effort was needed, because the model you creat is not referred to a specific cloud as a target. It will be matched, when a user orders it, with one of the cloud avaliable for him or for his project.

So I was
able to show the difference between a free, open source solution
(Terraform) and a commercial product (Cloud Center) in a similar
scenario.

The second option addresses different needs of the organization and offers a richer solution.

It’s up to you to evaluate which one fits your requirements better.

Modeling, policies and multitenancy

One of the
differences is that Cloud Center offers a graphical editor to model the
topology and the dependencies among all the building blocks of your
deployment.

You have
a library of services (software applications from a repository,
physical and virtual services like load balancers and firewalls).

Services can be dragged and dropped in the editor, then you set their properties and dependencies. The architecture of the application you're modeling can be based on a single server or a number of servers with different roles.

If the
application architecture has multiple tiers, every tier gets its own
attributes and policies: as an example, you can set the minimum and
maximum number of instances in a cluster of web servers (or application
servers or database servers).

Autoscaling
policies will tell the orchestrator to increase or decrease the number of servers based on metrics like consumption of cpu or memory, inbound/outbound
traffic, etc.

Everytime
the cluster changes, the orchestrator will modify the configuration of
load balancers and firewalls accordingly: no manual intervention is
needed.

Models are saved in the catalog and offered to users in a multitenant organization: every tenant is given a portion of resources (target cloud environments) and services (models available in the catalog to deploy applications) that the tenant administrator can offer to his own users and groups... and sub tenants. Every tenant cannot see other tenants' stuff.

A graphical editor to model blueprints for application deployment in hybrid cloud

Dashbooard and Reporting

Every user has a dashboard that shows the consolidated information about all the applications he has deployed (or the other users in the same tenant), and can manage the lifecycle of all the deployments.

Of course the administrator of the system sees the global view including all the assets.Active VM per cloud and per application are shown in the dashboard, as well as associated costs.

Cloud Center's Dashboard

A powerful reporting features allows to filter deployments and costs by user or group, application, environment and cloud.Data can also be exported in different formats, to be consumed by humans and other systems.

Unified reporting

Architecture

The architecture of the Cloud Center product is based on two Virtual Machines: the Manager (CCM) and the Orchestrator (CCO).

The Manager is the engine where policies and application models are defined, and where the user portal runs. The Orchestrator lives within each of the target clouds (indeed, there is one CCO in each cloud region), receives commands from the Manager and executes them locally using the API of the cloud platform.

Cisco provides orchestrator images that are specialized for every cloud supported by Cloud Center. So you have a single place to manage all your cloud resources, and a single model to maintain: you don't need a model, or a workflow, or a script for every target cloud where the syntax of that specific API is used. You create a single model, that is completely decoupled from the target of the deployment: this reduces the amount of work (a single model instead of many) and makes the maintenance of the model easier and more consistent (you don't have to evolve many models for the same application).

Cisco Cloud Center architecture

Comparison

Two solutions for the same use case, one for free and one at a cost?Indeed they address different requirements: as described above, Cloud Center is for enterprise organizations that need to rationalize their usage of cloud resources. It is used by the corporate IT to provide flexibiliy and agility to their developers (within a governance model), to standardize the architecture of their projects based on blueprints (including what products, what versions, what setup configuration they prefer) and to get reports on consumption.Service providers can use Cloud Center to broker third parties' resources, offering a single catalog to their customers. The hierarchical multi tenant organization and the sophisticated cost models that can be offered make it simple.

I suggest you to consider it if you are using, or plan to use, two or more cloud providers (counting also your private cloud or your virtualized data center). You will see an immediate benefit in terms of compliance and efficiency.

Terraform is one of the best open source tools to manage your Infrastructure as Code: it’s easy to install, learn and use (one hour).

You could start from tutorials and free examples available on Internet.

Here is an example of full automation (we'll try to get a little better result):

As a first step, to make the usage of Openstack easier on a large scale, we discussed the value of a managed service.

If the IT organization could just focus their effort on the development and operations of the business applications, instead of running the infrastructure, they would create more value for the internal customers (company's lines of business).

So I proposed the adoption of Cisco Metapod, that is Openstack as a managed service (delegation of all the tough administrative and operational work to a specialized 3rd party, while you just use the Openstack user interface and API enjoying a SLA of 99.99% uptime).

We created a lab where Openstack abstracts the resources from the physical and virtual infrastructure (etherogeneous servers, network and storage) and the configuration of different environments is managed by Terraform, so that you can create, destroy, restore and update a complex system in few minutes.

With Terraform you can describe the architecture in a declarative form (in a manifest file).

You simply describe what you need (the desired state), not how the different components (devices and software) must be configured with all their parameters and their specific syntax.

The goal of Terraform is to match the current state of the system with the desired state.

Desired State vs Current State

Terraform is used to create, manage, and manipulate infrastructure resources. Examples of resources include physical machines, VMs, network switches, containers, etc. Almost any infrastructure noun can be represented as a resource in Terraform. Terraform is agnostic to the underlying platforms by supporting providers. A provider is responsible for understanding API interactions and exposing resources. Providers generally are an IaaS (e.g. AWS, GCP, Microsoft Azure, OpenStack), PaaS (e.g. Heroku), or SaaS services (e.g. Atlas, DNSimple, CloudFlare).

Infrastructure as Code

Desired State

It contains all the resources you need to deploy a new Devstack instance (a all-in-one instance of Openstack, useful for developers) including the needed networks, public addresses, firewall rules on a target cloud platform. That, incidentally, is a Openstack instance (so we are deploying Openstack on Openstack).

Here is the content of the main.tf file used by Terraform: it references variables with the format ${variable_name}, including the output from actions on other resources. Dependencies between resources are managed automatically by Terraform. A separate file can contain the predefined values for your variables (like the references to the Openstack lab in my example).

If you are not interested in the content of this file (I guess it applies to 70% of my readers) you can skip it and go to next picture... there is also a good recorded demo down there :-)

main.tf (the manifest file where Terraform ready the desired state of all the resources):

To make it simple, for this blog post I replaced the part that deploys Devstack with a simpler setup of a web server (Apache).

deploy.sh (Terraform will copy and execute it on the remote instance as soon as it is created):

#!/bin/bash

# author: Joe Topjian (@jtopjian)

# source: https://gist.github.com/jtopjian/4ffc82bfcbbcc78d07e4

sudo apt-get update

sudo apt-get install -y -f apache2

The goal is to demonstrate how easy it is to create a new software environment on a Cisco Metapod Openstack target from scratch and run it.

The following pictures show the Metapod console before and after running the "terraform apply” command on my computer.

This is before I run the command:

The Openstack console from Cisco Metapod

And this is the expected result (network and server infrastructure created, apache installed):

Resources created in Openstack

Next video (the most important part of this post) is a recorded demonstration of the creation of the new Apache server: you can see the launch of the “terraform apply” command that, after reading the manifest file, creates a network, a subnet, a router with two interfaces, a floating ip and a instance on Openstack. Then the Apache web server is downloaded and installed in the new instance.

The Metapod console is left in the background and you see the Openstack objects pop up as they are created.

Finally the home page of the new web server is tested.

Conclusion

It is very easy to get rid of the delays, the misunderstandings and the inefficiency of many current IT organizations.

If you standardize the process that developers follow to obtain the environment for a new project - in all the phases of the life cycle - you can enable a faster go to market for new business initiatives making your customers happy.

It would be a first step towards DevOps (more is required, mostly in changing the culture of both developers and people in operations).

Infrastructure as code is a brilliant way to create the needed infrastructure on demand (and release it when no longer needed), to maintain it based on blueprints and manage the definition of the infrastructure with the same tools you use for the application source code: a text editor (or your preferred IDE), a version control system, an automation tool.

If you have a IaaS platform like Openstack, provisioning of both virtual and physical resources is made easy.

If you do a further step forward with a managed service, someone will grant that your Openstack is correctly configured for production, up to date and in perfect health. You enjoy all the benefits, without the hassle of setting it up and operating it daily.

A quick win encourages all the stakeholders (the project team, their clients, the lines of business that provide the budget, everybody up to the CEO).

Not only it demonstrates that the solution works, but it is a concrete measurement of the return on the investment.

Generally projects are not done because they are smart, but because they are supposed to generate a financial gain (more revenues or lower expenses). Even when the goal is described as a faster go to market, the ultimate target is generating more revenues.

Agile development is not the only way to achieve a quick win, but it helps.

It also helps in reducing the project risk because, if you have to fail, you fail soon (and save a useless effort).

So, when a colleague sent me this article to solicit my comment, I almost felt insulted by the author... though I'm pretty sure he was not referring to me :-)

A note on the author:

Matthew Kern has a long experience in the field, so he knows what he's talking about.

He’s been writing many posts since 2015 to explain that Agile is dead.

Definitely he knows the Agile methodology and its usage, so he deserves respect.

More, he published a followup of that post offering the correct interpretation: probably he received too many protests.

Nevertheless my first impression was negative, because he was criticizing my fundamental believes.

But reading it carefully I understood that he's not wrong. He criticizes the evolution of the Agile methodology and the usage that someone made of it as a marketing tool, also in the light of newest trends like DevOps.

In my opinion, some overstatements in the article - starting with the title - are a mean to get visibility.

Indeed, in the conclusion he explains what he really means (and I partially agree): he refers to the “Agile” brand, to politics and to commercial usage (literature, consulting, marketing...).

When he says that agile don't work for large enterprises, I would distinguish between vendors of software products and customers doing it for their own project.

The lifecycle of a software applications is completely different in these two scenarios, and so are the business requirements, the expected quality of the product, the variety of users, the frequency of the updates and bug fixes.

When he says that many projects fail, he highlights a fact that is common to all methodologies.

But, at least, with Agile you fail soon (that is one of the objectives: better to fail in one month than after 1-2 years of unproductive activities eating your time and money).

So, if we focus on the hype, on brands and marketing activity, Agile is being replaced by DevOps (that can be considered its evolution, taking care also of the Operations with continuous delivery and feedback) and later even DevOps will be replaced by next hype.

But they both produce a value for developers and for the IT: you can see it in the cultural shift and in the individual interpretation of the principles, rather than in coded best practices. As an example, I’ve seen that my colleagues in Cisco Advanced Services started using Agile with visible benefits for both themselves (less bureaucracy) and customers (better and faster projects).

In conclusion, definitions are important and they help to spread the knowledge.

But theory is important for professors only, while a good practice makes developers and project managers happy.

If they adopt the principles of Agile, they work - even using Scrum informally - implementing those guidelines and produce good results, would you stop them?

June 14, 2016

Openstack is mature (but complex).

Surveys and statistics show that Openstack is mature and provides a number of benefit to a broad spectrum of users, from small to large enterprises and service providers.

Almost every professional in the IT (including CIOs and CTOs) knows the advantage that Openstack would offer to his organization.

But many are also aware of the complexity of the technology, the need for new operational processes and skills to set up and operate Openstack.

A scalable and reliable production environment is different from a lab where you explore the capabilities of the new platform.

The journey to a mature adoption of Openstack is not easy and you need to invest time and money.

In addition, when you hire people (or train yours), there is a possibility that another company steals them with the offer of a better salary, given the scarcity on the market.

So, many IT organizations - excluding cloud service providers, because that’s exactly their business - started wondering if it’s worth spending time in running the infrastructure, rather than running their business applications.

If you are not a cloud provider, that makes money selling IaaS, why should you dedicate additional effort to installation, monitoring, troubleshooting and release upgrades to ensure reliability and performances to your applications (that’s the only asset you should care of, because your business relies on them)?

Focus on your real business.

Why don’t you delegate all the responsibility to a provider, signing a contract that puts the above tasks and SLA on them?

Doing so, you would be free to use Openstack, getting all the benefit that you expect from it, without the burden of the learning curve and the organization implied by the Openstack adoption.

You would focus onusing the infrastructure to develop and run your applications, no longer on running the infrastructure itself.

That is called a managed service.

You own the infrastructure and exploit the value of your Data Center assets (you don’t just drop them to escape to a public cloud).

An expert team (it’s just their business) installs Openstack in your DC and operates it everyday in a HA (high availability) configuration, granting 99.99% uptime.

They take care of all the version upgrades and the compatibility of all the new features released by the community by using a certified configuration.

The user interface (the Horizon console, the Openstack API and command line interface) is available to you so you can deploy virtual server instances, networks, storage at will. You get complete and granular reporting on the health of the system and its performances.

You are the owner, but you don't get your hands dirtywith the complex stuff :-)

You pay them for the service, they grant you the SLA.

Just taste if you like Openstack.

The approach described above can be a strategical decision, because you want to focus on your business applications.

But you could also use this trick to stand up a Openstack environment in very short time, test it (I mean if your organization adapts to it, if your applications run well, if the operational model - IaaS at home, on your infrastructure, no cloud provider lock in - is good for you, if your developers are more productive) for a while, e.g. 3 or 6 months, and finally decide if you want to adopt it.

At that time you can choose between continuing with the managed service or doing it yourself.

It is a zero risk trial of the technology and of the processes: if you don’t like, you haven’t wasted any time and effort to stand it up so you can happily retreat.

You simply do not renew the service contract and that’s all: you have made a real informed decisionabout the adoption of Openstack.

Cisco Metapod: Openstack as a managed service.

Cisco has a offer that allows you to do what I described above, that comes from the acquisition of a company whose business was exactly Openstack as a managed service, on your premises.

They had a Openstack distribution of their own, optimized and hardened to provide a smooth and effective service.

Now, thanks to a strong partnership with Red Hat, the team is using the Red Hat Enterprise Linux Openstack distribution (OSP8, based on Liberty).

The essential features of this service are:

- easy start: entry level contract for 90 days

- ready to go live in 2-3 weeks from the engagement

- HA included

- the infrastructure to run Openstack can either be yours or provided by Cisco

- both the Openstack API and the AWS API are exposed by the system

And the infrastructure to run it in production can be as simple as this:

The value you can get from it: a well defined SLA, installation included, maintenance and upgrade included, no cloud provider lock in.

I believe that Cisco Metapod is a very good option to start with Openstack.

You can put your foot in the water to test the temperature, then decide to take a bath if you like it.

May 10, 2016

Hybrid cloud is one of top mind projects for most IT managers, and there's little content that one can add to be original ;-)

The hype and the attempt of many vendors (including... Cisco) to provide relevant solutions have populated the space of an incredible number of offers that make it hard to distinguish what works, what's manageable and cost effective, from what is only marketecture.

Recently Cisco decided to invest even more on cloud and, with the advent of a new CTO and some acquisitions, a revision of our approach to hybrid cloud made it easier and more effective. This post is not from official marketing and is not echoing company's direction: it's my attempt to rationalize my understanding of the new framework and to solicit your comments and feedback, so that I can leverage it when I discuss with my customers and partners.
The following picture represents the area where Cisco plays a role, offering hardware and software solutions.
When it comes to the software stack to manage the infrastructure and provide services to the users, we have a mix of Cisco products, open source solutions and integration with 3rd parties. The objective is to offer a set of pre-validated stacks that can match the different needs, granting a deterministic result.

I shared some thoughts with a group of colleagues because we're planning educational activities for our field people: instead of just providing a reference architecture (that would end being a list of products to be forced in every deal) we tried to represent the functions in the system as components of a framework, from which we'll pull the specific architecture for a given project. This, used cum grano salis, should help to be pragmatic and realize quick wins (for both the customers - think of Fast IT initiatives - and of course for Cisco).

As a result, next picture is separating the different functional layers so that they can be explained to sales guys and to customers.
It could also help to manage the possible overlap with alternative solutions that customers may choose – or already have – because every element is replaceable in the picture, based on the open API they expose/consume (as well as any well designed 3rd party product).

It is important to note that the top two layers in the picture are optional, since not all customers need those functions in their system. Based on the level of Governance that they want to have, the existing processes and the way they develop business applications (or use commercial software that only need a resource pool to be deployed), the entry point could be directly at the third layer (Multi-Cloud Management) and ITSM and PaaS would be removed.

So, while we explain all the possibilities as said above, we need to make them feel confident that it’s doable and not overly complex.
In that regard, my motto is that “cloud is not a product (or a set of), it’s a project and it’s complex in nature… regardless the products set you choose”. Generally the cost of hardware and software products is lower than development and consulting services, and customers know it.
If we can claim that a pre-built integration makes the project easier (and we can), I would stress the value of reducing the project risk and delivering outcomes faster rather than a cheaper implementation.

Now, in the context of a end to end strategy defined with the customer, we can deliver projects based on agile methodologies (e.g. Scrum) and implement the architecture layers with a bottom up approach: from a strong capability to automate the Data Center (and the hybrid cloud) you can create services that are surfaced up to the consumption layers, including a self service catalog.

Software Defined What?

The bottom up approach stresses the value of the API exposed by UCS and ACI (with the further evolution from basic programmability to policy-based management, that I'm not mentioning yet - look out for next post). With the power and the granularity of those API, you can really realize a fully Software Defined Data Center (SDDC): servers and networks can be shaped via software interfaces.
By the way, I take the opportunity here to clarify that Software Defined Data Center does not mean Software Implemented Data Center: you don't necessarily need a software overlay that mimics the behavior of the hardware (living as a separate entity), you need software controllers that drive the shape and the behavior of both physical and virtual resources in the DC as a single system.
Better if they do that based on policies... like the Cisco architecture does :-)
You will see a post dedicated to policies and application intent soon on this blog.

Competition?

We also recognize that many customers have already an ITSM solution in place, or any other form of governance. So we don't engage in competitive fights, like imposing Cisco Prime Service Catalog vs Service Now, but we rather integrate our solution with the existing components: this is a sort of compromise with a competitor that hurts my pride, but since it's for our customers' benefit... it's a good solution.

Cisco Cloud Center as a broker: the recent acquisition of Cliqr brings a great solution to Cisco to address the multi-cloud management use cases, the most important ones for the majority of customers. In the logical schema above you can see that the hybrid cloud scenario has been qualified better as Multi-Cloud management.
This reflects the fact that having a application deployed partly in your Data Center and partly in the public cloud is still a relevant use case, but many companies are more attracted by other scenarios... like moving from one project stage to next (e.g. Dev-Test-QA-Prod) using different resource pools (on premise or in cloud), or moving their assets from one cloud provider to a different one.

Cloud Brokering and Multi Cloud Management

In the first one (promotion to next stage) it could be useful to
leverage resources that are allocated based on business convenience
(e.g. cost or flexibility) or compliance (e.g. data sovereignty), so the
application and all the needed infrastructure are moved back and forth to the
public cloud.
In the second the driver could be a dual provider
strategy, or maybe a change in the market conditions that makes one
provider more appealing than the current one, or a strategic switch from
private cloud to public (or vice versa).

In all these cases, we offer a solution to deploy a software stack (a complete custom application, a development platform, or a commercial software product) as a self service option, where the target can be selected dynamically from a list of available clouds.
You can deploy to your local private cloud, based on vmware or any other virtualization solution, or to a Openstack based cloud, or to any of the public cloud providers if you have an account there.
Any resource pool is a possible destination for the deployment (and the life cycle management, including autoscale or retirement of the application).
The model of the deployment of the application is completely de-coupled from the selection of the target, thanks to the capabilities of the orchestrator that can configure the needed resources in almost any cloud transparently.
It uses the API exposed by the element managers of a multi vendor infrastructure on premise (e.g. vcenter, UCS Manager, the ACI controller, etc.) and those exposed by public clouds like AWS, Azure, etc.

From a logical schema to a real deployment

So we can offer users a different entry point, based on their business needs (they might need a ticketing system, or a self service catalog, a PaaS solution or directly the web portal of the multi cloud manager to model deployments and deliver them).
The customer can have one or more resource pools, allocated wherever he likes (local or in cloud), and let the broker direct the selection of the target based on predefined policies.

The schema in next picture presents different products at every layer: a solution can be based on one of them, or a combination. We have the flexibility to match the specific needs with products from Cisco, from 3rd party vendors or open source.
As an example, MANTL is a new open source project that makes the development of microservices easier if you build cloud native applications.

I will expand the detail of the single products and the open source solutions shown in this picture in my next post.
Stay tuned...