Automate and scale using vCenter Orchestrator

Tag Archives: workflow

One of our customer’s was recently exploring using ServiceNow’s Service Catalog to initiate a provisioning request to vCloud Automation Center. There are several ways customers have requested vCloud Automation Center integrate with ServiceNow. These integrations can be bidirectional – ie, vCloud Automation Center -> ServiceNow, by generating a service ticket and updating the CMDB and invoking vCAC Services from the ServiceNow Catalog, and ServiceNow -> vCloud Automation Center, using our orchestrator to receive the request and invoke our own pre-built integrations to our suite. While it is still our best practice that vCAC serve as the main customer interface for self-service, this post looks at how we successfully integrated ServiceNow’s Service Catalog to provision services in vCloud Automation Center. And it also demonstrates how vCAC services could be invoked from any Service Catalog, using vCenter Orchestrator.

Special thanks to Tom Bonnano, Chris Decanini, Eric Hardcastle, Michael Steward, and Derek Reinhard for helping me with this integration and the vCAC Plugin.

Create the Workflow in vCenter Orchestrator to receive the ServiceNow request

In order to initiate the request to a third party system, your customer has to create a form to collect inputs and create a workflow to pass those inputs to the third-party system. Because every request takes a custom created form, the inputs (and how they are us

First, we created the master workflow that ServiceNow would invoke. I recommend you start very simply - for us, we only added a single variable, Hostname, which would receive a value from ServiceNow, and show that it passed all the way through to the server that was provisioned by assigning that hostname.

To do this we dragged the “Provision a virtual machine from a blueprint (Deprecated)” workflow from /Library/vCloud Automation Center/Infrastructure into our workflow, and assigned all of its values to attributes. ed) will differ from service to service. We opted to integrate ServiceNow directly to vCenter Orchestrator’s SOAP API, so we could leverage the 6.01 vCAC Plug0in and use that to invoke provisioning in vCloud Automation Center. This would also make editing and maintaining these workflows much easier due to vCenter Orchestrator’s rich object model and allow us to get updated workflows directly from VMWare (instead of having to re-write our own).

We assigned everything upfront to hard-coded values to complete the test (with the exception of the “custom” attribute, which is an array that contains each “customProperty=value” entry. To change the hostname of the provisioned server, we would have to insert “hostname=<something>” into this array.

We then used a scriptable task in front of that workflow, to create the “custom” array that would eventually contain our one custom property.
We made a simple script that pushed the formatted value onto this array so it was ready to go.

Make sure your workflow has one input that prompts the user, called “Hostname”. Now, when you think you’re ready, run this workflow, input a hostname, and see if it provisions a server! Make sure this works before moving on to the next step.

Link the form in ServiceNow’s Service Catalog to the workflow operations that invoke vCenter Orchestrator. NOTE: we found some limitations in ServiceNow’s SOAP message operations for some WebServices, where SOAPActions cannot be blank. Please see the Troubleshooting section for more information. We chose to use a Powershell operation in the ServiceNow workflow to call the SOAP endpoint (as it passed the SOAPAction header properly), and instead opted to pass the variables into this script. This allowed us greater control to override the SOAP client behavior (and to ignore self-signed certificates, as you’ll see below).

In ServiceNow, you should specify the workflow to be run from the form, and the workflow should invoke a Powershell operation.

We started with a completely hard-coded script, which would call the service and pass information.

This could then be modified to substitute the hard-coded values to the ServiceNow ${variable} syntax, which ServiceNow would replace with the form values on execution time.

Because this was a test environment and all of the certificates were self-signed, we ended up adding a line to the top of the script to allow Powershell to trust certificates that were not signed by their local CA chain:

And that’s it! Now, when the form is submitted, it provisions through to vCAC, setting the hostname to the one you hard-coded in the Powershell.

Troubleshooting

Testing vCenter Orchestrator using SoapUI

To ensure we could communicate with vCenter Orchestrator successfully, I used a program called SoapUI to send requests and to view the headers and responses. It’s a very nice client and will automatically generate all the soap actions for you. I used this to simulate API calls with my vCenter Orchestrator first, to get the inputs right.

This is what was generated, and I simply filled in the blanks to test if it worked.

Testing Powershell Outside of Servicenow

It was important to first test the commands being issued from the ServiceNow MID server to determine if we had connectivity to vCenter Orchestrator, and if it would accept our self-signed certificates. I recommend you invoke Powershell from your local ServiceNow MID server (which will receive the command from ServiceNow) to see if it works:

Displaying What ServiceNow is Sending vCenter Orchestrator

We found some limitations in ServiceNow’s SOAP message operations. vCenter Orchestrator’s SOAP messages specify that a blank SOAPAction header should be passed, but the SOAPMessage operator in ServiceNow cannot have a blank “SOAPAction” field. The request generated will omit the header “SOAPAction” if the action is blank (instead of passing a blank quoted string). vCenter Orchestrator considers missing headers a malformed request, and will output a 500 error in the server request log. You can see errors in the test scenarios of this SOAPMessage test:

We discovered that the operation SOAPMessage in ServiceNow must contain a SOAPAction, or else it will omit this header entirely.

We know from our previous testing with SoapUI we require that a blank header be sent, and SoapUI allows us to see what that header should be:

So we really need to understand what’s being sent by ServiceNow, it’s obviously not being correctly generated. You can examine any requesting service to see if the correct headers are being sent by starting a Mock Service on the server that has SoapUI installed, and sending the ServiceNow request there. Below is a screenshot of a mock service running on port 8088, where I print out the headers and content of the request, using a groovy script within the OnRequest field, located here: https://github.com/momecca/SoapUI

This will help you understand what is being sent, and compare it to other client’s and the requests they generate.

The vCenter Orchestrator gifts season started a few months ago with the general availability of vCO 5.5. The compelling release was announced just in time to share the vCO momentum at VMworld. Along with the amazing new features introduced, VMware continues with a lot more presents.

This year the holiday magic brings to vCenter Orchestrator users several updates, product integrations and learning gadgets that make automation with vCO more powerful than ever.

1. VMware has just announced the GA of VMware vCloud Automation Center 6.0(vCAC 6.0). The extension and creation of XaaS is done using the vCAC Advanced Service Designer which allows you to seamlessly leverage any vCO workflow and convert it to a catalog item or day 2 operation, available as a service in the vCAC Self-service portal.

2.vCO CLI is the code name of the new debugging extension which will not only facilitate all experienced vCenter Orchestrator users and also helps the newcomers to programmatically explore the rich vCO ecosystem through interactive command shell. Access to the vCO plug-in’s inventory is integrated into the tool UI for easy navigation to the integrated solution objects.

3. Our super powerful and generic HTTT- REST and SOAP plug-ins are now able to support proxy configurations and assure security compliance of your automated solutions.

4. The long awaited vCO Powershell plug-in 1.0.3 is updated to support Powershell 3.0 inventory and is fully backward compatible with your existing scripts.

VMworld US 2013 kicked off Monday with a keynote by Pat Gelsinger, CEO of VMware, mentioning the importance of management and automaton in the Software-Defined Data Center (SDDC). Also mentioned was the inclusion of vCloud Automation Center (vCAC) in the Standard, Advanced and Enterprise editions of the upcoming vCloud Suite release. With bidirectional integration between vCenter Orchestrator (vCO) and vCAC, native vCO workflows can be coupled with resources managed by vCAC, either as part of the provisioning process, or as a Day 2 operation. The upcoming vCAC Extensibility Package for vCO affords customers the ability to use vCO as a configuration tool for vCAC extensibility. Instead of manually reconfiguring stub workflows in vCAC to call vCO, the configuration workflows in the Extensibility package will do it for you. Customers choose a workflow to be executed at a given point in a machine's lifecycle (i.e. run a workflow before the machine is built), then select the blueprint(s) that should call the specified workflow. vCO then calls into vCAC and programmatically wires up the specified vCO workflow to the blueprint(s). Alternatively, vCO can expose and assign its own workflows as Day 2 operations to the contextual Machine Menu in vCAC (think right-click or hover menu), then enable that machine menu item on specified vCAC blueprints. Very powerful stuff!

In addition to the reference in the keynote, two vCO-specific breakout sessions were held on Monday:

VMworld US 2013 session VCM4875 - Part 1: Getting Started with vCO

VCM4875 - Part 1: Getting Started with vCenter OrchestratorThis session was vCenter Orchestrator's opportunity to shine, and the room was filled to capacity. The session was presented jointly by James Bowling, Cloud Architect at General Datatech LP, and Savina Iliena, VMware Product Manager for vCenter Orchestrator server.

James talked about his own experiences with vCO, and demonstrated a few things he put together, bravely presenting them in a live demo. Savina took over in the second half of the session to talk about the new features coming in vCO 5.5, specifically the new Debugger and the High Availability configuration. This news was well received by the crowd, and by the end, you could tell they were excited to try out vCO in their own environments.

This session was co-presented by Charlie Cano of F5 and Dan Mitchell of VMware. The session focused primarily on vCO's capabilities around provisioning, configuration and remediation using their brand new vCO plug-in.

Charlie started the session off by asking how many folks owned vCO, and only a few hands went up. He informed them they own vCO if they own vCenter, which seems to have caught a number of them by surprise.

F5 did a great job getting the initial release of their new plugin completed in time for VMworld. Thanks to Charlie Cano at F5!

Be sure to check out the other vCenter Orchestrator sessions at VMworld US 2013:

vCenter Operations performs constant analysis of the datacenter health, and launches alerts when certain problems or risks arise. These alerts can be sent externally via mail, or as SNMP trap messages. On the other hand, through its scripting API and its library of workflows that can be downloaded from VMware Solutions Exchange, vCenter Orchestrator is powerful enough to perform reconfiguration actions at almost any level of vCenter. And guess what? It can receive SNMP traps using its SNMP plug-in. I guess you already get the idea. And here is the picture to visualize it:

You said 5 minutes or less

Really anyone can write his/her own implementation of this scenario, using a vCenter Orchestrator appliance and the SNMP plug-in. However, it can take some time to track down what exactly happens on the SNMP level, parse all the valuable information, code the mapping between trap messages and remediation workflows in a clear and maintainable fashion. That’s why we decided to spare that effort to any user who does not necessarily feel the urge to be a code-hero. So we created a vCO .package with the following goals in mind:

Be easy to work with

Be easy to configure

Do the task, of course, of launching workflows on events from vCenter Operations

Start the policy. Start the newly created policy. If you want this policy to start automatically with server restart, edit it and change the Startup parameter on the General tab.

Attach remediation workflows. Go to the "Configuration elements" tab in "Design" menu, and locate the "SNMP/vCOps Configuration" element. In the attributes tab, you can find the mapping between vCenter Operations alerts (in the Name column) and workflows in the "Value" column.

There are two sample remediation workflows included in the package:

Default Action - it only prints the trap parameters in the log.

Capacity Remediation Action - It is a real remediation action, that takes the trap-specific inputs (like EntityName, EntityType, Criticality, etc.), takes only the ones needed, and forwards them to the real remediation workflow - "Library/vCOps Remediation/Capacity Remediation Action/vCOps Remediation Datastore Capacity". This is a non-intrusive workflow, that finds the Datastore object, corresponding to the datastoreName parameter, checks for its powered off VMs, analyses their disk usage, and the disk usage of their snapshots, then prepares an email report and sends it to the user. For this workflow to work, you have to add your email in the "toAddress" attribute in the General tab of the workflow. You also have to setup the right smtp server in the settings of the Mail plugin in vCenter Orchestrator Configurator. This workflow has to be assigned to the "riskCapacityNew" alert, so it can be triggered correctly.

Filtering. As there may be a lot of Alerts coming from vCenter Operations, we provide the possibility to filter the incoming traffic, and not launch any workflows, unless the filtering criteria are met.

This can most easily be accomplished with the help of the "Library/vCOps Remediation/Configuration/Configure Filters" workflow, although it is also possible to achieve the same directly in the "SNMP/vCOps Filters" configuration element. This workflow could fail validation on prior to vCenter Orchestrator 5.1 systems, and the workaround is to manually open the "SNMP/vCOps Filters" configuration element, and set empty array for each of the five attributes (just click the "Not Set" value and immediately hit the "Accept" button after this).

The examples

Although we are only providing non-intrusive examples, any workflow can be assigned to any alert, moving or deleting VMs based on some criteria, defined by the user. In fact, vCenter Orchestrator provides a library of thousands of out-of-the-box workflows that can integrate various third party management systems.

Implementation details

For the ones eager for technology detail - here is what happens under the cover:

A vCenter Orchestrator policy is waiting for trap messages from vCenter Operations.

Once a trap is received, the policy translates the trap to a javascript alert object, thus simplifying it quite a lot.

Then it checks in a map (associative array, actually) if there is an alert definition for the incoming SNMP OID.

For example: "1.3.6.1.4.1.6876.0.31" -> "riskCapacityNew"

It checks if there are filter conditions defined, and if the trap matches them, if defined.

The policy finds if there is workflow assigned to this alert, in the "vCOps Configuration" configuration element.

Launches the workflow if such a workflow has been defined.

How are the project goals achieved

All the complications, technical details and scripting resides in the policy.

Great flexibility can be achieved by setting of correct configuration and filters.

All configuration is moved to configuration elements.

There is a workflow for even easier configuration of the filtering.

Congratulations! Your system is now installed and configured.

No programming involved.

Summary

vCenter Operations is fully integrated with vCenter Orchestrator so you can leverage more of what you already have. Automated workflow triggers let you associate workflows created in vCenter Orchestrator with vCenter Operations alerts. For example, these workflows can be used to automatically delete old VM snapshots when available capacity falls below a critical threshold or to add resources when workload demands are rising above normal. You’re always in control and can customize workflows with simple drag and drop operations. With vCenter Operations you can finally pull the trigger on automation.

As 2012 comes to a close, we thought it would be a great time to end the year with some gifts to put under your (virtual) holiday tree!

2012 was a great year for automation in general, with the launch of the vCloud Suite 5.1, and orchestration in particular, with the release of vCenter Orchestrator 5.1. Our team was extremely happy to see a tremendous increase in vCO adoption, and a growing list of integrations with other management systems.

In that spirit, we are very glad to announce the availability of several integrations and learning tools to make your automation projects easier than ever before.

1. vCloud Automation Center 5.1, which was just released, provides the ability to extend pre-built processes and post-provisioning actions by invoking vCO workflows. This means that any technical integration or logic built in vCO can be leveraged by vCAC's lifecycle-management platform, thereby broadening the realm of self-service provisioning and basic administration for consumers of IT services.

2. Reversely, the new vCenter Orchestrator Plug-in for VMware vCloud Automation Center allows organizations to automate vCAC provisioning and post-provisioning tasks. With these two components, customers can leverage full bi-directional integration capabilities between vCloud Automation Center and vCenter Orchestrator.

3. Another new offering is the vCenter Orchestrator Elastic Service Plug-in. This plug-in provides a foundation for the self-scaling virtual datacenter, by automatically balancing the physical resources between virtual datacenters in VMware vCloud environments. This plug-in contains a rules engine that can analyze resource usage metrics (for instance, metrics captured by vCenter Operations Manager) and make scale-up or scale-down decisions automatically.

4. The vCenter Orchestrator Plug-in for VMware Service Manager enables organizations to automate operations around Configuration, Incident, Task and Service Request management. Thanks to this plug-in, repetitive tasks such as updating an Incident or creating a Configuration Item when a new virtual machine is provisioned can now be fully automated.

5. And to help you take advantage of all of the above gifts, the VMware Training department just released over 10 self-paced vCO training videos available for free!

For additional information on these materials, please visit the following sites:

Booths

We had some great discussions with many customers in San Francisco and are hoping to repeat that in Barcelona. So please, come by the VMware vCO/DynamicOps booth to see the latest 5.1 release, share your experiences, and get your questions answered.

As part of the broader announcements around vSphere 5.1, we are extremely glad to announce the general availability of vCenter Orchestrator 5.1!

As we mentioned last year, 2011 was all about bringing you new plug-ins (and more plug-ins...) to simplify multi-system integrations.

Whereas the emphasis on plug-ins has not stopped, we are extremely excited to announce that vCO 5.1 includes some major new capabilities!

Launch Workflows Directly from the vSphere Web Client

vSphere administrators and operators can now launch vCO workflows directly from the vSphere Web Client, thereby saving precious time and preventing the need to switch into and out of multiple user interfaces. Operators can use the vSphere Web Client to launch any workflow, whether pre-built, custom, and whether it interacts with VMware or partner applications! Operators can run workflows from the vSphere inventory browser in just a couple of clicks. Based on the object from which it is run (for instance a host or a VM), a workflow's input parameters get populated automatically to save time and eliminate errors. Operators can run multiple workflows concurrently, or schedule them as recurring or future off-hour tasks. For larger organizations, administrators can allow different groups of operators to have access to different categories of workflows.

Develop Workflows More Easily

Workflow developers can also benefit from a simpler, faster, and more enjoyable development experience thanks to a complete redesign of the Workflow Designer. The new Designer allows workflow developers to use multiple screens, detach windows, customize workflow icons, and perform many more operations in just a single click. Auto-attach and auto-layout capabilities also greatly reduce development time. And to simplify workflow administration, vCO 5.1 introduces new capabilities such as version control and automatic generation of workflow documentation in PDF.

Richer Integration Capabilities

vCenter Orchestrator 5.1 includes a new REST API that does everything covered by the current SOAP API... and more! The new REST API provides more control and flexibility when launching workflows programmatically. It also introduces support for new capabilities around content management such as workflow and package importing and exporting. In short, vCO administration itself can now be more easily automated.

These are just the major new capabilities but you'll find plenty of additional enhancements that simplify and enhance the automation of your virtual and cloud infrastructure. For a full list of new features and capabilities, please refer to the detailed overview or the release notes.

And finally, while vCO 5.1 is a major enhancement over version 4.2, you'll still be able to leverage all of the work you've already created in the previous release.

So don't wait any longer! Try it today and let us know what you think via blog comments, Twitter (#vCO) or the vCO Community.

This example shows that one can do useful things without writing JavaScript code in vCO.

Ever since the vSphere 5 upgrade, cold booting my home lab (which I generally do because I’m not going to pay to leave it running all the time plus it is noisy and it heats up my office) requires re-connecting my ESXi hosts.

I was tempted to write a Perl script or use PowerCLI but given my focus on vCO and with the new the vSphere 5 plugin, I thought, “I’ll write a workflow!”

So I started by looking at what was there and thinking it would be cool to do this in vCO without writing any JavaScript.

I easily found the delivered workflows for “Disconnect host” and “Reconnect host”.

Trying those one at a time on one of my ESXi hosts and they worked perfectly. Great. These are my first building blocks.

Next I thought, “Ok, now I will sequence these two flows.” No biggie. I created a new workflow called “Disconnect Reconnect ESX Host”. In the schema, I dragged a workflow item, select “Disconnect host” and repeated for “Reconnect host”. Add the “End Workflow” item then link together.

Next I clicked “Validate” and took the quick fix option to create an input attribute for “ESX host” which both embedded workflows can now share.

But I’m thinking that looks like too much work and I’m lazy. Besides, I would rather fall back on my bad habits and just write a “foreach” loop in JavaScript and call the workflow in the script. Not very “orchestrator-like” but I’m more of a “Perl” guy. I resisted the temptation.

“Well, maybe I can just use what is there in vCO and not write something. (I am so lazy.) Hey, what about that Batch thing. Yeah, baby.”

Since I had just added a new workflow that takes a single ESX host as input which qualifies to run from the batch workflow, I ran the “Fill batch configuration element”. Now I will be able to reference my “Disconnect Reconnect ESX host” workflow.

Now I want to verify that I can run my new workflow from the “Run a workflow on a selection of objects”. I chose ESX host as the object type, selected the Action “GetallHostSystemsofCluster”, using my cluster for the input and then chose my “Disconnect Reconnect ESX Host” workflow.

Outstanding, it cycled through my ESXi hosts disconnecting and then reconnecting each one very quickly.

Lines of code written: ZERO.

But you know I am really, really lazy. How can I run this batch workflow without having to go through and configure all the inputs each time? It was time for another workflow. I remember Christophe Decanini telling me this could be done.

Ok time for one more workflow to setup and call the batch workflow the way I want for this job. I called it “Disconnect Reconnect All ESX hosts of Cluster”. Add one workflow item which is the “Run a workflow on a selection of objects”, add the “End Workflow” item and link it together.

Next I used a trick that Christophe Decanini taught me. “Synchronize presentation” shows when you right-click on the embedded workflow.

This adds all the attributes of the embedded workflow as inputs to the current workflow. While this is pretty cool, remember that I want to run this without any inputs at all. Ok, there is a quick fix for this. Move all the inputs as attributes. Click on the top input, then Shift-Click on the bottom input. When all the inputs are selected, right-click and choose “Move as attribute”.

Now the visual binding looks like this:

This is a complex workflow but I only need a few of the parameters.

Now all that is left is to set the defaults that I want for my lab to run this specific job. This was actually pretty easy by looking back at the successful run of the batch flow previously and observing the variables that were actually used and their values.

The objectType is a string and the value needs to be “Host”. Not exactly intuitive but there it was in the variables from my previous run. I used the same Action “GetallHostSystemsofCluster” as before and selected my cluster for the input to the Action and finally chose my “Disconnect Reconnect ESX Host” workflow.

“Look Mom, no parameters for input!” And it worked fantastic. No loop code in JavaScript or multiple vCO scriptable tasks, decision items, etc… Dirt simple really.

This is good enough for me but if I am going to share this with others, I don’t want the workflow referring to the cluster in my lab. So the one last improvement is to use a Configuration instead to reference the cluster to use. This way you can set the Configuration to reference your cluster. Create a new folder and element for the Configuration.

Then edit the Configuration and add an element setting the Type to VC:ClusterComputeResource. I called it “MyCluster” and set the default value to point to “Cluster1” in my lab. Yes, this is quite the outstanding name, don’t you think?

Finally, I went back to the workflow and updated the “cluster” attribute default value to reference the configuration element. Click the little symbol in the Configuration column.

Now pick the “MyCluster” Configuration element.

Finally, I created of Package of my work and saved it out for safe keeping.

Cheers!

Bill Call

Central Region US Principal Systems Engineer

Solution Specialist vCloud/Chargeback

Bill has been at VMware a relatively long time (8+ years) and has worked with various automation tools over 25+ years in IT.