Automating IBM Websphere

Inline with Vibrato's mission to automate everything, Alvin Oh explores how to use CHEF to automate IBM Websphere (WAS v8.5) in a Microsoft Windows environment.

CHEF is a configuration management tool that is written in Ruby and Erlang and it’s used to streamline tasks to configure servers. It can be easily integrated to popular cloud-based platforms such as Microsoft Azure, Amazon Web Services (AWS) and Google Cloud Platform to automatically provision and maintain existing servers.

But can it handle automating installation of IBM Websphere (WAS v8.5) in a Microsoft Windows environment? While the answer is YES, we're going to have to jump over a few hurdles:

Deciphering the WAS v8.5 response file to execute can be painful and time-consuming. Often it doesn not consist of all of the relevant features that are required.

By default, the WINRM listener is not configured on Windows environments. It needs to be pre-configured together with ICF (Internet Connection Firewall) to allow Chef execution on that environment.

Installation of WAS v8.5 as a non-administrator on Windows environment is not recommended, as execution of certain operating system policies restrictions will apply.

No generic CHEF cookbooks available in the supermarket (https://supermarket.chef.io/) to cater for installation and configuration of WAS v8.5. It needs significant modification to selective CHEF cookbooks to cater for this installation.

In this article, I’ll describe how we overcame these hurdles using Chef and what the recommended ways are to get IBM Websphere (WAS v8.5) to work with Windows. As well as making it a standardised repeatable process that can be executed and extended across hosted infrastructure providers, or within enterprise on-premise infrastructure.

The following diagram shows the basic high-level components and the main interactions between them:

Using either the Microsoft Azure or the AWS (Amazon) platform, the Chef Server is installed on a single instance.

The local workstation has chefdk (Development Development Tool) and Berkshelf installed to interact with the Chef Server via the command line.

A separate node / instance is created and Chef recipes are used to install WAS 8.5 on that instance.

The installation is verified using a remote desktop connection or through the WAS 8.5 web admin user interface.

Hashicorp Packer is used to build standardised images for these environments

Building standardised Windows Image

Let’s begin by building a standardised Windows environment image that has the WINRM listener pre-configured with relevant ports enabled for Chef execution. Hashicorps Packer provides a nice straightforward way to achieve this as a repeatable and reliable.

Hashicorp Packer (https://packer.io/) is a tool for creating Operating System images for multiple platforms that’s governed from a single source configuration file. The advantages of using Packer include the creation of Operating System images that are automated and with multiple providers for portability into multiple Cloud-based or hosted platforms. If you're interested in know more checkout Hashicorp's page on the support of Packer with Microsoft Azure: https://www.hashicorp.com/blog/azure-packer-terraform.html

Let’s start off by creating a Packer image template, which is basically a simple .JSON file. If you compare the Azure and AWS resources in the Packer file, you'll notice that there aren’t many differences between them except for the different variables and builder formats that are used for creating images within these two environments.

AWS (Amazon) Environment (windows-2012r2-based-aws.json)

{

"variables": {

"aws_access_key": "",

"aws_secret_key": "",

"aws_vpc_id": "",

"aws_subnet_id": "",

"aws_security_group_id": "",

"aws_environment_tag": "",

"aws_implementer_tag": ""

},

"builders": [{

"name": "win-2012r2",

"type": "amazon-ebs",

"access_key": "",

"secret_key": "",

"region": "ap-southeast-2",

"source_ami": "ami-a1f7d5c2",

"vpc_id": "",

"subnet_id": "",

"security_group_id": "",

"instance_type": "c3.xlarge",

"ami_name": "w2012r2-winrm ",

"user_data_file": "ws2012-setup.ps1",

"tags": {

"Implementer": "",

"OperatingSystem": "Windows 2012 R2",

"Environment": "",

"Description": "Based on Microsoft's Official Windows AMI"

}

}],

"provisioners": [

]

}

Microsoft Azure Environment (windows-2012r2-based-azure.json)

{

"variables": {

"sn": "",

"ps": "",

"sa": ""

},

"builders": [

{

"type": "azure",

"publish_settings_path": "",

"subscription_name": "",

"storage_account": "",

"storage_account_container": "images",

"os_type": "Windows",

"os_image_label": "Windows Server 2012 R2 Datacenter, February 2016",

"location": "Central US",

"instance_size": "Small",

"user_image_label": "PackerMade_Windows2012R2DC"

}

],

"provisioners": [

{

"type": "azure-custom-script-extension",

"script_path": "./ws2012-setup.ps1"

}

]

}

Note that I have chosen NOT to separate the provisioners in the file (ws2012-setup.ps1), as it should be standardised regardless on which environments it would build to. Packer will execute the provision file during the image creation process and that’s where we pre-configure our WINRM listeners and ports:

I found using a self-signed functionality (makecert) from Windows SDK is sufficient enough to configure WINRM and so it could enable PowerShell remote execution on the Windows machine. There are other ways in making it more secure by securing WINRM for HTTPS transport protocol or even upgrading to a trusted CA signed certificate. But with Windows, it can often be unclear whether you are fighting a connection or authentication problem so making it simple would help you to eliminate these issues.

By default, WINRM uses ports 5985 (HTTP) and 5986 (HTTPS) as we enabled both to allow flexibility towards remote connectivity. The Set-ExecutionPolicy cmdlet determine which PowerShell scripts are allowed to be executed on the Windows machine. By enabling the policy to RemoteSigned, typically all downloaded scripts must be signed by a trusted publisher before it can be executed and by using the –Force parameter it would not asked to conform a confirmation.

There you have it. This is your first step towards automating IBM WebSphere (v8.5) on a Windows environment. I cannot remember how much I stressed to customers that base lining your OS images first before applying any configuration management is crucial in making environments – repeatable.

Side-Note: If you are running a Masterless Chef (chef-solo), you might want to install the chefdk (https://downloads.chef.io/chef-dk/windows/) during the Packer image baking process. It would save time in prepping for your Windows environment in the long run.

Enabling IBM WebSphere (v.8.5) to work with Chef

In the previous guide, we talked about the general structure of using Hashicorp Packer to build our Operating System image and the relevance and importance on achieving these objectives.

The next step is a brief introduction on setting up a centralised Chef Server v12.0 that would store and execute all configuration instructions and nodes profiling information. We would follow up by discussing in details on how we managed to install and configure WAS 8.5 with the relevant Chef recipes and bootstrapping a Chef node into the management of the Chef ecosystem.

Installing and Configuring Chef Server (v12.0)

Chef recommends at least 4 cores and 4GB of RAM with 50GB local disk (minimum) to install the Chef Server. At this present moment, Chef Server 12 is only available on the ‘Linux’ platform and its free up to 25 nodes before requiring to purchase additional nodes.

Chef Server operates on a resolvable fully qualified domain name (FQDN) pertaining to an IP address. It should be reachable via an address but if it’s not the case, the domain name with the IP address of the Chef Server needs to be set by modifying the host file:

sudovi/etc/hosts

The result should be a resolvable domain that the Chef Server can be reach anywhere in your current infrastructure.

Once the Chef Server instance is provisioned, we can install and configure the Chef Server. As root on the server, perform the following commands (note that it uses the wget command to download the Chef Server 12 package (Red Hat Enterprise Linux 7) and configure it)

Next, we will need to create an admin user and an organisation. It would generate two .pem files (admin.pem and organisation.pem) that are required by the administrator workstation (the computer from which cookbooks and nodes are managed) to gain access through Knife and Berkshelf command to the Chef Server.

Working on IBM WAS 8.5 chef cookbook recipes

It’s TRUE that automating WAS 8.5 can be a nightmare and you could end up deep inside a 'rabbit hole' debugging on why WAS 8.5 wasn’t installed properly if you decide to take shortcuts in the installation process.

In my experience, there are no shortcuts in installing WAS 8.5 and it would be wise to install WAS 8.5 manually first before trying to automate it in Chef, especially on Windows. IBM provides a baseline WAS 8.5 installation response file but it’s so plain that most of the WAS 8.5 features required would not be installed. The WAS 8.5 cookbook recipes require the WAS 8.5 response file as a based template to standardise the installation of WAS 8.5 on multiple environments.

The solution is to manually install WAS 8.5, selecting all the features required and generate the response file prior to the completion of the installation. Hence, template it up as .erb file as baseline install for Chef. That solves the first part of the solution but even before the correct response file could be generated, but what are the installation steps to successfully install a WAS 8.5 Application Server?

Based on documentation and recommendation from IBM, there are 2 installation paths that we could undertake when installing WAS 8.5:

The IBM HTTP Server (IHS) is a full feature web server that includes IBM WAS 8.5 as part of its package. Therefore, it depends on the features required by the organisation to which installation path they would undertake. The automation process in Chef shouldn’t differ from these installation processes that are recommended by IBM. Chef is merely a configuration management tool that governs the way each product is installed and configured.

Let’s take a closer look inside the WAS 8.5 recipes on how they are automated using Chef:

Every cookbook requires a metadata.rb file that is located at the top of every cookbook directory structure. The WAS 8.5 metadata content provides a configuration detail to the Chef Server in ensuring its node is deployed correctly. It defines the authenticity of this particular cookbook (name, maintainer, version) but the most important field is the depends field. It shows the WAS 8.5 cookbook is dependant on another cookbooks. The most important dependency WAS 8.5 cookbook requires is the 'IM' cookbook.

The IM cookbook installs and configures the IBM Installation Manager that is written specifically for Windows environment. Without it, WAS 8.5 wouldn’t be able to execute installation using its response file. Every mentioned process above consists of components that have their own downloadable packages that require installation. The pain-points are the location of packages, download durations (significant in size) and idempotency.

We placed our installation packages into a S3 Bucket or similarly Windows Azure Blob Storage with security rights to prohibit unauthorised access to these packages. Hence, it’s on the same network so the downloading speed shouldn’t be compromised. Chef cookbooks are meant to be able to run on an interval set by users and only apply new changes if they have been updated. So idempotency is really key part of developing correct Chef cookbooks.

The installation of WAS 8.5 runs against the response file that was generated previously and uses IBM Installation Manager to patch it to the latest stable version. Finally, the creation of a WAS profile to manage the WAS 8.5 Application Server (stop & start).

One last question remains – how does Chef know when the installation and configuration has been successfully completed?

Basically, what we are automating is a ‘SILENT’ mode install. As WAS 8.5 has many log files spreading across different directories, we found the easiest way to verify a successful installation is by checking for the .swtag files. All the above code snippets show verifying the existence of these files which if found indicate a successful installation.

The difficult part of automating WAS 8.5 in Chef is not the process nor the coding development standards but its verification and checking process in determining whether WAS 8.5 Application Server was installed properly.

The code base may look straightforward and simple but it took long hours and many failed attempts to successfully get it working on a Window environment.

Bootstrapping WAS node with Knife

With our Chef Server and WAS 8.5 recipes finalised, we can begin using Chef to bootstrap our newly created WAS node or Chef term (Chef client). Remember the two .pem keys (admin.pem and organisation.pem) that we generated from the Chef Server?

Those keys need to be used to configure Chef knife on the administrator workstation by opening up a file called knife.rb in your text editor:

Once the knife.rb has been configured, running this command would cause the Chef client to be installed on the WAS node, register it to the Chef Server and download and run all the recipes for its defined role.

knife bootstrap <node IP address>

--node-name <was node name>

--ssh-user root

--ssh-password <password>

--run-list role[was]

As I mentioned earlier, all recipes are expected to be idempotent so re-running them any number of times won’t change the final results unless the recipes are updated or there are manual changes on the server itself those are outside Chef. In this scenario, Chef would revert back the changes to its original state as specified in the recipes.

Summary

After following this blog post, you should have a basic understanding on how I used Chef to automate the WAS 8.5 installation in a Windows environment and the process I used to create a repeatable procedure that could be adopted on multiple different environments.

One last note, there are no shortcuts. I highly reccomend to follow the instructions produced by IBM and try to install it manually first before automating it.

If all else fails and you seem to be stuck debugging in a rabbit hole – Document what you have achieved so far and restart from scratch.

If you want to learn about how Chef can help you automate, especially in an Windows environment, check them out at http://chef.io