Some customers have been running into a problem when adding patches and fixes to their 8.0 WebSphere environments. The most common issue exists within existing environments that were not originally installed with Rational Automation Framework.

When installing WebSphere with IBM Installation Manager, depending on whether you do it silently or via the GUI. The agent dataLocation is set behind the scenes with a default. This default value, can be seen in the following locations (depending on if you are using Windows or Linux/Unix. I will explain using Windows in this example. Same as for Linux with variations to file paths.

This can be changed after an installation of WebSphere has been installed. So the data location may be different.

AGENT DATA LOCATION

The agent data location is used to store files such as Eclipse p2 repositories and installed.xsd/xsl and installRegistry.xml. These files are important for IM to recognize what you already have installed on your system. What version of WAS and keep track of patches and fixes that have been added.

The issue with RAF is that it launches the IBM Installation Manager using imcl and passes an argument called -dataLocation to the Installation Manager. Since RAF uses a different dataLocation than the IM default. There will be no record of WAS already being installed. This is different than if you had installed WAS with RAF. RAF would have set that dataLocation to the RAF default dataLocation on the install. Thus removing this issue during updates and fixes.

I do not recommend trying to change the ANT tasks. Although, it can be done. You can change the ANT task and remove the flag from being passed. I wont talk about this hear because unless you remember you did that. It could hinder future installs of WAS with RAF. Since RAF is intended to be an all or nothing product. At least in my opinion.

The best way to work around this issue, is to play with the response file. There are many ways to get a working response file. Which leads me into the second part of the issue when updating and applying fixes with RAF.

When installing WAS with RAF. RAF It passes in an offering ID and profile ID that are the same. So when it comes time to update, the ID's match. When installing WAS without RAF. Installation Manager creates the profile ID and offering ID which are different than what RAF would use. RAF my see what you have installed, but if the ID's do not match. It will either fail or attempt to install the "new product" because the ID's do not exist in the agent dataLocation.

The installation process works like this.

1. Checks the media tree location for disk1/disk.inf. (You can get away with just putting the disk.inf in the media tree location and nothing else. If this is an existing installation there is no need to have the full media for you are not installing anything.)

2. Checks to see if IM is installed. (Keep in mind, you must have the exact same IM media on your RAF server as you have installed on the target system. If you do not, it will install IM again with the different version. This you do not want. You can also not get away without having the IM media in the RAF media tree.)

3. If already installed, it checks to see if updates are needed. It will either update or not.

4. It then creates the response file and will install the fixes / patches. (As long as you have the PATCHES= or (portal) FIXES= etc. (You can find more on those values in the RAF IBM Infocenter)

The most least invasive way to fix the issue noted in this blog is to get your own response file and place it on the framework server. The easiest way is to start IM from the command line, passing in the -record and -skipInstall parameters. This will allow you to build a response file. The only thing to note would be the repository location being used.

-record and -skipinstall can be found in the IBM InfoCenter for the version of Installation Manager that you have.

Once you have a working response file. Put that response file on the framework server and use it to run your updates or fixes. The location on the framework server is as follows.

NOTE: If you have tried and failed previously. There will be a response file at the location below. You can open it up and place the new response file text there.

RAFServer/rafw/work/<environmentName>/cells/<CellName>/ etc.

If you have not attempted this yet. The location below:

RAFServer1\rafw\product\templates\

Will contain the templates used with the locations set as variables that are passed into the script before it is copied into the work directory location above.

Sometimes it is easiest to just run it once, let it fail, and use that response file. Since changing the template response file can be more dangerous for later installs updates etc.

You can then use this response file to do all of your updates to an existing environment for the future, until this issue has been fixed.

This can get really complicated. So I would encourage individuals using Portal and WAS to contact support with further issues. The easiest way to work with Portal updates is via the FIXES. There is a direct fix action that can be ran quickly. This is a "hidden" action because it can not be seen from the eclipse UI or the rafw.bat/sh CLI using -d or -p. It's a portion of the ANT task that is involved with installations. Thus it one of the actions called by the composite install action. I wont go into this here unless there is general interest. I do not work with many Portal customers.

This blog was generated by our twitter following. Join us to stay up to date on the latest Rational Automation Framework information. www.twitter.com/raf_WebSphere

Global Security within WebSphere allows for two different login options.

Global Security Disabled – Anyone can log into WebSphere without being prompted for a password. Anyone can start/stop and make configuration changes that would typically require a userName and passWord.

Global Security Enabled – This requires an individual to login to the WebSphere environment. Upon stopping/starting and other config changes, the user will be promoted for a set of credentials.

When getting started with Rational Automation Framework, you have two options.

Create a New WebSphere Cell – When the intention is to install WebSphere using RAF.

Read an Existing Cell Configuration – When a configured environment already exists and is brought into RAF to be managed.

When Enabling Global Security in the creation of a new WebSphere cell. You are given a drop down, asking whether to Enable Security. Yes or No.

If the answer is No. The predefined list of required fields does not change. See Below.

If Enable Security is set to Yes. The predefined list of required fields changes. See Below

The fields change requiring a WebSphere Administrative User and a WebSphere Administrator Password. These fields are the required fields used when attempting to login to the target system and for RAF to access the WebSphere DMGR. After the environment has been generated, the text for the fields is stored in the RAF_HOME/user/environments/<environmentName>/Cell/cellName/configuration.properties

The fields are stored above the OS userName and OS passWord. See Below

When the fields are not selected. The Global Security Information is not filled out. See Below

If the global security was enabled, but is now currently turned off, or was not enabled but now is.

Navigate to the Node scope. Find the DMGR Node and select the install.properties file. Inside this properties file there is a value. See below:

# turn on security by default
ENABLE_SECURITY_AT_CREATION=true

If the security enabled is true, then the WAS_USERNAME and WAS_PASSWORD from the configuration.properties will be used. If the value is set to false, they will not be used. If there are values for WAS_USERNAME and WAS_PASSWORD. There is no need to remove the values when turning on or off Global Security. Just change the Enable_Security_At_Creation = T/F.

When reading an existing cell configuration, RAF ask for the WebSphere username and password. If security is enabled, enter the correct credentials. If security is not enabled, you can enter whatever you would like into the fields. But I do believe they are required. This can pose a problem later on.

What if you read an existing cell, enter in the WebSphere userName and passWord as Hello and Hello. Then security is enabled later, and the values for WebSphere are wsadmin/wsadmin?

When reviewing the the configuration.properties files. You may have noticed how the passwords are encrypted. RAF will recognize a plain text password and encrypt it automatically. Thus, changing the encrypted password is not difficult to do in the event security has been enabled later on. Example below

NOTE: Don't forget to set the install.properites security value to true/false as well.

That is all I can think of regarding Global Security. If there is something else you would like to know. Tweet it to use at www.twitter.com/raf_WebSphere and we will do our best to answer your questions.

When it comes to performance within the realm of Rational Automation Framework, there are two places to look. Firstly, the database. Which we wont touch in this article. Secondly, is the projects structure. This document will walk you though a couple best practices and examples of ways to structure and set up your automation plans.

Rational Automation Framework has the ability to thread steps, iterate through steps using the .drill command or While loop. There is also a feature called Long Running Wsadmin in which you can run step with. I will start off by discussing threading and move to long running wsadmin and finish with the various step types.

Threading enables steps to run in parallel, either on the same server or on different servers. Threading is controlled by the Thread property setting for a step. By default, the Thread property is set to No. Threading helps reduce project execution time when there are parts of a project that can be run independently of each other. When the Thread property for multiple adjacent steps is set to Yes, the system attempts to run the step in parallel. Such steps are considered thread-enabled, and each step can be run separately while the rest of the job continues. Threading follows these rules.

At least two steps in sequence must have the Thread property set to Yes for threading to occur. A set of threaded steps in sequence is called a thread block. Thread blocks can continue into steps that are part of an Inline. For example, if a step in a project contains an Inline and the first step of that Inline is also threaded, the two steps are part of the same thread block. They run concurrently. The thread block follows threaded steps, including nested Inline steps, until a Join step or a nonthreaded step is encountered. Take care to avoid race conditions when using nested Inline steps. A race condition can be caused by an inlined threaded step depending on results or data from the threaded parent step.

This is important, considering most of the steps within RAF are ran via libraries that are inlined into the project.

A thread block is terminated by a step whose Thread property is set to Join or when it encounters a nonthreaded step. At that point step execution becomes sequential again.

Steps may end up running simultaneously on one server or on several servers,

If there are multiple thread blocks, the first thread block must complete before the next thread block can start.

Use the Max Threads property for the project to limit the number of threads that can run at the same time. Each thread-enabled step and its inline project, if any, can result in parallel processes. All processes are counted until the maximum for the parent project is reached. The system stops launching new parallel processes when it reaches the Max Threads value. It waits until the number of parallel processes for the project drops below the Max Threads value before continuing.

Most of the projects set up within RAF are launched via an inline that runs a library. Projects steps can be threaded, and so can the associated library steps. Something to keep in mind when designing your automation plan. This is particularly important when it comes to managing a single server. If there are two nodes on one physical machine, threading a step could potentially cause your project to fail. Take the step below as an example of this.

Notice that this step is threaded. It also has a condition that will cause the step to run in a loop for the amount of nodes to install. I can already see a problem when using one physical machine and trying to install more than one node at the same time. This will not work where as if each node were on a different physical machine. This would work.

Notice was_common_configure_jdbc_providers action runs more than one action. It also runs the was_common_configure_jdbc_datasources and was_common_configure_jdbc_was40_datasources. This is an example of a composite action.

Composite actions that are simply called. Will open a new jvm for each action. The composite action moves in sequence. Opening up a new JVM per operation. When running multiple steps against a server, it is possible to have a large number of JVM's running, thus slowing down the entire process. There is a way around this and it's called using Long Running Wsadmin.

Before using the useLongRunningWsadmin option, the necessary files must be in place on the framework server. The Environment Generation wizard for an existing cell puts those files where they need to be. If you have not run the Environment Generation wizard for an existing cell, use the was_common_configure_setup_lr_wsadmin action to copy the necessary files to the framework server. This is not supported on windows and before WAS 6.0.

This feature will launch a WsadminListener. This listener will open up a port and read in information in the form of a request. It puts a lock on the tmp/ directory and will execute each portion of an action using the same JVM. It also creates a larger more robust JVM. This can increase the time it takes to run composite actions.

The issue comes when running actions in long running while they are threaded. If there is another step that calls additional actions running at the same time. They will attempt to use the same Wsadminlistener. This works in some instances, while other instances it hangs the listener. This can depend on the type of actions being used, but rule of thumb. Just don't do it.

It is, possible to break the example composite action above into two steps. After all, one action just calls a series of other actions. Why cant you create your own automation plan calling these steps individually?

This can be done, and to some success as well. This would allow all the associated steps to be threaded. Which would cause them to run in their own individual JVM. This has a tendency to start up a lot of JVM's, but again, depending on the type of actions being ran. It may be faster to run each one in their own JVM. Allowing you to thread them all and avoid any .lock file issues or hung wsadmin processes.

Now let's talk about the different types of steps that can be used within Rational Automation Framework along with a .dot command that can be used in those steps.

Step type determines how the step is run. This property affects the contents of Command and the project specified in Inline, if any.

Regular: The step is run once.

Conditional: The step is run once if the expression in the Condition property evaluates to true. Selecting conditional causes the Condition, Else Inline, and Else Command properties to be shown. If the Condition property evaluates to false, then the Command and Inline are not run. Instead, the Else Command and Else Inline are run if they are specified.

While Loop: The step can be run multiple times. It is run until the expression in the Condition property is false or until the maximum number of iterations is reached. Selecting While Loop causes the Condition and Max Iterations properties to be shown.

The selector is evaluated each iteration of the While Loop to determine the server to use for the iteration.

The .drill command allows you to loop over a command, executing the command once for each member of a series of values. You can specify the values on the command line, or draw them from an environment variable or register. When the system runs a .drill command, the system uses the .drill syntax to construct a series of command lines and sends them to the agent for execution.

For example, the command .drill "A,B,C,D" "echo value $1" creates the following commands:

echo value A

echo value B

echo value C

echo value D

You can group the values and reference multiple values in each group using the $n syntax. $1 refers to the first value in the group; $2 to the second value in the group, and so on. For example, .drill through "(A,B,C,D,E),(B,C,D,E,F),(C,D,E,F,G)" grouped by "()" separated by "," exec "echo 1[$1] 2[$2] 3[$3] 4[$4] 5[$5]" creates these commands:

echo 1[A] 2[B] 3[C] 4[D] 5[E]

echo 1[B] 2[C] 3[D] 4[E] 5[F]

echo 1[C] 2[D] 3[E] 4[F] 5[G]

The .drill command is used in many of the “out of the box” actions. Take this configure library steps for example.

This step, is a regular step, but uses the .drill command to run this step on each individual servers on a node. It kinda works like a Java counter. Using the ${base_server_on_node} variable and the $1.

This is one way to set up a step. You can also set up a step using a while loop and an iteration condition. Such as the step in the example below, and our install example above.

There is also another step called conditional. This step allows you to have a condition, command and Else command.

There are also just plan old regular steps as in the example below.

It is even possible to use a while loop to iterate a condition, while using the .drill command as in the example below.

NOTE:

As Mentioned before, do not thread steps that are running wsadminLongRunning

Utilize the .drill command whenever possible to iterate through a series of steps. If you thread this step, be careful to make sure you are not running multiple steps on the same physical machine.

Use longRunningWsadmin for large composite actions such as was_common_configure_all.

It is ok to break down a composite action into many individual steps. Just because an action may not be visible in the CLI or Eclipse does not mean it can't be called in a step.

Be careful of the number of open JVM's you have, and be sure to check the target system after several failed attempts at running an action. This will ensure the JVM's are not still open and running.

Use the out of the box actions as a references for creating your own automation plans.

For any additional information on any of the topics discussed above. Please use the links below.

If MTV took notice of the IT world. DevOps would have won the buzz word of the year. Many applications are labeling themselves as DevOps to gain a boost in interest. Since the focus of DevOps has been geared towards the deployment of applications. Companies may be disappointed in the lack of features that exist as technology moves forward. Rational Automation Framework is an application that helped bridge the gap between Development and Operations, it continues to be a top performer. DevOps is an ever expanding word as more and more products begin to fall victim to it's labeling.

Rational Automation Framework is a middleware application that can be used to administrate and deploy your WebSphere and WebSphere Portal environments. It's important to note the difference between administrate and deploy. Deploy typically consist of installations. Whether it be installing applications, servers or environments. Administration on the other hand is having the ability to actually configure those servers once they have been deployed. Rational Automation Framework specializes in both. Rational Automation Framework also integrates seamlessly with IBM Advance Middleware Configuration and IBM Workload Deployer. Let's go through a scenario so you can better understand RAF capabilities.

Let's say I have an environment consisting of an administrative agent managing five stand-alone servers . A Deployment Manager with Five nodes, one server on each. I want to take my network deployment environment and deploy it into the cloud. I want to be able to use this same template to deploy to various other physical machines as well. At the same time, I want to import the configuration of all five of my stand alone environments, but I only want to promote two of those server configurations to another physical machine. I also want to stand up a series of stand alone servers of my choosing at any given time consisting of identical configuration as any one of the five stand alone servers that already exists. I then want to setup those two servers and my network deployment with Self Signed Certificates, SSL and LDAP/AD. Rational Automation can handle this without even a flicker of it's UI.

First, the stand alone servers. Rational Automation Framework has the ability to reach out to the administrative agent managing your stand alone servers and import all the configuration information. I can designate one, two or all five servers. In the event that I do not have a server installed to push the configuration into. I can create a template in RAF to deploy a stand alone server identical to the one's that I have read in. I can then use this template to stand up as many stand alone servers as I want. I stand up ten of these environments on accident. I then realize that five were suppose to have one configuration while the other five were suppose to have a different configuration. All I do is promote the configuration of my choosing to the already existing stand alone servers and wallah. No need to remove the five servers and stand up five new servers with my desired configuration. I can also utilize the promote.properites file where RAF will swap key value pairs to replace any value within the configuration I may want to change during this process. Such as host name. Just add the file to the desired scope and let RAF do the work.

Second, the network deployment. Rational automation Framework will read in the entire network deployment configuration. I can create a virtual pattern for IWD based off these configurations. I can also start from scratch if I choose. If I read in an environment and realize that every time this virtual pattern is deployed, a certain variable will need to be changed. I can create a custom question on that Virtual pattern. I can then sync those patterns to Rational Automation Framework counter part AMC (Advanced Middleware Configuration). Once the patterns have been created I can use IBM Workload Deployer to send those WebSphere environments into the cloud. I can use the exact same template I used to create the virtual pattern, and deploy that WebSphere environment to any one or set of physical machines. Once that has been accomplished. I can utilize one of the over one thousand different actions to set up my environment further. I can create Self Signed Certificates, exchange those certificates, and set up SSL, LDAP/AD or start, stop, remove applications. The list goes on.

As you can see, Rational Automaton Framework is a full on middleware solution for automating WebSphere. In the event that there is something that I want to do and can't. The products xml files, ANT and Jython scripts are all editable by the user. If wsadmin can do it, I can automate it. Couple this with Rational Build Forge capabilities to add logic to my steps, via conditions, iterations and loops. The possibilities are endless. The next time you hear the word DevOps, don't just limit yourself to applications. Think big, think environments, administration, applications, virtual patterns, cloud, re-use, logic, scheduling and highly customizable. Don't jump on the DevOps train like a robber in the wild wild west.

If you are interested in Rational Automation Framework, DevOps or just trying to stay up on what is happening in Rational. Please follow the @RAF_WebSphere twitter. Thanks!!!

It's Friday afternoon as your fingers slice across the keyboard and your mouse chirps click by click. You sit anxiously awaiting an email. DING! You race to open it and scribble down your thoughts before they magically disappear. The editor prompts “Format may not be correct”. To your surprise, It's all jumbled. You go strait to google and ten editors later, success. In many instances it is easier to download on autopilot while searching Facebook and monitoring your tweets than to do any actual reading of the documentation. We just keep downloading something different until we find what we need with the expectation that is exists. After all, you can stroll into Best Buy, buy a phone, link it to your tv and stream music videos in HD with surround sound. When approaching RAF I let my expectations shape my approach. It was a horrible idea and hopefully this will save you from going down the same path.

RAF leverages Rational Build Forge, agentless connectivity (RXA), SUDO permissions, ANT, Jython, wsadmin and a base root of automation plans to Administrate WebSphere with over a thousand actions. Why not give it the ol' college try. Why not jump in and learn trial by error? I will tell you just why not to do that.

When I dove right into Rational Automation Framework some time ago. I had no plan or direction. I was like Maverick in top gun. Needless to say I found an action called install was and flew with it. Hours later with a healthy amount of frustration. When finally completed. I was excited and disappointed. I had a poor understanding of the product. It seemed every time an install failed, I was just as baffled as the first time. Chasing down errors with no idea what I was doing and hoping for the best until I was successful again. Armed with very little knowledge. I new that an action performs a task. The action came with the install of RAF. So why wouldn't it work? When the action failed, I expected it was something I did. Expecting somehow RAF new what I was trying to do. Once I stopped, removed all expectations of any kind, and approached my install with a blank slate. I started making progress. Expectations are a state of mind, seemingly simplistic but often almost always over looked.

When approaching the application without expectations. I was forced to break down the process into many different sections. As if I were developing a product to install WAS myself. I would need to break an automation plan into Projects, Libraries, Steps, and properties. With documentation in hand, I learned a project is a definition of work made up of steps. Steps control the projects behavior through properties. I would have viewed the Project, and noticed that it had an in-line. An in-line is a Library that consists of a set of steps that do not change. These steps can be called from other projects or libraries by setting it as an inline. I navigated to the library page where I saw that there are multiple types of steps. Some steps have conditions, others have loops. I noticed that the install node step was conditional. It iterated through the number of nodes I wanted to install. I also noticed how it assumed I was doing this on two separate machines. two nodes two machines. I also noticed the property called “Thread”. I am installing two nodes on one machine at the same time. RAF threads these steps making it's own assumption that you are using two or more target systems. I would have fixed this first.. Not expecting RAF to know I only had one machine without telling it first.

Lets say I found an action was_common_configure_security_federated to import AD users. I ran this action against my test environment and it failed. It didn't just fail, But locked out all the users from being able to log in. Instead of plugging that action in and expecting it to work, I should have removed all expectations and broken it down.

What is an action? Is there a way to gain additional information on an action? If I had searched for “documentation on actions” in the IBM info center. I would have found RAFW command Syntax. This would have lead me to rafw.sh/bat <scope> -p <actionName>. This command finds the xml associated with this action and tells the location of that xml file. I would have also found that I can list all the available actions at a particular scope using rafw.sh/bat <scope> -l. sifting through the XML I can see that this action calls a script. Again, telling me the name and location of the script in question. I open the Jython script and notice it's an AdminTask. AdminTask commands are not recommended to run on an active server. If a conflicting configuration is saved, it can corrupt the server. This is probably what happened. By using the second command (-l) I may have found that was_common_configure_security_all was a better action to use. It is a composite action. That runs several actions underneath it to gain all the security data. Instead of running these steps threaded one by one, opening a new JVM for each one and causing slower performance. I could have ran this one composite action in long running wsadmin and opened one JVM to run all the associated action. Although, just because long running wsadmin is specified by -useLongRunning does not mean that it will work out of the box. Get out of here expectations!

By removing all expectations and clearing the slate. You are forced to use targeted discovery in order to break down the process, target each section of the process and understand it's purpose in relation to the bigger picture. You will be much more successful following this type of outline when dealing with RAF. Even the easiest thing such as starting or stopping a JVM can get incredibly complex in the event it fails. What if your server was installed as a windows service? You shut it down, but now you can't start it. You have no control over the associated servers. Remove expectations and use targeted discovery and you will be fine. Happy Automating.