Popular Posts

Friday, May 18, 2012

NOTE:

Please note that the instructions given in this blog post are applicable ONLY for ESB-4.0.X and G-reg-4.1.X, G-reg-4.5.0 versions. The deployment approach of the latest ESB versions has been changed and the recommended deployment synchronization approach has been switched to SVN based synchronizer. Because of that the following set of instructions cannot be applied to the latest WSO2 ESB and G-reg product versions.

Managing artifacts of a service oriented solution is one of the most important features expected from any SOA middleware platform. In a typical service oriented project, SOA artifacts are subjected to move through multiple phases. Usually, separate environments are maintained for the activities associated with those phases.
e.g:-

Development environment

Service development and system integration tasks are carried out in a separate physical environment. Depending on the requirements, there can be multiple SOA middleware solutions used in development environment to facilitate service development, integration and deployment processes.Once the development tasks are completed, SOA artifacts are transferred into QA environment for QA verification.

QA environment

Solution testing is done in an independent environment which is usually identical to the production settings. Various functional and non-functional tests are performed in QA environment. Upon successful QA verification, the artifacts are moved to production or staging environment

When the service oriented solution becomes complex and there are large number of SOA artifacts, automated artifact governance mechanism is required to ensure smooth transition between various environments. In this post, I will take you through a simplified SOA artifact governance process using WSO2 SOA middleware stack. We will explore a use case similar to the following.

Separate development and QA environments are used to maintain the SOA artifacts. For example, solution developers implement various configuration artifacts in ESB development node. After completing those development tasks, solution developers move those artifacts in to QA environment which allows the testers to carryout QA activities in an independent environment. The moving of artifacts will be done automatically using the features provided by central SOA governance solution.
In this example, we will use WSO2 ESB as the enterprise service bus middleware, WSO2 Governance Registry as the SOA governance solution.

Lets go through each of the steps in detail. We need to have 2 WSO2 ESB instances and 1 WSO2 G-reg instance which runs on mySQL. All these instances will be deployed in single host.

Setting up G-reg and ESB productsStep 1
Download the latest versions of WSO2 ESB and WSO2 G-reg from here. First we will configure WSO2 G-reg instance. Extract wso2greg-4.x.x.zip into your file system. We will refer to the extracted location as GREG_HOME.
By default, WSO2 products run on the file based embedded H2 database. Since we need two ESB instances connect to the database used by G-reg instance, we will configure G-reg to run on a mySQL database. Enter the following commands to create a database and grant permission to a user.

Step 3
We have completed the configurations of WSO2 Governance Registry instance. Lets move forward with setting up ESB instance in development environment.

Extract the downloaded wso2esb-4.x.x.zip into a directory in your file system. We will refer to this directory as ESBDEV_HOME. Since we are running all product instances in a single machine, we need to start them up in different ports. Therefore, change port offset parameter in ESBDEV_HOME/repository/conf/carbon.xml as follows.

<Offset>1</Offset>

Step 4
As shown in the above diagram, we will mount /_system/config collection, where the ESB artifacts are stored in to /_system/dev collection of G-reg instance. ESBDEV_HOME/repository/conf/registry.xml is used to configure that.

Start ESB dev instance. If you log in to management console of the ESB instance (https://localhost:9444/carbon) and access registry browser, you will notice the config and governance collections are mounted (See the arrow icons) to the remote G-reg database.

Step 5

Repeat step 3 and 4 with another copy of wso2esb-4.x.x.zip. Lets call it ESBQA instance. (The root directory of ESBQA instance will be referred to as ESBQA_HOME). Make sure to specify a different offset value in carbon.xml. In this instance, we will mount /_system/config collection to /_system/qa collection of G-reg instance. Update registry.xml of ESBQA node as follows.

Start the server. We are done with the product configurations. However, we have not done any artifact governance configuration yet. In the next steps, we will look into moving artifacts between the two ESB instances.

Using WSO2 Governance Registry to move artifacts between environments
As I explained at the beginning, we use central governance registry instance to manage the artifacts produced by each of DEV and QA environments. The ESB artifacts produced by development environment are stored under /_system/dev collection of governance registry where as the artifacts used in QA environment are stored in /_system/qa collection. When the development tasks are compelte and ready for QA, we can manually copy the artifacts into the relevant locations and configure the QA environment. Obviously it will be a painful task to copy large number of artifacts into various locations of ESB QA environment by hand. Therefore, we need some kind of automated artifact copying mechanism.
WSO2 Governance Registry provides us with registry extension features to extend the core functionality of G-reg to use in these types of situations. The lifecycle management feature is such a useful extension provided by WSO2 Governance Registry which can be used to manage life cycle of a resource stored in registry.
The standard WSO2 G-reg distribution is shipped with a default ServiceLifeCycle which can be used to move service artifacts among different environments. We will modify the default ServiceLifeCycle to copy artifacts stored under /_system/dev collection to /_system/qa when promoting from development stage to testing.

Log in to G-reg management console and navigate to Extensions --> Configure --> Lifecycles. Click on Add New LifeCycle and add the following life cycle.

You can see that we have used org.wso2.carbon.governance.registry.extensions.executors.CopyExecutor class to copy one or more resources from one environment to another. Executors are custom extensions to G-reg which used to trigger a custom execution logic at the time of state transition (e.g:- Dev to QA, QA to Production). When artifacts are promoted to Testing state from Development, the CopyExecutor is triggered and the events defined under transitionExecution element are carried out. In this lifecycle, CopyExecutor copies any resource which is at currentEnvironment to targetEnvironmentNote that, CopyExecutor has not been shipped by default in WSO2 G-reg-4.1.X versions. Hence, you need to implement org.wso2.carbon.governance.registry.extensions.interfaces.Execution interface. However, in current trunk based versions (which will be released within few months), this executor is included by default and you can use it out-of-the-box.
Once the life cycle is created and saved, we can apply it to the relevant collection in registry. In this example, we need to move the artifacts stored at /_system/dev to /_system/qa. Hence, we must apply ESBLifecycle to /_system/dev collection.

Navigate to /_system/dev collection in registry browser. Click on Lifecycle in the left pane. Select Add Lifecycle and choose ESBLifeCycle from the dropdown list. Click on Add to add ESBLifeCycle to /_system/dev collection.

We do not have any artifacts in /_system/dev collection yet. Therefore, lets add some ESB artifacts through ESB management console. Log in to the management console of ESB DEV instance and add a sequence. (say it is "devsequence). This will create a new sequence in ESB and it will be stored under /_system/dev in G-reg

Now, go back to G-reg management console and navigate to /_system/dev collection. Select the check-list items such as Configurations Completed, Transform Rules Defined etc.. and click on Promote.

After promotion, click on Search --> metadata in left menu in G-reg management console and search for the resource which has just been added ("devsequence"). The resource will be copied to /_system/qa collection.

We have just completed automatic promotion of resources which have been created from ESB DEV environment to ESB QA environment. However, you will notice that the ESB sequence artifact which has just been added, is not "deployed" in ESB QA instance. Because of that, you will not see "devsequence" in sequence list of ESB QA instance. The artifact is added to the relevant registry location but in order for deployment of the resource, ESB needs to check-out the resource from /_system/qa/repository/deployment/server/synapse-configs/default/sequences location in registry to the corresponding deployment directory of file system of ESB QA server.

DeploymentSynchronizer comes into action in this situation. As I have explained in a previous blog post, DeploymentSynchronizer can be used to synchronize deployment artifacts among cluster nodes.

- From WSO2 Carbon-4.0.0 release onwards, we need to enable Tribes based clustering in order for deployment synchronizer to communicate with cluster nodes. Open ESBDEV_HOME/repository/conf/axis2/axis2.xml and enable clustering and specify a unique domain name.

We have completed enabling deployment synchronizer for both ESB development and QA servers. Now, try out adding another sequence from ESB DEV instance (say it is "devsequence2"). Once the sequence is added, access G-reg management console and browse /_system/dev collection. Promote the lifecycle again. This time, you will notice that devsequence2 is deployed in ESB QA instance upon successful promotion

In this post, I took you through the steps of moving artifacts between different environments using WSO2 ESB and WSO2 G-reg. We created artifacts in ESB using management console. However, the recommended best practice is to build SOA artifacts using a tool like WSO2 Carbon Studio and upload the CAR artifacts to ESB. We will look into CAR based SOA governance mechanism in a future post.

Web Services Testing with soapUI

About Charitha

Charitha Kankanamge possesses over 14 years of experience in various technological domains including software testing/QA and middleware technical support. Currently, he works as a Software Quality Assurance Manager at Amazon. Previously, he worked for an open source software company, WSO2 as Senior Manager, Technical Support. He is also a committer of the Apache Software Foundation. Charitha's strengths and key experiences include functional testing, SOA testing, test planning and agile/exploratory test mechanisms.