A delve into the disturbing world of the Oracle Hyperion product set, uncovering the mysteries and facts beneath the shiny surface, yawn your way through yet another blog, let's hope this one is interesting...

Tuesday, 19 June 2012

I thought I would put together a write up on compact deployment in 11.1.2.2 as judging by some of the posts on the OTN forums lately there is a bit of confusion around this area.

I did write up a few posts in the past on compact deployment in 11.1.2.1 which can be read here, here and here, luckily the process in 11.1.2.2 has been much more simplified.

Now I must stress that all the information contained in this blog is just my interpretation and there may be some inaccuracies, as always if you feel there is then please get in touch and share your views.

So what is a compact deployment, well basically all it means is that instead of each web application being deployed to its own managed server they are combined and deployed as one, the number of web applications deployed to one managed server depends on what is selected when you run the EPM configurator which I will go through shortly.

So why have a compact deployment, well I think one of the main reasons the compact deployment came about was the big swing in the amount of resources required when entering the world of 11.1.2.x. In previous versions it was easy for say a consultant to run the full suite of Hyperion products on a laptop or run a POC, demo or training on an average spec machine, when 11.1.2.x came along this all changed as WebLogic had to be deployed and with the number of web applications and the amount of memory consumed this stopped many from being able to run what they could in previous versions.

Probably down to the amount of complaints and pressure then Oracle came up with the compact deployment method.

As all web applications share one JVM then overall performance may not be as good.

If there is an issue with one web application then this can stop the managed server from running.

Reduced logging information.

The advantages definitely out way the disadvantages if you are using EPM for development, training, POCs etc Oracle do say that compact deployment is now supported in production, I am not going to get into a debate of whether that is a good idea but personally I have reservations about using it in a production environment.

Oracle has also helped out those who want to quickly deploy Essbase, Planning or HFM to a one server machine which is known as a rapid deployment and uses the compact deployment method.

The rapid deployment documentation contains a step by step guide to deploying on Windows 2008 R2, the documentation even goes into detail on the OS configuration and steps for installing Oracle as the database.

Ok, so let’s go through the steps of a compact deployment but please note this is not a step by step configuration guide and does not contain screenshots of every panel.

After the required products have been installed the EPM configurator is started and all the Web Applications which are going to be deployed to one managed server can be selected, if there are web applications you don’t want to be part of the compact deployment and you want them to run under their own managed server you can deploy them later.

Please note that by default all the available Web Applications will be selected for deployment so if you don’t want to deploy all the web applications to one managed server only select the ones that you do.

If it is the first time the web applications are being deployed I make sure that the configure database is selected so the datasources are configured correctly in WebLogic.

It is important to understand which web applications you actually want to deploy to one managed server first as even though the memory footprint is reduced from a standard deployment the more web apps you select the higher the totally memory consumed will be.

There may be web applications that you want to deploy but are not going to use often so it is possible you wouldn’t include them in the compact deployment and deploy them to their own managed server and start when required though this could depend on the resources available.

If the configure database option is selected for multiple products then you will only be able to configure to one database, if individual database/schemas are required then it is possible to select the configure database for one product, configure and then move on to the next product to configure the database until all have been configured and then deploy the web applications.

As this is the first deployment the web applications will need to be deployed to a new domain, most of the time the default configuration can be kept and just a password provided.

Now for the most important panel in a compact deployment and I think this is where the biggest change is with 11.1.2.2 deployments, “Deploy the web applications to a single managed server” is selected so a compact deployment is the default.

You will notice that all web applications will be deployed to one managed server called EPMServer0 and all web applications will be running on one port which the default is 9000.

If you deselect “Deploy the web applications to a single manage server” then it is standard deployment where each web application will be deployed to its own managed server and running on a port which you are probably more accustomed to.

It is possible to change the default port of 9000 or SSL port 9443 if required.

I think it is worth going over the web server confiugration as well as I have noticed some confusion over it, Oracle HTTP Server comes as part of a separate download and it is possible to install without including it, now if you take this route the default web server will be the newly introduced Embedded WebLogic one.

The Embedded WebLogic HTTP Server uses a proxy servlet which is bound to Foundation services so whatever port Workspace/Shared Services is running on will define the port http server is running on, in a default compact deployment the port is 9000 so the http server will be running on port 9000.

If you have a compact deployment which includes Workspace/Shared Services using the default port of 9000 and using the embedded http server you would access over port 9000.

If you are using OHS as the http server then the default would be port 19000.

It can get confusing if Foundation was deployed to its own managed server and the embedded http server was chosen as then the default port for the http server would now be 28080.

It is worth noting that the embedded http server is not supported in production environments and I have also noticed rendering issues when using it with IE9 which don’t seem to exist if using Firefox, the Oracle http server is not a big overhead in terms of resources so it might be the best option anyway.

If connecting to say EAS and it is part of a compact deployment then to be able to connect successfully you would need to include the port, this could be either the direct port EAS is running on or via the http server port.

If the deployment to one managed server is on a windows server then a service called “Hyperion EPM Server – Web Application” will have been created.

It can also be started up by a script startEPMServer.bat/sh in <MIDDLEWARE_HOME>\user_projects\<instancename>\bin

If you start up the WebLogic admin server then you will see EPMServer0 has been deployed and is running on port 9000.

You will also be able to view all the web applications that have been deployed as part of the compact deployment.

The managed server can be monitored just like any other using Enterprise Manager which is installed and configured by default in 11.1.2.2

The combined logs are available in the services directory for windows and the starter directory for *nix.

The individual logs are available in <MIDDLEWARE_HOME>\user_projects\domains\EPMSystem\servers\EPMServer0\logs

I have noticed there is not as much information available as when the web applications are deployed to their own managed server.

So how does the memory consumption compare between a standard and a compact deployment.

With the ten web applications I deployed to one managed server this stabilised after start up at around 1.2GB

The equivalent memory when deployed to individual managed servers averaged at around 6GB so you can see a massive difference.

Remember there is the overhead of other services to consider which depends on the products installed e.g. RAF Agent, EPMA, Essbase, RMI

The maximum JVM size for a compact deployment is set to 2701mb which can be increased if required either in the registry under

<MIDDWARE_HOME>\user_projects\<instancename>\bin\deploymentScripts\setCustomParamsEPMServer.bat/sh and updating the set JAVA_OPTIONS line.

So how about a comparison on start-up times between compact and standard.

Please note this is not scientific and is based on the web applications I deployed and obviously depends on the hardware being used.

One managed server averaged at 3 ½ - 4 minutes.

The equivalent individual managed servers if started one by one and waiting until each one had fully started up before starting the next one was about 30 minutes, if all the individual services were started at once which is not always advisable then it took around 7 minutes.

What I did notice was that if you didn’t start the RAF agent first then the above errors appeared in the error log which caused the overall start-up time to increase.

This was not just applicable to the compact deployment as it happens if RAF web is deployed to its own managed server and the RAF agent is not started first, It doesn’t cause any problems if it is started up after it just slows down the start-up time, I am sure this did not occur in previous versions of 11.1.2

Anyway my testing of start-up times was done with the RAF agent started first.

It is possible to scale out the compact deployment to additional machines if required and to do this you first need to install the same set of web applications on the additional machine.

When you start up the EPM configurator an option will become available called “Scale out compact server on this machine”

This would then deploy exactly the same web applications as in the compact deployment on the first machine.

If you then look in WebLogic admin console you notice that EPMServer1 has been created and is part of the EPMServer cluster.

You also need to run the Web Server configuration again as this will then add the scaled out deployment to the http server configuration.

Make sure the Weblogic admin server is running before starting up the EPM Server on the scaled out server as otherwise it will not start and you will be hit with the above error, the admin server does not need to be running for subsequent start-ups.

If you want to add additional web applications to the compact server

In the configurator select “Deploy to Application Server” for web app you want to add to the compact deployment.

You should see the additional web application(s) available to be deployed to the managed server.

Make sure you also configure the “Foundation” - “Web Server” component so that newly deployed applications is also configured with the http server.

If you have already scaled out the deployment then this is what the documentation says when adding an additional web application

“If you add additional Web applications to the single managed server and you have multiple machines configured with the single managed server, for each additional machine (other than the WebLogic Administration Server machine), do the following:1. Install the additional applications.2. Restart the single managed server.”

I think this is missing a step because if you only follow those steps then the web applications on the scaled out server will not be added to the Shared Services registry and will not be added to the http server configuration.

So the registry will only contain web application information for the EPMServer0 and the http configuration will not add in the scaled out server even if you run the configure web server component again.

By selecting “Scale out compact server on this machine” again this should configure the additional web application.

Running the registry report displays the web application has been added against EPMServer1 and after running “Configure Web Server” the additional configuration for the scaled out server has been added.

Now on to the final piece and that is removing a web application from a single managed server, there may be a number of reasons why you want to do this such as if you want a web app to run in its own managed server instead of being part of the single managed server.

The documentation states

"Use the WebLogic Administration Console to remove any Web applications from the single managed server, and restart the single managed server on all machines.

If you uninstall any product from a machine that is part of the single managed server on that machine, the entire single managed server on the machine is removed."

It would be so much simpler if the configurator just provided the ability to deselect web applications from the “deploy to application server” panel.

Let me go through the first option “WebLogic Administration Console to remove any Web applications from the single managed server” and try and remove the planning web application.

The planning deployment is deleted from within the admin the console though the problem with this method is that from an EPM configuration perspective planning will still exist.

Restarting the web application and logging into workspace planning will still exist even though it has been deleted.

If you try to configure again the web application deployment still believe Planning is part of the compact deployment.

So what about the other statement in the documentation

“If you uninstall any product from a machine that is part of the single managed server on that machine, the entire single managed server on the machine is removed”

The uninstaller was run from <MIDDLEWARE_HOME>\EPMSystem11R1\uninstall\uninstall.cmd/sh and the planning web application removed.

Maybe I am misreading the documentation but the single managed server is not removed.

Anyway the process I finally went through to remove a web application from a compact deployment was

Uninstall the product on each server the single managed server has been deployed to.

Go to into the WebLogic admin console and delete all the deployments related to the web application.

“Configure Web Server” again in the EPM configurator to update the http server configuration.

If you wanted to then deploy the removed product to its own managed server you could reinstall the product, select “Deploy to Application Server” in the configurator just for that web application and in the deploy to application server panel deselect “Deploy the web applications to a single managed server”.

Monday, 4 June 2012

No doubt you aware of the EPM Shared Services registry but if you are a little unsure on what it is all about then here is the official Oracle description.

The Shared Services Registry is part of the database that you configure for Foundation Services. Created the first time that you configure EPM System products, the Shared Services Registry simplifies configuration by storing and reusing the following information for most EPM System products that you install:

• Initial configuration values such as database settings and deployment settings• The computer names, ports, servers, and URLs you use to implement multiple, integrated, EPM System products and components

Configuration changes that you make for one product are automatically applied to other products used in the deployment.

The Registry was introduced in version 11 and it was Oracle’s answer to centralise all the property files that were scattered everywhere in previous releases, I did write a blog about the registry and property files a few years ago so I am not going to go over old ground

The registry basically comprises of a set of six relational tables which are held together by component ids , it is not advisable to start changing the information directly in the tables as there are a number of supported methods to control it, this can be done by the command line epmsys_registry utility or for the majority of registry properties by using the LCM import/export features in Shared Services.

The registry works quite well most of the time but like most of products it is one that has improved over time, though saying that with a combination of maintenance releases, reconfigurations and bugs the registry can also get into a bit of a mess, sometimes it is not noticeable but other times it can cause nothing but a headache.

Luckily Oracle were aware of the issues that have arisen with the registry and have provided a utility to clean it up.

At the time of writing this there is not much documentation available on the utility so I apologise if there are any inaccuracies as it is just my own interpretation.

The utility is based on six rules defined in an XML file for cleaning up the registry and these are

Remove components without parent HOST node

Remove APP_SERVER components with just HOST components in parents

Remove components without any children or parent nodes

Remove HOST to HOST links

Remove webapps with invalid ‘serverName’ property

Merge duplicate webapps

From 11.1.2.2 the utility is installed by default and is available in <MIDDLEWARE_HOME>/user_projects/<instancename>/bin and can be run by executing registry-cleanup.bat/sh

The utility will go through each of the six rules and search for problems in the registry, it will skip certain component configurations which are also defined in the rules xml file.

This was run on a fresh install of 11.1.2.2 so as expected the utility didn’t encounter any issues.

If you are running 11.1.2.1 then the utility is also available as a patch and is definitely worth having a look, I would probably consider it a prerequisite if going through a maintenance release to 11.1.2.2

The nice feature of the utility is that you don’t have to take any action if problems are found and can just use it to see how the registry is looking.

I will now go through the process of getting up and running with the utility on 11.1.2.1

The patch 13807599 :Providing a cleanup script for Shared Services Registry issues is available from Oracle Support

Once downloaded extract the patch to <MIDDLEWARE_HOME>\EPMsytem11R1\Opatch

Once the patch has been applied you will find the utility in <MIDDLEWARE_HOME>\EPMSystem11R1\common\config\11.1.2.0 which is a different location than the one preinstalled on 11.1.2.2

Please note before running the utility make sure you perform a backup of the database/schema holding the Shared Services Registry as even though this utility is there to clean up the registry there is also a chance it could also do more damage.

The utility can be run from by executing registry-clean.bat/sh

So let’s give the utility a try on an EPM 11.1.2.1 instance though I am not sure how many issues it will actually find as I have not really had many problems with it.

Once the utility starts up it asks for the EPM instance location which is different than the 11.1.2.2 version, I think it just requires the epm instance so it can locate the reg.properties file which contains the connection information to the Shared Services Registry database.

The utility now steps through the six rules with the first being “Remove components without parent HOST node”

The utility finds no issues with that rule and skips four components according to the definition in the rules xml file.

The next rule is to “Remove APP_SERVER components with just HOST components in parents”

This time the utility finds one component matching the rule and provides five different options to take action upon.

Selecting I provides full details of the entry in the registry.

If you run a registry report using E:\Oracle\Middleware\user_projects\epmsystem1\bin\epmsys_registry.bat/sh then you can view the same entry in the registry.

It is also possible to run a SQL query against the registry tables to view the same information though please note these are not the only records in the registry tables relating to this entry.

Selecting Y deletes the component out of the registry.

A quick query confirms the records have been removed.

The utility runs through the remaining rules which with the instance I chose didn’t find any more issues.

So give the utility a trial as you don’t have to delete any entries and you have the ability to see what shape the registry is in.