The Oracle Management Service (OMS) renders the user interface for Oracle Enterprise Manager Grid Control, interacts with Management Agents on monitored hosts and stores persistent data in the Oracle Management Repository. The Grid Control installation comprises of these components:

Other optional components include WebCache, OCMRepeater (for OCM configuration in OMS home from 10.2.0.4 Version onwards).

The Enterprise Manager user-interface is called the Grid Console and can be accessed in any certified Web browser via Oracle HTTP Server (default port - 7778) or WebCache (default port - 7777). The WebCache is not a mandatory component for the OMS functionality, it is present as part of the Application server installation but does not provide significant performance improvement for the Grid Console pages.

Note 1080406.1: Description of Important Configuration files in the 10g Oracle Management Service (OMS) Home of the Enterprise Manager Grid Control

====================================================================

OMS Process Control (How to start, stop and check OMS status?)

The OMS can be started using two utilities:

<<OMS_HOME>>/opmn/bin/opmnctl<<OMS_HOME>>/bin/emctl

The usage of these utilities is described in detail in the following document:

Note 298991.1: How to Process Control (start, stop, check status) the Oracle Management Service (OMS) Component in Enterprise Manager Grid Control

====================================================================

Modifying OMS configuration files (for changing sysman pwd, port etc)

After the OMS installation, there may be a need to modify some of the OMS configuration details.The following documents describe the OMS configuration changes that can be done, grouped by the type of activity:

Changing the SYSMAN password in the Repository

Note 270516.1: How to Change the Password of SYSMAN User in Grid Control?

OMS to Repository Database Connection and Communication

Note 736061.1: How To Re-configure the OMS After Hostname/IP Address change on the repository database server

Note 369997.1: How to Re-configure the OMS After Port Change for the Listener Servicing the Grid Control Repository Database

Note 738075.1: How to Re-configure OMS After Change in the Grid Control Repository Database Name

Note.457854.1: How to configure OMS using existing DB when RAC database is used as Repository

Note 431410.1: How to Recreate the 'opmnctl' file from Scratch on Unix Machines

Changing hostname of the OMS Machine

Note 371865.1: Is it possible to Modify the EM 10g Grid Control OMS Hostname After the Installation?

Modification of Ports used by OMS

Note 235298.1: Overview of Default Ports Used by EM 10g Grid Control, DB Control and AS Control

Controlling the Log and Trace Files for OMS

Note 229627.1: How to Locate 10g Grid Control OMS Log / Trace Files and Control their Size and Other Details

====================================================================

Diagnostic Tools Available for Troubleshooting OMS Process Control and Configuration issues

RDA

The Remote Diagnostic Agent (RDA) can be executed specifically with the Grid Control / OMS profile name: GridControl in order to reduce the number of questions that need to be answered and also to collect all details of the OMS home correctly.

The steps to execute the RDA with GridControl profile is explained in:

Note 1057051.1: How to Run the RDA against a Grid Control Installation

It is highly recommended that the latest EMDiagkit is installed and executed in the OMS home, before running the RDA. This will ensure that the RDA picks up the latest data collected by the EMDiagkit.

EMDiagkit

The EMDiagkit is a diagnostic tool developed to assist in diagnosis and correction of Enterprise Manager 10g Framework issues. At present, the tool allows us to extract necessary troubleshooting data from the EM Repository Schema using the repvfy utility.

OMS is a J2EE application deployed in a Application Server OC4J instance and needs to connects to an Oracle Database hosting the Repository, for its operations.

Failures during Startup

The OMS startup failure could occur at any of the following component level:

1. OPMN and OC4J_EM components in the Application Server.2. HTTP_Server component in the Application Server.3. OMS Application and Repository Database Connectivity.4. Specific Objects / Configuration in the Grid Control Repository.5. WebCache component in the Application Server.

My Oracle Support contains several notes with steps to resolve the specific issue being faced during OMS startup, at each of the above component levels. The best way to find these notes is to login to My Oracle Support portal and query the 'Knowledge' with the following keywords:

OMS Startup Fails <symptom that you have found in the log/trace files>

Some examples:

OMS Startup Fails with "Communication error with the OPMN server local port."OMS Startup Fails: 'opmnctl status' Shows OC4J_EM Component in 'Stop' StatusOMS Startup Fails at HTTP_Server Component if the HTTP_Server Logs are > 2 GB in SizeOMS Startup Fails at HTTP_Server Component Startup with "Address already in use:make_sock: could not bind to port"

For steps to identify the symptoms and navigate through the log/trace files, refer to the troubleshooting steps suggested in:

All the processes that are started as part of the OMS startup should always be shutdown cleanly (no stuck / defunct processes) so that the state information held by the opmn is cleaned out and not re-used during the next startup. Also, all the sessions started by the OMS application in the repository should be closed successfully.

Sometimes, the OC4J_EM component may fail to shutdown gracefully within the default timeout period, due to a heavily loaded OMS server or latency on the network between the remote OMS machines and the repository. In such cases, the opmn timeout value can be increased using the steps in:

Note 358849.1: OMS Stops Fails with 'time out while waiting for a managed process to stop' When Using 'opmnctl stopall', How to increase the Timeout?

To find documents related to problems during OMS stop, login to My Oracle Support portal and query the 'Knowledge' with the following keywords:

The 'opmnctl status'command shows the status of the necessary opmn components needed for OMS functionality - HTTP_Server and OC4J_EM componentetc.The status of the components could be in these status :

Alive: indicates that the component is up and is responding to the opmn status check.Down: indicates that the component is not running.Init: indicates that the component is still in the process of initialization. This is a transitional status only. Stopped: indicates that the component has been stopped by the opmn as it is unable to start completely or the opmn is trying to clean up the state information. This is a transitional status only.

If the OC4J_EM component is in Alive status, it only means that the opmn-managed OC4J_EM component is running - the 'em' / OMS application may or may not be actually running.

emctl status oms

This command helps in verifying:

- whether the 'em' / OMS application is actually running or not. - whether the HTTP server is running or not.- whether the OMS is able to successfully connect to the sysman schema in the Grid Control Repository Database.- whether the emkey is configured properly.

Note: If the OC4J_EM status is Alive, but the 'emctl status oms' returns an error that the Management Service is not Up, then the Grid Console cannot be accessed from the browser.

To find documents related to problems during OMS status check, login to My Oracle Support portal and query the 'Knowledge' with the following keywords:

'emctl status oms' Fails <symptom that you have found in the log/trace files>

For steps to identify the symptoms and navigate through the log/trace files, refer to the troubleshooting steps suggested in:

The RDA output generated with the GridControl profile is very useful in obtaining all the configuration files and log/trace files together. Specific logs which will be helpful for each component are described in Note 730308.1.

The EMDiagkit output is very useful in diagnosing problems with Grid Control Repository objects, which can cause the OMS startup failures.

Note: It is highly recommended that the latest EMDiagkit is installed and executed in the OMS home, before running the RDA. This will ensure that the RDA picks up the latest data collected by the EMDiagkit.

====================================================================

Troubleshooting OMS Configuration

Modifying the OMS configuration in the emoms.properties, httpd_em.conf or any other configuration files may be done incorrectly, resulting in OMS startup failure or problems during its operations:

This section lists some of the best practices which will help prevent problems with OMS Process Control and Configuration:

EM Certification Checker

It is strongly recommended that you always use a certified combination of OMS, Agent and Repository Database for managing Targets which are certified with this combination.The Enterprise Manager certification details are available in:

Enable Log Rotation for the access_log and error_log files created by the httpd_em.conf file:Note 339819.1: How to Enable Rotation for the HTTP_Server and OC4J_EM Logfiles in the 10g Grid Control OMS Home?

Execute EMDiagKit and regular intervals (once per week or more frequently, depending on your setup) and check for any new problems that are reported.

Take valid backups of the OMS and Repository Database Homes at regular intervals, to restore back any configuration files that are deleted by accident.

Remove the emkey details from the repository and store the <<OMS_HOME>>/sysman/config/emkey.ora file in a safe place. If the emkey is lost, there is no way to recover it except for re-installing all the OMS and Repository.

OCM

Oracle Configuration Manager (OCM) works with My Oracle Support to enable proactive support capability that helps you organize, collect and manage your Oracle configurations by providing Proactive configuration-specific notification of Security and General Alerts, HealthCheck recommendations based on Support Best practices when using configuration auto-collection, Simplified Service Request logging, tracking and reporting and Project cataloging of key milestones and contacts associated with your configurations.

Healthchecks are executed dynamically against the Oracle Configuration Manager uploaded configurations in My Oracle Support. These checks, based on Oracle Best practices, will proactively notify you of potential problems in your environment, and provide recommendations that help you improve system performance and avoid problems in your Oracle environment.

If you are receiving any Healthcheck alerts in My Oracle support, then refer to the following document for the alert details and its corresponding document for resolving the same:

Critical Patch Updates (CPU) is the primary means of releasing security fixes for Oracle products. They are released on the Tuesday closest to the 15th day of January, April, July and October. This page lists all the currently available Critical Patch Updates (CPUs) in chronological order and is updated whenever new Critical Patch is released. You can also subscribe to the CPU Email Alerts using the steps listed here.

To obtain the latest CPU patch details for the Enterprise Manager Grid Control and its dependent products - Oracle Application Server and Oracle Database:

- In the page, click on the link shown for the latest CPU in the table under the 'Critical Patch Updates'.- The next page, lists all the products which have security fixes in the chosen CPU release. Scroll down to 'Patch Availability Table ..' topic and find the table with details for the Product Group and Patch Availability and Installation Information. - In the table, find the row related to Product Group: 'Oracle Enterprise Manager' and pick up the document number given in the Patch Availability and Installation Information column. In the document, navigate to:

Patch Set Updates (PSU) are proactive cumulative patches containing recommended bug fixes that are released on a regular and predictable schedule. PSUs are on the same quarterly schedule as the Critical Patch Updates (CPU), specifically the Tuesday closest to the 15th of January, April, July, and October. The PSUs serve as a new baseline version for reporting issues to Oracle, hence it is always recommended to be on the latest PSU release.

The PSU and CPU released each quarter contain the same security content. However, the patches employ different patching mechanisms, so customers need to choose wisely which patch satisfies their needs better:

·A PSU can be applied on the CPU released at the same time or on an any earlier CPU for the base release version. A PSU can be applied on any earlier PSU or the base release version. CPUs are only created on the base release version.

·Once a PSU has been installed, the recommended way to get future security content is to apply subsequent PSUs. Reverting from PSU back to CPU, while possible, would require significant effort, and so is not advised.

Getting CPU / PSU patch recommendations via OCM

OCM also collects and recommends the latest CPU and PSU patch that can be applied to a particular Oracle Home. These details can be seen in the My Oracle Support ->Patches and Updates -> Patch Recommendations section