Concurrent Processing - CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables (Doc ID 134007.1)
Manager Status Shows System Hold, Fix Manager before resetting counters (Doc ID 1507580.1) To BottomTo Bottom
Modified:22-Nov-2012Type:PROBLEM
Rate this document Email link to this document Open document in new window Printable Page
In this Document
Symptoms
Changes
Cause
Solution
References
This document is being delivered to you via Oracle Support's Rapid Visibility (RaV) process and therefore has not been subject to an independent technical review.
Applies to:
Oracle Concurrent Processing - Version 12.1.3 and later
Information in this document applies to any platform.
Symptoms
The Administer Concurrent Managers screen shows the Scheduler/Prereleaser Manager with a status / state of "System Hold, Fix Manager before resetting counters"
Changes
This issue started after bouncing all servers.
Cause
An orderly shutdown of the concurrent manager was not performed. The CP Diagnostic Request Analyzer" shows that a CP Shutdown was initiated at on 16-OCT-12 at 22:22:41 and then cancelled.
Yet, the database alert log file shows that the database was shutdown within four seconds of the concurrent managers being shutdown/cancelled, not providing enough time to perform an orderly shutdown of the concurrent processing server.
Solution
1. Shutdown the Concurrent Processing (CP) server in an orderly manner and verify that the shutdown is complete before invoking a shutdown of the database and/or the database listener. Check the last CP process to exit/terminate, the Internal Manager, with the Unix command: ps -ef | grep FNDLIBR
If a manual shutdown is being performed, use the same sequence found in the adstpall.sh – "Stop All Oracle E-Business Suite Enabled Services" script: first the CP Server--once down--shutdown the App Listener, the Forms Server, the Web (Apache) Server. Lastly, if necessary, shut down the database listener and/or database.
Note 1: The start up is the same, but in reverse order. In this case, a complete bounce of all servers (apps/db tier) restored a normal manager status/state.
2. In the event that the CP server was not shutdown properly, run cmclean.sql as per Doc ID Note 134007.1. In some cases, it may be necessary to bounce the database.
Note 2: Shutting down the database or the listeners without an orderly shutdown of the Concurrent Processing server is the equivalent of pulling the power cord on a Unix box. 999 times out of 1000, the Unix system will recover without any incident; however, at some point file/disk corruption or an odd state will occur; similarly, the same applies to the CP server, manager queue and process state information may not be current or correct.
Note 3: The cmclean.sql script is a safe script created by CP Development. It is a helpful script for re-initializing the managers, particularly when a manager's normal operation may have been compromised by manual intervention by a DBA or System Administrator (by manual killing of OS / RDBMS processes) or due to unplanned system outages (an accidental closing of the database without doing an orderly shutdown of the concurrent managers, power / network outages, etc).
References
NOTE:134007.1 - Concurrent Processing - CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables
Concurrent Processing - CMCLEAN.SQL - Non Destructive Script to Clean Concurrent Manager Tables (Doc ID 134007.1) To BottomTo Bottom
Modified:23-Oct-2013Type:SCRIPT
Language:
Rate this document Email link to this document Open document in new window Printable Page
Applies to:
Oracle Concurrent Processing - Version 11.5.10.0 to 12.2 [Release 11.5 to 12.2]
Information in this document applies to any platform.
Checked for relevance on 13-APR-2013
Applications Install 10.7 to 12.1.3
Purpose
This document provides a reference to self service Cleaning the Concurrent Manager tables
Non Destructive Script to Clean Concurrent Manager Tables
NOTE:
This script works with 10.7, 11.0, 11.5 & 12.1.3 Applications.
Information Center, Diagnostics, & Community
E-Business Concurrent Processing Information Center Document 1304305.1
Please reference this document regularly to review current offerings for Concurrent Processing needs.
Diagnostics
For additional help, please refer to one of the following documents on diagnostics to address current needs. Providing diagnostic output on an issue for support when logging a service request is very helpful.
Document 179661.1 for 11i or Document 421245.1 for Rel 12.x
Core Concurrent Processing Community
Visit the Core Concurrent Processing community for help from industry experts or to share knowledge.
Requirements
SQLplus
Configuring
Copy from the first REM statement to the last REM statement of this document and save as: cmclean.sql.
NOTE:
Ensure that No FNDLIBR processes are running as detailed within the Troubleshooting Note 104541.1 and that the Concurrent Manager is down.
Issue a commit once the script is run for the changes to take effect.
Instructions
Please run the Concurrent Manager Recovery feature to address any Concurrent Manager / Concurrent Processing issues within the Oracle Application Manager.
Using the Concurrent Manager Recovery wizard is the method to clear the errors upon bringing the internal manager back up.
The cmclean script can still be used for Application instances provided the managers are down and no FNDLIBR processes are still running.
For Concurrent Internal Manager failures, it is recommended to run the Concurrent Manager Recovery feature using the Oracle Applications Manager. This feature should be used for recovering from when the Internal Manager won't start. This is accessed from the Troubleshooting wizards available within applications logged in as the sysadmin userid.
Navigate:
Oracle Applications Manager > Concurrent Managers OR Concurrent Requests > Site Map > Diagnostics and Repair > Concurrent Manager Recovery
For information on the Concurrent Manager Recovery feature, please reference the Oracle Applications System Administrator's Guide - Maintenance provides information for frequent tasks such as monitoring your system with Oracle Applications Manager, administering Oracle E-Business Suite Secure Enterprise. Search, managing concurrent managers and reports, using diagnostic utilities including logging, managing profile options, and using alerts.
To run cmclean.sql:
Usage: sqlplus @cmclean
Caution
This sample code is provided for educational purposes only, and is not supported by Oracle Support. It has been tested internally, however, we do not guarantee that it will work for you. Ensure that you run it in your test environment before using.
Script
REM
REM FILENAME
REM cmclean.sql
REM DESCRIPTION
REM Clean out the concurrent manager tables
REM NOTES
REM Usage: sqlplus @cmclean
REM
REM
REM $Id: cmclean.sql,v 1.4 2001/04/07 15:55:07 pferguso Exp $
REM
REM
REM +======================================================================+
set verify off;
set head off;
set timing off
set pagesize 1000
column manager format a20 heading 'Manager short name'
column pid heading 'Process id'
column pscode format a12 heading 'Status code'
column ccode format a12 heading 'Control code'
column request heading 'Request ID'
column pcode format a6 heading 'Phase'
column scode format a6 heading 'Status'
WHENEVER SQLERROR EXIT ROLLBACK;
DOCUMENT
WARNING : Do not run this script without explicit instructions
from Oracle Support
*** Make sure that the managers are shut down ***
*** before running this script ***
*** If the concurrent managers are NOT shut down, ***
*** exit this script now !! ***
#
accept answer prompt 'If you wish to continue type the word ''dual'': '
set feed off
select null from &answer;
set feed on
REM Update process status codes to TERMINATED
prompt
prompt ------------------------------------------------------------------------
prompt -- Updating invalid process status codes in FND_CONCURRENT_PROCESSES
set feedback off
set head on
break on manager
SELECT concurrent_queue_name manager,
concurrent_process_id pid,
process_status_code pscode
FROM fnd_concurrent_queues fcq, fnd_concurrent_processes fcp
WHERE process_status_code not in ('K', 'S')
AND fcq.concurrent_queue_id = fcp.concurrent_queue_id
AND fcq.application_id = fcp.queue_application_id;
set head off
set feedback on
UPDATE fnd_concurrent_processes
SET process_status_code = 'K'
WHERE process_status_code not in ('K', 'S');
REM Set all managers to 0 processes
prompt
prompt ------------------------------------------------------------------------
prompt -- Updating running processes in FND_CONCURRENT_QUEUES
prompt -- Setting running_processes = 0 and max_processes = 0 for all managers
UPDATE fnd_concurrent_queues
SET running_processes = 0, max_processes = 0;
REM Reset control codes
prompt
prompt ------------------------------------------------------------------------
prompt -- Updating invalid control_codes in FND_CONCURRENT_QUEUES
set feedback off
set head on
SELECT concurrent_queue_name manager,
control_code ccode
FROM fnd_concurrent_queues
WHERE control_code not in ('E', 'R', 'X')
AND control_code IS NOT NULL;
set feedback on
set head off
UPDATE fnd_concurrent_queues
SET control_code = NULL
WHERE control_code not in ('E', 'R', 'X')
AND control_code IS NOT NULL;
REM Also null out target_node for all managers
UPDATE fnd_concurrent_queues
SET target_node = null;
REM Set all 'Terminating' requests to Completed/Error
REM Also set Running requests to completed, since the managers are down
prompt
prompt ------------------------------------------------------------------------
prompt -- Updating any Running or Terminating requests to Completed/Error canceled by CMCLEAN
set feedback off
set head on
SELECT request_id request,
phase_code pcode,
status_code scode
FROM fnd_concurrent_requests
WHERE status_code = 'T' OR phase_code = 'R'
ORDER BY request_id;
set feedback on
set head off
UPDATE fnd_concurrent_requests
SET phase_code = 'C', status_code = 'E'
WHERE status_code ='T' OR phase_code = 'R';
REM Set all Runalone flags to 'N'
REM This has to be done differently for Release 10
prompt
prompt ------------------------------------------------------------------------
prompt -- Updating any Runalone flags to 'N'
prompt
set serveroutput on
set feedback off
declare
c pls_integer := dbms_sql.open_cursor;
upd_rows pls_integer;
vers varchar2(50);
tbl varchar2(50);
col varchar2(50);
statement varchar2(255);
begin
select substr(release_name, 1, 2)
into vers
from fnd_product_groups;
if vers >= 11 then
tbl := 'fnd_conflicts_domain';
col := 'runalone_flag';
else
tbl := 'fnd_concurrent_conflict_sets';
col := 'run_alone_flag';
end if;
statement := 'update ' || tbl || ' set ' || col || '=''N'' where ' || col || ' = ''Y''';
dbms_sql.parse(c, statement, dbms_sql.native);
upd_rows := dbms_sql.execute(c);
dbms_sql.close_cursor(c);
dbms_output.put_line('Updated ' || upd_rows || ' rows of ' || col || ' in ' || tbl || ' to ''N''');
end;
/
prompt
prompt ------------------------------------------------------------------------
prompt Updates complete.
prompt Type commit now to commit these updates, or rollback to cancel.
prompt ------------------------------------------------------------------------
prompt
set feedback on
REM <= Last REM statment -----------------------------------------------------