Saturday, November 13, 2010

Process Scheduler Server Agent PSUNX is below the Log Space Threshold The Log/Output Directory /xyz/psft/pt/8.50/appserv/prcs/xyz/log_output for the Process Scheduler Server Agent PSUNX in server xyz.com for database xyz has 8 MB of free space. This is below the disk threshold value of 10 MB in the Log Space Threshold found in the Process Scheduler Configuration file. The system is suspending the server agent until more disk becomes available. Until then, no queued process requests will be processed in this Process Scheduler Server Agent.

This happened because, one of the user modified the Application engine Process definition, override options as follows and ran the process using process scheduler.

-debug Y –trace 3

This resulted in a trace file that keeps growing very fast and consumes the entire available disk space on the appserver.

If you have implemented password controls that locks the users account after x no of invalid attempts and the user has a crystal report scheduled, it makes crystal report process to remain in initiated status forever. It also generates an ever increasing log file, which has a potential of consuming entire available disk space and disrupting the other batch processes.

It appears that Crystal Report repeatedly calls the database sql statements and never comes out of it.

Steps to reproduce the issue. 1. Create a testid testps and assign roles PeopleTools and PeopleSoft User. 2. Schedule XRFWIN Crystal Report to run within next 5 minutes. 3. Update the testps user profile and lock the account. 4. Go back to process monitor and observe that process remains in initiated status. Also if you go back to server and check the log_output folder for xrfwin you will see that log file and trace file size keeps on increasing.

Only workaround is to Run a SQL to detect this situation and cancel the process. Unlocking the user account also fixes the issue. The fix is targeted in next Tools release 8.51. We observed this behavior in only PT 8.50 and Crystal Report 2008 SP1. We are currently using patch 8.50.10.

After the recent DST time change in USA on November 7th 2010, Scheduled processes are running one hour behind in our case. Our base time zone is EST. This happens due to a bug in the current Tools version less than 8.50.13. This issue is fixed in 8.50.13 and 8.51.

Workaround

To resolve this issue, restart all your batch server domains in DEV/QA/PROD. Both NT and unix server domains are affected by this issue.

For more detail see : E-PRCS: 8.50 After DST change processes run one hour ahead/behind scheduled time (Doc ID 1265111.1)

References NOTE:1163113.1 - E-PRCS Processes Are in "Queued" Status when Process Scheduler Resides in a Different Time Zone

Root Cause

This happens due to inability of process scheduler to detect change in sessiontimezone after the DST change and it’s use of the following SQL in 8.50

select ..... from PSPRCSQUE........where RUNDTTM <= SYSTIMESTAMP

By understanding the Oracle DBMS concept of SYSTIMESTAMP, the session time zone also takes effect when a TIMESTAMP value is converted to the TIMESTAMP WITH TIME ZONE or TIMESTAMP WITH LOCAL TIME ZONE datatype.

Therefore, for this conversion to take place : RUNDTTM <= SYSTIMESTAMP, the user's session timezone is returned and is not the same timezone as the database server.

Note: This only impacts those customers using Oracle and PT 8.50.12 or less. It does not impact non Oracle or lower tools release customers.