How to Apply a Standby-First PSU Patch in a 2 node RAC environment

Oracle provides since it’s 11.2.0.1 version a way to apply certain patches, to our standby environment first, without compromising the primary database, to allow to let it burn in the Standby for the time you deem appropriate and then apply the patch binaries to the primary BD ORACLE_HOME and these will cascade into our Standby DB.

The first thing you have to make sure before applying a patch like this, is that it’s certified in the MOS Patch note as “Standby-First”.

For this, I have the patch binaries in /u01/app/oracle/patches/11.2.0.3/PSUOct2013, and the databases that I will apply the patches are the primary DB TEST with oracleenespanol1/oracleenespanol2 nodes, and Standby TESTSTBY with oracleenespanol3 and oracleenespanol4 nodes. The PSU patch that I will apply is 17272731 which is 11.2.0.3.8.

Let’s start this long post in the Standby environment, so please be patient. The first thing I recommend is that you take a backup of your binaries, in a given case that there is an error, we can return to these without much problems. This is run as root. Similarly I recommend that before you start, to have the necessary backups of your database.

To continue, you must create as the oracle user, an OCM response file, this does not mean that we are going to install it, but this is necessary for a PSU. And remember that our directory where we have the patch binaries are shared, so we only have do it once.

I like to ensure that all my services are up or even down on my RAC environment before making any changes, so that I can crosscheck when I finish. I use this script called crs_status.sh to verify and I run with my user grid, all you have to change the value of CRS_HOME, hopefully this will also serve you.

The only thing I would do is to use again the crs_status.sh script to check my RAC services are up/down. For this type of application, you have to know that there is no extra step in the Standby, as the catbundle will be applied in the primary BD not in Standby. As you could see, it is a long process, because we have just finished our standby environment, we will now proceed to our primary environment. Here is where you will wait the time that you deem necessary to see if there are any errors that can impact your primary database when you apply it there or make you rollback the patch.

Now we will move to the primary environment.

For the primary, I’m not going to go over the same steps that had to be done standby environments (oracleenespanol3/oracleenespanol4) that you will also have to make in the primary environment (oracleenespanol1/oracleenespanol2) , but I will do a summary of what has to be done in the primary nodes

Take a backup of your GI_HOME/ORACLE_HOME and inventory

Verify you have a valid backup of your Primary DB

Run against your binaries in your primary environment opatch prereq CheckConflictAgainstOHWithDetail as we did with the standby environment.

Make sure that you have your ocm.rsp file created

Verify which services are up/down in your RAC environment

Once you’ve done the steps above, this is where the process of implementing the patch changes.

We verify again as we did in the standby environment with the opatch lsinventory that the patch 11.2.0.3.8 is applied properly in our binaries and with the script crs_status.sh we crosscheck which services are up/down in our primary environment.

What we have to do next is run the catbundle.sql as the oracle user from $ORACLE_HOME/rdbms/admin in our primary database, this has to run on one node only, not both.

At this point, we’re done with the primary database and the primary nodes. In oracleenespanol3/oracleenespanol4 servers (Standby), let’s put TESTSTBY recovery more. In my case I’m using Active Dataguard, just be careful with that, because if you are not using it, is an extra licensable option for your Standby DB.

And now there is nothing more to do, but to verify that all is well. Let’s check that the last ARCHIVED REDO LOG is applied, which was 82197, and the registry history of your Standby database and you’ll see that you’ll have the version 11.2.0.3.8.

This is very useful for maintaining high availability in your environment while allowing you to verify that the patch that you apply does not harm your primary database. LEt me know if you had a use-case for this type of patching and if you think this is useful to you.

P.S.- Happy 2013 Holidays :)

Share this article

6 Responses to “How to Apply a Standby-First PSU Patch in a 2 node RAC environment”

thank you for the kindness to share this info. Just a quick question. In the standby database patching, don’t you need to shut down the redo log apply before applying the patching?
If so, when should you turn the redo log apply back on? before patching the primary databae or after patching the primary database? thank you so much.

On the primary, it all depends if its RAC rolling or not. When patching the primary, we need to verify that is in Read Only Mode if the patch Readme advises Patches that are listed as RAC Rolling Installable in the patch README can be applied on the primary with the standby performing recovery in read only mode.

However, patches that are not RAC Rolling Installable must stop read only recovery on the standby,
bring the standby database to the mount state, and restart recovery after applying the patch to the primary database.