HOW DOES 11G R2 CLUSTERWARE START ASM WHEN ASM SPFILE IS STORED ON ASM ITSELF?

Beginning with the version 11g Release 2, the ASM spfile is stored automatically in the first disk group created during Grid Infrastructure installation.

Since voting disk/OCR are stored on ASM, ASM needs to be started on the node. To startup ASM, its SPfile is needed. But SPFILE is again located on ASM diskgroup only. How does clusterware resolve this issue?

- When a node of an Oracle Clusterware cluster restarts, OHASD is started by platform-specific means. OHASD accesses OLR (Oracle Local Registry) stored on the local file system to get the data needed to complete OHASD initialization

- OHASD brings up GPNPD and CSSD. CSSD accesses the GPNP Profile stored on the local file system which contains the following vital bootstrap data;
a. ASM_DISKSTRING parameter (if specified) to locate the disks on which ASM disks are configured.b. ASM SPFILE location : Name of the diskgroup containing ASM spfile

c. Location of Voting Files : ASM

– CSSD scans the headers of all ASM disks ( as indicated in ASM_DISKSTRING in GPnP profile) to identify the disk containing the voting file. Using the pointers in ASM disk headers, the Voting Files locations on ASM Disks are accessed by CSSD and CSSD is able to complete initialization and start or join an existing cluster.

– To read the ASM spfile during the ASM instance startup, it is not necessary to open the disk group. All information necessary to access the data is stored in the device’s header. OHASD reads the header of the ASM disk containing ASM SPfile (as read from GPnP profile) and using the pointers in disk header, contents of ASM spfile are read. Thereafter, ASM instance is started.

– With an ASM instance operating and its Diskgroups mounted, access to Clusterware’s OCR is available to CRSD.

– OHASD starts CRSD with access to the OCR in an ASM Diskgroup.

– Clusterware completes initialization and brings up other services under its control.

Demonstration :

In my environment, the ASM disk group DATA created with EXTERNAL redundancy is used exclusively for ASM spfile, voting and OCR files:

- Let us read gpnp profile to find out the location of ASM SPfile

[grid@host01 peer]$ cd /u01/app/11.2.0/grid/gpnp/host01/profiles/peer

gpnptool getpval -asm_spf

Warning: some command line parameters were defaulted. Resulting command line:

+ASM1.__oracle_base=’/u01/app/grid’#ORACLE_BASE set from in memory value

+ASM2.__oracle_base=’/u01/app/grid’#ORACLE_BASE set from in memory value

+ASM3.__oracle_base=’/u01/app/grid’#ORACLE_BASE set from in memory value

+ASM3.asm_diskgroups=’FRA’#Manual Mount

+ASM2.asm_diskgroups=’FRA’#Manual Mount

+ASM1.asm_diskgroups=’FRA’#Manual Mount

*.asm_power_limit=1

*.diagnostic_dest=’/u01/app/grid’

*.instance_type=’asm’

*.large_pool_size=12M

*.remote_login_passwordfile=’EXCLUSIVE’

a:O/

The same technique is used to access the Clusterware voting files which are also stored in an ASM disk group. In this case, Clusterware does not need a running ASM instance to access the cluster voting files:

Let’s check the location of voting disk :

[grid@host01 peer]$ crsctl query css votedisk

## STATE File Universal Id File Name Disk group

– —– —————– ——— ———

1. ONLINE 243ec3b2a3cf4fbbbfed6f20a1ef4319 (ORCL:ASMDISK01) [DATA]

Located 1 voting disk(s).

– Since above query shows that voting disk is stored on ASMDISK01 which maps to /dev/sdb1,

we will scan the header of /dev/sdb1

[root@host01 ~]# kfed read /dev/sdb1 | grep vf

kfdhdb.vfstart: 96 ; 0x0ec: 0x00000060

kfdhdb.vfend: 128 ; 0x0f0: 0x00000080

Here we can see that voting disk resides on /dev/sdb1 .

Once the voting disk is accessible and ASM is started using the SPfile read above, rest of the resources on the node can be started after reading the Oracle Local Registry (OLR) on the node.

I get the foll.:$ gpnptool getpval -asm_spfError: Can’t open profile ‘profile.xml’ for read: file not found$

Can you pls help me understand as to what could be the reason? The above is when entire RAC stack is up on the node. Believe the above should return correct value even when RAC is down on the node – isnt it?