Category Archives: Exadata

One challenge is with Exadata cells in a lab environment is that they are secure! This means that it has long lock out times in the event of an incorrect login and tough lock settings. You can manually change these.. but every time you update your cell there is a chance they will be reset.

As Oracle

Stop any databases running from the ORACLE_HOME where you want to enable DNFS.
Ensure you can remotely authenticate as sysdba, creating a password file using orapwd if required.
Relink for dnfs support

I was a little uncertain about the oradnfstab entries as most examples relate to a ZFS-BA which has many IB connections and 2 active heads, whereas the 7320 in this case was set in Active/Passive. I created $ORACLE_HOME/dbs/oradnfstab with the following entries.

A really useful command is ‘ibdiagnet’ available on Exadata, Exalogic and Sparc SuperCluster. It has several command line options, here I am asking for the simplest test, with 100 packets being used for each link.

This warning looks pretty scary, however it is just saying that the multicast group was created as 4 x SDR (10Gb) rtather than QDR speed (40Gb) even though all the nodes are QDR. Open SM defaults to a 10Gb group rate for multicast groups.

You can also check this out (look for amber lights!) through the managment bui on the switch. You can also use the BUI work out which physical port on the switch matches this Lid/Port combo (13A in this case)

I reseated the cable in port 13A, and cleared the error counter
[root@sscasw-ib2 opensm]# ibclearcounters

Now I’m monitoring the status of the port – if the error count increases again I will replace the cable.

After setting the listener_networks parameter I checked whether the services had registered correctly with the scan listener. Expected behaviour is to see all instances registered with all 3 scan listeners

– Set your environment to the GRID_HOME

- Check which nodes are running the scan listener as you can only interrogate the listener from that node
oracle@ssca01:~$ srvctl status scan
SCAN VIP scan1 is enabled
SCAN VIP scan1 is running on node ssca03
SCAN VIP scan2 is enabled
SCAN VIP scan2 is running on node ssca04
SCAN VIP scan3 is enabled
SCAN VIP scan3 is running on node ssca01

So on ssca01, I can check the status of LISTENER_SCAN1 ..
And it had no services registered. Strange. Checked all of my listeners and only LISTENER_SCAN3 had any services registered

So, it looked like pam had locked out the oracle user due to multiple failed login attempts. At this point on a production system you should start to investigate who has been trying to access your system, however,we knew what had caused the problem.

An Exadata X2-2 storage cell contains 12 disks. While you should not need to look at them via iostat to measure performance, as metrics are available via dbconsole, enterprise manager and asm disk views, it can sometimes be useful to be able to take a quick look at the system.

Note the lines highlighted in red:- on the first 2 disks the celldisks sit in the partition /dev/sd*3 to allow space for the software and operating system. On the remaining disks the cells have access to the whole disks.

This means that in iostat you cannot separate out the load for different ASM diskgroups whose griddisks share the same celldisk. You can see which celldisks are used by your griddisk by using

[root@ed2hcell01 ~]#cellcli -e list griddisk detail

Again the output to this command can be long, and it is better piped to a file.