Wednesday, November 2, 2011

We needed a way to have our existing monitoring system alert us if a project was running out of space. There's not a single CLI command that will show all projects, but this bit of ECMAscript 3will output an easily parsed table:

Monday, March 7, 2011

The EMC Celerra datamovers have the ability to log statistics about top talkers, which can be useful for tracking down problems. We run server_stats with these options to get top talker stats:

/nas/bin/server_stats server_2 -top nfs -i 5 -c 60

One thing worth noting is there's a column labeled "NFS Suspicious Ops". There's no documentation on this column, and it took EMC some time to dig up the answer. Here it is:

SUSPICIOUS EVENTS:
One of the TopTalker output columns lists Suspicious Ops/second.
"Suspicious" events are any of the following, which are typical of the patterns seen when viruses or other badly behaved software/users are attacking a system:

Saturday, January 1, 2011

We use Sun/Oracle's Fishworks (7000) ZFS Storage Appliances to store our Oracle archive logs and to replicate them to our DR datacenter.

We generate more than 2TB of archive logs per day, and ZFS' compression helps knock that down to a somewhat more manageable 500GB a day. Initially we were using project-based replication which was easy to configure, but unfortunately there was not enough parallelism to keep up with our change rate.

Sun suggested setting up replications for each share (we have 16 shares per database cluster) to improve throughput. It's worked well, but the user interface didn't provide an overview of replication status.

Fortunately, the CLI can be scripted using JavaScript, so it was easy to loop over the projects and shares and extract the replication status.

To run the script, just ssh to the appliance and redirect stdin from the script: