Posts in this blog are provided "AS IS" with no warranties, and confers no rights. Use of included script samples are subject to the terms specified in the Terms of UseAre you interested in having a dedicated engineer that will be your Mic

How to monitor SQL Agent jobs using the SQL Management Pack and OpsMgr

When you use Operations Manager to monitor your SQL servers using the SQL Server Management Pack, there are some options that you will need to think about up front. One of those areas is the SQL Agent, and SQL Agent jobs.

Out of the box, we don’t discover or monitor individual SQL agent jobs by default. What we do is discover the SQL Agent object:

By default – all we are monitoring with regard to the SQL Agent, is the SQL Agent service availability:

Keep in mind – on SQL clusters, we don’t monitor manual services by default. The assumption is that if the service is clustered, and is down, the cluster MP will alert that a clustered resource is partially offline. This keeps from having duplicate alerts about the same service availability issue. However, if you WANT the SQL MP to alert when a clustered SQL Agent service is down – you will need to override this monitor:

There are also several rules that target the “SQL Agent” object, that primarily look in the Application event log for issues/errors related to the SQL Agent:

Notice that the “An SQL job failed to complete successfully” is disabled out of the box. This was done to reduce out of the box noise…. as many customers have terribly monitored/maintained SQL environments and have so many jobs failing that this was almost un-actionable. If you want to be alerted to ANY sql agent job failure of any kind – then you might consider enabling this rule via override. I would recommend this if you aren't going to discover and monitor individual SQL agent jobs (covered later in this article)

Lastly – at the SQL Agent level – there is a monitor for alerting when SQL Agent jobs run too long. To reduce noise, this is also disabled out of the box.

If you want to monitor ALL jobs run under a specific SQL Agent with the same thresholds, you should consider enabling and adjusting this monitor. By default – the warning threshold is 60 minutes, and critical is 120 minutes. If you want to be able to control and override individual agent jobs that need longer run-times than others, leave this disabled and configure the SQL MP to discover and monitor individual SQL agent jobs (covered later in this article)

Ok – that covers the out of the box defaults, and SQL Agent only monitoring. Now, what if I need to go deeper, and have the ability to override settings for individual SQL Agent jobs? You will notice that in the view for SQL Agent Job State – there are no objects:

This is by design, because we don’t discover SQL Agent jobs individually out of the box. Perhaps I have some SQL Agent jobs that I don’t want to monitor at all, and others have especially long run-times that need unique thresholds (like SQL backups or maintenance on VERY large DB’s?) In this case – we will want to enable the discovery to discover SQL Agent jobs as discovered objects.

These are disabled by default. Create an override for each, “For all objects of class" and set Enabled = true. Save these overrides to your custom “SQL Overrides” management pack.

This discovery runs every 4 hours by default, so within 4 hours you should see your SQL Agent job state view populated with your individual SQL Agent jobs. (**Hint – to speed this process up, bounce the System Center Management service on your agents – as this will force a discovery to run on service startup)

Now – lets talk about what we monitor by default for discovered SQL Agent jobs. There are a total of two monitors. Last Run Status, and Job Duration:

“Last Run Status” monitor checks every 10 minutes, and will remain Yellow (Warning state) for any job whose last run was not a success. This gives you a nice “real time” view of the unhealthy jobs that need some attention.

“Job Duration” monitor runs every 10 minutes, and looks for jobs that have exceeded a run-time threshold. The default is 60 minutes (warning) and 120 minutes (critical). This monitor also does not generate alerts by default. If you want this monitor to generate alerts you must override that setting.

In the following example – I am creating an override for my SQL agent job that performs full backups, setting the thresholds higher and enabling alerting….. and saving it to my SQL Overrides management pack:

As you can see, this gives you SQL Agent job by job granularity.

You could easily also create groups of SQL agent jobs based on group name…. if you wanted to treat all jobs with the same or similar names the same from an override perspective. This will additionally ease the burden of maintaining overrides using a dynamic group based on criteria of the job:

I was wondering; this seems trivial however does not seem to be working for me.

I'm trying to setup the job monitor (not rule), which I've set appropriately as you've stated above but it does not work when using a dynamic group containing agent job objects. It does however work when I enable it on an individual job basis; same as those listed in the dynamic group.

I have many jobs across the enterprise and those populated with a specific category are the ones we want to alert which is what the dynamic group is for.

We need to generate alert only for the enabled job when it get failed and not for the disabled jobs which last run status is in failed state. I know there is a rule base Monitor that target the “SQL Agent” object, that primarily look in the Application
event log for issues/errors related to the SQL Agent but for this we need to enable notification in each SQL job which is not possible in our huge SQL environment. Thanks in Advance!!!

Hi Kevin,
Could you please help me out in genrating sql long runing job with an job name i have configured it to generate long running job successfully.
Am also getting all management server resource poll unavailable error. Requst ;you to kindly help me to resolve this issue