The solution in my case: Check each Management Packs’ LanguagePack. I would get above error when using certain special characters in the Name or Description element (e.g. single quotes). After removing those from the LanguagePack, the MP seals just nicely.

I have no idea how often I have used PowerShell scripts to build or extend SCOM Management Packs. Almost daily it must have been. Occasionally I stumbled over PowerShell version issues because the SCOM agent had its own logic of choosing a PowerShell version. To circumvent these, I basically limited myself to write (and test!!!) my scripts on PowerShell 2.0 and .NET 2.0 environments.

The current behavior for agents is to choose a CLR version based on the operating system version. For Windows Server 2012 and newer, the .NET Framework 4.0 is loaded. For operating systems older than Windows Server 2012, the .NET Framework 2.0 family is loaded. On management servers, the .NET Framework 2.0 family is loaded. This essentially maps the .NET Framework version used to the version available out-of-box on the server. The problem with the current behavior is that even if the Management Pack author knows that .NET Framework 4.0 is present on the system, it cannot be used.

In the new behavior, the agent loads the .NET Framework 4.0 if it is available else it falls back to the .NET Framework 2.0.

What this do not easily reveal: Since .NET Framework 4.0 is now chosen, the SCOM agent and SCOM management server can use PowerShell 4.0 or 5.0. That is fantastic news. It means the SCOM agent (Microsoft Monitoring Agent respectively HealthService) is now making use of the most recent PowerShell version installed on an OS.

SCOM PowerShell module behaviour

When using the Windows Library’s PowerShell modules (e.g. Microsoft.Windows.PowerShellProbe), the following logic applies:

SCOM 2012 R2

Machine Type

<= 7.1.10218.0 (UR 6)

>= 7.1.10229.0 (UR 7)

Management Server / Gateway

PS 3.0 if installed; otherwise PS 2.0

PS 4.0 or 5.0 (if .NET 4.0 and PS >= 4.0 are installed)

PS 2.0 (if .NET 4.0 or PS >= 4.0 are not installed)

Agent

PS 2.0 / .NET 2.0

Note on legacy Windows versions: PowerShell 2.0 is the minimum requirement to use the agent’s PowerShell modules. And PowerShell is always installed or upgraded as part of the “Windows Management Framework”.

To verify that newer PowerShell versions are used after upgrading to SOM 2012 R2 UR 7, I wrote a simple MP with agent tasks. The following table shows the sample output of a Windows 2008 R2 system, having WMF 5.0 preview installed:

before UR7

with UR 7

Operating
System: Microsoft Windows Server 2008 R2 Enterprise

PowerShell
Host: OpsMgr PowerShell Host

Powershell Version: 2.0

Common
Language Runtime .NET: 2.0.50727.5485

Operating
System: Microsoft Windows Server 2008 R2 Enterprise

PowerShell
Host: OpsMgr PowerShell Host

Powershell Version: 5.0.10105.0

Common
Language Runtime .NET: 4.0.30319.34209

So at last I can start writing PowerShell workflows for SCOM in 4.0 or 5.0 manner – provided I can guarantee that every agent has UR7 or later installed!

Using PowerShell.exe as an alternative (System.CommandExecuterProbe)

When you look closely at the output of the table above, you can see that the PowerShell host is “OpsMgr PowerShell Host”. SCOM PowerShell modules are not running on Windows’ native PowerShell host (ConsoleHost). Instead a SCOM internal host is used, that is optimized for agent use. Occasionally this can lead to issues with very advanced PowerShell workflows.

If this happens, and only then (!) the following approach may be used. Instead of using the native SCOM PowerShell modules, the System.CommandExecuterProbe can call PowerShell.exe. However; you have to make sure, the execution policy for our workflow is set no stricter than “RemoteSigned”. Otherwise the script will not be run.

Examples for this are found in SharePoint MPs or were used back in SCOM 2007 days to run PowerShell scripts. Or have a look at my own sample MP linked at the end of this post.

In the last 10 months I have added several minor features to the certificate verification management pack for SCOM 2012. Some of them rose from needs of clients but most were suggested by people using the MP. Many thanks for everyone providing feedback and especially for testing the pre-release builds.

And a few extra words on some of them

While some changes are pretty much self explanatory, I am going to give you some background information on others below. Plus two little PowerShell bits to make everyone’s lives a bit easier.

Added tasks

To make checking on the validity of certificates easier for SCOM operators, I have added agent tasks that will

list certificate properties (and check validity)

archive a certificate

disable respectively re-enable monitoring for a single certificate (append friendly name tag)

attempt immediate rediscovery

Archiving a certificate may obviously have a disastrous impact on payloads working with a certificate. However; it may be undone locally on an agent computer using some PowerShell code as in the following example:

Since there’s no undo option for deleted certificates, the MP does not contain a task to delete certificates.

Disabling monitoring for a single certificate is achieved by appending the tag “_DoNotMonitor” to the end of a certificate’s friendly name. Such certificates are then becoming members of a group and monitoring will stop by means of predefined overrides. Instead of using the SCOM task, a system administrator might simply use the certmgr snap-in to alter the friendly name.

Tag friendly name to disable monitoring

The rediscovery task will only be available, if the additional MP ReDiscoveryTasks has been imported. It will run a management server side PowerShell script, triggering the SCOM task “Trigger On Demand Discovery Task“. Overrides will be accounted for. Such, changes to an agent’s certificate store will be reflected much more quickly in SCOM. The progress can easily be verified by checking the “Task Status” view. Note that the ReDiscoveryTasks MP will also attempt to run an OnDemand discovery after Archive and Enable/Disable tasks have been performed.

Added Recoveries

Two disabled recoveries have been added to the certificate monitors:

Archive Certificate

Disable Monitoring

They do behave exactly the same as the related tasks mentioned above. They will show their effect on the SCOM repository within minutes if the optional ReDiscoveryTasks MP is imported. Otherwise monitoring will not stop until the next discovery cycle has passed.

Added discovery of the Web Hosting store

Windows Server 2012 and 2012 R2 feature a new certificate store called “Web Hosting”. Behind the scenes it provides more load efficient certificate handling and is thought to be used e.g. with IIS in web hosting scenarios (what a surprise). When using the QuickStart override MP, those stores and all signed certificates in them will automatically be discovered.

Monitoring wise it is important to understand that this store is thought to hold hundreds of certificates. Hence thinking about a good filtering strategy for monitoring may crucial. Failing to do so could lead to way too many objects being discovered, causing performance issues on SCOM. Consider making use of the Subject, Issuer and Template filtering capabilities.

Certificate Template information

Certificates issued by Enterprise CAs can now be filtered based on their template name (and OID). Consequently, the template property has been added as an additional property to the certificate and will be listed on inventory reports.

It is important to understand that the names of certificate templates are specific to an AD forest. They cannot be resolved if such a certificates is located on a computer outside of the forest. In that case, the MP will display the template OID value only. To check what the template property of a certificate is, run the following PowerShell commands on an agent computer:

Updated Alert Description

The alert description of monitors has been overhauled to be much more verbose. Specifically it is now showing both the subject and the issuer property. It also contains information about the RunAs account used for monitoring and gives details on the validity of the certificate chain.

Such it has become much easier to pinpoint issues when not the end certificate itself but a chain certificate has become invalid; possibly just in the context of the RunAs account of the agent.

An extra hint regarding the latter in case the agent is running under the local system account: Occasionally it can be observed that a certificate chain is reported invalid by the MP but is valid if a certificate is checked with an interactive user account. One possible reason for this can be the SYSTEM account not being able to automatically update CTLs (e.g. due to internet access restriction), while an interactive account can. If the issue cannot be resolved, consider using a tool like SysInternals PSExec.exe to launch the CertMgr.msc in the SYSTEM context to investigate the cause.

And now for the task scheduler: Today’s update has been on my to-do list for a very long time. Same story as with the certificate MP: I have completely re-written the scheduled task MP based on PowerShell, aiming to fully support SCOM 2012 and Windows 2012.

MP change history

Timing is mostly event-driven: significantly reduce the number of script calls required on the average agent system.

Verbose filters to adjust task discovery.

Disabled root discovery out-of-the-box (quick start MP included).

Disabled advanced monitors and rules out-of-the-box.

Configuration

Please read the included MP guide and the release notes carefully. Especially if you are upgrading from the previous (SCOM 2007) MP version.

Please make sure that PowerShell >=2.0 is installed on all agents that require task scheduler monitoring.

Previous versions of this MP had the root discoveries enabled. As this was changed to disabled, you will need to configure re-enable overrides to the required discoveries plus advanced monitors if you were relying on them.

The 1st update to the rewritten certificate management pack is ready. The update to 1.2.1.3 is mostly about more powerful filtering options for the certificate discoveries. It is now possible to use regular expressions to:

Include / Exclude based on “Subject”

Include / Exclude based on “Issuer”

Exclude based on “Enhanced Key Usage” OIDs

Note that the filters will have to be based on the exact string output of the certificate objects as presented in PowerShell. Hence check those before attempting to write RegEx filters using:

ls cert:\LocalMachine\My | fl Subject, Issuer

All characters (including blanks) are being taken into account. The discovery filters are using .NET RegEx expression syntax. Please test your expressions using a suitabe tool before using them for your overrides (I am often using Regex Hero but there are plenty of other options out there).

Once store discovery is enabled, the default filter settings of the MP will discover any certificates with the exception of self-signed and MS NAP ones. Refer to the MP guide and the release notes if you plan to make use of the advanced filter options. And remember to override the store discovery, not the certificate one.

Many years have passed since I first published the certificate MP back in summer 2009. Almost 5 years(!) later this management pack still fills a gap by keeping an eye on PKI certificates installed locally in servers’ certificate stores. Certainly about time for an update.

MP change history

SCOM 2012 / 2012 R2 support only (the legacy MP 1.0.1.20 is still available for use on SCOM 2007).

main monitoring script now uses PowerShell instead of VB Script, making it compatible with any system locale and easier to maintain.

new, advanced certificate verification flag overrides

dashboard view

Some extra words on the effort

The main aim with this update is to make the MP’s code easier to maintain. Hence I first recreated the entire MP as a Visual Studio project with the Authoring Extensions. This involves taking apart the MP’s elements, adding each one as a separate item to a VS project structure. Next I started writing a new discovery and monitoring script based on PowerShell. This script does most of the work by essentially enumerating certificates and certificate revocation lists in local certificate stores. Due to limitations in PowerShell regarding CRLs and alternate certificate stores, this script got rather complex. No chance of getting away with something easy and straight forward as ‘ls cert:\LocalMachine’. With the first CRLs getting discovered, tests, more tests, some extra testing plus updating the documentation were left.

While I did not clock the hours, the update kept me busy in much of my spare and commuting time during the last 4 months. And I must mention everybody helping with code samples, advise, by testing and reviewing. Pete, Vadim, Marc, Joel, Bob, Dan, Marnix, Stan, Tao and Dirk – this wouldn’t be here today without your help!

Today I was able to release an update to the SQL Server DB Mirroring Management Pack.

The update doesn’t bring any new features but fixes a potential issue on larger management groups which would lead to discoveries failing. In order to work around this I have altered the Powershell discovery scripts such that they do no longer make use of any SCOM console cmdlets. Instead direct SDK .NET calls are being used. I recommend to update existing installations specifically if you are using management group connectors, the Exchange 2010 correlation engine or other custom MPs which make use of SDK workflows.