SCOM Monitor reset logic

ResetButton

 

Ever want to reset SCOM monitors, and wish it was just a simple Reset Button for unhealthy monitors?

 

I’ve been using Scott Murr’s TechNet gallery loop to maintain my alerts, and ensure monitors are healthy for all my management packs.

 

The blurb I put in my DS/WA scripts to reset SCOM monitors.  I build on Andrew’s methods I didn’t realize (just think much uglier code!)

Cleaner PowerShell to help reset monitors and rules

 

 

My PowerShell variables to reset SCOM monitors, includes my Addendum and the core – DNS example provided below (thank you Andrew!)

 

## Grab the MP, get the Monitors and Rules from the MP, then grab all alerts found inside the Monitors/Rules

$SCOMCoreMP = Get-SCOMManagementPack -DisplayName “Microsoft Windows Server 2016 and 1709+ DNS Monitoring”
$SCOMAddendumMP = Get-SCOMManagementPack -DisplayName “Microsoft Windows Server 2016 DNS Monitoring Addendum”

$SCOMCoreRules = $SCOMCoreMP.GetRules()
$SCOMCoreMonitors = $SCOMCoreMP.GetMonitors()
$SCOMAddendumRules = $SCOMAddendumMP.GetRules()
$SCOMAddendumMonitors = $SCOMAddendumMP.GetMonitors()

$SCOMCoreReportAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMCoreRules.DisplayName) -or ($_.Name -in $SCOMCoreMonitors.DisplayName) }
$SCOMCoreReportAlerts.Count
$SCOMAddendumReportAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMAddendumRules.DisplayName) -or ($_.Name -in $SCOMAddendumMonitors.DisplayName) }
$SCOMAddendumReportAlerts.Count

$SCOMOpenReportAlerts = $SCOMAddendumReportAlerts | ? { ( $_.ResolutionState -ne “255” ) }
$SCOMOpenReportAlerts.Count
$SCOMOpenAddendumReportAlerts = $SCOMAddendumReportAlerts | ? { ( $_.ResolutionState -ne “255” ) }
$SCOMOpenAddendumReportAlerts.Count

$SCOMCoreRuleAlerts = Get-SCOMAlert | ? { ( $_.Name -in $SCOMCoreMonitors.DisplayName) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMCoreRuleAlerts.Count
$SCOMAddendumRuleAlerts = Get-SCOMAlert | ? { ( $_.Name -in $SCOMAddendumRules.DisplayName) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMAddendumRuleAlerts.Count

$SCOMCoreMonitorAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMCoreMonitors.DisplayName ) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMCoreMonitorAlerts.Count
$SCOMAddendumMonitorAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMAddendumMonitors.DisplayName ) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMAddendumMonitorAlerts.Count

$AutoClosed = $SCOMCoreMonitorAlerts.Count + $SCOMCoreRuleAlerts.Count + $SCOMAddendumMonitorAlerts.Count + $SCOMAddendumRuleAlerts.Count
$Test = $SCOMCoreReportAlerts.Count + $SCOMAddendumReportAlerts.Count
$OpenAlerts = $SCOMOpenReportAlerts.Count + $SCOMOpenAddendumReportAlerts.Count
$ResetMonitors = $SCOMCoreMonitors + $SCOMAddendumMonitors
$MonitorAlerts = $SCOMCoreMonitorAlerts.Count + $SCOMAddendumMonitorAlerts.Count

 

 

#
# If Cleanup needed, array of report monitors

# Reset Monitors Script
# Put ps1 in mgmtpacks folder
# https://sc.scomurr.com/scom-2012-monitor-reset-cleaning-up-the-environment/
# Download
# https://gallery.technet.microsoft.com/SCOM-2012-Batch-reset-63a17534

#Alternate
#https://gallery.technet.microsoft.com/scriptcenter/Auto-reset-script-for-d8b775ca

if ( $MonitorAlerts -gt 0 )
{
foreach ( $MonitorDisplayName in $ResetMonitors.DisplayName )
{
$Monitors = @( Get-SCOMMonitor -displayname $MonitorDisplayName )

# Set up monitor objects to reset
foreach ($Monitor in $Monitors)
{
$MonitorClass = Get-SCOMClass -Id $Monitor.Target.Id
$ActiveMonitors = Get-SCOMClassInstance -Class $MonitorClass | ? { ($_.healthstate -ne ‘Success’) -AND ( $_.healthstate -ne ‘Uninitialized’) -AND ($_.IsAvailable -eq $true) }
write-host “Found” + $ActiveMonitors.Count + “active monitors.”
if ( $ActiveMonitors -ne $null)
{
foreach ($ActiveMonitor in $ActiveMonitors)
{
write-host ” Resetting Health State on ‘” +$ActiveMonitor.FullName + “‘”
$ActiveMonitor.ResetMonitoringState($Monitor.ID)
}
}
}
}
}

Cleaner PowerShell to help reset monitors and rules

Like Meguiar's cleaner wax to your car's finish, this post will help utilize cleaner PowerShell to help reset monitors and rules
Like Meguiar’s cleaner wax to your car’s finish, this post will help utilize cleaner PowerShell to help reset monitors and rules

 

Cleaner PowerShell supplied by Andrew Bradley that’s helped simplify the PowerShell code included resetting/closing monitors and rules via a method call.  Hard to believe I’ve been quiet on the blog for the past year, as I’ve been working on SCOM management pack addendums.  The ‘cleaner PowerShell’ is being integrated into the various addendums.

 

have been helpful with many customers, by building out better ways to monitor, clean up alerts, and create daily reports.  The Addendum packs add report key insights for many 1P (1st party) Microsoft authored management packs.

Methods

 

 

Cleaner PowerShell to help reset monitors and rules

## Grab the MP, get the Monitors and Rules from the MP, then grab all alerts found inside the Monitors/Rules

$SCOMCoreMP = Get-SCOMManagementPack -DisplayName “System Center Core Monitoring”

$SCOMCoreRules = $SCOMCoreMP.GetRules()

$SCOMCoreMonitors = $SCOMCoreMP.GetMonitors()

$SCOMCoreReportAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMCoreRules.DisplayName) -or ($_.Name -in $SCOMCoreMonitors.DisplayName) }

Example for DNS management pack

## Grab the MP, get the Monitors and Rules from the MP, then grab all alerts found inside the Monitors/Rules
$SCOMCoreMP = Get-SCOMManagementPack -DisplayName “Microsoft Windows Server 2016 and 1709+ DNS Monitoring”
$SCOMCoreRules = $SCOMCoreMP.GetRules()
$SCOMCoreMonitors = $SCOMCoreMP.GetMonitors()

$SCOMCoreReportAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMCoreRules.DisplayName) -or ($_.Name -in $SCOMCoreMonitors.DisplayName) }
$SCOMCoreReportAlerts.Count

$SCOMOpenReportAlerts = $SCOMCoreReportAlerts | ? { ( $_.ResolutionState -ne “255” ) }
$SCOMOpenReportAlerts.Count

# Open alerts

$SCOMCoreRuleAlerts = Get-SCOMAlert | ? { ( $_.Name -in $SCOMCoreRules.DisplayName) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMCoreRuleAlerts.Count

$SCOMCoreMonitorAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMCoreMonitors.DisplayName ) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMCoreMonitorAlerts.Count

Adding parameters to datasource/probeaction moduletypes

Adding parameters to datasource/probeaction moduletypes

 

This post is adding parameters to datasource (DS) or probeaction (PA) moduletypes.  Sorry, found this draft from last year that I never published.  🙁 I’m in the ‘missing functionality’ boat.  Some would say I’m a dreamer, a good system admin, a car guy who has different ideas than the manufacturer, or something altogether different — you decide 🙂  Hope this blog post helps monitoring experts that author more functionality than what was delivered.  Specifically adding parameters to datasource/probeaction moduletype NOT delivered in the OotB functionality?!

 

 

Adding parameters to datasource/probeaction moduletypes
First – What is needed
Second – Verify dependencies required for a workflow
Third – Build on example ‘datasource’
Fourth – Configure Monitor/Rule to use Datasource/ProbeAction

Let’s go through step by step through ‘adding parameters to datasource/probeaction moduletypes’ to customize a data source. The datasource requirements are to include/verify the following parameters” TimeOut,TimeOutInMS,MatchCount,SampleCount (match/sample count are intended for rules/monitors)

 

Pre-reqs (what’s needed for a ModuleType to function)

Working Script – PowerShell/BASH/Perl/SH/KSH
ScriptArgs required at runtime
Other Configuration, or Overrideable Parameters
Using configured parameters properly
Verify ProbeActions (PA) inside DS have relevant parameters

 

Easiest way to summarize adding a configuration parameter
Must be added to Configuration, OverrideableParameters,ModuleImplementation,
When taking an Out of the box’ OotB’ moduletype to modify, where parameter(s) MUST be used in UnitMonitorType,Rule,Monitor

Quick background for MatchCount/SampleCount:
When adding parameters to datasource/probeaction moduletypes, it’s good to know why this is part of the conversation to be added to monitoring design/implementation.

MatchCount comes in handy for repeated failures BEFORE alerting (count 5 events before alerting)
SampleCount comes in handy for counting number of failed workflows BEFORE alerting (run workflow 6 times failing before alerting)

 

Example Unix.ShellCommand.Invoke.Script DataSource
Requirement = Add MatchCount/SampleCount (or TimeOut to the PA ProbeAction)

Download

Unseal, and open Microsoft.Unix.ShellCommand.Library.xml in NotePad++, VStudio, (or your favorite XML editor)

Screenshot of default Microsoft.Unix.ShellCommand.Invoke.DataSource
TimeOut and TimeOutinMS are baked in.  We begin by adding MatchCount/SampleCount

Adding MatchCount/SampleCount for Configuration, OverrideableParameters, and Module Implementation for DS/PA
Adding MatchCount/SampleCount for Configuration, OverrideableParameters, and Module Implementation for DS/PA

 

How to add MatchCount/SampleCount syntax

Adding MatchCount/SampleCount for Configuration, OverrideableParameters, and Module Implementation for DS/PA

NOTE – sometimes you don’t find an example!

This part gets complicated – how far down the rabbit hole do you need the parameters?
Does the DS workflow only need the respective parameters?
Do you have to add to the corresponding PA’s called in the workflow?

 

Starting simple, add to DS

Add MatchCount/SampleCount to DS Configuration
<xsd:element name=”MatchCount” type=”xsd:unsignedInt” maxOccurs=”1″ minOccurs=”0″ xmlns:xsd=”http://www.w3.org/2001/XMLSchema” />
<xsd:element name=”SampleCount” type=”xsd:unsignedInt” maxOccurs=”1″ minOccurs=”0″ xmlns:xsd=”http://www.w3.org/2001/XMLSchema” />

Add MatchCount/SampleCount to OverrideableParameters (if you want capability to override)
<OverrideableParameter ID=”MatchCount” Selector=”$Config/MatchCount$” ParameterType=”int” />
<OverrideableParameter ID=”SampleCount” Selector=”$Config/SampleCount$” ParameterType=”int” />

Add MatchCount/SampleCount to DS MemberModule
<MatchCount>$Config/MatchCount$</MatchCount>
<SampleCount>$Config/SampleCount$</SampleCount>

Add MatchCount/SampleCount to PA Configuration
<xsd:element name=”MatchCount” type=”xsd:unsignedInt” maxOccurs=”1″ minOccurs=”0″ xmlns:xsd=”http://www.w3.org/2001/XMLSchema” />
<xsd:element name=”SampleCount” type=”xsd:unsignedInt” maxOccurs=”1″ minOccurs=”0″ xmlns:xsd=”http://www.w3.org/2001/XMLSchema” />

Add MatchCount/SampleCount to PA MemberModule
<MatchCount>$Config/MatchCount$</MatchCount>
<SampleCount>$Config/SampleCount$</SampleCount>

Unix.ShellCommand.Invoke.Script
Alternate example for monitors, the SQL Windows Replication mgmt pack has a good UnitMonitor/UnitMonitorType example – Microsoft.SQLServer.Replication.Windows.Monitoring.xml

 

References

Kevin Holman has a good example for changing frequency and MatchCount here
https://kevinholman.com/2017/08/12/creating-a-scom-service-monitor-that-allows-overrides-for-interval-frequency-and-samples/

Find example by searching unsealed management pack repository (use Tyson’s SCOMHelper PowerShell module to unseal mp/mpb’s to facilitate a better unsealed mp search) https://monitoringguys.com/2019/11/12/scomhelper/

 

 

 

Detected malicious verification code error

'detected malicious verification code' errors
Detected malicious verification code when verifying element – ever run into this scenario while authoring?

 

 

Ever run into the ‘detected malicious verification code’ error while authoring?  I ran into the malicious verification error authoring, and couldn’t find any content for this error while authoring a pack.

 

Watch your copy/paste’s with additional monitoring changes to prevent ‘detected malicious verification code’ errors

In my authoring example, I received the ‘detected malicious verification code error’ after adding Rules, Datasources, and WriteActions (including tasks).  I was copying and pasting DataSources (DS) and WriteActions (WA), thought I had it all.  Uploaded > got the error, and GRR!   Hopefully this will help others authoring to know what to check to get the management pack uploaded.

 

Simply put – Watch out for typo’s to avoid ‘detected malicious verification code’ errors!

I stumbled across a few websites, but nothing really pointed out to what caused the ‘detected malicious verification code error’ when uploading a management pack.  First, check monitor and rules to verify the DS/WA are called correctly (no errors in file names.  Check the Tasks as well as DisplayStrings, to make sure everything matches.

 

Error Seen when uploading Management pack from SCOM Console GUI regarding ‘detected malicious verification code’ error

<ManagementPackNameHere> Reports could not be imported.

 

If any management packs in the Import list are dependent on this management

pack, the installation of the dependent management packs will fail.

 

Verification failed with 1 errors:

——————————————————-

Error 1:

Found error in

2|<ManagementPackNameHere>|1.0.0.6|<ManagementPackNameHere>|

| with message:

Detected malicious verification code when verifying element of type

Microsoft.EnterpriseManagement.Configuration.ManagementPackRule with inner

exception: System.Collections.Generic.KeyNotFoundException: The given key

was not present in the dictionary.

at System.ThrowHelper.ThrowKeyNotFoundException()

at System.Collections.Generic.Dictionary`2.get_Item(TKey key)

at

Microsoft.EnterpriseManagement.Configuration.ManagementPackRule.VerifyDataTy

pes(Dictionary`2 moduletypes)

at

Microsoft.EnterpriseManagement.Configuration.ManagementPackRule.Verify(Verif

icationContext context)

at

Microsoft.EnterpriseManagement.Configuration.Verification.VerificationEngine

.VerifyCollectionItems(Object context)


 

Additional links

Detected malicious verification code when verifying element

Forum https://social.technet.microsoft.com/Forums/en-US/ac50ae14-882a-4788-a8e4-6a975c498a29/detected-malicious-verification-code-when-verifying-element-of-type

Caution using Tags/Notes extending classes

Caution
Caution

 

Please take ‘caution using Tags/Notes extending classes’.  Please read below if you use Tags/Notes on SCOM classes.  Ran across examples where SCOM Class Properties were used for tags that used the .Notes field on various classes, causing orphaned properties, NOT removed from OperationsManager database.

 

 

Background

The Microsoft.Windows.Computer Class (insert class here) is updated using Tim McFadden’s blog.  This can cause issues with orphaned classes in the database because it is not currently handled as part of the stored procedure (i.e. the Notes property classes do not get marked for deletion).

 

First, identify which classes have Notes property.  Start from Management Server (MS) via PowerShell.   See attached TXT for additional examples to check and add/remove Notes Property on additional windows classes.

 

 

Set Notes property for Windows Operating System server

Second, we need to see how to set and clear the value, in order to clean up the Operations Manager database, to remove the orphaned instances.  The example below sets the value for one (1) server to ‘Production’.

 

$WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
$WOS.'[System.ConfigItem].Notes’

$WOS.'[System.ConfigItem].Notes’.Value = “Production”

$WOS.Overwrite()

$WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”

$WOS.'[System.ConfigItem].Notes’

 

 

Example Output

PS C:\Users\scomadmin> $WOS.'[System.ConfigItem].Notes’.Value = “Production”
PS C:\Users\scomadmin> $WOS.Overwrite()
PS C:\Users\scomadmin> $WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
PS C:\Users\scomadmin> $WOS.'[System.ConfigItem].Notes’

PropertyAccessRights : Unknown
Parent : Microsoft Windows Server 2016 Standard
Type : Notes
Value : Production
Id : 00000000-0000-0000-0000-000000000000
ManagementGroup : SCOM2016
ManagementGroupId : e39f5f53-9fbb-9d7f-4bfe-5f0324630ae5

 

 

Set Notes property to NULL

$WOS.'[System.ConfigItem].Notes’.Value = $null
$WOS.Overwrite()

$WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”

Verify Notes value

$WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
$WOS.'[System.ConfigItem].Notes’

 

 

Example Output
PS C:\Users\scomadmin> $WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
PS C:\Users\scomadmin> $WOS.'[System.ConfigItem].Notes’

PropertyAccessRights : Unknown
Parent : Microsoft Windows Server 2016 Standard
Type : Notes
Value : (null)
Id : 00000000-0000-0000-0000-000000000000
ManagementGroup : SCOM2016
ManagementGroupId : e39f5f53-9fbb-9d7f-4bfe-5f0324630ae5

 

Have a happy Holiday!

Good luck, hopefully this scenario isn’t something that impacted the monitoring environment!