Check your delegation settings

 

Sometimes as the monitoring admin, you may be responsible to secure your servers, being told from Security/Cyber Teams about new vulnerabilities.  The vulnerabilities may be from Tanium, ACAS, Tenable or other security tools.   This article is how to help you secure related SCOM web console, and SSRS reporting sites against Unconstrained Delegation vulnerabilities CVE-2020-17049, also AD STIG V-36435.

 

First we need to identify IF this is a true finding.

Typically this comes from Server/SystemsAdmin with domain admin access:

From PowerShell run:

Get-ADComputer -LDAPFilter
“(userAccountControl:1.2.840.113556.1.4.803:=524288)”

After identifying SCOM servers with unconstrained delegation (scope of blog post is focused on SCOM in a hybrid scenario (prem/cloud), we need to resolve.

With domain administrator rights, open the Active Directory Users and Computers MMC snap-in.

In ADUC, change Find drop-down to Computers
In the Computer name text box, enter <SCOMServer>  and click search

Right click the server in the results box > Select Properties.

Select the Delegation tab.

ADUC view of lab server delegation setting

 

Select Trust this computer for delegation to specified services only > Use any authentication protocol.

Under Services to which this account can present delegated credentials, select Add.

In the new dialog box, select Users or Computers.

Enter <SCOMServer>, and then select OK.

Click the Add button to add services

Select the w3svc and www processes

Select OK.

ADUC GUI adding services for delegation on SCOM server

Once set in AD, reboot server.  Running ‘gpupdate /force’ may not apply AD changes to the server object.

After reboot, reach out to SCOM Admins to test webconsole authentication

From edge browser, go to SCOM web console, typically @ https://<SCOMServer>/OperationsManager

On the Monitoring tab, click on Active Directory dashboard on left

Verify authentication works

 

Documentation

Pentestlab – Detecting Unconstrained Delegation Exposures in AD Environment

Petri.com find and block unconstrained delegation

Learn.Microsoft.com unconstrained kerberos article

Explanatory documents on what/why

Remove Unconstrained Kerberos Delegation

 

Configure MMA agent via PowerShell

A car mechanic uses battery jumper cables to charge a dead battery.

 

Do you feel like a mechanic having to jump start the agent configuration like a dead car battery?   Assuming the Agent is already installed, you can configure the SCOM agent via PowerShell.  Even better when you can PowerShell remote to multiple systems.  I hope the PowerShell commands below help you master PowerShell to configure the SCOM side of the MMA agent (house).

 

powershell

/*
# Find/replace variables to your environment like Kevin Holman’s fragments!
##SCOMMGMTGROUP1##
##SCOMMGMTGROUP2##
##SCOMMGMTSERVER1##
##SCOMMGMTSERVER2##
#
*/

$SCOMAgent = New-Object -ComObject AgentConfigManager.MgmtSvcCfg
$SCOMAgent.GetManagementGroup(“##SCOMMGMTGROUP1##”);$SCOMAgent.GetManagementGroup(“##SCOMMGMTGROUP2##”)

# If mgmt groups are incorrectly set
$SCOMAgent.RemoveManagementGroup(“##SCOMMGMTGROUP1##”)
$SCOMAgent.RemoveManagementGroup(“##SCOMMGMTGROUP2##”)

restart-service healthservice

# Domain
$SCOMAgent.AddManagementGroup(“##SCOMMGMTGROUP1##”,”##SCOMMGMTSERVER1##”,5723)

# Verify agent config
$SCOMAgent.GetManagementGroup(“##SCOMMGMTGROUP1##”)
# If you have a second management group

$SCOMAgent.GetManagementGroup(“##SCOMMGMTGROUP2##”)

# Restart and test connectivity
restart-service healthservice

# Check connectivity
test-netconnection -port 5723 -computername ##SCOMMGMTSERVER1##

 

 

SCOM Monitor reset logic

ResetButton

 

Ever want to reset SCOM monitors, and wish it was just a simple Reset Button for unhealthy monitors?

 

I’ve been using Scott Murr’s TechNet gallery loop to maintain my alerts, and ensure monitors are healthy for all my management packs.

 

The blurb I put in my DS/WA scripts to reset SCOM monitors.  I build on Andrew’s methods I didn’t realize (just think much uglier code!)

Cleaner PowerShell to help reset monitors and rules

 

 

My PowerShell variables to reset SCOM monitors, includes my Addendum and the core – DNS example provided below (thank you Andrew!)

 

## Grab the MP, get the Monitors and Rules from the MP, then grab all alerts found inside the Monitors/Rules

$SCOMCoreMP = Get-SCOMManagementPack -DisplayName “Microsoft Windows Server 2016 and 1709+ DNS Monitoring”
$SCOMAddendumMP = Get-SCOMManagementPack -DisplayName “Microsoft Windows Server 2016 DNS Monitoring Addendum”

$SCOMCoreRules = $SCOMCoreMP.GetRules()
$SCOMCoreMonitors = $SCOMCoreMP.GetMonitors()
$SCOMAddendumRules = $SCOMAddendumMP.GetRules()
$SCOMAddendumMonitors = $SCOMAddendumMP.GetMonitors()

$SCOMCoreReportAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMCoreRules.DisplayName) -or ($_.Name -in $SCOMCoreMonitors.DisplayName) }
$SCOMCoreReportAlerts.Count
$SCOMAddendumReportAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMAddendumRules.DisplayName) -or ($_.Name -in $SCOMAddendumMonitors.DisplayName) }
$SCOMAddendumReportAlerts.Count

$SCOMOpenReportAlerts = $SCOMAddendumReportAlerts | ? { ( $_.ResolutionState -ne “255” ) }
$SCOMOpenReportAlerts.Count
$SCOMOpenAddendumReportAlerts = $SCOMAddendumReportAlerts | ? { ( $_.ResolutionState -ne “255” ) }
$SCOMOpenAddendumReportAlerts.Count

$SCOMCoreRuleAlerts = Get-SCOMAlert | ? { ( $_.Name -in $SCOMCoreMonitors.DisplayName) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMCoreRuleAlerts.Count
$SCOMAddendumRuleAlerts = Get-SCOMAlert | ? { ( $_.Name -in $SCOMAddendumRules.DisplayName) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMAddendumRuleAlerts.Count

$SCOMCoreMonitorAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMCoreMonitors.DisplayName ) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMCoreMonitorAlerts.Count
$SCOMAddendumMonitorAlerts = Get-SCOMAlert | ? { ($_.Name -in $SCOMAddendumMonitors.DisplayName ) -AND ( $_.ResolutionState -ne “255” ) }
$SCOMAddendumMonitorAlerts.Count

$AutoClosed = $SCOMCoreMonitorAlerts.Count + $SCOMCoreRuleAlerts.Count + $SCOMAddendumMonitorAlerts.Count + $SCOMAddendumRuleAlerts.Count
$Test = $SCOMCoreReportAlerts.Count + $SCOMAddendumReportAlerts.Count
$OpenAlerts = $SCOMOpenReportAlerts.Count + $SCOMOpenAddendumReportAlerts.Count
$ResetMonitors = $SCOMCoreMonitors + $SCOMAddendumMonitors
$MonitorAlerts = $SCOMCoreMonitorAlerts.Count + $SCOMAddendumMonitorAlerts.Count

 

 

#
# If Cleanup needed, array of report monitors

# Reset Monitors Script
# Put ps1 in mgmtpacks folder
# https://sc.scomurr.com/scom-2012-monitor-reset-cleaning-up-the-environment/
# Download
# https://gallery.technet.microsoft.com/SCOM-2012-Batch-reset-63a17534

#Alternate
#https://gallery.technet.microsoft.com/scriptcenter/Auto-reset-script-for-d8b775ca

if ( $MonitorAlerts -gt 0 )
{
foreach ( $MonitorDisplayName in $ResetMonitors.DisplayName )
{
$Monitors = @( Get-SCOMMonitor -displayname $MonitorDisplayName )

# Set up monitor objects to reset
foreach ($Monitor in $Monitors)
{
$MonitorClass = Get-SCOMClass -Id $Monitor.Target.Id
$ActiveMonitors = Get-SCOMClassInstance -Class $MonitorClass | ? { ($_.healthstate -ne ‘Success’) -AND ( $_.healthstate -ne ‘Uninitialized’) -AND ($_.IsAvailable -eq $true) }
write-host “Found” + $ActiveMonitors.Count + “active monitors.”
if ( $ActiveMonitors -ne $null)
{
foreach ($ActiveMonitor in $ActiveMonitors)
{
write-host ” Resetting Health State on ‘” +$ActiveMonitor.FullName + “‘”
$ActiveMonitor.ResetMonitoringState($Monitor.ID)
}
}
}
}
}

SCOM Snapshot Synchronization alerts

Synchronized Swimmers

Updated 4 Apr 2023 with Tyson’s feedback!

 

First, some background – the Snapshot Synchronization alert just tells you there was a SQL issue running the workflow.

Second, the Snapshot Synchronization alert from a health model perspective, is NOT a critical issue (outage).  Create override severity to 1 (warning) to prevent false wake-up calls.  I’ll get this to my GitHub repo shortly!

 

 

 

Let’s troubleshoot the alert

Start with Tyson’s SCOM Maintenance pack, and run the tasks

Tyson’s SCOM Maintenance pack tasks

 

Alternative long steps

Login to server with SSMS installed –

Open SSMS > Connect to the SCOM OpsMgr DB > Click on New Query

***NOTE verify database dropdown shows Operations Manager!

Paste SQL query into the query textbox

Select WorkItemName, b.WorkItemStateName, ServerName, StartedDateTimeUtc, CompletedDateTimeUtc, DurationSeconds, ERRORMESSAGE
from cs.WorkItem a , cs.WorkItemState b
where a.WorkItemStateId= b.WorkItemStateId
and WorkItemName = ‘SnapshotSynchronization’

Screenshot

Snapshot Synchronization alert query

 

 

 

Example SQL Output

SQLQueryExecution

WorkItemName WorkItemStateName ServerName StartedDateTimeUtc CompletedDateTimeUtc DurationSeconds ERRORMESSAGE
SnapshotSynchronization Succeeded SCOMV01 2023-01-24 00:26:23.427 2023-01-24 00:27:46.100 83 NULL
SnapshotSynchronization Failed SCOMV02 2023-01-25 00:27:52.363 2023-01-25 00:28:07.520 15
SnapshotSynchronization Succeeded SCOMV00 2023-01-25 21:43:36.540 2023-01-25 21:45:07.947 91 NULL
SnapshotSynchronization Running SCOMV00 2023-01-25 21:45:32.227 NULL NULL NULL

Solution:
The jobs may show Succeeded by the time you login to SQL = EOJ (end of job)

If Failed is latest date/timestamp, re-run the task “Request Snapshot Synchronization” which can be found when we select  “Management Configuration Service Group” in the below mentioned view.

View:

From Monitoring Tab > Click on Operations Manager folder > Click on Management Group Health widget > Highlight unhealthy state from Management Group Functions.

Click on the ‘Request Snapshot Synchronization’ task to execute the Stored Procedure “SnapshotSynchronizationForce” on the OpsMgr DB.

 

NOTE: There are two tasks with same name but with different targets i.e. ‘Management Configuration Service Group’ and ‘Management Configuration Services’

 

The other task can be found on below view after selecting the Management Server you want the Task to be executed on

View:

From Monitoring Tab > Expand Operations Manager folder > Expand Management Configuration Service folder > Click on Services State view

 

 

Create Override for the alert

To change snapshot monitor to warning

From SCOM Console > Authoring Tab

Expand ‘Management Pack Objects’ > Click on Monitors

In the ‘Look for:’ bar type Snapshot synchronization state and hit enter

Monitor name = Snapshot synchronization state

Right click on monitor > Overrides > Override the monitor > For all objects of class

Click checkbox for Severity > change Critical to Warning

Click Edit – add comment – i.e. date/time changing to warning

Select your override pack > Click OK

Click OK to execute change

Override snapshot monitor to warning

 

 

Resources

Tyson’s MonitoringGuys blog for SCOM Maintenance pack – download here

Link to TechNet article