monitoring Archives - Kevin Justin's Blog

OS Addendum updates

More updates for your monitoring pleasure with OS addendum updates!

OS Addendum updates

Been busy in the monitoring ‘bat’ cave crafting up new ways to simplify things, automating recoveries, top process finds, STIG compliance, automatic services logic, and PowerShell transcription checks.

Download https://github.com/theKevinJustin/2016ServerAgnostic/

Automatic Services – monitors services with automatic service type, and includes built in recovery when monitor unhealthy

PowerShell transcription – be compliant for STIG V-257502 and V-257503, also includes Daily report (runs daily Monday-Friday)

Sets multiple overrides to reduce extraneous OS workflows that most customers never use.

Change/revision history

v1.0.7.6 13 Jan 2025 Updated EnableTranscription Report variables
v1.0.7.5 8 Nov 2024 Added additional services into excluded services, uncommented ServiceName variable
v1.0.7.4 7 Nov 2024 Updated ExcludedServices in AutomaticServices monitor DS
v1.0.7.3 21 Oct 2024 Logical Disk rollup disable, hourly monitor setting changes, event and perf collection rule disabled
v1.0.7.2 21 Oct 2024 Updated AutomaticServices DS/WA workflows
v1.0.7.1 18 Oct 2024 Updated Excluded Services array with additional services, updated AutomaticServices DS/WA workflows.
v1.0.7.0 17 Oct 2024 Updated AutomaticServices monitor timeout from 240 to 600 for OpsMgr event ID 22404
v1.0.6.8 16 Oct 2024 Updated Storport Timed Out Monitor from 5 to 10 events per hour
v1.0.6.8 11 Oct 2024 Additional EnableTranscription workflows – reports, report cleanup, transcription log cleanup
v1.0.6.7 11 Apr 2024 New weekly PowerShell Enable Transcription monitor for STIG V-257502 and V-257503, and new weekly report
v1.0.6.5 10 Apr 2024 Updated recovery logic for AutomaticServices DS/WA
v1.0.6.4 9 Apr 2024 Updated AutomaticServices DS/WA logic change, ServiceName variable change to Service

SQL STIGMonitor

Seriously, dream on! End the STIGma is a good thing, but STIGs can be a burden. Hit the easy button, if you’re not already using it. Contact your SQL Data and AI Cloud Solutions Architect for the latest SQL STIG Monitor 2024 Q4 build!

Latest SQL STIG monitor 31 Oct 2024 release includes

DISA UPDATES – see link
MS SQL Server 2016 Instance STIG, V3R2:

(NOTE: DISA has been contacted to remove related CCI STIGID for AzureSQLDB that was overlooked: ASQL-00-010700)

POWERSHELL MODULE
Updated version to 1.23
Added STIGID parameter to Invoke-StigMonitor allowing granular control over STIGID scanning.

DATABASE CHANGES
Updated Checklist Templates for Q4 Revisions.
Updated Instance & Database STIG for Q4 benchmark date.
Script updates include:

CNTNMIXDB: Not A Finding if using Windows Auth
FORCENRYPT: NA if using Windows Auth
PWDCMPLX: Updated Finding to remove OS STIG reference
AZDBPERMISS: Revised script with new version.
DBPERMISS: Revised script with new version.
ENFCACCSS: Revised script with new version.
PSERRPERM: Revised script with new version.
UNQSVCACC: Removed code stripping out port number.
AZAUDITSTATE: Properly returns No Finding when audit setup is correct.
Fixed bug in vDocumentation view causing POAMs to not display custom comment in exported documentation.
Added usp_RemoveInstance stored procedure to easily clean up a specific Instance from StigMonitor that no longer exists.

DOCUMENTS
Updated checklist templates, Approvals scripts, and Documentation Templates for Q4 Revisions.
Removed Set-CEIPRegKeys.ps1, Set-FIPSCompliance.ps1, and Set-SqlRegKey.ps1 in favor of Module commands.
Updated InfoPage with new StigMonitor logo and text references.
Documentation updated with new examples of Invoke-StigMonitor STIGID parameter.
Updated documentation to add Azure DB Permission for MS_SecurityDefinitionReader.
Added DatabaseName to CSV Export of Export-StigDocumentation.

REPORTS
Updated Report banner to display new StigMonitor logo and latest report versions.
Removed Adhoc scanning to Policy Management Report in favor of Invoke-StigMonitor parameter.
Removed references to Sunset 2012 and 2014 STIGs.
Added AzureSQLMI for future use.
Combined NF and Approved in Total Findings summary
Reduced Recent Scans to latest 6.

Also please send us your feedback if you get a chance to check this out.
If you want to be added/removed from this, click here (Subscribe /Unsubscribe) or send us an email.

DNS Scavenging alerts

Need DNS Scavenging alerts, to see what’s cleaned up, or that scavenging failed? Download the DNS Addendum pack from my GitHub repo https://github.com/theKevinJustin/DNSAddendumAgnostic

Latest revision first includes a EventID 2502 monitor for scavenging failed. Second, the monitor has count logic (setup to alert with 2 events in 30 minutes). Third, EventID 2501 rule details scavenging totals. Lastly, built a weekly report to summarize the scavenging alerts (cliff notes!).

Some quick ‘how-to’ setup DNS scavenging

Example of RegKey showing that Scavenging is setup – note Scavenging Interval key

Example of AD integrated DNS setup with 21 day scavenging interval, and prompts to configure (click OK twice)

DNS Scavenging setup on AD integrated DNS server

Import management pack, and run DNS scavenging.

Verify scavenging alerts

SCOM Monitoring Tab > Active Alerts > ‘Look for:’ scavenging

Example output

Additional SCOM PowerShell commands

Run PowerShell commands from the SCOM management server (MS)

$DNSAlerts = get-scomalert -name "*Scavenging*"
$DNSAlerts
$DNSAlerts | format-table PrincipalName,TimeRaised,Description -auto -wrap

Example Output

PS C:\Users\scomadmin> $DNSAlerts = get-scomalert -name “*Scavenging*”

PS C:\Users\scomadmin> $DNSAlerts

Severity Priority Name TimeRaised

——– ——– —- ———-

Warning Normal Windows DNS Event 2502 Scavenging Failed monitor addendum alert 8/19/2024 2:02:3…

Warning Normal Windows DNS Event 2502 Scavenging Failed monitor addendum alert 8/19/2024 1:07:0…

Information Normal Proactive DailyTasks DNSAlerts Scavenging Summary Report Alert 8/19/2024 10:11:…

DNS alerts formatted

PS C:\Users\scomadmin> $DNSAlerts | format-table PrincipalName,TimeRaised,Description -auto -wrap

PrincipalName TimeRaised Description

————- ———- ———–

DC02.testlab.net 8/19/2024 2:02:32 PM Windows DNS Event 2502 Scavenging Failed monitor alert 1 alert in 15 minutes

Event Description:

The DNS server has completed a scavenging cycle but no nodes were visited.

Possible causes of this condition include:

…

The next scavenging cycle is scheduled to run in 168 hours.

Learn articles for more details https://learn.microsoft.com/en-us/troubleshoot/windows-server/networking/dns-scavenging-setup

File Services Addendum

‘File Services Addendum’, named Microsoft Windows Server FileServices 2016 Addendum, adds replication health/backlog script, seed and group classes, replication/service monitors, recovery tasks, and overrides to tune monitored environment.

Quick Download HTTPS://GITHUB.COM/THEKEVINJUSTIN/FILESERVICESADDENDUM

Overview of File Services monitoring

Addendum assumes the file services version agnostic version 10. pack is installed.

Looking at XML file in Notepad++, the pack references are what packs the workflows refer to (other management packs). Kevin Holman taught building backwards compatibility with MP authoring. Backwards compatibility allows SCOM2012+ import without errors. To take this one step further, the v10.0.0.0 file services packs referenced represent the version agnostic packs.

NOTE: File Services Addendum references may need updates if the whole file services management packs are NOT installed.

References screenshot

Addendum logic

Capabilities

Daily report and close automation, on-demand tasks for reports

DFS backlog script errors

SmSvc, DFSN, DFSR service recovery and rule alerts (from Holman fragments library)

DFS replication backlog watcher, script, alerts

Notepad++ screenshot

Next, we look at the group/class discoveries

Update the Class/Group discoveries for DFS servers or script install paths for replication script.

Update Class/Group discoveries for DFS servers or script install paths for replication script.

Find and replace FilePath and ##DFSServerNamingConvention## variable.

Save file and Import

Documentation

Kevin Holman MP authoring with fragments https://kevinholman.com/2019/01/17/mp-authoring-with-fragments-introducing-combo-fragments/

Kevin Holman MP fragment library https://github.com/thekevinholman/FragmentLibrary

Addendum GitHub Repository HTTPS://GITHUB.COM/THEKEVINJUSTIN/FILESERVICESADDENDUM

MCM Addendum pack

Rebranding central – MEM, EM, MECM, SCCM, Configuration manager, depending on the synonym, we’re referring to the same product. Tune the most common critical alerts per the health model to warning.

QUICK DOWNLOAD https://github.com/theKevinJustin/MCMAddendum/

Background

Read Holman’s blog for more details.

Did you know – MCM discoveries are based on registry keys added with various role installs on windows servers. These registry keys are typically under this path: HKLM\SOFTWARE\Microsoft\SMS\Operations Management\Components

What capabilities does the ‘MCM addendum pack’ provide?

Quite simply, the pack provides warning severity overrides for common alerts, disable event collection rules.

Includes warning severity changes for the following rules and monitors:

Monitors

BackupStatus.StatusMessage.Monitor

ReportingPoint.RoleAvailability.Monitor

SoftwareUpdatePoint.RoleAvailability.Monitor

SoftwareUpdatePointSync.AlertState.Monitor

Rules

ComponentServer.ComponentStoppedUnexpectedly.Event.Rule

SiteComponentManager – CanNotFindObjectInAD.Event.Rule, CouldNotAccessSiteSystem.Event.Rule

StateSystem.FailedToExecuteSummaryTask.Event.Rule

WsusConfigurationManager.FailedToConfigProxy.Event.Rule

Utilize the ‘MCM Addendum pack’

Download Kevin Holman’s MCM pack from GitHub.

Download the Addendum here, to get alerts where manual intervention required.

Save packs

Enjoy some acronym humor and ‘who moved my cheese fun!’

Import into SCOM & Enjoy!

If you need more capabilities, reach out on the blog or GitHub.

Documentation

Github repository here

SCCM management pack

Holman blog for MEM, EM, MCM, MECM, CM, ConfigMgr, Configuration Manager

DHCP Addendum pack

Leverage DHCP addendum to tune DHCP subnet monitoring.

Leverage the ‘DHCP Addendum pack’. Why? DHCP manages IP ranges, particularly customer facing issues like VPN connectivity, VDI/AVD/appliance devices, as well as client workstations/laptops/GFE’s. The DHCP management pack alerts when a subnet is nearing zero available IP’s before you have an outage. This article will help you understand how the addendum’s new capabilities tune DNS monitoring to best practice.

QUICK DOWNLOAD(S)

2016+ HTTPS://GITHUB.COM/THEKEVINJUSTIN/DCHPAGNOSTIC

What capabilities does the ‘DHCP Addendum pack’ provide?

Two groups, one DHCP server group, and DHCP subscription group to configure notifications to SME for DHCP related classes

Overrides for common alerts, disable event collection rules

Utilize the DHCP Addendum

Download the DHCP Addendum on GitHub, to get alerts where manual intervention required.

Update XML

The pack greatly decreases alerts, and the XML authoring is an easy feat. After you import the pack, find/replace is required for two pieces.

Discovery group regular expressions (RegEx)

##DHCPServerRegEx##

Find ##DHCPServerRegEx## and replace with your DNS server expressions.

Example server names: 12dc01, 19dc01,19dc02,19dc03, etc.

RegEx = (?i)12dc0|19dc0

Using PowerShell on SCOM management server (MS) to determine group GUIDs for replace in Overrides/Discoveries.

Update group GUIDs, after installing this pack.

Find/replace the GUIDs, as they are unique to every SCOM management group, hard coding the group ID GUID is not possible. We will be running Get-SCOMClassInstance to determine the group GUID’s applicable in the management group.

From PowerShell, on your SCOM management server, run get-SCOMClassInstance commands for the two groups added.

get-scomclassinstance -DisplayName “Microsoft Windows DHCP 2016+ Servers” | ft Id

get-scomclassinstance -DisplayName “Microsoft Windows DHCP 2016+ subscription components” | ft Id

Example

Leveraging Notepad++ to find/replace the group GUID with SCOM environment specific GUIDs

Find/Replace the GUID in the pack with the ID from the output above.

Save pack

Import into SCOM & Enjoy!

PKI Addendum pack

The ‘PKI Addendum pack’ is a tricky pack, due to certificate hierarchy. The decisions included are part of the three pillars – health, Security, Compliance, as well as alerting WHEN manual intervention required.

QUICK DOWNLOAD https://github.com/theKevinJustin/PKIAddendum

The PKI pack provides discoveries of the server certificate stores to then analyze certificates for validity, chain, and expiration. The v1.4.3.0 release adds some task logic and script changes that helps discover more stores, trusted root, etc.

WHAT CAPABILITIES DOES THE ‘PKI ADDENDUM PACK’ PROVIDE?

Set timeframe for certificate per organizational standards.

Break out different expiration alerts based on internal/external certificate, or by AD Client Certificate enrollment templates (to build out the manual intervention required scenario when alerts are generated).

Create groups breaking out application self-signed, PKI certificates.

Separate RDP Auth, Domain Controller, Computer, and OCSP certificates.

If this sounds interesting, and you want to dabble in XML authoring…

Download the pack from GitHub to improve PKI monitoring on Windows Servers.

Additional screenshots of addendum components

Addendum pack creates multiple groups to break out various types of certificates that have different decisions/behaviors requiring unique timing

Groups

Discoveries

Leverage dynamic groups based on server name and EnhancedKeyUsageList (EKU) list

Overrides

Change PKI pack default discoveries, lifetime threshold expirations and more

DOCUMENTATION AND LINKS

Addendum requires the PKI Certificate MP release v1.4.3.0 download

Bob’s TopQuore blog

Active Directory 2012-2016 Addendum packs updated

Man time flies!

Thought I’d share some new functionality for AD DS (Active Directory Domain Services)

Ran across some customer errors with AD Event ID 1084, which exists in the old 8321 pack, but not in the v10.x pack.

Well, if you get these errors, your DC isn’t replicating, and most likely will need to be rebuilt.

Gallery download

Broke out the packs to separate the Recovery Tasks in their own pack, versus added functionality in the addendum.

Figured better to send packs NOT sealed, so that meant 2 packs,

WYSIWYG (wizzy-wig acronym)

What this means

v1.0.0.1 pack had just the AD DS Service Recovery Tasks

v1.0.0.2 pack has a Service Recovery Tasks pack, and the Addendum pack

What I think is cool is that the Addendum pack contains 2 rules, simple rule event (enabled by default), and also a PowerShell rule.

Rule Figured out how to simply look for criteria, count it, and alert on it.

We always look for alert suppression, some of the sliding/counting monitors are too much.

Starting with Holman’s alerting rule fragment, we can create more powerful combinations than just a single symptom.

Using Variations of the get-date command, we can actually specify how far back to look, to count for alerts.

Easier method to count events, to figure out an alert threshold.

From the rule in the Addendum pack

# Check blog for more detail https://blogs.technet.microsoft.com/heyscriptingguy/2015/01/21/adding-and-subtracting-dates-with-powershell/
# If you want this in other time increments – AddHours, AddSeconds, AddMilliseconds
#
$LastCheck = (Get-Date).AddMinutes(-65)

[int]$TempCount = (get-eventlog -logName “Directory Service” -Source “NTDS Replication” -InstanceID 1084 -Message “*8451 The replication operation encountered a database error*” -After $LastCheck).Count

IF ($TempCount -ge 1)
{
$Result = “BAD”
$Message = “The number of 1084 Replication Database error events was greater than 1”
}
ELSE
{
$Result = “GOOD”
}

Maybe we need multiple event ID’s, or search multiple event logs… you decide, and let me know.

Nutanix Monitoring on SCOM or OMS

Comtrade has been around quite some time now delivering custom management packs, in my experience for everything Citrix, F5 now, and Nutanix for SCOM and OMS.

Their profile is accurate in my opinion “The SCOM Extension Specialists”

Comtrade’s Channel Profile states “we natively integrate with System Center Operations Manager, providing a comprehensive monitoring of network (F5) and hyperconverged infrastructure (Nutanix) with insight into Citrix and Microsoft applications.”

Note: These MP’s are not free. Contact for a trial key and download.

If you use Nutanix hosts, this will provide insights on configuration, logs, resource performance, and overloaded clusters/hosts

SCOM MP dashboards don’t look that much different, but provide easy insight into your virtual environment

How the solution works