SNOW prerequisites
Update incident script and begin testing.
$ServiceNowURL=”https://##SERVICENOWURL##/api/now/table/em_event”
Operational monitoring tools including System Center, Azure Monitor
Use the Tangible SCOM management pack to monitor logins and ProV application registration issues. First, the management pack configures Seed class discovery. Second, the pack includes rules/monitors for Tangible ProV software. Third, rules and monitors for 2802 ‘Could not validate product key’ and 4402 ‘Could not validate the contents of user logon request context: AS-REQ contains an invalid or unknown username type’ events. Fourth, the service monitor, which uses Kevin Holman’s fragment library for service recovery scripts/rules. Fifth, scheduled and on-demand daily reports for audit and record keeping purposes. Lastly, alert cleanup logic, to reduce admin burden and overhead.
Reference the Tangible vendor’s website – Tangible ProV application website
NOTE: This may not apply for everyone, as the ProV application ‘Auto-provisions Active Directory user accounts for visitors or new employees whenever they want to work from one of your PCs.’
The Daily report piece of the pack makes things easier answering ‘what happened in the last 24-72 hours’ question. Gathers open/closed insights and organizes alerts.
Open ProV alerts = 13Total ProV alerts = 23 Auto-closed monitors = 22 Auto-closed rules = 0 Total automation closures: #————————— Auto-closed monitors = 262 Auto-closed rules = 0 # Unhealthy Tangible ProV service alert details #============================================== NetbiosComputerName TimeRaised RepeatCount Name ——————- ———- ———– —- DC01 8/11/2023 5:18:14 AM 0 Tangible ProV ProVService…
Since last report run: #———————–
All in all, the daily report utilizes get and set-SCOMAlert to accomodate large enterprise environments.
$OpenAlerts = get-scomalert -ResolutionState (0..254) -Name “Tangible ProV ProVService Service*”
$OpenAlerts = $OpenAlerts | ? { $_.TimeRaised -ge $Time }
# $OpenAlerts.count
# Closed alerts
$ClosedAlerts = get-scomalert -ResolutionState 255 -Name “Tangible ProV ProVService Service*” | ? { $_.TimeRaised -ge $Time }
# $ClosedAlerts.count
GitHub https://github.com/theKevinJustin/TangibleProV
Download here
My thanks to Aris Somatis for his deep dive reviewing the packs with me, particularly new use cases. The PowerShell below builds on Scott Murr’s initial TechNet published logic from years back. Consequently, the reset logic provides a ‘manual intervention required’ alerting/monitoring system.
Calling the reset method has been a game changer for my customers – including operators, system and application owners!
Scott’s reset logic, from SCOM2012, helped administrators reset unhealthy monitors where alerts may have been closed. Because Scott leveraged the ResetMonitoringState method, the community gained a way to keep true health. Additionally, many administrators and engineers built custom management packs to provide solutions. Second, the addendum packs blog brought in more options – best practices, lessons from the field (and customers), and health model accurate alerting for what was really broken in the environment. Third, addressing ‘gaps’ or ‘blind spots’ from product teams. As a result of NEW monitoring, the packs may include: rules/monitors, datasource/writeAction (DS/WA) workflows, recovery tasks and automation, count logic monitors, overrides, discoveries, and groups. Thirdly, to take monitoring to the next level. To top that off, with very little/NO cost compared to competitors!
Aris’s Age use case takes this even further. Using monitor age allows further analysis to dial down ‘monitor reset’ to object is X days old. Comparatively, the 24-72 hour setup default is used in the addendums, so Age provides a second option. Third option can rely on SCOM’s built-in cleanup, but that’s typically 14-30 days. Overall, flexibility is a good thing.
# Specify age variable for your environment
$Age = [DateTime](Get-Date).AddDays(-7)
First, the reset logic can pivot on the age requirement. Then, adjust the Age variable per requirements. Third, figure out which method applies to gather a unique list of classes, whether by partial string(s), or by management pack name(s).
Set age variable (how long ‘OLD’ monitors might be stale and need reset)
# Example sets $Age variable to 7 days ago (-7)
$Age = [DateTime](Get-Date).AddDays(-7)
Unpack two different ways to gather classes for monitors to reset
# When common string name exists in all classes
Example DFS/FileServices packs all have one of the three strings:
# DFS pack naming
$DFSClasses = @(Get-SCOMClass -Name “*FileServices*”; Get-SCOMClass -Name “*FileServer*”; Get-SCOMClass -Name “*DFS*” )
$DFSClass = $DFSClasses | sort -property Name -uniq
# Debug
$DFSClass.Count
# Get AD classes – Microsoft.Windows.Server.AD.2016.Discovery, Microsoft.Windows.Server.AD.Library
$ADLibrary = Get-SCOMManagementPack -name “Microsoft.Windows.Server.AD.Class.Library”
#get-scomclass -ManagementPack $ADLibrary
$ADMonitoring = Get-SCOMManagementPack -name “Microsoft.Windows.Server.AD.2016.Monitoring”
#get-scomclass -ManagementPack $ADMonitoring | fl DisplayName,Name,ID
$ADDiscovery = Get-SCOMManagementPack -name Microsoft.Windows.Server.AD.2016.Discovery
#get-scomclass -ManagementPack $ADDiscovery | fl DisplayName,Name,ID
# ADDS pack naming
$ADDSClasses = @(Get-SCOMClass -ManagementPack $ADLibrary; Get-SCOMClass -ManagementPack $ADDiscovery; )
# NOTE Excluded AD Monitoring pack as NO classes existed
$ADDSClass = $ADDSClasses | sort -property Name -uniq
# Debug count of unique classes
$ADDSClass.Count
Download from GitHub https://github.com/theKevinJustin/SCOMMonitorReset
NOTE debug logic enabled
0 Found 0 unhealthy monitors for class Microsoft.SystemCenter.HealthServicesGroup 1 Found 1 unhealthy monitors for class Microsoft.SystemCenter.HealthServiceWatcher Resetting Health State on ' + Microsoft.SystemCenter.HealthServiceWatcher:Microsoft.SystemCenter.AgentWatchersGroup;5e0 4f804-8b71-6eb6-0101-dcbb58022498 + ' Guid ---- 0218d239-3d37-f9b1-75d2-6d52c2c7c0c1
See the nice warm toasty updated packs
Fresh off the press, right to your door, just in time for that gift for your special someone! Time for new updates to keep you ever-green’d, up to date, fixes, etc. ;-P
Holman updated his SCOM.Management pack for SCOM2022 UR2
Github https://github.com/thekevinholman/SCOM.Management
Multiple packs with multiple updates. Removed debug detail for DS/WA (Data Source/Write Action workflows) Health Explorer outputs, simplified mgmt pack recovery tasks for single WA script.
Active Directory Certificate Services (ADCS) version agnostic 2016+ addendum https://github.com/theKevinJustin/ADCS2016-Addendum 2012 here See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/18/adcs-addendum-packs/
Active Directory Domain Services addendum https://github.com/theKevinJustin/ADDSAddendumAgnostic
See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/18/adds-addendum-pack/
Active Directory Federation Services addendum https://github.com/theKevinJustin/ADFSAddendum See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/18/adfs-addendum-pack/
FileServices Agnostic addendum https://github.com/theKevinJustin/FileServicesAddendum See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/31/file-services-addendum/
MCM/MEM/MECM/SCCM Configuration Manager addendum https://github.com/theKevinJustin/MECMSCCMAddendum See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/30/mecm-sccm-addendum-pack/
PKI certificate monitoring addendum https://github.com/theKevinJustin/PKIAddendum See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/24/pki-addendum-pack/
Proactive NOSC DailyTasks reports addendum https://github.com/theKevinJustin/ProactiveNOSCDailyTasks See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/15/proactive-daily-reports/
SCOM Core addendum https://github.com/theKevinJustin/SCOMCoreAddendum See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/30/scomcore-addendum-pack/
Top Process workflows tied to monitors in Tier1 https://github.com/theKevinJustin/TopProcessTier1 See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/15/proactive-daily-reports/
Tier0 https://github.com/theKevinJustin/TopProcess
Windows Server 2012/2012R2 Operating System Addendum https://github.com/theKevinJustin/2012OSAddendum See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/28/os-addendum-packs/
Windows Server 2016+ version agnostic Operating System Addendum https://github.com/theKevinJustin/2016ServerAgnostic See the blog post for capabilities here https://kevinjustin.com/blog/2023/08/28/os-addendum-packs/
Enjoy!
Updated DNS2012R2 Addendum overrides. Learned a few new things with Overrides workspace views, and why Authoring pane > Management pack Objects > Overrides may not load.
When your management pack has improper overrides, expect the loading icon. This may be caused due to overrides, whether error is with target, class/rule/monitor.
Sometimes, an Object of class error gets your hopes up (pointing at a non-existent object).
Example when Overrides loads properly
If Overrides view will not load, try creating a workspace view for Overrides.
Navigation Steps:
From SCOM Console
Click on My Workspace
Right Click > New > Overrides Summary View
Select checkbox ‘with a specific override management pack’ checkbox, then the ‘specific’ link to choose management pack(s).
Choose unsealed management pack(s) with overrides
Can select all – OR pick a few to see what loads without errors
Click OK
If you get the loading screen and error, now begins the pack analysis.
Clicking on the ‘Show’ link points to a non-existent object
Microsoft.EnterpriseManagement.Common.ObjectNotFoundException: An object of class ManagementPackClass with ID 76e2559c-aaf4-b1ec-60cf-d40ab4102fbc was not found.
How did I know that?
Run get-SCOMClassInstance command from PowerShell or Operations Manager shell
Example output of ‘get-SCOMClassInstance -ID “76e2559c-aaf4-b1ec-60cf-d40ab4102fbc” ‘
Work on the Overrides of the affected XML packs, and Import.
Once corrected, the Workspace view loads successfully, finite!
My Workspace https://learn.microsoft.com/en-us/system-center/scom/manage-web-console-my-workspace?view=sc-om-2022
Monitoring workspace https://learn.microsoft.com/en-us/system-center/scom/manage-using-monitoring-workspace?view=sc-om-2022
‘NiCE VMware addendum’ enhances VMware monitoring, tuning alerts to ‘manual intervention’ required alerting. The NiCE folks have been around for some time as a trusted Microsoft partner, creating additional monitoring functionality across Microsoft products. Having completed a number of projects implementing the VMware pack, it’s time to share the configuration and alert report capabilities.
Key breakdown of VMware ESX environment monitoring
Adjustments to vendor pack to further the mantra ‘alert when manual intervention required’.
Set monitor alerts to multiple samples over an hour (i.e. compute and performance of ESX environment)
Reports by team (requires regular expression updates for environment servers owned by each team)
Monitor reset logic, and service monitorType (count logic for X failures over Y time, before alert)
Overrides to change vendor pack provided discoveries, rules, monitors
Remove alert noise for unmanaged objects in ESX environment
Customize the ‘NiCE VMware addendum’ pack for specific environment. This means updating group discoveries, and GUIDs for group specific overrides. Further updates are required to update server naming conventions for team virtualization reports.
Classes/groups created for pack
Discoveries
Breakout of Discoveries that need pattern updates to match
Find/Replace ##ESXHostDataStoreNamingConventions## with names to exclude
Example of regular expressions for multiple customers
Update disable guest machine alerts
Disable guest machines in ESX environment to disable alerts.
Find ##ESXGuestServersDiskUsageNamingConventions##
Replace with relevant guest naming conventions
Example template/guest/virtual machine names typically disabled
Service MonitorType
Service MonitorType adds Samples and Intervals to alert after consecutive failures (x failures in y minutes then alert )
Rules, Monitors, Recoveries
List of workflows used to troubleshoot/resolve problems
NiCE VMware management pack https://www.nice.de/nice-vmware-mp/
‘File Services Addendum’, named Microsoft Windows Server FileServices 2016 Addendum, adds replication health/backlog script, seed and group classes, replication/service monitors, recovery tasks, and overrides to tune monitored environment.
Addendum assumes the file services version agnostic version 10. pack is installed.
Looking at XML file in Notepad++, the pack references are what packs the workflows refer to (other management packs). Kevin Holman taught building backwards compatibility with MP authoring. Backwards compatibility allows SCOM2012+ import without errors. To take this one step further, the v10.0.0.0 file services packs referenced represent the version agnostic packs.
NOTE: File Services Addendum references may need updates if the whole file services management packs are NOT installed.
References screenshot
Capabilities
Daily report and close automation, on-demand tasks for reports
DFS backlog script errors
SmSvc, DFSN, DFSR service recovery and rule alerts (from Holman fragments library)
DFS replication backlog watcher, script, alerts
Notepad++ screenshot
Next, we look at the group/class discoveries
Update Class/Group discoveries for DFS servers or script install paths for replication script.
Find and replace FilePath and ##DFSServerNamingConvention## variable.
Save file and Import
Kevin Holman MP authoring with fragments https://kevinholman.com/2019/01/17/mp-authoring-with-fragments-introducing-combo-fragments/
Kevin Holman MP fragment library https://github.com/thekevinholman/FragmentLibrary
Addendum GitHub Repository HTTPS://GITHUB.COM/THEKEVINJUSTIN/FILESERVICESADDENDUM
IIS addendum packs to tune IIS from 2012 forward.’IIS addendum packs’ to tune IIS from 2012 forward. The GitHub repository has two packs 2012/2016+ (version agnostic pack). This includes an IIS enabled group, Daily report and cleanup DataSource and WriteAction (tasks), as well as a regular expression to set up the IIS enabled group. The IIS enabled group is to enable IIS monitoring on servers IIS monitoring is needed.
Update addendums to server naming conventions for enabled IIS monitoring. Read below to better understand addendum functionality.
First, the addendums include class/group, datasource and write action alert reports and automated alert closure workflows, as well as event count logic/reset monitorType.
Second, the group discovery, find/replace the pattern to various application/web server naming conventions where IIS monitoring IS wanted.
Third, the version agnostic has overrides to disable most perf and rule alerts. Can provide OFF packs to turn off performance counter collection rules, to keep both the OperationsManager, and OperationsManagerDW databases cleaner, thereby faster with less data.
Lastly, once addendum updated, save file, move to SCOM MS, and import.
Enjoy the ‘IIS addendum packs’ for how few alerts, perhaps life changing?! (sarcasm)
Download Addendum packs https://github.com/theKevinJustin/IISAddendums
IIS2012 SCOM Management pack download https://www.microsoft.com/en-us/download/details.aspx?id=34767
IIS2016+ SCOM management pack download https://www.microsoft.com/en-us/download/details.aspx?id=54445