Tag: MP
NiCE VMware addendum
‘NiCE VMware addendum’ enhances VMware monitoring, tuning alerts to ‘manual intervention’ required alerting. The NiCE folks have been around for some time as a trusted Microsoft partner, creating additional monitoring functionality across Microsoft products. Having completed a number of projects implementing the VMware pack, it’s time to share the configuration and alert report capabilities.
Quick Download HTTPS://GITHUB.COM/THEKEVINJUSTIN/NICEVMWAREADDENDUM/
Changes to Nice vmware pack
Key breakdown of VMware ESX environment monitoring
Adjustments to vendor pack to further the mantra ‘alert when manual intervention required’.
Set monitor alerts to multiple samples over an hour (i.e. compute and performance of ESX environment)
Reports by team (requires regular expression updates for environment servers owned by each team)
Monitor reset logic, and service monitorType (count logic for X failures over Y time, before alert)
Overrides to change vendor pack provided discoveries, rules, monitors
Remove alert noise for unmanaged objects in ESX environment
Customize pack for environment
Customize the ‘NiCE VMware addendum’ pack for specific environment. This means updating group discoveries, and GUIDs for group specific overrides. Further updates are required to update server naming conventions for team virtualization reports.
Classes/groups created for pack
Discoveries
Breakout of Discoveries that need pattern updates to match
Find/Replace ##ESXHostDataStoreNamingConventions## with names to exclude
Example of regular expressions for multiple customers
Update disable guest machine alerts
Disable guest machines in ESX environment to disable alerts.
Find ##ESXGuestServersDiskUsageNamingConventions##
Replace with relevant guest naming conventions
Example template/guest/virtual machine names typically disabled
Service MonitorType
Service MonitorType adds Samples and Intervals to alert after consecutive failures (x failures in y minutes then alert )
Rules, Monitors, Recoveries
List of workflows used to troubleshoot/resolve problems
Documentation
NiCE VMware management pack https://www.nice.de/nice-vmware-mp/
MSSQL Addendum pack
The ‘MSSQL Addendum pack’ wouldn’t be possible without Brandon Pires contributions. Brandon dealt with my many questions to better alert! If you need more background, check the ‘why addendum pack’ post.
Quick Download(s)
2012+ https://github.com/theKevinJustin/MSSQLAddendum
Capabilities
The pack is based on the SQL engineering blog and program team making multiple updates per year for SQL monitoring. The addendum creates two groups for dev/test and notification/subscription modeling. Second, the overrides, man there are a bunch! aid consumption of real issues. Lastly, most environments should be SQL 2016+, as the 2012R2 EOL/EOSL is quickly approaching in October!
MSSQL group discoveries require updates to be applicable to environment
Tailor addendum
First, the Addendum pack requires the MSSQL packs MUST be installed. The addendum is based on the MSSQL 2016+ version agnostic is currently supported, as the 2012,2012R2 products are near end of support.
Find/Replace the variables as needed:
Example ##TESTSERVER##|##DEVSERVER##
Save file
Overrides
Addendum pack contains discovery, monitor, and rule overrides to tune MSSQL to CSA (old PFE/CE/CSAe Microsoft Field engineer recommendations), to match the health model reducing critical ‘wake me up in the middle of the night’ alerts.
Import
Download pack, and save to your environment
Import into SCOM
Enjoy!
MSSQL Addendum references
MSSQL Engineering blog and old post here
SQL Releases TechCommunity here
Engineering team latest management pack, TechCommunity release v7.2.0.0
Import ‘gotcha’ importing new custom functionality blog
Why Addendum packs
‘Why addendum packs’? What value can they bring to my customer? Kevin Holman started the Addendum thought process quite a while back. Added functionality to a core application/program/product. The first example of this pack naming convention is his SQL RunAs Addendum to simplify SQL monitoring. Let’s break down a number of examples how the SCOM community has built packs to better monitoring, and how I believe the addendum packs bring IT Ninja lessons from Microsoft experts monitoring to your environment.
Why Addendum packs
Better monitoring from the experts, including customer examples for other ‘blind spots’ in monitoring. Blind spots consist of ‘not monitored’ pieces of infrastructure, from simply an event, ping, service, tcp port check, process, web site, scripted workflow, with the purpose to identify a problem.
The goal of monitoring is to:
Identify, self-heal, automatically run recovery or diagnostic workflows alert when manual intervention is required. Doesn’t matter what tool you use, they all do some portion of these steps.
The addendum packs do these things, adding a few differentiators.
Auto closure daily scripts (close rules/monitors)
Auto reports of problems (M-F 0600-0700 local, reflecting last 24-72 hours of open/closed alerts)
Employ count logic (x in y time)
Self-heal monitors with no new events
Adjust alert severities to health model
where critical (red) = outage, warning (yellow) = issue, informational reports or FYI’s
Capable of updating alerts (status, owner, ticketID+)
Tasks to run workflows on-demand
Recovery tasks – (i.e. service restart automation or TopProcess, Logical disk cleanup, MECM Client cache clean )
Integrate additional monitoring (like DFS replication queue script/alerts)
Synthetic checks for DNS and web applications
Web Availability and Transactional monitoring, ADFS, CRL, PowerShell Invoke-WebRequest, and more
Security and Compliance checks
Imagine I forgot something capability wise.
Stay tuned, as this builds into an even better outcome, quality data into ‘a single pane of glass’ of multiple tools within PowerBI.
Detected malicious verification code error
Ever run into the ‘detected malicious verification code’ error while authoring? I ran into the malicious verification error authoring, and couldn’t find any content for this error while authoring a pack.
Watch your copy/paste’s with additional monitoring changes to prevent ‘detected malicious verification code’ errors
In my authoring example, I received the ‘detected malicious verification code error’ after adding Rules, Datasources, and WriteActions (including tasks). I was copying and pasting DataSources (DS) and WriteActions (WA), thought I had it all. Uploaded > got the error, and GRR! Hopefully this will help others authoring to know what to check to get the management pack uploaded.
Simply put – Watch out for typo’s to avoid ‘detected malicious verification code’ errors!
I stumbled across a few websites, but nothing really pointed out to what caused the ‘detected malicious verification code error’ when uploading a management pack. First, check monitor and rules to verify the DS/WA are called correctly (no errors in file names. Check the Tasks as well as DisplayStrings, to make sure everything matches.
Error Seen when uploading Management pack from SCOM Console GUI regarding ‘detected malicious verification code’ error
<ManagementPackNameHere> Reports could not be imported.
If any management packs in the Import list are dependent on this management
pack, the installation of the dependent management packs will fail.
Verification failed with 1 errors:
——————————————————-
Error 1:
Found error in
2|<ManagementPackNameHere>|1.0.0.6|<ManagementPackNameHere>|
| with message:
Detected malicious verification code when verifying element of type
Microsoft.EnterpriseManagement.Configuration.ManagementPackRule with inner
exception: System.Collections.Generic.KeyNotFoundException: The given key
was not present in the dictionary.
at System.ThrowHelper.ThrowKeyNotFoundException()
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at
Microsoft.EnterpriseManagement.Configuration.ManagementPackRule.VerifyDataTy
pes(Dictionary`2 moduletypes)
at
Microsoft.EnterpriseManagement.Configuration.ManagementPackRule.Verify(Verif
icationContext context)
at
Microsoft.EnterpriseManagement.Configuration.Verification.VerificationEngine
.VerifyCollectionItems(Object context)
Additional links
Caution using Tags/Notes extending classes
Please take ‘caution using Tags/Notes extending classes’. Please read below if you use Tags/Notes on SCOM classes. Ran across examples where SCOM Class Properties were used for tags that used the .Notes field on various classes, causing orphaned properties, NOT removed from OperationsManager database.
Background
The Microsoft.Windows.Computer Class (insert class here) is updated using Tim McFadden’s blog. This can cause issues with orphaned classes in the database because it is not currently handled as part of the stored procedure (i.e. the Notes property classes do not get marked for deletion).
First, identify which classes have Notes property. Start from Management Server (MS) via PowerShell. See attached TXT for additional examples to check and add/remove Notes Property on additional windows classes.
Set Notes property for Windows Operating System server
Second, we need to see how to set and clear the value, in order to clean up the Operations Manager database, to remove the orphaned instances. The example below sets the value for one (1) server to ‘Production’.
$WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
$WOS.'[System.ConfigItem].Notes’
$WOS.'[System.ConfigItem].Notes’.Value = “Production”
$WOS.Overwrite()
$WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
$WOS.'[System.ConfigItem].Notes’
Example Output
PS C:\Users\scomadmin> $WOS.'[System.ConfigItem].Notes’.Value = “Production”
PS C:\Users\scomadmin> $WOS.Overwrite()
PS C:\Users\scomadmin> $WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
PS C:\Users\scomadmin> $WOS.'[System.ConfigItem].Notes’
PropertyAccessRights : Unknown
Parent : Microsoft Windows Server 2016 Standard
Type : Notes
Value : Production
Id : 00000000-0000-0000-0000-000000000000
ManagementGroup : SCOM2016
ManagementGroupId : e39f5f53-9fbb-9d7f-4bfe-5f0324630ae5
Set Notes property to NULL
$WOS.'[System.ConfigItem].Notes’.Value = $null
$WOS.Overwrite()
$WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
Verify Notes value
$WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
$WOS.'[System.ConfigItem].Notes’
Example Output
PS C:\Users\scomadmin> $WOS = Get-SCOMClass -name Microsoft.Windows.OperatingSystem | get-SCOMClassInstance | where-object -property Path -eq “16db01.testlab.net”
PS C:\Users\scomadmin> $WOS.'[System.ConfigItem].Notes’
PropertyAccessRights : Unknown
Parent : Microsoft Windows Server 2016 Standard
Type : Notes
Value : (null)
Id : 00000000-0000-0000-0000-000000000000
ManagementGroup : SCOM2016
ManagementGroupId : e39f5f53-9fbb-9d7f-4bfe-5f0324630ae5
Have a happy Holiday!
Good luck, hopefully this scenario isn’t something that impacted the monitoring environment!
ADCS – Active Directory Certificate Services Addendum pack
Hello again, it’s time to talk about ADCS – Active Directory Certificate Services Addendum!
First, I’d like to call out Bob Williams and Vance Cozier for their help and expertise!
Background
ADCS is Active Directory Certificate Services, or what we would know as a Certificate Authority. The goal was to improve the pack, because the focus is on how important certificates are to a modern enterprise. Let’s begin the Active Directory Certificate Services Addendum pack review.
Collaboration
In this paragraph, let’s talk through the Certificate Services packs for 2016+, and how we as Microsoft consultants, and field engineers, recommend changes to the pack. First, for some background, the collaboration process gets a better result improving Microsoft products. Second, the collaboration result can vary. Third, collaboration input can be based on customer input, or field engineer experience. Most importantly, this is how we ‘would have liked’ the pack to work.
AD Certificate Services Monitoring
The Certificate services pack alerts on events/services. Therefore, the pack does NOT monitor the SCEP URL. For instance, a transaction web monitor was added. The collaboration effort was focused on improving the ADCS pack, resulting in the creation of the Active Directory Certificate Services Addendum and customizations packs.
Download File
Let’s delve into the download file
Review file contents
- Download.txt (in case you need to find it later!)
- Version.Info.txt (MP version history, what was added & when)
- XLS MP export of rules/monitors
- ADCS Addendum & Customizations packs
References
Configuring Certificate Services docs site
ADCS download
Management Pack wiki
SQL on Windows Addendum pack
It’s spring time; time to tune the SQL carb!
Carbs are way less easy to find these days, but I’ve been busy tuning the SQL agnostic pack (MSSQL on Windows).
Tuning the SQL Agnostic pack would be far less successful without expert help. My thanks to Brandon Pires – MCS SQL Consultant who helped provide a SQL DBA perspective. Brandon’s LinkedIn profile
Always grab an expert, and for SQL, it’s a DBA. If you’re new to SCOM, most product teams provide their management packs. SCOM PFE’s build addendum packs to improve a pack (from our perspective). Addendum packs make the a pack stronger, for an improved customer experience. I’m not complaining at what the pack delivers. The SQL Team is awesome for taking user feedback and making improvements quarterly!
Background:
Initially this journey started out with Tim McFadden disabling the duplicate rules/monitors in the SQL MP’s (here).
After talking with Tim and Kevin H, I set out to clean up the SQL version specific packs to remove bloat by creating the version specific OFF packs. The OFF packs disabled the plethora of SQL performance counters (see MP bloat blog here).
With the SQL Agnostic packs (thank God!), I wanted to deliver an addendum pack to tune the SQL alerts/health for what SQL PFE/Consultants recommended for an improved out of the box experience (OoBE).
MP Version history
v1.0.0.0 24 Feb 2020 Override to enable SQL Monitoring
v1.0.0.1 24 Feb 2020 Override pack cleanup to human readable format
v1.0.0.2 2 Mar 2020 Overrides for severities and SQL CPU samples
v1.0.0.3 2 Mar 2020 Overrides for SQL rules for warning
v1.0.0.4 4 Mar 2020 Completed overrides for SQL warning rules
v1.0.0.5 1 Apr 2020 Updated rules for backup failures when customer uses Netbackup vs. SQL agent/scheduled tasks
v1.0.0.6 9 Apr 2020 Created groups for seed discovery Test/Dev and Prod; excluded EXPRESS, disabled Securables monitor
v1.0.0.7 15 Apr 2020 Updated pack name to include ‘SQL Server’.
Updated AddendumGroupGUIDUpdate to include RegEx pattern replace
AddendumGroupGUIDUpdate will version pack to v1.0.0.7 for group GUID and regex changes
Please feel free to download the zip file, which includes the XLS for review of what was updated.
Additional References
The Agnostic OFF Pack to turn off the performance rules (found here)
The old SQL version specific OFF packs for the performance counters can be found here.
Updated Skype for Business 2015 Addendum pack
Continuing work with Nick Wood on the Skype pack for additional operational features.
Previously Blogged about this July 2018, and continue to make improvements
The TechNet gallery bundle is updated with new functionality.
Skype KHI addendum
Pack gathers the Skype KHI performance counters
Packets * Discards performance rules where greater than 100 discards are seen on NIC’s,
Monitoring Tab folder/performance view
Skype Custom Overrides
Includes common overrides for noisy monitors/rules.
Install SCVMM management packs from VMM Server
Time for some automation
Ever have to upgrade SCVMM packs every time a new Update Release (UR) comes out?
Copy the files off from the VMM server to your SCOM MS, install.
How long does that take?
Try this script out – assuming you have a login on the VMM Server
TechNet Gallery post here
# Set up some variables
$UR=”UR5″
$VMMServer = “16VMM01”
# Set up your path, this example is monadmin\backup
$date = Get-Date -UFormat “%Y-%m-%d”
# Set up backup path
$backupPath = “C:\monadmin\backup”
$backupDrive = “C:”
# Create some functions
Watch them roll, let PowerShell do your work!
UR6 packs
SCOM management packs backed up
Check out the SCOM Console Admin tab for updates!