Troubleshooting Service Map pack

Updated 14 Mar 2019

If you get these exceptions like me, the issue has been raised, with a deliverable targeted for SCOM2019UR1.

Disable the rule to reduce noise.

Are you using Service Map Management pack, and getting errors?

This alert is based on the 46651/46652 event ID in the Operations Manager event log

From SCOM Console > Authoring Tab > Management Pack Objects > Rules

Search in ‘Look for:’ bar GenericException (yes no space in between)

Rule

Rule Details

To enable debug on the MS

For collecting logs, please do the following:

Create folders “c:\Debug\ext\”
Now, Wait for an hour(which is the default time interval set in the rule for running service map api).
You will see some log files created in that folder “ext”. Please share the same in email.

The file showed up after the alerts, and listed debug INFO and WARN lines, and the time stamps match up to the generic exception rules.

Stay tuned for more information, I have been trying to get more answers on the exception

{WARN} [12:35:20.966] [ScomUtils] failed to export XML for Management Pack: System.NullReferenceException: Object reference not set to an instance of an object.

at ScomBridge.ScomUtils.WritePackXmlToFile(ManagementPack pack, String filename)

XML for Product or Company Knowledge

Digging in the archives…

From a discussion with some PFE’s – the question was ‘how do I create knowledge for a monitor/rule?’

Tyson Paul pointed out the system Center Wiki ‘Knowledge Article authoring’

When you create a knowledge article in an MP (let’s not even go into the console GUI! )

If the Knowledge Article references a sealed workflow (does it reference a sealed pack)

It’s Company Knowledge

Example

If the Knowledge Article references a sealed monitor, it will show up under the ‘Company Knowledge’ tab

XML example from Skype Addendum pack on TechNet Gallery

<KnowledgeArticles>
<KnowledgeArticle ElementID=”ML2MC!Microsoft.LS.2015.Monitoring.Internal.Health.DiscoveryRunner” Visible=”true”>
<MamlContent>
<maml:section xmlns:maml=”http://schemas.microsoft.com/maml/2004/10″>
<maml:title>Summary</maml:title>
<maml:para>Any added Skype servers will not be discovered in SCOM.</maml:para>
</maml:section>
<maml:section xmlns:maml=”http://schemas.microsoft.com/maml/2004/10″>
<maml:title>Causes</maml:title>
<maml:para>Discovery Failed. An internal exception has occurred during discovery.</maml:para>
</maml:section>
<maml:section xmlns:maml=”http://schemas.microsoft.com/maml/2004/10″>
<maml:title>Resolutions</maml:title>
<maml:para>Fix permission issues in alert.</maml:para>
<maml:para>Skype PowerShell module may not be installed.</maml:para>
<maml:para>Import-Module SkypeForBusiness</maml:para>
</maml:section>
</MamlContent>
</KnowledgeArticle>

</KnowledgeArticles>

If the Knowledge Article is referenced in a sealed pack, OR an UNsealed pack has a rule/monitor in the same unsealed pack)

It’s Product Knowledge

Sealed pack example

Unsealed pack Example

Skype for Business 2015 (premise) Addendum MP

Ever try to figure out a Skype alert, for which server in the pool(s) is failing?

While maybe not the clearest to find root cause, the Skype pack brings a bunch of functionality, including synthetic transactions.

I was lucky enough to collaborate with Nick Wood, Skype PFE, to help provide more detail, troubleshooting, impact on what is critical versus warning.

What the addendum pack brings

Do you think 656 monitors can all be critical?

Sets up service restart recovery tasks for all Skype services

Company Knowledge tab for troubleshooting/user impact

Gallery Download

Here is a visual of our Skype efforts for integrating troubleshooting details into SCOM console.

NOTE: Company Knowledge tab would be accessible from the alert as well

Company Knowledge

SCOM Console, Authoring tab, Dispatcher Queue monitor

Highlight monitor, right click, choose properties

Click on ‘Company Knowledge’ tab

Incorporated the XLS into SCOM under Company Knowledge for additional information on user impact, causes, and troubleshooting (under resolutions)

VSAE support for 2017

VSAE support for VS2017 has been released!

https://systemcenterom.uservoice.com/forums/293064-general-operations-manager-feedback/suggestions/18560653-updated-vsae-to-support-visual-studio-2017

VSAE download https://www.microsoft.com/en-us/download/details.aspx?id=30169

MomTeam Blog https://techcommunity.microsoft.com/t5/System-Center-Blog/System-Center-Visual-Studio-Authoring-Extension-VSAE-support-for/ba-p/351872?search-action-id=139696432720&search-result-uid=351872/

SQL native client for TLS1.2

Ever try to talk to someone when language is a barrier?

Sure, we can run an app, or search our phrase to pronounce, but it’s so much better when we can communicate seamlessly.

Post TLS1.2 for SCOM

Let’s talk SQL

Part of TLS1.2 is updating SQL Native Client to talk using a secure client that uses TLS1.2

That means a different executable should be called.

Why is that important in SCOM?

Maybe you have management packs that connect to SQL or run external commands.

On MS, there are multiple clues for various errors on Management Packs that use SSL or talk to SQL via a non-TLS method. NOTE this may mean that the SQL DB that management pack is connecting to may need the same pre-req SQL updates to a TLS 1.2 enabled version.

Do you have custom SQL queries being run, CMDB get’s, OLE DB Data Source checks?
Any Event ID 1401 or 11854 events in the Operations Manager Event log?
1. These events identify management pack scripts creating SCHANNEL events
  a. Event ID 1401 event example

Cause

SQLOLEDB connection strings will cause 36871 Sytem Log events

Example (TLS1.0)
sConnectString = “PROVIDER=SQLOLEDB;DATA SOURCE=<databaseServerFQDN>;DATABASE=MSSQLSERVER;trusted_connection=yes”
SQLNCLI11 driver for TLS1.2 connection strings

Example (TLS1.2)
sConnectString = “Provider=SQLNCLI11;DATA SOURCE=<databaseServerFQDN>;DATABASE=MSSQLSERVER;trusted_connection=yes”

Identify
Look for management packs with SQLOLEDB as the Connect string to reduce 36871 SCHANNEL events

In Windows Explorer, use the Advanced Options dropdown to select File Contents
In the Search bar (top right), enter SQLOLEDB (example shows SQLNCLI11)
NOTE SQL Discovery group pack IS compliant

In Windows Explorer, use the Advanced Options dropdown to select File Contents
In the Search bar (top right), enter SQLNCLI11

Additional offenders
HP Topology MP
SQL 2005 discovery MP (discontinued)
SQL Addendum MP’s (will work to update these with Holman)
SharePoint Foundation server (v15.0.4557.1000)
PRE TLS Microsoft.SystemCenter.2007

Resolution
Unseal (if necessary), update connection string, and reimport management packs
If Sealed vendor MP, request new MP via support Incident (and/or UserVoice if Microsoft sourced pack)
If Vendor will not release MP’s, accept risk with the logged errors, update MP, or remove from SCOM

MPViewer reloaded

The previous post covered the MPViewer 2012 version here

Jan Van Meirvenne spent the time to update the functionality even further

MPViewer “2012 Reloaded (release 1)” http://scug.be/jan/2016/06/06/mp-viewer-2012reloaded/

Use MPViewer reloaded will allow you to

Open multiple management pack files (MP and MPB)
See Modules to view underlying scripts (always wondering how a monitor got its state or property bag info)
Load files from Management Group (typically this required command line or Silect MP Studio!)
OpenWith file association (load MPViewer when clicking on files in explorer)

Using MP Viewer to unseal or export MP to XLS or HTML

Use MPViewer and open the management pack files (MP and MPB)

Updated 14 Dec 2018

Thanks to Daniele Muscetta for converting this so many years ago!

MPViewer tool originally at this blog https://blogs.msdn.microsoft.com/dmuscett/2012/02/19/boriss-opsmgr-tools-updated/

MPViewer v2.3.3 added to TechNet Gallery for download here

Jan Van Meirvenne spent the time to update the functionality even further

MPViewer “2012 Reloaded (release 1)” http://scug.be/jan/2016/06/06/mp-viewer-2012reloaded/

Load Management pack

GUI

In MPViewer,

Click on File, Load Management Pack

The 2012 Reloaded MPViewer allows you to open multiple management packs, or from a Management group

Go to your directory where you saved the UNIX SCOM 2016 UR2 management packs

If necessary, change the dropdown to mpb

Command line options

.\MPViewer.exe –help

Example syntax above

Remember to encapsulate your paths with quotes to be successful!

Opens MP and saves as HTML

.\MPViewer.exe “MP Path and file name” “Outputfilename.html”

Opens MP and saves as XLS

.\MPViewer.exe “MP Path and file name” “Outputfilename.xls”

Best practice is to keep same naming convention – makes it easier to track down the original MP

Example export MP to XLS

.\MPViewer.exe “S:\MonAdmin\scom\Management packs\sql\v7.0.7.0\2008-2012\Microsoft.SQLServer.2012.Monitoring.mp” “S:\MonAdmin\scom\Management packs\sql\v7.0.7.0\2008-2012\Microsoft.SQLServer.2012.Monitoring.xls”

Output

PS C:\Users\scomadmin\desktop> .\MPViewer.exe “S:\MonAdmin\scom\Management packs\sql\v7.0.7.0\2008-2012\Microsoft.SQLServer.2012.Monitoring.mp” “S:\MonAdmin\scom\Management packs\sql\v7.0.7.0\2008-2012\Microsoft.SQLServer.2012.Monitoring.xls”

PS C:\Users\scomadmin\desktop> gci “S:\MonAdmin\scom\Management packs\sql\v7.0.7.0\2008-2012\*.xls”

Directory: S:\MonAdmin\scom\Management packs\sql\v7.0.7.0\2008-2012

Mode                LastWriteTime         Length Name
—-                ————-         —— —-
-a—-       12/13/2018   8:40 AM         527618 Microsoft.SQLServer.2012.Monitoring.xls

To unseal MP to view in Notepad++

Once MP is loaded

Click File

Click Unseal MP (my path defaults to desktop)

Open file in Notepad++ or XML Editor, or your favorite XML viewer of choice

To export a management pack to XLS

Once MP is loaded

Click File

Click Save to Excel (my path defaults to desktop)

Choose path

My personal preference is to append filename with XLS for visibility

Copy file to a machine with Excel installed

Open the XML file in Excel, and hide all tabs but the Monitor tabs, and rules tab

Turn on auto-filter, etc.

Sealing SCOM MP’s

Sealing MP’s

This is an updated version of Kevin Holman’s blog, and Jonathan Almquist’s blog for SCOM2012R2 and 2016

First why seal?

If you seal the MP – we will be able to use the classes/groups created for overrides in any other override MP.

Unsealed MP – any overrides you use for classes/groups will be forced into this same MP.

If you don’t have Visual Studio 2013 and above with VSAE, or have other requirements, you will need to download the SDK to get the SN.exe utility

Download SDK

Win2008 & R2 SDK No longer available for Download

Win2012 & R2 SDK Download

Win10 SDK Download https://developer.microsoft.com/en-US/windows/downloads/windows-10-sdk

Release blog https://blogs.windows.com/buildingapps/2017/05/11/windows-10-sdk-preview-build-16190-released/

NOTE Install path to go grab the sn.exe file

Install SDK

Copy file to the local machine

Open PowerShell window as administrator

cd $HOME/desktop

.\sdksetup.exe # .\winsdksetup.exe for Server 2016/win10

Verify SN.exe is found after SDK install completes

Server 2008 – sn.exe located in C:\Program Files\Microsoft SDKs\Windows\v6.1\Bin\x64

Server 2012 – sn.exe located in C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools

Server 2016 and Win 10 – sn.exe located in C:\Program Files (x86)\Microsoft SDKs\Windows\v8.0A\bin\NETFX 4.0 Tools

Create the MPSeal folders

Repository where you want to keep the MPSeal.exe, SNK files, and related sealed packs for any MP sealed
C:\monadmin\MPSeal
C:\monadmin\MPSeal\unsealed
C:\monadmin\MPSeal\sealed # Reference sealed MP’s
C:\monadmin\MPSeal\key
C:\monadmin\MPSeal\output

PowerShell as Admin commands to create repository

new-item -itemtype directory -path c:\monadmin\
new-item -itemtype directory -path c:\monadmin\MPSeal
new-item -itemtype directory -path c:\monadmin\MPSeal\unsealed
new-item -itemtype directory -path c:\monadmin\MPSeal\sealed
new-item -itemtype directory -path c:\monadmin\MPSeal\key
new-item -itemtype directory -path c:\monadmin\MPSeal\output

Copy MPSeal utility from Support directory on SCOM ISO

On ISO, copy mpseal* from ISO SupportTools\AMD64 directory to c:\monadmin\MPSeal

Let’s get the Key file generated and start sealing MP’s!

Create SNK files
Note SN.exe only needs to be run once to create the SNK file
***Critical note – you need to keep a backup of this key… because it will be required for making updates to this MP in the future, re-sealing, and keeping the ability to upgrade the existing MP in production.

sn -k <yourDomainNameHere>.snk

Sample syntax from win2k8 server

Copy this SNK file to c:\monadmin\MPSeal\key

Copy Referenced MP’s
This is a good opportunity to add the MP’s referenced in the ISO, UR updates, and/or RTM folder when installing SCOM, Unix MP’s, etc.

Copy sealed MP’s to c:\monadmin\MPSeal\sealed

Seal MP

MPSeal.exe c:\monadmin\mpseal\unsealed\<mpNameHere>.xml /I “c:\monadmin\mpseal\sealed” /Keyfile “c:\monadmin\mpseal\key\PairKey.snk” /Company “CompanyName” /Outdir “c:\monadmin\mpseal\output”

References
How to Seal MP https://docs.microsoft.com/en-us/previous-versions/system-center/system-center-2012-R2/hh457550(v=sc.12)

Clarification on Registry Key discoveries

Ran across this in my travels, difficulty getting a monitor to work properly

To clarify some of the registry MP fragments, make sure you follow the whole path

This post is to help with using the Monitor.RegistryValue.Exists.mpx fragment

Example – Verify Registry Key under TestService

This is an excerpt from the MP Fragment header

%%
Description:
This fragment includes a Monitor which checks for the existence of a registry VALUE
RegValuePath – needs to be in the format of “SOFTWARE\Microsoft\CCM\HttpPort” or “SYSTEM\CurrentControlSet\Services\CcmExec\Start” as HKLM is assumed
RegValueName – needs to be the actual Reg VALUE name or your description of it (NO SPACES or special characters allowed) such as “HttpPort”
Version: 1.1
LastModified: 29-May-2017
%%

In the MP Fragment, you substitute the variables

<AttributeName>##RegValueName##</AttributeName>
<Path>##RegValuePath##</Path>

<AttributeName>ObjectName</AttributeName>
<Path>SYSTEM\SysInfo\AppName</Path>

Registry Key = HKLM\SYSTEM\CurrentControlSet\Services\HealthService\Test

Fragment variable (##RegValueName##) = SYSTEM\CurrentControlSet\Services\HealthService\Test

AttributeName or ##RegValueName## is simply whatever you want to call the attribute

Simply the name of the Registry value for my example is Test

Substitute ##RegValueName## for Test

If you’re testing in the lab, decrease frequency so you don’t have to wait as long

Remember to increase the frequency when you’re done

Upload MP (don’t forget to version your pack!)

Watch Health Explorer and test away adding or removing your key

Helpful testing tips to add a key to the registry and flip the health

reg add “HKLM\System\CurrentControlSet\Services\TestService” /v “Test” /t REG_SZ /d Test

reg delete “HKLM\System\CurrentControlSet\Services\TestService” /v “Test”

PowerShell Monitor Fragment with Run As

Stop!

Ever need to run a PowerShell command (or script) as a specific ID?

Maybe you need to know when the command fails to catch degraded application health?

Let’s work into the scenario with a MP Fragment

Download the latest fragments here