Scripting SCOM Registry key tweaks

 

Time to tune!

 

 

Had some requests to script the registry tweaks for SCOM

 

Starting off with Holman’s blog entry …

 

TechNet Gallery download here

 

Save .txt file as .ps1

 

On SCOM Management server(s)

Close out any SCOM Console session (to prevent SDK errors)

Run as administrator in PowerShell window

Restart SCOM services

restart-service omsdk; restart-service healthservice; restart-service chost

Verify services running

get-service omsdk; get-service healthservice; get-service chost

Workflow Manager Addendum MP for SQL Aliases

 

A SQL Alias is kinda like wearing disguise glasses…

 

From a security perspective, you can make things difficult for attackers by specifying a SQL alias and different port for SQL.

 

 

 

Symptom – discovery fails for WFM pack

 

Trying to monitor and figure out what the real database name, instance, etc. can be a challenge.

A couple of years ago, I was able to find an example for one customer where the registry key shed light on the alias.

 

The workflow manager management pack has a DataSourceModuleType “Microsoft.WorkflowManager.Addendum.v1.WFCommandExecuterDataSource”, where this change successfully retrieved the sql server name.

This datasource uses the PowerShell script (WorkflowPSDiscovery.ps1)

 

This function was changed in one example

# Get computer name from splitted dataSource
function GetPrincipalName {
param(
$ADDomain,
$ss
)

#$ssWithoutPort = $ss[0].split(‘,’)
#if (-not $ssWithoutPort[0].Contains(‘.’))
#{
# $ssWithoutPort[0] = $ssWithoutPort[0] + “.” + $ADDomain.Name
#}
#$principalName = $ssWithoutPort[0]

$key = ‘HKLM:\SOFTWARE\Microsoft\MSSQLServer\Client\ConnectTo’
$sqlfromalias = (Get-ItemProperty -Path $key -Name $ss).$ss
$sqlserverstr = $sqlfromalias.Split(‘,’)
$sqlserver = $sqlserverstr[1]
$principalName = $sqlserver

return $principalName
}

 

 

Ran into this discovery issue a second time, and the function didn’t solve the failure.

Real quick – a shout out and my thanks to Chuck Hughes and Mike Sadoff, for their time and testing this more robust discovery method.

 

 

 

Added logic to fix the assumed InstanceName ($instname) – Most likely why my first function worked (configuration had default SQL instance name of MSSQLSERVER )

Added GetSqlAlias function to help decode the disguise

 

 

Gallery download here

 

Don’t forget to override the original workflow manager discovery!

Microsoft.WorkflowManager.v1.Addendum.WFPSDiscovery

Test fire any event on any server from any application

Golden Oldies – always popular (tools vs music)

Old Holman blog that’s still relevant, even more powerful than EventLog Explorer

Basically anyone who wants to test fire events off a SCOM MP should use this tool.

Event Create, write-eventlog all have limitations (certain event sources that can be used to create events, or event ID number limitations)

First, download the 2007 R2 Admin ResKit here

MomTeam blog reference

Double click the downloaded MSI

I prefer to move extracted files under my SCOM tools/Management pack directory structure under MonAdmin (Monitoring Admin)

Copy extracted files to gold depot

Move to gold depot – SCOM \ tools \ <toolname here>

Go into the MPEventAnalyzer directory

Run the exe

MP Event Analyzer

Click on Investigate Event Sources Tab (bottom middle)

Don’t forget you can use the search bar (where I typed apm)

For my example, double click on APM Agent

Search Events on right hand pane

Check checkbox to select the 1319 APM event for configuration error (right hand pane)

Click the ‘Add selected events to execution list’

Once event verified in bottom box, click the green box to fire selected event(s)

Verify event in Event Viewer

Validate Management Pack

Stay tuned… this did not complete the validation process.

Re-learn an old but still relevant tool – EventLog Explorer

 

Sometimes we forget about tools that can make things easier.

 

Time to talk about EventLog Explorer.

 

Need to repro and test events for an installed program, to see what SCOM will handle?

Read this old mom team blog, courtesy of Kevin Holman blog

 

 

I wanted to try it to test fire some events, had a use case where we needed to test Skype events from the SCOM MP

 

Testing on my SCOM 2016 Management server

 

Download file, run EventLog Explorer

The Paste icon next to the X is ‘Add to Execution List’ and fills out the bottom pane

The Green Arrow is ‘go’ or execute (similar to PowerShell ISE)

 

Navigate through the Event Log and Event Source on the left hand pane

Mark events with the checkbox  

 

Add to Execution

 

Verify events added to bottom pane

(see my test yesterday for fired, and not fired events from today)

 

 

 

Click Green box with white arrow to fire events, and check Event Viewer

 

 

Yesterday’s test

 

 

 

Today’s test

 

 

Verify alerting occurred as expected!

Adding Management Solutions in Azure

Decoder ring applies!

 

OMS is Log Analytics is Azure Management Solutions.

 

 

 

Do you want to add solutions to your Azure subscription?

Pre-packaged visuals and insights on your data, whether azure or hybrid.

 

 

 

Adding Management Solutions

Login to the Azure Portal

Click on All Services

Type ‘solutions’, hit enter

Click star icon to favorite Solutions

 

 

Drag Solutions higher in your preferences (wasn’t in above screenshot)

 

 

Click Solutions

 

 

 

 

Click + to Add

Click on Security and Compliance

 

 

Click Create

 

 

Don’t forget solutions require MMA agents connected to a workspace to render any data/insights!

 

 

 

 

References

The Docs article lists how to use the management solutions

 

MMA Agent and SCOM Agent version numbers


 

FYI – Updated 24 June 2022

 

What are the MMA Agent and SCOM Agent version numbers?

This idea sprung from a discussion with Sr. PFE Brian Barrington, and it got me wondering…See below for more details on OMS/MMA, and SCOM agent versions, as well as how to verify agent from PowerShell.

 

 

FYI – If you’re running a SCOM agent, 2016 or above, various Log Analytics solutions may have pre-reqs.

The Content Dev team under Brian Wren added this to the docs.microsoft.com site

SCOM 2022     https://docs.microsoft.com/en-us/system-center/scom/release-build-versions?view=sc-om-2022

SCOM 2019     https://docs.microsoft.com/en-us/system-center/scom/release-build-versions?view=sc-om-2019

SCOM 2016      https://docs.microsoft.com/en-us/system-center/scom/release-build-versions?view=sc-om-2016

 

 

Azure Monitor Agent

AMA (Azure Monitor Agent)/ALA/OMS/MMA Agent can run on Windows/Linux operations systems.  Name has changed over the years, where AMA (Azure Monitor Agent) will be the name going forward for the cloud based offer.  See docs article here.

This also has been updated on the Docs site

https://docs.microsoft.com/en-us/azure/azure-monitor/agents/azure-monitor-agent-extension-versions

Download installer files here

Review what operating systems are covered here

Previously known as Windows OMS/ALA/MMA agent

Unfortunately, there’s no github repo that I’ve found.

Examples:

As of 6 Sep 2018, MMA agent = 8.0.11103.0

As of 17 Oct 2018, MMA agent = 8.0.11136.0

Skipping forward to 2020, the MMA agent is 10.20.18040.0

[!WARNING] The Log Analytics agents are on a deprecation path and will no longer be supported after August 31, 2024.

 

 

OMS Gateway

Older product published in 2016 – Download link here

OMS Gateway requires Microsoft Monitoring Agent (MMA)

(agent version – 8.0.10900.0 or later)

Simple English, that means SCOM2016 RTM agent or above

 

 

 

OMSAgent for xPlat

OMS-Agent-for-Linux

(Linux/Universal Linux)

Sep 16, 2021      OMSAgent_v1.13.40-0
Mar 08, 2021      OMSAgent_v1.13.35-0
Nov 16, 2020      OMSAgent_v1.13.33-0
Support for Red Hat Enterprise Linux 8, CentOS 8, Oracle 8, Ubuntu 20.04, SLES…
Nov 14, 2019       OMSAgent_v1.12.15-0
Jun 17, 2019      OMSAgent_v1.11.0-9
Apr 23, 2019      OMSAgent_v1.10.0-1
Feb 12, 2019      OMSAgent_v1.9.0-0
Nov 05, 2018     OMSAgent_v1.8.1.256
Oct 30, 2018      OMSAgent_1.8.0-256
Sep 03, 2018      OMSAgent_v1.6.1.3

 

 

Windows SCOM Agent Version numbers 

SCOM2016 

Build Number KB Release Date Description Step-by-Step
8.0.10918.0 Evaluate Oct 2016 SCOM 2016 RTM Link
8.0.10931.0 KB3190029 Feb 2017 SCOM 2016 Update Rollup 1 Link
8.0.10949.0 KB3209591 March 2017 SCOM 2016 Update Rollup 2 Link
8.0.10970.0 KB4016126 May 2017 SCOM 2016 Update Rollup 3 Link
8.0.10977.0 KB4024941 Oct 2017 SCOM 2016 Update Rollup 4 Link
8.0.10990.0 KB4090987 April 2018 SCOM 2016 Update Rollup 5 None
8.0.11004.0 KB4459897 Oct 2018 SCOM 2016 Update Rollup 6 Link
8.0.11025.0 KB4492182 April 2019 SCOM 2016 Update Rollup 7 Link
8.0.11037.0 KB4514877 Sept 2019 SCOM 2016 Update Rollup 8 Link
8.0.11049.0 KB4546986 April 2020 SCOM 2016 Update Rollup 9 Link
8.0.11000.0 KB4580254 Dec 2020 SCOM 2016 Update Rollup 10 Link
7.2.12335.0 KB5006871 Oct 2021 SCOM 2016 Update Rollup 10 Hotfix Link

SCOM1801
8.0.13053.0 RTM

SCOM1807

8.0.13067.0      General Availability release

 

SCOM2019

Build Number KB Release Date Description Step-by-Step
10.19.10050.0 Evaluate March 2019 SCOM 2019 RTM Link
10.19.10311.0 KB4533415 Feb 2020 SCOM 2019 Update Rollup 1 Link
10.19.10407.0 KB4558752 Sept 2020 SCOM 2019 Update Rollup 2 Link
10.19.10505.0 KB4594078 March 2021 SCOM 2019 UR3 Link
10.19.10550.0 KB5006871 Oct 2021 SCOM 2019 UR3 Hotfix Link

 

 

  • @Larry LeBlanc – thank you for the SCOM Agent version updates!

 

Verify what version is installed

Via SCOM – use Holman’s Agent Version Addendum management pack

 

If you don’t have SCOM

From PowerShell

$Agent = get-itemproperty -path “HKLM:\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup”

$Agent.CurrentVersion

 

 

 

 

 

 

Resources

SCOM Agent Version Addendum pack https://kevinholman.com/2017/02/26/scom-agent-version-addendum-management-pack/

SCOM Agent build numbers https://social.technet.microsoft.com/wiki/contents/articles/34312.system-center-operation-manager-momscom-list-of-build-numbers.aspx

Linux Agent can be downloaded from GitHub – github.com/Microsoft/OMS-Agent-for-Linux

Installing and configuring the MMA agent via PowerShell

 

GUI install option, see blog

 

 

Pre-reqs to build out an install script/package

MMA agent executable

Workspace ID

Workspace Primary Key

 

 

Download MMA agent

Click on Windows Servers from Connected Sources to download Windows Agent

Click on Linux Servers from Connected Sources to download Linux Agent

 

 

 

 

Obtain WorkspaceID

From the Azure Portal (https://portal.azure.com)

Click on Log Analytics, <your subscription >

Click on Advanced Settings

My view defaulted to Connected Sources > Windows Servers

 

Save the workspace ID and workspace key to notepad/OneNote for later

 

 

 

 

 

Build out command line for setup file

(optionally to include in Application Deployment package)

 

Grab pre-reqs above: (saved from above to build the command line)

Exe/msi file

Workspace ID

Workspace key

 

The setup.exe or MSI command line parameters to pass are:

MMA-specific options Notes
NOAPM=1 Optional parameter. Installs the agent without .NET Application Performance Monitoring.
ADD_OPINSIGHTS_WORKSPACE 1 = Configure the agent to report to a workspace
OPINSIGHTS_WORKSPACE_ID Workspace Id (guid) for the workspace to add
OPINSIGHTS_WORKSPACE_KEY Workspace key used to initially authenticate with the workspace
OPINSIGHTS_WORKSPACE_AZURE_CLOUD_TYPE Specify the cloud environment where the workspace is located

0 = Azure commercial cloud (default)

1 = Azure Government

OPINSIGHTS_PROXY_URL URI for the proxy to use
OPINSIGHTS_PROXY_USERNAME Username to access an authenticated proxy
OPINSIGHTS_PROXY_PASSWORD Password to access an authenticated proxy

Example:

setup.exe /qn NOAPM=1 ADD_OPINSIGHTS_WORKSPACE=1 OPINSIGHTS_WORKSPACE_AZURE_CLOUD_TYPE=0 OPINSIGHTS_WORKSPACE_ID=<your workspace id> OPINSIGHTS_WORKSPACE_KEY=<your workspace key> AcceptEndUserLicenseAgreement=1

 

 

 

Other helpful links

Docs site https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-quick-collect-windows-computer

Daniel Orneling Blog https://blog.orneling.se/2017/01/installing-oms-agent-with-powershell/

TechNet gallery https://gallery.technet.microsoft.com/scriptcenter/Install-OMS-Agent-with-2c9c99ab

Service Map SCOM pack configuration errors

Look for 6400 Event ID’s in the Operations Manager log on the management server if you do not have the correct information

 

Event ID 6400 in Operations Manager log helps show what’s missing with Azure AD error events

 

Follow steps outlined in the ‘Set up Azure Service Principal’ blog here

 

 

Sample 6400 event

 

Message: Microsoft.IdentityModel.Clients.ActiveDirectory.AdalServiceException: AADSTS90002: Tenant XXXXXXXXX not found.

This may happen if there are no active subscriptions for the tenant. Check with your subscription administrator.

Trace ID: 89abf27f-4884-4191-b577-de2fce100600

Correlation ID: c8a2470e-2383-4325-b91f-86b5e20ade57

Timestamp: 2018-08-06 20:34:49Z —> System.Net.WebException: The remote server returned an error: (400) Bad Request.

at System.Net.HttpWebRequest.GetResponse()

at Microsoft.IdentityModel.Clients.ActiveDirectory.HttpWebRequestWrapper.<GetResponseSyncOrAsync>d__2.MoveNext()

— End of stack trace from previous location where exception was thrown —

at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

at Microsoft.IdentityModel.Clients.ActiveDirectory.HttpHelper.<SendPostRequestAndDeserializeJsonResponseAsync>d__0`1.MoveNext()

— End of inner exception stack trace —

at Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext.RunAsyncTask[T](Task`1 task)

at Microsoft.SystemCenter.ServiceMap.REST.Credentials.AdCredentials.GetToken()

at Microsoft.SystemCenter.ServiceMap.UI.SubscriptionData.TestConnection()

ErrorCode: invalid_request

StatusCode: 400

 

Inner Exception

Message: The remote server returned an error: (400) Bad Request.

Response URI: https://login.windows.net/XXXXXXXXX/oauth2/token

Headers:

Pragma: no-cache

Strict-Transport-Security: max-age=31536000; includeSubDomains

X-Content-Type-Options: nosniff

client-request-id: c8a2470e-2383-4325-b91f-86b5e20ade57

x-ms-request-id: 89abf27f-4884-4191-b577-de2fce100600

x-ms-clitelem: 1,90002,0,,

Cache-Control: no-cache, no-store

Content-Type: application/json; charset=utf-8

Expires: -1

P3P: CP=”DSP CUR OTPi IND OTRi ONL FIN”

Set-Cookie: esctx=AQABAAAAAADXzZ3ifr-GRbDT45zNSEFEzFrPhp_xcoXIlYw2iOqAFXkz7NO-Hm1hJdVAn6298A0ylDD5VvX2VosFiRVxTDzmRz24sbVUbhiTuyHJsmeIkR47y1MU3SafDlFp6xPo91BwZhRqoDPtP6YTBi5D6mHGqy2lkSAEVQtg9D4lsWTmKipm9iLaB2twBZcYR0VkDhIgAA; domain=.login.windows.net; path=/; secure; HttpOnly,x-ms-gateway-slice=004; path=/; secure; HttpOnly,stsservicecookie=ests; path=/; secure; HttpOnly

Server: Microsoft-IIS/10.0

Date: Mon, 06 Aug 2018 20:34:48 GMT

Content-Length: 508

MMA Agent, cross platform, and Azure

Things that make you go hmmm….

 

 

Ran across a scenario where we were trying to connect Azure Cross-platform (Linux) VM’s and MMA/SCOM agents to SCOM management group.

 

Management group was 2012R2, discovery wizard from SCOM console, failed to install agent, certificate errors.

 

Researching, found this article first

Windows Azure VM monitoring blog

There’s a version history for the Azure Monitor VM extension here

Applies:

SCOM2012R2 after UR12 or SCOM 2016 UR2+ deprecated the SHA1 certificate

 

Deprecating SHA1 certificates
Tech Community blog

 

Product team nicely published a TechNet gallery script to help!

Gallery download – Script to update SHA1 certificates to SHA256 on cross-platform agents – SCOM

TechNet Gallery Download
https://gallery.technet.microsoft.com/scriptcenter/Script-to-update-SHA1-8a30c5ef

 

 

Service Map SCOM pack errors and events

Running Service Map SCOM management pack and getting errors?

 

 

 

Gotta love holidays

Good family time

Not at work if we’re lucky.

When you come back, do you have to go investigate some new/weird errors?

 

 

This was one of those holidays for me 🙂

 

 

 

Figured I’d document SCOM errors, indicate what Event Sources, event ID ranges that aid troubleshooting.

 

Event Source = MS ServiceMap OMS

Event ID range = 46649-46652

 

Long story short, the root cause for my case, my azure workspace was disabled (fun part with a lab is trying to see how much you can do before it disables!)

 

Digging in my inbox, found this over the weekend

Email subject: Your services were disabled because you reached your spending limit

 

 

SCOM Alerts seen:

 

Service Map Unknown Exception

 

SCOM Console alert example

 

Cause:    May point to Network Connectivity, proxy, or subscription disabled

REST request failed, so did name resolution (may indicate DNS issues)

 

Rule details

Operations Manager Event Log

Event Source = MS ServiceMap OMS

Event ID 46651

 

Operations Manager Event log

 

 

 

No Machines Alert

Rule Name = Microsoft System Center ServiceMap No Machines Alert

Event Source = MS ServiceMap OMS

Event ID = 46652

Event ID also seen is 46649 – Error in getting machine details

 

SCOM Console alert

 

 

 

 

Event ID 46649