Tag: scom agent

NOT monitored servers

Ever run into NOT monitored servers?
NOT monitored servers
NOT monitored servers
Unsure your experience, but recently, I’ve run across multiple occurrences where servers show up as ‘not monitored.  As a result, I’ve found three distinct sets of issues that might cause ‘not monitored’ status –
1) Orphaned Agent blog
This scenario usually entails deleting server from Managed Agents view in Administration tab, where alerts or other details may still exist.  The procedure leverages Holman’s orphaned agent blog (tried and true) post from years back to aid cleanup.
2) Do you have packs or connectors extending classes?
3) Rebuilding a server with the same name is a common server occurrence
Related to 1, Holman’s orphaned agent blog to be used to cleanup.
First, let’s test in the Lab.  Second, let’s talk about the express lane ‘easy button’.  Begin by deleting the Windows Computer orphaned object GUID.  Process the Windows Computer object (bottom), followed by the top two (2) are HealthService, and HealthServiceWatcher object properties (see three items highlighted).
16db02 properties
16db02 properties
Second piece, marking the Windows Computer GUID for deletion (IsDeleted=1) cleans up nearly ALL properties.  See the progress below, how this slight change makes short order of orphaned properties for ‘server’.
Windows Computer object marks all but SCVMM for deletion
Windows Computer object marks all but SCVMM for deletion
Third HealthService & SCVMM objects require manual deletion per GUID.
Note first screenshot shows health service properties marked ‘IsDeleted’ = 1 after manually processing each GUID.
HealthService marked for deletion
HealthService marked for deletion
Fourth, screenshot shows there the HealthServiceWatcher property is marked for deletion (IsDeleted=1)
HealthServiceWatcher marked for deletion
HealthServiceWatcher marked for deletion
If you have SCVMM, you will need to repeat for each of the SCVMM properties to clear out the orphans in the DB.
Why – the issue:
Typically, when servers are reimaged, i.e. NOT deleted from SCOM, there are two+ healthservice, HealthServiceWatcher, Windows Computer properties created for each image of example server.   Additional properties may show duplicated for any class discoveries that are common to the old and new image.
NOTE: Deleting the current agent may clean up objects for that instance of the discovered server, but NOT the old server image.

Configure SCOM agent from PowerShell

Configure SCOM agent from PowerShell
Configure SCOM agent from PowerShell

 

Ever have a need to configure a SCOM agent outside the SCOM Console GUI using SCOM.Management pack?   Tyson’s ‘SCOM Helper PowerShell module’ on monitoringguys.com may assist if you can update PowerShell and SnapIn’s.  If Tyson’s solution may not be an applicable solution in a locked down environment, the blog is a PowerShell script to manage agent configuration.

 

A shout out to Aris for his watchful eye testing and playing with this!

 

Configure SCOM agent from PowerShell

GitHub Download  https://github.com/theKevinJustin/SCOMAgentInstallApplication/

First, let’s begin the conversation.  Leverage the PowerShell script to configure SCOM/MOM agent per domain(s).  Whether DMZ forest(s), or multiple forests with SCOM gateways, the script will enable configuration management.  Alternatively, add script as a compliance check to the Configuration Management tool, or as a post install step to configure SCOM Agent (MomAgent.MSI) via application or package.  Third, utilize the PowerShell script where agent upgrade is a pre-req to resolve ‘a really old SCOM agent version’.   Lastly, the script also tests network connectivity, and verifies if configuration is requested/loaded.

 

It all starts with this object from the command:

$SCOMAgent = New-Object -ComObject AgentConfigManager.MgmtSvcCfg

 

Use the PowerShell script to:

Clean up extraneous management groups

Determine domain

Check network connectivity via test-netconnection

Reset healthservice to expedite agent configuration

 

 

Background

Because Kevin Holman blogged this almost 10 years ago in VB, and I’m pretty rusty in VB these days.  Check out Kevin’s blog titled ‘Using the agent scripting objects on a SCOM 2012 agent‘.

 

Documentation

Download https://github.com/theKevinJustin/SCOMAgentInstallApplication/

Kevin Holman’s blog for VB commands

Using the agent scripting objects on a SCOM 2012 agent

The agent configuration objects in MSDN http://msdn.microsoft.com/en-us/library/hh328967.aspx

 

SCOM agent application

Stop using the SCOM console to push agents. Move this to Endpoint Manager as application/package and task sequence to install and configure on every server built.
Stop using the SCOM console to push agents. Move this to Endpoint Manager as application/package and task sequence to install and configure on every server built.

Shout out to Neal Smith, for his help simplifying the ‘SCOM agent application’ install per ConfigMgr/MECM best practice!  Stop using the SCOM console to push agents.  Move this to Endpoint Manager as application/package and task sequence to install and configure on every server built. Need a MECM package (Application) for the SCOM agent.  Leverage a best practice, be more secure, include in task sequence, and automate manual install.

 

Easy button wrapper

Why?  Helpdesk and server admin teams don’t have access to Tier0 devices.  After no access, the Manual process (instruction steps get missed), then server is not monitored, becoming an outage resolution task/follow-up.

 

Use the below thread = ‘easy button’ to package SCOM agent when MECM administrator has availability to add SCOM agent to SCCM task sequence.

 

MECM/SCCM Application/package script :

Using Powershell.exe:
###############Startscript
“SCOM Super Installer”
start-transcript -path “c:\windows\ccm\logs\SCOMSUPERINSTALLER.log”

##ONLY THIS SECTION NEEDS TO BE EDITED, replace different domain FQDNS and the gateway/management server(s), management groups

# Leverage find/replace for the ##something## variables

# Provide SCOM Gateway or SCOM management server MS for $SCOMGATEWAYFQDN variable.

# Include SCOM Mgmt Group Name for ##SCOMMGMTGroupName##

Switch ($domain) {
“##DomainFQDN1##” {$SCOMManagementGroup=”##SCOMMGMTGroupName##”;$SCOMGATEWAYFQDN=”##SCOMServerName##”}
“##DomainFQDN2##” {$SCOMManagementGroup=”##SCOMMGMTGroupName##”;$SCOMGATEWAYFQDN=”##SCOMServerName##”}
}
########ONLY THIS SECTION NEEDS TO BE EDITED

“Install”
$commonArgs=”.\MOMAgent.msi /qn USE_SETTINGS_FROM_AD=0 USE_MANUALLY_SPECIFIED_SETTINGS=1 SECURE_PORT=5723 ACTIONS_USE_COMPUTER_ACCOUNT=1 NOAPM=1 AcceptEndUserLicenseAgreement=1 MANAGEMENT_SERVER_DNS=$SCOMGATEWAYFQDN MANAGEMENT_GROUP=$SCOMManagementGroup”

start-process MSIEXEC.exe -ArgumentList $commonArgs -wait -nonewwindow
stop-transcript
###############Endscript

 

Lastly, after SCOM agent added to Endpoint Manager, monitoring new servers should be a no-brainer.  One less manual step having ‘SCOM agent application’ as part of the task sequence.

Enjoy!

 

Documentation

SCOM2022 install agent manually, 2019 here

Don’t forget python as pre-req for agent install

Grocery List, items to get and notes
Grocery List

 

Hey guys, don’t forget python as pre-req for agent install!  Came across this again, where the docs site doesn’t mention python-ctypes as pre-req for agent install.  Let’s flip to GitHub for the agent.  GitHub lists the python pre-req here.  Otherwise, it’s Openssl 1.1.0 is only supported on x86_64 platforms (64-bit).

 

 

Let’s begin by starting with a Linux server.  I’ve used Ubuntu in my lab, specifically, Ubuntu v16.04.

Login and check if you have latest, or have the package installed (Debian Linux)

Regular user run   ‘sudo apt-get install python-ctypes‘

Super user/root     ‘apt-get install python-ctypes‘

 

screen capture of python install

 

As of 1 July, v1.13.7.0 is current (latest) 64 bit OMS for Linux agent released.

 

 

 

 

 

References

GitHub link https://github.com/Microsoft/OMS-Agent-for-Linux

GitHub Agent Download (AzMon/ALA/OMS/SCOM agent for Linux ) https://github.com/microsoft/OMS-Agent-for-Linux/releases/download/OMSAgent_v1.13.7-0/omsagent-1.13.7-0.universal.x64.sh

Python requirements https://github.com/Microsoft/OMS-Agent-for-Linux#python-requrements

Install guide https://github.com/Microsoft/OMS-Agent-for-Linux#azure-install-guide

Need to find the command UNIX pack runs for perf counter

Magnifying Glass

 

 

Have you ever needed to find the command UNIX pack runs for perf counter?   Say the processor time value doesn’t match what the Unix admin may be saying SCOM is showing.

 

Many times you can look at the SCOM management pack, and those commands trace back to the UNIX library.

 

Background:  The SCOM management server runs many of the cross-plat/xplat workflows to the UNIX agent through WinRM.

 

Agenda
  1. Unseal SCOM UNIX management pack to obtain URI
  2. Understand command line options from UNIX/Linux side, and how to view the output
  3. Enumerate command line
  4. Test Command line from SCOM MS

 

 

 

Unseal SCOM UNIX management pack

The screenshot below is unsealing the Solaris10 pack to XML, and then viewing/searching to show the processor reference.

Solaris 10 processor rules

NOTE that’s a URI, not a script

 

 

How UNIX admin may supply processor output

Example – Unix admin typically uses vmstat or iostat.

 

The screenshot uses ‘vmstat 2 10‘ – a snapshot every 2 second intervals, 10 times

vmstat output

 

We can discuss the vmstat output, but it shows way more than just processor (ready queue, swap, user, system, and cpu %) to help figure out which operating system component is the problem.

 

 

Enumerate command line test

How do we test the command line syntax, to verify what SCOM pulls when running the rule?

For example, we need to make the URI actionable from the management pack.  What is needed to make a usable command?

 

Grab the URI from the pack –

http://schemas.microsoft.com/wbem/wscim/1/cim-schema/2/SCX_ProcessorStatisticalInformation?__cimnamespace=root/scx

 

Because we know the URI, we now build out the syntax with WinRM

winrm enumerate http://schemas.microsoft.com/wbem/wscim/1/cim-schema/2/SCX_ProcessorStatisticalInformation?__cimnamespace=root/scx -auth:basic -remote:https://<servername>:1270 -username:<scomID, not necessarily root> -skipCACheck -skipCNCheck -skiprevocationcheck –encoding:utf-8

 

 

Test WinRM command from SCOM MS

For instance, we want to test the WinRM command from the MS to the UNIX server

winrm enumerate http://schemas.microsoft.com/wbem/wscim/1/cim-schema/2/SCX_ProcessorStatisticalInformation?__cimnamespace=root/scx -auth:basic -remote:https://ubuntu:1270 -username:scom -skipCACheck -skipCNCheck -skiprevocationcheck –encoding:utf-8

 

Example output

SCX_ProcessorStatisticalInformation
InstanceID = null
Caption = Processor information
Description = CPU usage statistics
ElementName = null
Name = 0
IsAggregate = FALSE
PercentIdleTime = 99
PercentUserTime = 0
PercentNiceTime = 0
PercentPrivilegedTime = 0
PercentInterruptTime = 0
PercentDPCTime = 0
PercentProcessorTime = 1
PercentIOWaitTime = 0

SCX_ProcessorStatisticalInformation
InstanceID = null
Caption = Processor information
Description = CPU usage statistics
ElementName = null
Name = _Total
IsAggregate = TRUE
PercentIdleTime = 99
PercentUserTime = 0
PercentNiceTime = 0
PercentPrivilegedTime = 0
PercentInterruptTime = 0
PercentDPCTime = 0
PercentProcessorTime = 1
PercentIOWaitTime = 0

 

Additional references for WinRM syntax and troubleshooting

Warren’s blog

Docs site

Use Unix MP’s for shell commands

 

MMA Agent and SCOM Agent version numbers


 

FYI – Updated 24 June 2022

 

What are the MMA Agent and SCOM Agent version numbers?

This idea sprung from a discussion with Sr. PFE Brian Barrington, and it got me wondering…See below for more details on OMS/MMA, and SCOM agent versions, as well as how to verify agent from PowerShell.

 

 

FYI – If you’re running a SCOM agent, 2016 or above, various Log Analytics solutions may have pre-reqs.

The Content Dev team under Brian Wren added this to the docs.microsoft.com site

SCOM 2022     https://docs.microsoft.com/en-us/system-center/scom/release-build-versions?view=sc-om-2022

SCOM 2019     https://docs.microsoft.com/en-us/system-center/scom/release-build-versions?view=sc-om-2019

SCOM 2016      https://docs.microsoft.com/en-us/system-center/scom/release-build-versions?view=sc-om-2016

 

 

Azure Monitor Agent

AMA (Azure Monitor Agent)/ALA/OMS/MMA Agent can run on Windows/Linux operations systems.  Name has changed over the years, where AMA (Azure Monitor Agent) will be the name going forward for the cloud based offer.  See docs article here.

This also has been updated on the Docs site

https://docs.microsoft.com/en-us/azure/azure-monitor/agents/azure-monitor-agent-extension-versions

Download installer files here

Review what operating systems are covered here

Previously known as Windows OMS/ALA/MMA agent

Unfortunately, there’s no github repo that I’ve found.

Examples:

As of 6 Sep 2018, MMA agent = 8.0.11103.0

As of 17 Oct 2018, MMA agent = 8.0.11136.0

Skipping forward to 2020, the MMA agent is 10.20.18040.0

[!WARNING] The Log Analytics agents are on a deprecation path and will no longer be supported after August 31, 2024.

 

 

OMS Gateway

Older product published in 2016 – Download link here

OMS Gateway requires Microsoft Monitoring Agent (MMA)

(agent version – 8.0.10900.0 or later)

Simple English, that means SCOM2016 RTM agent or above

 

 

 

OMSAgent for xPlat

OMS-Agent-for-Linux

(Linux/Universal Linux)

Sep 16, 2021      OMSAgent_v1.13.40-0
Mar 08, 2021      OMSAgent_v1.13.35-0
Nov 16, 2020      OMSAgent_v1.13.33-0
Support for Red Hat Enterprise Linux 8, CentOS 8, Oracle 8, Ubuntu 20.04, SLES…
Nov 14, 2019       OMSAgent_v1.12.15-0
Jun 17, 2019      OMSAgent_v1.11.0-9
Apr 23, 2019      OMSAgent_v1.10.0-1
Feb 12, 2019      OMSAgent_v1.9.0-0
Nov 05, 2018     OMSAgent_v1.8.1.256
Oct 30, 2018      OMSAgent_1.8.0-256
Sep 03, 2018      OMSAgent_v1.6.1.3

 

 

Windows SCOM Agent Version numbers 

SCOM2016 

Build NumberKBRelease DateDescriptionStep-by-Step
8.0.10918.0EvaluateOct 2016SCOM 2016 RTMLink
8.0.10931.0KB3190029Feb 2017SCOM 2016 Update Rollup 1Link
8.0.10949.0KB3209591March 2017SCOM 2016 Update Rollup 2Link
8.0.10970.0KB4016126May 2017SCOM 2016 Update Rollup 3Link
8.0.10977.0KB4024941Oct 2017SCOM 2016 Update Rollup 4Link
8.0.10990.0KB4090987April 2018SCOM 2016 Update Rollup 5None
8.0.11004.0KB4459897Oct 2018SCOM 2016 Update Rollup 6Link
8.0.11025.0KB4492182April 2019SCOM 2016 Update Rollup 7Link
8.0.11037.0KB4514877Sept 2019SCOM 2016 Update Rollup 8Link
8.0.11049.0KB4546986April 2020SCOM 2016 Update Rollup 9Link
8.0.11000.0KB4580254Dec 2020SCOM 2016 Update Rollup 10Link
7.2.12335.0KB5006871Oct 2021SCOM 2016 Update Rollup 10 HotfixLink

SCOM1801
8.0.13053.0 RTM

SCOM1807

8.0.13067.0      General Availability release

 

SCOM2019

Build NumberKBRelease DateDescriptionStep-by-Step
10.19.10050.0EvaluateMarch 2019SCOM 2019 RTMLink
10.19.10311.0KB4533415Feb 2020SCOM 2019 Update Rollup 1Link
10.19.10407.0KB4558752Sept 2020SCOM 2019 Update Rollup 2Link
10.19.10505.0KB4594078March 2021SCOM 2019 UR3Link
10.19.10550.0KB5006871Oct 2021SCOM 2019 UR3 HotfixLink

 

 

  • @Larry LeBlanc – thank you for the SCOM Agent version updates!

 

Verify what version is installed

Via SCOM – use Holman’s Agent Version Addendum management pack

 

If you don’t have SCOM

From PowerShell

$Agent = get-itemproperty -path “HKLM:\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\Setup”

$Agent.CurrentVersion

 

 

 

 

 

 

Resources

SCOM Agent Version Addendum pack https://kevinholman.com/2017/02/26/scom-agent-version-addendum-management-pack/

SCOM Agent build numbers https://social.technet.microsoft.com/wiki/contents/articles/34312.system-center-operation-manager-momscom-list-of-build-numbers.aspx

Linux Agent can be downloaded from GitHub – github.com/Microsoft/OMS-Agent-for-Linux

MMA Agent, cross platform, and Azure

Things that make you go hmmm….

 

 

Ran across a scenario where we were trying to connect Azure Cross-platform (Linux) VM’s and MMA/SCOM agents to SCOM management group.

 

Management group was 2012R2, discovery wizard from SCOM console, failed to install agent, certificate errors.

 

Researching, found this article first

Windows Azure VM monitoring blog

There’s a version history for the Azure Monitor VM extension here

Applies:

SCOM2012R2 after UR12 or SCOM 2016 UR2+ deprecated the SHA1 certificate

 

Deprecating SHA1 certificates
Tech Community blog

 

Product team nicely published a TechNet gallery script to help!

Gallery download – Script to update SHA1 certificates to SHA256 on cross-platform agents – SCOM

TechNet Gallery Download
https://gallery.technet.microsoft.com/scriptcenter/Script-to-update-SHA1-8a30c5ef

 

 

Installing and configuring the MMA agent

 

Maybe the MMA agent is like Venom?
Proof I’ve watched too many a Marvel movie…

 

An existential moment perhaps, but the MMA agent can be a bunch of strings stuck from one place to another, monitoring whatever its told to do.

 

 

 

If you are running SCOM2016 or above, the MMA agent is built-in with Log Analytics, just configure your workspace

 

 

 

 

Download and Install MMA agent

SCOM 2012R2 agent does not have MMA, so download MMA agent from Log Analytics workspace

Azure Portal > Log Analytics > Subscription > Advanced Settings

Click on Windows Servers from Connected Sources to download Windows Agent

Click on Linux Servers from Connected Sources to download Linux Agent

 

 

From the Azure Portal (https://ms.portal.azure.com)

Click on Log Analytics, <your subscription >

Click on Advanced Settings

My view defaulted to Connected Sources > Windows Servers

 

Save the workspace ID and workspace key to notepad/OneNote for later

 

 

 

< Assuming the MMA agent is installed with Log Analytics capability >

 

 

Update MMA Agent with Workspace ID and Key

From MMA agent, update the OMS Workspace with the GUID copied to notepad

 

Click on Start > Control Panel, System and Security > Microsoft Monitoring Agent

Click on Azure Log Analytics (OMS) tab on MMA agent

Click Add

 

Add Workspace ID and Key to agent

Click OK

Click OK again on MMA properties

 

Look for the healthy green checkbox’d circle

 

Troubleshooting Errors in the Operations Manager Event Logs

Blog posts – Verify, 55002

 

 

 

 

 

 

 

 

Azure Log Analytics Service Map Planning and Pre-reqs

My grandfather said two things:

An ounce of prevention is worth a pound of manure

Death and taxes are part of life

 

Planning out a deployment is a good thing.

My best friend would say “No one plans to fail, they just fail to plan”

 

 

This will be a multi-part blog – breaking out the high level steps, and my experience getting the solution set up.

 

What do we need for Service Map?

  • Azure connectivity
    • Setup Log Analytics workspace on MMA/SCOM agent article
    • Troubleshooting onboarding issues KB,
      • Check for Events in Operations Manager event logs blog
  • Computers in scope for visualization
    • What computers (Windows or Linux)
    • Pricing FAQ
  • Dependency agent installed on computers
  • Azure Service Principal
    • (think of it as an SSH shared key ID/password for Azure Apps to communicate)
    • Docs article

 

High level steps

  1. Overview blog
  2. Install the MMA agent blog
  3. Install the dependency agent blog
  4. Configure Azure Service Principal blog
  5. Configure Service Map on SCOM blog