Teams monitoring

Teams logo

IN case you’re assessing Teams monitoring options, the journey begins understanding the product roadmap.  Second, depending on your monitoring platform, there’s options for monitoring.  Depending on the monitoring products in your environment, whether SCOM, Azure Monitor, Splunk, SolarWinds or others, there are easier lifts to achieve some perspective into the SaaS model for the M365 platform. Lastly, I hope these links help point to proof of concepts to embrace ‘user experience’ deep monitoring.

M365 Product Roadmap

The Microsoft 365 roadmap provides estimated release dates and descriptions for commercial features. It includes updates that are currently in development, rolling out, or fully released. You can access the latest updates and detailed descriptions on the Microsoft 365 Roadmap

 

Microsoft Teams product group built the M365 pack

Download link https://www.microsoft.com/en-us/download/details.aspx?id=103379

Microsoft engineers built the M365 Supplemental

M365 Services Supplemental Monitoring Management Pack v3

 

Leverage Teams Real-Time analytics

Good for your Azure Tenant, Microsoft Teams Premium allows for real-time telemetry.  Build your own action alerts to be aware of user issues.  The solution uses real-time telemetry with details about devices, networks, and connectivity to troubleshoot user problems with Microsoft Teams scheduled meetings.

Link https://learn.microsoft.com/en-us/microsoftteams/use-real-time-telemetry-to-troubleshoot-poor-meeting-quality

Login URL https://admin.teams.microsoft.com/

 

M365 Admin Center

Alot of what might be called Office framework, the M365 Admin Center has API’s to help assess M365 Usage Analytics telemetry (analytic data).

Enable Usage analytics https://learn.microsoft.com/en-us/microsoft-365/admin/usage-analytics/usage-analytics?view=o365-worldwide

 

Teams Call Record Insights

https://techcommunity.microsoft.com/blog/microsoftteamsblog/announcing-call-record-insights-for-microsoft-teams/4108875

 

Have fun with this!

SNOW REST connectivity

SNOW REST connectivity network icon
SNOW REST connectivity network icon

 

Verify ServiceNow test/production  environment connectivity.  Are you ready to begin the checks for ‘SNOW REST connectivity’?

 

Verify the following pre-requisites are met before proceeding:

Proxy

TEST/PROD ServiceNow (SNow) URL(s)

ID

Password

 

High level outline:

Verify network connectivity via proxy, using SNOW URL’s, ID, password

Verify CredentialManager Snapin installed

Setup credentials in CredentialManager

 

 

Verify network connectivity via proxy, using SNOW URL’s, ID, password

Determine the SCOM notification account being used, see blog for details.

RDP to SCOM MS using notification account.

Open PowerShell as administrator

Verify network connectivity pasting command into PowerShell window

Example SNOW URL JustinTime.servicenowservices.com

Test-NetConnection -ComputerName JustinTime.servicenowservices.com -port 443

Test-NetConnection -ComputerName JustinTime.servicenowservices.com -port 443 -proxy ##PROXY##

 

Successful connectivity

PS C:\MonAdmin\SCRIPTS> Test-NetConnection -ComputerName servicenowservices.com -port 443

ComputerName     : servicenowservices.com

RemoteAddress    : 199.91.136.115

RemotePort       : 443

InterfaceAlias   : Ethernet 2

SourceAddress    : 10.0.0.30

TcpTestSucceeded : True

 

 

Verify CredentialManager Snapin is installed

Best practice – Encrypt credentials on SCOM MS to prevent cleartext in scripts.  To begin, this includes details to verify SnapIn, verify credentials stored, store credentials.

 

Verify CredentialManager Snapin is installed

RDP to SCOM MS server(s) as SCOM Notifications SVC account with SA access

Open PowerShell as administrator

Paste command(s) into PowerShell window to test network connectivity to SNOW environments

# Verify Credential Manager snap in installed

$CredMgrModuleBase = Get-Module -Name CredentialManager

if ( $Null -ne $CredMgrModuleBase.ModuleBase )

{ write-host -f yellow “CredentialManager PoSH Module Installed, ModuleBase = $($CredMgrModuleBase.ModuleBase)” }

if ( $Null -eq $CredMgrModuleBase.ModuleBase )

{ write-host -f red “CredentialManager PoSH Module NOT Installed” }

 

CredentialManager snapin PowerShell output
CredentialManager snapin PowerShell output

 

Verify Stored Credentials on server

First, verify any credentials stored on server, specific to ServiceNow or not.  Second, we begin to use the Get-StoredCredential command. Third, we will setup the credential for REST integration, lastly verifying credential.

Setup Credentials for SNOW RESTAPI

RDP to SCOM MS server(s) as SCOM Notifications SVC account with SA access

Open PowerShell as administrator

Paste command(s) into PowerShell window to test network connectivity to SNOW environments

 

If no output, there are no credentials stored under the RDP login.

Get-StoredCredential

Commands specific to ServiceNow to verify credentials exist

$Credential = Get-StoredCredential -Target “ServiceNowCredential”

$Credential = Get-StoredCredential -Target “svc_rest_scom”

CredentialManager Get-StoredCredential Output
CredentialManager Get-StoredCredential Output

Alternate command to verify stored credential

Get-StoredCredential -Target “ServiceNowCredential”

CredentialManager Get-StoredCredential -Target "ServiceNowCredential"
CredentialManager Get-StoredCredential -Target “ServiceNowCredential”

 

 

Setup Credentials for SNOW RESTAPI

RDP to SCOM MS server(s) as SCOM Notifications SVC account with SA access

Open PowerShell as administrator

Paste the following commands to create stored credential:

New-StoredCredential -Target “ServiceNowCredential” -UserName “svc_rest_scom” -Password “##Password##” -Persist ‘LocalMachine’

 

New-StoredCredential output example
New-StoredCredential output example

 

Verify credential being used by SCOM

$Credential = Get-StoredCredential -Target “ServiceNowCredential”

New-S

 

Therefore, network connectivity verified.  Lastly, encrypted credentials are stored in the SCOM notifications SVC account.  Continue the integration!

SNOW REST integration prerequisites

SNOW REST integration prerequisites
SNOW REST integration prerequisites

Did you know –

These PowerShell scripts allow organizations to specify which alerts/events that need to go to ITSM tool.

Does not matter to the tool, whether Broadcom (Spectrum/DXOI), SolarWinds, MECM/MEM/MCM, SCOM

Pretty much ANY tool that can leverage PowerShell scripts and/or REST calls can utilize this script.

The following ServiceNow ‘SNOW REST integration prerequisites’ are required before proceeding.

TEST/PROD ServiceNow (SNow) URL(s)

ID

Password

Incident short_description naming convention

ServiceNow SNOW Alert rule (to make events create incidents)

ServiceNow SNOW Incidents require additional variables to match ServiceNow selections

 

SNOW Incident short_description field is the title of the incident

Pre-define this in the SNOW REST Event/incident injection, to meet organizational naming conventions

Description can be additional details about the issue to be investigated, resolved.

SNOW Incident short_description, description fields
SNOW Incident short_description, description fields

 

Examples of short_description titles

# Setup SNOW Event Name standard

Example SNOWAlertName

$SNOWAlertName = “<Org> <Team> SCOM Test Event – $Alert”

Example SNOWAlertName

$SNOWAlertName = “<Team> <ORG> SCOM Event – $AlertName”

Example SNOWAlertName

$SNOWAlertName = “<Team> <ORG> SCOM $AlertName”

Example SNOWAlertName

$SNOWAlertName = “##CUSTOMER## ##TEAM## SCOM Event – $AlertName”

Example SNOWAlertName

$SNOWAlertName = “##TEAM## ##CUSTOMER##: SCOM – $AlertName”

Example SNOWAlertName

$SNOWAlertName = “##TEAM## ##CUSTOMER##: SPECTRUM – $AlertName”

Example SNOWAlertName

$SNOWAlertName = “##TEAM## ##CUSTOMER##: SOLARWINDS – $AlertName”

 

SNOW Incident fields

Direct Incident REST injection requires additional fields, such as caller, business_service, category, subcategory, channel, impact, urgency, priority, assignment_group
SNOW Incident fields
SNOW Incident fields
Additional SNOW incident fields
Additional SNOW incident fields

 

Additional information

REST/RESTAPI https://www.techtarget.com/searchapparchitecture/definition/RESTful-API

SNOW Utah Connector https://docs.servicenow.com/bundle/utah-it-operations-management/page/product/event-management/task/t_EMConfigureSCOMConnector.html

Additional blogs showing scope and options using ServiceNow for ITSM tool

https://kevinjustin.com/blog/2024/03/27/servicenow-event-integration/

https://kevinjustin.com/blog/2024/03/27/servicenow-incident-integration/

https://kevinjustin.com/blog/2024/04/30/servicenow-connector-for-scom/

https://kevinjustin.com/blog/2024/05/24/lots-of-options-to-create-servicenow-incidents-through-powershell-scripts-connectors-and-3rd-party-vendors/

Integrate SCOM and SolarWinds

Steve Irwin quote - what a beauty!
Steve Irwin quote – what a beauty!

I’m ISO (in search of) the mythical single pane of glass.  In my best Steve Irwin voice…  Integrate SCOM and SolarWinds – We are strong together.  To me, integration occurs everywhere, at home, in your job, anyone you work with.  Contribute, don’t consume 🙂   Everyone is unique, including preferences, and past experiences.  In my career, I’ve been lucky to administer both tools for Fortune 100 companies (and more tools) across my career.  I hope this blog provides a way to use both tools to get the full value for the least cost!

 

 

 

Integration typewriter picture - stronger together - integrate SCOM and SolarWinds
Integration typewriter picture – stronger together – integrate SCOM and SolarWinds

Integration

The real meat of this is how to get the most for the least cost!

Let’s ‘Integrate SCOM and SolarWinds’ into our unicorn.  To date, getting data from SolarWinds into SCOM allows for easiest PowerBI Apps/reports

In my experience, the MSI requires Solarwinds Support login to download the SolarWinds Management Pack for SCOM.msi.  Downlload and configure article here.

SCOM view of SolarWinds data
SCOM view of SolarWinds data

3rd party options (free)

Cookdown vendor method to grab SolarWinds data into SCOM

Webhooks SolarWinds – Cookdown details

 

SolarWinds Thwack entries – you can find my Thwack submissions there as well 😊

https://thwack.solarwinds.com/resources/thwack-command-center/f/forum/39833/scom-connector

https://thwack.solarwinds.com/product-forums/network-performance-monitor-npm/f/forum/70676/scom-integration

 

One possible PowerBI report

PowerBI App with multiple reports, dashboards
PowerBI App with multiple reports, dashboards

 

 

ITSM integrations

Remedy

Most times the Remedy application is old, deprecated, outside it’s service life, rarely integrated with monitoring.  One customer’s PowerBI report shows the utility at a glance (see picture below)

PowerBI report referencing ITSM insights for teams, totals, volume
PowerBI report referencing ITSM insights for teams, totals, volume

ITSM integration for ServiceNow (SNow)

Free – https://powerbi.microsoft.com/en-us/blog/explore-your-servicenow-data-with-power-bi/

SNOW PowerBI Connector (pay – requires support contract login)

https://store.servicenow.com/sn_appstore_store.do?#!/store/application/87a42573879e0110fb5033773cbb354f/

 

Documentation

2021 Blog https://www.upguard.com/blog/solarwinds-vs-scom

Blog https://www.trustradius.com/compare-products/microsoft-system-center-operations-manager-scom-vs-solarwinds-server-application-monitor

Compare SolarWinds and SCOM

My Big Fat Greek Wedding - we're all just fruits!
My Big Fat Greek Wedding – we’re all just fruits!

I think of My Big Fat Greek wedding to ‘Compare SolarWinds and SCOM’.  The wedding reception, where the father says the root of his daughter, and son-in-law’s last names, are from the greek word for Orange, and Apple.  “so in the end, we’re all fruits”   We are the same but different, where diversity and inclusion is key.  Everyone’s got a voice.  Contribute, don’t consume 🙂

 

First, I’ve been lucky to administer both tools for Fortune 100 companies (and more tools).  Second, I hope this blog provides some clarification of the strengths, weaknesses, and costs associated with both tools.  Here’s hoping wordpress readers identify with my background – saving money, cutting coupons, looking for on-sale, buy one get one deals.  Thirdly, while everyone’s past experiences may not be the same, cost is still a big factor.  Lastly, proprietary tools, Security, and other requirements can make or break an implementation.

 

 

Here’s a link to a PPT built to ‘Compare SolarWinds and SCOM’ feature wise, that goes along with ‘My Big Fat Greek Wedding’ and the fruit.  PPT title ‘better together’, is loaded with links and breaking out key capabilities.

 

Some items NOT covered in the PPT comparison

Example context – SAW/PAW/Red Forest

Both tools can store credentials within the application, obfuscated.

SCOM allows gMSA’s (managed service accounts) for key services including run as accounts.  View the Monitoring Guys blog plug here for CJ, Scott, and Tyson’s contributions 😛

 

COST

SolarWinds small enterprise example
Windows Server, SQL licenses (no cost given)

Monitors Windows, Non-Windows, Microsoft products

Community of custom application monitoring

Renewal cost per year in 2020 $48K/year
Add HA for SQL Enterprise licenses is same, where SW HA/High availability is the SolarWinds cost, not compute licenses for Windows Server, SQL
***500 license SAM, VOIP, IPAM, NPM/NCM.
Redesigning licensing to unlimited (site license) was $344K
Wow! Site licenses cost considerably more.
Though for clarification, 500 licenses equates to 500 monitors targeted at 500 servers.
SolarWinds costs broken out by feature
SolarWinds costs broken out by feature

Add unlimited VMAN, DPA, SCM, VNQM adds $256K

Add new SolarWinds features

 

Migrate functionality to site license ($48K > $344K)

Adding SolarWinds features with site unlimited licenses
Adding SolarWinds features with site unlimited licenses

 

SCOM small enterprise example

Windows Server, SQL licenses (no cost given)
No license limitation for products/features used, community built solutions

Monitors Windows, Non-Windows, Microsoft products

Large community of custom application monitoring

No yearly support costs (included with Microsoft support agreement)

SQL Enterprise licenses is same, where SW HA/High availability is the SolarWinds cost, not compute licenses for Windows Server, SQL

ESX monitoring via NiCE VMWare 3rd party pay pack is $10K/year
OpsLogix Teams integration helps with NOC/NOSC/SOC integration
Including NiCE Oracle monitoring $10k/year

 

I’ll leave the cost comparisons to you.

Securing the Applications and web consoles

SolarWinds (SW)

Secure SW website search, Smart Cards post, 2FA/MFA/RSA post

NPM (now N-Able RMM – Remote Management & Monitoring)

NCM Thwack forum

SCOM web console

Did you know – gMSA’s (managed service accounts) can be used with SCOM, Windows, AD, etc?  Monitoring Guys blog plug here for CJ, Scott, and Tyson 😛

Configuring AD Delegation, Smart cards and SSL certs (Client Certificate Mapping Authentication, IIS configuration, FIPS

Knowledge sources: Learn.Microsoft.Com, TechNet, blogs, STIG Library and more

 

Vulnerability mitigation

SCOM vulnerability mitigations Blog vuln search, SCOM STIGs plus IIS, Windows Server, SQL, WebServer ALL apply

Solarwinds vulnerability – Trust Center – CVE2023-23836, CVE2021-35211, CVE-2023-33231, all from searches.

NO DISA STIG for SolarWinds, so IIS, Windows Server, SQL, WebServer ALL apply

 

NOTE: I’ve NOT supported SolarWinds recently to see Security scans for other vulnerabilities and STIG settings (Windows Server, SQL, IIS, Network blog.  STIG dashboard ‘how to’

 

 

Licensing

Licensing is a big differentiator cost wise

SolarWinds needs an EA for Windows Server, SQL licenses.

SCOM has been part of the EA (Enterprise agreement) for at least 15+ years (since SCOM2007, if not MOM2005).  Windows Server license (now CPU based), SQL license, however NOT enterprise comes standard.  One reason the System Center suite is successful might be this built-in licensing, as well as the feature depth and cost the tools provide.

 

 

Hardware requirements

In my experience interacting with customers, SolarWinds support recommends hardware configuration well above vendor recommendations.  Support recommendations requesting high compute to provide memory level SQL speed and responsive web console.  However, the compute is basically ESX host level compute in the realm of 128GB of memory per server, in High Availability (HA), meaning x4 – 2 servers for 2 sites.

Monitoring tools are rarely Tier1 Applications with respective Service Level Availability (SLA).  Expectation alone presents a disparity, and false impression.  People just see a tool and base on personal experience.

Ferrari vs. GMC Cyclone - fooled you eh
Ferrari vs. GMC Cyclone – fooled you eh

Is it really surprising if one is faster than the other?