Category: PowerShell

SNOW REST connectivity

SNOW REST connectivity network icon
SNOW REST connectivity network icon

 

Verify ServiceNow test/production  environment connectivity.  Are you ready to begin the checks for ‘SNOW REST connectivity’?

 

Verify the following pre-requisites are met before proceeding:

Proxy

TEST/PROD ServiceNow (SNow) URL(s)

ID

Password

 

High level outline:

Verify network connectivity via proxy, using SNOW URL’s, ID, password

Verify CredentialManager Snapin installed

Setup credentials in CredentialManager

 

 

Verify network connectivity via proxy, using SNOW URL’s, ID, password

Determine the SCOM notification account being used, see blog for details.

RDP to SCOM MS using notification account.

Open PowerShell as administrator

Verify network connectivity pasting command into PowerShell window

Example SNOW URL JustinTime.servicenowservices.com

Test-NetConnection -ComputerName JustinTime.servicenowservices.com -port 443

Test-NetConnection -ComputerName JustinTime.servicenowservices.com -port 443 -proxy ##PROXY##

 

Successful connectivity

PS C:\MonAdmin\SCRIPTS> Test-NetConnection -ComputerName servicenowservices.com -port 443

ComputerName     : servicenowservices.com

RemoteAddress    : 199.91.136.115

RemotePort       : 443

InterfaceAlias   : Ethernet 2

SourceAddress    : 10.0.0.30

TcpTestSucceeded : True

 

 

Verify CredentialManager Snapin is installed

Best practice – Encrypt credentials on SCOM MS to prevent cleartext in scripts.  To begin, this includes details to verify SnapIn, verify credentials stored, store credentials.

 

Verify CredentialManager Snapin is installed

RDP to SCOM MS server(s) as SCOM Notifications SVC account with SA access

Open PowerShell as administrator

Paste command(s) into PowerShell window to test network connectivity to SNOW environments

# Verify Credential Manager snap in installed

$CredMgrModuleBase = Get-Module -Name CredentialManager

if ( $Null -ne $CredMgrModuleBase.ModuleBase )

{ write-host -f yellow “CredentialManager PoSH Module Installed, ModuleBase = $($CredMgrModuleBase.ModuleBase)” }

if ( $Null -eq $CredMgrModuleBase.ModuleBase )

{ write-host -f red “CredentialManager PoSH Module NOT Installed” }

 

CredentialManager snapin PowerShell output
CredentialManager snapin PowerShell output

 

Verify Stored Credentials on server

First, verify any credentials stored on server, specific to ServiceNow or not.  Second, we begin to use the Get-StoredCredential command. Third, we will setup the credential for REST integration, lastly verifying credential.

Setup Credentials for SNOW RESTAPI

RDP to SCOM MS server(s) as SCOM Notifications SVC account with SA access

Open PowerShell as administrator

Paste command(s) into PowerShell window to test network connectivity to SNOW environments

 

If no output, there are no credentials stored under the RDP login.

Get-StoredCredential

Commands specific to ServiceNow to verify credentials exist

$Credential = Get-StoredCredential -Target “ServiceNowCredential”

$Credential = Get-StoredCredential -Target “svc_rest_scom”

CredentialManager Get-StoredCredential Output
CredentialManager Get-StoredCredential Output

Alternate command to verify stored credential

Get-StoredCredential -Target “ServiceNowCredential”

CredentialManager Get-StoredCredential -Target "ServiceNowCredential"
CredentialManager Get-StoredCredential -Target “ServiceNowCredential”

 

 

Setup Credentials for SNOW RESTAPI

RDP to SCOM MS server(s) as SCOM Notifications SVC account with SA access

Open PowerShell as administrator

Paste the following commands to create stored credential:

New-StoredCredential -Target “ServiceNowCredential” -UserName “svc_rest_scom” -Password “##Password##” -Persist ‘LocalMachine’

 

New-StoredCredential output example
New-StoredCredential output example

 

Verify credential being used by SCOM

$Credential = Get-StoredCredential -Target “ServiceNowCredential”

New-S

 

Therefore, network connectivity verified.  Lastly, encrypted credentials are stored in the SCOM notifications SVC account.  Continue the integration!

SNOW REST integration prerequisites

SNOW REST integration prerequisites
SNOW REST integration prerequisites

Did you know –

These PowerShell scripts allow organizations to specify which alerts/events that need to go to ITSM tool.

Does not matter to the tool, whether Broadcom (Spectrum/DXOI), SolarWinds, MECM/MEM/MCM, SCOM

Pretty much ANY tool that can leverage PowerShell scripts and/or REST calls can utilize this script.

The following ServiceNow ‘SNOW REST integration prerequisites’ are required before proceeding.

TEST/PROD ServiceNow (SNow) URL(s)

ID

Password

Incident short_description naming convention

ServiceNow SNOW Alert rule (to make events create incidents)

ServiceNow SNOW Incidents require additional variables to match ServiceNow selections

 

SNOW Incident short_description field is the title of the incident

Pre-define this in the SNOW REST Event/incident injection, to meet organizational naming conventions

Description can be additional details about the issue to be investigated, resolved.

SNOW Incident short_description, description fields
SNOW Incident short_description, description fields

 

Examples of short_description titles

# Setup SNOW Event Name standard

Example SNOWAlertName

$SNOWAlertName = “<Org> <Team> SCOM Test Event – $Alert”

Example SNOWAlertName

$SNOWAlertName = “<Team> <ORG> SCOM Event – $AlertName”

Example SNOWAlertName

$SNOWAlertName = “<Team> <ORG> SCOM $AlertName”

Example SNOWAlertName

$SNOWAlertName = “##CUSTOMER## ##TEAM## SCOM Event – $AlertName”

Example SNOWAlertName

$SNOWAlertName = “##TEAM## ##CUSTOMER##: SCOM – $AlertName”

Example SNOWAlertName

$SNOWAlertName = “##TEAM## ##CUSTOMER##: SPECTRUM – $AlertName”

Example SNOWAlertName

$SNOWAlertName = “##TEAM## ##CUSTOMER##: SOLARWINDS – $AlertName”

 

SNOW Incident fields

Direct Incident REST injection requires additional fields, such as caller, business_service, category, subcategory, channel, impact, urgency, priority, assignment_group
SNOW Incident fields
SNOW Incident fields
Additional SNOW incident fields
Additional SNOW incident fields

 

Additional information

REST/RESTAPI https://www.techtarget.com/searchapparchitecture/definition/RESTful-API

SNOW Utah Connector https://docs.servicenow.com/bundle/utah-it-operations-management/page/product/event-management/task/t_EMConfigureSCOMConnector.html

Additional blogs showing scope and options using ServiceNow for ITSM tool

https://kevinjustin.com/blog/2024/03/27/servicenow-event-integration/

https://kevinjustin.com/blog/2024/03/27/servicenow-incident-integration/

https://kevinjustin.com/blog/2024/04/30/servicenow-connector-for-scom/

https://kevinjustin.com/blog/2024/05/24/lots-of-options-to-create-servicenow-incidents-through-powershell-scripts-connectors-and-3rd-party-vendors/

Create ServiceNow incidents from SCOM

Create ServiceNow incidents from SCOM
Create ServiceNow incidents from SCOM

Been working to ‘Create ServiceNow incidents from SCOM’ with Joe Kelly (Joe’s LinkedIn), Steven Brown (Steven’s LinkedIn), and Tim Fields (Tim’s LinkedIn).  Over the past few months, we’ve found quite a few methods available, some free, some with cost.  All methods are built on ServiceNow (SNow) REST commands.  The resulting PowerShell scripts can do most of the heavy lifting for other Tools like SolarWinds, vCenter, vRealize,  Zabbix, SAP manager, Kafka, Nagios, HP OpenView.

 

Last month’s blog showcased the ServiceNow API/Connector for SCOM

ServiceNow Connector for SCOM

 

Depending on your customer use cases, this may not be the preferred choice.

One SNOW flavor is like NetCool, where you inject ALL SCOM alerts into ITSM tool (i.e. SNOW events).  Another flavor allows you to create a group of objects which will inject SNOW events.

What if you only want incidents for a small subset, and don’t want to wait on SNOW team?

We got that covered.

What if SNOW team wants events only, how can you create an incident?

We got that covered.

 

ServiceNow connector options:

Inject SNOW incident from SCOM (direct) https://github.com/theKevinJustin/New-SNOWIncidents

Inject SNOW event (requires SNOW alert rules to process alert to incident) https://github.com/theKevinJustin/New-SNowEvent

SNOW SCOM connector

    1. Send all SCOM alerts to SNOW as events
    2. Send specific SCOM group of objects to SNOW as events

3rd party options

      1. OpsLogix SCOM ServiceNow connector (costs) https://www.opslogix.com/servicenow-incident-connector
      2. Cookdown SNOW incident connector https://www.cookdown.com/blog/how-to-build-a-servicenow-incident-connector-for-scom download MP https://www.cookdown.com/servicenow-monitoring-mp#download
      3. Tim McFadden SCOM ServiceNow connector https://www.scom2k7.com/introducing-the-scom-to-servicenow-connector/ (unsure if Tim still supports this)

 

Stay tuned…

Next steps are to build out the SCOM command execution channel using Holman’s blog https://kevinholman.com/2021/08/25/what-account-will-command-channel-notifications-run-as-in-scom/

ServiceNow Event integration

ServiceNow Event integration
ServiceNow Event integration
Time to integrate your Monitoring tools to ITSM tool.  First, this blog post documents ‘ServiceNow Event integration’.  Second, let’s explain the common acronym in my experience is SNOW/SNow.  Third, some background – ServiceNow has been around for some time as an Information Technology Service Management (ITSM), and discovery tool.  As a SaaS solution, companies can purchase a subscription and integrate tools via RESTAPI to create/update/close events or incidents.
First, let’s begin to discuss SCOM notification methods.  SCOM2022 adds a new capability with Teams integration.  Second, most people are familiar with notification methods leveraging Email (html or not), perhaps SMS, but not so much command channel, calling some script in shell, PowerShell, etc.  Generally, the command channel is basically a post processing script capability to execute notifications.  Third, example tools where command channel might be used – BMC BEM (BMC Event Manager), BMC Remedy, xMatters, DerDack; SNOW integration within SCOM, using notification channels.  Lastly, SaaS solutions (vendors like xMatters, and ServiceNow) allow RESTAPI crafted requests to take actions.
SNOW prerequisites
1) ServiceNow User/Password (or API key)
2) SNOW RESTAPI PowerShell needs to securely access credentials
For the Incident PowerShell, we store Credentials within Windows Credential Manager
3) Network connectivity to SaaS provider (use PowerShell test-netconnection from SCOM MS to test connectivity over whatever port(s) vendor requires.
4) ServiceNow CallerID GUID
5) Production and Test URL’s (also required for network connectivity tests)
6) Access to SNOW UI to verify required fields and values for the script parameters.
Update incident script and begin testing.
Download script from GitHub repo https://github.com/theKevinJustin/New-SNowEvent/
Download script, and copy to monitoring repository
Copy to SCOM management servers (MS)
NOTE Path, to run from management server
Update script, with pre-reqs above –
Credential Manager stored ID
For more detail, look at parameter examples below to verify UI.
Update with customer/ServiceNow SNOW subscription specific values:
##CallerID##
##CUSTOMER##    (customize SNOW short_description)
##TEAM##    (customize SNOW short_description)$Channel = “Direct”
$ServiceNowURL=”https://##SERVICENOWURL##/api/now/table/em_event”
$CallerID = “##CallerID##”
# if proxy is used, uncomment and replace with Proxy URL
#$Proxy = “##Proxy##”
# Test New-SNOWEvent.ps1
# Depending on how you want to randomly choose an alert to create SNOW event
Lab example
$Alerts = get-scomalert -resolutionstate 0 | where { $_.Name -like “System Center*” }
Gather Critical, New alerts
$Alerts = get-scomalert -ResolutionState 0 -severity 2
Debug for warning alerts
$Alerts = get-scomalert -ResolutionState 0 -severity 1
# Debug
$Alerts[0] | fl ID,Name,Description,Severity,MonitoringObjectDisplayName
.\New-SNOWEvent.ps1 -AlertName $Alerts[0].Name -AlertID $Alerts[0].ID -Impact 4 -Urgency 4 -Priority 3 -AssignmentGroup “System Admin” -BusinessService “System Management” -Category Support -SubCategory Repair -Channel Direct
Example output
PS C:\Users\scomadmin\Desktop> .\New-SNOWIncident.ps1 -AlertName $Alert.Name -AlertID $Alert.ID -Impact 4 -Urgency 4 -Priority 3 -AssignmentGroup “System Admin” -BusinessService “System Management” -Category Support -SubCategory Repair -Channel Direct
TEST ServiceNow URL specified.
CredentialManager PoSH Module Installed, ModuleBase = C:\Program Files\WindowsPowerShell\Modules\CredentialManager\2.0
The System Center Management Health Service 5E04F804-8B71-6EB6-0101-DCBB58022498 running on host 16DB02.testlab.net and s
erving management group with id {E39F5F53-9FBB-9D7F-4BFE-5F0324630AE5} is not healthy. Some system rules failed to load.
16DB02
Warning
impact 4
urgency 4
priority 3
ServiceNow Credential NOT stored on server

ServiceNow Incident Integration

ServiceNow Incident Integration
ServiceNow Incident Integration
Time to integrate your Monitoring tools to ITSM tool.  First, this blog post documents ‘ServiceNow Incident integration’.  Second, let’s explain the common acronym in my experience is SNOW/SNow.  Third, some background – ServiceNow has been around for some time as an Information Technology Service Management (ITSM), and discovery tool.  As a SaaS solution, companies can purchase a subscription and integrate tools via RESTAPI to create/update/close events or incidents.
First, let’s begin to discuss SCOM notification methods.  SCOM2022 adds a new capability with Teams integration.  Second, most people are familiar with notification methods leveraging Email (html or not), perhaps SMS, but not so much command channel, calling some script in shell, PowerShell, etc.  Generally, the command channel is basically a post processing script capability to execute notifications.  Third, example tools where command channel might be used – BMC BEM (BMC Event Manager), BMC Remedy, xMatters, DerDack; SNOW integration within SCOM, using notification channels.  Lastly, SaaS solutions (vendors like xMatters, and ServiceNow) allow RESTAPI crafted requests to take actions.
SNOW prerequisites
1) ServiceNow User/Password (or API key)
2) SNOW RESTAPI PowerShell needs to securely access credentials
For the Incident PowerShell, we store Credentials within Windows Credential Manager
3) Network connectivity to SaaS provider (use test-netconnection from SCOM MS to test connectivity over whatever port(s) vendor requires.
ServiceNow CallerID GUID
4) Production and Test URL’s (also required for network connectivity tests)
5) Access to SNOW UI to verify required fields and values for the script parameters.
Update incident script and begin testing.
Download script, and copy to monitoring repository
Copy to SCOM management servers (MS)
NOTE Path, to run from management server
Update script, with pre-reqs above –
Credential Manager stored ID
For more detail, look at parameter examples below to verify UI.
Update with customer/ServiceNow SNOW subscription specific values:
##Company##     (customize SNOW short_description)
##Team##     (customize SNOW short_description)$Channel = “Direct”
$ServiceNowURL=”https://##ServiceNowURL##/api/now/table/incident”
#$Proxy = “##CustomerProxyURL##”
$CallerID = “##GUID##”
# Test New-SNOWIncident.ps1
# Depending on how you want to randomly choose an incident
Lab example
$Alerts = get-scomalert -resolutionstate 0 | where { $_.Name -like “System Center*” }
Gather Critical, New alerts
$Alerts = get-scomalert -ResolutionState 0 -severity 2
Debug for warning alerts
$Alerts = get-scomalert -ResolutionState 0 -severity 1
# Debug
$Alerts[0] | fl ID,Name,Description,Severity,MonitoringObjectDisplayName
.\New-SNOWIncident.ps1 -AlertName $Alerts[0].Name -AlertID $Alerts[0].ID -Impact 4 -Urgency 4 -Priority 3 -AssignmentGroup “System Admin” -BusinessService “System Management” -Category Support -SubCategory Repair -Channel Direct
Example output
PS C:\Users\scomadmin\Desktop> .\New-SNOWIncident.ps1 -AlertName $Alert.Name -AlertID $Alert.ID -Impact 4 -Urgency 4 -Priority 3 -AssignmentGroup “System Admin” -BusinessService “System Management” -Category Support -SubCategory Repair -Channel Direct
TEST ServiceNow URL specified.
CredentialManager PoSH Module Installed, ModuleBase = C:\Program Files\WindowsPowerShell\Modules\CredentialManager\2.0
The System Center Management Health Service 5E04F804-8B71-6EB6-0101-DCBB58022498 running on host 16DB02.testlab.net and s
erving management group with id {E39F5F53-9FBB-9D7F-4BFE-5F0324630AE5} is not healthy. Some system rules failed to load.
16DB02
Warning
impact 4
urgency 4
priority 3
ServiceNow Credential NOT stored on server

SCOM maintenance schedules

SCOM maintenance schedules
SCOM maintenance schedules
Do your SCOM users need to know if a server is in scheduled maintenance?  This came about as Aris asked questions.
 First, let’s discuss specific maintenance mode and maintenance schedule scenarios users might ask.  Second, determining IF scheduled maintenance enabled, running, about to run.  Third, how does another user know when scheduled maintenance ends, allowing action and decision point to add/extend server maintenance.  Fourth, whenever scheduled maintenance entered by one user, is NOT automatically seen by other roles.  While product guidance states ‘maintenance schedules be added by someone in SCOM admin group’, self-service users still need visibility.  Lastly, can we figure out a way to answer these questions.  Given these points, users to be able to see server maintenance details.  Also, can solution adhere to best practice ‘no alerts during planned maintenance’.
From PowerShell on SCOM MS
Get-SCOMMaintenanceScheduleList
$ScheduleList = Get-SCOMMaintenanceScheduleList
$ScheduleList.ID
$ScheduleList.ScheduleID.Guid
foreach ( $ID in $ScheduleList.ScheduleID)
{
$Schedule = get-SCOMMaintenanceSchedule -ID $ID
# $Schedule.MonitoringObjects ;
(get-scomclassInstance -id $Schedule.MonitoringObjects.Guid).DisplayName
# Debug endtime
$Schedule | ft User,ActiveStartTime,ActiveEndDate,ScheduledEndTime
}
Example Output
SCOM Maintenance Schedule Output
SCOM Maintenance Schedule Output
Workflows:
Scheduled Maintenance report task
Maintenance mode report – what’s about to end maintenance mode.
Obviously, expect both workflows into the ‘Proactive NOSC DailyTasks’ pack.  GitHub repo  https://github.com/theKevinJustin/ProactiveNOSCDailyTasks

SQL STIG vulnerabilities V-213902, V-213935

Happy leap year, let’s talk Security and SQL STIG vulnerabilities V-213902, V-213935!
Happy Leap year
Happy Leap year

DISA DOD SQL STIG vulnerabilities V-213902, V-213935

SQL DBA team for RCC-C customer requesting documentation for exception, in light of vulnerabilities.
V-213902
V-213935
SCOM uses individual computer accounts in SQL for these findings
Holman documented this since 2012

SCOM SECURITY Documentation

SCOM2019 https://kevinholman.com/2020/07/23/scom-2019-security-account-matrix/

Both V-213902 AND V-213935 state same identification action.

Run this SQL Query on SCOM DB(s)
SELECT name
FROM sys.database_principals
WHERE type in (‘U’,’G’)
AND name LIKE ‘%$’
To remove users:
Run the following command for each user:
DROP USER [ IF EXISTS ] ;

V-213935 has a different identifier:

Launch PowerShell.
Execute the following code:
Note: <name> represents the username portion of the user. For example; if the user is “CONTOSO\user1$”, the username is “user1”.
([ADSISearcher]”(&(ObjectCategory=Computer)(Name=<name>))”).FindAll()
If no account information is returned, this is not a finding.
If account information is returned, this is a finding.

Tab delimited view –

Remove Computer Accounts DB SQL6-D0-000400 V-213902 CAT II Non-repudiation of actions taken is required in order to maintain data integrity. Examples of particular actions taken by individuals include creating information, sending a message, approving information (e.g., indicating concurrence or signing a contract), and receiving a message.
Remove Computer AccountsSQL6-D0-004200V-213935CAT IINon-repudiation of actions taken is required in order to maintain data integrity. Examples of particular actions taken by individuals include creating information, sending a message, approving information (e.g., indicating concurrence or signing a contract), and receiving a message.
Can provide one work-around to mitigate.
Awaiting CSS engagement for official mitigation from support and  SCOM PG.