Parse Events via PowerShell into table

Parsing Events via PowerShell into table
Optometrist eye testing equipment picture

 

Parse Events via PowerShell into table.  Ever have need to parse an event, and grab a field from the event description, then perform some action after that?

 

Here’s some PowerShell that may help you first to create a table, then setup columns, gather data, then parse what you need, and run a command to then output to the table

 

# Create Table for alerts

$Table = @()

$Table = $null

 

$Table = New-Object System.Data.DataTable “Failed Hosts List”

$Col1 = New-Object System.Data.DataColumn Host

$Col2 = New-Object System.Data.DataColumn IPAddress

$Table.Columns.Add($Col1)

$Table.Columns.Add($Col2)

 

 

$Alert20046 = Get-WinEvent -FilterHashtable @{LogName=’Operations Manager’;

ID=’20046′;}

 

$Alerts20046 = $Alert20046.Message

$Alerts20046.count

 

$Alerts20046uniq = $Alerts20046 | sort -uniq

$Alerts20046uniq.count

 

# $DeniedUniq = $Denied20046 | Sort-Object -Uniq

# $ServersDenied = @()

 

foreach ( $server in $DeniedUniq)

{

$Name = nslookup $server

foreach ($server in $Name)

{

# Add to Table   

# $Name.Split(“:”)[6]

# $Name.Split(“:”)[8]

$row = $Table.NewRow()

$row.Host = $Name.Split(“:”)[6]

$row.IPAddress = $Name.Split(“:”)[8]

$Table.Rows.Add($row)

}

}

 

Mining Windows Event Log

Mining Ore from the Windows Event Log and finding a way to make it portable

 

Use Get-WinEvent to use XML and filters from event viewer, to mine an event, including examples for a specific string, from a specific event, in a specific event log?

 

 

Hopefully this post will help with a few tips to simplify monitoring for events, whether in AzMon, SCOM, or via PowerShell.

 

 

Let’s start with the Dr Scripto blog post from quite a while ago –

https://devblogs.microsoft.com/scripting/data-mine-the-windows-event-log-by-using-powershell-and-xml/

 

Not sure how many people use get-WinEvent, but this is one tool in PowerShell that can help an admin parse the XML side of an event.

 

Example 1

Query Application Event Log for Severity, Event, and Event Data contains lync.exe

$query = @”

<QueryList>

  <Query Id=”0″ Path=”Application”>

    <Select Path=”Application”>*[System[Provider[@Name=’Application Hang’]

    and (Level=2) and (EventID=1002)]]

    and *[EventData[Data=’lync.exe’]]</Select>

  </Query>

</QueryList>

“@

Get-WinEvent -FilterXml $query

 

PowerShell output

Use Get-WinEvent to use XML and filters from event viewer
Lync.exe event example output

 

 

 

Use Get-WinEvent to use XML and filters from event viewer

The Tip or Trick part of this – leverage your Event Viewer Filter as a query to use with get-WinEvent

Credit for this tip comes from Andrew Blumhardt!

See below for examples to ‘use Get-WinEvent to use XML and filters from event viewer’

 

Navigating via Event Viewer:

Hop onto your favorite server, or connect to another server via Event Viewer

Go to the Event Log > Click Filter Current Log

Build out your filter (i.e. choose specific Event Sources, exclude events, include severities, timeframe (start/end), etc.)

Use Get-WinEvent to use XML and filters from event viewer
SCVMM Application Log Event ID 25933

Switch to the XML tab (and note you can edit your query further!)

SCVMM query example screenshot
Event Viewer filter XML tab

You can copy the query from the Event Viewer into your Get-WinEvent syntax

$query = @”

<QueryList>
<Query Id=”0″ Path=”Application”>
<Select Path=”Application”>*[System[Provider[@Name=’Microsoft.SystemCenter.VirtualMachineManager.2012.Monitor.UserRoleQuotaUsageMonitor’ or @Name=’Microsoft.SystemCenter.VirtualMachineManager.2012.Report.ServiceUsageCollection’ or @Name=’Microsoft.SystemCenter.VirtualMachineManager.2012.Report.VMUsageCollection’ or @Name=’Microsoft.SystemCenter.VirtualMachineManager.2016.EnableCredSSPClient’ or @Name=’Microsoft.SystemCenter.VirtualMachineManager.2016.Monitor.UserRoleQuotaUsageMonitor’ or @Name=’Microsoft.SystemCenter.VirtualMachineManager.2016.Report.ServiceUsageCollection’ or @Name=’Microsoft.SystemCenter.VirtualMachineManager.2016.Report.VMUsageCollection’] and (Level=2 or Level=3) and (EventID=25933)]]</Select>
</Query>
</QueryList>

“@

Get-WinEvent -FilterXml $query

 

PowerShell output

Use Get-WinEvent to use XML and filters from event viewer
SCVMM query example screenshot

 

 

 

 

Example 3

Grab System Event Log, Event ID 5827  (NetLogon denied events)

get-WinEvent -FilterHashtable @{LogName=’System’; ID=’5827′;}

 

PowerShell output

Use Get-WinEvent to use XML and filters from event viewer
get-WinEvent filter by logname and event ID

 

 

Documentation:

Get-WinEvent https://docs.microsoft.com/en-us/powershell/module/microsoft.powershell.diagnostics/get-winevent?view=powershell-7.1

MSFT DevBlogs https://devblogs.microsoft.com/scripting/data-mine-the-windows-event-log-by-using-powershell-and-xml/

Which subscription was the trigger?

Hello Again,

Surprise!

I am back, as a rusty nail, and back to make lemonade from lemons!

Rusty Nail through Lemon

Rusty Nail through Lemon

 

Ever run into an email you don’t want to get, but have difficulty finding

the subscription entry?

 

Do you get a subscription Email, and that channel has the Notification ID, but you’re not sure what subscription sent the alert email?

 

Let’s start with the notification email

Example

Notification subscription ID generating this message:

{AA0C1081-D04F-F5CA-DEB7-92B9ECA619E2}

 

On SCOM MS > Open PowerShell

Get-SCOMNotificationSubscription

 

Example

PS C:\windows\system32> Get-SCOMNotificationSubscription -ID

“BD52BB72-3FDE-9D7F-6214-B9A47A311896”

 

Configuration     :

Microsoft.EnterpriseManagement.Administration.AlertChangedSubscriptionConfiguration

ManagementGroup   : SCOMTestLab

Name              : Subscription168e29fd_a8e5_4ee4_956f_d9591b845475

DisplayName       : AD DailyTasks Report

Description       : +CRITERIA = Alert Name Contains ‘Proactive DailyTasks AD

Team Report’     +RESOLUTIONSTATE = (0)

New    +SUBSCRIBERS = AD team, USER Kevin Justin via EMAIL     +CHANNEL = SMTP Email

Actions           : {SMTPAction_a6a5314d_83f5_47c0_910a_e60040b4c808}

ToRecipients      : {USER <blank> via EMAIL, USER <blank> via EMAIL, USER

<blank> via Email, USER <blank> via EMAIL…}

CcRecipients      : {}

BccRecipients     : {}

Enabled           : True

Id                : bd52bb72-3fde-9d7f-6214-b9a47a311896

ManagementGroupId : 001b9265-3c9f-816c-aa36-a8687c05be8e

 

Get-SCOMNotificationSubscription | ? { $_.ID -eq

“BD52BB72-3FDE-9D7F-6214-B9A47A311896” }

 

Example

PS C:\windows\system32> Get-SCOMNotificationSubscription | ? { $_.ID -eq “BD52BB72-3FDE-9D7F-6214-B9A47A311896” }

Configuration     :

Microsoft.EnterpriseManagement.Administration.AlertChangedSubscriptionConfiguration

ManagementGroup   : SCOMTestLab

Name              : Subscription168e29fd_a8e5_4ee4_956f_d9591b845475

DisplayName       : AD DailyTasks Report

Description       : +CRITERIA = Alert Name Contains ‘Proactive DailyTasks AD

Team Report’     +RESOLUTIONSTATE = (0)

New    +SUBSCRIBERS = AD team, USER Kevin Justin via EMAIL     +CHANNEL = SMTP Email

Actions           : {SMTPAction_a6a5314d_83f5_47c0_910a_e60040b4c808}

ToRecipients      : {USER <blank> via EMAIL, USER <blank> via EMAIL, USER

<blank> via Email, USER <blank> via EMAIL…}CcRecipients      : {}

BccRecipients     : {}

Enabled           : True

Id                : bd52bb72-3fde-9d7f-6214-b9a47a311896

ManagementGroupId : 001b9265-3c9f-816c-aa36-a8687c05be8e

 

Update the subscription

Depending on the subscription criteria, you may need to adjust the classes, or rules/monitors, or even the criteria (properties)

 

Example

Using Subscription Description for more details into what is filtered, who alert is delivered to, and the channel used

SubscriptionDescription

 

Example

Expanding Subscription Criteria to see details into what criteria is filtered for subscription

SubscriptionCriteria

SubscriptionCriteria

See previous blogs for the best practice / how to set up subscriptions to show useful data without all the clicks

Subscription set up guide

Docs article How to Create Notification Subscriptions | Microsoft Docs

SCOM 2016 web console hot fix released

Burglar stealing a monitor
Security hotfix for SCOM 2016 web console released before your information is stolen

SCOM 2016 web console hot fix

Security teams may be contacting you for CVE-2020-1331 vulnerability on the 2016 web console.  In my example, the Tenable scanner listed ALL SCOM management group servers – under SCOM2016/2019).

NOTE KB does not install on server, so does not show up under ‘Installed Updates’

 

 

Background

HotFix DLL comes with a readme to replace the DLL for the SCOM 2016 WebConsole role

If you don’t already know this, the roles each get their own directory on your SCOM server

Security scanners run scripts to help validate if system is vulnerable.  It is possible that the scanner is just looking for some string for the install of SCOM, NOT the actual role that is vulnerable.

 

SCOM 2016 typically installs @ (‘\Program Files\Microsoft System Center 2016’)

SCOM 2019 typically installs @ (‘\Program Files\Microsoft System Center’)

 

 

Identify SCOM roles

Open PowerShell window to identify roles

cd “D:\Program Files\Microsoft System Center 2016\Operations Manager”

 

 

Resolve Web Console vulnerability

High level steps

Download the KB here

Execute KB

Copy dll and readme file

Backup DLL and replace

Reboot server

Contact Security Team to re-scan server

 

Mitigate vulnerability

Download the KB here

Extract downloaded the KB

Click Run to extract, and list extraction path

Click Run for Security Warning pop-up
Click Run for Security Warning pop-up

Copy Windows Explorer Path you want to extract to, and paste in the path

Example

S:\MonAdmin\MSDN images\SCOM\2016\WebConsole HotFix

Enter path to extract Hot Fix
Extract Hot Fix

 

Copy current DLL & replace with hotfix DLL

Open PowerShell window (as admin)

# Backup DLL

# Change Drive letter if you hopefully installed SCOM on D: drive (non-system drive)

copy “C:\Program Files\Microsoft System Center 2016\Operations Manager\WebConsole\MonitoringView\bin\Microsoft.EnterpriseManagement.OperationsManager.MonitoringViews.dll”  “C:\Program Files\Microsoft System Center 2016\Operations Manager\WebConsole\MonitoringView\bin\Microsoft.EnterpriseManagement.OperationsManager.MonitoringViews-old.dll”

# Replace DLL

copy “C:\MonAdmin\Microsoft*.dll” “C:\Program Files\Microsoft System Center 2016\Operations Mana
ger\WebConsole\MonitoringView\bin”

 

# Verify

cd “D:\Program Files\Microsoft System Center 2016\Operations Manager\WebConsole\MonitoringView \bin”

gci Microsoft.EnterpriseManagement.Operations*.dll

 

Sample screenshot from Windows Explorer view of Bin directory for replaced DLL – Same size, only timestamp changes July Page 4

Windows Explorer window showing DLL's
Windows Explorer window showing DLL’s

Reboot server

 

Test WebConsole functionality

Verify from SCOM Console > Administration Tab > Settings > Web

Find Console URL
Find Console URL

Example

http://16ms01/OperationsManager

 

Contact Security Team to re-scan SCOM asset(s)

 

 

References

CVE-2020-1331 https://portal.msrc.microsoft.com/en-US/security-guidance/advisory/CVE-2020-1331

 

Microsoft Support article

https://support.microsoft.com/en-us/help/4566040/prevent-javascript-injection-in-operations-manager-2016-web-console

Tenable forum post https://community.tenable.com/s/question/0D53a000074LGapCAG/plugin-137369-security-updatesfor-microsoft-system-center-operations-manager

 

Identify orphaned agent properties

Detective investigating items under a magnifying glass

 

Back again, I’m going to ‘Identify orphaned agent properties’.  For instance, does an agent still show up under Windows Computer, or more classes, like Windows Operating System?  Typically we have handled this by using Holman’s purge blog.

 

 

 

Deleting and Purging data from the SCOM Database

 

 

First, my thanks to Kevin H, Mihai S from the SCOM PG, & Premier Support CSS, for their help.  Let’s begin the ‘Identify orphaned agent properties’ discussion with ‘how’.  First, how do you get an orphaned property?  Second, how to you resolve?

 

Some example scenarios

    1. Server rebuilt with same name.  New agent runs discovery, and creates new set of GUID’s in the database.
    2. The Monitoring Tab > Windows Computer view contains unhealthy <gray> server objects.  Upon further inspection, the server does NOT show up in the Administration > Agent Managed view.
    3. Custom management pack authoring extends the Windows Computer class, or others (via SDK or PowerShell)

 

‘Identify and resolve’ orphaned agent properties

 

    1. Check for COMMIT or Overrides in management packs

PG recommended looking at Windows Computer extended class properties, and Connector Framework discoveries.

Microsoft.EnterpriseManagement.ConnectorFramework.IncrementalDiscoveryData.Commit()

or

Microsoft.EnterpriseManagement.ConnectorFramework.IncrementalDiscoveryData.Override()

 

Search for the ConnectorFramework

Search management packs (MP) via SCOM OpsDB (OperationsManager Database)

    1. Login to your SCOM OpsDB > New Query

select MPName, convert(xml, MPXML)

from ManagementPack

where

   MPXML like ‘%Commit(%’ or

   MPXML like ‘%Override(%’

Export management pack output or snag it/snippet screenshot

Example Snapshot from SQL query

SQL Query output of Management Pack output with Commit or Override
SQL query of MP Commit or Override pack matches

FYI – mgmt packs above use %Commit(%, but not the connectorFramework

 

Correct discoveries that use ConnectorFramework

Replace Discoveries

Update discoveries that contain:

New-Object Microsoft.EnterpriseManagement.ConnectorFramework.IncrementalDiscoveryData.Commit()

New-Object Microsoft.EnterpriseManagement.ConnectorFramework.IncrementalDiscoveryData.Override()

Replace with:

New-Object -comObject MOM.ScriptAPI for discovery

 

Test discoveries that use Remove method

Microsoft.EnterpriseManagement.ConnectorFramework.IncrementalDiscoveryData.Remove()

 

 

 

Example management pack discovery script

Contains

$discovery = New-Object Microsoft.EnterpriseManagement.ConnectorFramework.IncrementalDiscoveryData

$discovery.RemoveInternal($Instance,$ClassInstance.GetClasses()[0])

$discovery.Commit($mg)  <– This is the offender that causes the orphans

}

 

Don’t forget python as pre-req for agent install

Grocery List, items to get and notes
Grocery List

 

Hey guys, don’t forget python as pre-req for agent install!  Came across this again, where the docs site doesn’t mention python-ctypes as pre-req for agent install.  Let’s flip to GitHub for the agent.  GitHub lists the python pre-req here.  Otherwise, it’s Openssl 1.1.0 is only supported on x86_64 platforms (64-bit).

 

 

Let’s begin by starting with a Linux server.  I’ve used Ubuntu in my lab, specifically, Ubuntu v16.04.

Login and check if you have latest, or have the package installed (Debian Linux)

Regular user run   ‘sudo apt-get install python-ctypes

Super user/root     ‘apt-get install python-ctypes

 

screen capture of python install

 

As of 1 July, v1.13.7.0 is current (latest) 64 bit OMS for Linux agent released.

 

 

 

 

 

References

GitHub link https://github.com/Microsoft/OMS-Agent-for-Linux

GitHub Agent Download (AzMon/ALA/OMS/SCOM agent for Linux ) https://github.com/microsoft/OMS-Agent-for-Linux/releases/download/OMSAgent_v1.13.7-0/omsagent-1.13.7-0.universal.x64.sh

Python requirements https://github.com/Microsoft/OMS-Agent-for-Linux#python-requrements

Install guide https://github.com/Microsoft/OMS-Agent-for-Linux#azure-install-guide

Installing and configuring the MMA agent via Command line

Command prompt

 

GUI install option, see blog

PowerShell Agent configuration, see blog

Updated 1 Feb 2023

Pre-reqs to build out an install script/package

MMA agent executable

ALA Workspace ID

ALA Workspace Primary Key

 

 

Download MMA agent

Click on Windows Servers from Connected Sources to download Windows Agent

Click on Linux Servers from Connected Sources to download Linux Agent

 

 

 

 

Obtain WorkspaceID

From the Azure Portal (https://portal.azure.com)

Click on Log Analytics, <your subscription >

Click on Advanced Settings

My view defaulted to Connected Sources > Windows Servers

 

Save the workspace ID and workspace key to notepad/OneNote for later

 

 

 

 

 

Build out command line for setup file

(optionally to include in Application Deployment package)

 

Grab pre-reqs above: (saved from above to build the command line)

Exe/msi file

Workspace ID

Workspace key

 

Craft out your command line (MECM super installer code updated by Neal Smith

SCOM MECM Agent Package Installer Command Line

The setup.exe or MSI command line parameters to pass are:

MMA-specific optionsNotes
NOAPM=1Optional parameter. Installs the agent without .NET Application Performance Monitoring.
ADD_OPINSIGHTS_WORKSPACE1 = Configure the agent to report to a workspace
OPINSIGHTS_WORKSPACE_IDWorkspace Id (guid) for the workspace to add
OPINSIGHTS_WORKSPACE_KEYWorkspace key used to initially authenticate with the workspace
OPINSIGHTS_WORKSPACE_AZURE_CLOUD_TYPESpecify the cloud environment where the workspace is located

0 = Azure commercial cloud (default)

1 = Azure Government

OPINSIGHTS_PROXY_URLURI for the proxy to use
OPINSIGHTS_PROXY_USERNAMEUsername to access an authenticated proxy
OPINSIGHTS_PROXY_PASSWORDPassword to access an authenticated proxy

Example:

setup.exe /qn NOAPM=1 ADD_OPINSIGHTS_WORKSPACE=1 OPINSIGHTS_WORKSPACE_AZURE_CLOUD_TYPE=0 OPINSIGHTS_WORKSPACE_ID=<your workspace id> OPINSIGHTS_WORKSPACE_KEY=<your workspace key> AcceptEndUserLicenseAgreement=1

 

 

 

Other helpful links

Docs site https://docs.microsoft.com/en-us/azure/log-analytics/log-analytics-quick-collect-windows-computer

Daniel Orneling Blog https://blog.orneling.se/2017/01/installing-oms-agent-with-powershell/

TechNet gallery https://gallery.technet.microsoft.com/scriptcenter/Install-OMS-Agent-with-2c9c99ab

SQL MP bloat

Updated 25 Feb 2023

 

Ever wish alerts were like a wad of cash?

The more you solve, the more you make!

 

How about performance counter data?

 

 

The SQL management packs are awesome for visualizations, and provide a bunch of data.

 

Tim McFadden pointed out SQL Performance counters https://www.scom2k7.com/crazy-db-performance-collection-rules-in-the-sql-mps/

His blog brings up SQL MP Disk Latency performance counters.

 

His blog got me thinking about SQL DB and DB file design, where multiple DB files are on the same Drive, causes duplicate performance counters (SCOM workflows) on the agent, and will typically be one of the culprits for HealthService restarts.

 

SQL MP creates performance counters (per DB file, group, instance, engine)

 

Let’s start with how I figured out why all my money goes into storage.

 

Start in the SCOM console

Click on the Reporting Tab

Click on the ‘System Center Core Monitoring Reports’ folder

Double click on the Data Volume by Management Pack

View of SCOM report from console reporting tab

Select the timeframe (from, to)

Click Run

Data Volume MP selected

 

Reporting Data

I have 2 2016 DB’s and 1 2014 (SCVMM) database server monitored, and it’s 50% of my data volume!

 

 

 

Another example – had the DW shutdown for days

Data volume of SQL after

 

Did you know there are 60+ perf counter rules in 2012 alone, and nearly 200 in 2016?

 

How about an OFF pack, a management pack that turns off all the performance counter rules?

The monitors still exist for health, just no pretty performance graph, should you look.

 

 

Github repo link

Check out the Gallery post for download

TechNet gallery download

 

Zip file contains

  1.  OFF MP’s for 2008,2012,2014,2016
  2. XLS sheets to allow you to go to the SQL team and ask them what performance counters they use