Exchange Message Queue Monitoring

Q ?

Hopefully something else popped into your mind, not Star Trek – The Next Generation

 

 

How about monitoring a message queue?

 

 

This will help you monitor specifically Exchange 2013-2016 queue changes (state or message count beyond a value)

 

Just specify the queue and encapsulate into a monitor

OR… download the MP Fragment here

############################################################

# Test Queues

#Lab environment

#$queue1 = “Submission”
#$queue2 = “Submission2”

$queue1 = “CompanyName365-mail-onmicrosoft-com.mail.protection.outlook.com”
              $queue2 = “clustered.testlab.net”

##################################################################
# 1. How to monitor the status of the queues below (when the status is “retry” they need alert)

$SPEvent = ( get-queue | where { $_.NextHopDomain -like “$queue1” } | where { $_.Status -like “Retry” } | measure-object )
$SPEvent2 = ( get-queue | where { $_.NextHopDomain -like “$queue2” } | where { $_.Status -like “Retry” } | measure-object )

#
# 2. To get an alert when a message in any of these queues is stuck for > 5minutes
#

$SPEventStuck = ( get-queue | where { $_.NextHopDomain -like $queue1 } | where { $_.Status -like “Suspended” } | measure-object )
$SPEventStuck2 = ( get-queue | where { $_.NextHopDomain -like $queue2 } | where { $_.Status -like “Suspended” } | measure-object )

# 3. Also need alert and exact count when the message count in the above queue goes beyond 1000

$SPEventMC = ( get-queue | where { $_.NextHopDomain -like $queue1 } | where { $_.MessageCount -gt 1000 } | measure-object )
$MCQ1 = ( get-queue | where { $_.NextHopDomain -like $queue1 } )
$MQ1 = ($MCQ1.MessageCount)

$SPEventMC2 = ( get-queue | where { $_.NextHopDomain -like $queue2 } | where { $_.MessageCount -gt 1000 } | measure-object )
$MCQ2 = ( get-queue | where { $_.NextHopDomain -like $queue2 } )
$MQ2 = ($MCQ2.MessageCount)

 

 

Have fun!

Exchange 2013 monitoring Addendum

This is a good source for additional Exchange Server 2013+ monitoring, brought to my attention from Exchange PFE Dave Groll!

 

See Volkan Coskun’s blog post here

Update 30 Nov 2018

NOTE this content is no longer available – posted Volkun’s custom extension pack on TechNet Gallery here

 

 

This will help provide additional visibility to individual objects like Mailbox Databases or Transport Queues that are otherwise hidden in the health sets.

NOTE my Lab environment is Exchange 2016

PowerShell Monitor Fragment with Run As

Stop!

 

Ever need to run a PowerShell command (or script) as a specific ID?

Maybe you need to know when the command fails to catch degraded application health?

 

Let’s work into the scenario with a MP Fragment

 

Download the latest fragments here

Load Test MP fragment

Shout to Tyson Paul for his initial MP with 2016, catch his blog here!

 

This should help speed up building the MP if you have Visual Studio 2013 or 2015 with Visual Studio Authoring Extensions (VSAE).  Read Kevin Holman’s blog if this is new

 

Download the latest fragments here

 

Quicker method To build the MP as MP fragment, update the following:

  1. Import MP Fragment into Visual Studio for MP
  2. Replace variables in fragment
    1.   ##CompanyID##
    2.   ##AppName## – LoadTesting
    3.   ##ClassID## – WindowsServer
    4.   ##OSVersion## – 2012 = 6.2, 2012R2 = 2012.R2, 2016 = 100
    5.   ##MPReferenceID## – MWS2M for 2012, MWS2RM for 2012R2
    6.   ##RuleGUID##
  3. Obtain Rule names for Override Targets and Report rule GUID’s for the report parameters section

          get-scomrule | ? { $_.DisplayName -like “System Processor Queue Length*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “Current Disk Queue Length*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “Current Disk Queue*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “Current Dis*k Queue*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “*Current Disk Queue*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “*Average Disk Seconds Per Transfer*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “*Logical Disk Idle Time*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “*Processor Time Total*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “*Memory Available Megabytes*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “*Network Adapter Bytes Total*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “*Memory Pages per Second*” } | fl ID,DisplayName,Name

          get-scomrule | ? { $_.DisplayName -like “*System Processor Queue Length*” } | fl ID,DisplayName,Name

  1. Save and Import MP into environment
  2. Get Report parameter value for group ID

          get-ScomGroup | ? { $_.DisplayName -like “*Load Testing Group*” } | fl ID,DisplayName

  1. Update ##TargetGroupID##, MP version
    1. Save MP, and import into environment
  2. From SCOM Console, Authoring Tab
    1. Update group with explicit members
    2. Verify Group members
  3. From SCOM Console, Monitoring Tab
    1. Verify Performance view has performance counters
  4. From SCOM Console Reporting Tab (this may take a few minutes to push report to Reporting server)
    1. Open report and run
    2. Export data for analysis

Load Test MP with Report

Read below if you want a specific MP for load testing

 

I don’t know about you, but I’ve come across the situation where you need to compare performance.

 

This MP should help validate performance, whether to validate physical versus virtual, or a new Server Farm, storage performance between environments, etc.

 

Shout to Tyson Paul for his initial MP with 2016, catch his blog here!

 

 

Let’s start with the MP shell

 

Check out the upcoming Gallery download for MP’s and fragments here

NOTE: GUID’s will vary to your environment

 

To build the MP, you will need to update the following:

 

Pack ID to include the OS version

 

Add Reference for OS Monitoring MP (can use Visual Studio (VS) MP alias if VS is available for use in your environment)

Class Type ID with OS version

Discovery ID and Target with OS Version

 

Overrides ID with OS Version, and Target with OS MP Rule name and reference

Use the OS Monitoring MP to help with the rule names

The far right of the Override lists the Rule that must match to the OS MP

 

View ID with OS Version

 

Folder Item Element ID and ID

 

MP Display Strings with OS Version

 

Save and Import MP without the Report parameter to see what Group ID SCOM assigns the Load Test group

 

 

Part 2 – include report after group is imported

Don’t forget to update MP version under Identity!

Get Report parameter value for group ID

From PowerShell

          get-ScomGroup | ? { $_.DisplayName -like “*Load Testing Group*” } | fl ID,DisplayName

Alternatively, obtain from SQL SSMS

          select [ManagedEntityDefaultName],[ManagedEntityRowId]

          FROM [vManagedEntity]

          where [ManagedEntityDefaultName] like ‘%load%’

          order by displayname

SSMS Output

 

In MP XML, update View Target GUID to your Group ID

 

Add Report section, and update parameter values Rule GUID

 

PowerShell commands to run from MS or console installed machine

The GUID’s needed for the report parameters section

 

get-scomrule | ? { $_.DisplayName -like “System Processor Queue Length*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “Current Disk Queue Length*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “Current Disk Queue*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “Current Dis*k Queue*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “*Current Disk Queue*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “*Average Disk Seconds Per Transfer*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “*Logical Disk Idle Time*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “*Processor Time Total*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “*Memory Available Megabytes*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “*Network Adapter Bytes Total*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “*Memory Pages per Second*” } | fl ID,DisplayName,Name

get-scomrule | ? { $_.DisplayName -like “*System Processor Queue Length*” } | fl ID,DisplayName,Name

 

 

Verification

  1. From SCOM Console, Authoring Tab
    1. Update group with explicit members
    2. Verify Group members
  2. From SCOM Console, Monitoring Tab
    1. Verify Performance view has performance counters
  3. From SCOM Console Reporting Tab (this may take a few minutes to push report to Reporting server)
    1. Open report and run
    2. Export data for analysis

Have a lot of SCOM subscriptions?

You logged into SCOM because someone said they didn’t get a notification

Looked at subscriptions and it’s blank

 

Do you know if you backed up your subscriptions?

 

Hopefully, you read this and back up your management packs

Hope you read this blog, and it saved your life!

 

For those who love the Console UI

You can manually back up the MP

From the SCOM console

Click on the Administration Tab

Click on Management Packs (2012), or Installed Management Packs (2016)

Type ‘notification’ in the ‘Look for:’ bar

Hit Enter

Highlight the ‘Notifications Internal Library’ pack

Click on Export Management Pack

Name your path

Hit OK

 

For simple PowerShell

 

# Backup Management packs to C drive

# Set up your path, this example is monadmin\backup

$date = Get-Date -UFormat “%Y-%m-%d”

c:

cd monadmin\backup

new-item -itemtype directory -path c:\monadmin\backup\$date

cd $date

Get-SCOMManagementPack -Name *Notification* | Export-SCOMManagementPack -Path “C:\monadmin\backup\$date”

 

Subscription set up guide

Thought it funny when asking a child ‘what is a subscription?’ and the response isn’t that for movies, videos, streaming and such!?

My first thought was a magazine or newspaper.

 

…Anyways!

 

Somewhere in the middle of a Service and product, you need to know vital information

 

 

To set up a new subscription for the new monitors

Prerequisites:

SCOM Administrator ID required for update

Channel name

Subscriber Name set up in the SCOM environments for notifications

 

Log in to the SCOM Console

Click on Administration Tab

Expand Notifications

Click on Subscriptions

Click on New on top right hand pane

 

Enter Subscription name – Company AppID Environment Problem Description

Example TestLab Queue Monitors (Retry, Suspended, MessageCount GT 1000)

 

Recommend reviewing subscription naming convention to help leverage the description field for quick subscription criteria visibility

https://kevinjustin.com/blog/2017/04/24/building-a-subscription/

 

Update the description for SUBSCRIBERS and COMMENTS based on requirements

 

Sample description

+MONITOR = Exchange2013 Message Count, Exchange2013 Queue in * +SEVERITY = Critical +SUBSCRIBERS = JustinTime NOC team DL + Exchange support via Email +COMMENTS Created 2017-06-13 for Additional Exchange alerting

Click Next

 

Click the checkbox ‘created by specific rules or monitors’

In the bottom pane, click on the hyperlink blank

 

In the Popup window for Monitor and Rule Search

Type Exchange

If that is too broad, type in the name of the first monitor

(repeat for the second and third monitors)

Monitor names

Exchange Message Count

Exchange Queue in

Click on Search button for each search

 

Highlight the monitor(s) from the output

Click Add

Verify they are in the ‘Selected rules and monitors:’ pane

Click OK

 

Verify the 3 monitors are in the ‘created by hyperlinked data’

Click Next

 

Click Add to add a subscriber

 

Type in Subscriber name

Click Search

Highlight Available subscriber

Click Add

Verify subscriber is in ‘Selected subscribers’ box

Click OK

 

Verify that subscriber shows up in subscription wizard

Click Next

 

Click on Add

 

Type in channel name

Click Search

 

Select from ‘Available channels’

Click Add

 

 

Verify channel is in ‘selected channel’ pane

Click OK

Click Next to verify summary

Make sure ‘Enable this notification subscription’ checkbox is checked

Click Finish

 

Get to know your monitor

Ever need to disable a specific monitor?

I know I get tired of clicking through the console, maybe you do too?
Do you know the Monitor name and class?
If yes, then you can enable/disable monitors from PowerShell

 

So let’s get started.

From your management server, you can run SCOM commands as your ID (assuming your ID is set up in SCOM)

 

This example has 2 purposes:

  1. SQL2016 SP1 does NOT populate the proper fields, and will be fixed in SP2 per the SQL Engineering blog (Look at comments section – blog here)
  2. Tired of the warning alerts in my SCOM console

 

Find the monitors

$Monitor = get-scommonitor | where { $_.DisplayName -like “Service Pack Compliance” } | where { $_.Name -like “*Microsoft.SQLServer.2016.DBEngine*” }

 

Let’s focus for a second on some differences, and how you can interchange the two depending on what information you know

DisplayName attribute is what you see in the console (note the spaces)

Name attribute typically has dots for the spaces

 

Override a class

Disable-SCOMMonitor -Class $Class -ManagementPack $MP -Monitor $Monitor

Just in case you need to undo the override

Enable-SCOMMonitor -Class $Class -ManagementPack $MP -Monitor $Monitor

 

Override a group

$Group = (Get-SCOMGroup -DisplayName “Group*”)

 

# Enable the group

Enable-SCOMMonitor -Group $Group -ManagementPack $MP -Monitor $Monitor

 

# Disable the group

Disable-SCOMMonitor -Group $Group -ManagementPack $MP -Monitor $Monitor

 

 

Reference Links

Disable-SCOMMonitor https://docs.microsoft.com/en-us/powershell/systemcenter/systemcenter2016/operationsmanager/vlatest/disable-scommonitor

Enable-SCOMMonitor https://docs.microsoft.com/en-us/powershell/systemcenter/systemcenter2016/OperationsManager/vlatest/Enable-SCOMMonitor

PowerShell Rule and Monitor Template packs MP including fragments

 

Hit the easy button!

 

For all those diehard SCOM Console MP authoring folks, don’t forget about Wei Lim’s blogs to help add PowerShell script functionality into rules and monitors.

 

PowerShell Rules Blog https://blogs.msdn.microsoft.com/wei_out_there_with_system_center/2015/09/28/opsmgr-new-sample-powershell-collection-rule-wizards-in-the-ops-console/

PowerShell Monitor blog https://blogs.msdn.microsoft.com/wei_out_there_with_system_center/2015/07/09/opsmgr-new-sample-wizard-to-create-powershell-monitors-in-the-ops-console/

Performance Data blog https://blogs.msdn.microsoft.com/wei_out_there_with_system_center/2015/10/03/opsmgr-collecting-performance-data-using-a-powershell-script-collection-rule-created-from-a-wizard/

Download Rule https://gallery.technet.microsoft.com/Sample-Management-Pack-e48040f7

Download Monitor https://gallery.technet.microsoft.com/Sample-Management-Pack-17b76379

 

If authoring with Visual Studio or Notepad++, don’t forget Holman’s MP fragments!

Discover Class = Class.And.Discovery.Script.PowerShell.mpx

Monitor Timed Script PowerShell = Monitor.TimedScript.PowerShell.mpx

Monitor Timed Script SQL Query PowerShell = Monitor.TimedScript.PowerShell.SQLQuery.mpx

PowerShell Performance Rule = Rule.Performance.Collection.PowerShellScript.mpx

 

Download fragments here

 

Happy MP Authoring!