Using Unix MP’s for Shell commands and scripts

Ready to move out of the UI ?

Thanks to Saurav Babu, and Tim Helton’s help, I was able to push my MP authoring limits further.

The good thing with the Shell command template in SCOM is that your script is encoded.

Bad news

  1. If functionality doesn’t exist in the UI, you can’t easily pull the monitor and just add variables to get that functionality.
  2. Scripts and Shell commands are encoded (great news for security!)

Now to the use case – need Sample Count and Match Count to prevent false positive alerts

The UNIX Shell Command library allows us to use the following variables out of the box:

Interval, SyncTime, TargetSystem, UserName, Password, Script, ScriptArgs, TimeOut, TimeOutInMS, HealthyExpression, ErrorExpression

AND we can override Interval, Script, TimeOut, TimeOutInMS

If that’s not enough options, then read on!

When the built-in functionality doesn’t exist

For this UNIX shell command/script monitor, we required SampleCount and MatchCount

Variables explained

SampleCount is the number of times (samples for an alert).

If SampleCount = 4, this means 4 samples will generate an alert

MatchCount is the number of intervals before monitor state changes

If Interval = 60 (s), and MatchCount = 10, then it will take 10 minutes (600s before we alert)

Combining the 2 means 4 samples over 10 minutes will generate an alert.

Sometimes this is called alert suppression or counting failures before alerting

Built a custom DataSource, ProbeAction, and WriteAction, as the UNIX Shell Library MP did not include these additional variables.

Please review my updated MP Fragments TechNet Gallery for the custom MP and fragments!

https://gallery.technet.microsoft.com/Uncommon-Custom-MP-c5a12a86

Encoding the script or command to run

The other issue with UNIX scripts and commands, is the UI encodes the scripts.

How do we get around it you ask?

Since we are building an MP Fragment and MP, we must figure out how to encode.

To encode the script to put into your SCOM monitor (and MP Fragment)

Example

$script = ‘if [ `ps -ef | grep sleep | grep -v grep | wc -l` -eq “1” ]; then echo false; else echo true; fi’

# Verify script variable
$script

# Get $script bytes
$s = [System.Text.Encoding]::UTF8.GetBytes($script)

# Verify script bytes output (optional as bytes broken out by line)
$s

# Encode script to Base64
$encoded = [System.Convert]::ToBase64String($s)

# Verify $encoded
$encoded

# Optional
# Verify string converts back properly
[System.Text.Encoding]::UTF8.GetString($s)

$encoded output is what needs to be entered into the <script></script> variable in your monitor

Example Output

PS C:\Users\scomadmin\desktop> $script = ‘if [ `ps -ef | grep sleep | grep -v grep | wc -l` -eq “1” ]; then echo false;
else echo true; fi’
PS C:\Users\scomadmin\desktop> $script
if [ `ps -ef | grep sleep | grep -v grep | wc -l` -eq “1” ]; then echo false; else echo true; fi
PS C:\Users\scomadmin\desktop> $s = [System.Text.Encoding]::UTF8.GetBytes($script)
PS C:\Users\scomadmin\desktop> $s
PS C:\Users\scomadmin\desktop> $s = [System.Text.Encoding]::UTF8.GetBytes($script)

PS C:\Users\scomadmin\desktop> $encoded = [System.Convert]::ToBase64String($s)
PS C:\Users\scomadmin\desktop> $encoded
aWYgWyBgcHMgLWVmIHwgZ3JlcCBzbGVlcCB8IGdyZXAgLXYgZ3JlcCB8IHdjIC1sYCAtZXEgIjEiIF07IHRoZW4gZWNobyBmYWxzZTsgZWxzZSBlY2hvIHRydWU7IGZp
PS C:\Users\scomadmin\desktop> [System.Text.Encoding]::UTF8.GetString($s)
if [ `ps -ef | grep sleep | grep -v grep | wc -l` -eq “1” ]; then echo false; else echo true; fi
PS C:\Users\scomadmin\desktop>

References

Jonathan Almquist’s blog post

Kevin Holman’s blog on service with Samples

Using SharePoint On Premise Diagnostic tool

Futuristic, perhaps, but a powered screwdriver for space, almost looks like Han Solo’s pistol (sans scope)

From previous intro, we start using the tool to diagnose SharePoint problems.

Let’s install, and get to using it!

Once this is released, I expect this to be posted to a GitHub repository.

For now, there will be some mystery for obtaining the file bundle.

Copy folder from build zip file.

Paste to SharePoint machine, whatever standard you use.

From my own past, I prefer a MonAdmin (Monitoring Admin) directory, with a scripts sub-directory, then toolname/version

Example

cd ‘C:\Monadmin\Scripts\OPD-D2.0.1905.15001’

Start OPD via powershell

cd ‘C:\Monadmin\Scripts\OPD-D2.0.1905.15001’

.\OPD-console.ps1

Avoid some initial questions

cd ‘C:\Monadmin\Scripts\OPD-D2.0.1905.15001’

.\OPD-console.ps1 -mode SharePoint -ShareTelemetry Yes -AcceptEula

PS C:\Monadmin\Scripts\OPD-D2.0.1905.15001> .\OPD-console.ps1 -mode SharePoint -ShareTelemetry Yes -AcceptEula

Using OPD to check SharePoint environment

Start with the OPDLog Event Log

OPD Main menu

1 – Administration

Central Admin site

Current patch level

Emails

Timer jobs

2 – Performance

3 – Search

Search Hosts Online

Unable to retrieve topology

4 – Services

5 – Setup

Binaries

6 – User Profile

Firewall ports (duplicated from section 4)

Happy checking and to building new SharePoint checks!

SharePoint Management framework Private Preview

 

Do you have an Enterprise SharePoint farms that you manage health and performance via custom scripts?

Have you used SETH to manage SharePoint 2010 problems with the farm(s)?

 

Would you want a scalable tool you can add your own scripts and enable/check, and then alert on what you want?

 

 

Background

SharePoint Engineer Troubleshooting Helper (SETH) was a Microsoft tool for SharePoint 2010

Using SETH

Troubleshooting SETH

 

 

For SharePoint 2016 and 2019, the Customer Support team brought up the need for bringing back a utility to help with common SharePoint scenarios

On Premise Diagnostic (OPD) is the second generation of project (for SharePoint 2016 and 2019).

 

My goal was to help the Escalation Engineers have a full platform that can be implemented and is scalable for the technical community to maintain and use.

 

BTW, the only thing preventing 2013 SharePoint support is the dependency on WMF v5.0 or better on SharePoint servers.

 

 

SCOM management pack can be found here

 

Updated Skype for Business 2015 Addendum pack

Continuing work with Nick Wood on the Skype pack for additional operational features.

Previously Blogged about this July 2018, and continue to make improvements

The TechNet gallery bundle is updated with new functionality.

Skype KHI addendum

Pack gathers the Skype KHI performance counters

Packets * Discards performance rules where greater than 100 discards are seen on NIC’s,

Monitoring Tab folder/performance view

Skype Custom Overrides

Includes common overrides for noisy monitors/rules.

Install SCVMM management packs from VMM Server

Time for some automation

Ever have to upgrade SCVMM packs every time a new Update Release (UR) comes out?

Copy the files off from the VMM server to your SCOM MS, install.

How long does that take?

Try this script out – assuming you have a login on the VMM Server

TechNet Gallery post here

# Set up some variables

$UR=”UR5″

$VMMServer = “16VMM01”

# Set up your path, this example is monadmin\backup

$date = Get-Date -UFormat “%Y-%m-%d”

# Set up backup path

$backupPath = “C:\monadmin\backup”

$backupDrive = “C:”

# Create some functions

Watch them roll, let PowerShell do your work!

UR6 packs

SCOM management packs backed up

Check out the SCOM Console Admin tab for updates!

Troubleshooting Service Map pack

 

 

 

Updated 14 Mar 2019

 

If you get these exceptions like me, the issue has been raised, with a deliverable targeted for SCOM2019UR1.

Disable the rule to reduce noise.

 

 

Are you using Service Map Management pack, and getting errors?

 

This alert is based on the 46651/46652 event ID in the Operations Manager event log

From SCOM Console > Authoring Tab > Management Pack Objects > Rules

Search in ‘Look for:’ bar GenericException (yes no space in between)

 

Rule

 

 

Rule Details

 

To enable debug on the MS

 

For collecting logs, please do the following:

  • Create folders “c:\Debug\ext\”
  • Now, Wait for an hour(which is the default time interval set in the rule for running service map api).
  • You will see some log files created in that folder “ext”. Please share the same in email.

 

The file showed up after the alerts, and listed debug INFO and WARN lines, and the time stamps match up to the generic exception rules.

 

Stay tuned for more information, I have been trying to get more answers on the exception

{WARN} [12:35:20.966] [ScomUtils] failed to export XML for Management Pack: System.NullReferenceException: Object reference not set to an instance of an object.

   at ScomBridge.ScomUtils.WritePackXmlToFile(ManagementPack pack, String filename)

 

 

XML for Product or Company Knowledge

Digging in the archives…

 

 

From a discussion with some PFE’s – the question was ‘how do I create knowledge for a monitor/rule?’

Tyson Paul pointed out the system Center Wiki  ‘Knowledge Article authoring’  

 

When you create a knowledge article in an MP (let’s not even go into the console GUI! )

If the Knowledge Article references a sealed workflow (does it reference a sealed pack)

It’s Company Knowledge

 

 

 

 

Example

If the Knowledge Article references a sealed monitor, it will show up under the ‘Company Knowledge’ tab

XML example from Skype Addendum pack on TechNet Gallery

<KnowledgeArticles>
<KnowledgeArticle ElementID=”ML2MC!Microsoft.LS.2015.Monitoring.Internal.Health.DiscoveryRunner” Visible=”true”>
<MamlContent>
<maml:section xmlns:maml=”http://schemas.microsoft.com/maml/2004/10″>
<maml:title>Summary</maml:title>
<maml:para>Any added Skype servers will not be discovered in SCOM.</maml:para>
</maml:section>
<maml:section xmlns:maml=”http://schemas.microsoft.com/maml/2004/10″>
<maml:title>Causes</maml:title>
<maml:para>Discovery Failed.  An internal exception has occurred during discovery.</maml:para>
</maml:section>
<maml:section xmlns:maml=”http://schemas.microsoft.com/maml/2004/10″>
<maml:title>Resolutions</maml:title>
<maml:para>Fix permission issues in alert.</maml:para>
<maml:para>Skype PowerShell module may not be installed.</maml:para>
<maml:para>Import-Module SkypeForBusiness</maml:para>
</maml:section>
</MamlContent>
</KnowledgeArticle>

</KnowledgeArticles>

 

 

 

If the Knowledge Article is referenced in a sealed pack, OR an UNsealed pack has a rule/monitor in the same unsealed pack)

It’s Product Knowledge

 

Sealed pack example

 

Unsealed pack Example

Visual Studio and VSAE updates

 

For those using Visual Studio and Authoring Extensions to author management packs

 

 

Background

VSAE is Visual Studio Authoring Extensions, used to create management packs (with or without fragments)

See Holman’s Fragment library on TechNet Gallery for multiple hints at more than what SCOM delivers out of box (OoB)

 

 

 

 

If you or your customers are having issues after upgrading to VS2017 15.8.x.y. and above…there’s a new v1.3.0.1 VSAE download

NOTE – New VSAE updates are applicable to Visual Studio 2017/2015/2013/2012

View full features at the download link:

 

 

If you upgraded Visual Studio, and need to check the VSAE version

From Visual Studio, choose Help > About Microsoft Visual Studio

 

In my experience, I had to remove System Center Visual Studio Authoring Extensions’ to upgrade (I was running the private patch).

 

 

After upgrade, the version information:

Control Panel > Programs > Programs and Features

 

 

Visual Studio 2017

 

 

 

 

 

Visual Studio errors seen when VSAE is not updated

 

Unresponsive VSAE (shows as Visual Studio component ‘System Center Authoring Extensions’)

 

Exception Encountered

 

 

Missing .NET Frameworks on the system where Visual Studio runs

 

Missing Visual Studio components

 

 

 

Another warning

In Visual Studio, be careful to dock panes, if you’ve accidentally dragged the panes around/on the screen.

Right click on the Pane, and choose dock.

 

 

 

If you still have problems, upgrade to latest Visual Studio, open a Premier support case.

There is also UserVoice for Visual Studio feedback https://visualstudio.uservoice.com/forums/121579-visual-studio-ide
You can also upload pics to help illustrate the UI question.

 

 

 

References

Install VS2017 https://docs.microsoft.com/en-us/visualstudio/install/install-visual-studio?view=vs-2017

Workloads https://docs.microsoft.com/en-us/visualstudio/install/install-visual-studio?view=vs-2017#step-4—select-workloads

Gather Log Analytics/MMA agent version

Had some questions come up from the community to check the Log Analytics agent version.

Depending on how you are setup, the SCOM Integration makes this easy with Holman’s blog for the agent management pack.

If you have admin right in Operations Manager console then you can check this directly from SCOM server:

If you are an admin in SCOM, you can check from MS

$Server = “DC01.yourlabnamehere.net”
(Get-SCOMAgent -Name $ServerName).Version

Alternatively, from server registry:

(Get-ItemProperty “HKLM:\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\setup”)

# Just the Agent version variable

(Get-ItemProperty “HKLM:\SOFTWARE\Microsoft\Microsoft Operations Manager\3.0\setup”).AgentVersion

Log Analytics

Kusto query

// Servers and Versions

Heartbeat
| project Computer,Version

// Specific version

Heartbeat
| where Version == “8.0.10918.0”
| project Computer,Version

// Summarize by Version

Heartbeat
| summarize by Version

If you’re visual

From the Portal > Log Analytics > workspace > Workspace Summary > Agent Health

Scroll right to agent version

Monitor

Monitor > Overview > Agent Health Assessment

Scroll right to agent version

Azure Log Analytics for Windows Telemetry data

 

 

I blogged about this last year here

 

 

As best practice, the Upgrade Analytics script checks for far more than just injecting the workspace key and telemetry value.

 

 

FYI – This could also be managed in an SCCM Compliance setting.

Paul Fitzgerald – Platform PFE blogged about a non SCCM method here

 

 

Assess requirements for environment:

 

Barebones configuration requires Commercial ID, allow telemetry, and level of telemetry data to send

Optional – Create key for IEDataOptIn

Send data to Application Insights

Customer proxy setup

 

 

Script has 11 parameters specified, not all are needed (excerpt below from script)

Param(
# run mode (Deployment or Pilot)
[Parameter(Mandatory=$true, Position=1)]
[string]$runMode,

# File share to store logs
[Parameter(Mandatory=$true, Position=2)]
[string]$logPath,

# Commercial ID provided to you
[Parameter(Mandatory=$true, Position=3)]
[string]$commercialIDValue,

# logMode == 0 log to console only
# logMode == 1 log to file and console
# logMode == 2 log to file only
[Parameter(Mandatory=$true, Position=4)]
[string]$logMode,

#To enable IE data, set AllowIEData=IEDataOptIn and set IEOptInLevel
[Parameter(Position=5)]
[string]$AllowIEData,

#IEOptInLevel = 0 Internet Explorer data collection is disabled
#IEOptInLevel = 1 Data collection is enabled for sites in the Local intranet + Trusted sites + Machine local zones
#IEOptInLevel = 2 Data collection is enabled for sites in the Internet + Restricted sites zones
#IEOptInLevel = 3 Data collection is enabled for all sites
[Parameter(Position=6)]
[string]$IEOptInLevel,

[Parameter(Position=7)]
[string]$AppInsightsOptIn,

[Parameter(Position=8)]
[string]$NoOfAppraiserRetries = 30,

[Parameter(Position=9)]
[string]$ClientProxy = “Direct”,

[Parameter(Position=10)]
[int]$HKCUProxyEnable,

[Parameter(Position=11)]
[string]$HKCUProxyServer

 

 

 

Simple method to update machines to send Windows telemetry data:

 

 

PowerShell script

From PowerShell as Administrator

Set-Location HKLM:

 

$registryPath = “HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies”

$Name = “DataCollection”

$Name2 = “AllowTelemetry”

$CommercialID = “00000000-0000-0000-0000-000000000000”

$value = “2”  # Values from 0-3 accepted

$vIEDataOptInPath = “HKLM:\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\DataCollection”

$IEOptInLevel = “2”  # Values from 0-3 accepted

 

If ( (Test-Path $registryPath\$Name) ) { write-host -f green “Registry keys already exist” }

If ( ! (Test-Path $registryPath\$Name) )

{

New-ItemProperty -Path $registryPath -Name $name

New-ItemProperty -Path $registryPath -Name $CommercialID

New-ItemProperty -Path $vIEDataOptInPath -Name IEDataOptIn -Type DWord -Value $IEOptInLevel

New-ItemProperty -Path $registryPath\$Name -Name $name2 -Value $value `

    -PropertyType DWORD -Force | Out-Null

Write-host -f green “Registry keys added for Telemetry”

}

 

 

 

 

References

Configure telemetry

Get Started link

Win 7,8 Opt in link