PowerShell Archives - Page 4 of 6

SCOMCore Addendum pack

Time to configure the Microsoft System Center Core Monitoring pack per health model and best practice. That’s where the SCOMCore Addendum pack comes in. Addendum adds High Agent Handle count group, daily report and alert closure automation, and rule/monitor overrides. Some assembly required – update the discovery pattern for offending high handle counts, and high handle count group ContextInstance GUID after import.

Quick Download: https://github.com/theKevinJustin/SCOMCoreAddendum

Background:

While High Agent Handle count was more an issue before the x365 platform migrated UC, SharePoint, and email (i.e. Lync/Skype, SharePoint, Exchange on prem) went to the cloud. This is still seen where cloud scalability options and virtualization/storage limitations exist. Example typically is an over-utilized virtual machine in hybrid/IaaS/premise scenarios. Kevin Holman caught this performance issue years back, creating monitoring alerts pack and blog. In case you’re on SCOM jeopardy, the LAW/OMS/Microsoft Monitoring Agent/SCOM agent has a built-in health check. The built-in health check restarts service when Handle Count or memory of the HealthService (aka Microsoft Monitoring Agent service) ran too hot per SCOM PG. SCOM agent restarts caused config churn, and high compute, as workflows re-ran after the service restarted.

Assess agent restarts

Begin by verifying if you have Kevin Holman’s pack for SCOM agent restarts downloaded and installed, which sets memory/handle count informational alerts https://github.com/thekevinholman/SCOM.AgentThresholds

Validate pack installed

Verify SCOM Agent Thresholds pack installed.

Configure addendum for environment

Download and Install ‘SCOMCore Addendum pack’ here

Open saved XML in notepad or Notepad++ (your favorite XML editor here!)

Update the regular expression pattern line for offending servers in the

Figure out the group GUID for the high agent handle count

From PowerShell on SCOM management server, run:

Get-SCOMClassInstance -DisplayName “Proactive High Agent Handle Count servers” | fl DisplayName,ID

Find/Replace GUID

Save file and Import > enjoy less alerts!

Documentation:

Kevin Holman blog on SCOM agent restarts

Holman’s pack for SCOM agent restarts and setting memory/handle count alerts https://github.com/thekevinholman/SCOM.AgentThresholds

Addendum download https://github.com/theKevinJustin/SCOMCoreAddendum

OS Addendum packs

Download the ‘OS Addendum packs’ for new capabilities contains Event count logic monitor type, Disk cleanup, Group Policy, self-healing/reset monitors, as well as ‘eventLog full’ logic and reports. Additional monitors reduce alert noise. Examples of common alert scenarios are: StorPort storage errors, Group Policy 1096 identification and rebuild. Disk Cleanup & EventLog service recovery, which includes Event Log file expansion and rollover.

Quick DownloadS

https://github.com/theKevinJustin/2012OSAddendum

https://github.com/theKevinJustin/2016ServerAgnostic

Tune ‘OS Addendum packs’ as needed

Update logical disk paths and retentions. The default report contains quite a few common checks, including root folders broken out by path, highest to lowest GB’. The workflow is scalable to add additional application paths, as well as file retention timeframes. Workflow runs on a weekly basis to cleanup/archive log files, paths. See Disk cleanup logic blog for more details.

' OS Addendum packs' contains Logical disk breakdown of root folders to list paths were files stored, highest to lowest — ‘ OS Addendum packs’ contains Logical disk breakdown of root folders to list paths were files stored, highest to lowest

UpdateStorPortCountForRepeatedStorageErrors

StorPort storage errors typically cut lots of alerts with storage reads/writes. The ‘count’ monitors decrease the alerts, and the daily alert report consolidates the warning alerts (critical by default). If you’re seeing these alerts, the default should decrease overall alerts to near zero. Tune as needed for disk alerts, by updating MatchCount or TimerWait in Seconds (the x events in y time piece of the monitor logic)

Update StorPort Count for Repeated Storage read/write errors

Save file(s) and import

AD insight reports

Download the ‘AD insights pack’ for new capabilities to audit users, svc/MSA accounts, password last set, expiring, last login AD insights. Includes AD group audit alert capability.

Quick Download https://github.com/theKevinJustin/ADInsights/

AD audit

Time to provide key ‘AD insight reports’ into users and groups. Delve into different AD audit capabilities for users and groups. The pack also gathers DC Security events (rules), and lastly, on demand tasks for reports.

The question is what determines a problem?

Every domain admin has a different experience and perspective, whether cyber (hack) focused or not. Audit standards differ, from HIPAA, SOX, CCRI, STIG, etc.

Pack examples:

Users – service account naming conventions, password change frequency, expired date/time configured.

Groups – Choose your OU structure to audit WA in DA, SA in DA, WA in SA etc.

NOTE: Take caution on the OU group audit, to limit the output, as events have a size limitation

Configure ‘AD insight reports’

Now we can configure the user pack for applicable standards, like password age, last set, or AppOwners. The AppOwners is an array, so you can add whatever Application, system owners/teams in your organization. The password datasource (DS) rule runs weekly.

Configure the Password Time, last set, month, week and AppOwners to build out actionable svc/msa accounts failing audit artifacts.

Break out the regular expressions of whatever accounts each team uses, to tailor relevant data into the report alert. Find/Replace (Control-H) might be more effective, as the DS/WA repeat the logic for the on-demand task report, vs. the rule and monitor.

App Owner relevant service accounts by SamAccountName

Update patterns ID naming conventions

Tailor account names to environment to match ingested DC Security events.

Tailor the DC Security Events to account naming conventions.

Configure OU to environment

Configure OU structure to audit based on domain canonical names, groups, DC, etc.

Save file(s) and import

Disk cleanup logic

Logical disk cleanup, most times is harder vs. smarter manual intervention required, why not smarter vs. harder?

‘Disk cleanup logic’ traditionally follows manual intervention. Why would you want harder and manual? This article will present options to clean up system and non-system disks, by leveraging largest root folder, API’s and more. This is one step in the OS Addendum pack that needs explanation and can be tailored to applications where admins have regular manual cleanup actions.

Breakdown of Disk cleanup

We want to check system disks and non-system disks for different scenarios. Figure out Disk free space, user profiles, largest folder on root of disk, IIS cleanup, and MECM/SCCM client cache clear API. Second, utilize different behaviors depending on PowerShell version, application log(s) cleanup, and expand drive alerts when NO space after cleanup action.

Disk Free space

# Check Disk free space

#=====================

if ($Driveletter -eq “C” )

{

$CFreeSpace = gwmi win32_logicaldisk | ? { $_.DeviceID -eq “C:” }

$CFreeSpace.DeviceID

$CFreeSpace

$DeviceDriveLetter = $CFreeSpace.DeviceID

$DeviceDriveLetter

# Check folder size after cleanup

#==========================

$BeforeSize = (Get-ChildItem “$DeviceDriveLetter” -Recurse | Measure-Object -Property Length -Sum ).sum

$Before = [math]::Round($BeforeSize/1GB,2)

$DiskFreeSpace = [pscustomobject]@{

DeviceID = $DeviceDriveLetter

Size = [math]::Round($CFreeSpace.Size/1GB,2)

FreeSpace = [math]::Round($CFreeSpace.FreeSpace/1GB,2)

}

$DiskFreeSpace

$SoftwareDistribution = (gci C:\windows\SoftwareDistribution | measure length -s).sum / 1Mb

# Debug

$SoftwareDistribution

Check Software Distribution for ConfigMgr/SCCM/MECM client

Checking software distribution path was an item for discussion where the folder was larger than 3GB, stemming from customer and field engineers recommendations.

If ($DiskFreeSpace.FreeSpace -lt 15 )
{
# Audit Software Distribution
#==================================
If ( $SoftwareDistribution -lt “3000” )
{
Write-host “NO SME/SystemOwner/SysAdmin/Server Action required”
}

If ( $SoftwareDistribution -gt “3000” )
{
Write-host “SME/SystemOwner/SysAdmin/Server Action required, stopping Windows Update service, removing SoftwareDIstribution folder and restarting”
Get-Service -Name wuauserv | Stop-Service
Remove-Item -Path C:\Windows\SoftwareDistribution -Recurse
Get-Service -Name wuauserv | Start-Service
Write-host “Windows Update wuauserv service restarted after SoftwareDistribution directory removed”
}

Cleanup Application log folders

The nice part of this is you can reuse this by changing the path and deletion actions to tailor to customer environment. The script comes in handy for VEEAM, SQL, IIS instances and log directory on multiple drives.

# Cleanup IIS log files

#=====================

#if ( Test-Path C:\inetpub\logs\LogFiles\W3SVC1 )

## Years older than

#$HowOld = [DateTime]::Now.AddYears(-1)

#$RecentUse = [DateTime]::Now.AddDays(-90)

## Path to root folder

#$Path = “C:\inetpub\logs\LogFiles\W3SVC1\*.log”

## Deletion task

#get-childitem $Path -Recurse -Depth 1 -EA SilentlyContinue | where { $_.lastAccesstime -lt $RecentUse -and $_.CreationTime -lt $HowOld -and $_.LastWriteTime -lt $RecentUse } | remove-item -force -verbose

Sample report alert output

Documentation

CleanMgr https://learn.microsoft.com/en-us/windows-server/administration/windows-commands/cleanmgr

Delete client cache the right way https://sccm-zone.com/deleting-the-sccm-cache-the-right-way-3c1de8dc4b48

MECM client cache cleanup PowerShell https://learn.microsoft.com/en-us/powershell/module/configurationmanager/invoke-cmclientaction?view=sccm-ps

SCCM Client Cache cleanup https://rzander.azurewebsites.net/sccm-config-item-to-cleanup-ccmcache/

Stack Overflow disk cleanup https://stackoverflow.com/questions/28852786/automate-process-of-disk-cleanup-cleanmgr-exe-without-user-intervention

DNS2012R2 Addendum pack

Still running Server2012R2 servers with AD DCs with AD integrated DNS?

In case you’re still running Windows Server 2012R2, here’s the ‘DNS2012R2 Addendum pack’ giving the same functionality as the version agnostic 2016+ addendum. Why? DNS is a translation method to convert names to IP’s. Can you imagine if we wanted to connect to google via IP? The number of workflows in the SCOM DNS pack (built by the DNS Product Group) makes for an astounding number of workflows running on your DC every minute. Forward and reverse lookups are a good check, verifying DNS is functioning. In a complex environment with 100’s of zones, SCOM becomes a utilization culprit for a DC’s primary missions – authenticate and resolve. This article will help you understand how the pack will add new capabilities and tune DNS monitoring to best practice.

Quick Download HTTPS://GITHUB.COM/THEKEVINJUSTIN/DNSADDENDUM2012R2/

What capabilities does the ‘DNS Addendum pack’ provide?

Count logic monitors (i.e. x events in y time, and self heal)

Daily summary report of DNS alerts broken out

Daily alert closure workflow to close out DNS rules/monitor

DNS service(s) recovery automation

Synthetic internal/external nslookup monitor (scoped to PDC emulators versus ALL DNS servers

WMI validation alert recovery to prevent false positive alerts with weird one off scenarios – one example: Security tools randomly block WMI access.

Download the ‘DNS2012R2 Addendum pack’ on GitHub to improve AD Integrated (ADI) DNS monitoring on Windows Server 2016+ (version agnostic).

Save and Import pack, then update XML for group GUIDs

Update XML

First, update XML with the GUIDs from your management group. Second, map the group DisplayName to find/replace the GUID for each group.

Get-SCOMClassInstance output for DNS2012R2 groups

Third, using Notepad++ highlight the ContextInstance GUID and hit Control-H, and paste the group GUID then click Replace All.

Fourth – Rinse and repeat for the other three groups.

Lastly, save file, move to SCOM MS, and import!

Documentation and links

DNS Pack download

DNS2012R2 addendum blog including updates

GitHub Repository https://github.com/theKevinJustin/DNSAddendum2012R2/

DNS Addendum pack

nslookup to find out IP to name or name to IP resolution.

Simply put: Leverage the ‘DNS Addendum pack’. Why? DNS is a translation method to convert names to IP’s. Can you imagine if we wanted to connect to google via IP? The amount of workflows in the SCOM DNS pack (built by the DNS Product Group) makes for an astounding number of workflows running on your DC every minute. Forward and reverse lookups are a good check, verifying DNS is functioning. In a complex environment with 100’s of zones, SCOM becomes a utilization culprit for a DC’s primary missions – authenticate and resolve. This article will help you understand how the pack will add new capabilities and tune DNS monitoring to best practice.

QUICK DOWNLOAD(S)

2016+ https://github.com/theKevinJustin/DNSAddendumAgnostic

What capabilities does the ‘DNS Addendum pack’ provide?

Count logic monitors (i.e. x events in y time, and self heal)

Daily summary report of DNS alerts broken out

DNS service(s) recovery automation

Daily alert closure workflow to close out DNS rules/monitor

Synthetic internal/external nslookup monitor (scoped to PDC emulators versus ALL DNS servers

WMI validation alert recovery to prevent false positive alerts with weird one off scenarios – one example: Security tools randomly block WMI access.

Download the DNS Addendum on GitHub and the PDF install guide, to improve AD Integrated (ADI) DNS monitoring on Windows Server 2016+ (version agnostic).

XML authoring

The pack greatly decreases alerts, workflows on your AD integrated DNS servers, and the XML authoring is an easy feat. After you import the pack, find/replace is required for two pieces.

Group GUIDs update, after installing this pack.

Find/replace the GUIDs, as they are unique to every SCOM management group, hard coding the group ID GUID is not possible.

From PowerShell, on your SCOM management server, run these commands (after DNS Addendum installed)

Use get-scomclassinstance -DisplayName “GroupNameHere” | ft Id

Find/Replace the GUID in the pack with the ID from the output above.

Discovery group regular expressions (RegEx)

##DNSServerRegEx##

Find ##DNSServerRegEx## and replace with your DNS server expressions.

Example server names: 16dns01, 19dc01,16dns02,19dc02,19dc03, etc.

RegEx = (?i)16dns0|19dc0

DNS Group discovery example of RegEx for find/replace

Save and Import & Enjoy!

ADFS Addendum pack

Do you associate StarTrek when the word federation is used inside of federation services (ADFS)?

To begin, the ‘ADFS addendum pack’ needs acknowledgement of the contributors who dealt with my many questions to better alert on AD issues! My thanks to Jason Windisch for his help and expertise with Active Directory Federation Services (ADFS). If you need more background, check the ‘why addendum pack’ post. BTW, what do you associate with the word – Federation?

Quick Download(s)

2016+ https://github.com/theKevinJustin/ADFSAddendum

Overview of capabilities

The Active Directory Federation Services ‘ADFS Addendum pack’ configures ADFS group of related classes for notification/subscription modeling. Second, the rules, service monitors, tasks, service recovery, alert cleanup, and summary reports aid consumption of real issues. Third, if you have ADFS2012R2, I have an addendum pack, but coordination necessary to get the ADFS management packs MSI (not currently available). Lastly, most environments should be 2016+, as the EOL/EOSL is quickly approaching in October!

ADFS Addendum pack creates ADFS Group AND discovery requiring server names applicable to environment.

ADFS Group discovery requires server names applicable to environment

Tailoring the pack(s) to your environment

First, the Active Directory Federation Services management packs MUST be installed for the ‘ADFS Addendum pack’ to load. 2016+ agnostic is currently supported, as the 2012,2012R2 products are near end of support.

Find/Replace the variables as needed

##ADFSSERVERNAME1##|##ADFSSERVERNAME1##|##LAB##

Save file

Workflows

First, the DataSources (DS) and WriteActions (WA) clean up alerts, create daily reports, where the WA are the on-demand tasks versions.

Data source (DS) scheduled workflows run weekdays between 0600-0700 local SCOM management server local time. The summary and team reports (run during this time) summarize key insights. NOTE: the Monday report gathers the last 72 hours, so administrators get a ‘what happened over the weekend’ view. Tuesday-Friday reports are past 24 hours. Lastly, the group policy report summarizing unique GPUpdate error output.

Monitoring

Addendum pack rules schedule data source execution, add on-demand tasks. The service monitor, and Recovery tasks add service recovery automation to bring us to the ‘manual intervention required’ alerting. There are a few monitor/rule overrides to match the health model.

Import

Download updated ‘ADFS addendum pack’ and save to your environment

Import into SCOM

Enjoy!

Documentation

ADFS 2016+ management pack download

ADDS addendum pack

Active Directory monitoring - definitely needs an addendum! — Active Directory monitoring – definitely needs an addendum!

To begin, the ‘ADDS addendum pack’ needs acknowledgement of the contributors who dealt with my many questions to better alert on AD issues! My thanks to Bob Williams, Vance Cozier, Jason Windisch for their help and expertise with Active Directory (AD/ADDS). If you need more background, check the why addendum pack post.

Quick Download(s)

2012 HTTPS://GITHUB.COM/THEKEVINJUSTIN/ADDS2012ADDENDUM/

2012R2 HTTPS://GITHUB.COM/THEKEVINJUSTIN/ADDS2012R2ADDENDUM/

2016+ https://github.com/theKevinJustin/ADDSAddendumAgnostic

Overview of capabilities

The Active Directory ADDS Addendum pack(s) change how Tier0 health, and Domain Admins consume alerts. Then, AD product team re-wrote the packs back in 2016 to PowerShell workflows. Many workflows measuring replication, health of your forest(s), at the cost of less alert noise than the 2008 packs. Third, the addendums for 2012, 2012R2, and 2016+ version agnostic should help reduce alert ‘burden’. Lastly, most environments should be 2016+, as the EOL/EOSL is quickly approaching in October!

Workflows

First, the DataSources (DS) and WriteActions (WA) clean up AD pack alerts, create daily reports, team, and AD pack summary alerts, where the WA are the on-demand tasks versions.

DataSources (DS) and WriteActions (WA) clean up AD pack alerts, create daily reports, team, and AD pack summary alerts, and the WA are the on-demand tasks versions of the DS

Monitoring

Addendum pack rules schedule data source execution, adding on-demand task alerts, including new group policy rule alerts. The Recovery tasks add service recovery automation to bring us to the ‘manual intervention required’ alerting. There are a few monitor/rule overrides to match the health model. NOTE: The 2012R2 pack is missing the component alert, as there’s less than 2 months until the platform support ends.

The component alert is a new workflow that’s helped Tier0 admins.

Basically, this is a PowerShell workflow that checks SCOM alerts for multiple DC alerts to determine DC health. I don’t change the AD critical service monitors, but simply summarize the alerts to tell you when intervention is required.

Tailoring the pack(s) to your environment

First, the Active Directory Domain Services management packs MUST be installed for the ‘ADDS Addendum pack'(s) to load. The three versions currently supported have addendums, hopefully 2012,2012R2 are planned to be decommissioned in the short term.

Update the AD summary and team reports

The AD summary and team reports for specific Tier0 servers owned by Domain Administrators, AD Team (or any other aliases the SME’s may go by) group regular expressions.

In your favorite XML editor (mine is Notepad++), open the addendum pack(s), and find/replace for the following strings:

Look for the $ADDSServerAlerts

$ADDSServerAlerts = $ADDSReportAlerts | ? { ( $_.NetBiosComputerName -like “*A1*” ) `

Save pack

Import and enjoy!

Documentation

ADDS 2012+ management pack download

ADCS Addendum packs

ADCS 'gift' certificate - don't we all wish! — ADCS ‘gift’ certificate – don’t we all wish!

If only certificates were all gift certificates! The ‘ADCS Addendum packs’ disables noisy rules, adds OCSP seed, OCSP responder and OCSP group (classes). Recovery and service monitoring and nCipher event are the main highlights reducing alerts for ADCS 2012,2012R2,2016+. My thanks to Bob Williams CSA, for the assist!

Quick Download(s)

2012 HTTPS://GITHUB.COM/THEKEVINJUSTIN/ADCS2012QAddendum

2012R2 HTTPS://GITHUB.COM/THEKEVINJUSTIN/ADCS2012R2ADDENDUM/

2016+ https://github.com/theKevinJustin/ADCS2016-Addendum

Overview of addendum capabilities

Remember the why addendum packs for guiding purpose, transform!

The ADCS Addendum packs discover OCSP (seed class), and OCSP responder registry keys installed on monitored servers.

Group discovery tailors OCSP classes, for subscription or alert tuning.

OCSP server group can be used for subscription, or alert tuning (depending on class targets)

Monitors and service recoveries keep OCSP services monitored, and only alert when manual intervention is required.

OCSP service, certsvc monitors and service recovery automations built in

Tailoring the pack(s) to your environment

First, you must have at least ONE (1) set of ADCS Active Directory Certificate Services management packs so the ‘ADCS Addendum pack’ will load. The three versions currently supported have addendums, hopefully 2012,2012R2 are planned to be decommissioned in the short term.

Second, if you don’t have OCSP in your environment, download, and then import into your environment –

ELSE

Update the ‘OCSP Responder’ server name(s) for the group regular expressions.

In your favorite XML editor (mine is Notepad++), open the addendum pack(s), and find/replace for the following strings:

CAServer##

CERTIFICATESERVERS##

Save pack

Import and enjoy!

Documentation

ADCS 2016+ version agnostic pack download

ADCS 2012/2012R2 management packs download

Why Addendum packs

‘Why addendum packs’? What value can they bring to my customer? Kevin Holman started the Addendum thought process quite a while back. Added functionality to a core application/program/product. The first example of this pack naming convention is his SQL RunAs Addendum to simplify SQL monitoring. Let’s break down a number of examples how the SCOM community has built packs to better monitoring, and how I believe the addendum packs bring IT Ninja lessons from Microsoft experts monitoring to your environment.

Why Addendum packs

Better monitoring from the experts, including customer examples for other ‘blind spots’ in monitoring. Blind spots consist of ‘not monitored’ pieces of infrastructure, from simply an event, ping, service, tcp port check, process, web site, scripted workflow, with the purpose to identify a problem.

The goal of monitoring is to:

Identify, self-heal, automatically run recovery or diagnostic workflows alert when manual intervention is required. Doesn’t matter what tool you use, they all do some portion of these steps.

The addendum packs do these things, adding a few differentiators.

Auto closure daily scripts (close rules/monitors)

Auto reports of problems (M-F 0600-0700 local, reflecting last 24-72 hours of open/closed alerts)

Employ count logic (x in y time)

Self-heal monitors with no new events

Adjust alert severities to health model

where critical (red) = outage, warning (yellow) = issue, informational reports or FYI’s

Capable of updating alerts (status, owner, ticketID+)

Tasks to run workflows on-demand

Recovery tasks – (i.e. service restart automation or TopProcess, Logical disk cleanup, MECM Client cache clean )

Integrate additional monitoring (like DFS replication queue script/alerts)

Synthetic checks for DNS and web applications

Web Availability and Transactional monitoring, ADFS, CRL, PowerShell Invoke-WebRequest, and more

Security and Compliance checks

Imagine I forgot something capability wise.

Stay tuned, as this builds into an even better outcome, quality data into ‘a single pane of glass’ of multiple tools within PowerBI.