November 2017 - Kevin Justin's Blog

November 9, 2017December 20, 2019

Ruling out SCOM as the cause of SCHANNEL events

Ruling out SCOM notifications as the cause of SCHANNEL events

Still getting SCHANNEL error events and want to rule out SCOM

Management pack SQL events https://kevinjustin.com/blog/2017/11/08/sql-native-client-for-tls1-2/

SCHANNEL ciphers debugged https://kevinjustin.com/blog/2017/11/08/schannel-event-logging/

What command Channels are setup for notifications?

Validate Subscriptions aren’t the cause for email/text

Exchange 2013 and above typically use S/MIME to digitally sign/encrypt messages

Email communication can cause System 36871 events https://support.microsoft.com/en-us/help/305088/schannel-error-message-36871-when-receiving-an-ehlo-smtp-command

Do the events correlate with emailed alerts?
Tracing Notifications http://blog.scomskills.com/enable-tracing-of-the-notification-component-om07/

SCOM ETL traces

Run traces on suspect MS

2012R2 MS (adjust drive letter according to drive SCOM install)
cd “D:\Program Files\Microsoft System Center 2012 R2\Operations Manager\Server\Tools”
2012R2 GW (adjust drive letter according to drive SCOM install)
cd “C:\Program Files\System Center Operations Manager\Gateway\Tools”
2016 MS
cd ‘C:\Program Files\Microsoft System Center 2016\Operations Manager\Server\Tools\’

# Stop Tracing
.\StopTracing.cmd
# Clean up old files
remove-item C:\windows\Logs\OpsMgrTrace\*

# Start Traces

StartTracing.cmd VER

TraceLogSM.exe -stop TracingGuidsNative

TraceLogSM.exe -stop TracingGuidsUI

# Wait until notification fires and validate if 36871 SCHANNEL event ID is logged

# Stop and format the trace
.\StopTracing.cmd
.\FormatTracing.cmd

# Review txt files from C:\windows\Logs\OpsMgrTrace

November 8, 2017December 20, 2019

SCHANNEL event logging

First, my thanks to Bhuvnesh Kumar for his help!

Time to figure out what’s going on behind the curtain!

Are you seeing System Event Log, Event ID 36871 events?

Why does this matter?

Depending on OS versions and patches, the TLS Cipher Suites may not match on the various SCOM servers.

If you’re setting up TLS1.2, you need the SCOM servers to talk
The bad part, is this isn’t logged much on the GW but log more often on MS
Sometimes the 36871 events come with 36874, but in my experience they occur after Event Logging is enabled.

The unanswered question is “why are we seeing the 36871 events?”

In my example, the events only happened once a day, roughly 24 hours

Event Viewer

Are events related to the Cipher Suite, or is it a MP trying to run the old SQLOLEDB method?

This article will focus on verifying Cipher Suite on a server

See this article for MP analysis for SQL methods

SCHANNEL event logging setup

From Holman’s blog

Decimal	Description
0	Do not log
1	Log Error messages
2	Log Warnings
3	Log Error and Warning messages
4	Log Informational and Success events
5	Log Error, Informational and Success events
6	Log Warnings, Informational and Success events
7	Log Everything (Warnings, Errors, Informational and Success events

I’d recommend setting it to 3 to see errors and warnings, or 7 to see everything.

Remember to set this back to 1 when done resolving any issues.

Add

From Command Prompt or PowerShell (as administrator)

reg add “HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL” /v “EventLogging” /t REG_DWORD /d 7 /f

Disable

reg delete “HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL” /v “EventLogging”

Verification

reg query “HKLM\SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL” /v “EventLogging”

PowerShell verification

RegEdit Verification

Time to reboot!

Verify SCHANNEL events

Look at the System Event log, and filter for 36880 and 36874 events for clues

36880 provides Cipher Suite details

Event ID 36874 definitely describes the scenario

The easy answer to solve the cipher suite is to ask – is this server patched with latest security and .NET patches?

After all this, in my example, we confirmed that simple step was assumed, and inaccurate.

References
36871 event https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/dn786445(v=ws.11)

SCHANNEL events https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/dn786445(v=ws.11)

SChannel error codes https://docs.microsoft.com/en-us/windows/win32/secauthn/schannel-error-codes-for-tls-and-ssl-alerts

SChannel events https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2012-R2-and-2012/dn786445(v=ws.11)

SSL errors https://www.experts-exchange.com/questions/28996780/event-id-36871-Schannel.html

Troubleshooting https://docs.microsoft.com/en-us/iis/troubleshoot/security-issues/troubleshooting-ssl-related-issues-server-certificate

November 8, 2017December 20, 2019

SQL native client for TLS1.2

Ever try to talk to someone when language is a barrier?

Sure, we can run an app, or search our phrase to pronounce, but it’s so much better when we can communicate seamlessly.

Post TLS1.2 for SCOM

Let’s talk SQL

Part of TLS1.2 is updating SQL Native Client to talk using a secure client that uses TLS1.2

That means a different executable should be called.

Why is that important in SCOM?

Maybe you have management packs that connect to SQL or run external commands.

On MS, there are multiple clues for various errors on Management Packs that use SSL or talk to SQL via a non-TLS method. NOTE this may mean that the SQL DB that management pack is connecting to may need the same pre-req SQL updates to a TLS 1.2 enabled version.

Do you have custom SQL queries being run, CMDB get’s, OLE DB Data Source checks?
Any Event ID 1401 or 11854 events in the Operations Manager Event log?
1. These events identify management pack scripts creating SCHANNEL events
  a. Event ID 1401 event example

Cause

SQLOLEDB connection strings will cause 36871 Sytem Log events

Example (TLS1.0)
sConnectString = “PROVIDER=SQLOLEDB;DATA SOURCE=<databaseServerFQDN>;DATABASE=MSSQLSERVER;trusted_connection=yes”
SQLNCLI11 driver for TLS1.2 connection strings

Example (TLS1.2)
sConnectString = “Provider=SQLNCLI11;DATA SOURCE=<databaseServerFQDN>;DATABASE=MSSQLSERVER;trusted_connection=yes”

Identify
Look for management packs with SQLOLEDB as the Connect string to reduce 36871 SCHANNEL events

In Windows Explorer, use the Advanced Options dropdown to select File Contents
In the Search bar (top right), enter SQLOLEDB (example shows SQLNCLI11)
NOTE SQL Discovery group pack IS compliant

In Windows Explorer, use the Advanced Options dropdown to select File Contents
In the Search bar (top right), enter SQLNCLI11

Additional offenders
HP Topology MP
SQL 2005 discovery MP (discontinued)
SQL Addendum MP’s (will work to update these with Holman)
SharePoint Foundation server (v15.0.4557.1000)
PRE TLS Microsoft.SystemCenter.2007

Resolution
Unseal (if necessary), update connection string, and reimport management packs
If Sealed vendor MP, request new MP via support Incident (and/or UserVoice if Microsoft sourced pack)
If Vendor will not release MP’s, accept risk with the logged errors, update MP, or remove from SCOM

November 1, 2017December 20, 2019