Tag: dwdatarp

Alert on DWDataRP output

Holman's blog for DWDataRP is one way to Alert on DWDataRP findings
Holman’s blog for DWDataRP is one way to Alert on DWDataRP findings

I want to alert on DWDataRP output!  While everyone’s familiar with Holman’s SCOM SQL queries blog, read below to configure a new way to maintain data warehouse integrity and retention.

 

Data warehouse audits are included in monitoring platform checks.  For those new to monitoring, basically DWDataRP analyzes SCOM Data Warehouse issues for alert/event/performance/state retention.  One administration option is to utilize the SCOM Core Monitoring Addendum pack to run DWDataRP.  Another option is to run DWDataRP via Holman’s blog, or recently with Blake Drumm’s GUI tool.

 

Using the SCOM Core Monitoring addendum pack pre-configures a number of overrides, as well as adding DWDataRP monitor/rule options.  Consequently, the SCOM action account needs to have additional permissions on SCOM SQL servers where the OperationsManagerDW
resides.

Configure SCOM management server action account to alert on DWDataRP output

Example uses lab environment SVC.SCOM.PBIreader
Substitute the SCOM action account above for the SCOM data warehouse (OperationsManagerDW) databases on their respective SCOM management group(s).

Give SCOM Action account necessary rights
Update SVC account rights
Set and verify SVC account has Server role public

Click on User Mapping > select OperationsManagerDW database
Verify Default Schema shows DBO
Under Database Role Membership
Select db_datareader AND db_owner
Click OK



Verification
Reach out to SCOM team to verify execution
From SCOM, RDP to one of the management servers
Click on Start > Right click on Windows PowerShell
Click on More > click on Select PowerShell
Click on More > Click on 'Run as a different user'
Open PowerShell > right click > Run as a different user
Open PowerShell > right click > Run as a different user
On the Windows Security pop-up > Click on 'Use a different account'
Type the action account username and password
Click OK
Click Use a different account in the 'Run As different user' popup
Click Use a different account in the ‘Run As different user’ popup
Paste in the following commands, and verify output
cd "##YourPathtoDWDATARP.EXE##"
# cd D:\MonAdmin\TOOLS\DWDataRP"

# Check events
$Command = '.\dwdatarp.exe -s 16DB02 -d OperationsManagerDW -ds "Event Data
Set"'
$EventDataSet = Invoke-Expression $Command
$EventDataSet
$EventDataSet[2]
$LLineSplit = $EventDataSet[2].Split("(")
$EventDBPercent = $LLineSplit[1].Split("%")
$EventDBPercent[0]
DWDataRP PowerShell event output
DWDataRP PowerShell event output
Example PowerShell output when SVC Account cannot execute DWDataRP
PS C:\monadmin\tools\dwdatarp> whoami
testlab\svc.scom.pbireader
PS C:\monadmin\tools\dwdatarp> .\dwdatarp.exe -s 16db02 -d OperationsManagerDW -ds
Event
Dataset name
Aggregation name Max Age Current Size, Kb
----------------------------------------------------------------------------

Use DWdataRP utility on SCOM2016

Extra, Extra, read all about it!   Updated 24 June 2022

The DWdataRP utility works on SCOM2016+ 

 

 

This tool has been around since SCOM 2007, and blogged about by Kevin Holman and the SCOM Engineering Team

DW Retention https://kevinholman.com/2010/01/05/understanding-and-modifying-data-warehouse-retention-and-grooming/

Kevin added these –

Here is a link to the original command line tool, DWDATARP:   https://kevinholman.com/files/dwdatarp.zip

Here is a tool which makes this even simpler:  https://www.scom2k7.com/scom-datawarehouse-grooming-settings-super-easy-with-new-gui-tool/

And the latest new tool which adds features and simplicity, which is what I recommend: https://blakedrumm.com/blog/scom-dw-grooming-tool/

 

DWDataRP https://techcommunity.microsoft.com/t5/System-Center-Blog/Data-Warehouse-Data-Retention-Policy-dwdatarp-exe/ba-p/340415?search-action-id=139683980150&search-result-uid=340415

 

 

Download the link at the second link, and save to your machine (x86 or x64)

 

Run the executable

dwdatarp.exe -s <DB Server Name> -D <DW Database Name>

 

Example dwdatarp.exe -s 16DB02 -D OperationsManagerDW

 

 

Set Data sets according to SLA

 

 

Ever need to shorten up your Lab DB, so you can export it, to leverage PowerBI dashboards?

 

Example Output

PS C:\Users\sqladmin\Documents\DWDataRP\amd64> .\dwdatarp.exe -s 16db02.testlab.net -d OperationsManagerDW
Dataset name Aggregation name Max Age Current Size, Kb
—————————— ——————– ——- ——————–
Alert data set Raw data 180 28,736 ( 0%)
Client Monitoring data set Raw data 30 0 ( 0%)
Client Monitoring data set Daily aggregations 400 136 ( 0%)
Configuration dataset Raw data 400 259,968 ( 3%)
DPM event dataset Raw data 400 0 ( 0%)
DPM.Backup.DataSet Raw data 400 0 ( 0%)
DPM.Backup.DataSet Hourly aggregations 3 0 ( 0%)
DPM.Backup.DataSet Daily aggregations 182 0 ( 0%)
DPM.DiskMgmt.DataSet Raw data 400 0 ( 0%)
DPM.DiskMgmt.DataSet Hourly aggregations 3 0 ( 0%)
DPM.DiskMgmt.DataSet Daily aggregations 182 0 ( 0%)
DPM.DiskUtilization.DataSet Raw data 400 0 ( 0%)
DPM.DiskUtilization.DataSet Hourly aggregations 3 0 ( 0%)
DPM.DiskUtilization.DataSet Daily aggregations 182 0 ( 0%)
DPM.Recovery.DataSet Raw data 400 0 ( 0%)
DPM.Recovery.DataSet Hourly aggregations 3 0 ( 0%)
DPM.Recovery.DataSet Daily aggregations 182 0 ( 0%)
DPM.SLATrend.DataSet Raw data 400 0 ( 0%)
DPM.SLATrend.DataSet Hourly aggregations 3 0 ( 0%)
DPM.SLATrend.DataSet Daily aggregations 182 0 ( 0%)
DPM.TapeUtilization.DataSet Raw data 400 0 ( 0%)
DPM.TapeUtilization.DataSet Hourly aggregations 3 0 ( 0%)
DPM.TapeUtilization.DataSet Daily aggregations 182 0 ( 0%)
Event data set Raw data 100 456,144 ( 5%)
Exchange 2013: Mailbox Database data warehouse dataset Raw data 30 0 ( 0%)
Exchange 2013: Mailbox statistics data warehouse dataset Raw data 30 0 ( 0%)
Exchange 2013: Mailbox statistics data warehouse dataset Daily aggregations 400 0 ( 0%)
Performance data set Raw data 10 216,096 ( 2%)
Performance data set Hourly aggregations 400 5,552,832 ( 61%)
Performance data set Daily aggregations 400 221,312 ( 2%)
Process Monitoring: Performance Metric State data warehouse dataset Raw data 10 0 ( 0%)
Process Monitoring: Performance Metric State data warehouse dataset Hourly aggregations 90 0 ( 0%)
Process Monitoring: Performance Metric State data warehouse dataset Daily aggregations 180 0 ( 0%)
Process Monitoring: Process Health State data warehouse dataset Raw data 10 0 ( 0%)
Process Monitoring: Process Network Ports data warehouse dataset Raw data 10 0 ( 0%)
State data set Raw data 180 6,080 ( 0%)
State data set Hourly aggregations 400 2,335,968 ( 25%)
State data set Daily aggregations 400 99,840 ( 1%)

 

Commands to run to clean up the warehouse…

.\dwdatarp.exe -s 16db02.testlab.net -d OperationsManagerDW -ds Performance -a Hourly -m 60
.\dwdatarp.exe -s 16db02.testlab.net -d OperationsManagerDW -ds Performance -a Hourly -m 60
.\dwdatarp.exe -s 16db02.testlab.net -d OperationsManagerDW -ds State -a Hourly -m 60
.\dwdatarp.exe -s 16db02.testlab.net -d OperationsManagerDW -ds event -a raw -m 30

Example Output
PS C:\Users\admin\Documents\DWDataRP\amd64> .\dwdatarp.exe -s 16db02.testlab.net -d OperationsManagerDW -ds Performan
ce -a Hourly -m 60
Max data age set to 60 on dataset “Performance data set” aggregation type “Hourly aggregations”
PS C:\Users\admin\Documents\DWDataRP\amd64> .\dwdatarp.exe -s 16db02.testlab.net -d OperationsManagerDW -ds State -a
Hourly -m 60
Max data age set to 60 on dataset “State data set” aggregation type “Hourly aggregations”
PS C:\Users\admin\Documents\DWDataRP\amd64> .\dwdatarp.exe -s 16db02.testlab.net -d OperationsManagerDW -ds event -a
raw -m 30
Max data age set to 30 on dataset “Event data set” aggregation type “Raw data”

 

 

Don’t forget to execute cleanup

–This will manually run grooming for this dataset
DECLARE @DatasetId uniqueidentifier
SET @DatasetId = (SELECT DatasetId FROM StandardDataset WHERE SchemaName = ‘Alert’)
EXEC StandardDatasetGroom @DatasetId

–This will manually run grooming for this dataset
DECLARE @DatasetId uniqueidentifier
SET @DatasetId = (SELECT DatasetId FROM StandardDataset WHERE SchemaName = ‘Event’)
EXEC StandardDatasetGroom @DatasetId

–This will manually run grooming for this dataset
DECLARE @DataSetId uniqueidentifier
SET @DataSetId = (SELECT DatasetId FROM StandardDataset WHERE SchemaName = ‘Perf’)
EXEC StandardDatasetGroom @DataSetId

–This will manually run grooming for this dataset
DECLARE @DataSetId uniqueidentifier
SET @DataSetId = (SELECT DatasetId FROM StandardDataset WHERE SchemaName = ‘State’)
EXEC StandardDatasetGroom @DataSetId