Build FluentD conf file

Build trust one block at a time

Ready to build out a FluentD conf file?

 

Let’s build a FluentD conf file.  We can use the docs site for another example.  And now, let’s build a simple FluentD configuration file. Paste the XML code below, and save as <yourlogfile>.conf

Create custom log file to test

cd /etc/opt/microsoft/omsagent/scom/conf/omsagent.d/
# vi <yourlogfile>.conf

vi mylog.conf

# Example conf file

<source>
# Specifies input plugin. Tail is a fluentd input plugin – http://docs.fluentd.org/v0.12/articles/in_tail
type tail
# Specify the log file path. Supports wild cards.
path /var/log/mylog
# Recommended so that Fluentd will record the position it last read into this file.
pos_file /home/omsagent/fluent-logging/mylog.pos

# Used to correlate the directives.
tag scom.log.mylog

format /(?<message>.*)/
</source>

<filter scom.log.mylog>
type filter_scom_simple_match
regexp1 message 911
event_id1 911
</filter>

<match scom.log.mylog>
#Disable mutual Auth
enable_server_auth false

# Output plugin to use
type out_scom
log_level trace
num_threads 5

# Size of the buffer chunk. If the top chunk exceeds this limit or the time limit flush_interval, a new empty chunk is pushed to the top of the
queue and bottom chunk is written out.
buffer_chunk_limit 5m
flush_interval 15s
# Specifies the buffer plugin to use.
buffer_type file
# Specifies the file path for buffer. Fluentd must have write access to this directory.
buffer_path /var/opt/microsoft/omsagent/scom/state/out_scom_common*.buffer
# If queue length exceeds the specified limit, events are rejected.
buffer_queue_limit 10
# Control the buffer behavior when the queue becomes full: exception, block, drop_oldest_chunk
buffer_queue_full_action drop_oldest_chunk
# Number of times Fluentd will attempt to write the chunk if it fails.
retry_limit 10
# If the bottom chunk fails to be written out, it will remain in the queue and Fluentd will retry after waiting retry_wait seconds
retry_wait 30s
# The retry wait time doubles each time until max_retry_wait.
max_retry_wait 9m
</match>

Save (:wq!)

 

# Restart Agent

/opt/microsoft/omsagent/bin/service_control restart

# Check for errors – see blog

grep -i error /var/opt/microsoft/omsagent/scom/log/omsagent.log

# Test strings into your logfile

# Options

echo test >> /var/log/mylog

echo 911 error >> /var/log/mylog

# mimic syslog or messages syntax

echo `date +”%b %e %H:%M:%S”` MYLOG 911 test string. Call 911 >> /var/log/mylog

 

Please stay tuned for more management pack options to alert on the strings.  Refer to the part1/2 blogs for more details on unit testing for alerts.

2 Replies to “Build FluentD conf file”

  1. Hey Kevin,

    Do you know if it is possible to include, what we catch in the regular expression groups, in the description of the event we are sending to SCOM Management Server.
    I can see that it is included in the Event Parameter[5] but it is not part of the event description.
    My goal is to bring it to the Alert description. I have tried to include different parts of the Data Item in the Alert Parameters and pass it to the Alert description but surprisingly the rule stopped to create alerts 🙂

    1. Adam,
      Sorry for the late reply, guessing you’ll need to create your own datasource modeled after the ‘Microsoft.Linux.OMED.EventDataSource’ in the Microsoft.Linux.Library

      The DataSource (DS) includes 1 and 4 for the two Condition Detections –




      Params/Param[1]

      MatchesRegularExpression ^$Config/ComputerName$(\..+)*$




      $Data/EventOriginId$ $Data/PublisherId$ System Center OMED Service System Center OMED Service
      $Config/ComputerName$
      $Data/EventDisplayNumber$
      $Data/EventCategory$
      $Data/EventLevel$

      %1 $Data/Params/Param[4]$

Leave a Reply

Your email address will not be published. Required fields are marked *