Build FluentD conf file

Ready to build out a FluentD conf file?

Let’s build a FluentD conf file.  We can use the docs site for another example.  And now, let’s build a simple FluentD configuration file. Paste the XML code below, and save as <yourlogfile>.conf

Create custom log file to test

cd /etc/opt/microsoft/omsagent/scom/conf/omsagent.d/
# vi <yourlogfile>.conf

vi mylog.conf

# Example conf file

<source>
# Specifies input plugin. Tail is a fluentd input plugin – http://docs.fluentd.org/v0.12/articles/in_tail
type tail
# Specify the log file path. Supports wild cards.
path /var/log/mylog
# Recommended so that Fluentd will record the position it last read into this file.
pos_file /home/omsagent/fluent-logging/mylog.pos

# Used to correlate the directives.
tag scom.log.mylog

format /(?<message>.*)/
</source>

<filter scom.log.mylog>
type filter_scom_simple_match
regexp1 message 911
event_id1 911
</filter>

<match scom.log.mylog>
#Disable mutual Auth
enable_server_auth false

# Output plugin to use
type out_scom
log_level trace
num_threads 5

# Size of the buffer chunk. If the top chunk exceeds this limit or the time limit flush_interval, a new empty chunk is pushed to the top of the
queue and bottom chunk is written out.
buffer_chunk_limit 5m
flush_interval 15s
# Specifies the buffer plugin to use.
buffer_type file
# Specifies the file path for buffer. Fluentd must have write access to this directory.
buffer_path /var/opt/microsoft/omsagent/scom/state/out_scom_common*.buffer
# If queue length exceeds the specified limit, events are rejected.
buffer_queue_limit 10
# Control the buffer behavior when the queue becomes full: exception, block, drop_oldest_chunk
buffer_queue_full_action drop_oldest_chunk
# Number of times Fluentd will attempt to write the chunk if it fails.
retry_limit 10
# If the bottom chunk fails to be written out, it will remain in the queue and Fluentd will retry after waiting retry_wait seconds
retry_wait 30s
# The retry wait time doubles each time until max_retry_wait.
max_retry_wait 9m
</match>

Save (:wq!)

# Restart Agent

/opt/microsoft/omsagent/bin/service_control restart

# Check for errors – see blog

grep -i error /var/opt/microsoft/omsagent/scom/log/omsagent.log

# Test strings into your logfile

# Options

echo test >> /var/log/mylog

echo 911 error >> /var/log/mylog

# mimic syslog or messages syntax

echo `date +”%b %e %H:%M:%S”` MYLOG 911 test string. Call 911 >> /var/log/mylog

Please stay tuned for more management pack options to alert on the strings.  Refer to the part1/2 blogs for more details on unit testing for alerts.