Ready to build out a FluentD conf file?
Let’s build a FluentD conf file. We can use the docs site for another example. And now, let’s build a simple FluentD configuration file. Paste the XML code below, and save as <yourlogfile>.conf
Create custom log file to test
cd /etc/opt/microsoft/omsagent/scom/conf/omsagent.d/
# vi <yourlogfile>.conf
vi mylog.conf
# Example conf file
<source>
# Specifies input plugin. Tail is a fluentd input plugin – http://docs.fluentd.org/v0.12/articles/in_tail
type tail
# Specify the log file path. Supports wild cards.
path /var/log/mylog
# Recommended so that Fluentd will record the position it last read into this file.
pos_file /home/omsagent/fluent-logging/mylog.pos
# Used to correlate the directives.
tag scom.log.mylog
format /(?<message>.*)/
</source>
<filter scom.log.mylog>
type filter_scom_simple_match
regexp1 message 911
event_id1 911
</filter>
<match scom.log.mylog>
#Disable mutual Auth
enable_server_auth false
# Output plugin to use
type out_scom
log_level trace
num_threads 5
# Size of the buffer chunk. If the top chunk exceeds this limit or the time limit flush_interval, a new empty chunk is pushed to the top of the
queue and bottom chunk is written out.
buffer_chunk_limit 5m
flush_interval 15s
# Specifies the buffer plugin to use.
buffer_type file
# Specifies the file path for buffer. Fluentd must have write access to this directory.
buffer_path /var/opt/microsoft/omsagent/scom/state/out_scom_common*.buffer
# If queue length exceeds the specified limit, events are rejected.
buffer_queue_limit 10
# Control the buffer behavior when the queue becomes full: exception, block, drop_oldest_chunk
buffer_queue_full_action drop_oldest_chunk
# Number of times Fluentd will attempt to write the chunk if it fails.
retry_limit 10
# If the bottom chunk fails to be written out, it will remain in the queue and Fluentd will retry after waiting retry_wait seconds
retry_wait 30s
# The retry wait time doubles each time until max_retry_wait.
max_retry_wait 9m
</match>
Save (:wq!)
# Restart Agent
/opt/microsoft/omsagent/bin/service_control restart
# Check for errors – see blog
grep -i error /var/opt/microsoft/omsagent/scom/log/omsagent.log
# Test strings into your logfile
# Options
echo test >> /var/log/mylog
echo 911 error >> /var/log/mylog
# mimic syslog or messages syntax
echo `date +”%b %e %H:%M:%S”` MYLOG 911 test string. Call 911 >> /var/log/mylog
Please stay tuned for more management pack options to alert on the strings. Refer to the part1/2 blogs for more details on unit testing for alerts.