Pushing Apache HTTP Server logs using a fluentd relay

The preferred method to send Apache/Nginx logs is to use fluentd in case of Linux or td-agent in case of Windows or Mac OS X (the latter being a special version of fluent, packaged by Treasure Data). They are available at fluentd download page

The exact steps are following:

Step 1: Install fluentd or td-agent

Go to https://www.fluentd.org/download and select preferred version

Step 2: Prepare the config

The config consists of several sections. Source section specifies location of the input files. Filter section adds data required to associate the records with given account. Match section specifies the output of the log records.

The config typically resides in /etc/fluentd (Linux), /etc/td-agent (macOS) or C:\opt\td-agent\etc\td-agent\td-agent.conf (Windows)

Please replace CUSTOMER_TOKEN with the value available in Customer Tokens section of the user portal. The YOUR_IP value should be replaced with the IP you want the logs be visible as the source.

Also, please put the actual location of the log files.

# Access log
  @type tail
  path ENTER_FULL_PATH_OF_ACCESS_LOG_FILE, e.g. /var/log/apache/access.log
  pos_file ENTER_FULL_PATH_OF_ACCESS_LOG_POS_FILE, e.g. /var/log/apache/access.log.pos
  tag apache.access
  format apache2

# Error log
    @type tail
    tag apache.error
    path ENTER_FULL_PATH_OF_ERROR_LOG_FILE, e.g. /var/log/apache/error.log
    pos_file ENTER_FULL_PATH_OF_ERROR_LOG_POS_FILE.pos, e.g. /var/log/apache/error.log.pos

    format /^\[[^ ]* (?<time>[^\]]*)\] \[(?<level>[^\]]*)\] \[pid (?<pid>[^\]]*)\] \[client (?<client>[^\]]*)\] (?<message>.*)$/

<filter {apache}.**>
  @type record_transformer
    # Please fill the actual token!
    cs_customer_token CUSTOMER_TOKEN
    cs_pattern_key "message"
    cs_source_name "apache"

    # Uncomment and enter IP you want the logs to be
    # associated with as the source
    # cs_src_ip "YOUR_IP"

<match apache.**>
  @type forward

  transport tls
    host logs.logsense.com
    port 32714

  flush_interval 10s