A photo of Mitesh Shah

Mitesh Shah

Linux Enthusiast
System Administrator

Email Skype Github Twitter Resume Hire Me Keybase LinkedIn Stackoverflow


Overview

Import Squid3 Logs on Logstash

NOTE! We assume that you already setup/configure ELK Stack.

  • To Import squid3 Logs on Logstash, We have to create configuration file.
$ vim /etc/logstash/conf.d/squid.conf
input {
  file {
    type => "squid"
    start_position => "beginning"
    path => [ "/var/log/squid3/access.log" ]
  }
}

filter {
  if [type] == "squid" {
    grok {
      match => [ "message", "%{NUMBER:timestamp}\s+%{NUMBER:response_time} %{IP:src_ip} %{WORD:squid_request_status}/%{NUMBER:http_status_code} %{NUMBER:reply_size_include_header} %{WORD:http_method} %{WORD:http_protocol}://%{HOSTNAME:dst_host}%{NOTSPACE:request_url} %{NOTSPACE:user} %{WORD:squid}/(?:-|%{IP:dst_ip}) %{NOTSPACE:content_type}" ]
      add_tag => ["squid"]
    }
    geoip {
      source => "dst_ip"
    }
  }
}

Fix Squid3 Logs Permission

  • Let’s make squid3 logs are readable by Logstash
# Temp Fix
$ chmod 644 /var/log/squid3/access.log

# Permeant Fix
$ cat /etc/logrotate.d/squid3
#
#	Logrotate fragment for squid3.
#
/var/log/squid3/*.log {
	daily
	compress
	delaycompress
	rotate 2
	missingok
	nocreate
	sharedscripts
	postrotate
		test ! -e /var/run/squid3.pid || /usr/sbin/squid3 -k rotate
	endscript
  create 644 proxy proxy
}

Restart Logstash Service

$ sudo service logstash restart

Configure Kibana

  • Open http://192.168.0.1:5601/
  • Click on Settings > Objects > Import

  • Import Dashboard/Visualizations

Kibana Dashboard

  • Open http://192.168.0.1:5601/#/dashboard/Squid?_g=()

How to Monitor Squid3 Logs on ELK Stack





Post Navigation