A photo of Mitesh Shah

Mitesh Shah

Linux Enthusiast
System Administrator

Email Skype Github Twitter Google+ Resume Hire Me Keybase LinkedIn Stackoverflow


Overview

Import Squid3 Logs on Logstash

NOTE! We assume that you already setup/configure ELK Stack.

  • To Import squid3 Logs on Logstash, We have to create configuration file.
$ vim /etc/logstash/conf.d/squid.conf
input {
  file {
    type => "squid"
    start_position => "beginning"
    path => [ "/var/log/squid3/access.log" ]
  }
}

filter {
  if [type] == "squid" {
    grok {
      match => [ "message", "%{NUMBER:timestamp}\s+%{NUMBER:response_time} %{IP:src_ip} %{WORD:squid_request_status}/%{NUMBER:http_status_code} %{NUMBER:reply_size_include_header} %{WORD:http_method} %{WORD:http_protocol}://%{HOSTNAME:dst_host}%{NOTSPACE:request_url} %{NOTSPACE:user} %{WORD:squid}/(?:-|%{IP:dst_ip}) %{NOTSPACE:content_type}" ]
      add_tag => ["squid"]
    }
    geoip {
      source => "dst_ip"
    }
  }
}

Fix Squid3 Logs Permission

  • Let’s make squid3 logs are readable by Logstash
# Temp Fix
$ chmod 644 /var/log/squid3/access.log

# Permeant Fix
$ cat /etc/logrotate.d/squid3
#
#	Logrotate fragment for squid3.
#
/var/log/squid3/*.log {
	daily
	compress
	delaycompress
	rotate 2
	missingok
	nocreate
	sharedscripts
	postrotate
		test ! -e /var/run/squid3.pid || /usr/sbin/squid3 -k rotate
	endscript
  create 644 proxy proxy
}

Restart Logstash Service

$ sudo service logstash restart

Configure Kibana

  • Open http://192.168.0.1:5601/
  • Click on Settings > Objects > Import

  • Import Dashboard/Visualizations

Kibana Dashboard

  • Open http://192.168.0.1:5601/#/dashboard/Squid?_g=()

How to Monitor Squid3 Logs on ELK Stack





Post Navigation