Installing and configuring ELK stack 5.4.x (now Elastic Stack)

The basic config for this setup was done using this post:

http://pfelk.3ilson.com/

However, I was using ubuntu 17.04 instead of Debian and there were some changes required to get things running.

Changes:

  • Skip adding repo for Java or using the installer, just run
    sudo apt install openjdk-8-jdk
  • When actually running ELK install, you have to separate out the commands or you will get an apt install error:
  • sudo apt install elasticsearch
  • sudo apt install logstash
  • sudo apt install kibana

     Problems I ran into:

  • Config files downloaded didn’t fully work, Visualizations failed to import which led to dashboard being empty.  The searches worked well
    • Solution: Manually create my own dashboards and visualizations from the searches.  Good experience anyway!
  • pfSense wasn’t forwarding logs properly to logstash, data was emtpy
    • Solution: Multiple restarts and checking configs revealed issues with the IP of the VM ELK was running on.  Corrected that and double-checked IPs throughout the config.
  • For some reason restarting the services hasn’t been reloading the config to a working state
    • Solution: ??? (maybe failed shutdown, port not unbound?)  – Temp work around was to restart the VM to get all configs to pick up and logs starting to collect again.