Installing and Configuring Suricata+Snorby+Barnyard2

From Fyzix
Revision as of 19:59, 30 July 2019 by Fyzix (talk | contribs) (Created page with "Reference: https://www.frlinux.eu/?p=351 Reference: http://sublimerobots.com/2015/12/snort-2-9-8-x-on-ubuntu-part-7b/ This was built on Debian Jessie 64bit on 2016-09-05 =...")
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Reference: https://www.frlinux.eu/?p=351

Reference: http://sublimerobots.com/2015/12/snort-2-9-8-x-on-ubuntu-part-7b/

This was built on Debian Jessie 64bit on 2016-09-05

Suricata

Suricata is the IDS doing the sniffing of traffic to look for malicious evildoers and other fishy business.

Installation

apt-get install suricata

Configuration

Modify /etc/default/suricata

RUN=yes and set IFACE=

# Default config for Suricata

# set to yes to start the server in the init.d script
RUN=yes

# Configuration file to load
SURCONF=/etc/suricata/suricata-debian.yaml

# Listen mode: pcap or nfqueue
# depending on this value, only one of the two following options
# will be used
# Please note that IPS mode is only available when using nfqueue
LISTENMODE=nfqueue

# Interface to listen on (for pcap mode)
IFACE=eth0

# Queue number to listen on (for nfqueue mode)
NFQUEUE=0

# Load Google TCMALLOC if libtcmalloc-minimal0 is installed
# This _might_ give you very very small performance gain....
TCMALLOC="YES"

# Pid file
PIDFILE=/var/run/suricata.pid

Modify /etc/oinkmaster.conf to include the following in the # URL examples follows. section:

url = http://rules.emergingthreats.net/open/suricata/emerging.rules.tar.gz

Execute the following to fetch the rules

oinkmaster -C /etc/oinkmaster.conf -o /etc/suricata/rules

Note: Default rules are set to PASS to avoid interruptions to your traffic. It is up to you to tune this the right way.

Fix logging

Limit barnyard2 log size limit to 100mb.

/etc/suricata/suricata-debian.yaml

 # alert output for use with Barnyard2
  - unified2-alert:
      enabled: yes
      filename: unified2.alert

      # File size limit.  Can be specified in kb, mb, gb.  Just a number
      # is parsed as bytes.
      limit: 100mb

Make sure the logging: section of suricata-debian.yaml is configured properly.

logging:

  # The default log level, can be overridden in an output section.
  # Note that debug level logging will only be emitted if Suricata was
  # compiled with the --enable-debug configure option.
  #
  # This value is overriden by the SC_LOG_LEVEL env var.
  default-log-level: notice

  # The default output format.  Optional parameter, should default to
  # something reasonable if not provided.  Can be overriden in an
  # output section.  You can leave this out to get the default.
  #
  # This value is overriden by the SC_LOG_FORMAT env var.
  #default-log-format: "[%i] %t - (%f:%l) <%d> (%n) -- "

  # A regex to filter output.  Can be overridden in an output section.
  # Defaults to empty (no filter).
  #
  # This value is overriden by the SC_LOG_OP_FILTER env var.
  default-output-filter:

  # Define your logging outputs.  If none are defined, or they are all
  # disabled you will get the default - console output.
  outputs:
  - console:
      enabled: no
  - file:
      enabled: yes
      filename: /var/log/suricata.log
  - syslog:
      enabled: no
      facility: local5
      format: "[%i] <%d> -- "

Lastly, add logrotate rules.

/etc/logrotate.d/suricata

/var/log/suricata/*.log /var/log/suricata/*.json
{
    rotate 3
    missingok
    nocompress
    create
    sharedscripts
    postrotate
            /bin/kill -HUP $(cat /var/run/suricata.pid)
    endscript
}

Restart suricata

service suricata restart

Snorby

Reference: http://virtuallyhyper.com/2014/04/snort-debian/

Snorby is the front-end interface for the IDS solution.

Prerequisites

apt-get install libyaml-dev git-core default-jre imagemagick libmagickwand-dev wkhtmltopdf build-essential libssl-dev libreadline-gplv2-dev zlib1g-dev libsqlite3-dev libxslt1-dev libxml2-dev libmysqlclient-dev libmysql++-dev libcurl4-openssl-dev ruby ruby-dev mysql-server imagemagick apache2 libxml2-dev libxslt-dev

Assign a good password to mysql, if this is your first time installing. Referenced below as s3cr3tsauce. Substitute with your actual password.

Snorby installs a number of Ruby gems. To speed up their installation, run the following two commands to prevent the install of documentation when gems are installed:

echo "gem: --no-rdoc --no-ri" > ~/.gemrc
sh -c "echo gem: --no-rdoc --no-ri > /etc/gemrc"

Install bundler and rails

gem install bundler rails wkhtmltopdf
gem install rake --version=0.9.2

Installation

cd /opt
git clone http://github.com/Snorby/snorby.git

Fix gemfile.lock for newer mysql version (if not already fixed).

/opt/snorby/Gemfile.lock

    ' - do_mysql (~> 0.10.6)'
    '+ do_mysql (~> 0.10.17)'

    '- do_mysql (0.10.16)'
    '+ do_mysql (0.10.17) '
cd /opt/snorby
bundle install

Configuration

snorby_config.yml

Create /opt/snorby/config/snorby_config.yml

production:
  domain: 'demo.snorby.org'
  wkhtmltopdf: /usr/bin/wkhtmltopdf
  ssl: false
  mailer_sender: 'snorby@snorby.org'
  geoip_uri: "http://geolite.maxmind.com/download/geoip/database/GeoLiteCountry/GeoIP.dat.gz"
  rules:
    - ""
  authentication_mode: database

Snorby Configuration for MySQL

Create /opt/snorby/config/database.yml

# Snorby Database Configuration
#
# Please set your database password/user below
# NOTE: Indentation is important.
#
snorby: &snorby
  adapter: mysql
  username: root
  password: "s3cr3tsauce" # Example: password: "s3cr3tsauce"
  host: localhost

development:
  database: snorby
  <<: *snorby

test:
  database: snorby
  <<: *snorby

production:
  database: snorby
  <<: *snorby

This will do a basic setup.

cd /opt/snorby
RAILS_ENV=production bundle exec rake snorby:setup

Create lower privileged snorby user in MySQL

mysql -u root -p

At mysql> prompt

create user 'snorby'@'localhost' IDENTIFIED BY 'snorby';
grant all privileges on snorby.* to 'snorby'@'localhost' with grant option;
flush privileges;
exit

Modify /opt/snorby/config/database.yml to use the lower privileged user.

# Snorby Database Configuration
#
# Please set your database password/user below
# NOTE: Indentation is important.
#
snorby: &snorby
   adapter: mysql
   username: snorby
   password: "snorby" # Example: password: "s3cr3tsauce"
   host: localhost
 
development:
   database: snorby
   <<: *snorby
 
test:
   database: snorby
   <<: *snorby
 
production:
   database: snorby
   <<: *snorby

Snorby Setup

cd /opt/snorby
bundle exec rails server -e production

Start Snorby

bundle exec rails server -e production -b 0.0.0.0

Browse to http://your_host_address:3000

login: snorby@snorby.org or snorby@example.com (can reference db/seed.db for user setup)

password: snorby

Nuke Snorby database and start over (if needed)

If you need to nuke the database and start over at the Snorby Configuration for MySQL step.

mysql -u root -p
drop database snorby;

Repeat the above steps.

Note: The snorby user already exists. Thus you won't need to recreate it.

Snorby Enable Worker

Administration -> Worker & Job Queue -> Start Worker

Troubleshooting Snorby workers (if needed)

Reference: https://github.com/Snorby/snorby/wiki/Snorby-Worker-Troubleshooting

cd /opt/snorby
RAILS_ENV=production bundle exec rails c 

Snorby Worker

You should never really need to run the below commands. They are all available within the Snorby interface but documented here just in case.

Snorby::Worker.stop      # Stop The Snorby Worker
Snorby::Worker.start     # Start The Snorby Worker
Snorby::Worker.restart   # Restart The Snorby Worker

This will manually run the sensor cache job - pass true or false for verbose output

Snorby::Jobs::SensorCacheJob.new(true).perform

This will manually run the daily cache job - once again passing true or false for verbose output

Snorby::Jobs::DailyCacheJob.new(true).perform

Clear All Snorby Cache - You must pass true to this method call for confirmation.

Snorby::Jobs.clear_cache

If the Snorby worker is running this will start the cache jobs and set the run_at time for the current time.

Snorby::Jobs.run_now!

barnyard2

Barnyard collects alerts from Suricata and stuffs them into a database for Snorby front-end interface to display.

Prerequisites

apt-get install dh-autoreconf libpcap-dev libmysqld-dev libdaq-dev mysql-client autoconf

or

apt-get install dh-autoreconf libpcap-dev libmysqld-dev mysql-client autoconf flex bison

Install daq

Prerequisites

apt-get install bison flex

Fetch Source

cd /source
wget https://www.snort.org/downloads/snort/daq-2.0.6.tar.gz

Decompress

tar xvf daq-2.0.6.tar.gz

Build

cd /source/daq-2.0.6
./configure
make
make install

Install libdnet

Fetch from: http://libdnet.sourceforge.net/

cd /source
wget http://downloads.sourceforge.net/project/libdnet/libdnet/libdnet-1.11/libdnet-1.11.tar.gz?r=http%3A%2F%2Flibdnet.sourceforge.net%2F&ts=1461552505&use_mirror=iweb

Fix the crappy naming of the downloaded file

mv libdnet-1.11.tar.gz?r=http:%2F%2Flibdnet.sourceforge.net%2F libdnet-1.11.tar.gz

Decompress

tar xvf libdnet-1.11.tar.gz

Build

cd /source/libdnet-1.11
./configure
make
make install

Installation

Fetch source

cd /opt
git clone https://github.com/firnsy/barnyard2.git

Build source

cd /opt/barnyard2
./autogen.sh
./configure --with-mysql --with-mysql-libraries=/usr/lib/x86_64-linux-gnu/
make 
make install

Configuration

Modify eth0.201 to match the ethernet device you wish to listen on. This configuration is for CenturyLink FTTH. Most installations will use eth0

/etc/suricata/barnyard2.conf

config reference_file:      /etc/suricata/reference.config
config classification_file: /etc/suricata/classification.config
config gen_file:            /etc/suricata/rules/gen-msg.map
config sid_file:            /etc/suricata/rules/sid-msg.map
config interface: eth0.201
input unified2

# define the full waldo filepath.
config waldo_file: /var/log/suricata/suricata.waldo

# database: log to a variety of databases
output database: log, mysql, user=snorby password=snorby dbname=snorby host=localhost sensor_name=Cervin

Create log directory for barnyard2

mkdir -p /var/log/barnyard2

Start barnyard2

Start barnyard2

barnyard2 -c /etc/suricata/barnyard2.conf -d /var/log/suricata -f unified2.alert -w /var/log/suricata/suricata.waldo -D

Put NIC in promiscuous mode

ifconfig enp0s3 promisc

or

ip link set enp0s3 promisc on


Start Script (Suricata+Barnyard2+Snorby)

This script will change the ethernet properties and put the interfaces in promiscuous mode. After which, it will start Suricata, Snorby, and Barnyard2

I made this because I was having trouble with the traditional start scripts working with systemd.

/firewall/runids

#!/bin/bash

# 2016/09 Fyzix

echo "=== Killing any existing processes..."
PROCESS_ID1=`ps -ef|egrep /usr/bin/suricata|egrep suricata-debian.yaml|awk '{print $2}'`
PROCESS_ID2=`ps -ef|egrep barnyard2|egrep /var/log/suricata|awk '{print $2}'`
PROCESS_ID3=`ps -ef|egrep rails|egrep production|awk '{print $2}'`
PROCESS_ID4=`ps -ef|egrep delayed_job|egrep -v grep|awk '{print $2}'`
kill ${PROCESS_ID1}
kill ${PROCESS_ID2}
kill ${PROCESS_ID3}
kill ${PROCESS_ID4}
rm /var/run/suricata.pid
echo "=== Processes killed (if any)."

# Put ethernet in promiscuous mode and modify ethernet parameters so suricata can access the devices without error
echo "=== Reconfiguring interfaces for Promiscuous mode..."
ifconfig br0 promisc
ifconfig eth0 promisc
ifconfig eth0.201 promisc # Used for Centurylink FTTH # Disable if you don't have FTTH
ifconfig eth1 promisc
ifconfig eth2 promisc
ifconfig eth3 promisc
ethtool -K eth0 tx off rx off sg off gso off gro off
ethtool -K eth1 tx off rx off sg off gso off gro off
ethtool -K eth2 tx off rx off sg off gso off gro off
ethtool -K eth3 tx off rx off sg off gso off gro off
echo "=== Configuration changes complete."

# Make sure suricata rules are up-to-date
echo "=== Fetching latest emerging threat rules..."
oinkmaster -C /etc/oinkmaster.conf -o /etc/suricata/rules
echo "=== Rule fetch complete."

# Start Suricata
#/usr/bin/suricata -c /etc/suricata/suricata-debian.yaml --pidfile /var/run/suricata.pid -q 0 -D
#/usr/bin/suricata -c /etc/suricata/suricata-debian.yaml --pidfile /var/run/suricata.pid --af-packet -i eth0.201 -D
echo "=== Starting Suricata..."
/usr/bin/suricata -c /etc/suricata/suricata-debian.yaml --pidfile /var/run/suricata.pid --af-packet -D
echo "=== Suricata Started."

# Start Snorby
echo "=== Starting Snorby..."
cd /opt/snorby
# The below command will start Snorby in console mode
#RAILS_ENV=production bundle exec rails c
bundle exec rails server -e production -d -b 0.0.0.0
sleep 3
RAILS_ENV=production script/delayed_job start
sleep 3
RAILS_ENV=production /usr/local/bin/rails runner 'Snorby::Jobs::SensorCacheJob.new(false).perform; Snorby::Jobs::DailyCacheJob.new(false).perform'
echo "=== Snorby Started."

# Start Barnyard2
echo "=== Starting Barnyard2..."
barnyard2 -c /etc/suricata/barnyard2.conf -d /var/log/suricata -f unified2.alert -w /var/log/suricata/suricata.waldo -D

echo ""
echo "Browse to http://routerip:3000/"
echo "Be sure Snorby Workers are started."
echo ""

Fix permissions

chmod u+x /firewall/runids