Network monitoring with NTOPng

In our academic institution we have a pretty complex network with ~50 VLANs, few DHCP and DNS servers; and the requisite for monitoring and logging our traffic. Traffic from our main router Mikrotik CCR1036-8G-2S+ is collected, forwarded, and logged onto separate server. Server runs Debian Jessie operating system and ntopng + nprobe packages.
Ntopng is the next generation version of the original ntop, a network traffic probe that shows the network usage, similar to what the popular top Unix command does. ntopng is based on libpcap and it has been written in a portable way in order to virtually run on every Unix platform, MacOSX and on Windows as well.
Idea is that we can corelate all logs and informations from our network/servers/users and to have a REALTIME monitoring of a network. We have a centralized log managment system for all crucial system logs. System has two nodes/machines. One of them is configured like main syslog machine, which receives and stores all recieved logs locally, then forwards logs to another machine where we have Graylog software installed.
Main router is configured to forward data over its traffic-flow feature to the nprobe. MikroTik Traffic-Flow is a system that provides statistic information about packets which pass through the router. Nprobe is converting netflow/traffic-flow to JSON format adequate for ntopng, also it logs traffic flow to local syslog, which then forwards logs to the main syslog server, and main syslog server forward logs to Graylog server. Realtime network statistics can be seen on ntop webgui. Simple as that :)
Installation and configuration
Install and configure firewall (ferm in our case).
apt-get install ferm vim /etc/ferm/ferm.conf
# -*- shell-script -*- # # Configuration file for ferm(1). # table filter { chain INPUT { policy DROP; # connection tracking mod state state INVALID DROP; mod state state (ESTABLISHED RELATED) ACCEPT; # allow local packet interface lo ACCEPT; # respond to ping proto icmp ACCEPT; # allow NETFLOW saddr ROU.TER.IP.ADDR/32 proto (tcp udp) dport 2055 ACCEPT; # allow NTOPNG connections proto tcp dport (80 443) ACCEPT; # allow SSH connections proto tcp dport ssh ACCEPT; } chain OUTPUT { policy ACCEPT; # connection tracking #mod state state INVALID DROP; mod state state (ESTABLISHED RELATED) ACCEPT; } chain FORWARD { policy DROP; # connection tracking mod state state INVALID DROP; mod state state (ESTABLISHED RELATED) ACCEPT; } } # IPv6: #domain ip6 { # table filter { # chain INPUT { # policy ACCEPT; # # ... # } # # ... # } #}
Reload firewall
/etc/init.d/ferm reload
Install ntopng packages
wget http://apt.ntop.org/jessie/all/apt-ntop.deb dpkg -i apt-ntop.deb apt-get clean all apt-get update apt-get install pfring nprobe ntopng ntopng-data n2disk
Configure ntopng
vim /etc/ntopng/ntopng.conf
-G=/var/run/ntopng.pid -i="tcp://127.0.0.1:5556" --local-networks="10.0.0.0/8" #define your own local networks
Restart ntopng
/etc/init.d/ntopng restart
Install nginx as reverse proxy to ntop webgui
apt-get install nginx vim /etc/nginx/sites-available/ntop
# Upstream to Ntop frontend proxy_next_upstream error timeout; upstream ntop_web_interface { server localhost:3000 fail_timeout=0; } # Redirect everything to https server { listen 80 default_server; server_name ntop.example.com; return 301 https://ntop.example.com$request_uri; } server { listen 443 ssl default_server; server_name ntop.example.com; # SSL certificate ssl_certificate /etc/nginx/ssl/ntop.example.com.crt; # replace with your own certificate - https://letsencrypt.org/ ssl_certificate_key /etc/nginx/ssl/ntop.example.com.key; # replace with your own certificate - https://letsencrypt.org/ location / { proxy_pass_header Date; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header Host $http_host:$server_port; proxy_redirect off; proxy_set_header X_FORWARDED_PROTO $scheme; chunked_transfer_encoding off; proxy_pass http://ntop_web_interface; } }
Enable ntop site and restart nginx
cd /etc/nginx/sites-enabled ln -s ../sites-available/ntop . /etc/init.d/nginx restart
Start nprobe
nprobe -n none --syslog nprobe --collector-port 2055 --zmq tcp://0.0.0.0:5556 -b 2
Configure router via CLI
/ip traffic-flow set enabled=yes interfaces=all /ip traffic-flow target add address=NTOP.SERVER.IP.ADDR:2055 disabled=no version=5
Go to https://ntop.example.com and grab popcorns ...
Add new comment