Zeek Dashboard using Grafana

Combine the power of Zeek network analysis with Grafana visualisation to generate a dashboard of the traffic from any pcap.

This weekend's project highlights the power of integrating different open source tools. The two docker containers can be spun up immediately to get immediate insight into a network pcap; however the real advantage of this mini project is as a starting point to build more a custom solution and learn more about these excellent tools.

Explore the Grafana dashboard, and dig deeper into Zeek analysis to develop custom insights into your network environment. Go from a boring pcap to nice looking visuals in minutes.

Zeek Dashboard

Introduction to Zeek Visualisation

Zeek is a powerful network analysis tool and is commonly used with Elasticsearch and Kibana to build dashboards that visualise the data. Rather than go down that rabbit hole we have opted to use a lightweight solution using Grafana.

In the first part of this series we built a custom docker container for Zeek, to process the pcap and output the zeek log files. In the zeek container is a custom python script that will convert these log files to a sqlite3 database. Using a sqlite3 data source is a simple way to access the data within Grafana. There are many other ways this could be implemented, but this process is simple and portable.

A primary aim of this mini project is to demonstrate how combining open source tools and creating simple data processing pipelines we can create elegant solutions, while gaining a deeper understanding of how the tools work.

Generating the SQLite Data Source

In the previous part of this series we generated the zeek docker container and ran zeek against our pcap. This resulted in the zeek log files being generated into the /data/ directory.

From bash in the Zeek container we can run the custom script to convert the zeek log files to an sqlite3 database.

root@42f7fd693ca0:/data# zeek-to-sqlite.py
root@42f7fd693ca0:/data# ls
2015-07-28_mixed.pcap  conn.log  files.log        known_services.log  ocsp.log           software.log  telemetry.log  x509.log
analyzer.log           dns.log   http.log         loaded_scripts.log  packet_filter.log  ssl.log       websocket.log  zeek_logs.db
capture_loss.log       dpd.log   known_hosts.log  notice.log          pe.log             stats.log     weird.log
root@42f7fd693ca0:/data# 

Pay attention to the zeek_logs.db, this is the sqlite database. Viewing the db you will notice the logs from zeek are now tables within the database. Note that the column names have changed from any '.' to '_'. This is relevant when creating sqlite queries in Grafana.

Grafana and Docker Compose

Now that we have our SQLite datasource with the Zeek data. We can start up the Grafana docker container and start our analysis.

:~$ cd pcap-did-what/grafana-custom/
:~/pcap-did-what$ sudo docker-compose up -d
:~/pcap-did-what$ sudo docker ps -a
CONTAINER ID   IMAGE             COMMAND       CREATED         STATUS                      PORTS                                       NAMES
daba724a6548   grafana/grafana   "/run.sh"     1 hours ago    Up 1 hours                 0.0.0.0:3000->3000/tcp, :::3000->3000/tcp   grafana

If you wish to customise the build simply review the docker-compose.yml file. The dashboards and data sources directories contain the configuration information for the creation of the SQLite data source and the Zeek Dashboard within the newly created Grafana container. These files could be extend to build multiple dashboards or use other data sources.

Accessing Grafana and Zeek Dashboard

Grafana should now be running on its default port of 3000, so from your browser access https://127.0.0.1:3000 (or whichever IP you are running on).

The initial login will be admin/admin, and this will need to be changed on first login. The authentication information and any changes to the Grafana configuration will be saved in the Grafana storage that was created with the docker-compose.yml. This means stopping and starting the container will not affect these changes. To restart with a clean Grafana remove the ./grafana-storage directory.

- ./grafana-storage:/var/lib/grafana

After accessing the Dashboard, the first thing you may need to change to see the data will be date range. If you are using historical network pcaps then the date range on the dashboard will need to be adjusted to catch the relevant time frame.

Conclusion

In this post, we explored the powerful combination of Zeek and Grafana for network monitoring and visualization. By leveraging Zeek's extensive network analysis capabilities and Grafana's intuitive dashboard creation, we were able to gain valuable insights into our network traffic and identify potential security threats.

As you become more familiar with Zeek and Grafana, you'll discover more opportunities depending on your environment and use case for customization. This knowledge will enable you to fine-tune your Zeek analysis, create more sophisticated Grafana dashboards, and proactively detect and respond to threats.