ngrep and tcpflow – packet capture on a shoestring

The Ngrep and TCPflow packet capture tools are useful for fast access to packets on the wire. As you will see in the examples, they make grabbing text out of the network stream a piece of cake.

You may have heard of Wireshark (formerly Ethereal), a powerful network packet capture tool that enables a user to grab packets off the wire, load pcaps and analyse the data all in one GUI. While Wireshark is a must-have tool for many IT pro's there are times when a simple command line tool can get the job done faster.

Ngrep - or Network Grep Installation

On your Ubuntu (or Debian based) system install with apt-get. Under Fedora, Centos or RHEL if the package is not available in the repos, grab a copy of the rpm and install with a simple rpm -ivh (no dependencies required).

testbox:~#apt-get install ngrep
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  ngrep
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 29.1 kB of archives.
After this operation, 92.2 kB of additional disk space will be used.

Wow, take a look at that - 29.1kB had to be downloaded and 92.2 kB of disk space has been used by this tool. Maybe I should get a bigger hard drive!!

Ngrep Examples

A couple of basic examples to get you started.

testbox:~#ngrep -d wlan0 '^POST'
interface: wlan0 (192.168.1.0/255.255.255.0)
match: ^POST

The syntax is -d wlan0 for the device you wish to capture from, followed by the expression to match. This example will match packets with POST at the start of the line, or HTTP POST requests in a simple text output format. The '#' marks indicate packets that did not match the expression. Further filtering can be done on ports and ip addresses.

Here is a more telling example to give you an idea of the possibilities.

testbox:~#ngrep -t -d wlan0 'pwd'
interface: wlan0 (192.168.1.0/255.255.255.0)
match: pwd
#############
T 2013/05/08 23:30:46.559360 192.168.1.100:48187 -> 173.255.232.18:80 [AP]
  POST /wp-login.php HTTP/1.1..Host: hackertarget.com..User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0..Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8..Accept-Language: en-US,en;q=0.5..Accept-Encoding: gzip, deflate..Referer: http://hackertarget.com/wp-login.php..Connection: keep-alive..Content-Type: application/x-www-form-urlencoded..Content-Length: 106....log=admin&pwd=testpassword&wp-submit=Log+In&redirect_to=http%3A%2F%2Fhackertarget.com%2Fwp-adminF&testcookie=1                                                                                                          
###############################################################################################################^Cexit
124 received, 0 dropped

The addition of the -t will put a timestamp on the matching results. Notice what I have done here, a simple grep for the string 'pwd' has shown the HTTP POST request with my login and password for the https://hackertarget.com/ login page. A quick example that demonstrates the importance of using the SSL version of the site (https://hackertarget.com/).

tcpflow - logging all the data

With tcpflow the installation is similar to that of ngrep, at least under Ubuntu.

apt-get install tcpflow

tcpflow will log all the tcpflows - or TCP sessions into text files in the current directory where it runs. Use tcpdump command line switches for determining what to capture.

tcpflow -i wlan0 'port 80'

This example will capture all HTTP flows over port 80 and store them as text files. A great way to troubleshoot web applications, or network protocols.

Tshark - another worthy command line packet capture tool

tshark is part of the Wireshark package, and is basically a text or console based version of Wireshark. It has many options and can be used to perform much of what ngrep and tcpflow do. However, the advantage of ngrep and tcpflow is their simplicity and ease of use. It will often come down to what tools you have available on the system.

These examples just touch the surface whether troubleshooting or performing security analysis; any plain text protocol can be inspected, POP3, SMTP, IRC, DNS and HTTP are just a few possibilities. On a related note the excellent Zeek performs excellent flow analysis and is a tool worth investigating if you are performing security related packet captures.