You may have heard of Wireshark (formerly Ethereal), a powerful network packet capture tool that enables a user to grab packets off the wire, load pcaps and analyse the data all in one GUI. While Wireshark is a must have tool for many IT pro’s there are times when a simple command line tool can get the job done faster.

Ngrep – or Network Grep

On your Ubuntu (or Debian based) system it is a simple matter of installing with apt-get. Under Fedora, Centos or RHEL if the package is not available in the repos, grab a copy of the rpm and install with a simple rpm -ivh (no dependencies required).

testbox:~#apt-get install ngrep
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  ngrep
0 upgraded, 1 newly installed, 0 to remove and 0 not upgraded.
Need to get 29.1 kB of archives.
After this operation, 92.2 kB of additional disk space will be used.

Wow, take a look at that – 29.1kB had to be downloaded and 92.2 kB of disk space has been used by this tool. Maybe I should get a bigger hard drive!!

A couple of basic examples to get you started with ngrep.

testbox:~#ngrep -d wlan0 '^POST'
interface: wlan0 (192.168.1.0/255.255.255.0)
match: ^POST

The syntax is -d wlan0 for the device you wish to capture from, followed by the expression to match. This example will match packets with POST at the start of the line, or HTTP POST requests in a simple text output format. The ‘#’ marks indicate packets that did not match the expression. Further filtering can be done on ports and ip addresses.

Here is a more telling example to give you an idea of the possibilities.

testbox:~#ngrep -t -d wlan0 'pwd'
interface: wlan0 (192.168.1.0/255.255.255.0)
match: pwd
#############
T 2013/05/08 23:30:46.559360 192.168.1.100:48187 -> 173.255.232.18:80 [AP]
  POST /wp-login.php HTTP/1.1..Host: hackertarget.com..User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0..Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8..Accept-Language: en-US,en;q=0.5..Accept-Encoding: gzip, deflate..Referer: http://hackertarget.com/wp-login.php..Connection: keep-alive..Content-Type: application/x-www-form-urlencoded..Content-Length: 106....log=admin&pwd=testpassword&wp-submit=Log+In&redirect_to=http%3A%2F%2Fhackertarget.com%2Fwp-adminF&testcookie=1                                                                                                              
###############################################################################################################^Cexit
124 received, 0 dropped

The addition of the -t will put a timestamp on the matching results. Notice what I have done here, a simple grep for the string ‘pwd’ has shown the HTTP POST request with my login and password for the http://hackertarget.com/ login page. A quick example that demonstrates the importance of using the SSL version of the site (https://hackertarget.com/).

tcpflow – logging all the datas

With tcpflow the installation is similar to that of ngrep, at least under Ubuntu.

apt-get install tcpflow

tcpflow will log all the tcpflows – or TCP sessions into text files in the current directory where it runs. Use tcpdump command line switches for determining what to capture.

tcpflow -i wlan0 'port 80'

This example will capture all HTTP flows over port 80 and store them as text files. A great way to troubleshoot web applications, or network protocols.

Tshark – another worthy command line packet capture tool

tshark is part of the Wireshark package, and is basically a text or console based version of Wireshark. It has many options and can be used to perform much of what ngrep and tcpflow do. However, the advantage of ngrep and tcpflow is their simplicity and ease of use. It will often come down to what tools you have available on the system.

These examples just touch the surface whether troubleshooting or performing security analysis; any plain text protocol can be inspected, POP3, SMTP, IRC, DNS and HTTP are just a few possibilities. On a related note the excellent bro (no longer bro-ids) performs excellent flow analysis and is a tool worth investigating if you are performing security related packet captures.

The post ngrep and tcpflow – packet capture on a shoestring appeared first on Online Vulnerability Scanners and Port Scans.

Why you need an external port scanner

To understand how vulnerable your systems are to external attackers, you need to understand what they look like on the network from an external or Internet facing perspective. A port scan conducted from outside a network perimeter will map and identify vulnerable systems.

Technical operations staff need to know what their network perimeter looks like from the outside. The perimeter may be a single IP gateway, a hosted Internet server or a whole Class B network; it does not matter – you need to understand what services Internet based threats can see and what they are able to access.

If you are a systems administrator or a security analyst for an organisation having access to an external port scanner will provide a number of benefits; The most important being that you should understand and know exactly what services are listening on your perimeter. Testing should be performed at least monthly and ideally more often, to monitor for changes to the perimeter.

Firewall Testing

A firewall’s primary function is to block unauthorised packets from being able to reach listening services. The firewall can be situated on the perimeter of an organisations network or it can be on an internal network. It can also be on the end point whether that is a client desktop or a Internet server such as a web server or mail server.

Multiple firewalls and filtering devices increases the complexity of assessing a network. Using a port scanner one is able to quickly assess what ports are being permitted through the various layers of defence and are able to reach services on the end point host.

To effectively test a firewall and network for external access points, it is necessary to perform the port scanning from a remote host. By using the HackerTarget.com hosted online port scanner service you are able to quickly test a range of IP Addresses or a single IP address. All 65’535 ports can be tested at the click of a mouse, with the results delivered to your email address for review.

From the results of the port scan you are able to determine the state of a port:

Why ingress firewall filtering is required

Restrict access to vulnerable services, reduce attack surface of Internet facing systems and reduce ability of an attacker to open back-doors on Internet facing ports.

Why egress firewall filtering is required

Data ex-filtration and outbound initiated remote access. Command shells and other remote access can be achieved by a system initiating an outbound connection. Limiting the available outbound ports can make this outbound communication more difficult for an attacker. Note – this does not entirely solve the problem as advanced tools and attackers are able to initiate communication through multiple means including over https proxy servers, STMP and even DNS queries.

Troubleshooting Network Services

When installing and configuring Internet facing services it will often be necessary to troubleshoot a network configuration in order to get a service up and running. For example you may have correctly setup the service on the server with everything operating correctly, however an external firewall may be blocking remote access to this service.

While the situations in which network troubleshooting is required are varied, it is a common methodology to perform an external port scan against the network port or system to quickly understand where the problem may lie. If you are able to connect to a service from the internal host but unable to connect from external, you can make a pretty good guess at where the problem might lie. By performing a port scan using an external online port scan you are able to quickly confirm that all the required services are being filtered – hence your troubleshooting can move to looking at any external or host based firewalls that are blocking that port.

Mapping Networks and Services

In order to determine how vulnerable a network or host is to exploitation, it is necessary to know what services are running and whether they are externally facing (or accessible from the Internet). By performing a remote port scan against the network IP range or against a specific host it is possible to determine not only the open ports but also the types of services running on those ports. This is known as service detection and is a feature of most well known port scanners such as the nmap port scanning tool.

Further more identification of the actual operating system is also possible, either from the service identification or through more low level analysis of the packets coming back from the host.

System and network administrators will also utilize port scanners to map the external network of a host or organisation. Networks change over time and documentation is not always kept current, so a quick port scan of the services listening on a network will help a system administrator to understand the layout of the network.

The post Firewall Testing with a remote Port Scanner appeared first on Online Vulnerability Scanners and Port Scans.

Head over to the MaxMind website and grab the latest version of the GeoLiteCity.dat.gz file.

Download: http://dev.maxmind.com/geoip/geolite

Now uncompress the download with gzip.

gzip -d GeoLiteCity.dat.gz

If you take a look in /opt/splunk/etc/apps/maps/bin/ of your Splunk install you will see the version of the GeoLiteCity.dat file is August 20, 2011. Time to update it to the latest version.

cp GeoLiteCity.dat /opt/splunk/etc/apps/maps/bin/

Start searching Splunk with the latest GeoIP data from MaxMind. It really is that easy.

The post Update GeoIP data for Splunk App appeared first on Online Vulnerability Scanners and Port Scans.

Open Source Splunk Alternative

If you are interesting in a purely Open Source log search engine, take a look at ELSA – Enterprise Log Search and Archive this is a relatively new project that is making good progress. It has been included on the latest Security Onion release.

Another Open Source log management option is Greylog2. I am yet to test or explore this alternative to Splunk but I have read some good reviews and it looks promising.

Download Splunk for Ubuntu

Splunk runs on a wide range of computing platforms including Windows, Linux, FreeBSD, OSX, Solaris, AIX and even HPUX.

http://www.splunk.com/download?r=header

We are after the Linux download option, specifically the .deb file as Ubuntu uses the Debian based .deb package format for binary installs. It is a matter of selecting either 32bit or 64bit and then downloading the .deb file.

Not sure whether your Ubuntu is 32bit or 64bit? The easiest way to check this is to use a Unix command uname -a in a terminal window. Bring up a terminal window and type in that command. x64 indicates 64bit while i686 i386 indicates a 32 bit install.

The 32 bit package is about 35.5mb, you will need to signup for a Splunk account to begin the download. It is worth creating an account you will remember as this same acocunt will be used to download additional plugins (apps) from the Splunk site.

Once the download is complete you can install it with the following dpkg command as seen in the output below:

fred@x-wing1:~$ sudo dpkg -i Downloads/splunk-5.0.1-143156-linux-2.6-intel.deb 
[sudo] password for fred: 
Selecting previously unselected package splunk.
(Reading database ... 239507 files and directories currently installed.)
Unpacking splunk (from .../splunk-5.0.1-143156-linux-2.6-intel.deb) ...
Setting up splunk (5.0.1-143156) ...
----------------------------------------------------------------------
Splunk has been installed in:
        /opt/splunk

To start Splunk, run the command:
        /opt/splunk/bin/splunk start


To use the Splunk Web interface, point your browser at:

http://x-wing1:8000

Complete documentation is at http://docs.splunk.com/Documentation/Splunk
----------------------------------------------------------------------

Yes it is that easy, no dependencies or mucking around. Now its time to start the Splunk server.

sudo /opt/splunk/bin/splunk start

After some initial setup, you should see:

The Splunk web interface is at http://x-wing1:8000

Login and change your password. You now have your very own Splunk server, just like the cool kids.

Feed Splunk Data and Search!

Start getting data in the system and then you can search on that data. Data can be input from simple files for some one off analysis, it can read known log files or can listen on a port similar to a syslog server. It is very flexible, for example running it on a TCP port you could even use netcat to pipe a file over the network into Splunk server, or have a syslog server forward some of its logs to the Splunk instance. This would leave you with your existing syslog infrastructure intact for archival purposes but you also have the Splunk instance for easy analysis.

Now you are up to the point where it depends on your network and requirements, so think about how you are going to use it, feed it some data and start searching for stuff. The stuff could be configuration issues, errors, utilization trends or security events. If you want to do some easy testing, just grab a web server log file or other log and feed it in directly with the a file or directory option.

This video is a good introduction to performing Splunk log searches and pulling relevant information from your data.

While I suspect most people will find value from the first day, as you explore the capabilities of the Splunk search engine you will find stuff – its a rabbit hole for systems administrators.

The post Install Splunk on Ubuntu in 5 mins appeared first on Online Vulnerability Scanners and Port Scans.

This selection of tools when utilized by a moderately skilled attacker has the potential to wreak havoc on an organizations network.

If you are interested in testing these tools they are all available to download and use for FREE. Most are open source with a couple of exceptions. They should not be used against systems that you do not have permission to attack. You could end up in jail.

1. Metasploit Framework – an open source tool for exploit development and penetration testing Metasploit is well known in the security community. Metasploit has exploits for both server and client based attacks; with feature packed communication modules (meterpreter) that make pwning systems fun! The framework now includes Armitage for point and click network exploitation. This is the go to tool if you want to break into a network or computer system.

Defending against Metasploit:

2. Ettercap – a suite of tools for man in the middle attacks (MITM). Once you have initiated a man in the middle attack with Ettercap use the modules and scripting capabilities to manipulate or inject traffic on the fly. Sniffing data and passwords are just the beginning; inject to exploit FTW!

Defending against Ettercap:

3. sslstrip – using HTTPS makes people feel warm, fuzzy and secure. Using sslstrip this security can be attacked, reducing the connection to an unencrypted HTTP session, whereby all the traffic is readable. Banking details, passwords and emails from your boss all in the clear. Even includes a nifty feature where the favicon on the unencrypted connection is replaced with a padlock just to make the user keep that warm and fuzzy feeling.

Defending against sslstrip:

4. evilgrade – another man in the middle attack. Everyone knows that keeping software updated is the way to stay secure. This little utility fakes the upgrade and provides the user with a not so good update. Can exploit the upgrade functionality on around 63 pieces of software including Opera, Notepad++, VMware, Virtualbox, itunes, quicktime and winamp! It really whips the llamas ass!

Defending against evilgrade:

5. Social Engineer Toolkit – makes creating a social engineered client side attack way too easy. Creates the spear phish, sends the email and serves the malicious exploit. SET is the open source client side attack weapon of choice.

Defending against SET:

6. sqlmap – SQL Injection is an attack vector that has been around for over 10 years. Yet it is still the easiest way to get dumps of entire databases of information. Sqlmap is not only a highly accurate tool for detecting sql injection; but also has the capability to dump information from the database and to even launch attacks that can result in operating system shell access on the vulnerable system.

Defending against sqlmap:

7. aircrack-ng – breaking holes in wireless networks for fun and profit. A suite of tools that enables all manner of wireless network attacks.

Defending against aircrack-ng:

8. oclHashcat – Need to get some passwords from the hashes you grabbed with sqlmap? Use this tool to bust them open. Over 48 different hashing algorithms supported. Will use the GPU (if supported) on your graphics card to find those hashes many times faster than your clunky old CPU.

Defending against oclHashcat:

9. ncrack – Brute force network passwords with this tool from Fyodor the creator of Nmap. Passwords are the weakest link and Ncrack makes it easy to brute force passwords for RDP, SSH, http(s), SMB, pop3(s), VNC, FTP, and telnet.

Defending against ncrack:

10. Cain and Abel – Cracking passwords, sniffing VOIP and Man in the Middle (MITM) attacks against RDP are just a few examples of the many features of this Windows only tool.

Defending against Cain and Abel:

11. Tor – push your traffic through this onion network that is designed to provide anonymity to the user. Note your traffic from the exit node is not encrypted or secured. Make sure you understand what it does before using it, Tor provides anonymity not encrypted communication.

Defending against Tor:

If you are interested in testing these offensive security tools you should take a look at the BackTrack Linux distribution. It includes many of these and other tools pre-installed.

These tools are used by security professionals around the world to demonstrate security weakness.

The post 11 Offensive Security Tools for SysAdmins appeared first on Online Vulnerability Scanners and Port Scans.

The following are 10 essential security tools that will help you to secure your systems and networks. These open source security tools have been given the essential rating due to the fact that they are effective, well supported and easy to start getting value from.

1. Nmap – map your network and ports with the number one port scanning tool. Nmap now features powerful NSE scripts that can detect vulnerabilities, misconfiguration and security related information around network services. After you have nmap installed be sure to look at the features of the included ncat – its netcat on steroids.

2. OpenVAS – open source vulnerability scanning suite that grew from a fork of the Nessus engine when it went commercial. Manage all aspects of a security vulnerability management system from web based dashboards. For a fast and easy external scan with OpenVAS try our online OpenVAS scanner.

3. OSSEC – host based intrusion detection system or HIDS, easy to setup and configure. OSSEC has far reaching benefits for both security and operations staff.

4. Security Onion – a network security monitoring distribution that can replace expensive commercial grey boxes with blinking lights. Security Onion is easy to setup and configure. With minimal effort you will start to detect security related events on your network. Detect everything from brute force scanning kids to those nasty APT’s.

5. Metasploit Framework – test all aspects of your security with an offensive focus. Primarily a penetration testing tool, Metasploit has modules that not only include exploits but also scanning and auditing.

6. OpenSSH – secure all your traffic between two points by tunnelling insecure protocols through an SSH tunnel. Includes scp providing easy access to copy files securely. Can be used as poor mans VPN for Open Wireless Access points (airports, coffee shops). Tunnel back through your home computer and the traffic is then secured in transit. Access internal network services through SSH tunnels using only one point of access. From Windows, you will probably want to have putty as a client and winscp for copying files. Under Linux just use the command line ssh and scp.

7. Wireshark – view traffic in as much detail as you want. Use Wireshark to follow network streams and find problems. Tcpdump and Tshark are command line alternatives. Wireshark runs on Windows, Linux, FreeBSD or OSX based systems.

8. BackTrack – an Ubuntu based Linux distribution that is configured with hundreds of security testing tools and scripts. Backtrack is well known with penetration testers and hobbyists alike.

9. Nikto – a web server testing tool that has been kicking around for over 10 years. Nikto is great for firing at a web server to find known vulnerable scripts, configuration mistakes and related security problems. It won’t find your XSS and SQL web application bugs, but it does find many things that other tools miss. To get started try the Nikto Tutorial or the online hosted version.

10. Truecrypt – encrypt all the things. Truecrypt is a strong encryption utility that can encrypt entire volumes or create an encrypted container within a file system. Use Truecrypt to protect your flash drives. If it gets lost, even the NSA will have trouble reading the data.

The post 10 Essential Open Source Security Tools appeared first on Online Vulnerability Scanners and Port Scans.

Network architectures will vary but if you are deploying Internet facing Servers you generally should be configuring a host based firewall. It can provide protection to listening services that don’t need to be Internet accessible, in addition a firewall can make life more difficult for an attacker who does gain a foothold. Making it more difficult to create a backdoor listener for example.

When deploying an Ubuntu host based firewall you should also consider using the excellent open source HIDS software OSSEC.

The Ubuntu documentation portal has a good run down on implementing UFW.

Here is my shorter summary of UFW and Ubuntu Firewalls

Set the default rule, in case you are wondering this should be default DENY.

sudo ufw default deny

Logging is generally another good idea, lets enable it.

sudo ufw logging on

If you are connected over SSH then set your SSH allow rule now.

sudo ufw allow 22/tcp

HackerTarget.com runs SSH on 2222 to avoid brute forcing SSH bots. So the command is:

sudo ufw allow 2222/tcp

Now turn the firewall on (this applies the iptables commands).

sudo ufw enable

To turn the firewall off.

sudo ufw disable

Allow port 80 (for your webserver to server HTTP).

sudo ufw allow 80/tcp

Allow port 443 (as we have SSL enabled for our clients security).

sudo ufw allow 443/tcp

Allow port 25 (for your Email SMTP)

sudo ufw allow 25/tcp

You get the idea, it is also possible to enable rules that allow and block from specific IP addresses, after all it is just a script for iptables. See the Ubuntu Docs for details on this.

sudo ufw status

This command shows that the firewall is running and configured, now you should do a port scan and test it for real.

Since we run VPS servers on Linode and have configured dual stack IPv4 and IPv6 addresses our web server is happily serving on both protocols. iptables and ip6tables are two separate commands for the configuration of IPv4 and IPv6 firewalls. The excellent thing about UFW is the above commands enables the firewall on both IP stacks.

The post Firewalling Ubuntu with UFW for IPv4 + IPv6 appeared first on Online Vulnerability Scanners and Port Scans.

The detection focus of Bro IDS is more network flow rather than signature based and does not get the same attention as Snort or Suricata. In many installations where network defence is taken seriously Bro actually runs alongside Snort. Richard Bejtlich of TaoSecurity is fan, here is a video introduction.

Now lets get started on the Bro IDS Installation under Ubuntu 12.04

Grab the required packages with apt-get.

apt-get install libncurses5-dev g++ bison flex libmagic-dev libgeoip-dev libssl-dev build-essential python-dev libpcap-dev cmake swig2.0 libssl0.9.8

Some of these packages I already had installed, but it does not hurt to list all the requirements; apt-get will grab the missing ones and install them for us.

Now we will download bro-ids, we will download and install from source; they have a stable version 2.0 available for Debian 64 bit however there is a dependency issue.

So grab the source tarball, extract and install.

wget http://www.bro-ids.org/downloads/release/bro-2.0.tar.gz

tar zxvf bro-2.0.tar.gz
cd bro-2.0
./configure --prefix=/opt/bro2
make
make install

No errors? Good now add bro to your PATH.

export PATH=/opt/bro2/bin:$PATH

You can also add PATH=/opt/bro2/bin:$PATH to your ~/.profile file in your home directory to make the change permanent.

Bro is a powerful tool, for the most basic of installation steps we will follow the guide on the project page.

Edit the following files before starting:

$PREFIX/etc/node.cfg  -- configure network interface to monitor
$PREFIX/etc/networks.cfg -- configure local networks
$PREFIX/etc/broctl.cfg -- change MailTo address and the log rotation

To start the program simply enter broctl at a shell.

You are now in the broctl shell, from where you can give bro commands.

[BroControl] >

The first command to run, since this is a new installation is to run install. We will then run start.

[BroControl] > install
warning: cannot read '/opt/bro2/spool/broctl.dat' (this is ok on first run)
creating policy directories ... done.
installing site policies ... done.
generating standalone-layout.bro ... done.
generating local-networks.bro ... done.
generating broctl-config.bro ... done.
updating nodes ... done.
[BroControl] > start
starting bro ...
[BroControl] > status
Name       Type       Host       Status        Pid    Peers  Started              
bro        standalone localhost  running       22165  0      22 Aug 12:31:55 

You now have Bro-IDS running on your system. Woo hoo. This is just the beginning, check out the guide and follow the white rabbit.

The post Bro-IDS installation in Ubuntu 12.04 appeared first on Online Vulnerability Scanners and Port Scans.

What I have done is targeted the 3 different vulnerability scanners in a “black box” test against a Metasploitable version 2 Virtualbox.

Background Info

In 2010 I planned on doing an OpenVAS vs Nessus review, well it seems time got away and now its the middle of 2012. There is now a new high profile vulnerability scanner on the block; Nexpose from Rapid 7 has gained attention in recent years due to the adoption of its rock star big brother Metasploit.

In the testing I am deliberately focusing on the network vulnerability scanning capabilities rather than looking at the web application vulnerability detection in detail. It is my belief that a network vulnerability scanner should be capable of identifying poorly configured services, default services that have poor security and software with known security vulnerabilities.

Notes on the Vulnerability Scanner Testing

These results are only a quick overview I have not followed up every discovered vulnerability to determine false positives and false negatives.

Nessus 5
External Network Profile

Critical 3
High 6
Medium 22
Low 8
Info 137

Nexpose
Full Audit Scan Profile

Critical 49
Severe 103
Moderate 18

Analysing a specific sample of Security Issues

In order to look at some more meaningful results I have examined a sample set of exploitable and mis-configured services on the Metasploitable system.

At the last minute I decided to include Nmap with its NSE scripts against the Metasploitable host. The results were interesting to say the least, while not a full blown vulnerability scanner the development of the NSE scripting ability in Nmap makes this powerful tool even more capable.

the numbers get more interesting…

These are the numbers of vulnerabilities correctly discovered and rated by each vulnerability scanner; from the sample set of exploitable services.

Security Issue	Nessus	OpenVAS	Nexpose	Nmap
FTP 21 Anonymous FTP Access
FTP 21 VsFTPd Smiley Face Backdoor
FTP 2121 ProFTPD Vulnerabilities
SSH 22 Weak Host Keys
PHP-CGI Query String Parameter Injection
CIFS Null Sessions
INGRESLOCK 1524 known backdoor drops to root shell
NFS 2049 /* exported and writable
MYSQL 3306 weak auth (root with no password)
RMI REGISTRY 1099 Insecure Default Config
DISTCCd 3632 distributed compiler
POSTGRESQL 5432 weak auth (postgresql)
VNC 5900 weak auth (password)
IRC 6667 Unreal IRCd Backdoor
Tomcat 8180 weak auth (tomcat/tomcat)

Notes about the sample set of tests

Conclusion

Vulnerability scanning is an important security control that should be implemented by any organisation wishing to secure their IT infrastructure. It is recommended by the SANS Institute as a Critical Control and by the US based NIST as a Security Management Control.

The results show significant variation in discovered security vulnerabilities by the different tools. It may be helpful to compare vulnerability scanners to anti-virus solutions; they are both an important security control that can enhance an organisations security posture. However as with anti-virus, a vulnerability scanner will not find all the bad things.

This will be common knowledge for most in the security industry who have performed network vulnerability testing. When performing vulnerability scanning, it is necessary to check the results for accuracy (false positives) and to actively look for things that were missed (false negatives).

My recommended approach to vulnerability scanning is to:

Feedback and corrections are most welcome, drop me a mail – peter (at) hackertarget.com or use the comments below.

The post Nessus, OpenVAS and Nexpose VS Metasploitable appeared first on Online Vulnerability Scanners and Port Scans.

Background on the Bing Azure API

A couple of months ago Microsoft released an update to the API they use for developer access to Bing Search. To summarise, anyone with a Microsoft account (live, outlook.com etc), can access the Bing search engine via an API. The only restriction for Free users is that it is limited to 5000 queries per month. If you want more than that you have to pay, wait a month or find another method…

Getting Started in Ubuntu

I attempted to find the most simple way to perform this task, the difference with the old API is that previously you put the KEY in the URL as a parameter and you could use whatever client you wanted to such as wget, curl or Firefox. Now you need to perform authentication with your KEY in a different manner.

First step is to get yourself a valid Key. Head to the Microsoft Azure Marketplace Website and get one now.

As I mentioned I used Python and the Requests HTTP library.

Under Ubuntu you will need to install the Requests library, to do that I did:

apt-get install pipi
pip install requests

Ok, now for the Python Code, this mini script takes an IP address at the command line and does a Bing IP Address search on it (ip:123.123.123.123). Change the search parameters to whatever you like. Take note that it must be HTML encoded for the API to parse it correctly (ie %27 = ‘):

#!/usr/bin/python
import requests
import json
from sys import argv
ip = argv[1]
r = requests.get('https://api.datamarket.azure.com/Data.ashx/Bing/Search/v1/Web?Query=%27ip%3A' + ip + '%27&$format=json', auth=("LhzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzzoA=", "LhzzzzzzzzzzzzzzzzzzzzzzzzzzXC/HBoA=")).json
for i in r['d']['results']:
   print str(i['DisplayUrl'].encode('ascii', 'ignore')) + '\\' + str(i['Title'].encode('ascii', 'ignore'))

This prints the Displayed URL’s and the Title of each page found, we have then put this in our Domain Profiler and Server Info information gathering tools.

In case you missed it the keys go in the auth parameter on the requests.get line above.

The post Bing Azure API with a simple Python script under Ubuntu appeared first on Online Vulnerability Scanners and Port Scans.