Using Perl for NSLOOKUP from logfiles (Part 1)

Wow! It's been so long since I have put anything on here. Over the last six months I have been a TON of things, aside from the normal long hours at work.

A quick recap of the the past six months:
- Graduated from University of Michigan - Dearborn with my Masters of Science in Software Engineering
     - Of course, a MONTH after I walked the stage the school tells me that they screwed up and that I do in fact have to submit a project...grrrr!!!!!!
- Started four different Blog posts that I just haven't perfected enough to post :-(
- Took SANS610 in Orlando and am sitting the GREM exam next week.
- Dad had a heart attack, but he's getting stronger ever day through a life rehab program he is in
- Working on one web-based project with a friend, designing and installing a new small business network for another friend
- Trying to work on two different Android projects, although I haven't found any time in months to even look at what I have so far.
- AND, I want to start trying to help out on the SecurityOnion project (Doug has done such an AWESOME job with this tool and is looking for contributors - link at bottom of post)

In any event, I wanted to talk about a little tool I have been working on. It's a perl script that does two things and produces two output files.

What it does:
- Takes a file as an argument and parses ALL IP address from the file, writing them to "IPAddresses.txt"
- Parses "IPAddresses.txt" and performs an NSLOOKUP (Perl's version) of every address in this file
    - The exception is that I have inserted a quick check for 192.168 address space and ignore those addresses. This could be modified to ignore ALL Private IP space, but I really didn't want to do so yet.
    - The NSLOOKUP results are written to the file "NewIPLookupResults.txt."

A word of caution for anyone that wants to copy any of this script:
Absolutely NO ERROR CHECKING or validation is done. I am saving this for step 3 of this project.

So, time to break down the script:

The first part of the script takes the first argument, hopefully a log file, and parses out ALL IP addresses, pushing each one into an array. After this is done, a hash is setup that takes the unique values from the array. This hash is then sorted back into a final array.
This final array is then written to the "IPAddresses.txt" file, one address per line.

>> code begin <<
open MFILE, ">", "IPAddresses.txt" or die $!;
### This allows for the first passed parameter to be used
### in the loop
# Input: the first paramter passed
my @holder = ();
while(<>) {
  push(@holder, "$1\n") if /\D(\d+\.\d+\.\d+\.\d+)\D/;
}
#using a hash to weed out the dups
%listTemp = map { $_ => 1 } @holder;
@list_out = sort keys %listTemp;
for my $ipa (@list_out) {
 print MFILE $ipa;
}
close(MFILE);

>> end code

That is the "easy" part of this script.  The next section deals with getting the NSLOOKUP results using the perl library NET::NSLOOKUP. If you don't have this module installed, you will need to do so before running the remaining part of this script. Additionally, another GREAT library is probably going to need to be installed, NET::DNS, as NSLOOKUP depends upon this.

I will explain this part of the script below the pasting of it.

>> begin code <<

print "Processing NSLOOKUP of IP addresses\n";
open MFILE4, "<", "IPAddresses.txt" or die $!;
@lines = ;
print "Output file is NewIPLookupResults.txt\n";
open MFILE3, ">", "NewIPLookupResults.txt" or die $!;

for my $line (@lines) {
     #next here if private IP space?
     if($line =~ m/[1][9][2]/ )
     {
          next;
     }
     print "Performing NSLOOKUP for all IP adresses found in IPAddresses.txt\n\n";
     print MFILE3 "///////////////////////////////////////////////////\n";
     print MFILE3 "\t\tNSLOOKUP For: " . $line;
     print "Working on " . $line . "\n";
     print "Starting with A Records\n";
     print MFILE3 "**************** A Records: ****************\n";
     my @results = nslookup(domain => $line, type => "A");
     if (@results == 0)
     {
          print MFILE3 "\tNo A Records Found\n";
     }
     for my $res (@results) {
          print MFILE3  $res . "\n";
     }
     print "Now for PTR records\n";
     print MFILE3 "**************** PTR Records: ****************\n";
     my @results = nslookup(domain => $line, type => "PTR");
     if (@results == 0)
     {
          print MFILE3 "\tNo PTR Records Found\n";
     }
     for my $res (@results) {
          print MFILE3  $res . "\n";
     }
     print "Now for CNAME records\n";
     print MFILE3 "**************** CNAME Records: ****************\n";
     my @results = nslookup(domain => $line, type => "CNAME");
     if (@results == 0)
     {
          print MFILE3 "\tNo CNAME Records Found\n";
     }
     for my $res (@results) {
          print MFILE3  $res . "\n";
     }
     print "Now for MX records\n";
     print MFILE3 "**************** MX Records: ****************\n";
     my @results = nslookup(domain => $line, type => "MX");
     if (@results == 0)
     {
          print MFILE3 "\tNo MX Records Found\n";
     }
     for my $res (@results) {
          print MFILE3  $res . "\n";
     }
     print "Now for NS records\n";
     print MFILE3 "**************** NS Records: ****************\n";
     my @results = nslookup(domain => $line, type => "NS");
     if (@results == 0)
     {
          print MFILE3 "\tNo NS Records Found\n";
     }
     for my $res (@results) {
          print MFILE3  $res . "\n";
     }
     print "Now for SOA records\n";
     print MFILE3 "**************** SOA Records: ****************\n";
     my @results = nslookup(domain => $line, type => "SOA");
     if (@results == 0)
     {
          print MFILE3 "\tNo SOA Records Found\n";
     }
     for my $res (@results) {
          print MFILE3  $res . "\n";
     }
     print "Now for TXT records\n";
     print MFILE3 "**************** TXT Records: ****************\n";
     my @results = nslookup(domain => $line, type => "TXT");
     if (@results == 0)
     {
          print MFILE3 "\tNo TXT Records Found\n";
     }
     for my $res (@results) {
          print MFILE3  $res . "\n";
     }
     print MFILE3 "\n";
}
close(MFILE3);
close(MFILE4);

>> end code <<

And, minus the she-bang and use statements, that's the full script. This second part here was a lot more fun to work on than the first. I had never used perl's NSLOOKUP before today so I had a little fun with it which is the primary reason why I do a lookup for every possible record type that this library supports.

You might notice a LOT of print statements, both to the file MFILE3 as well as to STDOUT. I will change this as I tweak this tool and add the features I still want. However, right now I wanted as much verbosity as possible. That said, this script will print to STDOUT an output similar to:

>> Example Output <<

Working on 83.149.8.208
Starting with A Records
Now for PTR records
Now for CNAME records
Now for MX records
Now for NS records
Now for SOA records
Now for TXT records

>> end example output <<

The corresponding output that is written to the file is also somewhat verbose, or at least "pretty" if you like different symbols. :-)

>> Example File Output <<

///////////////////////////////////////////////////
  NSLOOKUP For: 83.149.8.208
**************** A Records: ****************
 No A Records Found
**************** PTR Records: ****************
gprs-client-83.149.8.208.misp.ru
**************** CNAME Records: ****************
 No CNAME Records Found
**************** MX Records: ****************
 No MX Records Found
**************** NS Records: ****************
 No NS Records Found
**************** SOA Records: ****************
 No SOA Records Found
**************** TXT Records: ****************
 No TXT Records Found

>> end example output <<


Only a portion of this section needs to be explained as the different checks are identical in nature.


First, I grab the file I just created:
open MFILE4, "<", "IPAddresses.txt" or die $!;

I copy this file to array:
@lines = ;
Created a new file for the NSLOOKUP results:
open MFILE3, ">", "NewIPLookupResults.txt" or die $!;

The script now loops through each address in the array. If the address belongs to the 192.168 space, then that line is ignored and the loop goes to the next line.
for my $line (@lines) {
     #next here if private IP space?
     if($line =~ m/[1][9][2]\.[1][6][8]/ )
     {
         next;
     }

Just some printed information for the file and STDOUT
print "Performing NSLOOKUP for all IP adresses found in IPAddresses.txt\n\n";
print MFILE3 "///////////////////////////////////////////////////\n";
print MFILE3 "\t\tNSLOOKUP For: " . $line;
print "Working on " . $line . "\n";
print "Starting with A Records\n";
print MFILE3 "**************** A Records: ****************\n";

Now, the part I really like, even as simple as it is (Which is WHY I like it because it yet again demonstrates the power that a simple script can have!!!)
Here, I create an array called results that holds the return value of the nslookup function. This function takes two arguments, the first being a 'target' of some kind. This can be domain, host, etc. The second argument is the type of record we want to look for: A, MX, SOA, etc.
my @results = nslookup(domain => $line, type => "A");
#If NO results, then print that and move on
if (@results == 0)
{
     print MFILE3 "\tNo A Records Found\n";
}
for my $res (@results) {
     print MFILE3 $res . "\n";
}

and that's really all this script does. However, it does ALSO provide a good framework for the beginnings of a multi-purpose tool for pentesters and intrusion detection analysts.

Path Forward:
I plan to make some changes over the next month, as time permits. I would like to make this a little more OO-like in that the NSLOOKUP section is one function, and other tools/sections that I want to add are also functions. These I plan to make 'active' during the scripts running via command line parameters.

Being that I work in network security, and have for a long time, I would be remiss if I didn't eventually add some code to handle errors and input, especially in terms of the files being passed as input as well as the IP address (valid, non-routed, etc). However, I can already 'see' the need for some custom switches, such as WHICH DNS records to actually get or how verbose the file or output needs to be for the user.

The final to things that I want to be able to do with this, after adding any other functionality that I think this needs without bloating it too much are:
- different output formats (how great would it be if I made this importable into the MetaSploit database!!!)
- The ability to parse files such as nmaps greppable or XML output and do more than just DNS lookups.


Unrelated and shamless promotion of an AWESOME TOOL from Doug Burks, SecurityOnion:
Google group:
http://code.google.com/p/security-onion/
http://securityonion.blogspot.com/