Hostintel – A Modular Python Application To Collect Intelligence For Malicious Hosts

0
6
Hostintel - A Modular Python Application To Collect Intelligence For Malicious Hosts

This instrument is used to gather numerous intelligence sources for hosts. Hostintel is written in a modular trend so new intelligence sources could be simply added.
Hosts are recognized by FQDN host identify, Domain, or IP handle. This instrument solely helps IPv4 for the time being. The output is in CSV format and despatched to STDOUT so the information could be saved or piped into one other program. Since the output is in CSV format, spreadsheets corresponding to Excel or database programs will simply be capable of import the information.
I created a brief introduction for this instrument on YouTube: https://youtu.be/aYK0gILDA6w
This works with Python v2, nevertheless it must also work with Python v3. If you discover it doesn’t work with Python v3 please put up a difficulty.

Help Screen:

$ python hostintel.py -h
utilization: hostintel.py [-h] [-a] [-d] [-v] [-p] [-s] [-c] [-t] [-o] [-i] [-r]
                    ConfigurationFile InputFile

Modular utility to search for host intelligence data. Outputs CSV to
STDOUT. This utility won't output data till it has completed all
of the enter.

positional arguments:
  ConfigurationFile     Configuration file
  InputFile             Input file, one host per line (IP, area, or FQDN
                        host identify)

optionally available arguments:
  -h, --help            present this assist message and exit
  -a, --all             Perform All Lookups.
  -d, --dns             DNS Lookup.
  -v, --virustotal      VirusTotal Lookup.
  -p, --passivetotal    PassiveTotal Lookup.
  -s, --shodan          Shodan Lookup.
  -c, --censys          Censys Lookup.
  -t, --threatcrowd     ThreatCrowd Lookup.
  -o, --otx             OTX by AlienVault Lookup.
  -i, --isc             Internet Storm Center DShield Lookup.
  -r, --carriagereturn  Use carriage returns with new traces on csv.

Install:
First, be certain your configuration file is right in your laptop/set up. Add your API keys and
usernames as applicable within the configuration file. Python and Pip are required to run this instrument. There are modules that have to be put in from GitHub, so ensure the git command is obtainable out of your command line. Git is straightforward to put in for any platform. Next, set up the python necessities (run this every time you git pull this repository too):

$ pip set up -r necessities.txt

There have been some issues with the inventory model of Python on Mac OSX (http://stackoverflow.com/questions/31649390/python-requests-ssl-handshake-failure). You could have to put in the safety portion of the requests library with the next command:

$ pip set up requests[security]

Lastly, I’m a fan of virtualenv for Python. To make a custom-made native set up of Python to run this instrument, I like to recommend you learn: http://docs.python-guide.org/en/latest/dev/virtualenvs/

Running:

$ python hostintel.py myconfigfile.conf myhosts.txt -a > myoutput.csv

You ought to be capable of import myoutput.csv into any database or spreadsheet program.
Note that relying in your community, your API key limits, and the information you’re trying to find, this script can run for a really very long time! Use every module sparingly! In return for the lengthy wait, you save your self from having to drag this knowledge manually.

Sample Data:
There is a few pattern knowledge within the “sampledata” listing. The IPs, domains, and hosts have been picked at random and on no account is supposed to focus on any group or particular person. Running this instrument on the pattern knowledge works within the following means:

Small Hosts List:

$ python hostintel.py native/config.conf sampledata/smalllist.txt -a > sampledata/smalllist.csv
*** Processing 8.8.8.8 ***
*** Processing 8.8.4.4 ***
*** Processing 192.168.1.1 ***
*** Processing 10.0.0.1 ***
*** Processing google.com ***
*** Processing 212.227.247.242 ***
*** Writing Output ***

Larger Hosts List:

$ python hostintel.py native/config.conf sampledata/largerlist.txt -a > sampledata/largerlist.csv
*** Processing 114.34.84.13 ***
*** Processing 116.102.34.212 ***
*** Processing 118.75.180.168 ***
*** Processing 123.195.184.13 ***
*** Processing 14.110.216.236 ***
*** Processing 14.173.147.69 ***
*** Processing 14.181.192.151 ***
*** Processing 146.120.11.66 ***
*** Processing 163.172.149.131 ***

...

*** Processing 54.239.26.180 ***
*** Processing 62.141.39.155 ***
*** Processing 71.6.135.131 ***
*** Processing 72.30.2.74 ***
*** Processing 74.125.34.101 ***
*** Processing 83.31.179.71 ***
*** Processing 85.25.217.155 ***
*** Processing 93.174.93.94 ***
*** Writing Output ***

Intelligence Sources:
You can get API keys on the websites under in your configuration file.

  • GeoLite2 (No community I/O required)
  • DNS (Network I/O required)
  • VirusTotal (Public API key and community I/O required, throttled when applicable)
  • PassiveTotal (API key, username, and community I/O required)
  • Shodan (API key and community I/O required)
  • Censys (API key, username, and community I/O required)
  • ThreatCrowd (Network I/O required, throttled when applicable)
  • OTX by AlienVault (API key and community I/O required)
  • Internet Storm Center (Network I/O required)

Resources:

  • The GeoIP2 Python library
  • The Python DNS library
  • The VirusTotal Python library
  • The Shodan Python library
  • The Censys Python library
  • The PassiveTotal Python library
  • The ThreatCrowd Python library
  • The OTX Python Library
  • The Internet Storm Center DShield Python Library

Notes:
Crude notes can be found here.

MoreTip.com

LEAVE A REPLY

Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.