Turbinia – Automation And Scaling Of Digital Forensics Tools

Turbinia - Automation And Scaling Of Digital Forensics Tools

Turbinia is an open-supply framework for deploying, managing, and operating distributed forensic workloads. It is meant to automate operating of widespread forensic processing instruments (i.e. Plaso, TSK, strings, and many others) to assist with processing proof within the Cloud, scaling the processing of enormous quantities of proof, and lowering response time by parallelizing processing the place attainable.

How it really works

Turbinia consists of various parts for the consumer, server and the employees. These parts may be run within the Cloud, on native machines, or as a hybrid of each. The Turbinia consumer makes requests to course of proof to the Turbinia server. The Turbinia server creates logical jobs from these incoming consumer requests, which creates and schedules forensic processing duties to be run by the employees. The proof to be processed might be break up up by the roles when attainable, and plenty of duties may be created to be able to course of the proof in parallel. One or extra employees run repeatedly to course of duties from the server. Any new proof created or found by the duties might be fed again into Turbinia for additional processing.

Communication from the consumer to the server is at present finished with both Google Cloud PubSub or Kombu messaging. The employee implementation can use both PSQ (a Google Cloud PubSub Task Queue) or Celery for process scheduling.
More data on Turbinia and the way it works may be found here.

Turbinia is at present in Alpha launch.

There is an rough installation guide here.

The fundamental steps to get issues operating after the preliminary set up and configuration are:

  • Start Turbinia server part with turbiniactl server command
  • Start a number of Turbinia employees with turbiniactl psqworker
  • Send proof to be processed from the turbinia consumer with turbiniactl ${evidencetype}
  • Check standing of operating duties with turbiniactl standing

turbiniactl can be utilized to start out the totally different parts, and right here is the fundamental utilization:

$ turbiniactl --help
utilization: turbiniactl [-h] [-q] [-v] [-d] [-a] [-f] [-o OUTPUT_DIR] [-L LOG_FILE]
                   [-r REQUEST_ID] [-R] [-S] [-C] [-V] [-D]
                   [-F FILTER_PATTERNS_FILE] [-j JOBS_WHITELIST]
                   [-J JOBS_BLACKLIST] [-p POLL_INTERVAL] [-t TASK] [-w]
                   <command> ...

non-compulsory arguments:
  -h, --help            present this assist message and exit
  -q, --quiet           Show minimal output
  -v, --verbose         Show verbose output
  -d, --debug           Show debug output
  -a, --all_fields      Show all process standing fields in output
  -f, --force_evidence  Force proof processing request in probably
                        unsafe circumstances
  -o OUTPUT_DIR, --output_dir OUTPUT_DIR
                        Directory path for output
  -L LOG_FILE, --log_file LOG_FILE
                        Log file
  -r REQUEST_ID, --request_id REQUEST_ID
                        Create new requests with this Request ID
  -R, --run_local       Run utterly domestically with none server or different
                        infrastructure. This can be utilized to run one-off Tasks
                        to course of information domestically.
  -S, --server          Run Turbinia Server indefinitely
  -C, --use_celery      Pass this flag when utilizing Celery/Kombu for process
                        queuing and messaging (as a substitute of Google PSQ/pubsub)
  -V, --version         Show the model
  -D, --dump_json       Dump JSON output of Turbinia Request as a substitute of
                        sending it
                        A file containing newline separated string patterns to
                        filter textual content based mostly proof information with (in prolonged
                        grep regex format). This filtered output might be in
                        addition to the whole output
  -j JOBS_WHITELIST, --jobs_whitelist JOBS_WHITELIST
                        A whitelist for Jobs that we are going to permit to run (notice
                        that it'll not power them to run).
                        A blacklist for Jobs we won't permit to run
  -p POLL_INTERVAL, --poll_interval POLL_INTERVAL
                        Number of seconds to attend between polling for process
                        state information
  -t TASK, --task TASK  The identify of a single Task to run domestically (should be used
                        with --run_local.
  -w, --wait            Wait to exit till all duties for the given request
                        have accomplished

    rawdisk             Process UncookedDisk as Evidence
    googleclouddisk     Process Google Cloud Persistent Disk as Evidence
                        Process Google Cloud Persistent Disk with an embedded
                        uncooked disk picture as Evidence
    listing           Process a listing as Evidence
    listjobs            List all out there jobs
    psqworker           Run PSQ employee
    celeryworker        Run Celery employee
    standing              Get Turbinia Task standing
    server              Run Turbinia Server

The instructions for processing the proof varieties of rawdisk and listing specify details about proof that Turbinia ought to course of. By default, when including new proof to be processed, turbiniactl will act as a consumer and ship a request to the configured Turbinia server, in any other case if --server is specified, it should begin up its personal Turbinia server course of. Here’s the turbiniactl utilization for including a uncooked disk kind of proof to be processed by Turbinia:

$ ./turbiniactl rawdisk -h
utilization: turbiniactl rawdisk [-h] -l LOCAL_PATH [-s SOURCE] [-n NAME]

non-compulsory arguments:
  -h, --help            present this assist message and exit
  -l LOCAL_PATH, --local_path LOCAL_PATH
                        Local path to the proof
  -s SOURCE, --source SOURCE
                        Description of the supply of the proof
  -n NAME, --name NAME  Descriptive identify of the proof

Other documentation


  • Turbinia at present assumes that Evidence is equally out there to all employee nodes (e.g. by domestically mapped storage, or by attachable persistent Google Cloud Disks, and many others).
  • Not all proof sorts are supported but
  • Still solely a small variety of processing job sorts supported, however extra are being developed.

Obligatory Fine Print
This will not be an official Google product (experimental or in any other case), it’s simply code that occurs to be owned by Google.



Please enter your comment!
Please enter your name here

This site uses Akismet to reduce spam. Learn how your comment data is processed.