Red Team’s SIEM – simple deployable software for Red Teams used for monitoring and alarming about Blue Team actions in addition to higher usability for the Red Team in long run operations.
Initial public launch at BruCON 2018:
Goal of the undertaking
Short: a Red Team’s SIEM.
Longer: a Red Team’s SIEM that serves three targets:
- Enhanced usability and overview for the pink crew operators by making a central location the place all related operational logs from a number of teamservers are collected and enriched. This is nice for historic looking out throughout the operation in addition to giving a learn-solely view on the operation (e.g. for the White Team). Especially helpful for multi-situation, multi-teamserver, multi-member and multi-month operations. Also, tremendous simple methods for viewing all screenshots, IOCs, keystrokes output, and many others. o/
- Spot the Blue Team by having a central location the place all visitors logs from redirectors are collected and enriched. Using particular queries its now attainable to detect that the Blue Team is investigating your infrastructure.
- Out-of-the-field usable by being simple to put in and deploy, in addition to having prepared made views, dashboards and alarms.
Here’s a conceptual overview of how RedELK works.
RedELK makes use of the everyday parts Filebeat (delivery), Logstash (filtering), Elasticsearch (storage) and Kibana (viewing). Rsync is used for a second syncing of teamserver knowledge: logs, keystrokes, screenshots, and many others. Nginx is used for authentication to Kibana, in addition to serving the screenshots, beaconlogs, keystrokes in a simple approach within the operator’s browser.
A set of python scripts are used for heavy enriching of the log knowledge, and for for Blue Team detection.
Supported tech and necessities
RedELK at present helps:
- Cobalt Strike teamservers
- HAProxy for HTTP redirector knowledge. Apache help is predicted quickly.
- Tested on Ubuntu 16 LTS
RedELK requires a modification to the default haproxy configuration with the intention to log extra particulars.
In the ‘basic’ part:
log-format frontend:%f/%H/%fi:%fp backend:%b consumer:%ci:%cp GMT:%T useragent:%[capture.req.hdr(1)] physique:%[capture.req.hdr(0)] request:%r
At ‘frontend’ part:
declare seize request len 40000 http-request seize req.physique id 0 seize request header User-Agent len 512
First time set up
./certs/config.cnf to incorporate the suitable particulars for the TLS certificates. Once performed, run:
preliminary-setup.sh This will create a CA, generate needed certificates for safe communication between redirs, teamserver and elkserver and generates a SSH keypair for safe rsync authentication of the elkserver to the teamserver. It additionally generates
elkserver.tgz that include the set up packages for every element. Rerunning this preliminary setup is just not required. But if you would like new certificates for a brand new operation, you possibly can merely run this once more.
Installation of redirectors
Copy and extract redirs.tgz in your redirector as a part of your pink crew infra deployment procedures. Run:
set up-redir.sh $FilebeatID $ScenarioName $IP/DNS:PORT
- $FilebeatID is the identifier of this redirector inside filebeat.
- $ScenarioName is the title of the assault situation this redirector is used for.
- $IP/DNS:PORT is the IP or DNS title and port the place filebeat logs are shipped to.
This script will set the timezone (default Europe/Amsterdam), set up filebeat and dependencies, set up required certificates, modify the filebeat configuration and begin filebeat.
Installation of teamserver
Copy and extract teamservers.tgz in your Cobalt Strike teamserver as a part of your pink crew infra deployment procedures. Run:
set up-teamserver.sh $FilebeatID $ScenarioName $IP/DNS:PORT
- $FilebeatID is the identifier of this teamserver inside filebeat.
- $ScenarioName is the title of the assault situation this teamserver is used for.
- $IP/DNS:PORT is the IP or DNS title and port the place filebeat logs are shipped to.
This script will warn if filebeat is already put in (necessary as ELK and filebeat typically are very choosy about having equal variations), set the timezone (default Europe/Amsterdam), set up filebeat and dependencies, set up required certificates, modify the filebeat configuration, begin filebeat, create a neighborhood person ‘scponly’ and restrict that person to SSH key-based mostly auth by way of scp/sftp/rsync.
Installation of ELK server
Copy and extract elkserver.tgz in your RedELK server as a part of your pink crew infra deployment procedures. Run:
set up-teamserver.sh This script will set the timezone (default Europe/Amsterdam), set up logstash, elasticsearch, kibana and dependencies, set up required certificates, deploy the logstash configuration and required customized ruby enrichment scripts, obtain GeoIP databases, set up Nginx, configure Nginx, create a neighborhood person ‘redelk’ with the sooner generated SSH keys, set up the script for rsyncing of distant logs on teamservers, set up the script used for creating of thumbnails of screenshots, set up the RedELK configuration information, set up crontab file for RedELK duties, set up GeoIP elasticsearch plugins and modify the template, set up the python enrichment scripts, and eventually set up the python blue crew detection scripts.
You are usually not performed but. You have to manually enter the small print of your teamservers in
/and many others/cron.d/redelk, in addition to tune the config information in
/and many others/redelk (see part under).
Setting up enrichment and detection
On the ELK server within the
/and many others/redelk listing you’ll find a number of information that you need to use to tune your RedELK occasion for higher enrichments and higher alarms. These information are:
/and many others/redelk/iplist_customer.conf: public IP addresses of your goal, one per line. Including an tackle right here will set a tag for relevant information within the redirhaproxy-* index.
/and many others/redelk/iplist_redteam.conf: public IP addresses of your pink crew, one per line. Convenient for figuring out testing performed by pink crew members. Including an tackle right here will set a tag for relevant information within the redirhaproxy-* index.
/and many others/redelk/iplist_unknown.conf: public IP addresses of gateways that you’re not positive about but, however do not wish to be warned about once more. One per line. Including an tackle right here will set a tag for relevant information within the redirhaproxy-* index.
/and many others/redelk/known_sandboxes.conf: beacon traits of identified AV sandbox programs. One per line. Including knowledge right here right here will set a tag for relevant information within the rtops-* index.
/and many others/redelk/known_testsystems.conf: beacon traits of identified check programs. One per line. Including knowledge right here right here will set a tag for relevant information within the rtops-* index.
/and many others/redelk/alarm.json.config: particulars required for alarms to work. This contains API keys for on-line companies (Virus Total, IBM X-Force, and many others) in addition to the SMTP particulars required for sending alarms by way of e-mail.
If you alter these information previous to your preliminary setup, these adjustments will likely be included within the .tgz packages and can be utilized for future installations. These information may be present in
./RedELK/elkserver/and many others/redelk.
To change the authentication onto Nginx, change
/and many others/nginx/htpasswd.customers to incorporate your most well-liked credentials. Or
./RedELK/elkserver/and many others/nginx/htpasswd.customers previous to preliminary setup.
Under the hood
If you need to have a look beneath the hood on the ELK server, check out the redelk cron file in
/and many others/cron.d/redelk. It begins a number of scripts in
/usr/share/redelk/bin/. Some scripts are for enrichment, others are for alarming. The configuration of those scripts is finished with the config information in
/and many others/redelk/. There can also be heavy enrichment performed (together with the technology of hyperlinks for screenshots, and many others) in logstash. You can examine that out immediately kind the logstash config information in
/and many others/logstash/conf.d/.
Current state and options on todo-record
This undertaking continues to be in alpha section. This signifies that it really works on our machines and the environment, however no prolonged testing is carried out on totally different setups. This additionally signifies that naming and construction of the code continues to be topic to alter.
We are working (and you might be invited to contribute) on the next options for subsequent variations:
- Include the true exterior IP tackle of a beacon. As Cobalt Strike has no data of the true exterior IP tackle of a beacon session, we have to get this way the visitors index. So far, we have now not discovered a real 100% dependable approach for doing this.
- Support for Apache redirectors. Fully examined and dealing filebeat and logstash configuration information that help Apache based mostly redirectors. Possibly further customized log configuration wanted for Apache. Low precedence.
- Solve rsyslog max log line difficulty. Rsyslog (default syslog service on Ubuntu) breaks lengthy syslog traces. Depending on the CS profile you utilize, this could turn into a problem. As a consequence, the parsing of a few of the fields are correctly parsed by logstash, and thus not correctly included in elasticsearch.
- Ingest handbook IOC knowledge. When you might be importing a doc, or one thing else, exterior of Cobalt Strike, it is not going to be included within the IOC record. We need a simple approach to have these handbook IOCs additionally included. One approach can be to enter the info manually within the exercise log of Cobalt Strike and have a logstash filter to scrape the data from there.
- Ingest e-mails. Create enter and filter guidelines for IMAP mailboxes. This approach, we are able to use the identical simple ELK interface for having an outline of despatched emails, and replies.
- User-agent checks. Tagging and alarming on suspicious person-brokers. This will most likely be divided in hardcoded stuff like curl, wget, and many others connecting with the right C2 URL’s, but additionally extra dynamic analysis of suspicious person-brokers.
- DNS visitors analyses. Ingest, filter and question for suspicious actions on the DNS degree. This will take appreciable work as a result of great amount of noise/bogus DNS queries carried out by scanners and on-line DNS stock companies.
- Other alarm channels. Think Slack, Telegram, no matter different approach you need for receiving alarms.
- Fine grained authorisation. Possibility for blocking sure views, searches, and dashboards, or masking sure particulars in some views. Useful for conditions the place you do not wish to give out all data to all guests.
First time login
Browse to your RedELK server’s IP tackle and login with the credentials from Nginx (default is redelk:redelk). You at the moment are in a Kibana interface. You could also be requested to create a default index for kibana. You can choose any of the out there indices, it does not matter which one you decide.
There are most likely two stuff you wish to do right here: have a look at dashboards, or look and search the info in additional element. You can change between these views utilizing the buttons on the left bar (default Kibana performance).
Click on the dashboard icon on the left, and you will be given 2 selections: Traffic and Beacon.
Looking and looking out knowledge intimately
Click on the Discover button to have a look at and search the info in additional element. Once there, click on the time vary you wish to use and click on on the ‘Open’ button to make use of one of many ready searches with views.
When deciding on the search ‘TimelineOverview’ you might be offered with a simple to make use of view on the info from the Cobalt Strike teamservers, a time line of beacon occasions for those who like. The view contains the related columns you wish to have, corresponding to timestamp, testscenario title, username, beacon ID, hostname, OS and OS model. Finally, the complete message from Cobalt Strike is proven.
You can modify this search to your liking. Also, as a result of its elasticsearch, you possibly can search all the info on this index utilizing the search bar.
Clicking on the small print of a report will present you the complete particulars. An necessary area for usability is the beaconlogfile area. This area is an hyperlink, linking to the complete beacon log file this report is from. Its means that you can have a look at the beacon transcript in an even bigger home windows and use CTRL+F inside it.
RedELK comes with a simple approach of all of the screenshots that had been made out of your targets. Select the ‘Screenshots’ search to get this overview. We added two huge usability issues: thumbnails and hyperlinks to the complete photos. The thumbnails are there to rapidly scroll by way of and provide you with an instantaneous impression: usually you continue to keep in mind what the screenshot regarded like.
Just as with screenshots, its very helpful to have a simple overview of all keystrokes. This search offers you the primary traces of cententi, in addition to once more an hyperlink to the complete keystrokes log file.
To get a fast record of all IOCs, RedELK comes with a simple overview. Just use the ‘IOCs’ search to get this record. This will current all IOC knowledge from Cobalt Strike, each from information and from companies.
You can rapidly export this record by hitting the ‘Reporting’ button within the prime bar to generate a CSV of this precise view.
Logging of RedELK
During set up all actions are logged in a log file within the present working listing.
During operations, all RedELK particular logs are logged on the ELK server in
/var/log/redelk. You most likely solely want this for troubleshooting.
Authors and contribution
This undertaking is developed and maintained by:
- Marc Smeets (@smeetsie on Github and @mramsmeets on Twitter).
- Mark Bergman (@xychix on Github and Twitter)