<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=2180921&amp;fmt=gif">

Threat Hunting with Suricata and Newly-Registered Domain Threat Intel (Open NRD) Part 2

This is a follow-up to our first blog on hunting using the publicly available Newly Registered Domains (NRD) threat intel lists. In this blog, we will explore a hands-on example of infection traffic and review one of three possible detection and investigation techniques that can unearth malicious behavior. 

It is important to note that NRDs are not necessarily bad by default, and an NRD alone is not enough of an IoC/match to determine whether or not it is malicious. We need to combine context with the detection itself to make that determination. But what information can help us come to our conclusion? This blog post should help give you a baseline of what to look for. 

This blog post is one of five blogs introducing Open NRD and sharing various ways it can be used with Suricata for threat hunting and investigation. To skip to the other blog posts in the series, click on one of the following links:

What tools are needed? 

Before beginning, there are some tools you need to ensure you can follow along:

Before we look into the actual hunting example, we must understand three major Suricata hunting concepts. They are as follows:

  1. 1. The wealth of data generated by Suricata (especially beyond alerts)
  2. 2. How correlation is performed with Flow ID
  3. 3. How flow labeling with Flowbits can impact context

 

Suricata-generated data - more than just alerts

One of the many powerful features of Suricata is that it can create protocol and transaction logs even in the absence of alerts. These logs include flow records, anomalies, alerts, protocol transactions, and file transaction logs, plus file extraction and packet capture (PCAP). 


Here is a full list and details of what those logs and transactions look like. 

You also might be interested in the many new things available in the recently released Suricata 7.   

Suricata data correlation with Flow ID 

Suricata produces all relevant network security monitoring logs: protocol, flow, file transaction, and anomaly logs, including the ones related to an alert - but also independent of alerts. In the regular JSON logs that Suricata generates (eve.json), you will find something called “flow_id” that correlates the network protocol data and evidence that Suricata has logged - to an alert event and that alert’s metadata. 

The ability to correlate any existing evidence/logs to an alert (“flow_id”) was introduced in 2014 by Suricata lead developer, Victor Julien

https://github.com/OISF/suricata/commit/f1185d051c210ca0daacdddbe865a51af24f4ea3 

Flow labeling with Flowbits can provide context

Suricata can match or highlight on specific simple events, occurrences, anomalies, patterns, or IoCs and also much more complex detection logic either via signature or a lua script – in flows/sessions or cross flows/sessions. 

Regardless of what type of match is used we can insert a piece of metadata in all protocol transactions generated by Suricata. This way, multiple different labeling of varying flows can occur that can later be used for detection logic SIEM/DB queries, alerting, and/or automation.

In other words, by using NRD matching all the Suricata protocol records to and from NRD communication that are found in either DNS Query (“event_type”:”dns”), HTTP Hostname (“event_type”:”http”), and TLS SNI (“event_type”:”tls”) will have the flowbit/marker piece of metadata inserted in them as well as any and all corresponding “event_type”:”fileinfo”, “event_type”:”flow”, “event_type”:”anomaly” event logs from those sessions/flows.

Here is an example of a flow in Suricata JSON output: 

The above picture has a flow or protocol record labeled (with “stamus.nrd.entropy”) as a Newly Registered Domain communication with high entropy from Stamus’s NRD daily updated lists. 

There could even be multiple matches and labeling:

{

  "flowbits": [

    "FBMproto_0",

    "stamus.nrd",

    "ET.http.binary"

  ]

}

….

The above record has a flow or protocol record labeled (with “stamus.nrd”) as a Newly Registered Domain communication from Stamus’s NRD daily updated lists. 

The visualization above is present in the SN-FLOW dashboard (based on Suricata “event_type”:”flow” log data) as part of free and Open Source

Dashboards that are available in GitHub or as part of SELKS

https://github.com/StamusNetworks/suricata-analytics/tree/main/kibana/7  

Detection method one - Executable downloaded from Newly Registered Domain 

Now that we have those 3 major points clarified, we can move on with our detection logic.

First, let's clarify what we mean by “detection” here. In this context, detection is simply marking and/or highlighting NRDs in the communication traffic. Then based on our hunting hypothesis we combine that knowledge with the protocol and alert information that Suricata produces. In some cases there could be no alerts, and that’s ok too. 

One point that is important to keep in mind is that after the hunt is successful we should check if  traces of such traffic or transactions happened in the past. For example, if we find any TLS certificates, file hashes, or domains, we could use Suricata protocol data to check if these occurred previously regardless of alerting.

We are basing this hunt on the hypothesis / idea that clear text executables from a newly registered domain is something worth investigating.

For this hands-on use case review, we run the PCAP through Suricata (via use of SELKS) and then we review the actual data we have available. If you need to install SELKS you can consult the online documentation here. However, it is not a requirement as you can also simply run the PCAP through any Suricata deployment as long as you have the OpenNRD rules installed

First we read in the PCAP in SELKS: 

 scripts/readpcap.sh -ac /home/pevma/Downloads/2023-10-12-DarkGate-infection-traffic.pcap

Alert-based hunting with Scirius

Once we have read the PCAP in SELKS, we can then head to the Scirius hunting interface (right upper corner widget switcher, click on Hunting) and look at the alerts and all other generated events.

We can see a few alerts as well as DNS and HTTP based NRD communication combined with a clear text executable download via HTTP:

If we select the “ET POLICY PE EXE or DLL Windows file download HTTP” alert, by way of Suricata’s defaultly available flow_id  correlation we can see (related events tab - fileinfo transaction, http log, and other alerts for example) that this is an executable downloaded from a NRD: 

The domain, according to Virus total, was registered only 9 days ago (at the time the original check was completed). 

Now let’s see if we can create a filter or a query based on the findings above.

We can create a simple Scirius Hunting query as follows (from the filter drop down, select ES query):

metadata.flowbits:*stamus.nrd* AND alert.signature:*exe* AND http.status:200   

Now we can easily zoom in on alerts that have executable downloads in clear text from a NRD.

Alert log example 

Here is an example of an executable download alert (“ET POLICY PE EXE or DLL Windows file download HTTP”) that has the flowbit / flow label enrichment of a newly registered domain as well - “stamus.nrd

NOTE: The response body, response body printable, and payload fields below as part of the alert have been clipped out on purpose as they are too big. 

{

  "timestamp": "2023-10-12T22:09:10.979074+0200",

  "flow_id": 1689769056868824,

  "pcap_cnt": 55,

  "event_type": "alert",

  "src_ip": "104.21.91.46",

  "src_port": 80,

  "dest_ip": "10.10.10.113",

  "dest_port": 64642,

  "proto": "TCP",

  "pkt_src": "wire/pcap",

  "metadata": {

    "flowbits": [

      "stamus.nrd",

      "ET.http.binary"

    ]

  },

  "ether": {

    "src_mac": "00:0c:30:5f:c4:a3",

    "dest_mac": "08:1f:71:62:50:7d"

  },

  "tx_id": 0,

  "alert": {

    "action": "allowed",

    "gid": 1,

    "signature_id": 2018959,

    "rev": 4,

    "signature": "ET POLICY PE EXE or DLL Windows file download HTTP",

    "category": "Potential Corporate Privacy Violation",

    "severity": 1,

    "metadata": {

      "created_at": [

        "2014_08_19"

      ],

      "former_category": [

        "POLICY"

      ],

      "updated_at": [

        "2017_02_01"

      ]

    }

  },

  "http": {

    "hostname": "hgfdytrywq.com",

    "url": "/a",

    "http_user_agent": "Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) WindowsPowerShell/5.1.19041.3031",

    "http_content_type": "application/octet-stream",

    "http_method": "GET",

    "protocol": "HTTP/1.1",

    "status": 200,

    "length": 41560,

    "http_response_body_printable": "MZ......................@...............................................!..L.!This program cannot be run in DOS mode.\r\r\n$........sD.R.*.R.*.R.*..C..P.*.....S.*._@..a.*._@....*._@..g.*.[j..[.*.[j..w.

*.R.+.r.*.......*.....S.*._@..S.*.R...P.*.....S.*.RichR.*.........................PE..L....q.Z..........\"...............................@...........................\r.......\r...@...@.......@.........................|.......P............

....

[clipped out]

.....

....",

    "http_response_body": "TVqQAAMAAAAEAAAA//8AALgAAAAAAAAAQAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAGAEAAA4fug4AtAnNIbgBTM0hVGhpcyBwcm9ncmFtIGNhbm5vdCBiZSBydW4gaW4gRE9TIG1vZGUuDQ0KJAAAAAAAAAAWc0SQUhIqw1ISKsNSEirDFEPLw1ASKsPMsu3DUxI

qw19A9cNhEirDX0DKw+MSKsNfQMvDZxIqw1tqqcNbEirDW2q5w3cSKsNSEivDchAqw+eMwMMCEirD54z1w1MSKsNfQPHDUxIqw1ISvcNQEirD54z0w1MSKsNSaWNoUhIqwwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAFBFAABMAQUAv3GqWgAAAAAAAAAA4AAiAQsBDAAA6AgAANgEAAAAAAD6fwIAABAAAAAACQAAAEAAA

BAAAAACAAAFAAEAAAAAAAUAAQAAAAAAAPANAAAEAACQ+A0AAgBAgAAAQAAAEAAAAABAAAAQAAAAAAAAEAAAAAAAAAAAAAAAzNALAHwBAAAAkAwAUNcAAAAAAAAAAAAAAIYNAKgcAAAAcA0ArHEAAJA7CQAcAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAIFsKAEAAAAAAAAAAAAAAAAAACQCECAAAAAAAAAAAAAAAAAA

....

[clipped out]

.....

...."

  },

  "files": [

    {

      "filename": "Autoit3.exe",

      "magic": "PE32 executable (GUI) Intel 80386, for MS Windows, 5 sections",

      "gaps": false,

      "state": "UNKNOWN",

      "stored": false,

      "storing": true,

      "size": 41560,

      "tx_id": 0

    }

  ],

  "app_proto": "http",

  "direction": "to_client",

  "flow": {

    "pkts_toserver": 19,

    "pkts_toclient": 34,

    "bytes_toserver": 1300,

    "bytes_toclient": 45430,

    "start": "2023-10-12T22:09:10.262358+0200",

    "src_ip": "10.10.10.113",

    "dest_ip": "104.21.91.46",

    "src_port": 64642,

    "dest_port": 80

  },

  "payload": "SFRUUC8xLjEgMjAwIE9LDQpEYXRlOiBUaHUsIDEyIE9jdCAyMDIzIDIwOjA5OjExIEdNVA0KQ29udGVudC1UeXBlOiBhcHBsaWNhdGlvbi9vY3RldC1zdHJlYW0NCkNvbnRlbnQtTGVuZ3RoOiA4OTM2MDgNCkNvbm5lY3Rpb246IGtlZXAtYWxpdmUNCkNvbnRlbnQtRGlzcG9zaXRpb246IGF0dGFj

aG1lbnQ7IGZpbGVuYW1lPSJBdXRvaXQzLmV4ZSINCkNGLUNhY2hlLVN0YXR1czogRFlOQU1JQw0KUmVwb3J0LVRvOiB7ImVuZHBvaW50cyI6W3sidXJsIjoiaHR0cHM6XC9cL2EubmVsLmNsb3VkZmxhcmUuY29tXC9yZXBvcnRcL3YzP3M9eDFua1FuUVJuWFRycmF6JTJCcE5COFVGY2FtVjVzNTZZMyUyQlJEd2QlMk

....

[clipped out]

.....

....",

  "stream": 1,

  "packet": "CB9xYlB9AAwwX8SjCABFAAWIjEVAADAG4WxoFVsuCgoKcQBQ/IJiHoBg3wnxhFAQAAggjgAA/+gCTwEAiwONj3j7//+LQAQDyOiFTwEAiwOLQAQDx4O4ePv//wB1zv8VVAhJAItP3IXJD4WOgQMAi0/QhckPhZeBAwAz24lf2ItPwIXJD4VVAQAAjU+giV/I6P9tAACNj3D+///oL1ABAI2PVP7//+g9n

QAAjY9k/f//6FdWAQCNj1T9///o020AAI2PRP3//+jSnv//jY80/f//6JBaAQCNjyT9///oS1wBAI2PFP3//+hAXAEAjY8E/f//6DVcAQCLj/z8//+FyQ+FH4EDAI2P6Pz//+h7ZP//i4/g/P//hckPhRqBAwCLj9T8//+FyQ+FIIEDAI2PwPz//+hcbQAAjY+o/P//6AOJAACLj5z8//+FyQ+FEIEDAImfpPz//4uPkPz

//4XJD4UQgQMAjY9M/P//iZ+Y/P//6B5tAACNjzz8///oE20AAI2PLPz//+gIbQAAjY/8+///6JhTAQBfXluL5V3Di4c4/f//iwSwiwCFwHQj/zD/NaxyTAD/FfQGSQCLhzj9//+LBLCLCIXJdAZR6N9dAQBGO7c8/f//D4P0/f//67+LcQRR6LJaAQCLzoX2D4SY/v//6+vMzMzMzMzMzMzMVYvsg+T4gey0AAAAU4tdE

FaLdQxXixOD7BCJTCQci0YEx0QkLAMAAACLBJCL1IsID794Col8JHiLAYkCi0EEiUIEi0EIiUIIi0EMi0wkHIlCDIHBNAEAAP8A6MGGAACLTCQMiUQkSIXAD4QcgAMAixOLgUgBAABCi3YEiYQkhAAAADPAiUQkcIlEJHjHRCR8AQAAAIlEJESJRCRAiUQkXIlEJGCJRCRMiUQkZIsElsdEJDSMCUkAx0QkOAAAAADHRCQ

8AAAAAGaDeAhHx0QkWBgsSQCJEw+Fwn8DAEIz/4kTiVQkKIsLiwSOD79ACIP4R30HjUEBiQPr64PoRw+EEQoAAEgPhaF/AwCF/w+Fu38DAIvBiUQkEECJA4tEJEiLQBA7BaCCTAAPj+wJAACFwA+O5AkAAMHgBAMF3IJMAItIBIlEJCSJTCRQiwFmg3gIAHUJgzgrD4R8fwMAvwMAAACLRCRIM9uJfCQUiVwkLIlcJCA5W

BQPj/gFAACLTCQQil0IhNsPhTCAAwCLdCQMi3wkSItEJCw7RxgPjJ6DAwA7RxQPj5WDAwA70Q+FjYMDAItHEMdEJGgAAAAAiUQkbIXAD4jIgAMAjUQkaLmAgkwAUOjmiP//i3QkaIX2D4W6gAMAg38UAQ+N5wIAAItcJAyLRCQsx4QkiAAAAAAAAADHhCSQAAAAAAAAAImDSAEAAIsDx4QklAAAAAEAAADGhCSYAAAAAMe

EJJwAAAAAAAAAi3gEA/vGhCSgAAAAAMeEJKgAAAAAAAAAx4QksAAAAAAAAADHhCS0AAAAAQAAAIB/DQDGhCS4AAAAAA+FAoIDAGpA6E5dAQCL8IPEBIX2D4ThCAAAjYQkiAAAAMdGCAAAAABQi87opZv//42EJKgAAADGRhAAjU4gx0YUAAAAAMZGGABQx0EIAAAAAOh/m///xkYwAItHCIlGOIl3CItEJEiNs1wBAAD/R

wSLu/QAAACLTCQMi0AQix5AUOhfCQAAOx4PgpOBAwCLXCQMiwOJu/QAAACLSASLRBkIA8uAeQ0AD4VVCAAAgHgQAItFGA+E3gYAAIB5DQDGAAEPhWqBAwCLeQiLdRQ793QZg34MBA+EYYEDAIvO6FOa//9Xi87o65r//4sDi3AEi0QeCAPzgH4NAA+FCwgAAIB4GAAPhU2BAwCAfg0Ai0YID4X8BwAAgHgwAA+FTYEDAIs

Di3AEA/OAfg0AD4U=",

  "packet_info": {

    "linktype": 1

  }

}

Elasticsearch/Kibana

The same result can be achieved with a similar query can also be used in Elasticsearch Suricata alert data: 

event_type:alert AND metadata.flowbits:*stamus.nrd* AND alert.signature:*exe* AND http.status:200

SIEM/DB Query based

Elasticsearch/Kibana - HTTP

In this example we will use purely HTTP protocol transaction logging from Suricata.

For a regular Elasticsearch query – based only on Suricata HTTP logs (not alerts) – we can use the following (based on “event_type”:”http” logs in Suricata):

event_type:http AND metadata.flowbits:*stamus.nrd* AND metadata.flowbits:*http.binary* AND http.status:200   

Which basically says: 

  • Give me all HTTP transactions 
  • That have Executables transferred to or from NRD
  • with HTTP status code 200 

File Transactions 

We can do similar for the Fileinfo transactions (based on “event_type”:”fileinfo” logs in Suricata):

event_type:fileinfo AND metadata.flowbits:*stamus.nrd* AND fileinfo.magic:*executable* 

Which basically says: 

  • Give me all file transactions 
  • That have Executables transferred to or from Newly Registered Domain

That way , in this PCAP review case we go down from over 7600 Suricata file transaction logs….. 

…to 1 that needs our attention:

The actual file transaction log contains all the relevant filename/filemagic/sha256,md5 sums and so on:

{

  "timestamp": "2023-10-12T22:09:13.481788+0200",

  "flow_id": 1689769056868824,

  "pcap_cnt": 1026,

  "event_type": "fileinfo",

  "src_ip": "104.21.91.46",

  "src_port": 80,

  "dest_ip": "10.10.10.113",

  "dest_port": 64642,

  "proto": "TCP",

  "pkt_src": "wire/pcap",

  "metadata": {

    "flowbits": [

      "stamus.nrd",

      "ET.http.binary"

    ]

  },

  "ether": {

    "src_mac": "00:0c:30:5f:c4:a3",

    "dest_mac": "08:1f:71:62:50:7d"

  },

  "http": {

    "hostname": "hgfdytrywq.com",

    "url": "/a",

    "http_user_agent": "Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) WindowsPowerShell/5.1.19041.3031",

    "http_content_type": "application/octet-stream",

    "http_method": "GET",

    "protocol": "HTTP/1.1",

    "status": 200,

    "length": 893608

  },

  "app_proto": "http",

  "fileinfo": {

    "filename": "Autoit3.exe",

    "magic": "PE32 executable (GUI) Intel 80386, for MS Windows, 5 sections",

    "gaps": false,

    "state": "CLOSED",

    "md5": "c56b5f0201a3b3de53e561fe76912bfd",

    "sha1": "2a4062e10a5de813f5688221dbeb3f3ff33eb417",

    "sha256": "237d1bca6e056df5bb16a1216a434634109478f882d3b1d58344c801d184f95d",

    "stored": true,

    "file_id": 1,

    "size": 893608,

    "tx_id": 0

  }

}

In the case above the file was also stored / extracted to disk - "stored": true - which can be useful for further file analysis during incident response.  

Conclusion

Some people might still consider Suricata a “legacy” intrusion detection system (IDS), but the data produced by Surcata speaks for itself. It is not only a highly capable IDS but has evolved into an impressive tool for gathering NSM data and full protocol, file transaction, flow, and anomaly logs along with file extraction and PCAP logging. And as we’ve shown at Stamus Networks, Suricata is in fact a powerful foundation on which to build a world-class network detection and response (NDR) system. 

We hope this article helps to dispel some of these misconceptions and can help users see the many ways to use and optimize Suricata beyond the basic alerts and signatures. 

Additional resources

Further reading on this topic can be found in our free book, “Suricata for Analysts.”” the world’s first practical guide to threat detection and hunting using Suricata. 

To read part 3 of this series, please visit this link

Please make sure you check out our free and Open Source contributions to Suricata on our Stamus Labs page.

If you wish to be notified when we publish similar content, please subscribe to the Stamus Networks blog

Finally, to receive near-real time updates, follow us on Twitter, LinkedIn, and Facebook, or join our Discord.

Peter Manev

Peter Manev is the co-founder and chief strategy officer (CSO) at Stamus Networks. He is a member of the executive team at Open Network Security Foundation (OISF). Peter has over 15 years of experience in the IT industry, including enterprise-level IT security practice. He is a passionate user, developer, and explorer of innovative open-source security software, and he is responsible for training as well as quality assurance and testing on the development team of Suricata – the open-source threat detection engine. Peter is a regular speaker and educator on open-source security, threat hunting, and network security at conferences and live-fire cyber exercises, such as Crossed Swords, DeepSec, Troopers, DefCon, RSA, Suricon, SharkFest, and others. Peter resides in Gothenburg, Sweden.

Schedule a Demo of Stamus Security Platform

REQUEST A DEMO

Related posts

Uncovered with Stamus Security Platform: Tapped on the Shoulder

In this series of articles, we explore a set of use cases that we have encountered in real-world...

SELKS 10: The Next Big Leap for Open-Source Network Security

Stamus Networks is pleased to announce the release and availability of SELKS 10, the newest version...