Indicators of compromise (IOCs) have become a key component of the threat detection practice. Modern sources of threat intelligence are now including meaningful context with their IOCs, but the context was not available in Suricata alert records. This has made analysis slower and more complex than it could be. Stamus Networks tackled this issue and contributed IOC matching with context to Suricata 8.0. It is now possible to ingest the JSON formatted contextual data from threat intelligence providers in Suricata and output the context directly in the alert record.

The Critical Value of IOCs

Indicators of compromise (IOCs) are a crucial part of any security practitioner’s tool box. IOCs are a set of values – such as domain names or IP addresses – that can be used to identify activity known to be malicious.

Security professionals compare a list of IOCs against metadata in security logs or live traffic to discover if any activity related to a threat actor has taken place in their organization. 

For example, an industry group with common interests (such as an Information Sharing and Analysis Center, or ISAC) might share a list of domains linked to various threat actors may, and the security team will have to check if these domains have been seen within DNS, TLS or HTTP network traffic in order to identify potential compromise. This task can be performed on application logs (DNS servers, proxy) or by inspecting data from passive network traffic analysis tools such as Suricata. 

Managing IOCs in Suricata

In the case of Suricata, the analysis can be done in two ways: 

Searching for IOCs in Suricata’s NSM log data is straightforward, and merely requires looking for a string matching the IOC in one of the fields produced by Suricata. Using our domain name list example from before, an analyst searching on dns.rrname for DNS, tls.sni for TLS and http.host for HTTP is sufficient to find traces of the domain usage. For this method to identify an active threat, it requires proactive threat hunting – a practice not always employed by many security organizations.

For live traffic matching, the Suricata dataset feature can perform the matching in real-time. This feature has been available since Suricata 5.0.0. The dataset feature enables IOC matching as follows: A file with one entry per line (encoded in base64) is written on disk and a signature referencing the file loads all the entries and performs a full string match on the specified buffer. For example, to match on TLS server name indication:

tls.sni; dataset:isset,my_ioc,type string,load myiocs.lst; 

This will only match if the TLS SNI is equal to one of the entries (line) in myiocs.lst.

The advanced reader will note that if the user wishes to match on a domain, then this signature is not correct as it will not match on subdomain. For instance if we have toto.com in the list, then www.toto.com will not match. This issue is addressed in Suricata 8 by using the domain transform.

tls.sni; domain; dataset:isset,my_ioc,type string,load myiocs.lst;

This feature was contributed by Stamus Networks, and users of our Clear NDR system have had access to this technique for a long time. This is, by the way, used in the Newly Register Domain feed we provide.

The Importance of Retaining Context in IOCs

IOCs are published by Computer Emergency Response Teams (CERTs) or dedicated organisations and it is more common to see IOCs being shared inside communities via tools such as ISAC, MISP, or OpenCTI. So organizations incorporate IOCs from different sources and reference different entities. This causes IOCs to be really numerous and to have a huge range of criticality. Because of that, many large organizations have found that an IOC without context is not valuable. And the problem with the original Suricata dataset feature is that we lose the context associated with the IOCs that may have been included in the threat intelligence feed. 

So, in order to access that valuable context, the only solution is to search for the IOC within the threat intel source itself to understand the context. This can work, but in practice it is far too much effort. 

In fact, it is nearly impossible to answer these simple questions using a SIEM or data lake:

• How do the observed IOCs align with the MITRE ATT&CK® framework?
• What are the known threat actors active on my network?

A new feature in Suricata 8.0 called “JSON Data for Datasets” is an evolution that is meant to fix this problem. This feature was contributed by Stamus Networks to the Suricata code base. The concept of this dataset change is to add the context that is included with the IOC to the alert event record. Now, with Suricata 8.0, the associated information included from the IOCs provider is not lost, and the analyst can directly use the alert in the SIEM without requiring any external correlation.

Instead of simply reading the value, Suricata reads the context along with the IOCs and outputs it in the alert.context key. This provides a result such as that shown in the following example:

"alert": {

    "signature_id": 1,

    "signature": "IOCs",

    "context": {

      "threat": {

         "ioc": "bad-actors.com", 

         "threat_actors": [

           "Killer Bear",

           "LSD kitten"

          ],

         "mitre": {

           "tactic_id": "TA0011",

           "tactic": "Command and Control",

           "technique_id": "T1573",

           "technique": "Encrypted Channel" 

         },

         "source": "mycti"

      }

   }

With data like this, an analyst can issue a query for alert.context.threat.mitre.technique, and they will get  access to the MITRE technique information and it is now possible to perform an additional search or do statistical analysis. Similarly, the threat actors can quickly be identified by querying alert.context.threat.treat_actors.

The JSON format is completely arbitrary. It just needs to have a key present that contains the field to use as IOC. Suricata will get the data from the file and output all JSON data for an IOC under a JSON key (here threat) that is defined in the signature.

For an output like the one in the previous example, Suricata requires the data file which is injected into the probe to contain the information available in the CTI. In this example, the associated data would be represented as follows (in newline delimited JSON), the first line is our match. See below:

{"ioc": "bad-actors.com", "threat_actors": ["Killer Bear", "LSD Kitten"], "mitre": { "tactic_id": "TA0011", "tactic": "Command and Control", "technique_id": "T1573", "technique": "Encrypted Channel"}, "source": "mycti"}

{"ioc": "vpn-provider.com", "network_role": "VPN Provider", "source": "mycti"}

And the dataset match would be:

tls.sni; domain; \

         dataset:isset,json_ioc,type string,format ndjson,load ioc.ndjson, \   

         context_key threat,value_key ioc;

The context_key is used to specify that output should be sent to alert.context.threat so we can have multiple datasets writing the same context. The value_key specifies in which field the IOC value is to be taken from (here the ioc key).

This feature will simplify the life of the rule writers and security analysts. 

• Rule writers - Only one signature is required per protocol to match the same dataset with a JSON file with one IOC type. 
• Security analysts - Benefit from easily searchable and fully contextualized data in the data lake.

Summary

The new "JSON Data for datasets" feature in Suricata 8.0 (available for download here >>), contributed by Stamus Networks, significantly enhances IOC handling by embedding contextual threat intelligence directly into alert records. This eliminates the need for external correlation, enabling security analysts to easily search and analyze fully contextualized data, such as MITRE ATT&CK techniques and threat actors, within their SIEM or data lake. This streamlines analysis, simplifies rule writing, and makes threat detection more efficient.