<img height="1" width="1" style="display:none;" alt="" src="https://px.ads.linkedin.com/collect/?pid=2180921&amp;fmt=gif">

Suricata Myth Busting: Alerts and NSM

As I mentioned in the introductory article in this series (see here >>), Suricata produces not only IDS alerts but also produces protocol transaction logs, flow records, full packet capture and extracted files.

In this article we attempt to clear up a couple of misconceptions (myths) and illustrate these lesser-known capabilities through real-world examples.

Data used in this article

The example that we will use in this article to showcase Suricata’s capabilities is from https://www.malware-traffic-analysis.net/2021/02/08/index.html

NOTE: The PCAP and the artifacts extracted or present (files/domains etc) in this example contain live malware. If you chose to explore or visit any of the domains/servers/files/ listed here, you should do so at your own risk, in a controlled manner and only in a sandbox environment.


Suricata Myth 1 - Alerts Only

Myth: Suricata produces only alerts/alarms and you need another tool to produce the network protocol log evidence (network protocol transaction or network security logs) for that alert in order to confirm if it is real or a false positive.

Fact: This is simply not true.

Suricata produces all relevant network security monitoring/protocol logs, including the ones related to an alert - but also independent of alerts. In the regular JSON logs that Suricata generates (eve.json), you will find something called “flow_id” that correlates the network protocol data and evidence that Suricata has logged - to an alert event and that alert’s metadata. So for example for a HTTP based alert, all the relevant HTTP and or file transactions and flow logs would be available.

The ability to correlate any existing evidence/logs to an alert (“flow_id”) was introduced in 2014 by Suricata lead developer, Victor Julien

https://github.com/OISF/suricata/commit/f1185d051c210ca0daacdddbe865a51af24f4ea3

There are, of course, other and many more network security and protocol logs produced by Suricata before and after an alert occurred that are also accessible to the user.

Classic IDS alert

Here is an example of a legacy/classic IDS alert

Alert log:
02/08/2021-17:00:13.183792 [**] [1:2014819:3] ET INFO Packed Executable Download [**] [Classification: Misc activity] [Priority: 3] {TCP} 8.208.10.147:80 -> 10.2.8.101:49757

You can see that it does not contain much information beyond source and destination IP address. In short, it is distinctly lacking in context that might be useful for incident investigation or threat hunting.

The Suricata alert difference

The Suricata difference can be seen below. 

NOTE: for easier reading, check these out on Pastebin here: https://pastebin.com/vXF59n6u

Notice the areas highlighted below in blue which demonstrate the additional data and indicate there are flow and file records associated with the alert. 

Alert log:
{"timestamp":"2021-02-08T17:00:13.183792+0100","flow_id":1890654805068045,"pcap_cnt":4153,"event_type":"alert","src_ip":"8.208.10.147","src_port":80,"dest_ip":"10.2.8.101","dest_port":49757,"proto":"TCP","metadata":{"flowbits":["exe.no.referer"]},"alert":{"action":"allowed","gid":1,"signature_id":2014819,"rev":3,"signature":"ET INFO Packed Executable Download","category":"Misc activity","severity":3,"metadata":{"created_at":["2012_05_30"],"updated_at":["2012_05_30"]}},"http":{},"app_proto":"http","flow":{"pkts_toserver":9,"pkts_toclient":11,"bytes_toserver":1042,"bytes_toclient":9759,"start":"2021-02-08T17:00:12.297229+0100"}}
{"timestamp":"2021-02-08T17:00:13.408464+0100","flow_id":1890654805068045,"pcap_cnt":4208,"event_type":"alert","src_ip":"8.208.10.147","src_port":80,"dest_ip":"10.2.8.101","dest_port":49757,"proto":"TCP","metadata":{"flowbits":["exe.no.referer","ET.http.binary"]},"tx_id":2,"alert":{"action":"allowed","gid":1,"signature_id":2018959,"rev":4,"signature":"ET POLICY PE EXE or DLL Windows file download HTTP","category":"Potential Corporate Privacy Violation","severity":1,"metadata":{"created_at":["2014_08_19"],"former_category":["POLICY"],"updated_at":["2017_02_01"]}},"http":{"hostname":"roanokemortgages[.]com","url":"/6lhjgfdghj.exe","http_user_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko","http_content_type":"application/octet-stream","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":41029},"files":[{"filename":"/6lhjgfdghj.exe","sid":[],"gaps":false,"state":"UNKNOWN","stored":false,"size":41029,"tx_id":2}],"app_proto":"http","flow":"pkts_toserver":24,"pkts_toclient":37,"bytes_toserver":1852,"bytes_toclient":46939,"start":"2021-02-08T17:00:12.297229+0100"}}

And Suricata’s Network Security Monitoring (NSM)

Below are a few of the network security and protocol logs produced by Suricata associated with the alert shown above. Note they are all tied together by a common flow_id value (highlighted in blue):

HTTP Log:
{"timestamp":"2021-02-08T17:00:12.648463+0100","flow_id":1890654805068045,"pcap_cnt":3883,"event_type":"http","src_ip":"10.2.8.101","src_port":49757,"dest_ip":"8.208.10.147","dest_port":80,"proto":"TCP","tx_id":0,"http":{"hostname":"roanokemortgages[.]com","url":"/0801.bin","http_user_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko","http_content_type":"application/octet-stream","accept":"*/*","cache_control":"no-cache","connection":"keep-alive","content_length":"876","content_type":"application/octet-stream","date":"Mon, 08 Feb 2021 16:00:13 GMT","last_modified":"Mon, 08 Feb 2021 13:20:38 GMT","server":"nginx","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":876,"request_headers":[{"name":"Accept","value":"*/*"},{"name":"User-Agent","value":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko"},{"name":"Host","value":"roanokemortgages[.]com"},{"name":"Cache-Control","value":"no-cache"}],"response_headers":[{"name":"Server","value":"nginx"},{"name":"Date","value":"Mon, 08 Feb 2021 16:00:13 GMT"},{"name":"Content-Type","value":"application/octet-stream"},{"name":"Content-Length","value":"876"},{"name":"Connection","value":"keep-alive"},{"name":"Last-Modified","value":"Mon, 08 Feb 2021 13:20:38 GMT"},{"name":"ETag","value":"\"60213aa6-36c\""},{"name":"Accept-Ranges","value":"bytes"}]}}

File Transaction log:
{"timestamp":"2021-02-08T17:00:12.648463+0100","flow_id":1890654805068045,"pcap_cnt":3883,"event_type":"fileinfo","src_ip":"8.208.10.147","src_port":80,"dest_ip":"10.2.8.101","dest_port":49757,"proto":"TCP","http":{"hostname":"roanokemortgages[.]com","url":"/0801.bin","http_user_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko","http_content_type":"application/octet-stream","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":876},"app_proto":"http","fileinfo":{"filename":"/0801.bin","sid":[],"magic":"data","gaps":false,"state":"CLOSED","sha256":"ee33a8fa2ae6f6b9366c97ed4c00c2796d98a371249dca725a01aca03caf747b","stored":false,"size":876,"tx_id":0}}

HTTP log:
{"timestamp":"2021-02-08T17:00:12.878417+0100","flow_id":1890654805068045,"pcap_cnt":3908,"event_type":"http","src_ip":"10.2.8.101","src_port":49757,"dest_ip":"8.208.10.147","dest_port":80,"proto":"TCP","tx_id":1,"http":{"hostname":"roanokemortgages[.]com","url":"/0801s.bin","http_user_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko","http_content_type":"application/octet-stream","accept":"*/*","cache_control":"no-cache","connection":"keep-alive","content_length":"913","content_type":"application/octet-stream","date":"Mon, 08 Feb 2021 16:00:13 GMT","last_modified":"Mon, 08 Feb 2021 13:20:37 GMT","server":"nginx","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":913,"request_headers":[{"name":"Accept","value":"*/*"},{"name":"User-Agent","value":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko"},{"name":"Host","value":"roanokemortgages[.]com"},{"name":"Cache-Control","value":"no-cache"}],"response_headers":[{"name":"Server","value":"nginx"},{"name":"Date","value":"Mon, 08 Feb 2021 16:00:13 GMT"},{"name":"Content-Type","value":"application/octet-stream"},{"name":"Content-Length","value":"913"},{"name":"Connection","value":"keep-alive"},{"name":"Last-Modified","value":"Mon, 08 Feb 2021 13:20:37 GMT"},{"name":"ETag","value":"\"60213aa5-391\""},{"name":"Accept-Ranges","value":"bytes"}]}}

File Transaction log:
{"timestamp":"2021-02-08T17:00:12.878417+0100","flow_id":1890654805068045,"pcap_cnt":3908,"event_type":"fileinfo","src_ip":"8.208.10.147","src_port":80,"dest_ip":"10.2.8.101","dest_port":49757,"proto":"TCP","http":{"hostname":"roanokemortgages[.]com","url":"/0801s.bin","http_user_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko","http_content_type":"application/octet-stream","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":913},"app_proto":"http","fileinfo":{"filename":"/0801s.bin","sid":[],"magic":"data","gaps":false,"state":"CLOSED","sha256":"7af0dc117d2dcd112f50889c4c8a14ac9ee55c2525a24fa66ff9a89b480b7e99","stored":false,"size":913,"tx_id":1}}

HTTP log:
{"timestamp":"2021-02-08T17:00:13.699055+0100","flow_id":1890654805068045,"pcap_cnt":4506,"event_type":"http","src_ip":"10.2.8.101","src_port":49757,"dest_ip":"8.208.10.147","dest_port":80,"proto":"TCP","metadata":{"flowbits":["exe.no.referer","ET.http.binary"]},"tx_id":2,"http":{"hostname":"roanokemortgages[.]com","url":"/6lhjgfdghj.exe","http_user_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko","http_content_type":"application/octet-stream","accept":"*/*","cache_control":"no-cache","connection":"keep-alive","content_length":"273422","content_type":"application/octet-stream","date":"Mon, 08 Feb 2021 16:00:14 GMT","last_modified":"Wed, 20 Jan 2021 09:59:19 GMT","server":"nginx","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":273422,"request_headers":[{"name":"Accept","value":"*/*"},{"name":"User-Agent","value":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko"},{"name":"Host","value":"roanokemortgages[.]com"},{"name":"Cache-Control","value":"no-cache"}],"response_headers":[{"name":"Server","value":"nginx"},{"name":"Date","value":"Mon, 08 Feb 2021 16:00:14 GMT"},{"name":"Content-Type","value":"application/octet-stream"},{"name":"Content-Length","value":"273422"},{"name":"Connection","value":"keep-alive"},{"name":"Last-Modified","value":"Wed, 20 Jan 2021 09:59:19 GMT"},{"name":"ETag","value":"\"6007fef7-42c0e\""},{"name":"Accept-Ranges","value":"bytes"}]}}

File Transaction log:
{"timestamp":"2021-02-08T17:00:13.699055+0100","flow_id":1890654805068045,"pcap_cnt":4506,"event_type":"fileinfo","src_ip":"8.208.10.147","src_port":80,"dest_ip":"10.2.8.101","dest_port":49757,"proto":"TCP","metadata":{"flowbits":["exe.no.referer","ET.http.binary"]},"http":{"hostname":"roanokemortgages[.]com","url":"/6lhjgfdghj.exe","http_user_agent":"Mozilla/5.0 (Windows NT 6.1; Win64; x64; Trident/7.0; rv:11.0) like Gecko","http_content_type":"application/octet-stream","http_method":"GET","protocol":"HTTP/1.1","status":200,"length":273422},"app_proto":"http","fileinfo":{"filename":"/6lhjgfdghj.exe","sid":[],"magic":"PE32 executable (GUI) Intel 80386 (stripped to external PDB), for MS Windows","gaps":false,"state":"CLOSED","sha256":"94e60de577c84625da69f785ffe7e24c889bfa6923dc7b017c21e8a313e4e8e1","stored":false,"size":273422,"tx_id":2}}

Flow record:
{"timestamp":"2021-02-08T16:58:15.247118+0100","flow_id":1890654805068045,"event_type":"flow","src_ip":"10.2.8.101","src_port":49757,"dest_ip":"8.208.10.147","dest_port":80,"proto":"TCP","app_proto":"http","flow":{"pkts_toserver":105,"pkts_toclient":207,"bytes_toserver":6226,"bytes_toclient":287136,"start":"2021-02-08T17:00:12.297229+0100","end":"2021-02-08T17:01:56.764859+0100","age":104,"state":"closed","reason":"shutdown","alerted":true},"metadata":{"flowbits":["exe.no.referer","ET.http.binary"]},"tcp":{"tcp_flags":"1b","tcp_flags_ts":"1b","tcp_flags_tc":"1b","syn":true,"fin":true,"psh":true,"ack":true,"state":"closed"}}

Suricata Myth 2 - NSM requires signatures/rules 

Myth: Suricata can only function with signatures/rules.

Fact: This is also not true. And it could be one of the most misunderstood capabilities of Suricata.

Suricata actually produces network security monitoring logs - protocol transactions and network flow data without signatures. In fact, if you were to switch off the rules/signatures in the example above, Suricata will still produce the exact same logs as above without the alert itself. That is, everything else will still be logged.

Here’s an example on Pastebin here:
https://pastebin.com/PxKDx2NF

In the next article, we will provide a more detailed review of the NSM capabilities of Suricata.

Further Reading on Suricata

Stamus Networks has produced a number of articles and papers on Suricata. Please see the list below for more information:

White Paper: Scaling Suricata for Enterprise Deployment
https://www.stamus-networks.com/en-us/landing/wp-scaling-suricata?hsLang=en

White Paper: Introduction to eBPF and XDP support in Suricata
https://www.stamus-networks.com/en-us/landing/introduction-to-ebpf-and-xdp-support-in-suricata?hsLang=en

Blog: Suricata - The First 12 Years of Innovation
https://www.stamus-networks.com/blog/suricata-the-first-12-years-of-innovation

Blog: The Other Side of Suricata
https://www.stamus-networks.com/blog/the-other-side-of-suricata

Blog: Advanced Deployment & Configuration with Suricata at SuriCon 2021
https://www.stamus-networks.com/blog/advanced-deployment-configuration-with-suricata-at-suricon-2021

Blog: Introducing the Stamus Networks App for Splunk®
https://www.stamus-networks.com/blog/introducing-the-stamus-networks-app-for-splunk

Blog: Just Released - Suricata 6
https://www.stamus-networks.com/hubfs/Library/Documents%20(PDFs)/StamusNetworks_TB-U37SURI-062021-2.pdf?hsLang=en

Tech Brief: Advanced capabilities in Stamus NDR for Suricata Sensors
https://www.stamus-networks.com/hubfs/Library/Documents%20(PDFs)/StamusNetworks_TB-U37SURI-062021-2.pdf?hsLang=en

 

 

 

Schedule a Demo of Stamus ND or Stamus NDR

REQUEST A DEMO

Related posts

The Art of Suricata QA GitLab Automation @ Suricon

I’m pleased to partner with the OISF QA Engineer and Automator - Corey Thomas for this year’s...

Unleash Suricata Superpowers with a Splunk App @ Suricon

At next week's Suricon 2021, I'll be sharing real world examples of how a new Splunk App can help...

Efficient Suricata: Migrating from Millions of Events to Manageable Insights @ Suricon

I’m pleased to partner with my colleague and OISF Board Member - Eric Leblond for this year’s...