July 2017

0

Suricata 4.0 is out and this switch from 3.x to 4.x is not marketing driven because the changes are really important. This post is not exhaustive on changes. It is Stamus Networks’ take on some of the important changes that have been introduced in this version.

Rust addition

This is the big step forward on the technology side. Suricata is written in C language. This gives performances and a good control over memory. But it goes with a series of well known problems. I name here buffer overflows, use after free, …

And the worse is that Suricata is parsing traffic content which is a kind of vice supercharged user input. If one should not trust user input, guess how careful we should be with network traffic. At Suricon 2016, Pierre Chifflier did present a proof of concept implementation of protocol parsers in Rust. The idea is to use the property of Rust that has been designed to avoid complete class of attacks on memory handling. But there is more in the approach as the implementation is using Nom which is a Rust parser combinator framework. It allows you to write protocol parser easily and in a reusable way. Thus the addition of Rust is two things at the same time: more security and easier code. Which means a lot of new protocols should be added in the near future.

Suricata 4.0 Rust support comes with NFS, DNS and NTP. NTP support is implemented via an external crate (read library): ntp-parser.

As mentioned before, the code uses Nom and the syntax is very different from traditional code. For instance, here is the code of ntp-parser parsing NTP extension:

named!(pub parse_ntp_extension,
    do_parse!(
           ty: be_u16
        >> len: be_u16 // len includes the padding
        >> data: take!(len)
        >> (
            NtpExtension{
                field_type:ty,
                length:len,
                value:data,
            }
        ))
);

This define a parsing function that read the stream of data. The code says, take 16 bits, store them as unsigned integer in ty. Then store the next 16 bits as unsigned integer in len. Then store in data a chunck of data of length len. And with that build a NTP extension structure. If the writing is concise and efficient, the best thing with Nom is under the hood. Nom is taking care of detecting the invalidities. For instance we could have a chunck of data of length 50, and len being set to 1000 (remember Heartbleed ?). Nom will see that there is not enough data available in the chunck and return it wants more data.

Better alerts

As you may know, the preferred output of Suricata is the EVE JSON format. It is flexible, easy to extend and easy to read by human and tools. Suricata 4.0 is introducing some major changes here:

  • ‘vars’ extraction mechanism
  • The new target keyword
  • HTTP bodies logging
HTTP body output

Suricata is able to uncompressed HTTP body on the fly and match on the uncompressed content. This means that if you get the payload of the stream triggering the alert in your event, you will just see compression noise and won’t be able to analyze why the alert was triggered. Suricata is now able to include the HTTP bodies in the alert. The analyst can then directly see from the event the content that did trigger the alert.

The following event shows how payload_printable is completely compression noise and the http_response_body_printable is readable:

Target keyword

The new target keyword is a fix on a very old problem. It is not possible to know in an alert event which side of the source or destination is the target of the attack. This is a problem as it is not possible to automate things due to that lack of information. The target keyword allow the rules writer to specify which is side is the target. Doing so automated analysis and better visualization can be made.

Usage is simple, signature has to contain the target keyword with value dest_ip or src_ip. For example, in a simple scan alert we have:

alert tcp $EXTERNAL_NET any -> $HOME_NET 3306 (msg:"ET POLICY Suspicious inbound to mySQL port 3306"; flow:to_server; flags:S; threshold: type limit, count 5, seconds 60, track by_src; reference:url,doc.emergingthreats.net/2010937; classtype:bad-unknown; target: dest_ip; sid:2010937; rev:2;)

If target is present in a signature, the alert is added an alert.source and alert.target field:

For example, on a visualization where node are IP address and links are alerts between the two, we can get an idea of the possible compromised path. With the target addition, we can switch from a non oriented graph:

To an oriented graph that show which paths were really possible:

If you know French, you can learn more about this subject with Eric Leblond’s talk at SSTIC 2017.

Vars extraction

This is one of the most expected feature of Suricata 4.0. This has been described by Victor Julien in an extensive blog post. The concept is to be able to define in signature data to extract and store them in a key value form. There is a lot of possible usage ranging from application version extraction to getting exfiltred data. For example, let’s consider there is a domain we are interested in. One interesting information is the list of email addresses where mail are sent to. To do so we can use the following signature:

alert smtp any any -> any any (msg:"Mail to stamus"; content:"rcpt to|3A|"; nocase; content:"stamus-networks.com"; within: 200; fast_pattern; pcre:"/^RCPT TO\x3a\s*<([\w-\.]+@stamus-networks.com)>/ism pkt:email"; flow:established,to_server; sid:1; rev:1;)

The magix here is the groupe in the regular expression ([\w-\.]+@stamus-networks.com) that is save in a packet var named email by the pkt:email in the regular expression definition.

Using that signature we get this kind of alerts:

The key point here is the vars sub object:

  "vars": {
    "pktvars": [
      {
        "email": "eleblond@stamus-networks.com"
      }
    ]
  },

We have an extraction of the data and this can be easily search by tool like Elasticsearch or Splunk.

Conclusion

Suricata 4.0 is really an important milestone for the project. Introduction of Rust is opening a really interesting path. The alerts improvement may change the way signatures are written and it will help to provide really accurate information to the analysts.

Suricata 4.0 is already available in SELKS and it will be available in Stamus Probe by the end of August. To conclude on a personal note, we, Stamus Networks, are really happy to have contributed to this release with features such as via HTTP body logging and target keyword.

0

After a very valuable round of testing and feedback from the community  we are pleased to announce the SELKS 4 RC1 availability.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a the release candidate of a new major branch with an updated storage visualization stack and latest Suricata.

New Features

  • Suricata IDS/IPS/NSM 4.0.x – latest git master Suricata packaged with Hyperscan enabled for extra performance boost. This edition of Suricata besides many improvements and bug fixes also includes extra alert data like for example http body added to the alert json logs wherever available.
  • Elasticsearch 5.5.0  – part of the ELK5 stack upgrade making available a ton of new features and enhancements.
  • Logstash 5.5.0 – performance improvement over 2.x and ES5 compatibility.
  • Kibana 5.5.0 – taking advantage of the latest dashboarding features of ES.
  • Scirius 1.2.2 – bugfixes, better correlation capability with EveBox and introduction of IPS rules support.
  • Evebox – many new features including reporting and comments on the log events.
  • Debian Stretch – All new features, kernel and tools.

EveBox

Alert event with a comment field.

Kibana

Verbose HTTP logging

Kibana

GeoIP heat maps

EveBox

Supplemental alert data logging

 

Download

To download SELKS4-RC1:

Usage

Usage and logon credentials (OS and web management user)

  • user: selks-user
  • password: selks-user (password in Live mode is live)

The default root password is StamusNetworks

To remotely access the web management interface :

  • https://your.selks.IP.here/ – Scirius ruleset management and a central point for all dashboards and EveBox alert and event management.

Howto

Upgrade

To upgrade your existing SELKS 3 to SELKS 4 preview, please refer to SELKS-3.0-to-SELKS-4.0-upgrades wiki page.

It is recommended to follow the onscreen instructions and if needed answer “yes” to all changes. At the end of the upgrade you will be asked to enter the interface that you will use for IDS/sniffing. Please enter (eth0 for example) the interface name and reboot when the script is done.

Create your own ISO

To create your own SELKS 4 preview ISO (if your host OS is Jessie):

git clone https://github.com/StamusNetworks/SELKS.git
git checkout SELKS4-dev
./install-deps.sh
cd /usr/share/live/build/data/debian-cd/ && ln -s squeeze stretch
./build-debian-live.sh

It will take probably 30-40 min and you should end up with the SELKS.iso under the Stamus-Live-Build folder.

Once installed/upgraded
  • Please feel free to choose the IDS sniffing/listening interface either via the desktop icon Setup-IDS-Interface or via the cmd calling /opt/selks/Scripts/Setup/setup-selks-ids-interface.sh
  • Any further upgrades are done via a wrapper script located in /opt/selks/Scripts/Setup/selks-upgrade_stamus.sh
  • Recommended set up for SELKS 4.0RC1 is 2CPUs 5-6Gb RAM
  • If you need to reset/reload all the dashboards  – you can do like so
    • In Scirius on the top left corner drop down menu select System Settings
    • click on the Kibana tab
    • choose Reset SN dashboards

Feedback is welcome

Give us feedback and get help on:

While this test upgrade/installation has been verified and tested and aims at upgrading your current SELKS 3.0 to  SELKS 4.0RC1 please make sure you try it in your test/QA set up first and give us any feedback.

Thank you!