Our Blog

0

Introduction

This is a short tutorial of how you can find and store to disk a self signed TLS certificates with Suricata IDPS and Lua (luajit scripting).

What does self signed TLS certificate mean – the quick version  from Wikipedia here. In other words – “certificate that is signed by the same entity whose identity it certifies” or anyone can create and deploy such a certificate. This kind of events are signed of some poorly setup TLS servers and that’s why it is good to keep an eye on such events in your network for the purpose of contingency monitoring – the very least.

TLS support in Suricata allows you to do match on TLS subject, TLS issuer DN and things like TLS fingerprint. For instance, one can do

alert tls any any -> any any (msg:"forged ssl google user";
            tls.subject:"CN=*.googleusercontent.com";
            tls.issuerdn:!"CN=Google-Internet-Authority"; sid:8; rev:1;)

to detect that the TLS issuer DN is not the one we were waiting for a given TLS subject. But there is no way to compare the two different fields. So how do we catch all those self signed certificates without knowing any details about them and/or any network/domain/port specifics either. And we want them all caught and stored to disk!

This case is one example where the Lua support in Suricata IDPS  shines – more than shines actually because it empowers you to do much more than chasing packet bytes with rule keywords and using PCREs inside a rule – and all those still deliver limited functionality in this particular scenario.

Lua and Suricata

Since version 2.0, Suricata has support for Lua scripting. The idea is to be able to decide if an alert is matching based on the return of a lua script. This script is taking some fields extracted by Suricata (the magic of it all) as parameters and return 1 in case of match and 0 if not. This lua scripting allows rules to implement complex logic that would be impossible with the standard rule language. For instance, Victor Julien was able to write a performant Heartbleed detection with lua scripting in the afternoon (the very same day) the problem/exploit was announced.

The syntax is the following, you prefilter with standard signature language, and you add a lua keyword with parameter being the script to run in case of partial match:

alert tls any any -> any any ( \
    msg:"TLS HEARTBLEED malformed heartbeat record"; \
    content:"|18 03|"; depth:2; lua:tls-heartbleed.lua; \
    classtype:misc-attack; sid:3000001; rev:1;)

For more information on Suricata Lua scripting, please read how to write luascripts for Suricata.

For self signed certificate detection, you need to write a script – shall we say “self-signed-cert.lua” and save it in your /etc/suricata/rules directory. Then you can use it in a rule like so –

alert tls any any -> any any (msg:"SURICATA TLS Self Signed Certificate"; \
  flow:established; luajit:self-signed-cert.lua; \
  tls.store; classtype:protocol-command-decode; sid:999666111; rev:1;)

Now let us explain that in a bit more detail below.

At the time of this writing we are using this branch in particular – TLS Lua rebased . This is going to be merged to the Suricata git (latest dev) soon and later into beta, stable editions.

You need to make sure Suricata is compiled with Lua enabled:

# suricata --build-info
This is Suricata version 2.1dev (rev b5e1df2)
...
Prelude support:                         no
PCRE jit:                                yes
LUA support:                             yes
libluajit:                               yes
libgeoip:                                yes
...

In suricata.yaml make sure the tls section is enabled:

# a line based log of TLS handshake parameters (no alerts)
- tls-store:
  enabled: yes  # Active TLS certificate store.
  certs-log-dir: certs # directory to store the certificates files

We have the following rule file (self-sign.rules) content located in /etc/suricata/rules/ :

alert tls any any -> any any (msg:"SURICATA TLS Self Signed Certificate"; \
  flow:established; luajit:self-signed-cert.lua; \
  tls.store; classtype:protocol-command-decode; sid:999666111; rev:1;)

Make sure you add the rule to your rules files loaded in suricata.yaml and that you copy the associated lua script in the same directory. Here you can find the self-signed-cert.lua script.

Then you can start Suricata IDPS the usual way you always do.

The active part of the lua script is the following:

function match(args)
    version, subject, issuer, fingerprint = TlsGetCertInfo();
    
    if subject == issuer then
        return 1
    else
        return 0
    end
end

When suricata will see a TLS handshake (regardless of IP/port), it will run the Lua script. This one uses the fact that an equality between subject and issuer DN constitutes most of the self-signed certificates. When it finds such an equality it will return 1 and an alert will be generated.

This script is showing a really basic but useful code. However you can use all the power of Lua with the given info to do whatever you want/need.

The result

self-cert-extract

Extracted self signed SSL certificate

Some meta data about the certificate (in /var/log/suricata/certs/ you will find the .meta and pem  files):

TIME:              07/14/2015-14:45:16.757001
SRC IP:            10.0.2.15
DST IP:            192.168.1.180
PROTO:             6
SRC PORT:          49966
DST PORT:          443
TLS SUBJECT:       C=FR, ST=IDF, L=Paris, O=Stamus, CN=SELKS
TLS ISSUERDN:      C=FR, ST=IDF, L=Paris, O=Stamus, CN=SELKS
TLS FINGERPRINT:   80:e7:af:49:c3:fe:9a:73:78:29:6b:dd:fd:28:9e:d9:c9:15:3e:18

 

Further more from the alert info in JSON format (/var/log/suricata/eve.json log) we also have extra TLS info for the generated alert :

{"timestamp":"2015-07-14T14:45:18.076794+0200","flow_id":137451536,"in_iface":"eth0",
"event_type":"alert","src_ip":"192.168.1.180","src_port":443,"dest_ip":"10.0.2.15","dest_port":49966,"proto":"TCP","alert":
{"action":"allowed","gid":1,"signature_id":999666111,"rev":1,"signature":"SURICATA TLS Self Signed Certificate","category":"Generic Protocol Command 
Decode","severity":3},"tls":{"subject":"C=FR, ST=IDF, L=Paris, O=Stamus, CN=SELKS","issuerdn":"C=FR, ST=IDF, L=Paris, O=Stamus, 
CN=SELKS","fingerprint":"80:e7:af:49:c3:fe:9a:73:78:29:6b:dd:fd:28:9e:d9:c9:15:3e:18","version":"TLS 1.2"}}

To get the extra TLS info in the JSON alert you need to enable it in the suricata.yaml like so:

  # Extensible Event Format (nicknamed EVE) event log in JSON format
  - eve-log:
      enabled: yes
      filetype: regular #regular|syslog|unix_dgram|unix_stream
      filename: eve.json
      types:
        - alert:
            #payload: yes           # enable dumping payload in Base64
            #payload-printable: yes # enable dumping payload in printable (lossy) format
            #packet: yes            # enable dumping of packet (without stream segments)
            http: yes              # enable dumping of http fields
            tls: yes               # enable dumping of tls fields <<---
            ssh: yes               # enable dumping of ssh fields

so you can get all the JSON data in Kibana/Elasticsearch as well.

Conclusion

If you would like to learn more about what can you do with Lua and Suricata IDPS, those links below will put you off to a good start:

Suricata EVE JSON format is becoming the de-facto standard for this IDS. All type of events are now exported to this format. The JSON format allows a nice handling of data in external tool like Elasticsearch or even DOM. The output is readable by human but as an event/record can contain a lot of data it can be difficult to do a by-eye analysis when looking at a file. The following screenshot give you an idea of the possible output:

Tailing EVE

Using standard unix tools like grep on the EVE JSON file is not the perfect idea. For example if you want to extract a field to get some statistics you may want to try using grep, cut or awk but you may find it painful. And it is worthed to mention here that JSON fields are not ordered.

Here to the rescue comes the jq utility. jq is a tool dedicated to the transformation/parsing of a JSON entry. It is Debian packaged, so a simple apt-get install jq is enough for the install.

Some jq examples

The most basic usage is to colorize the entry. To do that, just do something like

$ tail -n100 eve.json| jq '.'

The output is done the pretty way:
JQ displaying an event
To get a one line per event output, just add the -c flag to the command:
One line

To extract a single field from the JSON events, one can do:

$ jq '.src_ip' eve.json
"58.218.211.155"
"58.218.211.155"
"58.218.211.155"

The point to remember is that the point in .src_ip is a place holder for the current entry.

By default when a field is not present null is displayed in the output. To fix that, it is possible to filter the event to only get the one we are interested in. This is done via the select keyword. For instance to select the SSH events and extract the information about the client part one can do:

$ tail eve.json | jq -c 'select(.event_type == "ssh")|.ssh.client'
{"proto_version":"2.0","software_version":"PUTTY"}
{"proto_version":"2.0","software_version":"PUTTY"}

Far more things can be done with jq. Good starting points are the jq manual and wiki.

0

Stamus Networks is proud to announce the availability of SELKS 2.0  release.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a major SELKS upgrade.

New Features

  • Debian Jessie based  – With Debian Jessie  being released last week  – 25 April   SELKS makes the switch as well (from Debian Wheezy and 3.2 kernel). The new Debian release Jessie will enable SELKS for much better HW compatibility, new kernel 3.16 and all the performance improvements, features and benefits with it right out of the box.
  • Elasticsearch 1.5  – upgrade from 1.4
  • Scirius 1.0 – upgrade from 1.0rc3

 

Some screenshot examples

Scirius

SELKS2.0-Scirius-2

Screenshot from 2015-04-20 22:07:08

IDS/IPS dashboards
SELKS2.0-1

12 ready to use IDS/IPS dashboards

VLAN3

By VLAN break down File Transactions/SSH

VLAN2

By VLAN break down DNS/TLS

SMTP-Attachments

By file attachment breakdown SMTP

VLAN1

By VLAN break down Alerts/HTTP

UPGRADE from SELKS1.2

For those that use SELKS 1.2 and would like to do an in place upgrade to SELKS 2.0 you can follow THIS GUIDE.

NOTE: Please make sure that you test the upgrade in your test/QA environment first before doing it on your production systems.

Please note that default login/password for HTTPS access (Dashboards or Scirius icons) is selks-user/selks-user.

More about SELKS 2.0

Stamus Networks is proud to announce the availability of Scirius 1.0. This is the first stable release of our web interface for Suricata ruleset management. It is providing an efficient way to manage and update the ruleset.

Scirius is displaying some graphics that will help you to get an idea of the activity of your Suricata probe and easily select rules that may be noisy and need to be deactivated:
Screenshot from 2015-04-20 22:07:08

But the main focus is on ruleset handling. You can for example follow the change of a signature source:

Screenshot from 2015-04-20 22:05:36

Scirius is not meant to replace a good dashboard interface, so it is providing a link to Kibana dashboard:
Screenshot from 2015-04-20 22:38:48

Scirius also allows you to search inside its database to find the elements you are looking for:

Screenshot from 2015-04-20 22:06:06

Scirius is able to handle multiple sources. So you can mixed local rules and rules download from outside sources such as Emerging Threats or SSLBL from abuse.ch:
Screenshot from 2015-04-20 22:05:52

Scirius is fetching activity information from Elasticsearch and it is now even able to display some interesting information about the state of your cluster
Screenshot from 2015-04-20 22:06:51

Scirius 1.0 is part of SELKS our live and installable Suricata NSM/IDS distribution. Happy SELKS can upgrade to scirius 1.0 via a simple apt-get update && apt-get upgrade. Other users can simply grab the release from Github.

The development will now focus on getting Scirius ready to handle IPS. So the changes will mostly be about rules transformation and the main features of Scirius should stay alike.

Stamus Networks is proud to announce the availability of the third release candidate of Scirius 1.0. This new release features minor bugfixes and an improvement.

Scirius 1.0-rc3 brings a new set of graphs that improve the visualization of the probe activity. The following video demonstrate the usage of this feature:



To sum up, the suricata page has now one zoomable sunburst graph who split the rules by classification:
sunburst-crop

There is also an alternate display via hierarchic circles:
circles-crop

The selection between the different type of graphs is done via the local settings:
Screenshot from 2015-04-03 09:07:31

You can download Scirius 1.0-rc3 from Github.

0

Stamus Networks is proud to announce the availability of SELKS 2.0 BETA1 release. With Jessie getting a target release date for 25 April 2015  SELKS will make the switch as well. The new Debian release Jessie will enable SELKS for much better HW compatibility, new kernel (3.16) and all the performance improvements, features and benefits with it right out of the box.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

Please give it a try and do not hesitate to feed back!

SELKS2.0Beta1-1

 

 

Download SELKS 2.0 BETA1

More information: Howto and README

Follow us on Twitter, Google+ and Github

Get help at Freenode IRC on the #SELKS channel and/or  Google Mailing list.

 

0

Some words about PRscript

PRSCript is a script that run a series of builds and tests on a given branch. It was reserved to some developers so they can check the quality of their work before submission. The test builds are run on Suricata buildbot which is composed of some different dedicated hardware system. buildbot is an open-source framework for automating software build, test, and release processes. In the case of Suricata instance, it is set up to run various builds, unit tests as well as functional tests (such as pevma’s regression script).

The fact that this script was reserved to some users was a limitation as many contributors are not registered as Suricata buildbot users. As well, the fact that the code has to be public was not convenient as you could have to expose code before it is ready (with shameful TODO inside). Another point is that you were not able to customize your build. For instance, if you were introducing a new library as dependency it was not possible to test it before a global modification of the buildbot.

PRscript with docker support

To get over these limitations, Victor Julien and I have discussed on using Docker to allow developers to simply run a Suricata dedicated buildbot. As you may/should already know Docker is an open platform for distributed applications for developers and sysadmins. It allows you to quickly install install, manage and run containers. In our case, the idea was to start a pre-configured buildbot container using your local git as reference code. This way you can simply start test builds on your private code without even needing.

So, I have worked on this Docker based buildbot installation dedicated to Suricata and it has been merged in Suricata mainstream by Victor Julien.

It is now possible to use the prscript locally via Docker. Installation had been made simple so you should just have a few commands to run before being ready.

The buildbot will run various builds (gcc and clang, different builds options) and run suricata against some pcaps to check against possible crash.

Screenshot from 2015-04-07 16:22:19

Installation

Prerequisites

You need to have docker and python-docker installed on your system. Optionally you can install pynotify on your system to get desktop notification capability. On recent Debian based distribution you can use:

sudo apt-get install docker python-docker python-notify

Create the container

This operation has only to be done once. From the root of
Suricata sources, run:

sudo qa/prscript.py -C

It will take some times as the download is several hundred Mo. The result will be a docker container named ‘suri-buildbot’.

Using the buildbot

Start the buildbot

When you need to use the buildbot, you can start it from the command line:

sudo qa/prscript.py -s

You can check it is running via:

sudo docker ps

And you can connect to the buildbot web interface via http://localhost:8010

Start a build

Once the buildbot is active, you can start a build:

qa/prscript.py -d -l YOUR_BRANCH

This will start a build of the local branch YOUR_BRANCH without requiring any connectivity.

To get warned of the result of the builds via a desktop notification:

qa/prscript.py -d -l YOUR_BRANCH -n

Stop the buildbot

When you don’t need the buildbot anymore, you can stop it from the command line

 sudo qa/prscript.py -S

For further details, check Suricata docker QA page on OISF redmine.

Advanced usage

Build customisation

Buildbot will make suricata read all the pcap files available in qa/docker/pcaps/. So you can use this directory to add your own test pcaps.

Buildbot configuration is stored inside your suricata sources. It is the file qa/docker/buildbot.cfg. So, you can change the Buildbot configuration by editing this file. Then stop and start the docker container to get the new version used. This can be for example used when you need to add a flag to the configure command to activate a new feature.

What is great about this docker way of doing things is that it solves easily some complex points. For instance, if the buildbot configuration were coming from the Docker image then it will not be possible to easily edit it. Furthermore developer will loose any changes made in case of image upgrade. Also, the configure flags used by the buildbot will always be related to the current state of the code. So there will be no issue with running builds even if you are working on some older code as your buildbot configuration will be synchronized first.

Connect via ssh

The docker instance can be accessed via SSH using the admin account (password being ‘admin’ too). To get the port to use for ssh run the following command to get the port to use:

$ sudo docker port suri-buildbot
22/tcp -> 0.0.0.0:49156
8010/tcp -> 0.0.0.0:8010

and then connect:

ssh admin@localhost -p 49156

This can be used to install new dependencies inside the container. For instance if you are introducing a new library in Suricata, you may have to install the library in the docker instance.

Customizing the Docker image

On Docker side, the build recipes is available from GitHub. Feel free to modify it or propose updates and fixes.

0

Stamus Networks is proud to announce the availability of SELKS 1.2 stable release. SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

New features:

  • Suricata 2.1beta3  – Lua support for Stats output and Modbus parsing and matching as additional main features
  • Scirius 1.0-rc2 rule manager
  • Elasticsearch 1.4.3  – upgrade from 1.1.2
  • New Desktop icons – easy access to Dashboards and Scirius
  • Conkya free, light-weight system monitor for X, that displays any information on your desktop.”

 

system-status-scirius

Desktop-SELKS1.2

Desktop icons and Conky

You can download SELKS 1.2 from Stamus Networks’ open source page. Happy users of SELKS 1.1 can upgrade to SELKS 1.2 by using the traditional apt-get update && apt-get dist-upgrade. Please note that default login/password for HTTPS access (Dashboards or Scirius icons) is selks-user/selks-user.

NOTE – Elasticsearch upgrade for SELKS

If you were running Elasticsearch 1.1.2 with SELKS 1.1 this is the way to upgrade to Elasticsearch 1.4.3:

make sure your /etc/apt/sources.list.d/elasticsearch.list  looks like so

root@SELKS:~# cat /etc/apt/sources.list.d/elasticsearch.list
deb http://packages.elasticsearch.org/elasticsearch/1.4/debian stable main
deb http://packages.elasticsearch.org/logstash/1.4/debian stable main

then run

apt-get update && apt-get dist-upgrade

Please make sure you consider some testing/verification for ES in a QA/test environment before doing the upgrade in the production environment.

Download SELKS 1.2

More information: Howto and README

Follow us on Twitter, Google+ and Github

Get help at Freenode IRC on the #SELKS channel and/or  Google Mailing list.

Stamus Networks is proud to announce the availability of the second release candidate of Scirius 1.0. This new release features bugfixes and improvements.

On the bugfix side, the main one is a fix in the display of graph that could fail on fresh install due to a problem in the Elasticsearch request
Screenshot from 2015-02-12 18:10:59

The system status has been improved to feature a warning phase on disk and memory usage.

The only new feature is the System Settings menu:

Screenshot from 2015-02-12 18:11:15

For now, it allows the administrator to setup two things:

  • HTTP proxy parameters: if activated and setup it will allow scirius to fetch rules updates using the specified proxy
  • Elasticsearch usage: some people are using scirius without Elasticsearch so displaying empty graph is not interesting for them. By unchecking elasticsearch, the graphs based on elasticsearch information are not displayed anymore.

Screenshot from 2015-02-12 18:05:54

You can download Scirius 1.0-rc2 from Github. SELKS users can upgrade to this release by doing apt-get update && apt-get dist-upgrade.

Stamus Networks is proud to announce the availability of version 1.0-rc1 of Scirius, our web interface for Suricata ruleset management. This new release is first 1.0 release candidate. You can download it from Github download page.

It features a lot of bug fixes and improvements over the previous (beta) release. Among the new features, Scirius is now displaying a system status in the left sidebar.

system-status-scirius

It displays :

  • Status of the Elasticsearch cluster (in SELKS and if setup).
  • Status of Suricata.
  • Memory usage: alerting if swap is used.
  • Disk status: alerting if disk is filled in.

An other important improvement is the support of flowbit, scirius now disables all rules sharing a flowbit if one is disabled. This helps preventing entering is some weird state where an incomplete set of rules could trigger a lot of events.

Last but not least, the copyright has been updated with a new year inside. Happy new year 2015 from Stamus Networks team.

SELKS user can upgrade to Scirius 1.0-rc1 via apt-get update && apt-get dist-upgrade.