Our Blog

Stamus Networks is proud to announce the availability of version 1.0, nicknamed “glace à la vanille”, of Amsterdam, our container based ready to use Suricata IDS. Amsterdam is a fully web managed software appliance that is using Docker to provide:

  • Network Intrusion Detection and Network Security Monitoring via Suricata
  • Log storage and analysis via the Elastic stack: latest Logstash, Elasticsearch and Kibana are part of the Amsterdam
  • Suricata ruleset management and basic reporting via Scirius our web interface
  • Alerts listing and acknowledgement via Evebox

Scirius homepage

Each component is running in its own container and Amsterdam is using by default the official image on Docker Hub. This guarantees you fast update and heavily tested software. The orchestration of the different containers is done via Docker compose but all the details are hidden to you and Amsterdam should be your only interface in daily usage.

Installation is just a few commands:

pip install amsterdam
amsterdam -d ams -i wlan0 setup
amsterdam -d ams start

Once every containers are running, you can simply point your browser to https://localhost/ to start analyzing the traffic and fine tune the system. Kibana is coming with a set of predefined dashboards so you don’t have to build your own before starting to work.

Kibana Alert dashboard

Amsterdam offers you really easy upgrade via integrated commands:

amsterdam -d ams upgrade
amsterdam -d ams restart

Amsterdam is multi instances. For example, let’s say you have two customers where you analyzed the traffic when on site. You can set up two instances:

amsterdam -d customer1 -i wlan0 setup
amsterdam -d customer2 -i eth0 setup

and start the first one when at customer 1

amsterdam -d customer1 start

and second one when at customer 2

amsterdam -d customer2 start

The two different instances are not sharing any data, so you can freely show the interface to any of the customer if running the good instance. All data and configuration files are in customer1 directory for first customer and customer2 for the second one.

Amsterdam can digest any JSON formatted data. For that is is enough to copy a file to analyzed in the suricata directory inside the instance:

cp /path/to/passwords.json customer1/suricata/

This method makes it really easy to combine different sources of information into Kibana dashboards:
Pshitt and Suricata information

Amsterdam is also really easy to tune. The configuration files are stored for each components in the config directory so you can easily update Suricata, Logstash or Nginx configuration.

Stamus Networks is really excited by this first stable release of Amsterdam and we think that it has never been so easy to sniff and understand your network.

This release is dedicated to the memory of Edith Leblond.

0

After some hard team work, Stamus Networks is proud to announce the availability of SELKS 3.0RC1.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a the release candidate of a new major branch with an updated storage visualization stack and latest Suricata.

New Features

  • Suricata IDS/IPS/NSM 3.0.x – latest git master suricata packaged.
  • Elasticsearch 2.3  – latest available ES edition featuring speed, scalability, security improvements and more.
  • Logstash 2.3 – performance improvement ES 2.3 compatability, dynamically reload pipelines on the fly and more
  • Kibana 4.5 – taking advantage of the latest features and performance improvement of ES
  • Scirius 1.1.6 – support for xbits, hostbits, thresholding, suppression, backup and more
  • Evebox – alert management/viewer interface for Suricata/ES  allowing easy export of payload into pcaps

SELKS comes with 11 ready to use Kibana dashboards using more than 190 visualisations.

Please feel free to try it out, spread the word, feedback and let’s talk about SELKS 3.0.

Thresholding-2

Thresholding with Scirius

Thresholding

Suppression with Scirius

Thresholding-1

Threshold and suppress ruleset view with Scirius

 

Pcap-1

Payload pcap generation (Evebox)

Pcap-2

Payload pcap generation (Evebox)

 

Dashboard-3

Dashboards

Dashboard-1

Dashboards

 

 

 

To get you started (the download link is below this paragraph):

Once installed in order to upgrade all components follow the guide here.

Usage and logon credentials (OS user)  – user: selks-user, password: selks-user (password in Live mode is live). The default root password is – StamusNetworks

Upon log in double click the Scirius icon on the desktop. Credentials are  – user: selks-user, password: selks-user. In the left upper corner click the drop down menu and choose “ALL” dashboards. Choose default index(click on logstash-* and then the green star) as depicted below. Then choose “Dashboards” and choose your desired dashboards from the 11 available.

enable-index-kibana

 

More about SELKS 3.0RC1

Stamus Networks is proud to announce the availability of Scirius 1.1.6. This new release brings interesting new features and a lot of bugfixes to our Suricata ruleset manager.

Rule page in scirius 1.1.6

The main new features in release are:

  • Backup support
  • Threshold support
  • Xbits and hostbits support
  • Down detection of scirius
  • Top src and destination in rule page
  • Fix of test system that takes Suricata local config into account

The backup system adds a set of new commands to manage.py to backup and restore completely a Scirius instance. scbackup will do a backup and screstore will erase everything and restore latest backup. Backup can be done locally but it is also possible to use FTP, Dropbox or Amazon AWS to store and fetch backups.

On the usability feature side the most important is the support of thresholding. Scirius is now managing a threshold.config that is used by Suricata to limit or suppress alert(s) for a signature under certain conditions. Easiest way to access this feature is to start from a rule page and look at new top source and destination tables:

Top src and dest IP for a signature

The arrow down and the cross can be clicked to trigger edition of a form for a threshold (limit) or a suppression. For instance if you click on the cross, you will get something like:
Suppression
If there is already a suppression activated for the network/IP, you get a warning:
Adding a suppression

Latest ruleset management feature is the handling of the new xbits and hostbits. When a rule is disable, all the rule sharing a flowbits, a xbits or a hostbits are also deactivated.

At last, browser is now detecting that Scirius is down allowing you to avoid to navigate away from a form you were editing till connection is not restored:
Scirius down

Scirius 1.1.6 may be a minor release for the number in term of features it adds a lots of things users were asking for. You can already get scirius 1.1.6 in latest amsterdam. And it will be part of SELKS 3.0 that will be available really soon.

0

Suricata stats in EVE JSON format

Suricata 3.0 will come with a lot of improvements on the output side. One of them is the ability to output Suricata internal statistics in the EVE JSON format.

Stats event in EVE JSON format

This functionality is already used by scirius to display statistics graphs of the Suricata running in SELKS, Amsterdam or Stamus Networks’ appliances:

Stats in scirius

This statistic sometimes help to visualize the impact of configuration change. For example, in the next screenshot the generic receive offloading on the capture interface has been disable at 23:33:

Impact of iface offloading

Impact is cristal clear as the counter of invalid decoding did stop increasing.

Using Kibana Timelion plugin

Amsterdam came with Kibana 4 and the Timelion plugin is preinstalled. Timelion is a plugin providing a new interface and language to graph timeline.

As Suricata stats data are fed into Elasticsearch, we can use it to graph Suricata performance data.

For example to graph DNS and HTTP memory usage, one can use the following syntax:

.es(metric=’avg:stats.dns.memuse’).label(‘DNS’) .es(metric=’avg:stats.http.memuse’).label(‘HTTP’)

Result is the following graph:
Screenshot from 2016-01-07 11-01-48

If you have a counter and want to graph rate, then you can use:

.es(metric=’avg:stats.capture.kernel_packets’).derivative().label(‘PPS’) .es(metric=’avg:stats.capture.kernel_drops’).derivative().label(‘Drops’)

And you get the following graph:

Screenshot from 2016-01-07 10-59-01

One interesting thing with Timelion is that you can use Lucene query to get a count of something really easily. For example to get a view on the rate of different event type, one can use:

.es(q=’event_type:http’) .es(q=’event_type:tls’) .es(q=’event_type:dns’)

Rate of different event types

Both method can be mixed so, if you have different probes (let’s say probe-1 and probe-2) you can do something like:

.es(q=’host.raw:”probe-1″‘, metric=’avg:stats.dns.memuse’).label(‘Probe-1 DNS’) .es(q=’host.raw:”probe-2″‘, metric=’avg:stats.dns.memuse’).label(‘Probe-2 DNS’)

Conclusion

The new Suricata statistic output is really improving the information we can use when doing performance analysis. Combined with timelion, we get a really easy and powerful solution. If you want to give a try to all these technologies one of the easiest way is to use Amsterdam which comes with latest Suricata and a pre installed timelion.

Stamus Networks is proud to announce the availability of the first technology preview of Amsterdam.

Amsterdam is a ready-to-use Suricata IDS/NSM based system running on Docker. It features the same components as SELKS our live and installable Suricata distibution. So by running Amsterdam you get:

  • Suricata: latest version of the IDS/NSM engine
  • Elasticsearch: the powerful search engine
  • Logstash: the data pipeline application injecting Suricata events in the database
  • Kibana: Version 4 of the famous dashboard interface
  • Scirius: Stamus Networks’ Suricata ruleset management interface

Each component runs in a separate container and Amsterdam is using the official images for ELK stack to provide you an always up-to-date experience. Amsterdam is available under GPLv3 license.

Using Amsterdam is really simple. Once installed, you need to setup a directory that will contain data and configuration. For example to create and use a directory named ams-wlan0 and sniff the wlan0 interface on host, one can run once:

amsterdam -d ams-wlan0 -i wlan0 setup

You can then start Amsterdam by running:

amsterdam -d ams-wlan0 start

Scirius running in Amsterdam

Once all containers are built and started, you can point your browser to http://localhost:8000 to get access to the management interface.

It is possible to run multiple instances of Amsterdam on the same system by using different data directories. Each data directory contains the configuration files of the components so you can easily tune your installation.

More information and source code available on Github.

0

 

Stamus Networks is proud to announce the availability of SELKS 2.1  release.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a major SELKS upgrade.

New Features

  • Elasticsearch 1.7  – upgrade from 1.5 (security fixes and faster recovery after restart)
  • Scirius 1.1 – upgrade from 1.0 (suricata and logstash performance stats)
  • Logstash 1.5.4 – upgrade from 1.4 (performance improvement
    in JSON handling and better security)

 

Some screenshot examples

Source addition page

Suricata memory usageElasticsearch and Logstash information

 

UPGRADE from SELKS 2.0

For those that use SELKS 2.0 and would like to do an in place upgrade to SELKS 2.1 you can follow THIS GUIDE.

NOTE: Please make sure that you test the upgrade in your test/QA environment first before doing it on your production systems.

Please note that default login/password for HTTPS access (Dashboards or Scirius icons) is selks-user/selks-user.

More about SELKS 2.1

Stamus Networks team is proud to announce the availability of Scirius 1.1. This new release brings a ton of new features:

  • Easier enabling/disabling of rules and categories
  • Delete events from Elasticsearch corresponding to all alerts of a given rule
  • Compatibility with Kibana 4
  • Add a graphic presenting Logstash insertion speed
  • Graphics with Suricata performance indicators
  • Greatly improved source addition
  • Preliminary support for Sourcefire ruleset
  • Validity checking of ruleset, imported sources and rules

One of the main new feature is the validity checking system. Scirius is now able to test whether a rule, a source or a ruleset will be valid for Suricata. This is not a simulated test as Scirius is using the configuration testing capability of Suricata to get the information.

One of the updated page is the the rule details page. It does now contain the validity of this rule respectively to the existing rulesets:
Validity check

In our example, the rule is invalid for first ruleset because the source containing the lua script is not active in the ruleset. In the second ruleset, the rules is not valid because the Suricata on the system is not supporting the not-yet-official(as of the moment of this writing) TLS extension for lua scripting. As you can see, the validity checking system is much more than a simple parsing and gives you a real view on the validity of rules on your own system. This system will work with any Suricata but it will give accurate information if the system is hosting the latest Suricata. This is due to the fact that Scirius is parsing the JSON console output of Suricata which has been added recently to the Suricata git master. Our team has packaged the latest Suricata and made it available in Stamus Networks repository to the users to give them the best of this new feature.

This testing system has been used to revamp the Source addition. Adding a source to an existing ruleset was complicated and involving multiple steps. Now, user can select to which rulesets a new source should be added when creating the source. So there is now basically one step:
Source addition form
The validity checking system will warn the user about a possible problem during the source creation:
Source addition page
As some rulesets may contain invalid signatures, user has the capability to choose whether or not he/she wants to ignore the detected problems.

The second main change in Scirius is its ability to graph some performance indicators. It is using the logstash metrics capability to get statistics on the insertion speed. This is a good indicator of the load of your logging system.

Elasticsearch and Logstash information

The second set of graphics is building performance indicators for Suricata. The Suricata page of Scirius is now able to graph:

  • Capture stats
  • Memory usage
  • Problem indicators

If capture stats is showing the usual drop and accept, the two others tabs are more interesting as they are showing the memory consumption of Suricata subsystem succh as TCP, Flow, DNS and HTTP protocols:
Suricata memory usage
The third tab is showing some Suricata indicators that can help to diagnose problem. All these indicators are extracted from Suricata statistics output in JSON format that will be part of Suricata 2.1.

Happy SELKS users can simply run apt-get update && apt-get dist-upgrade to get new version. Others can get it from Github.

0

Introduction

This is a short tutorial of how you can find and store to disk a self signed TLS certificates with Suricata IDPS and Lua (luajit scripting).

What does self signed TLS certificate mean – the quick version  from Wikipedia here. In other words – “certificate that is signed by the same entity whose identity it certifies” or anyone can create and deploy such a certificate. This kind of events are signed of some poorly setup TLS servers and that’s why it is good to keep an eye on such events in your network for the purpose of contingency monitoring – the very least.

TLS support in Suricata allows you to do match on TLS subject, TLS issuer DN and things like TLS fingerprint. For instance, one can do

alert tls any any -> any any (msg:"forged ssl google user";
            tls.subject:"CN=*.googleusercontent.com";
            tls.issuerdn:!"CN=Google-Internet-Authority"; sid:8; rev:1;)

to detect that the TLS issuer DN is not the one we were waiting for a given TLS subject. But there is no way to compare the two different fields. So how do we catch all those self signed certificates without knowing any details about them and/or any network/domain/port specifics either. And we want them all caught and stored to disk!

This case is one example where the Lua support in Suricata IDPS  shines – more than shines actually because it empowers you to do much more than chasing packet bytes with rule keywords and using PCREs inside a rule – and all those still deliver limited functionality in this particular scenario.

Lua and Suricata

Since version 2.0, Suricata has support for Lua scripting. The idea is to be able to decide if an alert is matching based on the return of a lua script. This script is taking some fields extracted by Suricata (the magic of it all) as parameters and return 1 in case of match and 0 if not. This lua scripting allows rules to implement complex logic that would be impossible with the standard rule language. For instance, Victor Julien was able to write a performant Heartbleed detection with lua scripting in the afternoon (the very same day) the problem/exploit was announced.

The syntax is the following, you prefilter with standard signature language, and you add a lua keyword with parameter being the script to run in case of partial match:

alert tls any any -> any any ( \
    msg:"TLS HEARTBLEED malformed heartbeat record"; \
    content:"|18 03|"; depth:2; lua:tls-heartbleed.lua; \
    classtype:misc-attack; sid:3000001; rev:1;)

For more information on Suricata Lua scripting, please read how to write luascripts for Suricata.

For self signed certificate detection, you need to write a script – shall we say “self-signed-cert.lua” and save it in your /etc/suricata/rules directory. Then you can use it in a rule like so –

alert tls any any -> any any (msg:"SURICATA TLS Self Signed Certificate"; \
  flow:established; luajit:self-signed-cert.lua; \
  tls.store; classtype:protocol-command-decode; sid:999666111; rev:1;)

Now let us explain that in a bit more detail below.

At the time of this writing we are using this branch in particular – TLS Lua rebased . This is going to be merged to the Suricata git (latest dev) soon and later into beta, stable editions.

You need to make sure Suricata is compiled with Lua enabled:

# suricata --build-info
This is Suricata version 2.1dev (rev b5e1df2)
...
Prelude support:                         no
PCRE jit:                                yes
LUA support:                             yes
libluajit:                               yes
libgeoip:                                yes
...

In suricata.yaml make sure the tls section is enabled:

# a line based log of TLS handshake parameters (no alerts)
- tls-store:
  enabled: yes  # Active TLS certificate store.
  certs-log-dir: certs # directory to store the certificates files

We have the following rule file (self-sign.rules) content located in /etc/suricata/rules/ :

alert tls any any -> any any (msg:"SURICATA TLS Self Signed Certificate"; \
  flow:established; luajit:self-signed-cert.lua; \
  tls.store; classtype:protocol-command-decode; sid:999666111; rev:1;)

Make sure you add the rule to your rules files loaded in suricata.yaml and that you copy the associated lua script in the same directory. Here you can find the self-signed-cert.lua script.

Then you can start Suricata IDPS the usual way you always do.

The active part of the lua script is the following:

function match(args)
    version, subject, issuer, fingerprint = TlsGetCertInfo();
    
    if subject == issuer then
        return 1
    else
        return 0
    end
end

When suricata will see a TLS handshake (regardless of IP/port), it will run the Lua script. This one uses the fact that an equality between subject and issuer DN constitutes most of the self-signed certificates. When it finds such an equality it will return 1 and an alert will be generated.

This script is showing a really basic but useful code. However you can use all the power of Lua with the given info to do whatever you want/need.

The result

self-cert-extract

Extracted self signed SSL certificate

Some meta data about the certificate (in /var/log/suricata/certs/ you will find the .meta and pem  files):

TIME:              07/14/2015-14:45:16.757001
SRC IP:            10.0.2.15
DST IP:            192.168.1.180
PROTO:             6
SRC PORT:          49966
DST PORT:          443
TLS SUBJECT:       C=FR, ST=IDF, L=Paris, O=Stamus, CN=SELKS
TLS ISSUERDN:      C=FR, ST=IDF, L=Paris, O=Stamus, CN=SELKS
TLS FINGERPRINT:   80:e7:af:49:c3:fe:9a:73:78:29:6b:dd:fd:28:9e:d9:c9:15:3e:18

 

Further more from the alert info in JSON format (/var/log/suricata/eve.json log) we also have extra TLS info for the generated alert :

{"timestamp":"2015-07-14T14:45:18.076794+0200","flow_id":137451536,"in_iface":"eth0",
"event_type":"alert","src_ip":"192.168.1.180","src_port":443,"dest_ip":"10.0.2.15","dest_port":49966,"proto":"TCP","alert":
{"action":"allowed","gid":1,"signature_id":999666111,"rev":1,"signature":"SURICATA TLS Self Signed Certificate","category":"Generic Protocol Command 
Decode","severity":3},"tls":{"subject":"C=FR, ST=IDF, L=Paris, O=Stamus, CN=SELKS","issuerdn":"C=FR, ST=IDF, L=Paris, O=Stamus, 
CN=SELKS","fingerprint":"80:e7:af:49:c3:fe:9a:73:78:29:6b:dd:fd:28:9e:d9:c9:15:3e:18","version":"TLS 1.2"}}

To get the extra TLS info in the JSON alert you need to enable it in the suricata.yaml like so:

  # Extensible Event Format (nicknamed EVE) event log in JSON format
  - eve-log:
      enabled: yes
      filetype: regular #regular|syslog|unix_dgram|unix_stream
      filename: eve.json
      types:
        - alert:
            #payload: yes           # enable dumping payload in Base64
            #payload-printable: yes # enable dumping payload in printable (lossy) format
            #packet: yes            # enable dumping of packet (without stream segments)
            http: yes              # enable dumping of http fields
            tls: yes               # enable dumping of tls fields <<---
            ssh: yes               # enable dumping of ssh fields

so you can get all the JSON data in Kibana/Elasticsearch as well.

Conclusion

If you would like to learn more about what can you do with Lua and Suricata IDPS, those links below will put you off to a good start:

Suricata EVE JSON format is becoming the de-facto standard for this IDS. All type of events are now exported to this format. The JSON format allows a nice handling of data in external tool like Elasticsearch or even DOM. The output is readable by human but as an event/record can contain a lot of data it can be difficult to do a by-eye analysis when looking at a file. The following screenshot give you an idea of the possible output:

Tailing EVE

Using standard unix tools like grep on the EVE JSON file is not the perfect idea. For example if you want to extract a field to get some statistics you may want to try using grep, cut or awk but you may find it painful. And it is worthed to mention here that JSON fields are not ordered.

Here to the rescue comes the jq utility. jq is a tool dedicated to the transformation/parsing of a JSON entry. It is Debian packaged, so a simple apt-get install jq is enough for the install.

Some jq examples

The most basic usage is to colorize the entry. To do that, just do something like

$ tail -n100 eve.json| jq '.'

The output is done the pretty way:
JQ displaying an event
To get a one line per event output, just add the -c flag to the command:
One line

To extract a single field from the JSON events, one can do:

$ jq '.src_ip' eve.json
"58.218.211.155"
"58.218.211.155"
"58.218.211.155"

The point to remember is that the point in .src_ip is a place holder for the current entry.

By default when a field is not present null is displayed in the output. To fix that, it is possible to filter the event to only get the one we are interested in. This is done via the select keyword. For instance to select the SSH events and extract the information about the client part one can do:

$ tail eve.json | jq -c 'select(.event_type == "ssh")|.ssh.client'
{"proto_version":"2.0","software_version":"PUTTY"}
{"proto_version":"2.0","software_version":"PUTTY"}

Far more things can be done with jq. Good starting points are the jq manual and wiki.

0

Stamus Networks is proud to announce the availability of SELKS 2.0  release.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a major SELKS upgrade.

New Features

  • Debian Jessie based  – With Debian Jessie  being released last week  – 25 April   SELKS makes the switch as well (from Debian Wheezy and 3.2 kernel). The new Debian release Jessie will enable SELKS for much better HW compatibility, new kernel 3.16 and all the performance improvements, features and benefits with it right out of the box.
  • Elasticsearch 1.5  – upgrade from 1.4
  • Scirius 1.0 – upgrade from 1.0rc3

 

Some screenshot examples

Scirius

SELKS2.0-Scirius-2

Screenshot from 2015-04-20 22:07:08

IDS/IPS dashboards
SELKS2.0-1

12 ready to use IDS/IPS dashboards

VLAN3

By VLAN break down File Transactions/SSH

VLAN2

By VLAN break down DNS/TLS

SMTP-Attachments

By file attachment breakdown SMTP

VLAN1

By VLAN break down Alerts/HTTP

UPGRADE from SELKS1.2

For those that use SELKS 1.2 and would like to do an in place upgrade to SELKS 2.0 you can follow THIS GUIDE.

NOTE: Please make sure that you test the upgrade in your test/QA environment first before doing it on your production systems.

Please note that default login/password for HTTPS access (Dashboards or Scirius icons) is selks-user/selks-user.

More about SELKS 2.0