Our Blog

0

After a very valuable round of testing and feedback from the community  we are pleased to announce the SELKS 4 RC1 availability.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a the release candidate of a new major branch with an updated storage visualization stack and latest Suricata.

New Features

  • Suricata IDS/IPS/NSM 4.0.x – latest git master Suricata packaged with Hyperscan enabled for extra performance boost. This edition of Suricata besides many improvements and bug fixes also includes extra alert data like for example http body added to the alert json logs wherever available.
  • Elasticsearch 5.5.0  – part of the ELK5 stack upgrade making available a ton of new features and enhancements.
  • Logstash 5.5.0 – performance improvement over 2.x and ES5 compatibility.
  • Kibana 5.5.0 – taking advantage of the latest dashboarding features of ES.
  • Scirius 1.2.2 – bugfixes, better correlation capability with EveBox and introduction of IPS rules support.
  • Evebox – many new features including reporting and comments on the log events.
  • Debian Stretch – All new features, kernel and tools.

EveBox

Alert event with a comment field.

Kibana

Verbose HTTP logging

Kibana

GeoIP heat maps

EveBox

Supplemental alert data logging

 

Download

To download SELKS4-RC1:

Usage

Usage and logon credentials (OS and web management user)

  • user: selks-user
  • password: selks-user (password in Live mode is live)

The default root password is StamusNetworks

To remotely access the web management interface :

  • https://your.selks.IP.here/ – Scirius ruleset management and a central point for all dashboards and EveBox alert and event management.

Howto

Upgrade

To upgrade your existing SELKS 3 to SELKS 4 preview, please refer to SELKS-3.0-to-SELKS-4.0-upgrades wiki page.

It is recommended to follow the onscreen instructions and if needed answer “yes” to all changes. At the end of the upgrade you will be asked to enter the interface that you will use for IDS/sniffing. Please enter (eth0 for example) the interface name and reboot when the script is done.

Create your own ISO

To create your own SELKS 4 preview ISO (if your host OS is Jessie):

git clone https://github.com/StamusNetworks/SELKS.git
git checkout SELKS4-dev
./install-deps.sh
cd /usr/share/live/build/data/debian-cd/ && ln -s squeeze stretch
./build-debian-live.sh

It will take probably 30-40 min and you should end up with the SELKS.iso under the Stamus-Live-Build folder.

Once installed/upgraded
  • Please feel free to choose the IDS sniffing/listening interface either via the desktop icon Setup-IDS-Interface or via the cmd calling /opt/selks/Scripts/Setup/setup-selks-ids-interface.sh
  • Any further upgrades are done via a wrapper script located in /opt/selks/Scripts/Setup/selks-upgrade_stamus.sh
  • Recommended set up for SELKS 4.0RC1 is 2CPUs 5-6Gb RAM
  • If you need to reset/reload all the dashboards  – you can do like so
    • In Scirius on the top left corner drop down menu select System Settings
    • click on the Kibana tab
    • choose Reset SN dashboards

Feedback is welcome

Give us feedback and get help on:

While this test upgrade/installation has been verified and tested and aims at upgrading your current SELKS 3.0 to  SELKS 4.0RC1 please make sure you try it in your test/QA set up first and give us any feedback.

Thank you!

Stamus Networks is proud to announce the availability of Scirius 1.2.0. This release of our Suricata ruleset management interface comes after 4 months of development bringing two new major features: rules transformations to manage IPS and users activity logging to ease collaboration.

Rules transformation

With rules transformations, Scirius can now manage Suricata in IPS mode but also add the filestore option to specific rules allowing the user to transform existing rules coming from feed in rules realizing file extraction.

A signature can be transformed per ruleset to a drop or reject rule as shown in the following capture:

The filestore transformation will trigger file extraction by Suricata in case of alert. This allows user to have file extraction without the need of cloning existing rules.

User activity logging

The second big new feature is user activity logging. It is now possible to comment actions. A team collaboring on the same Scirius can now comment actions such as disabling a rule or adding a threshold.

It is also possible to simply comment on a rule.

All these features are already available in Scirius Enterprise and Amsterdam and will be available in SELKS in the coming days.

Eric Leblond gave a talk entitled “The adventures of a Suricata in eBPF land” at netdev 1.2, the Technical Conference on Linux Networking. This talk reviewed Stamus Networks’ work in the field of bypass and showed how the eBPF technology can be used to implement this feature.

eBPF is a technology that extends the traditional Berkeley Packet Filter that you can for example use with tcpdump. For instance eBPF filter can be written in a subset of C and allows kernel and userspace to share data via maps that can be for example an array or hash table. This technology has been used to implement a kernel bypass in Suricata. The idea is that Suricata is asking the Linux kernel to stop sending  it (bypass) packets for particular flow once it has decided that no further inspection is needed to be done.

For detailed information on the subject, you can get the Slides of “Suricata and eBPF” or watch the video that is already available thanks to the great work of Netdev team:

0

Introduction

Stamus Networks was working on a new Suricata feature named bypass. It has just been merged into Suricata sources and will be part of the upcoming 3.2 release. Stamus team did initially present his work on Suricata bypass code at Netdev 1.1, the technical conference on Linux networking that took place in Sevilla in February 2016.

In most cases an attack is done at start of TCP session and generation of requests prior to attack is not common. Furthermore multiple requests are often not even possible on same TCP session. Suricata reassembles TCP sessions till a configurable size (stream.reassembly.depth in bytes). Once the limit is reached the stream is not analyzed.

Considering that Suricata is not really inspecting anymore the traffic, it could be interesting to stop receiving the packets of a flow which enter in that state. This is the main idea behind bypass.

The second one consist in doing the same with encrypted flows. Once Suricata sees a traffic is encrypted it stops inspecting it so it is possible to bypass the packets for these flows in the same way it is done for packets after stream depth.

In some cases, network traffic is mostly due to session we don’t really care about on the security side. This is for example the case of Netflix or Youtube traffic. This is why we have added the bypass keywords to Suricata rules language. A user can now write a signature using this keyword and all packets for the matching flow will be bypassed. For instance to bypass all traffic to Stamus Networks website, one can use:

alert http any any -> any any (msg="Stamus is good"; content:"www.stamus-networks.com"; http_host; bypass; sid:1; rev:1;)

This is for sure just an example and as you may have seen our website is served only on HTTPS protocol.

Currently, Netfilter IPS mode is the only capture supporting the bypass. Stamus team represented by Eric Leblond will be at Netdev 1.2, first week of October 2016, to present an implementation of bypass for the Linux AF_PACKET capture method based on extended Berkeley Packet Filter.

And if you can’t make it to Japan, you will have another chance to hear about that during suricon, the Suricata user conference that will take place in Washington DC beginning of November.

Suricata bypass concepts

Suricata bypass technics

Suricata is now implementing two bypass methods:

  • A suricata only bypass called local bypass
  • A capture handled bypass called capture bypass

The idea is simply to stop treating packets of a flow that we don’t want to inspect anymore as fast as possible. Local bypass is doing it internally and capture bypass is using the capture method to do so.

Test with iperf on localhost with a MTU of 1500:

  • standard IPS mode: 669Mbps
  • IPS with local bypass: 899Mbps
  • IPS with NFQ bypass: 39 Gbps
Local bypass

The concept of local bypass is simple: Suricata reads a packet, decodes it, checks it in the flow table. If the corresponding flow is local bypassed then it simply skips all streaming, detection and output and the packet goes directly out in IDS mode and to verdict in IPS mode.

Once a flow has been local bypassed it is applied a specific timeout strategy. Idea is that we can’t handle cleanly the end of the flow as we are not doing the streaming reassembly anymore. So Suricata can just timeout the flow when seeing no packets. As the flow is supposed to be really alive we can set a timeout which is shorter than the established timeout. That’s why the default value is equal to the emergency established timeout value.

Capture bypass

In capture bypass, when Suricata decides to bypass it calls a function provided by the capture method to declare the bypass in the capture. For NFQ this is a simple mark that will be used by the ruleset. For AF_PACKET this will be a call to add an element in an eBPF hash table stored in kernel.

If the call to capture bypass is successful, then we set a short timeout on the flow to let time of already queued packets to get out of suricata without creating a new entry and once timeout is reached we remove the flow from the table and log the entry.

If the call to capture bypass is not successful then we switch to local bypass.

The difference between local and capture bypass

When Suricata is used with capture methods that do not offer the bypass functionality of eBPF/NFQ mark – pcap, netmap, pfring – it will switch to local bypass mode as explained above. Bypass is available for Suricata’s IDS/IPS and NSM modes alike.

Handling capture bypass failure

Due to misconfiguration or to other unknown problems it is possible that a capture bypassed flow is sending us packets. In that case, suricata is switching back the flow to local bypass so we handle it more correctly.

0

Yes, we did it: the most awaited SELKS 3.0 is out. This is the first stable release of this new branch that brings you the latest Suricata and Elastic stack technology.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

Suricata page in Scirius

Suricata page in Scirius

Main changes and new features

Suricata 3.1.1

SELKS 3.0 comes with latest Suricata namely 3.1.1 bringing a big performance boost as well as some new IDS and NSM capabilities.

Elasticsearch 2.x and Kibana 4

But the main change in SELKS 3.0 is the switch to the latest generation of the Elastic stack. On user side this means Kibana 3 has been replaced by Kibana 4. And this really means a lot. Kibana 4 is a complete rewrite of Kibana 3 being non backward compatible on data side. So, our team had to redo from scratch all dashboards and visualizations. The result is a new set of 11 ready-to-use dashboards and a lots of visualizations that you can use to build your own dashboards.

Kibana Alert dashboard

Kibana Alert dashboard

correlate-alerts

Complete flow and rule correlation view of an alert

Latest Scirius Community Edition

On the ruleset management side, SELKS 3.0 comes with Scirius Community Edition 1.1.10 that has support for advanced Suricata feature like xbits.

Thresholding

Suppression with Scirius

Thresholding-1

Threshold and suppress ruleset view with Scirius

Thresholding-2

Thresholding with Scirius

Scirius CE also brings thresholding and suppression support as well as an integrated backup system which allows for back up to be done (besides locally) in locations such as :

  • FTP
  • Amazon AWS
  • Dropbox
Evebox

SELKS 3.0 comes with Evebox an alert management/viewer/report interface for Suricata that presents events as a mailbox to provide classification via acknowledgement and escalade.

Mailbox view in Evebox

Mailbox view in Evebox

One of the other interesting features of Evebox is the capability to create and export pcap generated from events:

Pcap-1

Payload pcap generation (Evebox)

Pcap-2

Payload pcap generation (Evebox)

Features list

  • Suricata IDS/IPS/NSM  – Suricata 3.1.1 packaged.
  • Elasticsearch 2.3.5  – latest available ES edition featuring speed, scalability, security improvements and more.
  • Logstash 2.3.4 – performance improvement ES 2.3 compatability, dynamically reload pipelines on the fly and more
  • Kibana 4.5.4 – taking advantage of the latest features and performance improvement of ES
  • Scirius 1.1.10 – support for xbits, hostbits, thresholding, suppression, backup and more
  • Evebox – alert management/viewer/report interface for Suricata/ES  allowing easy export of payload/packets into pcaps
  • 4.4.x longterm kernel – SELKS 3.0 comes by default with 4.4.16 kernel.
  • Dashboards – reworked dashboards with flow and rule correlation capability.

SELKS comes with 11 ready to use Kibana dashboards. More than 190 visualizations are available to mix, match, customize and make your own dashboards as well.

Please feel free to try it out, spread the word, feedback and let’s talk about SELKS 3.0.

To get you started

Once downloaded and installed, you can get access to all components via https://your.selks.IP.here/

The default user and password for both web interface and system is:

  • user: selks-user
  • password: selks-user

The default root password is StamusNetworks.

Please note that in Live mode the password for the selks-user system user is live.

Upgrades

There is no direct upgrade path from SELKS 2.0 to SELKS 3.0 due to a number of breaking and compatibility changes in Elasticsearch 1.x to 2.x and Kibana 3.x to 4.x. The only proposed upgrade path is SELKS 3.0RC1 upgrade to SELKS 3.0

More about SELKS 3.0

Stamus Networks is proud to announce the availability of version 1.0, nicknamed “glace à la vanille”, of Amsterdam, our container based ready to use Suricata IDS. Amsterdam is a fully web managed software appliance that is using Docker to provide:

  • Network Intrusion Detection and Network Security Monitoring via Suricata
  • Log storage and analysis via the Elastic stack: latest Logstash, Elasticsearch and Kibana are part of the Amsterdam
  • Suricata ruleset management and basic reporting via Scirius our web interface
  • Alerts listing and acknowledgement via Evebox

Scirius homepage

Each component is running in its own container and Amsterdam is using by default the official image on Docker Hub. This guarantees you fast update and heavily tested software. The orchestration of the different containers is done via Docker compose but all the details are hidden to you and Amsterdam should be your only interface in daily usage.

Installation is just a few commands:

pip install amsterdam
amsterdam -d ams -i wlan0 setup
amsterdam -d ams start

Once every containers are running, you can simply point your browser to https://localhost/ to start analyzing the traffic and fine tune the system. Kibana is coming with a set of predefined dashboards so you don’t have to build your own before starting to work.

Kibana Alert dashboard

Amsterdam offers you really easy upgrade via integrated commands:

amsterdam -d ams upgrade
amsterdam -d ams restart

Amsterdam is multi instances. For example, let’s say you have two customers where you analyzed the traffic when on site. You can set up two instances:

amsterdam -d customer1 -i wlan0 setup
amsterdam -d customer2 -i eth0 setup

and start the first one when at customer 1

amsterdam -d customer1 start

and second one when at customer 2

amsterdam -d customer2 start

The two different instances are not sharing any data, so you can freely show the interface to any of the customer if running the good instance. All data and configuration files are in customer1 directory for first customer and customer2 for the second one.

Amsterdam can digest any JSON formatted data. For that is is enough to copy a file to analyzed in the suricata directory inside the instance:

cp /path/to/passwords.json customer1/suricata/

This method makes it really easy to combine different sources of information into Kibana dashboards:
Pshitt and Suricata information

Amsterdam is also really easy to tune. The configuration files are stored for each components in the config directory so you can easily update Suricata, Logstash or Nginx configuration.

Stamus Networks is really excited by this first stable release of Amsterdam and we think that it has never been so easy to sniff and understand your network.

This release is dedicated to the memory of Edith Leblond.

0

After some hard team work, Stamus Networks is proud to announce the availability of SELKS 3.0RC1.

SELKS is both Live and installable Network Security Management ISO based on Debian implementing and focusing on a complete and ready to use Suricata IDS/IPS ecosystem with its own graphic rule manager. Stamus Networks is a proud member of the Open Source community and SELKS is released under GPLv3 license.

This is a the release candidate of a new major branch with an updated storage visualization stack and latest Suricata.

New Features

  • Suricata IDS/IPS/NSM 3.0.x – latest git master suricata packaged.
  • Elasticsearch 2.3  – latest available ES edition featuring speed, scalability, security improvements and more.
  • Logstash 2.3 – performance improvement ES 2.3 compatability, dynamically reload pipelines on the fly and more
  • Kibana 4.5 – taking advantage of the latest features and performance improvement of ES
  • Scirius 1.1.6 – support for xbits, hostbits, thresholding, suppression, backup and more
  • Evebox – alert management/viewer interface for Suricata/ES  allowing easy export of payload into pcaps

SELKS comes with 11 ready to use Kibana dashboards using more than 190 visualisations.

Please feel free to try it out, spread the word, feedback and let’s talk about SELKS 3.0.

Thresholding-2

Thresholding with Scirius

Thresholding

Suppression with Scirius

Thresholding-1

Threshold and suppress ruleset view with Scirius

 

Pcap-1

Payload pcap generation (Evebox)

Pcap-2

Payload pcap generation (Evebox)

 

Dashboard-3

Dashboards

Dashboard-1

Dashboards

 

 

 

To get you started (the download link is below this paragraph):

Once installed in order to upgrade all components follow the guide here.

Usage and logon credentials (OS user)  – user: selks-user, password: selks-user (password in Live mode is live). The default root password is – StamusNetworks

Upon log in double click the Scirius icon on the desktop. Credentials are  – user: selks-user, password: selks-user. In the left upper corner click the drop down menu and choose “ALL” dashboards. Choose default index(click on logstash-* and then the green star) as depicted below. Then choose “Dashboards” and choose your desired dashboards from the 11 available.

enable-index-kibana

 

More about SELKS 3.0RC1

Stamus Networks is proud to announce the availability of Scirius 1.1.6. This new release brings interesting new features and a lot of bugfixes to our Suricata ruleset manager.

Rule page in scirius 1.1.6

The main new features in release are:

  • Backup support
  • Threshold support
  • Xbits and hostbits support
  • Down detection of scirius
  • Top src and destination in rule page
  • Fix of test system that takes Suricata local config into account

The backup system adds a set of new commands to manage.py to backup and restore completely a Scirius instance. scbackup will do a backup and screstore will erase everything and restore latest backup. Backup can be done locally but it is also possible to use FTP, Dropbox or Amazon AWS to store and fetch backups.

On the usability feature side the most important is the support of thresholding. Scirius is now managing a threshold.config that is used by Suricata to limit or suppress alert(s) for a signature under certain conditions. Easiest way to access this feature is to start from a rule page and look at new top source and destination tables:

Top src and dest IP for a signature

The arrow down and the cross can be clicked to trigger edition of a form for a threshold (limit) or a suppression. For instance if you click on the cross, you will get something like:
Suppression
If there is already a suppression activated for the network/IP, you get a warning:
Adding a suppression

Latest ruleset management feature is the handling of the new xbits and hostbits. When a rule is disable, all the rule sharing a flowbits, a xbits or a hostbits are also deactivated.

At last, browser is now detecting that Scirius is down allowing you to avoid to navigate away from a form you were editing till connection is not restored:
Scirius down

Scirius 1.1.6 may be a minor release for the number in term of features it adds a lots of things users were asking for. You can already get scirius 1.1.6 in latest amsterdam. And it will be part of SELKS 3.0 that will be available really soon.

0

Suricata stats in EVE JSON format

Suricata 3.0 will come with a lot of improvements on the output side. One of them is the ability to output Suricata internal statistics in the EVE JSON format.

Stats event in EVE JSON format

This functionality is already used by scirius to display statistics graphs of the Suricata running in SELKS, Amsterdam or Stamus Networks’ appliances:

Stats in scirius

This statistic sometimes help to visualize the impact of configuration change. For example, in the next screenshot the generic receive offloading on the capture interface has been disable at 23:33:

Impact of iface offloading

Impact is cristal clear as the counter of invalid decoding did stop increasing.

Using Kibana Timelion plugin

Amsterdam came with Kibana 4 and the Timelion plugin is preinstalled. Timelion is a plugin providing a new interface and language to graph timeline.

As Suricata stats data are fed into Elasticsearch, we can use it to graph Suricata performance data.

For example to graph DNS and HTTP memory usage, one can use the following syntax:

.es(metric=’avg:stats.dns.memuse’).label(‘DNS’) .es(metric=’avg:stats.http.memuse’).label(‘HTTP’)

Result is the following graph:
Screenshot from 2016-01-07 11-01-48

If you have a counter and want to graph rate, then you can use:

.es(metric=’avg:stats.capture.kernel_packets’).derivative().label(‘PPS’) .es(metric=’avg:stats.capture.kernel_drops’).derivative().label(‘Drops’)

And you get the following graph:

Screenshot from 2016-01-07 10-59-01

One interesting thing with Timelion is that you can use Lucene query to get a count of something really easily. For example to get a view on the rate of different event type, one can use:

.es(q=’event_type:http’) .es(q=’event_type:tls’) .es(q=’event_type:dns’)

Rate of different event types

Both method can be mixed so, if you have different probes (let’s say probe-1 and probe-2) you can do something like:

.es(q=’host.raw:”probe-1″‘, metric=’avg:stats.dns.memuse’).label(‘Probe-1 DNS’) .es(q=’host.raw:”probe-2″‘, metric=’avg:stats.dns.memuse’).label(‘Probe-2 DNS’)

Conclusion

The new Suricata statistic output is really improving the information we can use when doing performance analysis. Combined with timelion, we get a really easy and powerful solution. If you want to give a try to all these technologies one of the easiest way is to use Amsterdam which comes with latest Suricata and a pre installed timelion.

Stamus Networks is proud to announce the availability of the first technology preview of Amsterdam.

Amsterdam is a ready-to-use Suricata IDS/NSM based system running on Docker. It features the same components as SELKS our live and installable Suricata distibution. So by running Amsterdam you get:

  • Suricata: latest version of the IDS/NSM engine
  • Elasticsearch: the powerful search engine
  • Logstash: the data pipeline application injecting Suricata events in the database
  • Kibana: Version 4 of the famous dashboard interface
  • Scirius: Stamus Networks’ Suricata ruleset management interface

Each component runs in a separate container and Amsterdam is using the official images for ELK stack to provide you an always up-to-date experience. Amsterdam is available under GPLv3 license.

Using Amsterdam is really simple. Once installed, you need to setup a directory that will contain data and configuration. For example to create and use a directory named ams-wlan0 and sniff the wlan0 interface on host, one can run once:

amsterdam -d ams-wlan0 -i wlan0 setup

You can then start Amsterdam by running:

amsterdam -d ams-wlan0 start

Scirius running in Amsterdam

Once all containers are built and started, you can point your browser to http://localhost:8000 to get access to the management interface.

It is possible to run multiple instances of Amsterdam on the same system by using different data directories. Each data directory contains the configuration files of the components so you can easily tune your installation.

More information and source code available on Github.