“By failing to prepare, you are preparing to fail.” - Benjamin Franklin
When it comes to cybersecurity, preparedness is your most valuable asset. By regularly testing your defenses through interactive cybersecurity exercises, your organization can identify weaknesses, refine your incident response plans, and ensure your team is ready to act in the event of an attack. With two main cybersecurity exercises — life-fire and tabletop — choosing the right approach can be difficult.
In this blog post, we will explore the benefits and limitations of both types of exercise, give recommendations on how to choose which is right for your organization, and discuss how combining these methods can create a more complete training program for your security team.
What is a Live Fire Cybersecurity Exercise?
Have you ever faced a real cyber attack? Where your network is being assaulted and critical data is being exfiltrated? Live-fire cybersecurity exercises attempt to realistically simulate this scenario in a controlled environment, attacking your actual infrastructure and tools using modern tactics, techniques, and procedures, including live malware.
In a live fire exercise, the red team (attacking) would use simulated malware or other tactics, techniques, and procedures used by real threat actors to breach the defenses of the defending blue team, evade detection, and infiltrate the network. In these events, the red team would be adjusting their strategy to adapt to your defenses just as a real attacker would, creating a pressure cooker environment for participants.
Your team’s response is monitored in real-time, assessing their decision-making and execution of incident response plans in a high-pressure and time-constrained environment, much like the type they would experience during a real attack. The goal of this type of exercise is to identify vulnerabilities and gaps in your security posture.
The benefits of live fire exercises are undeniable:
- Stress-testing your incident response: Live fire exercises expose your team to real-time pressure, mimicking the urgent decision-making of a true attack. This allows them to practice their skills under pressure and identify areas for improvement in their response procedures.
- Uncovering hidden vulnerabilities: Through simulated attacks, live fire exercises can expose security weaknesses and configuration errors that may have gone unnoticed in traditional vulnerability assessments. This proactive approach allows you to patch these vulnerabilities before real attackers can exploit them.
- Validating incident response plans: Live fire provides a testing ground for your established incident response protocols, letting you identify any gaps or inefficiencies in your response process. This ensures your plan is truly effective when the time comes.
However, live-fire exercises aren't without potential drawbacks. The cost and complexity often required to set up and execute live fire exercises require significant resources. You might need specialized expertise or dedicated infrastructure to avoid disrupting your actual operations. Even if cost is not an issue, simulated attacks can still impact business activities by taking time away from your security team. Careful planning and scheduling is required to minimize disruptions.
What is a Tabletop Exercise?
While live fire cybersecurity exercises offer a highly interactive experience with simulated attacks, tabletop exercises function more as thought experiments. In this type of exercise, teams gather around a table (not always literally) to discuss hypothetical attack scenarios and strategize their response.
Tabletop cybersecurity exercises often involve moderator-led discussions, where diverse attack scenarios are presented to the group. Participants might not be limited only to the security team, as tabletop role-play is just as beneficial for business executives, PR teams, and customer support as it is to security practitioners. These exercises provide a structured way to examine existing incident response plans and business continuity strategies through open discussion and debate.
Tabletop exercises have several advantages:
- Cost-effective and scalable: Tabletop exercises require minimal resources and are easily adaptable for large teams, making them an efficient way to train and engage personnel across your organization.
- Fosters communication and collaboration: By encouraging open discussion and role-playing, tabletop exercises strengthen communication and teamwork skills between your security team and other members of your organization. This collaborative environment facilitates creative problem-solving and improves overall cyber resilience.
- Tests incident response plans in a safe environment: Tabletop exercises allow you to explore potential pitfalls and refine your response procedures without exposing your systems to real attacks or using your actual infrastructure. This Socratic approach minimizes risks while maximizing learning opportunities.
Of course, tabletop exercises also have limitations. The hypothetical nature of a tabletop exercise cannot fully replicate the pressure and uncertainty of a real attack. This could lead to unrealistic assumptions and responses that would not translate well to a live situation where attackers are unpredictable. Tabletop exercises are also vulnerable to bias, as preconceived notions or personal opinions can sway discussions. This could potentially neglect alternative perspectives and overlook critical aspects of the scenario.
Finding the Right Fit: Which Exercise is Right For You?
The choice between live fire and tabletop exercises should not be an either-or proposition. If at all possible, performing both exercises could create a more comprehensive training approach. Tabletop exercises are excellent ways to prepare for, debrief, and analyze the results of live-fire exercises. This helps teams anticipate potential issues, analyze their response in hindsight, and refine their procedures for future live fire simulations. You could also consider basing your live fire exercises on specific vulnerabilities identified during tabletop sessions, maximizing the learning potential. By rotating between live fire exercises and tabletop exercises, you keep your team challenged by different, complimentary training styles that help ensure they are prepared for all types of cyber attacks.
If you are unable to do both types of exercises, consider the following to help decide which might be right for your organization:
- Consider your budget and resource constraints. Live fire exercises require more investment in expertise and infrastructure, while tabletop exercises can be conducted with minimal overhead.
- Evaluate your organizational risk profile and threat landscape. If you face frequent targeted attacks or handle sensitive data, live fire may be essential. For more general preparedness, tabletop exercises can be a cost-effective starting point.
- Define your desired learning outcomes and training objectives. Tabletop exercises are excellent for refining plans and fostering communication, while live fire helps stress-test your defenses and uncover vulnerabilities.
Expert Recommendations
Many cybersecurity experts recommend both live fire and tabletop exercises. Notably, the NIST CSF (National Institute of Standards and Technology Cybersecurity Framework) includes tabletop exercises as part of its “Identify” and “Protect” functions, providing recommendations on developing and conducting tabletop exercises. Additionally, the NIST Special Publication 800-84 provides a guide to designing, conducting, and evaluating both tabletop and live fire exercises.
Other respected government agencies, such as the United States Cybersecurity and Infrastructure Security Agency (CISA), also explicitly recommend tabletop exercises and allude to the value of live fire exercises. In their framework, CISA claims that “investments in planning and exercises support timely response and recovery actions, resulting in reduced impact to the delivery of services.”
It isn’t just government institutions that advocate for simulated cybersecurity exercises, however. Many renowned cybersecurity experts also support the value of live-fire exercises. For example, infamous hacker Kevin Mitnick was very outspoken in support of simulated exercises to test defenses. Although Kevin Mitnick passed away in July of 2023, his company Mitnick Security continues to offer vulnerability testing and red team operations similar to those conducted in many live fire exercises.
Stamus Networks’ Experience with Live Fire Exercises
Every year, the NATO Cooperative Cyber Defense Centre of Excellence (CCDCOE) hosts two major cybersecurity exercises, Locked Shields and Crossed Swords. These large-scale live fire exercises bring together expert teams from various NATO nations to defend simulated networks against complex cyber attacks. Since 2016, Stamus has partnered with the CCDCOE at both Crossed Swords and Locked Shields, providing training and expertise while deploying the Stamus Security Platform (SSP) in support of the exercises. Notably, at Crossed Swords 2022 the 2 representative team members from Stamus Networks used SSP to identify 60% of the total threats discovered by the yellow team. This type of involvement from our team showcases our commitment and belief in these types of exercises. Peter Manev, co-founder and Chief Strategy Officer of Stamus Networks, recently discussed his experience at Crossed Swords in an article on tabletop exercises published by Dark Reading.
Preparedness Pays Off
Ultimately, the choice between live fire and tabletop exercises is less about determining which is better, and more about creating a training program tailored to your organization’s unique needs and goals. By understanding the strengths and limitations of each type, employing a strategic framework, and considering a combined approach, you can equip your security team with the knowledge, skills, and confidence to face any cyber threat that may come their way. Remember, preparedness is your most valuable asset.
To be notified of new blog posts and other news, make sure to subscribe to the Stamus Networks blog and the Stamus Spotlight Monthly Newsletter, follow us on Twitter, LinkedIn, and Facebook, or join our Discord.
