enterprisesecuritymag

Making Micro-purple Testing Work

By Matt Stiak, Director, Cyber Risk and Threat Operations, Delta Dental of California

Matt Stiak, Director, Cyber Risk and Threat Operations, Delta Dental of California

You’ve invested millions into your cybersecurity incident response capabilities.  How do you know that when under attack you’ll actually be able to find the needle in the haystack that allows you to quickly identify, contain, eradicate, and recover?  Traditional methods include running end-to-end tabletop war games, executing a red team exercise, or just documenting your successes against actual attacks.  While valuable, each of these methods has its shortcomings.

Tabletops and red team exercises can be costly and disruptive and are usually infrequently executed.  Red teams sometimes will take the shortest path to the goal, leaving avenues of attack unexplored, or sometimes repeatedly hammer on known issues, yielding little additional insight.  Taking the approach of waiting around for an attacker to evaluate your defenses has, let’s say, issues.

What is needed is a structured, complete, low cost, low risk, and continuously executing mechanism to evaluate your defenses against a constantly changing threat landscape.  You need to leverage the results of that evaluation to make incremental, yet meaningful improvements.  The continuous improvement cycle needs to be agile, and the testing strategy needs to adapt as well.

"By planting needles in your haystack, micro-purple testing gives you and your team an additional level of assurance that you will be ready when the attack comes"

A structured and complete evaluation framework should account for the myriad of attacker behaviors that may be used against you.  Work over the past few years on frameworks such as MITRE ATT&CK has given us such a library of attacker behaviors. They’ve analyzed thousands of advanced attacks in order to find the common tactics and techniques, or building blocks, that an attacker will string together.

Having these building blocks gives a low cost, low-risk way to structure tests.  Instead of an end-to-end test of the system, like a red/purple team exercise or a tabletop, we can create “unit,” or micro-purple, tests.  Each test is used to simulate a specific behavior.  If these tests are scriptable, it is straightforward to execute them on-demand or continuously.

When executing the micro-purple tests, ask three questions to understand your effectiveness.  Was I able to block the behavior?  If I didn’t block it, could I alert on it?  When I alert on it, do I have a procedure to respond to the specific alert?  Measure the results, report your improvement over time, and identify additional needs as attacker behaviors evolve.  At a strategic level, the results of the tests can support the case for additional investment into controls.

As you implement the micro-purple method, there are a number of decisions that will need to be made.  There aren’t right or wrong answers but thinking through each of these decisions in advance will prepare you for a successful implementation.

What behaviors should I simulate or test?  MITRE ATT&CK includes all attacker behaviors, even those that overlap with legitimate user behaviors.  Differentiating between an attacker and a legitimate user can be very difficult and lead to high rates of false positives with little gain, especially if you are just getting started.  Choose behaviors that are clearly malicious to start.

The behaviors are generic, so how should I write my test?  The frameworks include base examples, which is how to get started.  If you’re handling those examples with ease, you’ll want to uptick the sophistication of each behavior.  Learning how to be a better attacker is a fantastic exercise for blue team members.  You can also engage your red team to build out specific tests.  Finally, you might consider a vended solution, which is essentially a pre-built test battery.

Where should I test?  This is important to ensure coverage of your capabilities.  Select a variety of network locations and system types.  In a perfect world, you’d test on all permutations, but starting with network behaviors that cross key choke-points and system behaviors on the most deployed variants of operating systems yields the most immediate value.

What do I do with the results?  Think through what you’ll find and how you might go about fixing it.  Our implementation identifies missing or incomplete data and visibility; missing or misconfigured blocking or detection rules; and missing or under-specified response playbooks.  Just in these categories. It’s clear that multiple teams can be engaged in the process of improvement, so workflow and task tracking are critical.  We use a Kanban board with a clearly defined workflow for opening, closing, and transferring work tickets.

These tips will get you started.  Start small and start slowly.  For many teams, this method can represent a significant change in the way work is done.  A two-week scrum, replete with retrospectives to reflect on and tune the process itself can help the team prioritize work and quickly identify how to make micro-purple work for them.  Through an Agile approach, you’ll also gain the ability quickly pivot to changes in your threat profile as well as adapt to developments in attacker techniques.

By planting needles in your haystack, micro-purple testing gives you and your team an additional level of assurance that you will be ready when the attack comes.  Continuous testing and the associated continuous improvement cycle focus the team on building visibility, control, and response capabilities that can be proven to work.

Read Also

Evaluating Challenges and Impacts of Anti-Forensics

Evaluating Challenges and Impacts of Anti-Forensics

Steve Stawski - Director of eDiscovery and Forensics for a large Global Company
DDoS Attack... for Dummies!

DDoS Attack... for Dummies!

Thierry Derungs, Chief Digital Officer, BNP Paribas Wealth
Artificial Intelligence Against Online Financial Fraud

Artificial Intelligence Against Online Financial Fraud

Nicolas Miailhe, Co-Founder & President, The Future Society
Outcomes Rising from the IT Roadmap Leveraging Microsoft Azure

Outcomes Rising from the IT Roadmap Leveraging Microsoft Azure

Chad Leverenz, Vice President, Information Technology, Mercy Housing

Weekly Brief