Making Micro-purple Testing Work

By Matt Stiak, Director, Cyber Risk and Threat Operations, Delta Dental of California

Matt Stiak, Director, Cyber Risk and Threat Operations, Delta Dental of California

You’ve invested millions into your cybersecurity incident response capabilities.  How do you know that when under attack you’ll actually be able to find the needle in the haystack that allows you to quickly identify, contain, eradicate, and recover?  Traditional methods include running end-to-end tabletop war games, executing a red team exercise, or just documenting your successes against actual attacks.  While valuable, each of these methods has its shortcomings.

Tabletops and red team exercises can be costly and disruptive and are usually infrequently executed.  Red teams sometimes will take the shortest path to the goal, leaving avenues of attack unexplored, or sometimes repeatedly hammer on known issues, yielding little additional insight.  Taking the approach of waiting around for an attacker to evaluate your defenses has, let’s say, issues.

What is needed is a structured, complete, low cost, low risk, and continuously executing mechanism to evaluate your defenses against a constantly changing threat landscape.  You need to leverage the results of that evaluation to make incremental, yet meaningful improvements.  The continuous improvement cycle needs to be agile, and the testing strategy needs to adapt as well.

"By planting needles in your haystack, micro-purple testing gives you and your team an additional level of assurance that you will be ready when the attack comes"

A structured and complete evaluation framework should account for the myriad of attacker behaviors that may be used against you.  Work over the past few years on frameworks such as MITRE ATT&CK has given us such a library of attacker behaviors. They’ve analyzed thousands of advanced attacks in order to find the common tactics and techniques, or building blocks, that an attacker will string together.

Having these building blocks gives a low cost, low-risk way to structure tests.  Instead of an end-to-end test of the system, like a red/purple team exercise or a tabletop, we can create “unit,” or micro-purple, tests.  Each test is used to simulate a specific behavior.  If these tests are scriptable, it is straightforward to execute them on-demand or continuously.

When executing the micro-purple tests, ask three questions to understand your effectiveness.  Was I able to block the behavior?  If I didn’t block it, could I alert on it?  When I alert on it, do I have a procedure to respond to the specific alert?  Measure the results, report your improvement over time, and identify additional needs as attacker behaviors evolve.  At a strategic level, the results of the tests can support the case for additional investment into controls.

As you implement the micro-purple method, there are a number of decisions that will need to be made.  There aren’t right or wrong answers but thinking through each of these decisions in advance will prepare you for a successful implementation.

What behaviors should I simulate or test?  MITRE ATT&CK includes all attacker behaviors, even those that overlap with legitimate user behaviors.  Differentiating between an attacker and a legitimate user can be very difficult and lead to high rates of false positives with little gain, especially if you are just getting started.  Choose behaviors that are clearly malicious to start.

The behaviors are generic, so how should I write my test?  The frameworks include base examples, which is how to get started.  If you’re handling those examples with ease, you’ll want to uptick the sophistication of each behavior.  Learning how to be a better attacker is a fantastic exercise for blue team members.  You can also engage your red team to build out specific tests.  Finally, you might consider a vended solution, which is essentially a pre-built test battery.

Where should I test?  This is important to ensure coverage of your capabilities.  Select a variety of network locations and system types.  In a perfect world, you’d test on all permutations, but starting with network behaviors that cross key choke-points and system behaviors on the most deployed variants of operating systems yields the most immediate value.

What do I do with the results?  Think through what you’ll find and how you might go about fixing it.  Our implementation identifies missing or incomplete data and visibility; missing or misconfigured blocking or detection rules; and missing or under-specified response playbooks.  Just in these categories. It’s clear that multiple teams can be engaged in the process of improvement, so workflow and task tracking are critical.  We use a Kanban board with a clearly defined workflow for opening, closing, and transferring work tickets.

These tips will get you started.  Start small and start slowly.  For many teams, this method can represent a significant change in the way work is done.  A two-week scrum, replete with retrospectives to reflect on and tune the process itself can help the team prioritize work and quickly identify how to make micro-purple work for them.  Through an Agile approach, you’ll also gain the ability quickly pivot to changes in your threat profile as well as adapt to developments in attacker techniques.

By planting needles in your haystack, micro-purple testing gives you and your team an additional level of assurance that you will be ready when the attack comes.  Continuous testing and the associated continuous improvement cycle focus the team on building visibility, control, and response capabilities that can be proven to work.

Weekly Brief

Read Also

Managed Security Services - A Business Partnership

Managed Security Services - A Business Partnership

Peter Stevens, Head of Security Operations, Royal London Group
Automate, Orchestrate, and Delegate

Automate, Orchestrate, and Delegate

Ian Hill, Global Director of Cyber Security, BAM
Becoming a Leader in Enterprise Security

Becoming a Leader in Enterprise Security

RANDY RAW, VP of Information Security, Veterans United Home Loans
How Blockchain can Support Future Industrial Evolution

How Blockchain can Support Future Industrial Evolution

Odile PANCIATICI, Blockchain Project VP, Groupe Renault
How Modernized Encryption Standards and TLS 1.3May Impact Your Security Strategy

How Modernized Encryption Standards and TLS 1.3May Impact Your...

Ben Schoenecker, CISSP, Director of Information Security, Hendrick Automotive Group
IT Security: A Practical Approach

IT Security: A Practical Approach

Christopher McCarey, Director of IT Security for Gila River Hotels & Casinos – Wild Horse Pass, Lone Butte and Vee Quiva