Last quarter your organization ran a tabletop exercise. A facilitator walked the team through a ransomware scenario, everyone said the right things, the runbook was referenced, and the after-action report concluded that the incident response plan is “mature and well understood.” Somebody printed it. Somebody put it in a board deck.
Congratulations. You passed a test that was designed to be passed. And I need to tell you what that is worth on the night it actually happens, which is roughly nothing.
The exercise everyone was going to pass
Here is the thing about most tabletop exercises: they are theater, and everyone in the room is in on it. The scenario is circulated in advance. The people who would actually be unavailable at 3am are sitting right there at the 10am meeting, coffee in hand, fully briefed. The facilitator is a friendly consultant whose renewal depends on you feeling good about the outcome. Nobody’s pay, promotion, or weekend is on the line. Under those conditions, of course the plan works. You have removed every single variable that makes a real incident hard.
A real incident does not open with a slide. It opens with an alert nobody is sure about, at an hour nobody wants, involving a system whose only expert left the company in March. The runbook points at a wiki page that 404s. The emergency contact list has two people who changed roles and one who is on a plane. Legal wants to be looped in before anything is said, and nobody agreed in advance what “anything” means. That is the exercise. The tabletop tested the plan. The incident tests the organization, and those are not the same thing.
What the passing grade is hiding
The reason a clean tabletop is worse than useless is that it manufactures confidence without earning it. You now believe you are ready, which means you will stop investing in getting ready, which is exactly backwards. The Verizon Data Breach Investigations Report has spent years documenting the same gap between what organizations think their detection and response looks like and what actually happens during a breach, and the distance is not small. An exercise that closes that gap by pretending the hard parts away is not measuring readiness. It is measuring how well your team performs when told the answers in advance, which is a skill with no application on the day it matters.
But here is what nobody tells you: the value of an exercise is entirely in what it breaks. An exercise where everything works taught you nothing you did not already believe. An exercise where the backup restore failed, the on-call could not be reached, and it took forty minutes to decide who was allowed to pull a production system offline, that one was worth every hour it cost, because it found the failures in a room instead of in a headline.
Things That Make Me Grumpy
- Scenarios circulated a week early, so the “response” is a recital.
- Exercises that never once say the words “and that person is unreachable.”
- Testing the incident plan without ever testing a restore from backup, which is the one control the whole plan quietly assumes works.
- Measuring success by whether the team felt confident, instead of by how many gaps got written down.
- The after-action report that lists zero action items. If nothing needs fixing, you did not run an exercise, you ran a ceremony.
Fine, here is how to run one that is worth the afternoon
Do not announce the scenario. Hand it to the incident commander cold, the way reality will. Then start removing people. Tell the database expert they are on a flight and cannot be reached, and watch what happens, because in the real thing they will be. Inject a second problem an hour in, because incidents do not wait politely for you to finish handling the first one. And at least once a year, do not simulate the backup restore, actually perform it against real data, because a recovery plan that has never been executed is a hypothesis, not a control.
Measure the things that hurt on the night: time to first decision, time to a customer-ready statement, time to a verified-clean restore. Not “did we follow the plan.” The plan is a starting position. The SANS incident-handling literature has been saying a version of this for two decades, and the version I will say is shorter: an exercise that everyone passes was a waste of everyone’s time.
Run the next one to fail. If your team walks out a little rattled and holding a list of things that are broken, that was not a bad exercise. That was the only kind that was ever worth running. The certificate on the wall and the clean tabletop report belong to the same genre of fiction, and the attacker has not read either one.