Twenty years in this business and I can tell you the most dangerous thing in any security architecture is not the firewall you forgot to configure. It is the off switch nobody told you about, sitting inside the thing you installed on purpose.
Let me explain what I mean, because this is one of those problems that never goes away. It just changes costume.
You build defense in depth. Firewall, endpoint agent, logging, the whole liturgy. You spend a fortune and you feel good about it. Then some Tuesday afternoon an application team files a ticket that says the EDR agent is “impacting performance” on the payment app, and could you please add an exclusion. And you do, because the alternative is a meeting with the C-suite about why checkout is slow. So now there is a folder on that server the endpoint agent politely ignores. Congratulations. You bought a very expensive lock and then handed a copy of the key to the one process most likely to get popped.
I have watched this exact movie for two decades. The “temporary” any-any firewall rule that outlives the engineer who added it. The antivirus exclusion list that nobody has reviewed since the Obama administration. The break-glass admin account with a password in a wiki. Every one of them is the same story: a control that works fine, right up until the moment someone flips the switch that turns it off, and that someone is almost never on the security team.
The switch is usually held by convenience
Here is the thing about off switches. They are rarely installed by attackers. They are installed by us, for good reasons, usually performance or convenience, and then forgotten. The attacker’s job is not to build the bypass. The attacker’s job is to find the one you already built and left unlocked.
MITRE has a whole catalog entry for the attacker half of this. It is called Impair Defenses, technique T1562, and it is one of the busiest neighborhoods in the entire ATT&CK matrix. Disable the tool, add the exclusion, kill the logging, blind the sensor. The reason it is so busy is that we keep leaving the switches out where anyone can reach them.
You want to see this disease in miniature, stripped of enterprise budget and vendor theater, look at what just happened in the WordPress world.
A speed feature that turns off the firewall
A popular analytics plugin shipped a performance optimization. To make its tracking requests a hair faster, it installed a small module that, on any request it decided was headed for its own endpoint, unloaded every other plugin on the site for the duration of that request. Skip the page builder, skip the SEO plugin, skip the cache, get the tracking pixel done quickly. Reasonable, if you squint.
Except the check it used to decide “is this one of my requests” was a plain substring match against a value that is printed in the page source of every single page. So an unauthenticated attacker could take that value, staple it onto any URL as a junk parameter, and the module would obligingly unload every plugin on the site for that request. Including the security plugin. Including the firewall. There is a full writeup of the bypass with the confirmed WAF evasion and the vendor’s fix, and it is worth your time, but you already know the shape of it because it is the same shape as your EDR exclusion. The attacker half is the boring part. The ATT&CK matrix files it under Disable or Modify Tools, and the design flaw has a tidy catalog number too, CWE-693, Protection Mechanism Failure, which is a bureaucratic way of saying the guard was real but somebody left a door around him. Someone optimized for speed, the security model was collateral, and the off switch ended up in the hands of whoever bothered to read the page source.
Small stakes, sure. It is a blog plugin, not a core banking system. But the mechanism is identical to the one that will ruin your quarter, and the design review that let it through made the same trade every enterprise architecture review makes: performance had a person in the room arguing for it, and the security model did not.
Why we never learn
Performance always wins the argument. Always. Nobody gets promoted for the checkout flow that stayed three milliseconds slower because the security team refused an exclusion. They get promoted for shipping. So the exclusion goes in, the bypass mode gets built, the fast path skips the annoying control, and everyone moves on to the next sprint.
And nobody threat-models the seams. We threat-model the firewall. We threat-model the app. We almost never threat-model the interaction between the caching layer and the firewall, or the plugin that can unload other plugins, or the integration that runs as domain admin because getting least-privilege right was a two-week project and we had a demo Friday. The vulnerabilities live in the seams, and the seams are exactly where nobody is looking. I have said this before in the context of firewall management and I will keep saying it until it stops being true, which is to say forever.
Things that make me grumpy
- Exclusion lists that no human has read since they were created, growing quietly like the pile of AV exceptions I once found that had turned an endpoint agent into expensive wallpaper. I have opinions about the AV industry, but even good tools die by a thousand exclusions.
- The word “temporary” attached to anything in a firewall ruleset. There is no temporary. There is only “not yet forgotten.”
- Features that trade the security model for a benchmark number and call it an improvement.
- The assumption that a control which is installed is a control which is on. Installed and enforcing are two very different states, and the gap between them is where you get breached.
- Design reviews where performance sends a director and security sends a calendar invite that nobody accepts.
What to actually do
Go find your off switches. Every one of them. The exclusions, the bypass modes, the fast paths, the “temporary” rules, the integrations with more privilege than they need. Write them down. For each one, answer two questions: what control does this turn off, and who or what is able to flip it. If the answer to the second question is “an unauthenticated request” or “any application team with a ticket,” you do not have a control. You have a suggestion.
You do not need another tool for this. You need an afternoon, a spreadsheet, and the discomfort of looking honestly at what your defenses actually depend on. That is less fun than a proof of concept with a nice dashboard. It is also the only part of this job that has ever worked.