Disaster Recovery Plans That Look Good on Paper (and Fail in Reality)

Most organisations have a disaster recovery plan. Most of those plans have never been tested under realistic conditions. And most of the ones that have been tested have failed in ways that the document did not anticipate. The gap between a DR plan that looks good on paper and one that works when you actually need it is one of the most consequential and most consistently underinvested areas of enterprise risk management.

The DR Plan That Nobody Has Read

The most common disaster recovery failure mode begins before any disaster occurs. The DR plan exists — it was written eighteen months ago, approved by the CISO, and filed in the compliance management system where it satisfies the audit requirement for documented DR procedures. Nobody on the current incident response team has read it. Two of the people named as DR coordinators have left the company. The recovery procedures reference systems that have since been deprecated. The document is a compliance artefact, not an operational guide.

This situation is the norm rather than the exception. DR documentation tends to be produced in response to audit requirements or post-incident commitments, not in response to a genuine desire to improve operational readiness. The document gets written, the audit passes, and the document ages out of relevance without anyone noticing — until the disaster arrives and the plan provides no useful guidance.

Why Untested DR Plans Are Worse Than No DR Plan

An untested DR plan creates false confidence. Leadership believes the organisation has a DR capability because a document exists describing one. Engineers believe a recovery procedure will work because it has been approved. When the disaster occurs, the false confidence produces delayed escalation, poor decision-making, and costly improvisation — all of which would have occurred faster and more effectively without the plan, because at least then the team would have known it was improvising.

The discovery that a DR plan does not work should happen during a controlled test, not during an actual disaster. Testing under controlled conditions with time to iterate is inexpensive. Testing under disaster conditions with real customer impact, regulatory scrutiny, and stakeholder pressure is extraordinarily expensive — in recovery time, in customer trust, and in the organisational trauma that follows a poorly managed incident.

The RTO/RPO Theatre Problem

Recovery Time Objective and Recovery Point Objective are the two most important numbers in a DR plan, and they are almost universally set incorrectly. RTOs and RPOs are typically determined by asking business stakeholders how long they can tolerate downtime and how much data loss is acceptable — reasonable questions. The answers are then committed to in the DR plan without testing whether the engineering architecture can actually achieve them.

The result is RTO/RPO commitments that are operationally unachievable. A four-hour RTO requires a DR infrastructure that can be activated in four hours. If your DR environment is built manually during a disaster, your actual RTO is more likely twenty-four to forty-eight hours. The plan says four hours; the reality says otherwise. This gap is RTO/RPO theatre — the appearance of defined recovery commitments without the engineering foundation to meet them.

Five Failure Modes of Real-World Disaster Recovery

• Stale infrastructure: the DR environment has not been updated to match production configuration changes made since the last DR test. The environment that starts up during recovery does not match the environment that was running before the disaster.
• Data synchronisation gaps: replication lag means the DR environment’s data is hours or days behind production at the time of failover. The RPO commitment is violated immediately on activation.
• DNS and certificate failures: DNS cutover and TLS certificate management are consistently underspecified in DR plans and consistently problematic during actual failovers. Propagation delays and certificate mismatches produce extended recovery time.
• Human dependencies: the recovery procedure requires specific people who are not available — on holiday, in a different timezone, or no longer with the organisation. The plan assumes institutional knowledge that is not documented.
• Third-party dependency failures: the DR plan accounts for the organisation’s own infrastructure but not for the third-party services it depends on. When those services have their own availability issues during the same disaster event, the recovery is blocked.

DR as a Product, Not a Document

The most important conceptual shift in effective DR is treating it as a product rather than a document. A DR product has an owner, a roadmap, a set of acceptance criteria, and a release process — just like any other engineering product. The acceptance criteria are the RTO and RPO commitments. The release process is the DR test. The owner is accountable for the product meeting its acceptance criteria.

This framing changes how DR is resourced. A document is produced once and maintained occasionally. A product requires ongoing investment: infrastructure updates that keep the DR environment in sync with production, regular testing that validates the recovery procedures, and a backlog of improvements that address the gaps discovered in each test. This investment is budgeted and planned, not absorbed as operational overhead.

GameDay Testing: Making DR Muscle Memory

GameDay testing — scheduled disaster simulation exercises in which the team activates DR procedures in a controlled environment — is the most effective mechanism for building genuine DR capability. The first GameDay almost always reveals significant gaps between the documented procedure and the actual recovery capability. Subsequent GameDays validate the fixes and build the team’s confidence in the procedure.

GameDays should be conducted at least annually for critical systems, and more frequently for systems with aggressive RTO/RPO commitments. They should be treated as engineering investments, not compliance exercises — the goal is to find and fix gaps before a real disaster requires the procedure to work flawlessly under pressure.

The Human Factor

Technology is the easier half of disaster recovery. The harder half is the human organisation that activates and executes the recovery. Clear decision authority — who declares a disaster, who activates DR, who communicates to customers and regulators — must be documented, understood, and practiced. Communication trees that rely on a single point of contact fail when that contact is unavailable. Decision criteria that require judgment calls that the on-call engineer has never made before produce delays.

The best DR organisations have practiced these human processes as much as they have tested the technical procedures. Tabletop exercises that walk teams through disaster scenarios without actually activating infrastructure build the decision-making muscle memory that technical tests do not.

What Good DR Governance Actually Looks Like

Effective DR governance includes: an annual review that assesses the currency of the DR plan against current architecture and personnel; quarterly testing of individual DR components even if full DR activation is annual; post-incident reviews that evaluate actual recovery performance against documented RTO/RPO commitments; and a clear owner with executive accountability for DR readiness. The governance model treats DR as a risk management function with measurable outcomes, not a compliance function with documents as deliverables.