A backup you haven't restored is a hope, not a backup
What recovering a corrupted HR system with zero data loss in a day and a half taught me — and the HA/DR checklist I now use for every design.

Every HA/DR design I review now starts with one question: when did you last restore this? Here is the incident that taught me to ask it.
What happened
An ERP upgrade at an enterprise customer corrupted their HR data. That alone is recoverable — it's what backups are for. Then the backup failed validation on restore. Payroll and HR records for the whole organisation were at risk, and there was no clean restore point.
How we recovered
We stopped looking for a single fix and split the response into three parallel streams, each with a named owner:
- Backup restore — in case a partial or older restore could still be salvaged.
- Vendor escalation — getting the product's own engineers working on the corruption.
- Transaction-log recovery — rebuilding the data from the database's own logs.
Leadership got updates on a fixed clock, whether or not there was news. Nobody had to ask for status, which kept the technical teams focused.
The transaction-log path worked. We recovered everything — zero data lost, in a day and a half — and reconciled the data before handing the system back. The customer later sent a thank-you letter, which in our line of work is rarer than an outage.
The root cause
It wasn't really the upgrade. Upgrades fail; that's normal. The root cause was a backup that had never been restore-tested. It existed, it ran every night, it showed green — and it was useless on the one day it mattered.
The checklist I use now
- Restore, don't just back up. Schedule restore tests into an isolated environment, and time them. The restore time is your real RTO.
- Agree RPO and RTO per system, in writing, with the business owner — not just the infrastructure team.
- Keep the logs. Transaction, redo and archive logs, retained and shipped off the host. They are your second line of defence.
- Recover in parallel streams with owners. Don't run your hopes one after another.
- Communicate on a clock. Every two hours, even when the update is "no change".
- Rehearse failover. A DR site that has never been failed over to is a theory, not a capability.
- Reconcile before you declare victory. "The database is up" and "the data is right" are different statements.
In the cloud this is easier — and easier to skip
Managed backups, point-in-time recovery and cross-region replication remove a lot of the heavy lifting. They also make it very easy to assume the problem is solved. Defaults are not a strategy:
- Test restores of managed backups too.
- Protect backups from the same credentials that run production — ransomware goes after backups first.
- Keep at least one copy in another region, and know how long it takes to bring it back.
The one-line version
A backup you haven't restored is a hope, not a backup.
More notes

Your OCI Backup Is Not a Recovery Strategy
OCI resilience is an application workflow, not a collection of snapshots.

Let's Build It: A Hands-On Agent-to-Agent Walkthrough
Part 9, the finale of AI From Zero to Agents. We build three small agents (a trip planner, a currency agent and a weather agent) that find each other through A2A Agent Cards and work together over the A2A protocol, and read the real messages they exchange.

The USB-C Moment for AI: MCP vs A2A vs ACP, and How Agents Talk to Each Other
Part 8 of AI From Zero to Agents. The standards that let AI plug into tools and into other AI: MCP (agent to tools), A2A (agent to agent) and the three different things people call “ACP”, with pictures, a side-by-side comparison and a timeline.