The "If Somebody Would Have Told Me" Paradox

Every bad outage ends the same way. In the retro, someone stares at the timeline and says it out loud: if somebody would have told me, I could have stopped this. And they're right — which is the maddening part. The failure wasn't unknowable. It was legible. The ingest lag had been creeping for three weeks. The refund rate had quietly detached from its baseline. One upstream table had been landing an hour late every night for a month. Nobody hid any of it. Nobody was watching it either.

That's the paradox. The information that would have saved you almost always already existed. It just wasn't pointed at anyone.

Your alerts only watch what you already fear

Ask why the tell went unseen and you get the same answer everywhere: alerts are set per report, on the handful of things you already thought to worry about. Someone wired a threshold to last quarter's fire — the one metric that burned them — and moved on. So your monitoring becomes a monument to your last incident instead of a watch on your next one.

It's the same failure mode as a metric defined five different ways in five reports, pointed at coverage instead of definitions. Each dashboard owner sets a tripwire in isolation, on the number they own. The breach that actually hurts is the one nobody set a tripwire for — because nobody had been burned by it yet. You cannot add your way out of that. Ten thousand thresholds still only watch the doors you already knew to lock.

Rules are how you expand your context

Here's the shift. Stop treating an alert as a tripwire on one number, and start treating it as a rule that runs over your whole governed domain. Not "email me when P1s clear ten," but "watch every metric in the model for the shape of trouble — a series detaching from its baseline, a leading input drifting, a cohort bending toward a cliff — and say so before it lands."

A rule like that needs somewhere to stand. It needs one governed definition of every metric it watches, on a declared grain, so "detaching from baseline" means the same thing everywhere and can't be gamed by a per-team query. The semantic layer isn't only what makes your reports agree. It's the surface a watchtower stands on — the same map an agent needs to reason over the model rather than guess — and it's what lets one rule see the entire domain at once instead of one dashboard at a time.

That's what using rules to expand your context means. A person watches the five things they can hold in their head. A governed rule watches the ten thousand things in the model — including the nine thousand nobody thought to fear.

Anomalies, indicators, predictors

Domain-wide watching sorts into three heights, and the value climbs as you go up:

• Anomalies — something is already off its baseline. Useful, but it's still a report about now.
• Indicators — a leading input is moving: ingest lag, queue depth, the adoption slope. The metric you care about hasn't broken yet, but its cause is in motion.
• Predictors — the model has seen this shape before and knows where it goes. This is the one that buys you the days.

Report-specific alerts rarely get past the first rung. They fire when the number you were watching finally breaks — which, for the failures that actually cost you, is after the SLA is already gone. A watchtower over the whole domain lives on the second and third rungs: it catches the drift and the pattern, and it hands you lead time instead of a receipt.

Run the pre-mortem

Here's the test. Take your last real incident — the one with the painful retro — and ask the easy question: what was the earliest thing in the data that was already true when the timeline started? There is almost always something. An input that drifted. A cohort that bent. A table that slipped its schedule.

Now ask the hard one: would any rule running today have caught it? If the honest answer is no — if the only way you'd have known was to have already feared exactly this failure — then you don't have a monitoring gap. You have a context gap. And the fix isn't another threshold. It's a rule that watches the whole domain, so that next time, something actually does tell you.

That's what the Data Concierge puts in the tower: governed rules that read the entire model and raise the anomaly, the indicator, and the predictor — before the retro, not during it.