The Concierge Test

Everyone says they have a semantic layer now. Almost nobody has tested whether it's actually good enough to trust — and "we modeled the data" and "we built something that survives an executive's first hard question" turn out to be very different claims.

I've argued that the same governed layer should serve both an executive and an AI agent — one kitchen, two guests. But "governed" isn't a checkbox. A thin semantic layer is worse than no semantic layer at all, because it looks authoritative right up until it's wrong.

Additive isn't the same as correct

The easiest mistake to miss: not every metric can be summed up a hierarchy. Revenue by region rolls up cleanly — add the regions, get the total. A conversion rate doesn't. Neither does a point-in-time balance, a distinct-customer count, or anything built on a ratio. Sum four regional conversion rates and divide by four, and you've built a number that looks like an average and means nothing.

A semantic layer that only knows how to add things up will hand that wrong number to an executive with total confidence — and hand the identical wrong number to an AI agent, which will then explain it back with even more confidence. Non-additive measures need their own logic, defined once, or they'll get re-derived wrong in every tool downstream that touches them.

Your fiscal year doesn't start in January

Ask a "governed" semantic layer for Q3 revenue and watch what happens if the business runs a fiscal calendar — a retail 4-4-5 calendar, a fiscal year starting in February, a week defined Sunday-to-Saturday instead of the ISO standard. If the layer only understands calendar dates, every "quarter" query it answers is quietly wrong for a fiscal-calendar business, and nobody notices until finance does the math by hand and gets a different number.

This is exactly the kind of business logic that lives in an analyst's head and nowhere in the schema — which is the whole reason a semantic layer exists in the first place. If the layer doesn't encode the calendar the business actually runs on, it hasn't captured the business logic. It's captured the easy 80%.

Portable, or it's not really governed

The last test is the one teams skip because it doesn't show up until year two or three: can the logic survive a tool migration? If "net revenue" is defined inside one BI tool's proprietary semantic model, you haven't governed your metrics — you've moved the mess to a nicer UI, and you'll re-derive everything from scratch the day you switch warehouses or dashboards. A governed definition that only one vendor can read isn't governed. It's rented.

This is the same conviction as declaring the grain first, one level up the stack: define the concept once, somewhere every tool can reach it, and make the wrong fork — a second, slightly different definition living in someone's dashboard — genuinely harder to create than reusing the real one.

The same test, run on a human

Here's the part that ties back to the concierge idea directly: the rigor that keeps an AI agent from hallucinating is the same rigor that turns a dashboard tile into an actual headline instead of decoration. A tile that says "Q3 revenue: $4.2M" isn't a headline — it's a number with no story. A real headline states the finding, the driver, and the tension in one sentence: "Q3 revenue is up 4%, entirely from mid-market renewals — new-logo growth is still flat." That sentence is only possible if the layer underneath can actually answer "compared to what, and why" without someone hand-building the comparison in a spreadsheet first.

Cognitive overload and hallucination are the same failure wearing different clothes — a consumer, human or agent, handed raw material and expected to assemble the meaning themselves. The fix, both times, is doing the assembly once, correctly, upstream.

Run the test before you trust the layer

Before you tell an executive — or an AI agent — that your data is "modeled," check it against the parts that actually break: does it handle non-additive measures correctly? Does it know the calendar the business runs on, not just the one on the wall? Does the definition live somewhere every tool can reach, or just the one you happen to be using this year?

A semantic layer that passes all three isn't just technically correct. It's the layer good enough to actually host a guest — human or otherwise.