The worst data incident I have ever been part of did not involve a single error message anywhere. An upstream team changed how a source table was populated, which quietly introduced duplicate rows, which doubled a downstream join, which inflated our reported revenue by roughly 2x on the executive dashboard. The pipeline ran green every single day. The dashboard rendered beautifully, all its charts pointing confidently up and to the right. The number was simply wrong, and it stayed wrong for nine days — including, in a detail I will never fully live down, the specific day we presented it to the board. Nobody noticed, because nothing broke. That is the real nightmare of data work: not a crash, which announces itself, but a confident, well-formatted lie that announces nothing.
When we finally caught it — a sharp person in finance squinted and said 'that revenue number can't possibly be right' — the post-mortem question was brutal and completely fair: how do we ever trust any dashboard again if a clean 2x doubling can slip through in total silence for over a week?
Tests, not vibes
The honest answer was that we had exactly zero automated checks on the data itself. We had unit tests on our transformation code, we had CI, we felt responsible. But we tested our code and we did not, in any way, test our data — the actual rows flowing through. So I described our core revenue table to the Build a Data Quality Test Suite for a Table prompt: the full schema, what it feeds downstream (the executive revenue dashboard and the monthly board deck), and the specific known issue, which was the fresh, painful duplicate-rows incident we had just finished living through. Claude Sonnet 4.6 returned a prioritized suite where the very first, block-severity check was a uniqueness assertion on the table's grain — the precise check that would have caught our exact duplication on day zero instead of day nine.
What the suite actually contained
- A primary-key uniqueness assertion (block severity) that returns the duplicated rows themselves rather than just a count, so you can immediately see exactly what doubled and trace it upstream.
- A freshness check (block) that fails loudly if the table has not loaded within the expected window, with the threshold written as a fill-in parameter instead of a confidently-wrong guessed number.
- A row-count anomaly check (warn) that flags any daily volume more than 2 standard deviations from the trailing average — which is the generalized, automated version of 'why did this number suddenly double.'
- A referential-integrity check against the customers table, because, as it correctly inferred from our context, half of our historical 'missing revenue' tickets had traced back to orphaned customer_ids with no matching parent.
The detail I would have skipped on my own
Left entirely to my own devices, I would have written lazy assertions that returned counts — 'there are 14 duplicate rows' — and then cheerfully spent the following hour figuring out WHICH fourteen rows and where they came from. The prompt's explicit rule that every assertion returns the offending rows, not a count, turned debugging from a small investigation into a single glance. And because it handed me both the raw SQL assertion and the equivalent dbt test for each check, I dropped the whole suite straight into our existing dbt project in an afternoon. The block-versus-warn severity split meant the pipeline now genuinely halts on a real integrity failure, instead of emailing a soft warning into a Slack channel that everyone muted in 2023.
It also did something I had not asked for and did not expect: it refused to over-test. I half-expected a generic 'validate every column' wall that would have been slow, noisy, and ignored within a week. Instead it tied every single check back to either the table's stated purpose or the known incident I described, and it told me, in plain words, which columns were not worth a freshness or distribution check because nothing downstream depended on them. A test suite that flags everything is functionally the same as no test suite, because the team learns to ignore the alerts. Restraint is what keeps a quality suite alive past its first month, and this one had restraint baked in.
The takeaway, learned the expensive way
Data quality is not a vibe you maintain by being a careful and conscientious person. Careful people presented that wrong number to the board. Data quality is a concrete set of assertions that fail loudly the instant reality drifts away from your stated assumptions about it. The difference between us-before and us-now is not diligence; it is that we wrote the assumptions down as code that runs every day. Grab the Build a Data Quality Test Suite for a Table prompt on Prompt Dock, point it at the single most important table behind your most-watched dashboard, and let it write the exact check that would have caught your last silent incident. I now run it on every new table the moment my Profile and Clean a Messy CSV with Reproducible pandas Code prompt finishes loading the first batch — clean the data once, then guard it loudly forever.