Product analytics data quality is a slow-motion disaster at most companies, and it is almost never the engineers' fault. It is because the requirements they get are too vague to implement consistently. 'Track when someone exports a file' leaves a dozen questions wide open: does the event fire before or after the API call, does it fire on an error too, what properties does it carry, what exactly do we name it, and is it one event or secretly two. Three engineers will make three perfectly reasonable but different guesses, and six months later you are staring at an export funnel that does not reconcile with your billing data, and nobody in the room can say why, and the meeting slowly turns into a séance.
I have lived this exact horror. I once spent a full sprint reconciling a single 'signup_completed' event that turned out to fire in four subtly different places, because the original ticket said, in its complete entirety, 'fire when signup is done.' Done according to whom? Done at the form submit, the email verify, the first login, or the welcome modal? All four, apparently, depending on which engineer touched which surface.
What a complete plan actually looks like
We were adding a prompt_revealed event to our analytics, and I was determined not to repeat the séance. I described the interaction precisely — user clicks Reveal on a premium prompt, the API returns a 200, the body renders on screen — set the analytics tool to Segment, and pasted our product context into the Design a Tracking Plan for a New Product Event prompt. Claude Sonnet 4.6 returned a plan that left genuinely nothing to interpretation: the event name in snake_case following Segment's convention, a trigger pinned explicitly to the 200 response, a seven-row property table with types and required flags, a Do Not Track section, and a QA checklist that spelled out the failure case as its own line item.
The two sections that earned their keep
The Do Not Track section told us, specifically and unprompted, not to fire the event on a cached reveal where the body was already in memory from a previous view — which is exactly the kind of silent double-count that would have inflated our reveal numbers and quietly broken the analytics behind our daily-cap UI. And the QA checklist's failure case ('the event must NOT fire when the API returns a 4xx') caught a real, shipped-to-staging bug: our first implementation fired on error responses too, counting failed reveals as successful ones. We caught it in QA on a Tuesday instead of in a confusing quarterly metrics review three months and ten thousand bad rows later.
- Engineering implemented it in a single PR with zero clarifying questions, which is genuinely a first in my career and made me suspicious until I saw the data come in clean.
- The PII rule made it flag that I had casually included the prompt's title as a property — which can contain user-authored content — and it suggested sending a stable prompt_id instead.
- The Downstream Use line ('feeds the daily-reveal-count funnel for the 50/day cap UI') reminded the engineer to test the cap logic in the same PR, which we would otherwise have tested separately and later.
- The property table's explicit 'Required?' column quietly ended a long-running argument about whether the plan property should be nullable for anonymous users.
The math on clean data
Clean tracking from day one is worth an absurd amount more than cleaning up six months of dirty data later, and I can say that with confidence because I have done the cleanup version more than once. The cleanup version involves a spreadsheet titled 'events we cannot trust,' a series of apologetic Slack messages to everyone who built a dashboard on those events, and a slow, creeping suspicion that none of your numbers have ever been real. Defining the event correctly up front, by contrast, costs about ten minutes and one careful read of the plan before you paste it into a ticket.
If your analytics have ever handed you a number you were not quite sure you could trust, the rot almost certainly started back at the tracking plan, in a three-sentence ticket. Grab the Design a Tracking Plan for a New Product Event prompt on Prompt Dock and run it on the very next event you add, before an engineer has to guess. Once the data is finally flowing cleanly and you trust it again, my Turn a Metrics Dump into a Skimmable Executive Summary prompt is what turns that trustworthy data into something leadership actually reads.