This is Post 3 of The Production Ready Series.
"The schema conversation is not a one time event. It is the moment a team decides to stop building for today and start building for production."
Two Engineers, Two Mental Models
Every AI system has a schema conversation eventually. The question is whether it happens before the layer gets built or after something breaks.
Two engineers working on the same system will produce two different schemas. Not because one is wrong. Because each is solving from a different vantage point. The application engineer sees the system from the request layer down. The data engineer sees it from the table layer up. Neither view is complete. Both are necessary.
The schema conversation is where those two views become one. Where assumptions get named. Where gaps get surfaced. Where the team stops building what seemed right and starts building what production actually requires.
Skip that conversation and you ship a metrics layer that reflects one perspective. It will look complete until the moment it is not — and that moment will arrive at the worst possible time.
MVP Got You Here. It Won't Get You There.
MVP schema design optimizes for speed. Get the tables down, get the data flowing, get the application running. Good enough is the goal. It should be.
But good enough for MVP is not good enough for v1. The difference is accountability. MVP is a proof of concept. V1 is a production commitment. The business sponsor is now watching. The system is now being used by real users on real data. The questions being asked are no longer does it work but how well does it work, how do we know, and what happens when it does not.
That shift requires a different schema conversation. Not what did we build but what does production accountability actually require. Not what columns do we have but what questions can we not currently answer that we should be able to.
The MVP to v1 transition is a maturity milestone. The schema review is how you cross it deliberately rather than stumbling across it when something breaks.
What the Review Actually Looks Like
It is not glamorous. It is methodical.
You go through every table. Every column. Every assumption baked into the MVP design. You ask the same questions at each one — what is this capturing, is it capturing it correctly, what is missing, what does the next six months of this system's life require that this table cannot currently answer.
You bring the right people. Not just the engineers who built it. The testing resource needs to be in that room. They are writing verification logic against this schema. If they are not aligned before the layer gets built out further the cost of misalignment lands on everyone later. Testing alignment is not a downstream concern. It is a precondition for v1.
You document the output. Not because process requires it. Because six months from now someone will ask why a column exists or why a table is structured a certain way. The schema conversation produces decisions. Decisions need to be findable.
What Came Out of Ours
Consistent typing across all six tables. Not assumed — explicitly reviewed and locked. Bigints for identity columns carrying SQL Server lineage from the relational source systems this application sits alongside. Timestamps as timestamps. No type surprises when you join across the taxonomy.
Naming conventions confirmed. The _events suffix throughout. OperationalMetrics as the deliberate exception — structurally different by design, key/value pattern, flexible by intent.
ingest_date on every table. Stamped at ingestion. A reliable timeline anchor for every record. Simple enough to feel obvious. Missed often enough to be worth saying explicitly.
And the one that came out of the review that should have been there from day one.
HeartbeatEvents.
A regular structured pulse from each service confirming it is running and healthy. No transactions. No user events. Just a timestamp, a service identifier, and a status. The most basic signal in the taxonomy. Nobody had it on their list going into the review. The process surfaced it.
That is what the schema conversation is for. Not to validate what you already know. To find what you do not know you are missing.
The Discipline That Makes It Hold
Schema discipline is not exciting. It is the difference between a metrics layer that holds up over time and one that fragments quietly until nobody trusts it.
Type consistency is a contract. Every table in the taxonomy needs to speak the same language. Inconsistent types are technical debt that compounds every time someone writes a query across tables. Lock them in the review. Hold the line after.
Naming conventions are documentation. The _events suffix tells the story without a wiki page. Six months from now when someone new is maintaining this layer the naming should be self-explanatory. If it is not the naming is wrong.
The schema evolves but not arbitrarily. OperationalMetrics uses a key/value pattern precisely because LanceDB telemetry and infrastructure signals are still maturing. New metrics get added without a schema change. That flexibility is intentional — but it is bounded. The tables with stable, well-understood domains get rigid schemas. The tables that are still maturing get designed for evolution. Know which is which before you build.
Data quality gates are part of the schema conversation. Null checks. Pipeline stops on threshold failures. A metrics layer fed with bad data is worse than no metrics layer — it produces false confidence at exactly the moment you need accurate answers. The review is where you define what clean data looks like for this layer and how you enforce it.
The Process Is the Output
The schema conversation produces a schema. But the more valuable output is the alignment.
Two engineers who went through that process together understand the layer the same way. They make consistent decisions when the schema needs to evolve. They catch each other's blind spots before they become production issues. They can both stand in front of a business sponsor and explain what the system captures and why.
That alignment does not happen by accident. It happens because the team made time for the conversation before v1 shipped. Because the right people were in the room. Because the output was documented and owned.
The schema is an artifact. The alignment is a capability.
Build both.
What's Next in This Series
Post 4 — What Databricks Gives You and What It Doesn't: What Databricks gives you natively and where you still need to build
Post 5 — Follow the Request: Walking a request chain from endpoint to vector store and back
Post 6 — Closing the Loop: Feeding clean signal to your observability platform and making the business conversation possible
Clarity through the chaos.