What a Semantic Layer Is and Why Your AI Needs One to Do Real Analysis
If you’ve tried using an AI tool to analyze your business data and come away underwhelmed, there’s a good chance the problem wasn’t the AI. It was what the AI was working with.
Most people who start using AI for business analysis do something like this: they export a spreadsheet, upload it to an LLM, and ask a question. Or they grab a screenshot of a dashboard and ask the AI to explain what it’s showing. The AI responds with something that sounds reasonable. It describes the data, identifies a trend or two, maybe offers a general interpretation. And it feels useful, until you try to act on it and realize the analysis wasn’t actually grounded in anything solid.
What’s happening in those cases isn’t that the AI is bad at analysis. It’s that the AI is doing something closer to reporting. It’s describing what it sees. Real analysis — the kind that tells you why something happened, what’s driving a trend, and what you should actually do about it — requires something more.
The Two Things AI Struggles With in Data Work
AI language models are genuinely good at a specific set of things in the context of analysis. They’re good at sequencing a set of investigative steps: if you ask why your website traffic dropped, a well-prompted AI knows to look at traffic by page, by channel, by campaign, by time period. It can hold a complex analytical plan in mind and work through it systematically. It’s also good at taking data and explaining it in plain language, which is valuable for making analysis accessible to people who aren’t analysts.
What AI is not reliably good at is the math required to turn raw data into correctly calculated metrics.
Pete Caputa, CEO of Databox, explains why this matters. Take something as straightforward as a weekly engagement rate for a website. If you want to know what percentage of visitors stayed on your site rather than leaving immediately last week, you can’t just average the daily values. On a day with 1,000 visitors you might have a certain bounce rate. On a day with 10 visitors you’d have a different one. Averaging those two numbers gives you a figure that’s mathematically incorrect. You have to look at the aggregate numbers across the full period and calculate from there.
Every metric has its own version of this problem. Some metrics require summing. Some require taking the latest value in a period. Some require weighted averages. Some involve more complex calculations that depend on how your specific business defines the metric. An AI working from raw data tables will sometimes get these calculations wrong in ways that aren’t obvious from the output. The answer looks plausible. It’s just not right.
What a Semantic Layer Does
A semantic layer is a system that sits between your raw data sources and your AI, and handles two specific problems.
The first is relationships between data. In most business systems, the data you care about is spread across multiple tables that need to be joined correctly to produce a meaningful answer. In a CRM, for example, getting closed revenue by sales rep requires connecting a table of deals to a table of users to a table of close dates, at minimum. That joining logic has to be defined somewhere. A semantic layer defines it, so when you ask the AI a question that requires pulling from multiple tables, it doesn’t have to figure out the relationships on its own. The semantic layer handles that correctly every time.
The second problem is metric definitions. A semantic layer stores the specific calculation for each metric in your business: how your engagement rate is calculated, how your pipeline velocity is defined, what counts as a converted lead versus a marketing qualified lead. When the AI needs to return a metric, it doesn’t calculate it from raw data. It calls the semantic layer, which returns the correctly calculated value. The AI then takes that value and uses it in the analysis.
The result is a system where the AI handles what it’s actually good at — the logic, the sequencing, the explanation — and the semantic layer handles what requires precision. Together, they can do analysis that neither could do reliably alone.
Why This Is a Recent Development
A year or two ago, most of this infrastructure didn’t exist in a form that was practically usable for growing businesses. The pieces were there in enterprise data warehousing, but connecting them to an AI in a way that worked smoothly required significant custom engineering.
What changed, according to Caputa, is the emergence of MCP servers and similar interoperability standards that allow AI tools to connect to external systems in a structured way. Databox, for example, has built an MCP server that allows Claude to query Databox’s metric definitions and connected data sources directly. When Caputa asks Claude a question about his business, Claude isn’t working from a spreadsheet export. It’s pulling real-time, correctly calculated metrics from a system that knows exactly how those metrics are defined.
That level of integration is becoming more accessible. More tools are building MCP servers. More semantic layer products are becoming available at price points that make sense for companies outside the enterprise. The technical barrier is lower than it was 12 months ago, and it’s continuing to fall.
What This Means for Companies That Aren’t There Yet
If your current AI analysis workflow involves uploading exports or screenshots, you’re not getting the full picture. You’re getting a description of data you already had, filtered through a model that may or may not be calculating your metrics correctly. That can still be useful. It’s just not what AI-powered analysis is capable of when it’s set up properly.
The path forward starts with your data foundation. Before worrying about semantic layers and MCP servers, make sure the underlying data is clean and connected. Your CRM should be capturing activity, not just results. Your marketing tools should be connected to your revenue data. Your systems should be talking to each other in a way that gives you a complete picture of the business.
From there, the question is how to define your key metrics clearly enough that a system can calculate them consistently. What counts as a lead in your business? How do you calculate pipeline velocity? What’s your definition of an active customer? These are questions most companies have answered informally. Writing them down precisely is the prerequisite for building a semantic layer that actually reflects how your business works.
None of this is overnight work. But for companies that get it right, the payoff is AI that can actually analyze the business rather than just describe it — analysis that’s grounded in real calculations, pulled from live data, available to anyone who needs it.
Listen to the full podcast episode here.