The Vision: One LLM to Rule Them All

Every SaaS vendor is quietly calling an LLM, and someone has to govern it.

Walk into any large enterprise right now and you'll see the same pattern. IT and data leadership are putting a wall around a small set of sanctioned LLM providers, typically some combination of Anthropic's Claude, Microsoft Copilot, and a flavor of OpenAI through Azure, and they're locking the perimeter down hard.

The reasoning is sound. Most enterprises will happily run two or three models, often behind an internal gateway that routes and enforces policy. The real concern is the explosion happening outside that wall. Every SaaS vendor on the market now ships an "AI-powered" feature, which means every vendor is quietly calling some LLM provider on the backend. Multiply that by the hundreds of tools a large enterprise touches and you're looking at hundreds of ungoverned LLM calls with no consistent data handling and no audit trail.

I've sat through enough IT security reviews to know how this lands: the point isn't how many models live inside the perimeter, it's having a perimeter at all.

This is the right move, and most enterprises are already making it. But governance alone just gets you a well-secured chatbot. The value comes from a second force, and how the two fit together.

The central LLM should orchestrate tools, not analyze raw data.

Once you've centralized, the instinct is to make the LLM the brain of the organization. Hook it up to everything, let employees ask it anything. That instinct is right too. A central LLM should be able to answer questions nobody in the company could answer in under a week of manual digging.

The problem is what happens next.

First, LLMs are unreliable at end-to-end analysis of raw business data. Not because the reasoning is weak, it usually isn't, but because the information needed to get the answer right is rarely in the data. Ask a model to find your unprofitable customers from transaction and cost data and it will do something defensible: allocate costs, compute margin, rank the losers. It will not know that the cost table it queried leaves out transfer price on goods coming from your other manufacturing entity, so every account served from that plant looks more profitable than it is. Nothing in the data says so. The analysis is competent, the answer is wrong, and it reads as authoritative either way.

This is the part people expect the next model release to solve. It won't. Better reasoning does not tell a model what its data left out.

Second, LLMs are stochastic. The same question, asked twice against raw data, can produce two different answers. That's fine when you're drafting an email. It is not fine when the VP of Supply Chain asks "what's our exposure on the Q3 inventory plan?" and gets a different number than her counterpart in Finance got an hour earlier. The credibility of the whole system collapses, and rightly so.

Strategic questions depend on layers of business logic, the segmentations, forecasts, and simulations the organization has spent years building, debating, and aligning on. You cannot ask the LLM to reconstruct your portfolio segmentation from raw transactional data every time someone references it. The segmentation is a deliberate artifact of how the business thinks about itself, not something waiting to be rediscovered in the warehouse.

So the fix isn't to stop the LLM from analyzing. It's to keep it out of the analyses whose outputs are a) heavily context-dependent, and b) have to be deterministic. The pattern that wins: composable, deterministic tools that don't just expose the business's data, but encode its logic as something the central LLM can run, query, and modify. Composable meaning the output of one becomes the input to the next, so the LLM can chain them into an analysis nobody built in advance. Deterministic meaning the same call returns the same answer, every time, to everyone.

The semantic layer was the first hint. dbt, Cube, Looker, the whole category exists so that "revenue" means the same thing whether you ask a dashboard, a notebook, or an LLM. But a semantic layer is read-only, and that's where almost everyone has stopped. A knowledge graph, RAG over your documents, a governed metrics layer: all of it just lets the LLM look things up. Looking things up is table stakes. It is not deciding.

To make a real decision, the LLM has to act on the business's logic, not just retrieve it: run a simulation, ask a what-if, weigh alternatives, optimize against a goal. That's the difference between exposing more data and building a tool the LLM can actually use, and it's the harder problem by far. On the complex end, a tool like Pillar AI's River can build a probabilistic simulation of a company's supply chain. Instead of looking at the raw planned inventory data and inventing a new analysis, the LLM queries River, gets a probability of a stockout, changes an assumption, and re-runs the simulation to see what happens. Your segmentation, your demand forecast, your pricing model are the same kind of artifact: deliberate, governed, and, crucially, executable.

Let the LLM be creative about what to analyze and how to compose the analysis. Don't let it be creative about the analytical machinery the business depends on.

Triaging five thousand SKUs.

Put several of these tools together. Suppose you ask your central LLM to figure out which of your underperforming SKUs can be fixed and which should be rationalized. It's a real strategic question, and exactly the kind of thing that gets a bad reputation as a use case for LLMs today.

Here's how it could go. The LLM pulls your strategic segmentation, the one your leadership team aligned on, and identifies SKUs with weak commercial performance and poor operational execution. Let's say five thousand candidates. For each one, the LLM queries your supply chain simulation, running roughly a hundred thousand simulations to ask whether service can be improved while reducing working capital, and at what cost. Then it goes to your price elasticity simulation, another deterministic tool, and runs another hundred thousand simulations to evaluate pricing and distribution levers. Finally, it synthesizes across all of it, grouping the five thousand SKUs into buckets: fixable operationally, fixable commercially, fixable through a combination, or genuinely worth rationalizing.

Notice what the LLM did, and didn't do. It didn't build a segmentation. It didn't estimate elasticities on its own. It didn't simulate the supply chain in its head. Every quantitative claim traces back to a governed tool the business trusts, so two executives running the same workflow get the same recommendation. But the LLM did something no human analyst could have done in a reasonable timeframe: it composed the workflow, drove two hundred thousand simulations across five thousand SKUs, and turned the results into a structured recommendation. That's a real contribution to a real strategic decision, and it's only possible because the deterministic machinery was there to be called.

The future of enterprise AI isn't a bigger model. It's a better-equipped one.

The enterprises that get the most out of their central LLM over the next few years will start making its surrounding tooling richer. The LLM is a remarkable interface. It is not a remarkable analyst, at least not for the decisions where consistency and defensibility matter most.

Give it deterministic, business-aware tools to read, query, and where appropriate modify. Don't ask it to be the analyst, the forecaster, the segmenter, and the simulator all at once from raw data.

The future of enterprise AI isn't a bigger model. It's a better-equipped one.