Twelve percent more accurate. Eighty percent cheaper. Same model, same task, the only thing that changed was what sat underneath it.
That’s the headline from a benchmark released this month, and it’s worth sitting with if you’re weighing up your next AI project. The model race gets all the attention right now, GPT versus Claude versus Microsoft’s own MAI models, open-weight versus hosted.
But the real advantage sits somewhere else entirely, and once you see it, you’ll start asking different questions before your next AI rollout.
The Model Race Is a Distraction
Here’s where those numbers came from. Pinecone, a company that builds knowledge infrastructure for AI systems, tested its Nexus product against Sierra’s τ-Knowledge benchmark, an open test that measures how well AI agents handle real customer service work involving multiple steps and strict policy rules.
On the benchmark’s hardest task set, giving GPT-5.2 a proper knowledge layer through Nexus made it 12% more accurate while cutting the cost per task by 80%. GPT-5.5 kept its accuracy steady while cutting costs by 77%. Both models needed roughly half as many tool calls and model calls to reach the same answer.
Pinecone also ran Nexus behind its own customer support queue. The share of tickets its agent could resolve without a person touching them jumped from around a quarter to over half. That’s not a rounding error. That’s the difference between an AI pilot that quietly stalls and one that earns its keep.
Worth being clear on one thing: the model didn’t change in that test. GPT-5.2 was still GPT-5.2. What changed was the quality of the information it could draw on.
What a Knowledge Layer Actually Is
So what does “the knowledge underneath the model” actually mean in practice? Let’s make it concrete, because “knowledge layer” can sound like more jargon if I don’t explain it properly.
Think about everything your business knows. Your policies, your product details, your past customer conversations, your pricing rules, your compliance requirements. Right now, that knowledge probably sits scattered across SharePoint sites, old PDFs, someone’s inbox, a CRM, maybe a spreadsheet nobody’s touched since last year.
When you ask Copilot or any AI agent a question, it has to go find the relevant bits of that scattered knowledge, work out what’s current, and stitch together an answer.
That’s slow, it’s expensive in computing terms, and it’s where a lot of AI answers go wrong. The model isn’t dumb. It’s just being asked to do detective work every single time, with no map to guide it.
A knowledge layer is that map. It’s a structured, governed version of your business knowledge that’s been organised once, so agents can query it directly instead of rediscovering everything from scratch each time.
Done well, it also keeps the relationships between facts intact, flags conflicts, like two documents giving different pricing, and tracks where each answer actually came from.
Microsoft has been building this same idea straight into its own platform, which tells you it’s not a niche concept from one vendor. Foundry IQ, part of Microsoft Foundry, is designed as a shared knowledge layer that multiple AI agents can draw on, built on Azure AI Search.
In Microsoft’s own testing, adding proper query planning and follow-up search through Foundry IQ lifted answer quality by an average of 36% compared to simply searching all sources at once, with the biggest gains on harder questions that need information pulled from more than one source.
CG TECH’s governance work with clients increasingly starts here, with the knowledge layer, rather than with picking a model.
Why This Matters More Than Your Model Choice
Once you’ve seen how much the knowledge layer moves the needle, the model conversation looks a lot less urgent. Here’s the honest breakdown for most businesses.
Your knowledge layer decides how accurate your answers are. An AI model, however capable, can only work with what it’s given. If Copilot can’t see your current pricing sheet, or it’s pulling from three conflicting versions of a policy document, no model upgrade fixes that.
This is exactly why the permissions and structure of your Microsoft 365 content matter so much to how well Copilot performs day to day.
Your knowledge layer decides your costs. The Pinecone results make this plain. Cutting the number of model calls and tool calls an agent needs isn’t just faster, it’s dramatically cheaper.
If your business is running or planning to run agents at any scale, the compute cost of poor retrieval adds up fast.
Your knowledge layer protects you from vendor change. This part matters more than people realise right now. OpenAI recently confirmed it’s slowed development of its next major model, Astra, after internal testing raised safety concerns, and paused its largest planned training run while it builds better monitoring.
Microsoft, meanwhile, keeps shifting which models power different parts of Copilot and Foundry, mixing its own MAI models with OpenAI and Anthropic depending on the task.
If your AI strategy is built around “we use Model X,” you’re exposed every time a vendor changes course. If it’s built around a knowledge layer that any reasonably good model can plug into, you’re not.
Your knowledge layer is what makes agents safe to trust. This connects directly to the security side of AI I’ve written about before. When an AI agent has clear, permission-aware knowledge to work from, rather than open-ended access to search wherever it likes, it’s easier to audit what it did and why.
That’s the same thinking behind giving every agent its own managed identity rather than letting it borrow broad access.
What This Looks Like Inside a Microsoft-First Business
All of that sounds reasonable in theory, so let’s ground it in what it actually looks like day to day.
You don’t need to build a knowledge layer from scratch or buy a specialist product to get started. For most businesses I work with, this shows up in three places already inside their Microsoft environment.
Copilot for Microsoft 365. Copilot only answers well when it can see clean, well-permissioned content. Messy SharePoint sites, outdated file versions and unclear sharing settings all weaken the knowledge layer Copilot draws from, no matter which underlying model is running it.
Agent-based security and operations tools. Microsoft’s agentic security tools, including Project Perception, rely on structured, accurate data about your systems and vulnerabilities to work properly. An agent can only act well on data that’s actually coherent.
Custom agents built on Microsoft Foundry. If your business is building agents for specific tasks, whether that’s answering HR questions or summarising customer accounts, most of the real engineering effort should go into defining what knowledge that agent can access and how it’s structured, not which model sits on top.
A Practical Starting Point
So where do you actually start? If you’re weighing up your next AI project, here’s where I’d begin instead of shopping for a new model.
First, get an honest list of where your critical business knowledge actually lives, and how messy or clean it is.
Second, fix the basics in Microsoft 365 before layering AI on top, tidy up SharePoint permissions, remove duplicate documents, and retire anything outdated.
Third, pick one use case where better knowledge would make the biggest difference, like customer support or policy questions, and pilot it properly with clear measurement of accuracy and cost, not just speed.
Fourth, make your knowledge layer part of your AI governance conversation, not an afterthought your IT team deals with quietly.
Models will keep changing. That’s not going to stop, and it shouldn’t worry you as much as the headlines suggest. What compounds in value over time is the knowledge layer sitting underneath.
Get that right, and you’ll get more from whichever model you’re using this year, and the one after that.
If you want a hand working out what your knowledge layer actually looks like right now, that’s a conversation worth having before your next AI project, not after.
About the Author
Carlos Garcia is the Founder and Managing Director of CG TECH, where he leads enterprise digital transformation projects across Australia.
With deep experience in business process automation, Microsoft 365, and AI-powered workplace solutions, Carlos has helped businesses in government, healthcare, and enterprise sectors streamline workflows and improve efficiency.
He holds Microsoft certifications in Power Platform and Azure and regularly shares practical guidance on Copilot readiness, data strategy, and AI adoption.
Sources
- Pinecone: Nexus GA — It’s the Knowledge, Not the Models – τ-Knowledge benchmark results on accuracy, cost, tool calls and ticket resolution.
- Microsoft Tech Community: Foundry IQ response relevance results – Microsoft’s own benchmark data on agentic retrieval performance gains.
- Microsoft Azure: Foundry IQ product page – overview of Foundry IQ as a unified knowledge layer for agents.
- Tech Xplore: OpenAI slows advanced AI development after its tools launched a cyberattack – reporting on OpenAI’s Astra pause and safety concerns.