A few clients have asked me some version of the same question lately. If Copilot already does most of what we need, why would we ever want to run our own AI model?
It’s a fair question, and it’s come up again this month for a good reason. Microsoft just made OpenAI’s open-weight models, gpt-oss-120b and gpt-oss-20b, available on Azure AI Foundry and Windows AI Foundry.
Anthropic also weighed in on the open-weight debate in July, clarifying its own position after facing criticism. Between the two, it’s becoming clear that “open-weight AI” is turning into a real business decision, not just a developer trend.
So let’s break down what open-weight AI actually is, how it’s different from the hosted Copilot experience most businesses already use, and where each one makes sense if you’re running on Microsoft 365.
Hosted AI in Plain English
Most businesses already use hosted AI without thinking about it that way. When you use Microsoft 365 Copilot, ChatGPT, or Claude through the browser or an app, you’re using a hosted model. The provider runs the model on their own infrastructure. You send a prompt, they process it, and you get an answer back.
Hosted AI is simple by design. You don’t manage servers, you don’t patch anything, and you get updates automatically when the provider releases them.
That’s exactly why Copilot works the way it does inside Word, Excel, Teams and Outlook. Microsoft handles the model, the compute and the security, and your business just switches it on for the right licence.
The trade-off is control. You’re relying on the provider’s data handling, their uptime, and their decisions about what the model can and can’t do.
For most day-to-day tasks, like drafting emails, summarising meetings or building a first-pass spreadsheet, that trade-off makes complete sense. You don’t need to own the plumbing to get the value.
Open-Weight AI in Plain English
Open-weight AI works differently. Instead of only accessing a model through someone else’s service, you get the actual model file, or “weights,” and you can run it yourself. That might be on a cloud platform like Azure, on your own servers, or in some cases directly on a laptop.
This is exactly what happened with gpt-oss-120b and gpt-oss-20b. OpenAI released the model weights publicly, and Microsoft made both available through Azure AI Foundry, with gpt-oss-20b also running through Windows AI Foundry for local use on capable hardware.
Microsoft describes gpt-oss-120b as delivering performance close to its o4-mini model, but small enough to run on a single enterprise-grade GPU. gpt-oss-20b is built to be lighter still, aimed at tasks like code execution and tool use, and can run on Windows devices with a solid graphics card rather than needing a data centre at all.
Open-weight doesn’t mean “free for all” or “no rules.” Most releases still come with licence terms, and running a model yourself means you take on the responsibility for hosting it securely, keeping it updated and monitoring how it’s used.
What changes is where the model actually lives, and how much visibility and control you have over it.
Why Anthropic’s Position Matters Too
It’s worth mentioning that open-weight AI isn’t just an OpenAI and Microsoft story.
In late July, a coalition of tech companies, including Nvidia, Microsoft, Meta and Palantir, published a letter urging policymakers not to rush into restrictions on open-weight models, partly in response to debate over Chinese open-weight releases.
Anthropic didn’t sign that letter, and its CEO Dario Amodei followed up with his own post making Anthropic’s position clear. The company has never called for banning open-weight models as a category, even though it doesn’t release open-weight versions of Claude itself.
Amodei’s actual concerns were narrower than a general stance on open models.
He pointed to keeping advanced chips out of authoritarian governments’ hands, cracking down on large-scale “distillation,” where a smaller model is trained by copying a bigger one’s outputs, and mandatory safety testing for any sufficiently capable model, open or closed.
That’s a useful data point for business leaders, even if Anthropic itself sits on the closed side of the ledger. It shows that “open-weight AI” isn’t a simple two-team contest between open and closed vendors.
It’s a live policy debate with real disagreement about risk, and it’s happening at the same time as Microsoft is actively building open-weight options into its own platform.
Where This Fits Into a Microsoft-First Strategy
Here’s the important part for anyone running a Microsoft-first business.
You don’t have to choose one approach and abandon the other. Microsoft’s own direction backs this up. Azure AI Foundry now sits alongside Copilot as a platform where you can mix hosted models, like GPT-5.6 or Claude, with open-weight models such as gpt-oss or Mistral’s models, depending on the task.
Think about it as two different tools in the same toolbox rather than a fork in the road.
Hosted AI, through Copilot or a similar service, remains the better fit when:
- Your team needs AI inside the apps they already use every day, like Word, Excel, Teams and Outlook
- Speed of rollout matters more than deep customisation
- You don’t have in-house capability to manage model deployment, fine-tuning or monitoring
- The task is genuinely everyday work rather than something specialised to your business
Open-weight AI is worth exploring when:
- Data residency or sovereignty is a genuine requirement, not just a nice-to-have
- You want a model fine-tuned on your own data or documents, rather than a general-purpose assistant
- You’re running AI in a bandwidth-constrained or offline environment, such as a remote site or a secure network
- Cost predictability matters more than convenience, since running your own model shifts spend from per-use fees to infrastructure and management
If you’ve already been through the work of building an AI operating model for Microsoft 365, this fits neatly into that same structure.
Your decision owner now has one more lever to pull. It’s not just “which Copilot feature do we turn on,” it’s “does this task need a hosted assistant, or would it be better served by a model we control ourselves.”
The Governance Question Doesn’t Go Away
If anything, running your own model raises the governance bar, not lowers it.
With hosted AI, a lot of the security and compliance heavy lifting happens on the provider’s side, which is part of why Microsoft has been able to fold Claude into Microsoft 365 Copilot with a consistent set of admin controls around it.
With open-weight AI, your business owns more of that responsibility directly.
That means asking practical questions before you deploy anything: who has access to the model and its outputs, how is usage logged and reviewed, what data does it get exposed to during fine-tuning, and who’s responsible if something goes wrong.
These aren’t new questions if you’ve already worked through governance for Copilot or Cowork. They just apply to a slightly different environment.
The same multi-model thinking is showing up elsewhere in the Microsoft ecosystem too.
The expanded Microsoft-Mistral partnership already lets Azure customers run certain models in a cloud-connected or fully disconnected setup, depending on how sensitive the workload is.
Open-weight AI through Foundry is really an extension of that same idea: more choice over where your AI actually runs, not just which brand name is on the box.
What To Do Next
You don’t need to become a machine learning shop overnight, and for most businesses, hosted Copilot will remain the right day-to-day tool for a long time yet.
But it’s worth putting one question on your radar for the next planning cycle: are there any tasks in your business where owning the model, rather than renting access to one, would genuinely change the outcome?
That might be a compliance-heavy workflow, a specialised internal tool, or a use case where your data simply shouldn’t leave your own environment.
If the honest answer is “not yet,” that’s a completely reasonable place to be. If the answer is “maybe,” that’s worth a proper conversation before you commit budget either way.
We’re always happy to have that conversation.
Whether it’s reviewing your current Copilot setup, working through an AI operating model, or scoping a pilot for an open-weight model on Azure, get in touch and we’ll help you figure out where the value actually is for your business.
About the Author
Carlos Garcia is the Founder and Managing Director of CG TECH, where he leads enterprise digital transformation projects across Australia.
With deep experience in business process automation, Microsoft 365, and AI-powered workplace solutions, Carlos has helped businesses in government, healthcare, and enterprise sectors streamline workflows and improve efficiency.
He holds Microsoft certifications in Power Platform and Azure and regularly shares practical guidance on Copilot readiness, data strategy, and AI adoption.
