Of all the debates raging about the potential downsides of artificial intelligence, one concern has been causing the most hand-wringing among AI enthusiasts in Silicon Valley: the fear that the giant AI labs selling proprietary models are acting like Trojan horses. The worry is that as startups and enterprises use AI models from labs such as OpenAI and Anthropic, those labs gain ever-increasing access to the companies’ most sensitive business information. The model makers can then use that knowledge for themselves, potentially becoming competitors to their own customers. Prominent voices like venture capitalist Jason Calacanis and Palantir CEO Alex Karp have already issued such warnings.

Now, in a surprising blog post published on Sunday, Microsoft CEO Satya Nadella has joined this crowd. Nadella warns that AI users — whom he calls the “buyers” — are paying twice. They knowingly spend money on AI token usage, but they also, obliviously, hand over valuable data in the process. “You essentially pay for intelligence twice, once with money, and again with something even more valuable: the proprietary knowledge you must reveal to make that intelligence useful. The better you want the model to perform, the more of that knowledge you have to feed it,” he writes. Most dangerously, enterprises are literally teaching the models about the nuances of their businesses.

The Hidden Cost of Proprietary Models

Nadella elaborates that models learn from “exhaust” — the prompts people write, the tools agents use, and especially the corrections people make when the model is wrong. “Every correction is distilled into institutional know-how,” he writes. This is “the kind of knowledge a competitor could never buy,” yet enterprises are handing it over without realizing it. The core of his argument is that if AI companies are allowed to freely scrape the internet to train their models, it is only fair that enterprises get to study — or “distill” — those models in return. Distillation is the practice of using a model’s own outputs to learn how it works and to train a new, often cheaper, model based on those insights.

In February 2026, Anthropic accused Chinese open-source models of sending millions of prompts to Claude as a way to improve their own models, urging the U.S. government to crack down on export controls. Nadella’s point is that model makers cannot have it both ways: it is hypocritical for them to freely train on the world’s data while restricting others from doing the same to their models. “While the great innovation that comes from model providers having fair use rights to train models on public data is needed, I find it ironic that the status quo is to then turn around and impose restrictive terms on distillation,” he writes. Nadella is particularly concerned when model makers “reserve the right to learn from customer usage and interaction data.”

Nadella’s Proposed Solutions

Nadella’s solution is the kind of thing the CEO of a giant cloud provider would suggest. He wants companies to “retain ownership” of their data, including prompts, feedback, and other interaction data. To achieve this, he urges them to build their own “proprietary learning environments” on the cloud — where their data is likely already stored anyway and, conveniently, could mean Microsoft’s cloud, Azure. He also wants companies to build in what he calls “orchestration layers” — essentially, a way to easily switch between AI models from different providers rather than being locked into one. Tools like AI “gateways” that let companies do exactly this have become increasingly popular.

While Nadella never uses the words “open source” as the method for retaining ownership, this is an obvious subtext. Yet there is another subtext. Large companies, many of which still have their own data centers in addition to using the cloud, are already moving to open-source models installed on their own premises (known as “on-prem” in industry jargon). Idit Levine, founder and CEO of Solo.io — a company that makes networking and security software to help enterprises manage AI systems — says she is seeing exactly this shift play out with her own customers. After experimenting with proprietary model makers, they start asking themselves: “Can I take an open-source model and run it on-prem? It will do almost 90% of what the big one’s doing. It will cost way less,” she explains. “They understand that, and they can control it.”

The Rise of On-Prem Open-Source Models

Solo.io’s technology was selected by the Linux Foundation to power its Agentgateway project. Her company counts enterprises like T-Mobile, ADP, and SAP as customers. She sees companies increasingly installing on-premise open-source models and considers it the next big wave in enterprise AI use. She is not alone. Vercel, best known as a platform for building and hosting websites that has recently added AI model-switching tools, and OpenRouter, a company that helps developers route requests across different AI models, are both seeing a surge in traffic to open-source models. In fact, open models accounted for 29% of all traffic routed through Vercel’s gateway last month.

This trend is driven by a growing awareness of the hidden costs of proprietary AI. Beyond the financial cost, the loss of proprietary knowledge can be detrimental to a company’s competitive advantage. Enterprises are realizing that every prompt, every correction, every interaction with a proprietary model feeds the model maker’s knowledge base, potentially enabling them to offer competing services. The fear is that the very models meant to empower businesses are being used to extract their trade secrets.

Historical Context and Broader Implications

Satya Nadella’s warning is particularly striking given Microsoft’s deep investments in both OpenAI and Anthropic. As the CEO of a company that has poured billions into proprietary AI, his public shift in tone signals a significant strategic reconsideration. This move aligns with Microsoft’s broader cloud strategy: by encouraging customers to build proprietary learning environments on Azure, Microsoft positions itself as the neutral platform that hosts enterprise data, while still benefiting from the AI ecosystem. However, it also reflects a genuine concern that the current model of AI consumption is unsustainable for the long-term health of the industry.

The concept of distillation, while technically complex, is central to Nadella’s argument. Distillation allows a smaller, cheaper model to learn from a larger, more powerful one by analyzing its outputs. Proponents argue that this democratizes access to AI, enabling smaller players to build competitive solutions without paying exorbitant fees. Critics, however, warn that unrestricted distillation could lead to a loss of intellectual property for model makers and undermine their business models. The debate over distillation rights is likely to intensify as more enterprises seek to extract value from the models they use.

Another key factor is the rapid advancement of open-source AI. Models like Llama (from Meta), Mistral, and others have narrowed the performance gap with proprietary offerings. For many enterprise use cases, an open-source model running on-premises can achieve 90% of the performance at a fraction of the cost. This cost advantage, combined with the ability to retain control over data, is driving the migration toward on-premise deployments. Additionally, regulatory pressures around data sovereignty and privacy are pushing companies in Europe and elsewhere to keep their data within their own infrastructure.

The broader implication of Nadella’s warning is that the era of unquestioning trust in proprietary AI vendors may be coming to an end. Enterprises are becoming more sophisticated in their understanding of how AI models work and what they cost beyond the invoice. The hidden cost of data leakage is now a boardroom-level concern. As more executives read Nadella’s words, they are likely to demand audits of their AI usage and explore alternatives that offer greater control.

Meanwhile, the response from the proprietary AI labs has been cautious. OpenAI and Anthropic have both defended their data practices, arguing that they have strict privacy policies and that customer data is not used to train models without explicit consent. However, critics point out that the terms of service often include broad rights to analyze usage patterns, which can still reveal sensitive information. The line between anonymized usage data and proprietary knowledge is blurry, and enterprises are right to be wary.

Nadella’s proposed solution — building proprietary learning environments with orchestration layers — is technically feasible but requires significant investment. Companies need to set up their own infrastructure, integrate multiple AI models, and manage the complexity of orchestration. This is where cloud providers like Microsoft Azure come in, offering managed services that simplify the process. But it also opens the door for new startups that specialize in AI gateways and data privacy tools. Solo.io, Vercel, and OpenRouter are examples of companies capitalizing on this trend.

In conclusion (though we refrain from a formal conclusion, the article ends with a natural thought), the shift toward on-premise open-source models and distillation rights is reshaping the enterprise AI landscape. With the CEO of Microsoft now openly urging caution, the momentum behind this movement is only likely to grow. Nadella’s final words resonate: “In consuming intelligence, you are creating intelligence. And what you create should belong to you.” Enterprises would do well to heed his advice.

Source:TechCrunch News