Hello and welcome to Eye on AI…In this edition: DeepSeek drops another impressive model…China tells companies not to buy Nvidia chips…and OpenEvidence scores an impressive result on the medical licensing exam.
Hi, it’s Jeremy here, just back from a few weeks of much needed vacation. It was nice to be able to get a little distance and perspective on the AI news cycle. (Although I did make an appearance on Rana el Kaliouby’s “Pioneers of AI” podcast to discuss the launch of GPT-5. You can check that out here.)
Returning this week, the news has been all about investor fears we’re in an “AI bubble”—and that it is about to either pop or deflate. Nervous investors drove the shares of many publicly-traded tech companies linked to AI-related trades, such as Nvidia, CoreWeave, Microsoft, and Alphabet down significantly this week.
To me, one of the clearest signs that we are in a bubble—at least in terms of publicly-traded AI stocks—is the extent to which investors are actively looking for reasons to bail. Take the supposed rationale for this week’s sell-off, which were Altman’s comments that he thought there was an AI bubble in venture-backed, privately-held AI startups and that MIT report which found that 95% of AI pilots fail. Altman wasn’t talking about the public companies that stock market investors have in their portfolios, but traders didn’t care. They chose to only read the headlines and interpret Altman’s remarks broadly. As for that MIT report, the market chose to read it as an indictment of AI as a whole and head for the exits—even though that’s not exactly what the research said, as we’ll see in a moment.
I’m going to spend the rest of this essay on the MIT report because I think it is relevant for Eye on AI readers beyond its implications for investors. The report looked at what companies are actually trying to do with AI and why they may not be succeeding. Entitled The GenAI Divide: State of AI in Business 2025, the report was published by MIT Media Lab’s NANDA Initiative. (My Fortune colleague Sheryl Estrada was one of the first to cover the report’s findings. You can read her coverage here.)
NANDA is an acronym for “Networked-Agents and Decentralized AI” and it is a project designed to create new protocols and a new architecture for an internet full of autonomous AI agents. NANDA might have an incentive to suggest that current AI methods aren’t working—but that if companies created more agentic AI systems using the NANDA protocol, their problems would disappear. There’s no indication that NANDA did anything to skew its survey results or to frame them in a particular light, but it is always important to consider the source.
Ok, now let’s look at what the report actually says. It interviewed 150 executives, surveyed 350 employees, and looked at 300 individual AI projects. It found that 95% of AI pilot projects failed to deliver any discernible financial savings or uplift in profits. These findings are not actually all that different from what a lot of previous surveys have found—and those surveys had no negative impact on the stock market. Consulting firm Capgemini found in 2023 that 88% of AI pilots failed to reach production. (S&P Global found earlier this year that 42% of generative AI pilots were abandoned—which is still not great).
You’re doing it wrong
But where it gets interesting is what the NANDA study said about the apparent reasons for these failures. The biggest problem, the report found, was not that the AI models weren’t capable enough (although execs tended to think that was the problem.) Instead, the researchers discovered a “learning gap”—people and organizations simply did not understand how to use the AI tools properly or how to design workflows that could capture the benefits of AI while minimizing downside risks.
Large language models seem simple—you can give them instructions in plain language, after all. But it takes expertise and experimentation to embed them in business workflows. Wharton professor Ethan Mollick has suggested that the real benefits of AI will come when companies abandon trying to get AI models to follow existing processes—many of which he argues reflect bureaucracy and office politics more than anything else—and simply let the models find their own way to produce the desired business outcomes. (I think Mollick underestimates the extent to which processes in many large companies reflect regulatory demands, but he no doubt has a point in many cases.)
This phenomenon may also explain why the MIT NANDA research found that startups, which often don’t have such entrenched business processes to begin with, are much more likely to find genAI can deliver ROI.
Buy, don’t build
The report also found that companies which bought-in AI models and solutions were more successful than enterprises that tried to build their own systems. Purchasing AI tools succeeded 67% of the time, while internal builds panned out only one-third as often. Some large organizations, especially in regulated industries, feel they have to build their own tools for legal and data privacy reasons. But in some cases organizations fetishize control—when they would be better off handing the hard work off to a vendor whose entire business is creating AI software.
Building AI models or systems from scratch requires a level of expertise many companies don’t have and can’t afford to hire. It is also means that companies are building their AI systems on open source or open weight LLMs—and while the performance of these models has improved markedly in the past year, most open source AI models still lag their proprietary rivals. And when it comes to using AI in actual business cases, a 5% difference in reasoning abilities or hallucination rates can result in a substantial difference in outcomes.
Finally, the MIT report found that many companies are deploying AI in marketing and sales, when the tools might have a much bigger impact if used to take costs out of back-end processes and procedures. This too may contribute to AI’s missing ROI.
The overall thrust of the MIT report was that the problem was not the tech. It was how companies were using the tech. But that’s not how the stock market chose to interpret the results. To me, that says more about the irrational exuberance in the stock market than it does about the actual impact AI will have on business in five years time.
With that, here’s the rest of the AI news.
Jeremy Kahn
jeremy.kahn@fortune.com
@jeremyakahn
