Hello and welcome to Eye on AI.
Today we’re starting with some breaking news in the world of generative AI. The U.S. FTC this afternoon announced it’s launching an inquiry into generative AI partnerships between Big Tech and startups. Specifically, the agency is investigating three multi-billion dollar deals that have shaped the AI landscape as we know it: Microsoft and OpenAI, Google and Anthropic, and Amazon and Anthropic. The FTC issued orders to all of the involved companies, seeking specifics about their agreements, the practical implications of these partnerships, analysis of the transactions’ competitive impact, competition for AI inputs and resources, and more information. This investigation could have major ramifications for these companies and the AI and technology landscape—and we’ll be paying close attention as it pans out.
Now, let’s get into our main story, which is in some sense related to the FTC’s concerns about concentration of power in the nascent generative AI market: We’re talking about models, and more specifically, why most companies have quit building them and what this means for gaining a competitive advantage in AI. To start, here’s a staggering statistic: according to Menlo Ventures’ recent survey of more than 450 enterprise executives, almost 95% of AI spend is now on inference, or running AI models, as opposed to training them.
“People are not building models from scratch for the most part anymore,” Tim Tully, an engineer and partner at Menlo Ventures told Eye on AI. “We see it empirically. You see it through the survey data. We see it from talking to companies. It’s plainly obvious.”
Creating an end-to-end model from scratch is massively resource intensive and requires deep expertise, whereas plugging into OpenAI or Anthropic’s API is as simple as it gets. This has prompted a massive shift from an AI landscape that was “model-forward” to one that’s “product-forward,” where companies are primarily tapping existing models and skipping right to the product roadmap. By 2027, the total value of APIs to AI software specifically will reach an estimated $5.4 trillion, representing 76% growth in five years, according to a report from open-source API company Kong.
This shift has been incredibly lucrative for companies like OpenAI, but for everyone else, it’s a massive sea change that’s forced many companies to quickly turn the ship around. For example, Menlo portfolio company TrueEra, which provided machine learning observability capabilities for companies training models in-house, had to completely pivot its product strategy when its customers started using existing models instead of building their own, according to Tully.
There’s an argument to be made that this shift levels the playing field, making it possible for any company of any size to access and deploy advanced AI. After all, everyone is now just an API away from best-in-class models. But if everyone is using the same models, what will be the competitive differentiator?
One aspect is how well you can prompt engineer, which explains the mad rush of prompt engineering research, training, and the creation of high-priority prompt engineering roles that are fetching salaries over $300K. But as it goes in AI, it really comes back to the data, particularly the proprietary data you have access to and how well you can incorporate it.
“It’s what kind of documents can you come up with? How well can you parse and extract data from those documents? How can you convert the unstructured data to structured?” Tully said.
A recent article in Harvard Business Review on how companies can turn generative AI into a competitive advantage similarly boils down to 1. adopt publicly available tools and 2. supercharge them with your own data.
Last fall, I wrote about RAG (retrieval-augmented generation), a now incredibly popular technique for getting an existing AI model to work with new information it was never trained on. RAG is currently a major driver of how companies can incorporate their own data and get the most out of their off-the-shelf models, but AI practitioners are already wondering what comes next.
“We have models, but then what? How do you use them in increasingly interesting ways that build sophisticated applications over time?” Tully said. “I think the question that I have is, how do you evolve RAG in a way that helps applications continue to differentiate? That’ll be something to keep an eye on.”
And with that, here’s more AI news. Today’s issue also premiers two new sections for Eye on AI—one to help you get the most out of LLMs like ChatGPT, and another to keep you up-to-date on the most important AI-related events coming up soon.
Thanks for reading!
Sage Lazzaro
sage.lazzaro@consultant.fortune.com
sagelazzaro.com
