Alibaba Group Holding (NYSE:BABA) has introduced a new computing pooling system called Aegaeon, which dramatically reduces the reliance on Nvidia (NASDAQ:NVDA) GPUs by 82% for AI models.
Three Months In Testing
This innovation was tested in Alibaba Cloud's model marketplace for over three months, according to a research paper presented this week at the 31st Symposium on Operating Systems Principles (SOSP) in Seoul, South Korea.
The Aegaeon system has successfully decreased the number of Nvidia H20 GPUs required from 1,192 to just 213 for serving models with up to 72 billion parameters.
"Aegaeon is the first work to reveal the excessive costs associated with serving concurrent LLM workloads on the market," the researchers stated in the paper.
Researchers from Peking University and Alibaba Cloud emphasized the high costs associated with serving concurrent large language model workloads.
One GPU For Multiple Models
Alibaba Cloud, the AI and cloud services division of the Hangzhou-based Alibaba, aims to boost efficiency by pooling GPU resources, allowing a single GPU to support multiple models.
The system addresses resource inefficiency, as previously, 17.7% of GPUs were allocated to serve only 1.35% of requests in Alibaba Cloud's marketplace.
Cloud service providers like Alibaba Cloud and ByteDance‘s Volcano Engine manage thousands of AI models simultaneously, often leading to inefficiencies. The Aegaeon system seeks to optimize this process by reducing the number of GPUs needed.
Growing Concerns Over Nvidia's China Presence
This development comes amid growing concerns over Nvidia’s presence in China. Recently, China raised security concerns about Nvidia’s H20 chips, especially regarding potential backdoor risks. As part of its deal with Nvidia, the Trump administration has struck an agreement for a 15% revenue share from the company's chip sales to China.
Nvidia CEO Jensen Huang stated that Nvidia’s market share in China has plummeted from 95% to zero. He expressed concerns over the impact of U.S. policies on Nvidia’s market presence in China.
Despite these challenges, Nvidia has financially insulated itself from potential escalations, as its guidance assumes zero revenue from China, according to Huang.
Read Next:
Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors.
Photo courtesy: Shutterstock
