I recently had a chance to speak with Hrishikesh Dewan, founder and CEO of Ziroh Labs, where we discussed their new platform, Kompact AI. It’s a tool developed that enables the execution of powerful AI models on regular computers (CPUs) rather than expensive graphics cards (GPUs). My conversation with Mr. Dewan challenged many assumptions we normally have about how AI should run. So, let’s take a closer look at what Kompact AI is, its benefits, limitations, and more from Mr. Dewan’s perspective.
Making AI Work Without the GPU
One thing that Mr. Dewan made clear from the start is that Kompact AI is not a new AI model. It’s a platform that allows existing large language models (LLMs) to run efficiently on CPUs. Yes, you read that right: regular processors, not expensive GPUs.
When I asked him to explain Kompact AI in simple terms, he broke it down beautifully: AI models are essentially math, primarily linear algebra and probability equations. Unlike other software systems that deal with databases or files, these models are just equations computed one after the other. Kompact AI is about making those equations run fast and accurately on CPUs.
However, what makes the Kompact AI unique is that it preserves the quality of results while speeding up performance even on less powerful hardware. “Throughput and quality are the two key metrics,” he told me. Throughput measures how fast the model returns an output, and quality measures how accurate that output is. Kompact AI adheres to model benchmarks established by the original designers, and in some cases, such as Meta’s Llama, it has even outperformed the original.
Privacy by Design
As the use of AI tools grows, concerns about privacy are also gaining momentum. So, when I asked Mr. Dewan about the privacy aspect of Kompact AI, he had a very clear viewpoint:
When you send input to an external AI model, you’re giving up control over that data. Your input could be a question, photo, or personal data, and it’s all part of your intellectual property. With Kompact AI, since models run locally on your own machine, the input stays with you. This model of local inference protects users from privacy risks, especially as generative AI becomes more capable of manipulating personal content.
Hrishikesh Dewan, founder and CEO of Ziroh Labs
Limitations—and Why That’s Okay
When I asked Mr. Dewan about the limitations of Kompact AI, I got to know that it currently supports models up to 50 billion parameters, not the massive trillion-parameter giants. However, Mr. Dewan pointed out that today, most real-world applications use much smaller models, which are more efficient for specific use cases, such as medical diagnostics, radiology, or code analysis.
Does this mean that going forward, we will only need small models? Well, the answer is no. Mr. Dewan believes that both large and small models will coexist. Big models might be great at writing jokes, but for mission-critical sectors like healthcare, smaller, local models are often the better fit, especially due to regulatory concerns.
In places like India, South America, or Africa, where GPU infrastructure is limited, Kompact AI allows people to use what they already have—CPUs. “You don’t need a GPU for everything,” he repeated. And he’s right. Many practical AI tasks, like diagnosing an image or generating a few lines of text, don’t require massive hardware.
Future of Software Development
At the end of our chat, I asked him about the rise of low/no code tools and what will be the future of software development. His answer was simple: these platforms are great for removing “boilerplate,” the boring, repetitive coding tasks. But when it comes to AI-generated apps, he said building is one thing, and making sure they actually work is another. He believes that going forward, developers may need to focus more on reading and debugging AI-generated code instead of writing from the ground up.
