Every now and then, a new product will be launched, boasting a nifty bit of tech. But rather than making me think about how cool it all is, I just end up going, 'Hang on, haven't we seen this before?' One such example is a new PCIe 5.0 switch for data centers, that allows data to stream directly from SSDs to GPUs, using a system called GPUDirect. That's exactly what Nvidia's RTX IO was supposed to do, except it doesn't seem to any more, which might explain why it's pretty much disappeared from everyone's radar.
The switch in question is a HighPoint Rocket 7638D, as reported by Tom's Hardware. It's of no use to a gaming PC, as it's exclusively for Nvidia's AI GPUs that support a bit of tech called GPUDirect. It basically provides a direct data path between storage, such as an NVMe, and a GPU's VRAM, neatly bypassing the CPU and system memory. It's been around since 2019, a little earlier than RTX IO.
That was a much-hyped feature of GeForce RTX 30-series GPUs, and Nvidia expressly said that it allows "streaming directly from SSD to GPU memory." However, if you read the current webpage for RTX IO, that's not mentioned at all, and instead just focuses on its other performance-enhancing aspects.
A couple of years ago, RTX IO was demonstrated using Valve's Portal to highlight the speed at which assets can be loaded into VRAM, but all the talk was about GPU decompression and the saving of CPU cycles. The whole 'SSD directly to GPU' thing was notable by its total absence.
Which is all very puzzling. Why does Nvidia's data centre stuff get GPUDirect but its gaming products don't? Why was this part of RTX IO to begin with, but then gets quietly dropped? In fact, what's going on with RTX IO full stop? I suspect the answer lies with Microsoft's DirectStorage.
That's an API which provides a far more streamlined way of transferring data from storage to GPU memory. Unlike GPUDirect, the CPU and system RAM aren't skipped, but the API handles the calls for data in a far more parallel manner than is normally done, reducing the workload on the CPU.
A revision of DirectStorage added GPU decompression, where files can remain compressed until they're in VRAM and then get unpacked via shader cores. One of the compression formats DirectStorage supports is GDeflate, which was developed by Nvidia for RTX IO.
None of this explains why "streaming directly from SSD to GPU memory" is missing from RTX IO now, of course, but it could be down to it not working very well, or a lack of interest from game developers. After all, DirectStorage isn't widely used even now, and the few games that do use it typically don't enable GPU decompression.
Anyway, while I'm waiting for RTX IO to be used properly in a full game, I'll just sit here staring jealously at consoles and data centres, all enjoying blisteringly fast load times and SSD-GPU harmonious data flinging. I know it sounds a bit like I'm flinging my toys out of my pram, but it'd be nice to have the toys I was promised in the first place.
