Twice a year, NVIDIA, a Silicon Valley-based technology company that pioneered and popularized the use of Graphics Processing Units (GPUs), runs a conference called the GPU Technology Conference (GTC) aimed to showcase advancements in the use of processing technology for emerging technology use cases. It should come as no surprise that recent GTC events have focused on AI, given how compute-intensive machine learning training and inference can be. Each year, it seems that the GTC events have become more and more about AI only, to the virtual exclusion of other areas where GPUs are being applied. That being said, there were a number of interesting announcements that came out of the NVIDIA GTC 2019 event that are of interest beyond those that follow the company’s own products and solutions.
US Postal Service is using AI to speed up mail handling
One of the most interesting announcements was not about a new product or feature. Rather, the company announced that the US Postal Service (USPS) is now using AI and computer vision technology to process around 146 billion pieces of mail and 6 million packages a day. Using image recognition powered by machine learning, the USPS claims that it will be able to process these items up to 10 times faster and with greater accuracy than currently available. Specifically, the solution uses AI to read labels which assists in automatic routing and sorting. The system will be fully operational by end of spring 2020.
AI performance continues to improve at the edge
Another interesting innovation happening across the AI market is increasing performance of machine learning models, both for training as well as in inference phase, also known as operationalization. Many companies realize that the power of AI shouldn’t just be leveraged in the data center or in the cloud, but also on devices themselves. Edge devices ranging from mobile phones to cameras and robots are increasingly requiring the use of machine learning models while disconnected from the Internet. To do so requires processing power that can handle increasingly complicated machine learning models.
Last year, NVIDIA released their Jetson product which reduces the form factor for edge-based AI to a small size at an equally small price, while packing a performance punch. This year, they announced the Jetson Xavier NX, which the company claims is the world’s smallest “AI supercomputer” focused on robotic and embedded computing devices at the edge. Smaller than a credit card, the company’s AI edge computing device can deliver performance usually associated with server machines while consuming as little as 10 watts of power. It will be interesting to see what sort of applications find themselves suited to these sort of edge computing needs.
The battle of the benchmark
There has been a bit of a boom lately in hardware companies focused on building AI chipsets. While many of these hardware companies are focused on specific AI application areas such as computer vision, natural language processing, sensor handling, and other focused areas, some are focused on building general purpose machine learning processing power capable of handling a wide range of model training and inference workloads. As a result of this diversity there has been greater emphasis on the use of benchmarks to compare performance across the different technology implementations.
Created in 2018, the MLPerf benchmark aims to measure performance for both training and inference in a way that can be independently verified and replicated. Formed as a collaboration of companies and researchers from educational institutions, the benchmark measures how fast the processors are able to handle standardized workloads for ML training as well as inference. It goes without saying that as technology vendors continue to make significant improvements to their chipsets, they continue to battle each other for benchmark performance.
At this NVIDIA GTC DC 2019 event, the company claims to have bested the benchmark for ML inference performance, in addition to its performance in the training benchmark. MLPerf has five inference benchmarks that focus on four inferencing scenarios including image classification, object detection and translation. As can be expected, we might see similar announcements soon from other vendors who are working to increase their own performance.
The market for models is heating up
NVIDIA wasn’t the only company making announcements at the event. Booz Allen made waves with its announcement of Modzy, a platform for managing machine learning models in operation as well as a marketplace for curated models. While much of the focus of AI efforts have been applied to the development and creation of machine learning models, the attention now is shifting to the needs of those who are more concerned with consuming and using models developed by others. As a result, we’re starting to see the emergence of a new class of solutions focused on machine learning model usage and consumption, so-called ML “Ops” platforms.
Focusing on this need, Booz Allen Hamilton, a technology and management consulting and engineering firm, announced their Modzy platform which focuses on the needs of public and private sector companies to rapidly deploy, manage and secure AI models. The Modzy platform provides functionality for organizations to not only scale their own ML models, but also provides a model “marketplace” that consists of customer models, Booz Allen models, and curated models developed by partners including NVIDIA, HyperGiant, Paravision, Orbital Insight, AI.Reverie, AppTek, and others.
As AI continues to be something companies of all sizes are eager to invest in, we expect to see more events that have traditionally been product focused to give greater emphasis on AI. Given this observation, it should be no surprise that NVIDIA GTC DC had a heavy AI focus.
(Disclosure: I have no commercial relationship with NVIDIA. Cognilytica has a relationship with Booz Allen)
