AI takes center stage at Computex 2024
Computex is an exciting venue for tech companies to showcase their latest and greatest and this year, generative AI is taking center stage. The biggest chip design and manufacturers in the world are showing how they are supporting AI training and inferencing on everything from power-sipping laptops to 100kW racks. The path forward to continue AI development will not only require bigger and more powerful supercomputers to help train and finetune cutting-edge models but also the ability for models created on these systems to run efficiently in consumer devices. These two use cases will require vastly different chips, but as demonstrated at Computex, the chipmakers of the world are rising to the challenge.
Betting big on VRAM
Training and inferencing of large generative AI models requires both massive quantities of fast VRAM but also the bandwidth to use it. While the obvious answer me seem to be simply putting more memory on a single accelerator, current leading generative AI models, to say nothing of those in the near future, are already far too large to fit on the amount of memory that designers (or the laws of physics) allow to be placed on a single PCB.
Nvidia has long offered NVLink and NVLink Switch as a solution to this, providing massive interconnect bandwidth to up to 8 discrete GPUs in a single compute node at a time, allowing them to function as a single, virtual, accelerator with the aggregated resources of the entire group. Now though, they are scaling that even further, up to a staggering 576 GPUs per NVLink Switch, allowing multiple racks of GPUs to function as a single, massive accelerator by providing up to 1.8TB/s of GPU-to-GPU bandwidth and up to 1PB/s through the switch itself. Combined with Nvidia’s own custom GPU compute-optimized networking, their next generation accelerators will be ready to tackle even the largest generative AI models.
While Nvidia focuses on a scalable network, AMD is investing in a bigger accelerator and open standards. AMDs latest Instinct MI325X AI accelerator provides up to 288GB of HBM3E VRAM on a single card, more than double what Nvidia’s current top offering, the H200, provides. With such a large quantity of memory onboard, AMD claims their new card can run up to 1-trillion parameter AI models without needing to split the work across multiple accelerators. Even while showcasing such a powerful single device, AMD is also joining a consortium of tech leaders including Intel, Meta, and Microsoft, to develop and push a new industry standard to compete with the proprietary NVLink called UALink. UALink will provide an open protocol for AI accelerators to talk to each other, allowing for easy expansion of accelerator pods to meet the needs of demanding workloads in the future.
Efficient acceleration on a (power) budget
AI isn’t just about the datacenter. As more and more AI applications move away from large theoretical models in the datacenter and gain a place in everyday professional and consumer software, being able to quickly and efficiently leverage compact AI models on a local device is a critical next step in adoption.
Both AMD and Intel are bringing AI-accelerating NPUs to their next generation of mobile processors that will power everything from inexpensive consumer laptops to high-end mobile workstations. These accelerators can provide up to 50 TOPS (Tensor Operations per Second) of AI performance within the tiny power envelope of a laptop, often below 50w for the entire CPU package, while still providing significant acceleration to integrated AI and neural networks embedded within end-user software. Beyond specific applications, NPUs are required to enable the new AI features of Microsoft’s Copilot+ PCs, including an integrated copilot, Recall, automated translation and captioning and much more. While it remains to be seen what AI all has to offer in the daily lives of users, integrating power-efficient AI accelerators will be a crucial step in finding out.
AI acceleration was a major theme at this year’s Computex and, as generative AI continues to move beyond the experimental phase and into everyday life, one of growing importance. With the biggest chipmakers in the industry all committed to accelerating AI at every level from the datacenter down to the laptop, it will be exciting to see what the future of generative AI holds.
Siemens Digital Industries Software helps organizations of all sizes digitally transform using software, hardware and services from the Siemens Xcelerator business platform. Siemens’ software and the comprehensive digital twin enable companies to optimize their design, engineering and manufacturing processes to turn today’s ideas into the sustainable products of the future. From chips to entire systems, from product to process, across all industries. Siemens Digital Industries Software – Accelerating transformation.
