SemiEngineering: Hidden Costs In Faster, Low-Power AI Systems

Excerpt from article: “Hidden Costs In Faster, Low-Power AI Systems”

Rising design costs
Another piece of this puzzle involves the element of time. While there has been much attention paid to the reliability of traditional von Neumann designs over longer lifetimes, there has been far too little for AI systems. This is not just because this technology is being applied to new applications. AI systems are notoriously opaque, and they can evolve over time in ways that are not fully understood.

“The challenge is understanding what to measure, how to measure it, and what to do to make sure you have an optimized system,” said Anoop Saha, market development manager at Siemens EDA. “You can test how much time it takes to access data and how fast you process data, but this is very different from traditional semiconductor design. The architecture that is optimized for one model is not necessarily the same for another. You may have very different data types, and unit performance is not as important as system performance.”

This has an impact on how and what to partition in AI designs. “When you’re dealing with hardware-software co-design, you need to understand which part goes with which part of a system,” said Saha. “Some companies are using eFPGAs for this. Some are partitioning both hardware and software. You need to be able to understand this at a high level of abstraction and do a lot of design space exploration around the data and the pipeline and the microarchitecture. This is a system of systems, and if you look at the architecture of a car, for example, the SoC architecture depends on the overall architecture of the vehicle and overall system performance. But there’s another problem here, too. The silicon design typically takes two years, and by the time you use the architecture and optimize the performance you may have to go back and update the design again.”

This decision becomes more complex as designs are physically split up into multi-chip packages, Saha noted.

Read the entire article on SemiEngineering originally published on January 20th, 2021.