​3D IC technology trends: Microarchitecture in IC design

John McMillan [0:08]

Hello and welcome to the Siemens EDA podcast series on 3D IC Chiplet Ecosystems, brought to you by the Siemens Thought Leadership team. I’m your host, John McMillan. This podcast series dives into the exciting world of semiconductor chip integration and advanced technology platforms using two and a half and 3D techniques.

I talk to industry leaders and subject matter experts to discuss the latest 3D IC industry trends and roadmaps and uncover how the semiconductor industry is working to diligently make 3D IC mainstream.

If it’s your first time listening to the 3D IC podcast, welcome and if you’ve been following this podcast series, welcome back. In today’s podcast, I’m excited to be joined by my special guest, Pratyush Kamal. Pratyush is the director of R&D for 3D IC Solutions Engineering at Siemens Digital Industries.

And today we’re going to dive into the world of micro architecture in IC design. Welcome, Pratyush. We’re very excited to have you on the 3D IC podcast, but before we dive into today’s discussion, please tell our listeners a little about yourself, your background and current role with Siemens EDA.

Pratyush Kamal [1:13]

Thank you, John. Very nice to meet you. I’m currently a 3D IC R&D director, as you mentioned at Siemens EDA. And I lead a team at a technical level of a subject matter experts, and these are people from different fields in industry, from mechanical design background, from electrical analysis background, from package design background, as well as IC design background.

So our focus is on enabling 3D IC design and EDA support, as well as looking at roadmaps towards advanced technology node integration because when we look at the new leading edge technology nodes that the TSMC and Samsungs are offering, they are inevitably taking us towards 3D IC with introduction of those nodes.

Prior to joining Siemens, I spent a lot of years doing IC design as well as IC architecture and systems architecture work. My focus over the last 20 years has been primarily in mobile space where I worked for Qualcomm as well as Google in their mobile chip divisions.

I do have some experience with the 3D IC research at Qualcomm where we looked at the hybrid bond-based integration many years ago. Very excited to be here.

John McMillan [2:31]

Thanks. Very impressive, and I’m really excited you joined the podcast today. So, let’s kind of dig right in. My first question is just to kind of educate us all, what is microarchitecture in IC design?

Pratyush Kamal [2:44]

So, microarchitecture in IC design is essentially a hardware implementation of the architecture we put in. Now an architecture is essentially how a software engineer sees the system. Microarchitecture is essentially how the hardware engineer sees the system and the implementation of that architecture into hardware components, for example, register files, arithmetic logic units, memories, et cetera.

And essentially microarchitecture, you can break it down into two classes. One is relating to the data set itself and the other one is control of the data. A microarchitecture is essentially trying to make sure that the data is flowing as designed, is conditioned as design, and all the associated controls related to that data conditioning and data flow components are in place.

John McMillan [3:35]

Oh, great. So, let’s take the next step and try to discover what does microarchitecture mean in 3D IC design?

Pratyush Kamal [3:44]

So, when you look at 3D IC, you have considerations about how the data is moving across the 3D stack and how the controls are going back and forth across the 3D stack for a seamless flow of data. So, this could involve components like your die-to-die interface making use of the 3D stack, and that interface would have both the data plane as well as the control plane support built in, but it goes beyond that.

When we look at the new emerging trends in 3D IC integration, we are talking about a submicron pitch density. And what that means is that even the components that you use to build up your microarchitecture could themselves be realized in 3D fashion.

So, it is not only the organization of your control of your hardware blocks when it comes to 3D IC microarchitecture, but it’s also leveraging the integration density that we have today and finding out how we can build a system that uses 3D components in itself.

So, imagine when you play a Tetris game, and you have these three-dimensional blocks, and you have to fit them in. So that’s the kind of challenge we are looking at with 3D IC, both in terms of physical implementation of the design as well as the logical implementation of the design.

John McMillan [5:01]

So how early do we need to consider microarchitecture in 3D IC design?

Pratyush Kamal [5:05]

Not just in 3D IC design, but overall, as you see the landscape changing, the IC design industry is shifting to what we call a software-defined silicon paradigm where you’re using a software to customize the hardware functionality of your core.

Now, 3D IC adds another layer of complexity to this new paradigm. As a result, you really need to start very early on. 3D IC is expensive, the design process is expensive, the manufacturing process is expensive, so anything you can catch up early on is a lot of savings for your organization.

So, we need to start upfront as we are defining the system architecture, the system partitioning, and not just in terms of where we place the blocks, but also how we realize the functionalities, whether in terms of software or whether in terms of hardware.

3D IC designs especially, you need to look at it all very, very early on.

John McMillan [6:03]

So, considering the increasing complexity of microarchitecture, what are the main issues that system designers are currently facing?

Pratyush Kamal [6:12]

3D IC brings in itself a lot of challenges. Now imagine if you take a design that is laid out in 2D today and you fold it onto itself to realize a 3D equivalent design of it. The design still needs the same amount of power delivered to it.

Now you have half the area to deliver that power too, so your power density is going up and as a result of that, your thermal challenges go up in 3D IC. But not only that, you have to think about test considerations, you have to think about debug consideration, failure analysis, because when we go to 3D IC, just the nature of it takes away some of the capabilities of current failure analysis from us.

We just cannot do certain things that we have gotten used to doing in industry. So, for example, the circuits need to be a lot more resilient because you’re going to lose some ability to do your debug to an extent, your failure analysis to an extent.

You’re also going to lose your ability to do tests, some portions of the test. So, all of that has to be taken into consideration for 3D IC very, very early on. One of the challenges of a test is when we shrink the 3D integration page to one micron density, we cannot physically probe those pins anymore.

So, we’ll have to come up with alternative solutions. So, your burden of tests, your burden of what I call resilience… Resilience is essentially your ability to bounce back from a failure essentially, that needs to go up for 3D ICs a lot.

And this could even mean going beyond a level of microarchitecture and probably looking at circuit level innovations to address some of these problems.

John McMillan [7:55]

Sounds really complex. So how do you deal with the increased complexities and what enables decision-making early on?

Pratyush Kamal [8:03]

So, two things, we really need to focus a lot on automation. In the past, a lot of automation has been achieved in the IC design flow itself. But when it comes to taking a piece of silicon and packaging it into a package and putting it on the PCB board, there is a lot of traditional flows are still very manual.

And there are windows-based platforms commonly used that needs to change. The complexity requires us to automate a lot of these flows as much as we can. These flows don’t only need automation, but also integration.

And when I say these flows, I really mean the design flow, the integration flow, the electrical analysis flow, the mechanical analysis flow, thermal analysis flow. They need to be looked up all in conjunction with each other.

And when we are putting things so close to each other, and there could be a heterogeneity to the system as well where you are putting different materials on top of each other. You could be putting a silicon with a non-silicon substrate, you could have silicon interposer with an organic substrate in there.

So, you need to do what we call multi-physics analysis early on to make sure… Because remember the design cost is very high design manufacturing is very high. So, your goal is to do a lot of very early predictive analysis before you actually engage your design teams in implementing those designs.

So, automation is a key. And the second key point as I mentioned, is early predictive analysis that really two critical components.

John McMillan [9:42]

So, with that, how are roles changing or developing to enable a more holistic outlook on 3D IC microarchitecture?

Pratyush Kamal [9:50]

So, what I see happening is we will be moving away from a human-centered flow to a machine-centered flow. Again, in the packaging world, in the 3D assembly world, we have to go back and see what the IC designs in the past have been done.

Simply because of the complexity of the IC design where you’re dealing with a million or a billion transistors at a time, there’s a diminishing participation from humans and more and more work is given to the machine. Definitely we need machines to aid in our work more.

We are lucky that AI is at a point where it is a very powerful tool for humans to deploy. We can make use of AI agents and tomorrow when we have multiple agents working together, we can make use of agent AI to do a lot of these works on our behalf.

We will be replacing a lot of human expertise with machines essentially. The need for us is rather than being very vertically ingrained into a particular domain, we need to have a breadth of domain. Designers need to be aware of mechanical aspects of a design.

They need to be aware of how this electrical analysis is done. Designers should be able to do these complex analyses on their own without the requisite background or training. A lot of cross-pollination needs to happen within the teams.

A lot of skill enhancement, I would say, a breadth of skill rather than a depth of skill is required at this time. As humans, we have a tendency to lose interest in things we don’t understand well. So, there’s a lot of education required.

Retraining of the workforce is required for it to work.

John McMillan [11:30]

Gotcha. What would you describe are things that would enable this non-silo design practice?

Pratyush Kamal [11:37]

Cross-training is very important. Skill development is very important because that way once we understand something, we can take interest in it. So, today’s designs, you have an architect handing over the piece of the specification over to microarchitect and the microarchitect in their own silo implementing it in their way.

But as we look towards more software defined systems, these two, as an example, architect and microarchitect have to work together very closely to enable this. And same thing goes to once you have microarchitected the design and send it over to implementation, the same collaboration needs to happen more closely.

Again, the key is cross-training, cross-training across disciplines as well as growing your own expertise in a breadth of field basically.

John McMillan [12:29]

Great. Thanks. Anything else you’d like to add before we wrap this podcast up?

Pratyush Kamal [12:32]

3D IC implementation is only beginning. We have seen very few examples in the industry so far. We have seen custom examples in a very application specific manner. The most generic example that we can see in industry today for 3D IC is what AMD put out, where they put an SRAM on top of a logic die.

The 3D ICs of tomorrow will have to do more. Now when we just take the example of AMD’s architecture of choice, we can just look at layers of decision-making that must have been involved there.

For example, when you’re putting the SRAM out of a die, you have to look at the read and write latency. And typically, if your kernel, the logic core that decides the data movement, dictates the data movement to and from the SRAM code. If it’s sitting on the opposite side of SRAM, you would naturally have higher read latency compared to write latency.

Most applications, for example, you would care about less read latency than write latency. So how do you partition these kernels across the two dies? It could be a key for tomorrow’s decision-making. When we go to a more heterogeneous integration, for example, when we look at the mobile industry or the IoT industry where the cores are not as homogeneous as you will see for AI/ high-performance compute platforms where the cores are more homogeneous, the heterogeneity of these other markets would drive different architecture considerations.

For example, a lot of focus is today on a mesh-based architecture for multi-chiplet systems, multi-die systems. Tomorrow’s systems have to probably think about a hub-and-spoke kind of architecture. Same thing with the memory access.

A lot of work is being done. The industry is going towards a non-unified memory access, and it just goes on basically to the fundamental idea that not all latencies are equal. So, you don’t want to penalize cores if they have lower latency to the DRAM compared to a core that has higher latency.

So, we have gone with this non-unified memory access approach, but when we look at mobile and IoT space, we have to think differently. Same thing with resilience, we have to bring in a lot of redundancy in tomorrow’s design.

When a lot of designs today also… In fact, this is an industry-wide practice today, that when we run a chip, we run it based on the environment it is operating at. If it’s getting hot, we slow it down to cool it down.

So, we call it thermal throttling. Those things have to be looked at more carefully, more closely. One of the effects of 3D IC is that to get the connections from the backside, you thin the silicon a lot.

Silicon is a very good conductor. So, when you’re thinning the silicon, you’re losing the ability to transfer the heat laterally on your die. So how does that change your thermal throttling strategy? That has to be looked at. A lot of design pieces that we embed in our silicon today.

They may have to move out onto the package to have a better view of how the system is performing at a given time. I talked about redundancy, but I keep coming back to it because I do see physical failure analysis as a showstopper to adoption of 3D IC for heterogeneous systems.

So, I mentioned earlier, we have to think about failure resilient systems. How do we do that? We probably have to think of redundancies. We have to think of analog design practices and implementing them into the digital domain to overcome some of these challenges.

John McMillan [16:21]

Thank you, Pratyush. This has been a very insightful and informative podcast. Thank you for joining me today and sharing your knowledge and thoughts on this exciting, fast-moving and growing area of technology. To all our listeners and viewers, I hope you found this podcast educational and insightful as I have.

That is it for today’s episode, but thank you for joining us and be sure to check out the show notes to learn more about 3D IC and the Siemens EDA