The last dance OR not another GPU blog!
I have some unfortunate news*. This could well be the last time I write about GPUs.
Am I being replaced with an AI blog writer? No! Is it because no one is interested in GPUs for CFD anymore. Well, obviously, No! Is it because I am tired of it. Again, No!
Then what is it? Spoiler Alert!
It is the fact that with the release of Simcenter STAR-CCM+ 2606 I quite simply don’t have any more GPU-native combustion features to talk about. It is due to the fact we have reached a significant milestone in our ability to model combustors with GPU hardware.
*Or fortunate, depending on whether you have enjoyed any of my previous blogs.
Looking back
If this is to be my last ever GPU blog, perhaps you will allow me to indulge in a trip down memory lane. For me, it all started with overnight LES on GPUs. Although, nothing was yet on fire (even if it felt like it when meeting the release deadline), this was a key building block for accurate resolution of turbulence in combustion systems.
Next, we introduced our first taste of GPU-native combustion with the flamelet family of models. Now combustion chambers could be on fire (no not the deadlines this time!), which is great and even better we demonstrated excellent success on real industrial geometries in collaboration with Siemens Energy.
Sprinkling in some nice additions to combustion analysis such as conjugate heat transfer and radiation modelling really began to round out the GPU-native opportunities for combustion design.
Having said that, we still had some roadblocks to covering the full range of analysis on GPU for combustion systems. Firstly, we were limited to gaseous fuels only – think methane and hydrogen. This is ok for many land-based gas turbines focused on power generation, but looking to the sky and jet engines then we need to be able to model liquid fuel.
The second issue was that we were still limited to flamelet combustion models. For steady burning scenarios (e.g. a gas turbine running at full power) then this is suitable. But for accurate modelling of ignition and extinction events, slowly forming pollutants such as soot and NOx, as well as part load conditions, then we often move beyond the applicability of the flamelet regime.
Here we rely on reacting species transport models such as Complex Chemistry. Unlike the flamelet models, where chemistry is reduced to a 0D/1D representation with pre-calculated combustion tables, Complex Chemistry directly calculations reactions “online” within the CFD calculation and all species are transported. Although more accurate, this comes with more cost compared to flamelet models and is therefore a perfect candidate to take advantage of the power of GPU acceleration!
And looking forward
This brings me back round to my bad news. As of Simcenter STAR-CCM+ 2606, these roadblocks have been lifted. Our Lagrangian spray and Complex Chemistry models are now GPU-native, meaning that a comprehensive suite of gas turbine combustion scenarios can now take advantage of the performance, cost and energy savings of GPU hardware.
To demonstrate this, the NASA Energy Efficient Engine (E3) was studied. This is an annular combustor which represents a real-world aerospace combustor. The combustor is liquid fueled, using JetA1, and therefore this makes it a great test case for the latest enhancements to Simcenter STAR-CCM+.
The small print
To get into the details, we model this combustor using Lagrangian multiphase for the liquid spray. This includes drag force, the stochastic secondary break-up (SSD) model, quasi-steady evaporation (with Ranz-Marshall method for Sherwood Number calculation) and Two-Way coupling with the flow.
For the Complex Chemistry model, we utilize the Laminar Flame Concept (LFC) with cell clustering to reduce chemistry overhead. The reaction mechanism used is the JetA HyChem mechanism [1,2]. For the combustion gurus, it’s worth noting that the Eddy Dissipation Concept (EDC) and Thickened Flame Model (TFM) options for turbulence-chemistry interaction are also available on GPU in this release as well as our Eddy-Break Up (EBU) combustion model.
The mesh contains 40M polyhedral cells and models a periodic segment of the combustors. As is common practice for such applications, Large Eddy Simulations (LES) are required to capture the correct turbulent flow behavior and fuel/air mixing. We also have radiation modelling active. This is therefore a true, high-fidelity, multi-physics simulation as we are covering fuel spray, turbulence, combustion and radiation.
Crunching the numbers
We compared this simulation on 1024 CPU cores (16 nodes of a HPC cluster) to 4 x A100 GPU cards (typically half a node) as well as 4 x GH200 GPU cards. The results speak for themselves. Using CPUs this detailed simulation would take over 4.5 days to complete 10 flow throughs. Only 4 x A100 GPUs brings this down under 4 days (meaning a single A100 GPU card is performing equivalent to 310 CPU cores) and utilizing 4 x GH200 GPUs brings the runtime down even further to 2.5 days (meaning a single GH200 GPU card is performing equivalent to 540 CPU cores).
What this means is that instead of having most of your HPC resource blocked for the best part of a week, 4 x GH200 GPUs gets the job done in almost half the time. And as we have discussed before, faster time to solution is not the only benefit. Leveraging GPU hardware results in reduced energy consumption and therefore reduced running costs.
Introducing energy consumption reporting
Here’s a little extra on top: such benefits can now be directly monitored in Simcenter STAR-CCM+ 2606 with the introduction of a new in-built report: Total Solver Energy Consumption. As the name suggests, this report can clearly outline the energy used during the solution process directly inside your simulation file. If we compare for example the 1024 CPU cores with 4 x A100 GPUs, the results are revealing.
For our simulation, leveraging GPU hardware means you reach your solution using around one third of the energy compared to CPU hardware. We can then translate this directly into a cost [3] and CO2 emission [4] per simulation. The numbers here are based on the simulation being conducted in the USA. Over the course of a year running one hundred of these simulations, not only do you save on time, but $2600 dollars of running costs and 5600kg of CO2 emissions (that’s about 4 return flights from London to New York!) are saved. So not are we helping engineers to design more eco-friendly gas turbines, we are helping them to do so in a more sustainable way.
Some final thoughts
So there you have it, Simcenter STAR-CCM+ 2606 is another significant milestone in leveraging GPU-native simulation across industries. After we boosted the automotive and aerospace aerodynamicists early on, tackled the marine sector in the 2602 release, now aerospace combustor designers can enjoy the benefits of GPU hardware. As for me, well I’ll just have to put my feet up and head to the beach or find something else to harp on about. I think my continued employment at Siemens may force me to choose the latter!
[1] H. Wang, R. Xu, K. Wang, C.T. Bowman, D.F. Davidson, R.K. Hanson, K. Brezinsky, F.N. Egolfopoulos, A physics-based approach to modeling real-fuel combustion chemistry – I. Evidence from experiments, and thermodynamic, chemical kinetic and statistical considerations, Combustion and Flame 193 (2018) 502-519.
[2] R. Xu, K. Wang, S. Banerjee, J. Shao, T. Parise, Y. Zhu, S. Wang, A. Movaghar, D.J. Lee, R. Zhao, X. Han, Y. Gao, T. Lu, K. Brezinsky, F.N. Egolfopoulos, D.F. Davidson, R.K. Hanson, C.T. Bowman, H. Wang, A physics-based approach to modeling real-fuel combustion chemistry – II. Reaction kinetic models of jet and rocket fuels, Combustion and Flame 193 (2018) 520-537.
[3] https://www.globalpetrolprices.com/map/electricity_industrial/
