Building the foundation of LLMs in simulation
New technology is constantly being developed, refined, and finding a place in the real world. Generative AI is the latest in a long line of groundbreaking technology to follow this trajectory as today more and more people adopt it as everything from a conversation partner to a powerful automation assistant. However, in an industrial setting, adopting new technologies is a slower process but one with the potential to yield even greater rewards.
In a recent podcast, experts Justin Hodges, Senior AI/ML Technical Specialist and Todd Tuthill, Vice President of Aerospace, Defense, and Marine Industries from Siemens and Fatma Kocer-Poyraz, Vice President of Engineering Data Science from Altair continued a discussion on applying AI to simulation in the design process. Now, they continue that conversation with a broader look at generative AI in design and what it will take to make an industry-ready foundation model.
Keep reading for some of the highlights of that conversation or click here to listen to the full episode.
Generative AI finds a place in industry
To answer the question of “where does generative AI find its place in industry?” Todd looked back to when simulation was first introduced alongside traditional physical prototyping methods. With the benefit of hindsight, we now see that digital simulations never completely replaced physical prototyping, yet the modern design process finds both indispensable. Going forward, it is equally clear that AI and machine learning will find a similar equilibrium.
Modern engineering is all about the data, with AI being a key tool in understanding and utilizing that data, which is not to say that providing AI with the sum total of human knowledge on math, physics and engineering will result in a generative model capable of replacing traditional simulations, Todd clarifies. Building on the idea that generative AI can augment the capabilities of simulation, while AI will become an increasingly powerful tool for the future of the industry just like simulation and physical prototyping before it, AI will be a technology that enhances, not replaces, what came before.
Modern design practices, like simulation, will continue to serve as a foundation, a source of trust, for designers and engineers going forward but by building on that foundation with AI it becomes possible to create products faster and in more innovative ways than ever before.
Understanding foundation models
Data is of critical importance to AI but equally so is trust and capabilities. Connecting any off-the-shelf LLM with engineering data isn’t going to yield a trustworthy or necessarily useful result, instead, industrial foundation models are key part of bringing the goal of AI-enhanced design to life.
To understand the importance of an industrial foundation model, understanding the relationship between foundation models and large language models is key. Generalizability should be considered the key differentiator, where an LLM can be expected to give reasonable results within the range of its training data, a foundation model must generalize beyond its training data to other, similar tasks. For example, a language foundation model trained on the hundred most spoken languages in the world would be able to generalize to another language its not trained on with just a few examples to work from, rather than further exhaustive training.
An industrial foundation model would hope to achieve the same result, but for math, engineering and process data rather than language. Compared to language though, industrial data is far less voluminous making it difficult to reach the goal of generalization and since our knowledge of physical world is still incomplete, training a truly general foundation model is still a distant dream that may not even be achievable. However, what Justin describes as a narrow foundation model, which can apply within a specific field or project, is where the near future of industrial foundation models lies.
Developing these narrow foundation models will be key in developing the ROMs that will help accelerate simulation processes, which can then be further extended using simulation data. These models will be easier to train with smaller data sets, making them a better fit for the limited data available in many sectors. Simulation data will still be key in training narrow foundation models as they support each other, rather than foundation models taking over simulation as a whole.
Adopting new technologies into complex fields is always a difficult task but, just as with simulation before it, bringing generative AI into the fold will be well worth the effort. Finding the balance between old and new technologies presents a challenge in its own right as the new seems to completely supplant the old. However, as understanding continues to develop, the synergies become clear. Going forward AI will play a key role in simulation and product design becoming a valuable tool for every engineer alongside tried and tested methods, helping them develop new and more innovative technologies.
To learn more, click here.
Siemens Digital Industries Software helps organizations of all sizes digitally transform using software, hardware and services from the Siemens Xcelerator business platform. Siemens’ software and the comprehensive digital twin enable companies to optimize their design, engineering and manufacturing processes to turn today’s ideas into the sustainable products of the future. From chips to entire systems, from product to process, across all industries. Siemens Digital Industries Software – Accelerating transformation.
