{"id":12908,"date":"2026-02-20T12:53:31","date_gmt":"2026-02-20T17:53:31","guid":{"rendered":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/?p=12908"},"modified":"2026-03-26T12:35:51","modified_gmt":"2026-03-26T16:35:51","slug":"topics-in-ai-systems-part-ii-gnn-and-llm-types","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/topics-in-ai-systems-part-ii-gnn-and-llm-types\/","title":{"rendered":"Topics in AI Systems: Part II \u2014 GNN and LLM Types"},"content":{"rendered":"\n<p>In <a href=\"https:\/\/blogs.sw.siemens.com\/thought-leadership\/topics-in-ai-systems-part-i-gnn-and-llm-fusion\/\">Part I<\/a>, we introduced the idea that solving complex digital-engineering problems requires combining structural reasoning (GNNs) with semantic understanding (LLMs). We used examples from electro-mechanical systems, manufacturing, and service lifecycle to show why this fusion is powerful. In this article, we move from <em>why<\/em> to <em>what<\/em>. Specifically, we introduce the major types of GNNs and LLMs so developers and architects can select the right building blocks when designing fused AI solutions for digital industries (see Figure 1). For ease of comparison, the current top-twelve GNNs and LLMs are presented in two tables.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img loading=\"lazy\" decoding=\"async\" width=\"1024\" height=\"683\" src=\"https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/19\/2026\/02\/GNN_LLM_Types-1024x683.png\" alt=\"\" class=\"wp-image-12919\" srcset=\"https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/19\/2026\/02\/GNN_LLM_Types-1024x683.png 1024w, https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/19\/2026\/02\/GNN_LLM_Types-600x400.png 600w, https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/19\/2026\/02\/GNN_LLM_Types-768x512.png 768w, https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/19\/2026\/02\/GNN_LLM_Types-900x600.png 900w, https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/19\/2026\/02\/GNN_LLM_Types.png 1536w\" sizes=\"auto, (max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"has-text-align-center\">Figure 1: Illustrating <em>what<\/em> are the major GNN and LLM model families available to AI system developers.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">I. The GNN Table with Example Application Use Cases<\/h3>\n\n\n\n<p>The legend in the last two columns of the table for the current top-twelve GNNs are defined as:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Complexity\/effort = engineering + needed infrastructure + tuning cost<\/li>\n\n\n\n<li>Maturity\/accuracy = reliability + predictability in real deployments<\/li>\n<\/ul>\n\n\n\n<p><strong>Table 1: Graph Neural Network (GNN) Model Types<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><td><strong>GNN Type<\/strong><\/td><td><strong>Best Applied Problem Types<\/strong><\/td><td><strong>Implementation Complexity<\/strong><\/td><td><strong>Maturity \/ Typical Accuracy<\/strong><\/td><\/tr><\/thead><tbody><tr><td>GCN (Graph Convolutional Network)<\/td><td>Node classification in product graphs, part similarity, baseline digital-twin analytics, baseline molecular property prediction<\/td><td>Low<\/td><td>High<\/td><\/tr><tr><td>GraphSAGE<\/td><td>Inductive embeddings for evolving product structures, supply chains, and asset fleets<\/td><td>Low<\/td><td>High<\/td><\/tr><tr><td>GAT (Graph Attention Network)<\/td><td>Systems where some connections matter more than others (failure propagation, reliability analysis)<\/td><td>Medium<\/td><td>High<\/td><\/tr><tr><td>GIN (Graph Isomorphism Network)<\/td><td>Graph classification, molecular fingerprints, tasks requiring strong structural discrimination<\/td><td>Medium<\/td><td>High<\/td><\/tr><tr><td>MPNN (Message Passing Neural Network)<\/td><td>Molecular property prediction, chemistry\/material science, physics-inspired problems<\/td><td>Medium<\/td><td>High<\/td><\/tr><tr><td>R-GCN (Relational GCN)<\/td><td>Enterprise knowledge graphs linking requirements, parts, simulations, suppliers, and service data<\/td><td>Medium<\/td><td>High<\/td><\/tr><tr><td>Heterogeneous GNN (HAN, HGT)<\/td><td>Multi-type nodes\/edges: recommender systems, enterprise knowledge graphs<\/td><td>High<\/td><td>High<\/td><\/tr><tr><td>Graph Autoencoders (GAE\/VGAE)<\/td><td>Link prediction and anomaly detection in supply chains, manufacturing networks, and service fleets<\/td><td>Medium<\/td><td>Medium<\/td><\/tr><tr><td>DiffPool \/ Hierarchical GNNs<\/td><td>Graph classification requiring hierarchical structure (proteins, materials, circuits)<\/td><td>High<\/td><td>Medium<\/td><\/tr><tr><td>Graph Transformers<\/td><td>Long-range dependencies, large molecular graphs, protein folding, materials discovery<\/td><td>High<\/td><td>Medium<\/td><\/tr><tr><td>SE(3)\/E(n) Equivariant GNNs<\/td><td>3D molecular modeling, quantum chemistry, physics simulations<\/td><td>High<\/td><td>High (scientific domain)<\/td><\/tr><tr><td>SchNet \/ PhysNet<\/td><td>Atomistic simulations, energy\/force prediction<\/td><td>High<\/td><td>High (scientific domain)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">II. Choosing GNNs for Digital Industry Use Cases<\/h3>\n\n\n\n<p>For most digital-engineering applications, the starting point is usually a heterogeneous product or lifecycle graph. A practical selection pattern often looks like:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Start with GCN \/ GraphSAGE for early experimentation and baseline value.<\/li>\n\n\n\n<li>Move to R-GCN or Heterogeneous GNNs once multiple data types (requirements, parts, suppliers, simulations) are included.<\/li>\n\n\n\n<li>Consider GAT when uncertain about which components play the critical roles in system.<\/li>\n\n\n\n<li>Use physics-aware GNNs (SE(3), SchNet) when geometry and simulation fidelity become critical.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">III. The LLM Table with Example Application Use Cases<\/h3>\n\n\n\n<p>For the LLM table, including the current top-twelve models, we have a similar legend as above:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Complexity = engineering + infrastructure + tuning pain<\/li>\n\n\n\n<li>Maturity\/accuracy = reliability + predictability in real deployments<\/li>\n<\/ul>\n\n\n\n<p><strong>Table 2: Large Language Model (LLM) Types<\/strong><\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><td><strong>LLM Type<\/strong><\/td><td><strong>Best Applied Problem Types<\/strong><\/td><td><strong>Implementation Complexity<\/strong><\/td><td><strong>Maturity \/ Typical Accuracy<\/strong><\/td><\/tr><\/thead><tbody><tr><td>Vanilla Transformer (GPT\/BERT-style)<\/td><td>General NLP, chat, summarization, coding, classification<\/td><td>Medium<\/td><td>Very High<\/td><\/tr><tr><td>Encoder-only (BERT\/RoBERTa)<\/td><td>Semantic search across requirements, documentation, simulation reports, and service records<\/td><td>Low<\/td><td>Very High<\/td><\/tr><tr><td>Decoder-only (GPT-style)<\/td><td>Engineering copilots, requirements assistants, code generation, workflow automation<\/td><td>Medium<\/td><td>Very High<\/td><\/tr><tr><td>Encoder\u2013Decoder (T5\/BART)<\/td><td>Translation, summarization, structured text transformation<\/td><td>Medium<\/td><td>High<\/td><\/tr><tr><td>Instruction-tuned LLMs<\/td><td>Assistants, enterprise copilots, task-following<\/td><td>Medium<\/td><td>Very High<\/td><\/tr><tr><td>RLHF \/ RLAIF tuned LLMs<\/td><td>Safe assistants, customer-facing chatbots<\/td><td>High<\/td><td>Very High<\/td><\/tr><tr><td>Retrieval-Augmented Generation (RAG)<\/td><td>Enterprise knowledge copilots grounded in PLM, ALM, MES, and service documentation<\/td><td>Medium<\/td><td>Very High<\/td><\/tr><tr><td>Tool-using \/ Agent LLMs<\/td><td>Automated engineering workflows interacting with CAD, simulation, and data platforms<\/td><td>High<\/td><td>High<\/td><\/tr><tr><td>Mixture-of-Experts (MoE)<\/td><td>Massive-scale models, cost-efficient inference at scale<\/td><td>Very High<\/td><td>High<\/td><\/tr><tr><td>Long-context Transformers<\/td><td>Analysis of long engineering specifications, certification documents, and design histories<\/td><td>High<\/td><td>High<\/td><\/tr><tr><td>Multimodal LLMs (Vision-Language)<\/td><td>Engineering drawings, CAD screenshots, inspection images, and technical documentation<\/td><td>Very High<\/td><td>High<\/td><\/tr><tr><td>Domain-Finetuned LLMs<\/td><td>Engineering, manufacturing, aerospace, automotive, and industrial automation domains<\/td><td>Medium\u2013High<\/td><td>Very High (in-domain)<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<h3 class=\"wp-block-heading\">IV. Choosing LLMs for Digital Industry Use Cases<\/h3>\n\n\n\n<p>In digital industries, the \u201cright\u201d LLM is less about raw benchmark scores and more about how the model will be grounded, governed, and integrated into engineering workflows. A practical selection pattern often looks like:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Start with Instruction-tuned, decoder-only LLMs for developer productivity and copilots.<br>This approach would be best when we need: Q&amp;A, summarization, explanation, and workflow assistance for engineers.<\/li>\n\n\n\n<li>Use Encoder-only models (embeddings) when retrieval is the real product. This would be best when we need: semantic search across requirements, specifications, simulation reports, service notes, and lessons learned.<\/li>\n\n\n\n<li>Adopt RAG early for enterprise use, not as an afterthought. Best when we must ground outputs in authoritative and up-to-date sources: PLM\/ALM artifacts, engineering standards, test reports, maintenance manuals, and internal wikis.<\/li>\n\n\n\n<li>Choose Long-context Transformers when engineers work with \u201cgiant documents.\u201d Best when the key inputs are large: certification documents, multi-hundred-page specifications, and system-level design histories.<\/li>\n\n\n\n<li>Use Tool-using \/ Agent LLMs when the output must change the configuration (CAD\/CAE\/MES), not just talk about it. Best when the model needs to: query databases, run checks, generate scripts, call simulation workflows, open tickets, and produce traceable artifacts.<\/li>\n\n\n\n<li>Reserve Domain-Finetuned LLMs for repeatable, high-value tasks with stable data. Best when we have: consistent internal corpora and recurring tasks (requirements normalization, defect triage, service summary classification).<\/li>\n\n\n\n<li>Use Multimodal LLMs only when visual inputs are central. Best when the workflow involves: drawings, inspection photos, UI screenshots, and diagram interpretation.<\/li>\n<\/ul>\n\n\n\n<h3 class=\"wp-block-heading\">V. Putting the Pieces Together for Fusion<\/h3>\n\n\n\n<p>Selecting models for fusion is not about choosing the most advanced architecture. It is about matching system structure with knowledge sources and regulatory\/safety requirement. Here are a few examples of the typical problems in digital industries, and choice of GNN and LLM to provide a GNN+LLM solution:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><td><strong>Digital Industry Problem<\/strong><\/td><td><strong>Likely GNN Choice<\/strong><\/td><td><strong>Likely LLM Choice<\/strong><\/td><\/tr><\/thead><tbody><tr><td>Digital twin of complex product<\/td><td>R-GCN \/ Heterogeneous GNN<\/td><td>RAG + Instruction-tuned LLM<\/td><\/tr><tr><td>Manufacturing planning<\/td><td>GraphSAGE \/ Temporal GNN<\/td><td>Tool-using LLM + RAG<\/td><\/tr><tr><td>Service lifecycle intelligence<\/td><td>Temporal GNN<\/td><td>Long-context LLM + RAG<\/td><\/tr><tr><td>Engineering knowledge assistants<\/td><td>Knowledge-graph GNN<\/td><td>Domain-finetuned LLM<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p>Obviously, specific GNN and LLM choices shown in these examples could differ based on the specific industry use case and evolution of AI methods.<\/p>\n\n\n\n<h3 class=\"wp-block-heading\">VI. Final Remarks and Next Steps<\/h3>\n\n\n\n<p>In <a href=\"https:\/\/blogs.sw.siemens.com\/thought-leadership\/topics-in-ai-systems-part-i-gnn-and-llm-fusion\/\">Part I<\/a>, we introduced the idea that fusing GNNs and LLMs enables AI systems that understand both the structure of engineered systems and the knowledge surrounding them. In this article, we explored the major model families that developers can use to build these fused solutions. A key takeaway is that most real-world implementations do not require the most complex models immediately. Successful projects typically start with mature, production-ready approaches and evolve toward more specialized architectures as value is proven. Across digital industries, the opportunity is clear:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>GNNs provide the structural backbone of the digital thread and digital twin.<\/li>\n\n\n\n<li>LLMs provide the interface to human knowledge and engineering workflows.<\/li>\n\n\n\n<li>Fusion enables a new class of engineering copilots and intelligent automation.<\/li>\n<\/ul>\n\n\n\n<p>In <a href=\"https:\/\/blogs.sw.siemens.com\/thought-leadership\/topics-in-ai-systems-part-iii-gnnllm-architecture-and-implementation-patterns\/\">Part III<\/a> of this series, we will explore practical architecture and implementation patterns for building GNN+LLM fusion systems in real industrial environments. Note: This blog was created with the help of AI.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>In Part I, we introduced the idea that solving complex digital-engineering problems requires combining structural reasoning (GNNs) with semantic understanding&#8230;<\/p>\n","protected":false},"author":73027,"featured_media":12913,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spanish_translation":"","french_translation":"","german_translation":"","italian_translation":"","polish_translation":"","japanese_translation":"","chinese_translation":"","footnotes":""},"categories":[1],"tags":[],"industry":[],"product":[],"coauthors":[2485],"class_list":["post-12908","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news"],"featured_image_url":"https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/19\/2026\/02\/GNN_LLM_Types.png","_links":{"self":[{"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/posts\/12908","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/users\/73027"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/comments?post=12908"}],"version-history":[{"count":4,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/posts\/12908\/revisions"}],"predecessor-version":[{"id":13005,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/posts\/12908\/revisions\/13005"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/media\/12913"}],"wp:attachment":[{"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/media?parent=12908"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/categories?post=12908"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/tags?post=12908"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/industry?post=12908"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/product?post=12908"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/thought-leadership\/wp-json\/wp\/v2\/coauthors?post=12908"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}