{"id":74979,"date":"2026-07-02T04:19:54","date_gmt":"2026-07-02T08:19:54","guid":{"rendered":"https:\/\/blogs.sw.siemens.com\/simcenter\/?p=74979"},"modified":"2026-07-02T04:27:51","modified_gmt":"2026-07-02T08:27:51","slug":"simulation-model-automated-validation","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/simcenter\/simulation-model-automated-validation\/","title":{"rendered":"From Manual Checks to Automated Validation: Ensuring Model Integrity with Predefined KPIs\u00a0"},"content":{"rendered":"
\n
\"\"<\/figure><\/div>\n\n\n

In the world of Electric Vehicle (EV) thermal management, accurate and reliable simulation models are paramount. But how do we ensure these complex models truly reflect reality and can be trusted for critical design decisions? The answer lies in robust and automated validation methods, and the Test Execution Manager (TEM)<\/strong>, a free add-on for Simcenter Amesim, is a critical capability in automating this crucial process.<\/p>\n\n\n\n

This article explains how the Test Execution Manager (TEM)<\/strong> enables automated validation of models by defining objective KPIs, executing test suites, and integrating with version control<\/a> workflows, enabling simulation engineers to reduce validation time by up to 90% while improving model integrity.<\/p>\n\n\n\n

The challenge of model validation<\/h2>\n\n\n\n

Traditionally, validating models often involved tedious manual comparisons of simulation results against test data. This approach is not only time-consuming and prone to human error but also lacks scalability.<\/p>\n\n\n\n

As models grow in complexity and new versions are developed, manually re-validating every change becomes an unsustainable bottleneck.<\/p>\n\n\n\n

Consider the initial stages of validating an EV cooling system model. We start with simple test cases, comparing coolant temperatures between the model and test data.<\/p>\n\n\n\n

While visual inspection might suffice for a single, straightforward scenario, it quickly breaks down when dealing with multiple parameters, complex cycles, and frequent model updates. We need a more objective, repeatable, and automated approach.<\/p>\n\n\n

\n
\"\"
Figure 1: Test harness used to validate the cooling system model against requirements<\/em><\/figcaption><\/figure><\/div>\n\n\n

Objective KPIs replace subjective visual inspection with quantifiable Pass\/Fail criteria <\/h2>\n\n\n\n

To move beyond subjective visual checks, we introduce Key Performance Indicators (KPIs), <\/strong>quantifiable metrics that define acceptable model performance, such as maximum temperature error or cumulative deviation from test data. and clear sign-off criteria<\/strong>, predefined thresholds that determine whether a model passes or fails validation, enabling objective, repeatable assessment.<\/p>\n\n\n\n

These objective metrics allow us to quantify model accuracy and define acceptable thresholds for validation, such as maximum temperature error or cumulative deviation from test data.<\/p>\n\n\n\n

For instance, a model might be deemed valid if:<\/p>\n\n\n\n

    \n
  • The absolute temperature error at the end of a cycle is less than 0.1\u00b0C.<\/li>\n\n\n\n
  • The absolute sum of errors over a cycle is less than 1\u00b0C.<\/li>\n<\/ul>\n\n\n
    \n
    \"\"
    Figure 2: Using Simcenter Amesim post-processing to define sign-off criteria<\/em><\/figcaption><\/figure><\/div>\n\n\n

    In addition, from Simcenter Amesim 2604, TEM supports direct comparison with previous time-series results, further supporting regression tests when updating models to the latest versions of the software.<\/p>\n\n\n\n

    By defining these criteria, we establish a clear benchmark for model performance, ensuring that any new version or modification meets predefined standards. <\/p>\n\n\n\n

    Automated test execution reduces validation time<\/h2>\n\n\n\n

    This is where the TEM steps in as an indispensable tool. TEM is specifically designed to automate the process of repeating tests on new model versions seamlessly and automatically, drastically reducing manual validation efforts.<\/p>\n\n\n\n

    Here’s how TEM streamlines your automated validation workflow:<\/p>\n\n\n\n

      \n
    • Environment Setup:<\/strong> After setting up your model environment in Simcenter Amesim, TEM analyzes the available post-processing variables and identifies validation-ready criteria.<\/li>\n\n\n\n
    • Pass\/Fail Criteria Definition:<\/strong> Test engineers define the pass\/fail criteria based on your established KPIs. This might involve setting target values, tolerances, and conditions for various parameters.<\/li>\n<\/ul>\n\n\n
      \n
      \"\"
      Figure 3: Setting Pass\/Fail criteria in TEM<\/em><\/figcaption><\/figure><\/div>\n\n\n
        \n
      • Automated Test Execution:<\/strong>\u00a0TEM can execute individual tests or entire suites of tests, running your model against defined boundary conditions and comparing the results to predefined criteria without manual intervention.<\/li>\n\n\n\n
      • Integration with Version Control:<\/strong>\u00a0TEM integrates with Simcenter Client for Git, making it a powerful component of your collaborative model development and automated validation workflow. When a new model branch is developed and ready for integration, TEM can automatically run pre-defined tests. If the tests pass, the results can be confidently merged back into the main branch for re-use.<\/li>\n<\/ul>\n\n\n
        \n
        \"\"
        Figure 4: Model development process using Simcenter Systems Simulation tools<\/em><\/figcaption><\/figure><\/div>\n\n\n
          \n
        • Comprehensive Reporting:<\/strong> TEM collects results from multiple models and test scenarios into a single, comprehensive report, providing a clear overview of model performance and highlighting any areas that require further attention.<\/li>\n<\/ul>\n\n\n
          \n
          \"\"
          Figure 5: Collated model sign-off report<\/em><\/figcaption><\/figure><\/div>\n\n\n

          Conclusion: Building confidence, accelerating development<\/h2>\n\n\n\n

          TEM transforms model validation from a manual chore into an automated, reliable, and integral part of your simulation workflow. This not only builds trust in your EV thermal models but also significantly enhances productivity and accelerates innovation.<\/p>\n\n\n\n

          \n

          \ud83d\udcccKey takeaways<\/h2>\n\n\n\n

          By leveraging TEM, engineers can:<\/p>\n\n\n\n

            \n
          • Reduce Validation Time and Effort:<\/strong> Automating repetitive tests frees up valuable engineering time, allowing them to focus on more complex design challenges.<\/li>\n\n\n\n
          • Improve Model Integrity:<\/strong>\u00a0Consistent and objective automated validation ensures that model integrity is maintained across different versions and modifications.<\/li>\n\n\n\n
          • Increase Confidence in Simulation Results:<\/strong> Clear pass\/fail criteria and automated testing provide a high level of confidence in the validity and reliability of your models.<\/li>\n\n\n\n
          • Accelerate Development Cycles:<\/strong>\u00a0Faster automated validation leads to quicker iterations and ultimately, a more efficient product development process for EV thermal systems<\/li>\n<\/ul>\n<\/div><\/div>\n\n\n\n

            How does your team currently perform systems model validation and verification tasks?<\/strong><\/p>\n\n\n\n

            In the next blog, we’ll explore how search capabilities in Simcenter Client for Git enable efficient model-based design through model re-use<\/p>\n\n\n\n

            \n

            FAQ<\/h2>\n\n\n\n
            Q: What is the Test Execution Manager (TEM)?<\/strong><\/summary>\n

            A: Test Execution Manager (TEM) is a free add-on for Simcenter Amesim that automates the validation of simulation models against predefined test criteria. It executes tests, compares results to Key Performance Indicators (KPIs), and generates pass\/fail reports\u2014eliminating manual validation efforts and ensuring model integrity across versions.<\/p>\n<\/details>\n\n\n\n

            \n\n\n\n
            Q: How does TEM integrate with Simcenter Client for Git?<\/strong><\/summary>\n

            A: TEM integrates directly with Simcenter Client for Git to automate validation in version-controlled workflows. When a new model branch is ready for integration, TEM automatically runs predefined test suites. If tests pass, results are merged back into the main branch. If tests fail, the branch is flagged for review, preventing invalid models from entering production.<\/p>\n<\/details>\n\n\n\n

            \n\n\n\n
            Q: What types of validation criteria can TEM evaluate?<\/strong><\/summary>\n

            A: TEM accesses Simcenter Amesim\u2019s post-processing variables enabling support for multiple validation criterion types including:<\/p>\n\n\n\n

              \n
            • Time-series cumulative error, compared to previous results<\/li>\n\n\n\n
            • Absolute error thresholds (e.g., temperature error < 0.1\u00b0C)<\/li>\n\n\n\n
            • Cumulative error limits (e.g., sum of errors over cycle < 1\u00b0C)<\/li>\n\n\n\n
            • Peak value constraints (e.g., maximum pressure < 2.5 bar)<\/li>\n\n\n\n
            • Time-based conditions (e.g., settling time < 30 seconds)<\/li>\n<\/ul>\n<\/details>\n\n\n\n
              \n\n\n\n
              Q: Can TEM validate models against real test data?<\/strong><\/summary>\n

              A: Not directly, but this is possible by including the test data comparison in the Simcenter Amesim model to be validated.<\/p>\n<\/details>\n\n\n\n

              \n\n\n\n
              Q: Can TEM run tests on multiple model versions simultaneously?<\/strong><\/strong><\/summary>\n

              A: Yes. TEM can execute test suites across multiple model versions or configurations in parallel, generating comparative reports that highlight performance differences. This is particularly useful when evaluating design alternatives or tracking model evolution over time.<\/p>\n<\/details>\n\n\n\n

              \n\n\n\n
              Q: What happens when a test fails?<\/strong><\/summary>\n

              A: When a test fails, TEM generates a detailed report identifying:<\/p>\n\n\n\n

                \n
              • Which specific KPI(s) failed<\/li>\n\n\n\n
              • The magnitude of the deviation from acceptable criteria<\/li>\n<\/ul>\n\n\n\n

                The models with results are stored in the tool\u2019s working directory, so Engineers can then investigate the root cause and iterate on the model before re-running validation.<\/p>\n<\/details>\n\n\n\n

                \n\n\n\n
                Q: Is TEM compatible with CI\/CD workflows?<\/strong><\/summary>\n

                A: Yes. TEM supports command-line execution and API integration, enabling it to be incorporated into Continuous Integration\/Continuous Deployment (CI\/CD) pipelines.<\/p>\n<\/details>\n\n\n\n

                \n<\/div>\n","protected":false},"excerpt":{"rendered":"

                In the world of Electric Vehicle (EV) thermal management, accurate and reliable simulation models are paramount. But how do we…<\/p>\n","protected":false},"author":36441,"featured_media":75386,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"spanish_translation":"","french_translation":"","german_translation":"","italian_translation":"","polish_translation":"","japanese_translation":"","chinese_translation":"","footnotes":""},"categories":[1,179,182],"tags":[82,1823,16],"industry":[89,63664],"product":[590,64098],"coauthors":[29828,45824],"class_list":["post-74979","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-news","category-product-updates","category-tips-tricks","tag-digital-twin","tag-simcenter","tag-system-simulation","industry-automotive-transportation","industry-battery","product-simcenter-amesim","product-simcenter-client-for-git"],"featured_image_url":"https:\/\/blogs.sw.siemens.com\/wp-content\/uploads\/sites\/6\/2026\/06\/Simcenter-systems-automated-validation.jpg","_links":{"self":[{"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/posts\/74979","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/users\/36441"}],"replies":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/comments?post=74979"}],"version-history":[{"count":4,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/posts\/74979\/revisions"}],"predecessor-version":[{"id":76497,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/posts\/74979\/revisions\/76497"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/media\/75386"}],"wp:attachment":[{"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/media?parent=74979"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/categories?post=74979"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/tags?post=74979"},{"taxonomy":"industry","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/industry?post=74979"},{"taxonomy":"product","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/product?post=74979"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/blogs.sw.siemens.com\/simcenter\/wp-json\/wp\/v2\/coauthors?post=74979"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}