{"id":273,"date":"2020-02-20T12:17:00","date_gmt":"2020-02-20T17:17:00","guid":{"rendered":"https:\/\/blogs.sw.siemens.com\/jt-open\/?p=273"},"modified":"2026-03-26T08:30:17","modified_gmt":"2026-03-26T12:30:17","slug":"using-multibody-simulation-in-ar","status":"publish","type":"post","link":"https:\/\/blogs.sw.siemens.com\/jt-open\/using-multibody-simulation-in-ar\/","title":{"rendered":"Using Multibody Simulation in AR"},"content":{"rendered":"\n

Technische Universit\u00e4t Darmstadt
Department of Computer Integrated Design
Johannes Olbort, M.Sc. ; Prof. Dr.-Ing. R. Anderl<\/p>\n\n\n\n

\"Technische
Technische Universit\u00e4t Darmstadt<\/figcaption><\/figure>\n\n\n\n

Using Multibody Simulation in Augmented Reality<\/h4>\n\n\n\n

The current state of the art allows to transfer Multibody Simulations (MBS) from a simulation program to the AR. This has some advantages and allows for example the exact simulation of CAD models in reality by means of an AR device. In the following blog I will show you how to transfer a MBS of a CAD model from Siemens NX to the AR of the Microsoft HoloLens. Thereby the MBS definition from NX is taken over and made available to the HoloLens.<\/p>\n\n\n\n

Basics: Transfer of simple CAD models into the AR<\/strong><\/p>\n\n\n\n

The basis for the transfer from a MBS to the\nAR is firstly the fact, that simple CAD models can be transferred to the AR of\nthe HoloLens. In earlier research work this was investigated with regards to\nthe most suitable programs and tools. Without going into too much detail, we\nhave developed a process chain that transfers CAD models into the AR of the\nHoloLens. The process chain starts with a CAD program, for example Siemens NX,\nand is transferred to a Game Engine via a DCC program, for example 3ds Max.\nWhen using a Game Engine, you can compile an application for the HoloLens\ndirectly. As a Game Engine Unity has proven to be the most suitable. Using this\nprocess chain, CAD models can now be transferred directly as applications to\nthe HoloLens. Interesting in this respect is the so-called JT Open Plugin,\nwhich enables the import of JT-files into Unity. This enables JT-files to be\ndisplayed in the HoloLens. <\/p>\n\n\n\n

In current research work we are dealing with\nextensive process chains, where CAD models contain a MBS or further\ninformation. As an example, we will now look at a factory planning scenario, where\nit is important to simulate the movement of robots in reality using the\nHoloLens.<\/p>\n\n\n\n

The Process chain from CAD to MBS to AR<\/strong><\/p>\n\n\n\n

The difference between the import of a pure\nCAD model and the import of a CAD model with a defined MBS is the so-called Kinematic\nModule, which must be created in this context. The Kinematic Module creates an\ninterface to transfer information into the AR. The informational flow of the\ndata can be shown with activity charts, as we see below:<\/p>\n\n\n\n

\"The
The process chain from CAD to MBS to AR<\/figcaption><\/figure>\n\n\n\n

Here we proceed similar to the basic approach\nabove, but we do this without a DCC program and instead insert the Multibody Simulation\ntogether with the Kinematic Module.<\/p>\n\n\n\n

We build a robot arm consisting of 6 separate\nassemblies and a gripper in Siemens NX. The motion body and the joints and\ndrives are then defined for the Multibody Simulation. This allows the necessary\ncalculations for the simulation to be carried out. In any case, the results\nshould be evaluated to avoid unpleasant surprises.<\/p>\n\n\n\n

\"RobotArm\"
A robot arm consisting of 6 separate assemblies and a gripper<\/figcaption><\/figure>\n\n\n\n

Now we’ll get started on the Kinematic Module.\nThe Kinematic Module<\/em> essentially has\nthree parts: the Reader<\/em>, Converter<\/em> and the Factory<\/em>. The Reader<\/em> reads\nXML files and gives a list of joints as output. The Converter<\/em> then transfers the vectors of the Game Object into a\ncoordinate system, whereupon the Factory<\/em>\ncreates the joints and defines the properties. It is only necessary to read in\nand adjust the data of the Multibody simulation. The Kinematic Module<\/em> then creates all necessary kinematic elements\nautonomously.<\/p>\n\n\n\n

This created data can now be exported and\ninserted into Unity. During a test run, you can see the created joints in Game Mode.\nThe robot arm can be controlled and new positions of the Multibody system can\nbe set easily.<\/p>\n\n\n\n

\"UnityControl\"
Editor Mode and Game Mode in Unity<\/figcaption><\/figure>\n\n\n\n

This process however is very time consuming\ndue to the high number of degrees of freedom and the restricted functions of\nthe Game Engine environment. Also, if a strong limitation of the degrees of\nfreedom is reached, this often results in an unrealistic behavior of the\nMultibody system due to the segregation of the joints from each other.<\/p>\n\n\n\n

Now you can use the MBS for further projects\nin Unity. Since we are pursuing the goal of transferring the MBS to the\nHoloLens, some HoloLens specific settings will be set and the MBS will be\ntransferred to the HoloLens using the Universal Windows Platform. The final\nresults can be seen in the following video:<\/p>\n\n\n\n