Empowering Shop Floor Workers with Industrial AI – Podcast Transcript
The promise of future technologies is always exciting yet the reality can sometimes fail to live up to the dream. While technologies like mixed reality and Industrial Metaverse may have had a rocky start, today, they are now capable of offering tangible benefits to frontline workers with the promise of very real advancements in the future.
In this episode, host Conor Peick is joined by guest Theo Papadopoulos, senior consultant and head of the Metaverse Lab at Siemens to explore the ways mixed reality and artificial intelligence can help leverage the Industrial Metaverse into a real tool for workers today and tomorrow.
Check out the full episode here or keep reading for a transcript of that conversation.
Conor Peick:
Welcome to the Future Ready Podcast. I’m really excited to have you joining me today. I’m here with Theo Papadopoulos, who’s a senior consultant and the head of the Industrial Metaverse Lab at Siemens. We’re going to be here talking about the Meta Rayban glasses and how that’s helping workers on shop floors access data and talk to machines a little bit. So Theo, as we look at this technology, I would love to know from your perspective, you know, why didn’t this sort of thing work maybe five years ago and what has made it real and accessible today?
Theo Papadopoulos:
First of all, thank you very much for the invitation. Really excited to be here today with you, Conor. And that’s a great question. I’m asked quite a lot this question. What’s the difference? I mean, let’s say the smart glasses or XR glasses have three key building blocks. AI, hardware, connectivity, let’s say. So where we’re standing five years ago and why it didn’t work is back then we had the AI. The AI could see, so it can tell us that’s a pump, but it didn’t have the context. It couldn’t understand what’s the interaction with the environment. And also, if we speak about AI, the natural language process, the NLPs, were not so, let’s say, mature like today. So we had noisy environments problems to understand what the people were saying. or the technical terms were not trained for that. And even for people with pronunciation, so speaking as a Greek English, it was difficult for them to capture that. And if we go to the hardware, there back then also was quite bulky. You remember the first headset quite bulky, so you couldn’t wear them for a lot of hours. Also with the heat, the CPC had problems. It was getting too hot for the user. And the battery was lasting only two hours. So You can see many problems there. And because the first glass has a very limited computational power, we are very dependent on the cloud. Here comes the connectivity. So we didn’t have the speeds to transfer data really fast. So and I want to add here another context, which is the content itself. So when we were asking something, we had to pre-program in advance the reply. So we just provide information to the user. And that was connected with a huge manual effort. So we may have had very successful POC, but that was not scalable. And what now changed now in these five years is this advancement in AI. So now AI has reasoning, can understand the context. The LLMs now can perceive better what we are saying, even myself, I mean, the Greek English, let’s say, or the Greek German, easily capture them. And the content is self-created based on the AI, based on the requests, it create on demand what I want. So if I ask, can you help me with that, on demand can generate the content I want to consume as a field engineer. And connectivity is faster, of course. And the hardware, it’s now the form factor, much, much better. It’s almost close to normal glasses that people wear. And the battery lasts up to 8 hours, so it’s one shift more or less today, so you can cover. And battery lasts longer as a form factor, and computational is also higher. So you can see it’s multiple factors that change. We have huge advancements in this key technology for five years. Of course, still it’s the the beginning of the journey. I don’t want to over problems like we did five years ago and we have faced the same issue, but I am more positive, let’s say, nowadays compared to five years ago.
Conor Peick:
Sure, yeah, That’s amazing. And you know, talked about already, you mentioned AI, of course, and the connectivity and how in the past we’ve had things like VR, XR, and AR, right? Lots of topics that maybe have blended into buzzwords in some fashion in industrial discourse. But So with these today, if we were to kind of take away all the hype, the external opinions and things, what’s like a practical problem that this solves on the shop floor for the people using them?
Theo Papadopoulos:
Yeah. So if I may say, it’s the cost of hesitation. I know that you didn’t expect that answer. But what I mean with that is imagine that the people have a very good ability to understand there is a problem, but then comes the question, why and how to solve it? And if people are not experienced, don’t feel confident, what solution comes in mind to apply that? So they have to ask people, they have to go to manuals and discuss time. And time equals money in the industry. You know that better than me also. And that’s the thing is we want to democratize Here, the access to information gets this decision faster. So to eliminate this hesitation that the employees may have on the shop floor, and that’s here, I think, the added value here.
Conor Peick:
Yeah, minimize, yeah, eliminating that hesitation, I suppose, like you said. It’s interesting because you have these glasses that are obviously a really impressive piece of hardware, but then there’s also what it enables, right, in the business. So yeah, helping workers just do their job better and faster.
Theo Papadopoulos:
Exactly. And now they can feel confident and empowered because I don’t have to ask someone else how to solve the problem. I can ask first my copilot, my assistant, to give me some instructions, and then I can myself solve the problem. So I feel now I have, let’s say, superpowers, when we were always If they can solve any problem on the software myself, I’m not dependent on others. And that’s also quite important for the employees to feel that they’re important and they can themselves do stuff.
Conor Peick:
And that ties in with what you were saying earlier about with the language, potentially a language barrier, understanding, providing to an employee in a language they can understand.
Theo Papadopoulos:
Exactly. And also, I mean, we are speaking a lot with our factories because we’re doing some tests. It’s the test phase, everything within the company. And one of the feedback we got is that they have employees from multiple countries, Eastern Europe, Spain, Germany, Bavaria. Bavaria has another accent. So it’s very difficult also to translate and make accessible this information to all the language, but with the help of AI now that’s possible. But the other way around, the people feel more confident to do documentation through cameras, through streaming, and things like that in their own language, and then we can translate and make available to the other language themselves. So you can see how the inclusion of the employees now gets higher. And everyone now, it’s part of the solution and not kept aside, only for limited people.
Conor Peick:
Yeah, no, that’s really cool. So back to sort of the technology a little bit as well. With the camera embedded in the glasses, obviously they can see what the worker is looking at. And what’s really interesting to me is thinking about how a computer system can look at an image and it obviously sees what the worker is seeing, but how does it actually develop an understanding of what it is looking at through that camera?
Theo Papadopoulos:
So what the camera transmits, it’s just pixels. Let’s say there’s no content. Then comes first computer vision, which is first of all detects the object, it’s object detection or the pose, how this object is positioned in 3D space. And then on top, we have VLM, the visual language model there, which then demystifies this image. So with the help of AI, now we can understand what is the context of this equipment. maybe read data from the machine and also interpret this data and compare with the values that should have been the golden standard, the golden state, let’s say data. And all this is doing, all this stuff is done by AI real time because it’s very fast and they provide these answers to the employee where a human being there, we may need a lot of time to consume all this information and process it. And back then, if when this happens, then it’s an answer generated and provided back through a display or through headphones to the employee and also or step-by-step guidance. Another thing also what’s happened here is the AI also understands based on the request we are doing the level of your experience so it can adjust the replies based on your expertise. So if you are a newbie you can say I need step-by-step guidance so it’s more detailed. If you’re more experienced, you figure out based on the request, if you’re more experienced, then summarize, because the people who are more experienced, they say, Why do you tell me what to do? I know that, so they give some indications that’s a problem, and then they know how to repair themselves. So yes, that’s impressive.
Conor Peick:
That’s very, I mean, all of that is going on in the background in a matter of milliseconds, seconds.
Theo Papadopoulos:
Seconds, exactly. I mean, everything should be fast. As we said before, every second counts. And okay, we have the limitations of the latency of the cloud systems and also the time that the LLMs or the VLMs needs to think, but it’s a matter of seconds to get a reply.
Conor Peick:
Really, really quite impressive. And then obviously also transmit it to the glasses that you talked about with the connectivity. So of course, one of the big things with introducing maybe some automation AI into systems, especially on a shop floor where mistakes can be pretty costly. if this were to make a recommendation or provide some instruction or guidance to an employee that is incorrect, that’s obviously a big problem and it might cause people to lose trust in the system and stop using it. So how do you address that? How do you make sure that it is providing trustworthy guidance and reliable information?
Theo Papadopoulos:
First of all, trust is not claimed, it’s gained. So you have to end the trust with the solution. And as you said, you are right. I mean, still today we know that a lot of LLMs out there are still hallucinated. There’s major advancement in the last years, but still hallucinate. In the private life, that may be annoying, but in the business, as you said, that may be dangerous. Maybe it may create a downtime with the whole factory of a machine, destroy a machine. So it’s very important what we deliver. It can be trusted from the employee, the information. And for that now we are using RAG, retrieval augmented generation of data. So these models are mapped to our internal standard operating processes, manuals, and the ticketing system, documentation, everything, the internal information of the factory. So the AI always use this information as a reference, deliver an answer. And of course we have ensured in our developments now we’re doing here that we have here, let’s say 4 pillars. The first one is the bounded confidence. So the machines disable the AI to understand what it knows and what doesn’t. It’s not afraid to say, I don’t know, because that’s important. And to be honest, because then you can trust it. Then if something is not based on that, it’s also the second one to understand the constraints. The AI, so we set the limits, so operation limits should take the consideration before giving an answer, and with that I mean it should not violate these limits, and if that comes with the first one, so if something goes around then the AI should say, No, I cannot provide you an answer, and that comes the third, and third is here is be able to also redirect. So ask an expert. So if something is not able or it violates the limits, then we give an indication, speak with an expert or speak with your manager. And the 4th one is, of course, to provide explainability. So the AI should be able to say, I suggest you, if you ask them, why did you give me the answer, to explain you detail based on which facts this reply comes. So if you use these four pillars, then we believe you can gain trust. And that’s why currently we are exploring a new agenting workflow from our partner NVIDIA. We are among the first ones. It’s XR AI for smart glasses workflow. Okay. So where we have a reasoning model. And at the beginning, where it can really fast detect our request and point to the right direction, to the right RAG system or to the right LLM, VLM model, and then we get a better result and also integrate our processes there. So it’s something we’re experimenting also currently because we believe that’s part of the trustedness we need on the shop floor.
Conor Peick:
Yeah, that’s really interesting. Are there like feedback mechanisms that people on the shop floor or perhaps those experts that you mentioned where they can influence the model to say, hey, this guidance you offered, I suppose. If you say you didn’t know you weren’t able to provide, you can update potentially the model to expand its applicability.
Theo Papadopoulos:
Exactly. That’s also part of this process. And for example, the employees, if something is not documented or is not available, they can take videos or pictures and with a voice, give some instructions. And this is kept, then approved by the experts. If it’s validated, then we incorporate to the general knowledge that we have here in the back end.
Conor Peick:
Okay, interesting. Yeah, so it’s keeping that human.
Theo Papadopoulos:
Because that’s very important. And also you don’t have to forget that the factories are a living organization. So the factory is never the same. Every day something new comes. Maybe it’s not automatically updated the database or the knowledge in the back end. So we must be able also to expand the knowledge that we have there.
Conor Peick:
But that’s all the time we have for this episode, once again, I’ve been your host Conor Peick joined by Theo Papadopoulos on the Future Ready podcast.
Siemens Digital Industries Software helps organizations of all sizes digitally transform using software, hardware and services from the Siemens Xcelerator business platform. Siemens’ software and the comprehensive digital twin enable companies to optimize their design, engineering and manufacturing processes to turn today’s ideas into the sustainable products of the future. From chips to entire systems, from product to process, across all industries. Siemens Digital Industries Software – Accelerating transformation.
