188. Idea: Augmented Reality for the Industrial World

00:00 Chris: 

Welcome to EECO Asks Why. Today we have an idea episode and we're gonna be talking about what is industrial augmented reality. And to have us walk through this, I brought in the expert, the man himself, Bob Meads, who was the president at IQ agent. So welcome Bob. 

00:20 Bob: 

Hi Chris. Thanks for having me. I'm excited. 

00:22 Chris: 

I'm excited. I remember we met a couple years ago pre-Covid. You came through our lab there in Raleigh and I just remember it was like a kid in a candy store, like, is this stuff for real, like this is really cool. So, you know, I'm excited for this conversation and I kind of want, I was thinking about it. We hear all these terms about, AI, machine learning, IIOT. And if someone were at a dinner party, you and I, and somebody says, what is that augmented reality from an industrial stand point? Where are you going to go? What are you going to tell them? 

00:54 Bob: 

Well, it's one of my favorite topics and I like to boil it down and not use big words. And the way that I usually explain it as augmented reality is just a way to place information directly in context of the real world. And so I may get a funny look at that, but then I ask him, how do you watch football? I'm in the south, everybody here watches football. And I said, okay, you ever notice how the first downline just magically appears? It's not painted on the field, the players can't really see it, but you can see it and it moves.

That's basically augmented reality or at least what it looks like. It's giving you intuitive information that you don't really have to think about because they put it directly in context of the real world. And that's what industrial augmented reality can do for you. So in the plant let's say that you're looking at a machine and you've got a roller, rolling out paper, and then there's this little floating thing that's showing you so many feet per minute, intuitively, you're going to say, oh, okay, well that role was going at, 82 feet per minute or something. And that's what augmented reality can do for you. It doesn't have to be this magical, game-like thing it's really meant to be something that is very helpful. And very intuitive so that people can work more efficiently and more safely. 

02:18 Chris: 

I love it. I love it. Cause I typically when I thought about it in the past, I automatically go to the games, like the Pokémon Go, but I hadn't even thought about that football line, I guess that's been in place for years. 

02:31 Bob: 

I mean, it's not, from a very technical standpoint, that's not technically augmented reality, but it's exactly what it looks like when Microsoft HoloLens, which is what they call mixed reality came out the very first version. It was all about games and you know, and when they brought it out and they did the big thing on the stage. It was a game they showed and it was a like robot attack and your walls and your house would open up and robots would pour out and you would shoot them with laser fingers.

When HoloLens 2 came out, there were no games shown. It was only industrial applications. And that is very, very telling because Microsoft understood that the sweet spot for augmented reality lies in manufacturing. They have money. They can really improve their life and that's what they went with. And I think that once you look at that and realize that it shows you what kind of potential that augmented reality has for process and manufacturing. 

03:34 Chris: 

And, you've been in the augmenting reality game for a while. Now, I'm curious from your vantage point, some of the advancements I'm sure technology is moving so quickly. What have been some of the coolest things that has advanced over the last three to five years? 

03:50 Bob: 

You know, there has been a lot and we actually created iQagent in 2011. And back then, which is our flagship product. We were more assisted reality than augmented reality and assisted reality just means, okay, we're going to pop stuff up on your screen and you can generally see the real world, but it may not be attached to a real world object. So it's really an information display or something like that, where augmented reality tends to have an anchor in the world, but we came up with the idea back in 2011, but you know, you're holding a tablet or you're holding a phone. So one of the things back then was one of your hands or both of your hands is encumbered if you're trying to see information displayed, that means you can't work. And that means you have to put that device down. So, one of the big things to come out this is the new real wear a headset, the navigator. And it's actually this is the next one from their first one. It's much lighter, much more comfortable, but now, you know, because it's got this camera, you can see information, but your hands are free.

So one of the biggest innovations is we're starting to get these hands-free devices that are very comfortable to wear that you can wear all day. You can't wear a HoloLens 1 all day, but the HoloLens 2, you can wear a lot more comfortably, a lot longer, same thing with a real, where I have to give them credits because they, this new product is really good.

But then you've got fustics, you know, you've got third eye, you've got some of these others. Also the processing power, you know, some of the advancements in technology 5G is going to mean a lot because you need a lot of bandwidth in order to do this. And so the technology for graphics processing has increased the battery technology, which Elon Musk has helped move along, has also done this.

And then the third advancement is really the buy-in from some of your bigger players. So Apple, you know, they came out with their augmented reality library and, we feel that they're going to be coming out with glasses. They've all, but come out and said it, and you can see down in the developer notes and then the beta release, you know, These mentions or suggestions of, gesture, interfaces and things like that. So you've got, you've got AR core. You've got, what Google's doing, you've got buy in from Microsoft, from Google, from Android, from Apple. That they are going to be supporting this. And I think you're going to see the iPhone, the mobile phones are going to start to have competition. Once the wearables get into a form factor of something like this, but still be meaningful. And this all started, this all started with Google glass. And it was called The Explore and it had this tiny peripheral vision.

And that was really one of the first things we looked at, but it was too small to do anything. You could put a single piece of information on there and, and there's just a lot of things, but it was a start and it kind of got things started. Those are really the three biggest areas, you know, that I've seen the most advancement and you're going to see more, you're going to continue to see wearables that are increasingly comfortable to wear, less intrusive. And I'm hoping, I think a lot of these manufacturers are looking at the plant floor and there's a lot of things that they have to do in order to be accepted on the plant floor. So I think you're going to see a lot of that. 

07:17 Chris: 

Yeah. I mean, it almost feels like science fiction is going to meet industrial. Pretty soon, right? 

07:24 Bob: 

Oh yeah. Yeah. Well, it really is. And it's being driven by science fiction, I am a huge fan of science fiction. And if you think of Minority Report where Tom cruise is moving things in the air. And a lot of that is really kind of like an augmented reality or a mixed reality display. And it does drive it. We see that we want that somebody works on it and I'm still waiting on my Jetson car because I was promised that when I was a kid.

07:50 Chris: 

That's right. Well, I am curious, from a culture standpoint for the industrials, that can be a little bit harder road. Sometimes they are more of the late adopters in the early adopters. So how do we get these industrial facilities to start moving forward and changing their culture with augmented reality? Because I think that the value's there, but culture stuff in a lot of these industrial facilities.

08:12 Bob: 

That you're absolutely right. That is the $64,000 question, but I have seen the acceptance level really come up. When I started in automation. I got out of the Navy in 91. I started working for Siemens. I was working with HMI software WinCC and Windows 95 just came out at that time. And they were working on windows NT. So Siemens had an office at Microsoft, it was called Siemens and Redmond Office and Windows 95 was just not stable enough. We felt to go into the plant, but one to where I felt it was just fine. And that's how a Wonderware really took off. If you remember, they were on Windows 3-1 and things like that.

So Siemens, you know, they were really looking at new technology and everything and, back then the adoption of new technology was seven to 10 years. But when Windows NT came out as an industrialized version and things like that, then that came out and then automation of vendors in turn started writing software to be run in plants. And then that's where you saw PLCs and things like that. But I've seen a progression where, okay, Windows doesn't go on a plant and then it was okay, well, Windows NT is going to go on a plant, but we're not doing any of these laptop computers because it's crazy to walk around with a computer.

You got to sit at a desk with a computer, you'll be distracted and then it was okay. You can have laptops, but they're not going to be wireless because that's too much freedom. Okay. And then it yet to be well wireless, okay, we'll do wireless, but X, Y, and Z. Now, you know, you've got these giant iPads, you've got these iPhones and you've got, what were kids back then who grew up with this technology. The iPad two came out in 2011, which is why I started developing iQagent. So you've got these people that are now comfortable. And they're white collars in the plants and they're thinking, okay, well, let's get this technology in the plant floor and I realized that the adoption rate had really exceeded because 2011, the iPad two comes out, it's got a camera, it's got an accelerometer, it's got all the bits on it that you can actually make something that makes sense to be on a mobile tablet. And my first customer for iQagent, global pharmaceutical company, everyone would know exactly who they are. If I said who they were, but one of the guys, their guys came to us and we did some programming and stuff and they said, we just bought 50 iPads.

I said, what are you doing with them? He said, I'm asking you. So they looked at that and they knew that these things were going to be extremely useful on the plant floor. So usually apps will drive hardware cells. In this case, they saw that, okay. And they came up with the term mobility, they had a mobility initiative. And so the adoption rate has gotten a lot faster, but the problems, the barriers, which is everybody's biggest topic, favorite topic. Number one, do you have a good wireless network? Because if you don't have a good secure wireless network where you can reach the plant, then what are we talking about? Now we do have, you know, like iQagent does have offline capabilities for forms and for documents and things like that.

But you know, that, that comes later. So, they have to do that. The other thing is they, need to adopt the use of electronic forms and the electronic documents, if they don't have that information in that format and organized, and you would be very surprised how many don't, then they're going to have some work to do. And they also, a lot of plants for forever, basically didn't want you using your phone or an iPad out on the plant floor and here's a real life story. And I, this is a very passionate topic for me. So I tend to go, but I, we were also in a major automotive vendor. And so I was showing this, this was 2013. And so I'm showing this and they've got the vice-president of manufacturing down there. I'm doing the demo in their lab at this university I'm nervous as heck, and this guy, he's asking all these questions, you know, for two hours, we're going around and he's scanning QR codes and he really likes it.

So, at the end, talking to him and I said, okay, so what do you think? And he says, Bob, this is a fantastic application. This is so useful. I can think of a lot of things. We will never use this in the plant and it got quiet. And I said, okay, why? And he said, well, I don't know if you've been in an automotive plant. I had, he said, but there's this big sign when you go in, it says, there's no cameras or recording devices that are going to be allowed on this floor. I said, really? He said, yeah, and I said everybody's got one of these in their back pocket, and this is a high definition, 40 megapixel camera that can record. So either that rule, you're not enforcing it or you don't care about it.

And he said, you've got a point there. They changed their policy. They officially changed their policy and the policy was okay, you can't do this. You can't record everything in the world, but you can use this technology to use these applications and here's areas where you can't, and so that's where, your device management comes up, but you know, one of the big barriers is thinking just, okay, well, you can't have a camera here. Well, the benefits of a camera, the benefits of that technology or mobile technology platform outweigh in many cases, the risk, which, securities risks, people walking around, looking at Facebook, getting run over by something, zooming across the plant floor, that's a risk you got to mitigate that, but you can really make people very efficient and get their job done and like doing what they're doing.

The other thing with wearables specifically is, you know, think about having something over your face and you know, something like a little warning pops up and it distracts you and then you get hurt something happens. So you know that's one of the big things in AR that's been talked about for a long time, is how do you make it so it's not distracting. And device management can actually take care of that with a good device management software, like AirWatch or there several others, if you're wearing a headset and you're connected to it, if you walk into a secure area, you have a room saying, okay, you're in this room, shut the camera off. Can't use it. It's not enough to tell the guy, okay, you can't record in here because that may or may not stop the person or you may forget so, the biggest barriers are having their data organized. So that we can bring this data in, data from PLC data, from your SCADA, your PDCs systems, from your production databases, your documents, your schematics, because what iQagent does is you take your phone and point it at a motor and I can get all the information you have about that motor and pull up the schematics. I can fill out a work order. I can see a video on how to restart it. I can see, the points, but in order to do that, it's got to come from somewhere.

And if they're not organized, this a lot of work. So, one of the biggest barriers that me and a lot of things. In my field face is they look at the amount of engineering or the perceived amount of engineering that they're going to have to do in order to use the system. And they go, yeah, we don't have time to organize all those documents that are on this gigantic, shared drive. And, it's not really organized and somebody, we'd have to spend hours and hours now. And everybody has that problem, and they are very, very, you know, organized. And that's probably the surprisingly, probably the biggest one that they have.

And then of course, another thing is just the perceived, too new technology that, some of the older people, may have an unconscious bias against, you know, about using these things. COVID has been probably the thing that has forced the adoption of wearables. More than anything, because back in 2019, I mean, RealWear was doing good HoloLens, some of these other wearables, but some of the wearables actually did very well with COVID because now they had this need. They had the toothache. And they had to do something. They couldn't have everybody in the plant. So they had a skeleton crew at the plant and they're thinking, well, the expert is at home and just waving your phone around and hey look at this and doing this, wasn't cutting it. And so they need the application where he's wearing a camera, his hands are free and the guy's telling him what to do.

So remote mentorship, kind of came about. And that really, I think injected some much needed motivation on the part of consumers, which then now they're willing to spend the money and, the vendors for these things started seeing, Hey, we need to get these out there, but we, you really gotta solve the problems.

Yeah. And I know I'm going, but I'm on a point here and the point. I personally felt okay, well, if I can get more hardware out there, that means I'm going to have all my software also and that's not true because what I saw is a lot of people would just buy a wearable and they would put the free version of Microsoft Teams. That's it. They're not buying remote mentoring software. They're not buying XYZ because right now they're just trying to dig out of the COVID hole. And you know, so now software is trying to kind of catch up and get on these systems that people would buy. So I know that was kind of a long answer, but did you get out of that what you needed?

17:52 Chris: 

I think so for sure. I mean, a couple of things that really jumped out when I think about the CMMS side of it, and there's still a lot of clipboards that are out there from a maintenance standpoint where you know, I'm going here. I'm going to look at this piece of equipment. I'm going to collect this data because this has been done for last 20 years. This is how I got to do it. And augmented reality challenges that, and if I'm not supportive of that with my management, with my culture at the plant, that would probably be a pretty stiff rub to get past that. I mean, that one jumps out, but then conversely, we hear about the skills gap challenge in industry all the time.

And if I think about the skills gap and bringing new people into the plants to do these maintenance tasks, augmented reality could be a way to accelerate that skills gap and to get that training at their fingertips, right there in front of them while they're doing it in the moment. So you're not relying on the travel knowledge necessarily, there's still value in travel knowledge, but if you can automate some of that and put it right there in front of the new employees. I see tremendous value in that could be a culture shift as well. 

19:05 Bob: 

That is a fantastic point. I'm glad you brought it up. There is this concept called just-in-time training and the knowledge gap, you know, which is a big thing that people are facing, especially now because COVID really affected the job market. And a lot of people that were at, or near retirement just said the heck with it. And so, you know, the first point you brought up with a CMMS, and, utilizing that system, one of the barriers is that sometimes, software, like iQagent the perception is that is overlaps would SCADA or overlap with CCMS. So it's not really true if you really look into it. And so, we look to integrate with those systems, but the skills gap, a lot of augmented reality is a perfect fix for that, because let's think about it. Here's a real world example. And I love telling this story, but if you think about information in the context of the real world, that is a perfect platform to enable people to use resources to do jobs. And so here's the example.

Two years ago, my wife or my daughter rather, she has got a 2014 Toyota Corolla. It happens to be silver and the headlight went out. And so, like a good dad I took it down, to you know, the guy that does my truck. And I said, Hey, how much to replace a headlight? And then he said a figure that made my mouth go dry. So like now I'm not doing that. I'm going to do it myself. So I go to AutoZone. Guy gets me the right headlight and what do I do? I go home. I pull out my phone. I go to YouTube, how to change a headlight in a 2014 Toyota Corolla. And I had four videos to choose from, I picked the guy that had the same color car as my daughter. I propped it up on the engine. And I did what he did and I was able to change the headlight and I didn't have any parts left. 

So let's think about that. So I was operating at a higher capacity than I normally could because I had information right at the place that I need it, not in the classroom, not at my desk, but right there. And I was able to use it. When I was in manufacturing, I used to work for NEC Technologies and I was a production engineer. I an electronics technician. And we had a lot of temporary workers, you know, they just hired temps and the temps would just kind of do the drudgery and they would, we make computer monitors and they're in the heat up line. We had the agent in the CRT and they would go and check these. And every once in a while, they would get to something and they didn't know how to do. So we're paying them what, eight, nine bucks an hour. They had to walk away. And go find somebody. Okay. So there's a gap of time. We're still paying and then they get somebody. Now we're paying two people for this job, walk them back and then they get instructions or whatever. There may be some delays. So that problem may have cost us $35. But let's say that now with this technology, That there's a video floating right next to this thing saying, Hey, okay, to reset this alarm, do this and this, or here's the instructions it's no different than me changing the headlamp and the 2014 Toyota Corolla.

The information is right there is exactly where you need it. It's not a four hour long maintenance course. It's a 30 second clip on exactly what we have to do. And that is a huge benefit of augmented reality. And it's one of the stories that I tell where if you look at the knowledge gap, you get the older people that really know what the heck they're doing, that are maybe not so computer centric or, they're too busy saving the world and you get them and say, Hey, I'm going to shoot a video of you changing this filter, put a good shirt on let's do it.

Do you video them. Then you drag that in and associate it right there with that filter. You just close the knowledge gap by a little bit, and if companies could take this, they get a huge benefit because they're closing the knowledge gap, but they've got less experienced workers performing at a higher level with less errors.

23:16 Chris: 

That's the beauty of it. For sure. I'm curious from your standpoint, you've worked with a lot of industrial. Who owns it? Who inside these plants, who's taking the lead, who's taken the initiative who when it's not working, who's getting called into the office? And then but also who's going to be that advocate? So I'm just curious from your standpoint. 

23:35 Bob: 

That's a great question. And that's also one of those $64,000 questions. Although with inflation is probably more like a 64 million question, but you would think, and when we started out, I started going after senior production engineers, maintenance manager. You know, I can impress the daylights out of a production engineer, but they don't have a button. And usually the maintenance person, depending on the company, doesn't have time and they're worried about their system. And they probably tend to see this as overlap with their system where I've had the most success. And we're in some big companies, some, global brand-named companies and the biggest success with them is that the corporate level, because you have corporate level technicians, corporate level managers, whose job it is to help these other plants bring in new technology. 

So they have titles like, external technology acquisition or lean six Sigma, anyone that's involved in making people more efficient, making the process more lean and they like nothing more, cause they usually have their own evil laboratory where they got all kinds of stuff and so I can get in with those people and get them working on the bench. And there, all the people from the plants are coming to them. Hey, we're having this problem. And so, then they like nothing more to say, Hey man, check this out. Here's kind of a solution. And then you go get a pilot in that plane. The thing is if you're going through the plant in these big companies, a lot of times they can't make those decisions, it's got to be a corporate. And so that's the number one thing is going after corporate. Some of the companies we're in that the beauty of our solution is, you don't have to be a big corporation to enjoy the benefits of this because our price point is so much more practical, that's why I call it practical augmented reality, but we are in a small bakery up in Canada and they use it on their ovens. They use it on the roll racks. They use it in the maintenance. Just to pull up documents and a little bit of data. And so for that, then you're looking at, your leaders knew that might have a budget and, basically he'll look at it and go to his boss or her boss and say, Hey, I need, you know, 4,000 bucks and I can outfit everything. Here's how we save that money back and you can do it. It depends on who you're talking to, but you know, obviously you're going to do better with the logical operations, because it has a much longer sales cycle.

25:57 Chris: 

Well, it's a wonderful technology. And I tell you what Bob, one thing our listeners can tell is you're passionate about it, man.

26:05 Bob: 

I love this. I've been living this and I am extremely passionate about. And, when I've told my son and I've told my daughter find something that you're happy with and you're passionate about it because it's going to make your life a lot better. 

26:21 Chris: 

No doubt. Now we call it EECO Asks Why Bob, and we always wrap up with the why. I'm curious for you and speak to that industrial listener out there.

So why is augmented reality that technology that they really should embrace as they build for the future?

26:37 Bob: 

Another great question. And it's because it comes full circle and here's what I mean. Back in the day before transistors were invented, we didn't really have PLCs. And so you walk up to a piece of equipment and you have all the information around there. You can look at the valve and tell what position, the valves, and you've got a pressure gauge and you've got a power light. You've got in this hardwired system. When PLCs came about, we had this idea of process visualization. We put in controllers and then we moved the visualization up to the control room or out to an HMI panel.

And what that did is it took that information away from the one place we really need it, which is standing right in front of the equipment. I can't go look at the motor, and it may be across the safety gate. I may not even be able to tell if it's running, much less what the RPMs are. So what do we have to do?

We got to go walk away and find an HMI panel. So now, as we get more and more technology, augmented reality brings us back full circle where we don't have to invest in, a physical HMI panel. If we just need that kind of information, we can just point our device had it, grab this information, see it on a point of interest display or see it in augmented reality display. And get the information I need. And not only that, but everybody can see that even if they're not by that valve, they can pull it up. So, it's because it's kind of bringing it full circle. And the other thing that augmented reality does, which, it's really the same situation as you look at how manufacturing is done is augmented reality, decentralizes your information because, you know, if you look at a, say a motor, SharePoint might have the schematics and maybe user manuals. They may have to walk back to the maintenance shop, to go into the CMMS system, to get the maintenance history. They may have to go to the vendor's website to get information, about make, model, parts, things like that. They might have to go and look up a production report to see how it's been producing or, X, Y, and Z.

So, they met deploying the PLC to see what's going on as five or six or seven different things. And you can't get a complete picture and it's because all your information is in all these different systems. And so with, AR, and especially with mobile apps, it's these technologies that have to come together to basically make this possible. Now we're pulling information from all those systems and giving it back to you where it matters most right in front of the equipment. 

29:09 Chris: 

Love it. Love it, Bob, this has been so much great information for listeners out there. Check out the show notes. You have ways to connect with Bob, with iQagent to see his solutions. Bob, thank you so much again. Anything else you got? 

29:23 Bob: 

No, sir. I really appreciate it. And you guys listening out there. Thank you for listening. I appreciate it. Give us a hit up if we can help you with anything. Absolutely. 

29:30 Chris: 

You have a wonderful day, sir. 

29:33 Bob: 

Thank you, Chris. I've enjoyed it. Thank you.