Is AI-Powered Machine Vision Beneficial to Manufacturing?
TRANSCRIPT
Automation World - David Greenfield
Welcome to the automation world get your questions answered podcast where we connect with industry experts to get the answers you need about industrial automation Technologies. I'm David Greenfield, editor in chief at Automation World. And the question we'll be answering in this episode is, is AI powered machine vision really beneficial to manufacturing? Joining me to answer this question is Kevin McCabe with IDS Imaging Development Systems, a supplier of machine vision system technology and related software. So thanks for joining me today, Kevin.
IDS - Kevin McCabe
Hey, thanks for having me, David. Glad to be on. So, you know, let's start with the obvious question, you know, why use artificial intelligence in vision systems at all? And you know, by that, I mean, you know, what benefits does it bring to machine vision specifically, and brings just hype right now?
Automation World - David Greenfield
There's some of that going around with AI now. Yeah. Oh, yeah. For sure.
IDS - Kevin McCabe
What a crazy storm even like fast. Yes. Absolutely. Honestly, the technology in specifically machine vision has been around a while.It's sort of, yeah, sort of started with like, neural networks way back, I think in like the 80s. Now we're using them quite often. But the bottom line is, why do you want to use it, it kind of gives you another approach to solving qualitative problems, which can sometimes be difficult for kind of more traditional rules based image processing approach to solve.
So what I mean by qualitative is, let's take a really, really simple example. Let's say we have like a mixed fruit production line, we've got apples and oranges, for some reason on the same line. And we have the problem. Okay, we have to sort these.
So you can start with traditional rules approach, okay, we have an expert in machine vision go, how can I differentiate these, okay, maybe there's some difference in color. Maybe the circularity is different between an apple and orange, and apples got a stem, you can kind of continue on and on to kind of create these rules, and then you sort of end up having exceptions along the way. And this process takes quite a long time to come to a final solution. And that final solution, you know, could be destined in the field, and then end up not actually working. So instead, with the most common AI, type of network called a convolutional neural network. With that type of network, you instead feed a big data set to it, basically, and big can be relative can be 50. Images can be 100, can be 1000. And you feed, you know, pictures of apples and pictures or pictures of oranges to this network. And you tell it, which picture is an apple in which picture is an orange. And then that network, during the training process, figures out what you mean by an apple and an orange.
So that's like, one of the big things you can get this kind of initial prototype out pretty quickly learn from the field. And it's, it's an alternative approach to some rules based stuff.
Automation World - David Greenfield
Can you dig a little deeper, you know, kind of give us more information about specifically how AI enables vision system use in quality control and assurance in manufacturing processes?
IDS - Kevin McCabe
Yeah, so you can kind of you can take one of these networks and insert it to help a quality assurance or quality control process that a human can do alongside it. So it's, again, networks like this or AI like this is good for qualitative things, you know, is there screw Miss screws missing on this part? I only see, you know, three out of five. Okay, that's a bad part. Oh, this, this parts a little askew. There's object detection, which stuff is like, again, counting the screws, there's classification. In the case of like, mixed use lines, it can say, you know, oh, this is this part. Let's switch the system to what that part needs versus all of the other parts you have running along the system. And then there's also segmentation which is it's still useful in quality assurance.
But it's sort of one of the steps into using maybe some other techniques. And then finally, there's a really powerful one called anomaly detection. Those types of networks. Instead of having like this giant data set of like good and bad images that you'd have to provide, like a quality control process, which is tough, right? Most modern manufacturing has a pretty good yield rate. So you end up not having even if you decide to capture data of like all your parts, you end up having way more examples of what a good part is versus what a bad part is. And those bad parts can vary quite a bit in what they are, like, maybe it could be like a scratch on a metal piece, or the metal pieces cracked, those are already two different types of anomalies that need that we need examples in kind of more of a classification network. But with anomaly detection, you give it good images, you tell it over and over again, hey, this is the right part, you know, oh, it's in this different orientation. This time, it's still right. And it will learn what is anomalous, right? Exactly in the name anomaly detection. So it will trigger Oh, hey, this looks something off, kick it to another part of the process and have someone look at it.
So it can assist humans in getting a better idea of what's going wrong with their products and, and catching those products before they get to customers.
Automation World - David Greenfield
It's been really amazing to see over the past, I guess, it's several years now how these systems, how easy they have become to train them, where it used to be hundreds or 1000s of images that you had to show these systems. Now it can be just a handful of them. I can remember, you know, learning about these years ago thinking, you know, is it worth it given the amount of time to train some of these systems?
IDS - Kevin McCabe
Yeah, but now it's it's relatively easy in comparison, it's kind of gone both ways, too, right? They've become easier to train. But also, there's just more data out there now. Like, there are cameras and processes that we didn't think needed vision before. As soon as you learn one thing is like, Oh, wait, we could do this. And you can do that, that just keeps adding on.
Automation World - David Greenfield
So you know, speaking of that, you know, before we get further into this, you know, I did want to take just a quick second. To clarify, you know, what kind of AI we're talking about here, with machine visions, just because with all the headlines right now about the large language models like cat and GPT, I just want to make sure the audience understands the difference between that kind of AI and what we're talking about here with vision systems.
IDS - Kevin McCabe
So they're actually kind of related. So a large language model is also a neural network, similar to a convolutional neural network. So it has a lot of parameters, and it's trained on a whole bunch of data. The most recent ones are just like, here's a bunch of data, I'm not telling you what it is. So it's like totally self supervised learning, they call it where, again, the user isn't giving you a description of what they're giving you. It's just a there's a bunch of text figured out where convolutional neural networks are similar is, again, lots of parameters, it can be trained on a huge quantity of data. So where they differ is, again, at least how convolutional neural networks are applied in vision is, you have these large language learning models that are almost more poised at solving any problem that you give them.
How well they do that, you know, is getting better, but as up to up to our judgments still. And in the case of machine vision, we're really talking more of back to a narrow approach. So you have this convolutional neural network that if you know a little bit about neural networks, there's, you know, input nodes, and then there's the final output. In convolutional neural networks. Those first input nodes are actually things are actually like filters already. So filters that are more used in real space machine vision, like edge detection circularity. So you don't have to reteach a network, like, what's an edge? What's the circular things like that? And then the network kind of chooses what rules based techniques almost, to use on that image when you once you've trained it. So it's a very narrow approach to a problem rather than this big, broad generality.
Automation World - David Greenfield
Alright, thanks for explaining that. I think that really helps not just myself but the audience as well really understand with so much coming out of that AI now, you know, to really differentiate between the different types and understand what the similarities are. So, you know, speaking that I, you know, I know that IDS offers IDS NXT, which is an AI powered vision system that it's designed to be easy to use, even for manufacturers who haven't used AI vision systems before. Can you explain a bit about IDS NXT and how it uses AI and why you say it's easy to use, even for those with no experience working with AI.
IDS - Kevin McCabe
Yeah, so the NXT, it's already done, like the heavy lifting. So it already, you don't need to learn how to create a convolutional neural network on your own. You don't have to have a giant rack of GPUs ready to train that network, you don't even have to have coding experience. So it's really trying to this is this is a little bit of a buzz, it's trying to democratize it saying there are a lot of different user groups that could potentially, you know, touch a system like this, let's make it as usable as possible for, you know, the vision experts, the programming experts, the people on the line, the engineers somewhere in between that have their knowledge of, you know, setting up the rest of the system in terms of automation, but don't have a lot of experience and vision.
Offloading that task of,or really changing that task of programming a bunch of rules based rules, or rules based the rules based approach of machine vision to a network that will help you decide what is best to use, and then spit out a result for you really changes the paradigm quite a bit. Again, it changes it from programming problem over to just a data and labeling problem. And the knowledge shifts to how do you like not introduce biases and things like that into the network. But the bottom line is, with NXT, you're not starting from scratch, you're using a full ecosystem. The NXT is an embedded vision platform. So it's a camera running a full full operating system, it's got like a arm slash FPGA processor onboard. So it's running the neural networks on the camera itself. So that's the hardware part done for you.
The training part, and programming part is done by lighthouse, which is a web based platform where you upload images to a cloud server, all of your data stays around, don't worry, we're not stealing anything. Yeah, and, you know, because it offloads the need for, you know, a big giant mainframe. To train the network's we customized a visual coding platform called the Google Blockly that basically just allows you to, you know, do any logic that you need in a really easy visual format. So, you know, you have the basic loop of the camera, hey, it's capturing every time. Okay, set these parameters up before we start to capture exposure time game, normal cameras, stuff like that, maybe set up some GUI and your, your vision app or whatever. That's like the kind of web based front end front end of whatever you want to do with the network. And then, you know, once an image is captured, oh, run this neural network to figure out whether it's an apple or an orange. And then Okay, switch to this next neural network that I've trained to tell me whether this apple is good or bad.
Automation World - David Greenfield
So, you know, with that explanation of basically how the system works, and how it's been set up to be easy to use, can you give us an example or two of customers of yours in the manufacturing industries who are using IDs and the types of results they've been able to achieve with it.
IDS - Kevin McCabe
So one that comes to mind is actually a 2D pick and place application. So robot arm has to, you know, find, find a park and find its orientation, and then find a pic point and move it to the next event. A customer of ours, based on the NXT hardware trained a neural network in order to do that, so it was able to, I guess you could consider like an object detection network. But additionally, they added on their own ability to also tell the orientation of the part. And so with the orientation and in that kind of center known location, they were able to find a pic point and, and move the move the part on to the next next part, at least they demonstrated it with screwdrivers. So not ultra precise kind of thing. But you know, something that was that had a pretty obvious like orientation in one direction. But it also could tell the rotation of it in the other directions to from like, the flathead the flathead front, because you know, the rest of it is kind of similar. So it paid attention to like, you know, where the label was on the handle, as well as, like I said, the flathead screws, what what orientation the flathead head was on?
Automation World - David Greenfield
How long did it take them to do that training to you? Because like you said, it wasn't a super precise application, but it was fairly detailed. How long did that take to train the system.
IDS - Kevin McCabe
So they, they initially started out just using the object detection network. And as far as I can recall, that took like, a day to set up, okay. And then they wanted to kind of take the technology a little bit further than what we were ready to do with it. So they were also AI experts on the other end. So they they handed it off to another team that was an AI expert and went, Okay, now we need to know the orientation. So they created their own network, then I uploaded it to the camera. And it worked. That part I'm not sure how to how long it took, because obviously it takes longer to create your own network.
But that first part that first initial prototyping was, yeah, within a day, you know, now that we've established, you know, a bit of a baseline about AI use and vision systems and how you're approaching the application of the technology industry. Can you explain, you know how adaptable AI vision systems are in terms of accommodating new processes or changes in a manufacturing line. And I'm asking this, because I keep hearing more and more about how this is a major issue for manufacturers of all kinds today is, you know, they adapt to changing customer demands, reacting to supply chain issues. And also just with the rise in high mix, low volume manufacturing overall, for a lot of manufacturers, especially contract manufacturers, yeah, they're pretty flexible. Like I said before, it kind of changes that rules based problem from a programming problem into a data driven problem.
So if you get a new part, you take some images of it, and you know, different orientations or things like that.So if you've got the data, you can take that same vision system that you have already installed in a line that's, let's say, pre trained for one particular part, you can retrain another network, and then maybe have a classification network beforehand, again, telling the process which part is coming, and then doing the like quality control process on it. So pretty easy. Once you have the camera in, again, it's still data driven. So it's going to take, you know some time to get those images, as well as have a human label them, and then train the network. But still quite a bit faster than then rules based, I believe. Yeah.
Automation World - David Greenfield
And you kind of referenced this earlier talking about the data going into a cloud based system and being secure, you know, just with this idea of AI, you know, consuming and generating so much operational data. Are there any specific data privacy and security implications associated with using AI powered industrial cameras? And if so, how is that being addressed?
IDS - Kevin McCabe
Yeah. So anytime you start collecting images on on your products, it's going to be a risk, right? You know, I defer to IT specialists to have, you know, the real network security type of stuff there. But what we can talk about is how changing like the methods of computation can increase the security. So you have a spectrum of choices, you have the expensive, like self training within a facility. So you've got this data set, there's a little risk there already, right. But let's say it's on a closed network, no problem, you just carrying a bunch of images, no big deal. You're keeping it in your company, totally. You're doing it with you're doing self training with like a big GPU rack that you've already got, or a big server rack. So not much risk there. And then you have the whole other end of the spectrum. That's okay, I have this completely cloud based AI decision making platform that I send an image to, and then it tells me what it is. So every single time a decision is being made.
There's a risk there going to that centralized server, you can sort of make that choice of having something in between. And I think that's where having an edge based system can potentially be a little more secure. So you move that computing from the big cloud based stuff on to the camera itself, the image doesn't even have to leave the camera itself, you can just send the decision out from the camera.
So it gets rid of some of the risks there. And obviously, there's the, the elephant in the room for, well, we have an edge based platform, but we have a cloud based training. Yeah, again, there's going to be some risk, right? Anytime you upload anything on the internet, someone could potentially grab it. But you can make like some mitigating steps towards that. So something simple is like, you know, I hate to hate to hate on the US here, but like, go to European server, they have to comply with GDPR, like really hard.
So even even having something like that, you know, it won't stop malicious actors, as much as you know, just companies grabbing your data. But again, another another mitigating factor. Another kind of tangentially related thing is, with all this data capturing even within your company, I've worked with two or three companies now, I think that really focused on like, they were capturing pretty broad images of like a process. And sometimes a one of their workers could be in there. And they wanted to make sure that data was an organized so like that, it they would still be sending the image off of the embedded platform on to somewhere else for like storage, you know, just gather more more for their dataset, but they wanted to remove the person from him. Hey, an AI network can do that for you. If you can run a person detector network on the camera, and then it goes, Okay, what are the section now there's a person here, you're already improving privacy there.
Automation World - David Greenfield
So it sounds like the word from the way you describe it. It's a lot of the same cybersecurity protection methods that are used for all industrial systems. I mean, there's the IT focused operations in house, there's verifying the cloud security, you know, based on what type of public or private cloud you're using, I guess the difference is that anonymizing feature that you might have to use, depending on what sort of pictures, that's the difference, but the processes are largely the same.
IDS - Kevin McCabe
Yeah.
Automation World - David Greenfield
So one last question for you, Kevin. You know, how can businesses start integrating AI vision, you know, what are the first steps they should take?
IDS - Kevin McCabe
So it's gathered data, right? So we've talked about the shift of AI from program based problem to data driven problem, you got to have the data in order to solve it with AI, for sure. So plunk a camera on your process somewhere that's, that's like step 101. Right? After you've gotten the data, you can start out with something like, we recently brought some of our AI networks actually just to our 2d cameras. So the it's the AI stuff we have is no longer exclusive to NXT. Or you can take a 2d camera, use it with our just our regular free API, and start training network. So that's, that's definitely a good path forward, you have just to just a 2d camera, you have the your computer doing your work, it's going to be slow, right? But it's, it's a start, it'll, it'll give you an understanding of what the possibilities are, for it to solve.
And obviously, reach out to companies like us, we've done a lot of work with a lot of people. And you know, we can't obviously, say all the specifics, but we have an idea of what kind of what kinds of problems can be solved with this technology. Because I suspect there's a lot of companies who are looking at this who are getting ready to take the leap and figure out how they can start using it. And I guess, like you said, the first step is to take that first step, and just start with something simple and go from there. computing power has come a long way. That's that's the big reason why we're even talking about Convolutional Neural Networks right now. It was a brute force problem, though, and that they couldn't solve in the 80s. I think the 80s with, you know, their current computing power. And oh, man, has it changed and is changing. Yeah. And it's, it seems to be what we're seeing now with AI applications just is changing month after month.
Automation World - David Greenfield
Well, thank you again for joining me for this podcast. Kevin. And thanks, of course, to all our listeners. And please keep watching this space. For more installments of automation world get your questions answered. And remember, you can find us online at automation world.com to stay on top of the latest industrial automation, technology insights, trends and news.
