What Are LiDAR Sensors?

TRANSCRIPT

David Greenfield, Automation World
Welcome to the Automation World Gets Your Questions Answered podcast where we connect with industry experts to get the answers you need about industrial automation technologies. I'm David Greenfield, editor in chief at Automation World and the question will be answering in this episode is: What are LiDAR sensors and joining me to answer this question is Aaron Rothmeyer, product manager for dynamic ranging products at Sick USA, a supplier of sensors, safety, machine vision and networking technologies and services. So thanks for joining me today, Aaron.

Aaron Rothmeyer, Sick USA
Absolutely. Happy to be here, David. Thank you.

David Greenfield, Automation World
So you know, Aaron, let's just start by explaining what LiDAR sensors are and how they differ from other position detection sensors commonly used in manufacturing like proximity, position or vision sensors.

Aaron Rothmeyer, Sick USA
We hear a lot about LiDAR and it seems like a very complex technology, but in reality it's quite simple.  It's just a sensor that's designed to do distance and ranging, as is implied in the name. LiDAR is basically an acronym for light based detection and ranging and, as the name implies, you're using lasers or light to basically do detection of objects or do ranging and that's based off of the time-of-flight principle, where in the LiDAR sensor we're sending out a bunch of pulses of laser light, we can calculate the amount of time it takes for those laser pulses to go out and back. And then based on the speed of light, we can figure out exactly how far it is to a target.

So it's a very simple technology. But there's certainly benefits of using this type of technology versus. say, basic proximity or position sensors. One of the big ones is range. Since we're using lasers, obviously lasers have very long range typically. So where a typical proximity sensor might max out at around 1 meter or so, with a LiDAR sensor we can often go much farther, 80 to 100 meters is no problem at all.

This reach depends on a variety of factors, but these are much longer rage sensors and the resolution is you typically outstanding as well. Even at those longer ranges, at tens of meters, we can often get very good resolution—sub centimeter or even sub millimeter resolution depending on the technology.

So, while often there's a tradeoff between range and resolution with LiDAR, oftentimes we can get both. Both get very high resolution, even at longer ranges.

And then another important factor on top of those two are ambient effects. If there's weather or anything like that, if you're using a camera or if you're using a certain technologies that aren't really meant for outdoor use, you might end up with issues if there is rain, snow, heavy sunlight, etc. LiDAR is typically fairly immune to those effects.

And then the field of view—with a traditional proximity sensor or industrial capacitive sensor or anything like that, you're going to end up with issues with field of view. You're very limited. It's just a single point of detection with both 2D and 3D technologies. With LiDAR you get a much larger field of view. It could be up to 360 degrees depending on the technology.

David Greenfield, Automation World
So you said these were simple sensors and I understand that, but they can do quite a bit. So that's a very interesting. Thanks for explaining the expansive capacity of LiDAR sensors compared to more of the standard sensors used in manufacturing.

You mentioned how LiDAR sensors can generate 2D and 3D images. How did they do that and what is that kind of data used for?

Aaron Rothmeyer, Sick USA
I would say that the technology varies pretty greatly between different sensors, so there are a lot of ways to do this. But with 2D sensors, primarily, it's just a spinning motor and that spinning motor rotates a mirror and we bounce a laser off of that mirror. And so, as the motor rotates, we're basically sweeping a laser to generate this range or this large field of view of laser distances. That's primarily how it ends up getting done with two dimensional sensors.

Now when we say 3D, we're not just doing left to right or an XY. We're actually going vertically as well, and when you go vertically, well, there's gotta be another way to generate that and that, right now, is a very hot topic. What's the best way to do that? If it is, for example, a mechanical system where there's some type of movement inside the sensor to generate that that vertical field of view. It's often done by basically spinning a large number of lasers, so instead of just taking one laser and rotating it, we take advantage of a technology called VECSEL, a vertical external-cavity surface-emitting laser. Here, you basically create a large cloud of lasers and spin them, and that's one of the more popular ways to do it right now. But there's certainly people working on other technologies as well, such as solid state LiDAR where a variety of technologies are being employed to basically create a much larger cloud of measurement points.

David Greenfield, Automation World
One of the most common applications for LiDAR sensors that I've seen is around their use on autonomous mobile robots. Can you explain why the use of LiDAR is so prevalent on AMRs?

Aaron Rothmeyer, Sick USA
Sure. But I will say first of all, LiDAR is a great technology. It's not the only technology that's used on AMRs. For example, many times when you look at AMR, you'll see a LiDAR sensor on there. But you'll also often see other technologies and that's an important point to make because you know, we can't say you just throw a LiDAR on any system and make it, you know, reliable and safe.

And that is the goal if you got an autonomous vehicle traveling through facility. It has to be reliable to maintain uptime and it has to be safe so that nobody is getting injured by this technology. And in order to make that happen, oftentimes engineers have to use multiple types of sensing to make it reliable and safe. And so with LiDAR you often see it next to a camera, or maybe a radar to supply some multimodal sensing technology.

But I think it's mostly popular in all AMRs because it has that large field of view. If you think about a camera or about a radar or ultrasonic sensors or things like that, the field of view is typically quite limited. With a LiDAR however, if you've got a spot, you know right on top of the machine that you can mount one of these, you can take advantage of that full 360 degrees. You can really cover a lot of area around the AMR as it's traveling with one sensor and I think that's a pretty important distinction between other technologies.

But also it's also an active sensor, right? Because we are sending out our own laser light, we're not dependent on, say, ambient light. So you hear a lot of buzz about, you know, lights out facilities and stuff like that. Even if it's not a lights-out facility, lighting changes as you travel through a warehouse, right? People open dock doors, windows get opened and, as the sun travels from east to west, there are all kinds of things affecting the ambient light. So, if you're relying on a technology that uses ambient light, that can create issues with the reliability of your system. Since the LiDAR sends out its own pulses of light and captures them when they come back, it's very immune to any kind of ambient light issues or other things that change in the environment, like dust entering the air or sawdust getting kicked up, and that makes for a more reliable system.

Also, because it’s sending out our its own light, it can deal with really nasty targets as well. So certain targets that are, say, dark or glossy or dirty, they're going to interact with light differently. And so, if we're just relying on ambient light, that can lead to issues, but we're sending out our own light that has a high intensity of very precise wavelength. So when that light hits something like, say, forklift forks, which is a very common target that we hear about a lot from our end users, it can be a reliable detection method because, as we send out a pulse of high intensity light, it bounces off that dark target. We receive it back. It's easy for us to detect it. But with a camera or other sensor technologies, maybe not so much.

So that's one of the reasons it’s so prevalent and you see it on nearly every AMR.

David Greenfield, Automation World
Thanks for explaining that. I did not realize that about LiDAR—about it's the precision in different lighting environments. It makes sense when you think about it, especially as you mentioned with AMRs. But that's something that I hadn't really realized was a key differentiator in the technology before.

Now I noticed you mentioned using LiDAR to detect the forks on a forklift truck and other issues with things being dark in different areas of a factory and LiDAR being able to detect them. Are there other applications in the manufacturing industries for LiDAR beyond AMRs?

Aaron Rothmeyer, Sick USA
Oh yeah for sure. At Sick, we've been around for 75 years in the industrial space and spent a lot of time making sure that other applications within any kind of warehouse or industrial manufacturing facility are also seeing the benefits of LiDAR. And so some of the ones that we've seen are quality control and guidance of a traditional industrial robotic arms, inventory management and general process optimization where you're doing volumetric calculations of maybe a load on a palette. With LiDAR you can make sure that the load on the palette is properly sized before it leaves your facility.

Outside of that, the range in the field of view leads to some really good applications that that aren't typically solved by other technologies. For example, crane anti-collision. If you've got a large gantry crane overhead, typically as these cranes travel back and forth, they've got a pretty good understanding of where they are using various technologies. But determining where the actual hook of the crane is as it travels through the facility is difficult to predict. But using LiDAR and its large field view, we can look down and make sure that crane hook as it's traveling through the facility isn’t going to collide with racking or any other cranes.

And AS/RS systems, automated storage and retrieval. A lot of people with those types of systems use LiDAR to make sure that as those vehicles travel through those narrow aisles, they're not colliding with stuff that's hanging out of their racks. One of our favorite stories is about a user that stores frozen turkeys in a large cold storage freezer. And as the AS/RS system goes back and forth, if these pallets of turkeys are hanging out in the aisle and a collision occurs, well, it's a very large mess, creating a lot of downtime that everybody wants to avoid. So using LiDAR, we can look right down that aisle and, as the vehicle travels, if there's anything hanging off of the rack, we can make sure we don't have turkeys on the floor.

David Greenfield, Automation World
Yeah, you don't need turkeys on the floor. That's for sure.

You know Aaron, one thing you mentioned there in your explanation of the other applications for LiDAR, you mentioned quality control. Can you explain that a little bit better? How it's used there? Because I'm used to seeing vision sensors used here. With LiDAR is it sued here more for presence absence type detection or is it something different?

Aaron Rothmeyer, Sick USA
Yeah, vision systems are outstanding at that and we, you know, we're 3D vision manufacturers as well. We do a lot of cool stuff with with 3D vision, but quality control has a wide definition. If we want to, say, create a digital twin of an object and then verify that the produced object matches that digital twin, we can do that with LiDAR versus 3D vision a lot of times. If you're looking at, you know, a 3D vision system, it's going to be very high-resolution—sub millimeter—whereas often with LiDAR you're going to end up with something more like sub centimeter as far as your resolution and your accuracy goes, and so you gotta pick and choose which one is the best one for that object. If it is a very large object, like a vehicle body, and you want to determine that the vehicle body is formed correctly, that's something you could accomplish with LiDAR. These are cases where it would be difficult to get that field of view in that range out of something like a 3D vision system. So it’s just a question of what your object is and how precise your tolerances are.

David Greenfield, Automation World
OK. Understood. Thanks for explaining that, Aaron. So how do LiDAR sensors compare in terms of cost to other more standard position and detection sensors?

Aaron Rothmeyer, Sick USA
Yeah, that's a very insightful question, David. And to be quite honest, it's typically higher. It depends, you know, on the exact technology you're comparing to. But there are a couple things to keep in mind here.

One is that because of the range and field of view of LiDAR, often you can replace several sensors. You kind of have to think about it from a system standpoint. If you are able to replace, say, 100 traditional proximity sensors with one LiDAR, well that can be a cost savings we’ve got to keep in mind.

The other thing is just, you know, I've been watching LiDAR for last 20 years and in that time the cost has dropped dramatically—by an order of magnitude. Most LiDAR sensors used to be in the five figures. Since then, we've seen LiDAR drop into four figures and in many cases now we're able to get LiDAR sensors on the order and several hundred dollars for a single user. And so that cost, while still high, is definitely dropping dramatically. And I really look forward to seeing where it goes in the next decade or two.

David Greenfield, Automation World
So with that higher cost of LIDAR sensors versus standard sensors, how can a manufacturer determine if the use of a LiDAR sensor in an application is the right way to go versus the use of a more standard position sensor?

Aaron Rothmeyer, Sick USA
You've got to look at from a system standpoint and it all comes down to the requirements of that particular application. In most cases, if a manufacturer is looking at it, you know a sensor suite for whatever application that they have. If we know that they are looking for something with a longer range, for example, a larger field of view, if they need to see 360 degrees around a vehicle for example, or they need to see several tens of meters out or they that it's going to be a particularly difficult environment. For example, they know there's going to be a large amount of dust  or there's going to be no ambient light or very high ambient light, or it's gonna be a very difficult application. Well, then LiDAR might be a good sensor to consider.

If the application needs a longer range or higher field of view, or if there's any kind of precipitation or fog or anything like that in the atmosphere, it might be something worth taking a look at LiDAR for.

David Greenfield, Automation World
OK, now I know you've given a few examples of LIDAR use beyond AMRs here in our discussion today, but since we're talking about application examples and determining if LiDAR makes sense, can you provide an example or two of manufacturers using LiDAR in production operations and why they chose to use LiDAR in those applications?

Aaron Rothmeyer, Sick USA
Well, the first one may seem obvious at first, but I do want to point out that you know, we use our own products as an industrial manufacturer of automation equipment as well and in particular uses our products in our own manufacturing processes.

So one example of where we do that is in an application that's called pick wall. So in a lot of warehouse facilities and also just in light manufacturing facilities will have what's called pick wall where it's a series of bins, could be tens of meters long, could be just a bench top. Either way it's a series of bins and as the operator is working at that wall, they're reaching into these bins and pulling out various components, right, it could be O-rings, it could be bolts, it could be T-shirts, it could be anything, but they're reaching into these bins and they're picking out objects and the control process wants to know how many times has the operator reached into this bin or reached into that bin and the application is been around for years and it's been solved in a variety of ways, but with Sick we use one of our LiDAR products called the TiM— a 2D sensor—to create a vertical wall in front of the pick wall and as the operator reaches through the plane of our LiDAR, it's detecting where the happened. It basically translates the reach into the bin into an XY cartesian coordinate that says, OK, the operator reached into this bin.

And so now that our process control system can look at that and say OK, the operator reached into that bin, go ahead and move on to the next step. And it works very reliably with our process and it allows us to verify that the operators are doing things correctly with a single sensor.

You don't need to have, you know, a wide variety of sensors all wired up across those walls. One sensor can do it all, and so that is a big value to us not having to wire up, you know, 100 different sensors for a single wall.

But we also have other manufacturers out there, such as Omron. Omron is a customer of ours for long time. Coming from the Omron facility and those have been using LiDAR sensors for a long time basically to detect small objects. One particular use case that we ran across was on top of these AMRs. When they start up, they want to verify that nothing is around the AMR so that it can start moving in the direction that it needs to move in and those objects around it could be quite small—they could be cables, they could be wire racking, stuff like that, very difficult to detect and so they needed a sensor. The high resolution needed to pick up those types of small objects, even right on startup, the LiDAR sensors are able to do that quite readily.

So there's a lot of reasons that people use LiDAR, but those are a couple of that we have for you.

David Greenfield, Automation World
Thank you again for joining me for this podcast Aaron and thanks of course to all our listeners and please keep and please keep watching this space for more installments of Automation World Gets Your Questions Answered. And remember you can find us online at automationworld.com to stay on top of the latest industrial automation technology insights, trends and news.