Assembling a complex piece of machinery like a gas turbine is a costly and time-consuming process. It’s no wonder that it can take several years for a fitter to be properly certified—typically weeks of classroom learning followed by additional blocks of hands-on training, working with experienced counterparts to master the intricacies of assembly.
As part of a series of pilot projects to explore potential use cases for augmented reality (AR), Siemens Power Generation Services partnered with Daqri, which makes AR smart glasses, to try to significantly reduce assembly training time for the burner portion of its gas turbine. Together the companies developed an AR app that facilitates the training process by swapping out paper- or tablet-based technical drawings, checklists and manuals with virtual step-by-step work instructions that show trainees exactly what to do during the assembly process using 3D visualizations. The AR app dramatically increased the speed and efficiency of the training, particularly for novices, who were able to complete a portion of the curriculum in 45 minutes instead of the usual day.
“Today, a classic training always requires a trainer who is explaining content to trainees and who is supervising them,” says Frank Voßnacker, innovation manager for Siemens Power Generation Services. “Having an AR app available on the Daqri helmet lets users learn at their individual speed, and lessons can be repeated again and again without needing to involve the trainer or interrupt other trainees.”
Like Siemens, manufacturers across industry are knee-deep in exploring new ways to harness the power of mixed-reality technologies like AR and virtual reality (VR). According to IDC, 2017 was an inflection point for AR adoption—worldwide spending on AR/VR technology is expected to leap to $17.8 billion this year, almost double the $9.1 billion spent in 2017. Commercial deployments will represent more than 60 percent of AR/VR spending this year, growing to more than 85 percent by 2021, IDC reports. On-site assembly and safety, process manufacturing training and industrial maintenance will be among the most widely deployed use cases of AR/VR in the manufacturing sector, the research showed.
For PTC—a leading champion of AR vis-à-vis its acquisition of Vuforia in 2015—service and maintenance applications currently account for more than one-third of its customer use cases. However, manufacturers are starting to venture into lesser known territory, including experimenting with AR technology for collaborative design reviews, virtual work instructions for shop floor and industrial asset operators, customer self-service, pre-sales aids and, much like the Siemens gas turbine example, for next-generation training.
As the physical and virtual worlds converge as part of Industry 4.0 digital transformation, PTC contends AR is more about becoming a vehicle for interpreting and visualizing the Big Data of the Internet of Things (IoT). “Given the amount of data everyone is inundated with, there’s no better way than AR to make sense of it,” says JP Provencher, PTC’s vice president of connected operations solutions. “It simplifies data and puts it in context of the machine you’re looking at.”
Even so, AR isn’t a candidate for every training or maintenance and service use case. “It makes sense where there’s complexity or if you’re not doing something often,” Provencher says. In those cases, “AR helps with reducing training time, improving execution time, and can help validate that someone performed the right tasks.”
One possible use case for Daqri’s Worksense is getting hands-free remote expert guidance.
AR at work
Since it rolled out its HoloLens commercial suite in July 2016, Microsoft has seen an uptick in activity on the industrial AR front, across vertical industry segments with many customers moving from pilot stage into production. Independent software vendor (ISV) partnerships, including one with PTC, are essential to driving deployment activity, making it easier for companies to develop AR apps with out-of-the-box functionality instead of requiring the skilled resources of do-it-yourself (DIY) custom programming, says Greg Sullivan, Microsoft’s director of communications for mixed reality.
For example, as a result of the Microsoft/PTC collaboration, ThingWorx Studio now supports native authoring and publishing of AR experience for HoloLens. “Really quickly, the program transitioned from a bunch of folks kicking the tires and thinking about the realm of possibilities to production deployments,” Sullivan says.
AR/VR technology provides step-by-step instructions for Honeywell customers to perform inspection procedures in a much simpler way.
Here are some of the more popular use cases where AR is gaining a foot hold:
Operator work instructions. Given the rising complexity of heavy equipment and industrial assets, experts see big potential for leveraging AR as a more intuitive and engaging way to deliver work instructions to shop floor operators and workers in the field. Highly configured products like offshore power generation units or agricultural combines are complex to assemble, typically requiring operators to juggle binders of paper documents and several screens of how-to instructions on proper procedures. “AR is a way to deliver all the right information in one screen in the right context and overlay it on top of the physical unit where there is work in progress,” explains Milan Thakker, an IoT manufacturing solutions specialist for PTC. “The operator doesn’t have to put things down or pick things up and they don’t have to look at a specific set of work instructions to see the information they need in context to what they’re doing.”
Pre-sales support. For highly customized and complex products, AR provides a way for customers to visualize the product in their specific setting well before it’s built. While this application is a natural fit for cars and other industrial products, thyssenkrupp is using Microsoft HoloLens to visualize, configure, price and quote a stair lift in real time. No two stair lifts are alike, with each system custom-designed and tailored to an individual’s requirements; HoloLens ensures customers can visualize what their chair will look like on their staircase. At the same time, the AR tool, combined with a custom-built Configure Price Quote (CPQ) solution, lets the thyssenkrupp team measure, visualize, configure, price and quote a stair lift in one visit, greatly streamlining the process. “This fully digitized engineering-to-order process allows both sales and manufacturing operations to sync up in the digital realm and significantly reduce the time from initial call to final delivery,” a thyssenkrupp spokesman says. “Above all is the impact the process has on enhancing the customer experience.”
Remote expert guidance. It takes highly specialized expertise to maintain and operate complex industrial assets, but that manpower isn’t always available on site and it can be expensive to get it there. AR—supplemented with 3D annotation capabilities and in some cases video calling—connects field technicians and operators with remote experts, who can see what’s happening with a piece of equipment without having to be physically present. Thyssenkrupp’s elevator group is leveraging Microsoft HoloLens to give elevator support repair personnel the ability to reach out to expert advice since they are not always 100 percent familiar with every elevator they are called on to service. Another HoloLens partner, Trimble, is tapping AR technology as part of its Connected Mine offering to allow remote teams to share 3D holographic data to improve communication, save on travel time, and make more informed decisions, says Aviad Almagor, director of the mixed reality program at Trimble. “The digital and physical worlds are merging, and mixed reality plays a vital role,” he says. “Enriching mixed reality with AI [artificial intelligence] and machine learning processes will enable automation of workflows such as quality control and production control.”
HMI replacement. AR/VR also has potential as an alternative to the traditional human-machine interface (HMI), according to Andy Stump, business manager for design software at Rockwell Automation. Instead of manually checking individual HMI screens on equipment, AR allows operators to walk the floor and see relevant information such as tank capacity, temperature or line speed overlaid on the equipment and contextualized in line of sight, Stump explains. Coupled with a digital twin, which serves as a real-time data feed, there’s greater potential for optimization and predictive maintenance. “Using the digital twin and AR, you can optimize in the virtual world before you do something in the physical world,” he says. “It may not always be practical to test something out in the physical world.”
Quality control and inspection. Given the harsh environments in the process industry, quality control is a more effective application for AR than maintenance, notes Youssef Mestari, program director for Honeywell’s Connected Plant program. Field operators are tasked with making regular inspection rounds to check that equipment temperature is right or that there are no leaks or sounds coming from a compressor. Using AR tools and with a 3D model of the asset or plant, operators can receive a step-by-step inspection route with the right data overlaid directly on the asset, he explains. For example, Continental Tire has a pilot application that leverages AR technology with Iconics HMI/SCADA systems to facilitate tire inspection. “It makes a nice inspection quality control system that is hands-free,” explains Russ Agrusa, Iconics’ CEO and president.
Iconics delivers mixed-reality capabilities that augment HMI/SCADA applications, allowing operators to view real-time and analysis data in hands-free operation.
Challenges remain
While there have been huge strides made in hardware, some of the AR devices tend to be heavy and bulky, making them uncomfortable to wear for full-day shifts. In addition, there are still safety concerns as operators equipped with AR glasses could get caught up in the virtual world and lose grounding in their physical surroundings. “Creating situational awareness is key for successful deployment,” PTC’s Thakker says.
3D and AR development skills are another barrier because most manufacturers don’t have the on-staff expertise to develop AR apps. In fact, the lack of packaged AR solutions prompted the recent rollout of Daqri’s Worksense productivity suite of software for its AR smart glasses. Worksense offers five common use case apps for AR, including: Show, for hands-free remote expert guidance and annotation; Tag, for attaching critical information like notes or multimedia assets like training videos to objects and assets to improve management and operations; Scan, for mapping out the environment and equipment to create a 3D digital model to aid in maintenance tasks or plant retrofits; Model, initially for creating immersive, full-scale walkthroughs using Autodesk BIM Docs; and Guide, for creating multimedia work instructions.
Microsoft HoloLens mixed-reality technology allows thyssenkrupp stair lift customers to visualize in real time what products will look like in their own homes while reducing the delivery time to the customer by up to four times.
“Over the last year or two, a lot of companies have done proof of concepts around AR, but it required custom software development in order to make it work,” says Jason Haggar, Daqri’s vice president of global partner and developer programs. “We’re trying to take existing business processes and make them better through AR. It’s less about doing something like a science experiment and more about making it accessible for companies of all sizes to do real AR apps.”
