It might not be the holodeck on the Starship Enterprise, but the simulator at Sandia National Laboratories certainly shows that virtual reality is not just science fiction anymore. Sandia’s technicians use their simulator to practice disassembling and reassembling a fairly large classified device that is too sensitive to permit frequent handling. While immersed in virtual space, they learn to service the device without ever touching it.
To begin a session, a technician dons a headset and a pair of gloves. An array of sensors in the gear broadcasts the position of the technician’s head, hands and fingers so that the software can generate the appropriate view on the headset’s display. The software even simulates the sensations of touch and weight by sending the necessary force feedback instructions to robotic arms attached to the technician’s hands.
Successful applications like this suggest that it won’t be long before immersive virtual technology will make major inroads into manufacturing. Especially with recent developments, virtual technology could be poised to become a standard tool for optimizing assembly lines and reducing manufacturing costs.
>> Google Earth Connects Ford's Virtual Assembly. Click here for more information.
One of these developments is VRTools from Integrated Engineering Solutions (IES, www.ie-sol.com), a consulting company in the research park at Washington State University (WSU) in Pullman. Because the founders of the company are professors at WSU, they developed an early version of VRTools for Sandia by applying lessons from earlier research sponsored by the Virtual Assembly Technology Consortium.
“Our goal was to do research in virtual assembly, using immersive tools such as headsets, gloves and other tracking devices,” explains Sankar Jayaram (Jay), president of IES and director at WSU’s Virtual Reality Computer Integrated Manufacturing (VRCIM, www.vrcim.wsu.edu) Laboratory.
Founded in 2000, the consortium pools the resources of WSU’s VRCIM Lab, the National Institute of Standards and Technology (NIST) and several industry partners. Called the Virtual Assembly Development Environment (VADE), the software that resulted from their work permits users to interact with models created on computer-aided design (CAD) systems.
VRTools takes the capabilities of VADE to the next level. “Our technology gives people the ability to go into a visualization environment and see all the parts, to create cross-sections and look at them,” Jay says.
Not only can engineers see an assembly in its actual size and context, they can manipulate it to improve its design and the process to assemble it. “If you’re sitting in an airplane seat, for example,” offers Jay, “you can turn your head to see what the rest of the plane looks like. You can push the seat away from you a little to get a few more inches of legroom.”
Users can also immerse themselves at an assembly line. While standing in a virtual assembly station, they can check whether parts go together as they should and perhaps even wield virtual tools like wrenches, screwdrivers and power tools. In fact, IES developed a virtual saw so Sandia’s technicians could cut through panels and create cross-sections without actually altering the device.
“The user would run the saw through the model,” recalls Charles DeChenne, director of R&D at IES. “Just as you would when cutting a two by four, you see the slit that the saw blade made through the material. And when you cut all the way through, the material splits into two pieces.” Throughout the cutting process, the software updates the CAD model, removing the material from the kerf in real time.
VRTools can do this because the researchers were careful to maintain an intrinsic link to the CAD model. “Engineers do almost all of their work in CAD,” explains Uma Jayaram (Uma), executive vice president at IES and associate director of WSU’s VRCIM Lab. “So, unless the software is connected in a meaningful manner, it’s unlikely that the information will have an impact on what needs to be done. You would write a report that would just sit on somebody’s desk.”
To avoid such inaction, VADE and VRTools maintain bi-directional associativity with the CAD model in two ways—through the assembly trees and the parametric definitions. The assembly tree in the actual assembly process must have a one-to-one correlation with the tree in the CAD model. “At the beginning of our research, we thought naively that the order we would receive from the CAD system would be pretty close to how they would be assembled,” says Uma.
When she and her colleagues discovered that the trees in the CAD models typically did not conform to the actual assembly trees, they invested the time to develop a way to modify the CAD trees to match the actual, as-assembled trees. So now, once the user determines how the pieces will be assembled, the software propagates the new tree back to the CAD software to update the model.
It also has a mechanism for propagating any parametric adjustments made to the virtual assembly during the simulation back to the CAD model. Consider a scenario where, after importing the CAD model into the virtual environment, the user discovers that the clearance between two parts is too small to access a bolt with a wrench. “You can move either the hole or a part a little to see if it would make a difference,” says Jay.
Because those changes occur parametrically, they go back to the CAD software and update the model, which is then used to refresh the image in the virtual space. Interference or any other problems that the proposed changes would pose for the CAD model would be flagged right away.
Will your plan work?
Testing assembly strategies in virtual space before implementing them has a number of advantages. To illustrate one, consider the problem of putting together something simple, such as a prefabricated bookshelf that you might buy at an office supply store. Because the kit may have only 50 parts or so, you might start putting it all together without reading the directions.
“It may look obvious at first glance,” suggests Uma. “But when you come down to the final steps, you realize that this extra piece should have been the one used in step number five.” Correcting the mistake now, which may be anywhere between five and 10 steps later, creates a lot of rework.
The complexity of such sequencing problems is only magnified in the assembly of vehicles, appliances and other complex products made on assembly lines. “The whole problem gets way out of hand very quickly when you are looking at maybe 1,000 subassemblies,” says Uma. “The nomenclature, the order, and how they behave are just much more complex.” For these situations, she recommends testing your assembly process in virtual space rather than simply tracking the sequence on a spreadsheet because it fosters analysis and troubleshooting.
Simulations in virtual space are not only useful for planning the sequence of operations within a line. They also hold promise for reducing the cost and risk of building the line itself. Consider the work conducted by WSU researchers and an engineer from Tokyo-based Komatsu Ltd. over a year and a half. Among the problems that the team studied using VADE was the assembly of the machine tool builder’s big tandem and transfer presses in its customers’ facilities.
Once Komatsu builds and proves a large press in its own factory, it has to disassemble it into pieces that are practical to ship to the customer’s site. Then its technicians reassemble the pieces in the spot prepared for the press. “There is a lot of time and money that goes into this, so they are constantly looking for new tools and ways of doing things that would help mitigate their risk,” says Uma. The researchers were able to use VADE to take the constraints and tools at the site into consideration in developing their delivery and assembly plans.
Optimizing existing lines
The researchers have also investigated how virtual tools like VADE and VRTools might help manufacturers to improve their existing assembly lines and trim their current costs. “Very often, vehicle manufacturers, for example, would like try something different,” says Uma. “In no way, however, can you bring the whole line down.”
Not only does virtual technology permit engineers to test new ideas without shutting down a real line, but it also can streamline complex, comparative studies. An example is the virtual models that the researchers at the consortium made in VADE for an automotive supplier that wanted to standardize a portion of its chassis lines across several plants. The models replicated both how the overhead cranes presented the assembly to each station in the process and what happened within each station.
“Using the models, the engineers proposed changes for standardization,” reports Jay. “They came up with a process for assembling this piece in exactly the same way in all locations, given the constraints at the physical factories and the requirements for each operation.”
In another study conducted without stopping the line, the researchers investigated the risk of repetitive-stress injury to the wrists of workers who were inserting snap rings into pistons on an engine assembly line. The researchers generated a model of the workstation in Pro/Engineer and put the appropriate CAD-modeled parts into the workstation’s various bins. They then imported the models into VADE. By immersing the workers in the virtual workstation, they were able to capture their movements and analyze the position of each joint with a Rapid Upper Limb Assessment (RULA) algorithm.
“We discovered that the risk of wrist injury was high when a worker was reaching for part of the piston on the top shelf,” says DeChenne at IES. Another contributor to the risk was the height of the assembly fixture. Simple adjustments for height and reach relieved the stress.
As powerful as these technologies are, virtual assembly has found few industrial applications to date. The reason is that the technology is still in its infancy and therefore has been too expensive for most assembly plants to justify. “The very first immersive system that I put together 20 years ago cost a half million dollars,” notes Jay.
Although costs have been falling since then, they have remained way too high to put on the typical engineering workstation. Over the last 10 years or so, for example, a headset with engineering-grade capacity would typically cost tens of thousands of dollars. Gloves and other tracking devices have been in the same price range. By the time you buy the software and computers capable of running it, the price tag of adding virtual reality to an engineering workstation could easily exceed $50,000.
The situation has changed within the past year or so, however. Because the trend in computer games has been toward providing users with an immersive experience, the gaming industry has invested the resources necessary to jump the cost hurdle. “The processing capabilities for computer graphics have exploded,” offers Jay. “This means that the large CAD models that required very powerful computers to view can now be seen on gaming computers, which cost $2,000 or $3,000 each.”
Meanwhile, the cost of the required tracking devices has also plummeted. A headset with the required capacity, like the Occulus Rift, is now available for a little more than $300. Intel and others have developed similarly priced camera-based tracking devices to replace the gloves and other sensors. “You can move your fingers and hands in front of the camera, and it will detect and compute your finger movements in real time,” says Jay.
Cost is not the only obstacle to wider use of virtual assembly technology. Another has been the learning curve. “Engineers are used to using the CAD tools on their desktops on a daily basis, so using something different is not a change that you are going to bring about overnight,” explains Jay. “Any change has to come from within.” In other words, it has to streamline what they are doing if it is going to be accepted and used.
For this reason, Jay and his colleagues at IES developed VRTools to work inside two CAD packages—Catia from Dassault Systèmes (www.3ds.com) and Pro/Engineer from PTC (www.ptc.com). A user clicks on an icon inside the CAD program to launch VRTools. “Once you put on the headset, you’re inside the truck cab, airplane or other environment and can start touching things and moving them around,” he says.
This confluence of developments promises to transform virtual assembly into an affordable and practical technology. Just as advances in consumer communications have made mobile devices practical for an increasing number of industrial settings, so do advances in gaming technology seem ready to do the same for immersive virtual-assembly technology.
