A couple of years ago, executives at Fusion UV Systems Inc., a provider of ultraviolet curing systems and equipment, decided to upgrade the company’s software system. While the manufacturing system at the Gaithersburg, Md., UV plant was doing just fine, reporting was in terrible shape, and the sales team couldn’t access information on orders. “We were doing lots of workarounds and everybody was using Excel spread sheets,” says Julie Sharpe, Fusion UV’s sales support manager. “We couldn’t tell customers when their orders would ship, and we had no idea when the system might collapse.”
Being a mid-size manufacturer, Fusion UV executives were surprised that SAP AG, the giant Walldorf, Germany-based enterprise software vendor, was willing to set them up with an enterprise resource planning (ERP) system. The tricky part was linking the new ERP system to the plant.
Like many manufacturers, Fusion UV decided to make the move when the plant was slow. “We shifted to SAP over the 2003 Christmas holiday and used that slow time as cover,” says Sharpe. “During the holiday, most of our shipments are just spare parts.”
The company went live on Jan. 5, sending out automated order acknowledgments for the first time. “It sounds simple, but it was a lot of work,” says Sharpe, noting that weeks of planning and training preceded the upgrade. “We had to reconstruct all of our BOMs (Bills of Material) before the Christmas shutdown.”
Keep it simple
The company also chose to do as little custom work as possible, as it would be easier to upgrade in the future if the system was basic. “We planned to keep it as simple as possible,” explains Sharpe. “The more custom the system is, the more expensive it is to upgrade it later. So we adjusted ourselves to the software rather than adjusting the software to our practices.”
Even though Fusion UV’s automation platform itself was not replaced, the upgrade to connect to the SAP ERP system meant that the plant had to be shut down and effectively rebooted. Upgrading plant software is difficult business. A small security upgrade that may be a no-brainer to the Human Resources department can be a monstrous disruption on the plant floor. Plant managers must carefully plan their upgrades, which today are coming at an increasing pace. Plant managers are feeling pressure to connect the plant system to business systems, install audit trails to satisfy regulators and add functionality that extends the life of the plant system.
Plant operators are finding it difficult to avoid upgrades. Economic and regulatory pressures usually drive the upgrade. “Typically, the upgrade is more for economic reasons than for maintenance,” says Dave Woll, vice president of consulting at ARC Advisory Group Inc., Dedham, Mass. “The only justification for an upgrade is that the unreliability of the automation system may affect the activity of the plant.”
Upgrades are usually only conducted if they can demonstrably improve the plant. “There is a golden rule to upgrades. You have to make sure the result is something equal to or better than what you started with,” says William Robertson, director of worldwide services at Emerson Process Systems and Solutions, in Austin, Texas. “The software upgrade should allow you to extend the investment life of the system, or increase the system’s results with new technology.” Robertson notes that plants make major investments in their control systems as capital assets, and they want to preserve that asset as long as possible. Upgrades allow them to increase the life of the asset.
Managing upgrades requires intricate planning at most plants, because downtime must be kept to an absolute minimum. Often, the planning process of the upgrade involves a thorough look at what potentially needs upgrading. “While simple version upgrades are most common, frequently, production upgrades are planned from the bottom up, from the PC hardware and operating system to the applications and connectivity,” says Nancy Venable, manager for OEM/VAR and services programs at Invensys’ Wonderware division, based in Lake Forest, Calif. “When production systems run 24/7, upgrades are planned to coincide with scheduled maintenance downtime.”
IT Jumps In
Plant managers are not alone in their upgrade activity these days. Often, the information technology (IT) department also rolls up its sleeves to help plan the plant upgrades. “Many of the plant operators at large multinational manufacturers are talking more to the IT folks,” says Emerson’s Robertson. “IT and control both need to be involved in upgrades. When you’re passing information between a control valve and a server, you don’t want anything inappropriate to get passed along.”
Some vendors go to great effort to make generation upgrades easy on their customers. “Our upgrade package performs updates automatically,” says Roy Kok, senior manager for Proficy at GE Fanuc Automation, based in Charlottesville, Va. “The upgrade will recognize the old system and migrate files from the old system and make sure they’re compatible.”
Software upgrades are easy for Rick Spreeman, plant technician at Waupaca Foundry, a Waupaca, Wis.-based automotive supplier that makes iron castings. “We shut down a couple of weekends a month, which allows us to get access to the computers,” says Spreeman, a customer of Tecnomatix, a unit of Plano, Texas-based UGS Corp. During the shutdown, plant operators can take computers up to the IT department for security patches and Windows upgrades. “We can have it switched over in no time,” says Spreeman.
Spreeman’s situation is an exception to the general rule that upgrades are difficult. Yet even with the difficulty, an upgrade doesn’t necessarily mean that production must be halted. “You don’t have to shut the plant down to do an upgrade,” says Sudipta Bhattaharya, vice president of manufacturing applications at SAP. “You can conduct the upgrade in phases. Or you can run a parallel system and make a gradual transition.”
ARC’s Woll notes that many vendors are turning to online connectivity to upgrade parts of the control system without stopping the plant’s production. “There’s a big push to build online upgrade capability, where the vendor can do the upgrade without shutting down the plant.”
Software vendors often design their upgrades so only a small portion of the upgrade requires a plant shutdown. For some upgrades, a shutdown is not even necessary. “Upgrading is hard, but backing up the software can be done while the plant is active,” explains Gregg LeBlanc, product director and strategist at OSIsoft Inc., San Leandro, Calif. “We can do it hot. We can just swap in the new software and swap out the old while the plant is running.”
Suites or best-of-breed?
One way to keep upgrades simple is to choose a software suite with a number of modules rather than cobbling together software from a collection of vendors. Each vendor has its own upgrade cycle and each has its own compelling reason to upgrade to new releases. The more vendors you use on your system, the more frequent your upgrades. Plant managers also like to minimize the number of vendors that have to be held accountable for the smooth running of the system.
In recent years, suites have grown to match much of the best-of-breed specialty software. “Manufacturers used to go out and search for best-of-breed, but today, all products are pretty darn good,” says GE Fanuc’s Kok. “So it has become more important to buy an entire solution from one vendor. You get a better level of productivity.”
With all the pressure to frequently upgrade the plant system, managers must plan their software maintenance carefully. Most upgrades are critical. One may extend the life of the plant system, another allows the plant to meet new regulatory demands, and a third lets the ERP system dip into the plant database to check on inventory consumption. Because of the constant pressure to upgrade, those who cut their teeth on automation systems now have to become IT specialists as well.
As one plant manager told Emerson’s Robertson, “I graduated as an instrumentation and control engineer, but now I’m at a large plant that runs 50 servers, and I feel more like an IT manager than a control systems manager.”
For more information, search keywords “software upgrades” at www.automationworld.com.
See sidebar to this article: Why Do We Need an Upgrade, Anyway?
