Formally organized in 1834, Waterford Township is located geographically in the center of Oakland County, Michigan, and is home to over 72,000 residents. With 360 miles of water main and 355 miles of sanitary sewer, water management in Waterford is no small task. The Department of Public Works (DPW) operates and maintains 19 production wells, 3 storage tanks, 11 treatment plants, and 63 sewer lift stations.
To run all this, they invested years ago in integrating core applications, including geographic information systems (GIS), asset management systems (AMS), enterprise content management (ECM), and supervisory control and data acquisition (SCADA).
That system delivered a lot of value over the years, but nothing lasts forever.
Time to upgrade
In 2017, Russell Williams, director of public works, and Frank Fisher, engineering superintendent, at Waterford DPW started on a project to upgrade their core SCADA infrastructure. The next year, they attended a conference announcing the release of Opto 22’s groov EPIC and were excited by the potential of MQTT Sparkplug to eliminate some long-standing systemic limitations.
With more than 90 controllers on their network, the polling mechanism they used, combined with the limited bandwidth of their radio network, meant that data from each site would update only every 3-5 minutes. Sometimes a lift station would run briefly in between polling cycles, creating gaps in their reporting and inhibiting operators’ ability to accurately detect issues until alarms eventually made their way through. And for each I/O point they added to the system, this latency only grew worse.
It seemed clear that MQTT’s report-by-exception behavior could significantly reduce bandwidth usage and ensure delivery of important system actions.
“We have many lift stations that will spend most of their time sitting,” Williams explains, “[So] why transfer data all the time?”
And with no dependence on a central polling program, they saw the possibility to eliminate a systemic bottleneck and potential point of failure.
From proof-of-concept to production
To help them execute their vision, Waterford DPW engaged Perceptive Controls, a Michigan-based system integrator specializing in industrial and process control applications for the water/wastewater, food and beverage, and oil and gas industries. But building an MQTT system for the first time came with a learning curve, according to Kevin Finkler, software engineer at Perceptive.
MQTT’s publish-subscribe communication model is a definite departure from that of traditional industrial protocols in a few key ways:
- Each field device connects only to the broker, regardless of where its data needs to go.
- When using Sparkplug payloads, each device publishes (sends) a list of its available data items, called topics, upon establishing a connection to the broker.
- Other MQTT clients may also connect to the broker, see the available topics, and then subscribe to (request) updates on those topics when published by field devices.
After experimenting with a few popular SCADA packages, Perceptive Controls decided on Ignition by Inductive Automation because it offered very tight MQTT integration, including the ability to serve as an MQTT broker itself.
Even though understanding the MQTT communication model took Finkler some work at first, establishing communication was straightforward in the end.
“It kind of happens automatically,” Finkler says. “You basically define a few parameters [in Ignition] to set up the broker. And each of the EPIC devices was pretty simple. You just point it at the broker and it starts sending tags.”
“I love that both of these sides have embraced MQTT,” adds Fisher. “It makes the connection seamless.”
