Proving the Reliability of Cloud-Based Control

Most industrial engineers are familiar and comfortable knowing that a local, hardened, industrial grade PLC is controlling their process. That’s why control via cloud-based algorithms is, to say the least, an unnerving prospect for many. 

After all, what happens if communications are severed? What if the system is compromised? How can it be integrated into local operator screens? Will I still have real control? 

If the process to be controlled is in any way safety related, or requires millisecond scan times, then you should stick with a traditional PLC. However, many processes and monitoring tasks work just as well with a slower control response and are perfect candidates for cloud-based control. 

Another application that works well with cloud-based control involves adding control elements to an existing facility. 

The control strategy described below details the replacement of a $160,000 PLC cabinet with an $800 IIoT (Industrial Internet of Things) communication system. 

Wastewater processing application

At EOSi, our focus is wastewater processing. Our main business is to provide MicroC carbon supplements used to meet permitted effluent nitrogen levels. 

Wastewater processes typically have process times that encompass many hours, even days, so a 15-minute update is well-suited for a cloud-based control application. Aside from wastewater processing, checking the degradation of a filter as part of a preventative maintenance routine only requires measurements a couple of times per day, since it takes months for this measure to trend into the “red” indicating that cleaning or repair is necessary.

With those examples in mind, let’s explore the infrastructure that now supports the cloud-based control of these processes. 

First, a Windows server is installed on an Azure virtual machine, which in turn hosts an industrial application platform called Ignition from Inductive Automation. Ignition includes a full-blown historian using an Azure SQL server, Cirrus Link’s MQTT Engine as well as Twilio’s email/SMS alarm notification module.

In the field, Ranger cellular transmitters from SignalFire are used to post discrete analog process values to the MQTT broker in Ignition. IIoT gateways from Moxa with Ignition Onboard are used to pull and push, where appropriate, other data points from memory space shared between the gateway and the customer’s PLC network. The Moxa devices have firewalls, are accessible only via VPN and have physically separate NIC cards to ensure there is no active connection between the subnet used for the PLCs, the network subnet used internally by the MQTT gateway, or the cellular network. The MQTT protocol transmits tiny amounts of non-contextual numerical data on this third cellular connection which is also encrypted and linked to CA certificates, therefore interception or hacking is not a concern. 

Functionally, MQTT enables live process values to appear in the Ignition tags. Formulas in what Ignition calls “expression tags” provide the control algorithm. The results of the algorithms are then sent back down via MQTT to the IIoT gateway and then to the local PLC for further analysis or direct control of analog outputs. 

Using Inductive Automation’s Perspective templates, an HMI is developed for each site or customer at very little additional cost, since the Ignition license allows unlimited users, unlimited tags and unlimited screens. These HMIs can run on any platform—Windows, Mac or Linux, as well as iOS and Android phones and tablets. 

Ensuring system reliability

Our initial concern with this cloud-based control system was reliability. The cloud is a great place to store your backups but using it to actively compute process values for real-time control is another issue entirely. To alleviate our concerns here, each interface we developed featured a watchdog element monitored on the Ignition platform.

Following is a rundown of the connections and communications we developed for the system to ensure its reliability:

• The Ignition platform is monitored from uptime.com, a third-party service.
• Site PLC communication focused on a heartbeat value from the PLC to ensure site communication was functional. 
• Another heartbeat value was created to ensure communication between the on-site IIoT gateway and Ignition platform. 
• Internal checks between Ignition and the SQL database were developed to monitor this connection. 
• If any of these watchdogs fail, notifications configured with Twilio notify plant operators and EOSi personnel via email or text.  After four years of installations, the only failure mode we’ve seen has involved communication between the site PLC and local IIoT gateway. In each case it was found to be a power failure at the PLC—sometimes intentional due to maintenance or caused by a local power outage. In one case, we were able to notify a customer of the issue via our monitoring of the system before they had noticed it themselves! 

It is worth noting that because these watchdogs are monitored in the cloud, they are always active, regardless of local conditions. In other words, we’ve found this methodology to be more reliable, not less, than locally installed PCs and PLCs.

Cost savings

This cloud-based monitoring system allowed us to move from using the traditional full-blown PLC cabinet to a very small box and reduced the cost by almost two orders of magnitude. As a business model, this cost reduction has allowed us to go from the tedious process of bids, submittals and hurry-up-and-wait PLC cabinet installations to one where we can provide our Virtual Nitrack BNR (biological nutrient removal) control system at no additional charge as long as the client is buying our MicroC products.

Our usual deliverable is a small electrical enclosure containing the communication gateway and power supply. The smallest version of the system we’ve created was implemented by us mailing an $800 gateway to a controls electrician who snapped it onto an inch of DIN rail in an existing PLC cabinet, wired power to it and plugged in the Ethernet cable. Because we had configured the gateway before mailing it, the electrician had everything up and running in hours, not days. 

Randall Pulsifer is automation manager at Environmental Operating Solutions Inc.