The message queuing telemetry transport (MQTT) protocol is a key contender for the most favored method of data transference. The main reason why is MQTT’s open-source design and lightweight stature make it well suited to connect disparate devices to supervisory control and data acquisition (SCADA) systems as well as other industrial networks.
As Omer Qadri, product marketing manager for edge and HMI products at Aveva, explains, MQTT uses a publish/subscribe architecture that reduces bandwidth utilization by 95% compared to traditional polled communications and client/ server communications using the hypertext transfer protocol (HTTP). “An HTTP header is typically around 8,000 bytes,” he says, “but the MQTT protocol uses only two bytes and a few lines of code.” This is key in an era where millions of IIoT (Industrial Internet of Things) devices have been deployed, many with low internal memory and processing power.
Besides having a much smaller footprint on the network, MQTT’s publish/subscribe architecture is also flatter than the architecture used by traditional industrial automation protocols, such as Modbus, EtherNet/IP, and Profinet. “This [MQTT] architecture replaces the traditional automation pyramid,” observes Garrett Schmidt, senior product manager for communication interfaces at Phoenix Contact USA.
Whereas clients in a client/server architecture communicate directly with an endpoint or server, publishers and subscribers— message senders and recipients, respectively—never talk directly to each other in a publish/subscribe architecture. Rather, they communicate with an intermediator called a broker; the publisher supplies the broker with data and the subscribers consume the data.
“The broker can reside anywhere—on the cloud, on a private server, or just running on a PC somewhere,” says Schmidt. “It filters the incoming messages and distributes them to the appropriate subscribers.”
He adds that this decoupling of publishers and subscribers enhances flexibility in IIoT applications in at least three ways: “First, publishers and subscribers only need to know how to contact the broker, not each other. Second, a broker can store messages for clients that are not online and deliver them when the resource is available. And third, operations do not have to be interrupted when waiting to receive or publish a message to coincide with the asynchronous nature of most client libraries.”
MQTT also has the advantage of being an open-source protocol built upon TCP/IP (transmission control protocol and internet protocol). In essence, MQTT permits users to send TCP/IP messages back and forth, according to Arlen Nipper, a co-creator of MQTT and president and chief technology officer at Cirrus Link Solutions.
Like HTTP, MQTT defines only a transport protocol. It doesn’t provide for security; it relies on TCP/IP for that. Like HTTP, MQTT also doesn’t define a payload specification. Although being payload agnostic offers the flexibility to transfer any payload, including those from legacy systems, it can complicate the connection of some devices. In these cases, a programmer would be required to translate the data.
To eliminate this translation work and streamline implementation, the open-source Sparkplug payload specification was released in 2016. “It marked the first attempt to standardize on an interoperable format for MQTT in industrial applications,” says Josh Eastburn, director of technical marketing at Opto 22.
In 2018, the Eclipse Foundation sponsored the Tahu Project, which collected reference implementations of Sparkplug. The result has been the emergence of plug-and-play IIoT devices using MQTT.
Nipper says Sparkplug does for IIoT what the hypertext markup language (HTML) did for the Internet of People. Consequently, he is expecting IIoT applications to explode, as the Internet of People did once both HTTP and HTML were defined.
Explosive growth expected
MQTT is already making significant inroads
in industrial automation, as well as enjoying
widespread use in other applications. Facebook,
for example, adopted it as the transport
layer for its Messenger app back in 2011.
“Literally overnight, 800 million people were using MQTT,” notes Andy Stanford-Clark, MQTT’s other co-creator and distinguished engineer and master inventor at IBM UK.
Since then, other Big Tech companies have followed suit. Amazon’s AWS, Microsoft’s Azure, IBM’s Watson, and Google IoT platforms, for example, all are using MQTT. With such broad uptake, MQTT overtook HTTP in 2018 as the transport protocol of choice for the Internet of Things, reports Stanford-Clark.
Many automation suppliers expect MQTT to eventually dominate the industrial networking space. “We believe MQTT will become the de facto industrial standard in the next 10 years,” predicts Qadri. “It will enjoy widespread adoption as industry replaces legacy Modbus, OPC, and other telemetry protocols that are still predominant in SCADA applications.”
Key milestones
The success of MQTT in the consumer space has obscured some fundamental facts about its origins. Namely that the protocol has been around for 23 years now and was originally developed for industrial automation, specifically for Phillips 66.
The development of MQTT occurred after AT&T had been broken up and a number of vendors began offering their own SCADA systems to deliver data in real time by satellite. “Every one of those companies had a proprietary transport layer,” recalls Nipper, who, at the time, was with Arcom Control Systems Inc., a company he had co-founded and which is now part of Eurotech.
The one exception was AT&T, which designed its new SCADA offering to run natively on TCP/IP. Phillips 66 had installed one of these systems and asked Nipper for help with increasing the efficiency of realtime data flows between field devices and multiple data consumers. “Polling over a VSAT [very small aperture terminal] is slow,” explains Nipper. “And it was very expensive if you had hundreds of sites, like we did at Phillips 66.” Other constraints included the use of devices reliant on 8-bit embedded microprocessors and 300-baud communications.
Because the SCADA manager at Phillips 66 wanted to replicate the success the IT department had been having with messageoriented middleware (MOM) from IBM, he introduced Nipper to IBM’s Stanford-Clark. In 1999, the pair developed MQTT for MOMbased SCADA.
Despite being an efficient, open-source protocol, MQTT would not gain much momentum for nearly a decade. “It wasn’t until the protocol became available in a royalty- free license that it began to catch on outside of IBM,” explains Eastburn. “In 2010, Mosquitto, the first open-source MQTT broker was released, proving that MQTT had a life outside of IBM and marking a turning point in its adoption.”
Two other milestones in industry’s adoption of the protocol occurred in 2011. First was the Eclipse Foundation initiating the Paho Project, which collected MQTT clients implemented in various languages. “In 2011, IBM and Eurotech donated MQTT client implementations in C and Java to the foundation, allowing for a complete MQTT system to be built from open-source components,” says Eastburn.
That same year, IBM also began the standardization process of MQTT with the Organization for the Advancement of Structured Information Standards (OASIS) ultimately adopting version 3.1.1 as a standard in 2014. Then, in 2016, the International Organization for Standardization (ISO) and the Geneva-based International Electrotechnical Commission (IEC) also approved it as ISO/IEC 20922:2016.
To keep up with advances in related technologies, OASIS released version 5 of MQTT in March 2019. This version allows users to do new things with MQTT via the cloud, large distributed infrastructures, and clusters of multiple brokers. “We were careful not to let too many things creep into it, as we have to stick to the founding principles of keeping the protocol easy to understand and not very chatty on the wire,” says Stanford- Clark. ISO is currently considering adoption of version 5 as well.
Potential application concernsDespite the success MQTT and its publish/ subscribe architecture have had, it’s not optimal for every application, according to Kenneth Tran, founder and CEO of Koidra Inc., a supplier of artificial intelligence-driven IoT technologies. “We find the pub/sub model is often not the best solution for higher-level applications, in part because they must be configured to consider asynchronous data availability,” he says. “In a factory, it’s typical to have many sensors connected to a controller, server, or sensor hub in the field.”
In the IoT systems that Koidra offers, an on-premises IoT hub aggregates data pulled from a factory’s sensors via smaller, local sensor hubs. “These IoT hubs perform lightweight data cleansing, processing, and compression— and then push the resulting information to the cloud,” explains Tran. In this case, “because there is only one consumer, the central cloud, the pub/sub framework would be overkill.”
Another potential pitfall is getting locked into a particular vendor’s proprietary IoT platform. This can happen with data sent to the vendor’s cloud services, which can happen despite MQTT’s open-source origins. In these instances, users buy their edge device and software and connect it using MQTT.
“But you have no access to the data if it all remains within the vendor’s cloud environment,” explains Travis Cox, co-director of sales engineering at Inductive Automation.
Consequently, Cox urges users to make sure the configuration of these cloud-based systems allows them access to their data. “You can send the data to their cloud,” he says, “but ultimately you should be able to send that data to your systems too.”
A second way to get locked into proprietary technology, despite the use of MQTT, is through the payload format. This can occur because MQTT can transfer payloads in any format, including a vendor’s proprietary binary format.
“If you don’t understand what’s being sent, then it’s going to be very hard for you to take advantage of it,” Cox points out. To avoid this pitfall, insist on either having a definition that tells you what the data look like or use the open-source Sparkplug payload specification.
Cox also recommends building a resilient architecture. “If you were to lose a connection or access to your central broker, then your applications would be blind,” he says. One way that he suggests for building resiliency against such interrupted connections would be to store data in a local cache so it can be forwarded when the connection is re-established. Another way to enhance resiliency is to have two brokers, so that one can continue working if the other should fail.
Leaders relevant to this article: