The State of Big Data Collection and Analysis

Read the transcript below:

Welcome to Automation World’s Technology Matters. I’m David Greenfield, Director of Content, and today I’m going to share some insights we gathered recently from research into industry’s use of Big Data collection and analytics technologies.

Though the name Big Data sounds really trendy, it’s really just the name that’s new. Companies have been collecting time series data from assets for decades now. And while plant managers and maintenance workers have, of course, been using this data for operations improvements, they were rarely, if ever, analyzed with wide business transformation in mind. And that’s what Big Data is really all about—capturing increasing amounts of data by deploying more sensors and other data gathering technologies and—most importantly—analyzing the data for specific business improvement insights…not just capturing it to let it sit there for some possible use to be identified later.

So for this research project, in which we surveyed both end users and system integrators, we focused on the use of a variety of data collection and analysis technologies, ranging from data acquisition systems, historians, and computerized maintenance management systems to edge and cloud computing, and advanced analytics software.

So here’s a little sampling of what we found. While most end user respondents—86 percent—indicate that they collect data from equipment and devices specifically for production improvement initiatives, most only started doing this within the past five years. Only 27 percent of respondents indicate that they’ve been collecting data for such purposes for more than six years.

An interesting point among the end user responses is that 98 percent plan to gather even more data from their production systems in the next two years. But only 30 percent plan to do so for specific operational improvements.

Now, this could indicate that, of those who have been collecting and analyzing data for a few years, many may have already discovered numerous ways of improving their production operations and could be looking to leverage the data they collect now for other, more strategic business purposes.

Another interesting finding in the survey was that end users and integrators agreed on the the top three business drivers for data collection and analysis. Those drivers are: the improvement of specific line or equipment operations, improving maintenance operations, and being part of the company’s overall Industry 4.0 or digital transformation initiative.

With regard to specific technologies used for Big Data collection and analysis, we found it interesting, though not too surprising, that most manufacturers still rely on data collection and analysis technologies that existed long before the development of the technologies receiving most of the Big Data attention today.

That’s not to say newer collection and analysis technologies are going unused—that’s definitely not the case. Edge and cloud technologies, for example, are being used widely across industry. Even so, there remains a lot of headroom left for their growth. Only 29 percent of system integrator clients use hybrid cloud and edge technology and just 14 percent use stand-alone cloud systems.

According to the survey results, most manufacturers rely on three principal methods of data collection and analysis—and one of those is handwritten data collection. System integrator respondents say that 57% of their clients still rely on handwritten data collection that’s then entered into spreadsheet software. But only 29% of end users say they use handwritten methods of data collection. Even if you split the difference between integrator and end user responses here, that’s still a lot of companies relying on handwritten data collection and that really can’t support a true Big Data initiative.

The other two principal methods used are historians and computerized maintenance management systems—both of which have long histories of use across industry.

The results of this study will be featured in our October 2022 issue, so keep an eye out for the full report to be published online and in print.

So I hope you enjoyed this Technology Matters episode. Keeping watching this space for regular updates on advances in and applications of industrial automation technology.