Securing The U.S. Power Grid From Cyber Attack

In addition to ensuring continuing manufacturing and production, the electric power grid performs a key role in helping developed countries maintain the standard of living its citizens enjoy. The U.S. terrorist events of Sept. 11, 2001 followed by the massive Northeast United States blackout in 2003 raised the U.S. government’s awareness of the critical nature—and the vulnerability—of the grid. Eventually responsibility for the grid was vested upon the North American Electric Reliability Corp. (NERC). Its stated mission is to ensure the reliability of the North American bulk power system. The series of rules and recommendations NERC is creating affects every power generating plant in the U.S., and is influencing industrial network security initiatives in all industries. But its decisions are not without controversy.

NERC is the electric reliability organization (ERO) certified by the Federal Energy Regulatory Commission (FERC) to establish and enforce reliability standards for the bulk-power system. NERC develops and enforces reliability standards; assesses adequacy annually via a 10-year forecast, and summer and winter forecasts; monitors the bulk power system; and educates, trains and certifies industry personnel.

One problem cited by many is that this organization is composed largely of the companies that it regulates. This leads to charges that it will incorporate change slowly or try to define regulations in such a way that they don’t apply to the companies. Other charges are that those in charge do not realized the critical role played by control assets such as distributed control systems (DCSs), programmable logic controllers (PLCs), remote terminal units (RTUs), control networks and other devices.

NERC is charged with developing standards for compliance. Over the past eight years or so, it has developed a set of standards relative to this discussion dubbed Critical Infrastructure Protection or CIP. The set of standards are often referred to as NERC/CIP. The crux of the work thus far has been to define what is a “critical infrastructure.”

One problem with a standard is whether adherence to it will solve the problem—or create others. Rick Kaun, head of Matrikon’s Industrial Security and Compliance group now part of Honeywell Process Solutions, the Phoenix-based automation systems supplier, nails the situation: “Will everyone focus on compliance and lose focus that this is supposed to give better security? Many feel it’s less secure to be compliant. But according to the standard now, if you’re doing something more stringent, then you have to continue. So, if you slip internally but still are doing more than the standard, you’d be out of compliance.”

A timeline of the development of the standard is provided in the accompanying sidebar. Following the events of 9/11, heightened awareness of the possibility of terrorist attack on U.S. critical infrastructure led to NERC issuing Urgent Advisory 1200. This was followed by UA 1300 and then CIP standards 002 through 009.

Andrew Ginter, chief security officer for security firm Abterra in Calgary, Canada (he also writes the Findings from the Field blog at www.findingsfromthefield.com), provides insight into development of the standards. “NERC version 1 was approved and immediately panned due to perceived weaknesses in wording. A NERC manager wrote a memo saying that large numbers of entities reported they had almost no critical assets, and that can’t be right. Subsequent big changes have had to do with firming language, especially over just what critical assets are. They now have a bright-line rule that sets firm criteria for saying your asset is in or out of coverage by the standard. But criticism continues in a different direction. Instead of allowing companies to define their critical assets, they have set a bar that many say is too high, such that it excludes too many assets. Implementation is being delayed while FERC evaluates it.”

Who is affected?

Kaun, writing on his “Insecurity” blog (insecurity.matrikon.com), says, “If you have been following the events around CIP versions 1, 2, 3 and the proposed version 4, you are certainly aware that v4 is an attempt to remove the subjective component of CIP 002 and replace it with ‘bright-line criteria,’ which is intended to clarify which components of the grid need to be considered high impact. To date, NERC has submitted v4 to FERC and the general consensus has been that FERC was going to approve the version in Q1 or Q2 of this year and then the fun would begin as newly identified facilities would need to come up to speed on complying with NERC CIP 003-009.

“Well, I have recently discovered a potential diversion from that plan. In essence, it appears that FERC is not going to just rubber-stamp the NERC offering. Rather, FERC is asking NERC for a number of clarifications. Some involve wording/interpretation clarifications and others include base statistics—numbers that spell out how much of the grid would be affected by the proposed wording and therefore how much would *not* be affected or would still potentially be out of scope. The final, and perhaps most interesting, type of clarification FERC is looking for is for NERC to provide updated timelines and background data. From the outside looking in, I see a real possibility here that FERC is going to expect more components of the grid in a more concise (and perhaps shorter) timeline.”

In an April 7, 2009 memo, then NERC director of security Michael Assante addressed concerns about owner/operators self-identifying critical assets- —indeed, calling into question the very basis of the potential problem.

Assante noted that the bulk power system is designed to withstand the most severe single contingency, and in some cases multiple contingencies, without incurring significant loss of customer load or system stability. But in the new world of cyber security concerns, the game changes.

“System planners and operators will need to consider the potential for the simultaneous manipulation of all devices in the substation or, worse yet, across multiple substations,” Assante wrote. “I have intentionally used the word ‘manipulate’ here, as it is very important to consider the misuse, not just loss or denial, of a cyber asset and the resulting consequences, to accurately identify CAs under this new ‘cyber security’ paradigm.”

Assante concluded, “One of the more significant elements of a cyber threat, contributing to the uniqueness of cyber risk, is the cross-cutting and horizontal nature of networked technology that provides the means for an intelligent cyber attacker to impact multiple assets at once, and from a distance. The majority of reliability risks that challenge the bulk power system today result in probabilistic failures that can be studied and accounted for in planning and operating assumptions. For cyber security, we must recognize the potential for simultaneous loss of assets and common modal failure in scale in identifying what needs to be protected. That is why protection planning requires additional, new thinking on top of sound operating and planning analysis.”

Markus Braendle, group head of cyber security at automation and power solutions supplier ABB Group in Zurich, adds, “CIP specifies what you should consider as a critical asset. At first it was left to the utilities to determine, but now they are becoming more specific. But this is still not in the context of an actual system. CIPs should be based upon sound risk assessment and risk management, not just on a list from NERC. Further, the CIPs still talk about a single electronic perimeter, but today’s best practice is defense in depth.”

Not reaching the control level

Perhaps the most vocal critic of NERC is security specialist and Managing Partner of Applied Control Solutions, Joe Weiss, who also writes the “Unfettered” blog (www.controlglobal.com/unfettered). One of his big concerns is that the standards do not reach to the control level enough. They are too limited. “Stuxnet would be excluded from consideration and protection because it directly attacked a PLC with a non-routable protocol. Components that fall under the standard are defined as those that have an IP (Internet protocol) routable protocol. Stuxnet was spread through thumb drives. In addition, modems are still in use in some utilities. These are cyber-hackable, but are not covered because these also are non-routable protocols.”

A more troubling observation from Weiss is his view that not only do the CIPs not go far enough in protecting the entire system, but they also increase vulnerability. “Electrons don’t have org charts—neither do hackers,” says Weiss, who goes on to ask and answer, “Is NERC/CIP a roadmap for attacking the electric grid? It absolutely positively is. Number 1, it’s public. They have explicitly told everybody what is covered and what is not, and even a timetable for when to do something. So not only do they say what needs to be a protected, but also when it’ll be done. Thatis a roadmap for hackers.”

Weiss cites examples of incidents in addition to Stuxnet such as the Sept. 9, 2010 San Bruno, Calif. natural gas line explosion, an Aurora diesel generator cyber attack and the June 10, 1999 Bellingham, Wash. pipeline incident. “None of these incidents were viruses, worms or other IT-related problems. They were all control system stuff. And that’s what needs to be protected,” he says.

Do suppliers care?

Do process systems suppliers care about this and the threats their customers face? Gary Woodward, director of business development and product marketing for the Power & Water Solutions division of Emerson Process Management in Pittsburgh, notes, “Ninety percent of our revenues come from generating plants worldwide. We work very closely with this user group. When they see things coming at them, we start studying. We actually have products and services and even had to change business practices—we assure that our technicians have training and awareness of security at customer’s sites because we know so much about the systems.” Woodward relates that Emerson is responding by adding more anti-virus protection, continually hardening the system, automated patch management and more.

Roger Pan, Ovation security program manager at Emerson Process Management, says that he has noticed even those utilities and power generation customers who had procrastinated about implementing NERC/CIP or other cyber security measures have been awakened to the problem by the Stuxnet event. “They are putting things in place because they know it’s good business.”

As to the objection that NERC is more concerned with IT than with control systems, Pan says, “Control systems today really are IT systems. Maybe years ago they weren’t digital, but today’s are different. The situation is more of a mindset of what to do. An instrumentation and control (I&C) engineer is responsible for the plant system and knows you can’t just reboot and hope it’ll work. What we see as a battleground is that IT realized early the need for NERC/CIP and now I&C engineers do, also. Often I get involved with customers and gather all the groups together.”

One problem the Emerson people still see is that too many people are still running Microsoft Windows NT. “The new regulations are requiring everyone to stay more current,” adds Pan. “We do a good job of going back to people and updating every three to five years.”

Lots of record-keeping

As Pan notes, part of the new requirements from NERC require a lot of record keeping and analysis of all the related assets. Nick Cappi, director of integrity solutions for PAS, an automation supplier in Houston, says, “Companies are required to document everything from physical security to application security— even to placing an appropriate banner on a device itself so that people on it know the intent of the device. It’s forcing vendors to give an approved list of application levels, anti-virus, third party applications and the like. Then they need a physical inventory of all covered devices including PLCs, PCs, DCSs and the like. Power generation companies will have to provide a very complete inventory of hardware and software installed in a facility.”

Compiling and maintaining this information will be a huge task. Cappi has heard estimates of six people for two weeks per site to gather the information. But PAS has developed tools in use for years that help maintain all this information. “The goal is to assure people have control over their systems—both internal and external—to assure they are not being invaded,” Cappi continues. And to recognize that threats are not just external. PAS Vice President of Global Business Development Mark Carrigan adds, “From everything I’ve seen, the risk is actually greater internally—through errors, missed updates and such—than from an external source.”

While the controversy continues about whether the requirements are stringent enough, company executives have awakened to the new realities of safely doing business in the Internet age. They are finally taking cyber security seriously. 

SIDEBAR: 7 Ways to Secure a Facility, click here.

SIDEBAR: NERC Standard Development timeline, click here.