Mark is a customer and solution oriented person with extended experience and professional knowledge in power system

As cyber threats grow and become more sophisticated, utilities must constantly evaluate and improve the effectiveness of security programs. Moreover, utilities must manage their security programs across both traditional information technology (IT) and operational technology (OT) systems.

Utilities must also be able to quantify the investments and technologies needed to satisfy rate case requirements. Determining a utility's existing cybersecurity posture is a necessary first step. Lord Kelvin famously observed, "When you can measure what you are speaking about, and express it in numbers, you know something about it."

As part of the Electric Power Re-search Institute's (EPRI) research program in cybersecurity, we initiated a collaborative effort with the Edison Electric Institute, the American Public Power Association, the National Rural Electric Cooperative Association, the Utilities Telecom Council, and the SANS Institute to examine security metrics for the electric sector. In 2015 the collaboration created a security metrics methodology and a framework for creating security metrics. In 2016 the group revised the methodology and developed specific metrics for utilities to use as a starting point in evaluating their own posture and path forward.

A Practical Methodology for
Cybersecurity Metrics Development

EPRI's research approach incorporated five common-sense rules to our metrics development:

• Utility cybersecurity metrics must be based on quantitative and repeat-able data

• Metrics must be independent of compliance to mandatory standards

• Metrics must allow for tailoring across the utility's business units, functions, and ownership structures

• Metrics must take into ac-count difference between IT and OT architectures

• Metrics must be able to clearly communicate the utility's state of cybersecurity to different stakeholders

EPRI's approach, as shown in the metrics "pyramid" (Figure 1) organizes data points, then rolls them up and assigns a weight of importance to either an operational, tactical, or strategic metric. The resulting tiers of data will help a broad range of utility stakeholders gain improved knowledge about cybersecurity postures and thus inform decision-making about policies, investments, and action plans.

More than 100 data points provide the quantitative foundation for the metrics, consisting of various operational statistics collected from different points in utility operations. The availability and quality of these data are important factors in metrics calculations.

Operational metrics measure real-time, day-to-day operations such as logs, rule sets, and signatures. Tactical metrics address programmatic health and progress in the organization. Strategic metrics measure corporate risk and alignment of the metrics to the direction of the business.

A Cybersecurity Scorecard for Utilities
Each succeeding layer of metrics is based on rolling up the lower level metrics to the higher level ones. As shown in the figure, the top-level, three strategic metrics are calculated from 11 tactical metrics; and each tactical metric is calculated by summarizing relevant operational metrics. As data points shift, the impacts are reflected in metrics calculations and scorecards.

A Path Forward
As a relatively new field, security metrics is not as mature or robust as metrics in finance, reliability operations, or safety. However, EPRI's collaborative research and practical methodology offers an optimal, standardized and complementary approach utilities can use to evaluate their own postures and resulting action plans.