More than Monitoring: How Observability Keeps Power Flowing

Grid operators believe observability tools can prevent outages and save money, according to a report from New Relic, an observability platform designer. Further, most use artificial intelligence tools to detect, analyze, and prevent problems.

“The powerful combination of observability and AI can support organizations in creating a greater understanding of telemetry data and address the challenges associated with ever-expanding data sets,” Peter Pezaris, New Relic's Chief Strategy and Design Officer, told EEPower.

New Relic surveyed 132 technology professionals about their use of observability, artificial intelligence, technology plans, and cybersecurity. The findings were part of New Relic’s 2023 State of Observability for Energy and Utilities report, highlighting the growing role of digital tools and artificial intelligence in grid management. 

Digital grid tools. Image used courtesy of Pacific Northwest National Laboratory

Monitoring vs. Observability

Grid monitoring systems are already commonly used, but observability takes a more holistic view.  

“Modern systems' complexity exceeds performance monitoring's capabilities as the best method for tracking software performance and operations,” Pezaris explained. “Today, observability, the comprehensive real-time report of the health, performance, and status of software and IT operations, is the best approach for managing the complex world of microservices and distributed systems.”

Observability platforms allow grid operators to collect and analyze data to monitor complex power systems, plan maintenance and anticipate future problems. About half of the survey respondents said observability helps them create a technology strategy.

“Energy and utility providers must juggle reliability of service alongside their desire to modernize and increasing scrutiny from regulators and consumers demanding green energy options, cost transparency, and access to real-time energy usage data,” Pezaris said.

Observability involves all aspects of power distribution but must be individualized to an energy company’s specific grid assets and topography. Observability could apply to assessing the real-time performance of transmission lines, transformers, circuit breakers, disconnectors, smart meters, and substations. 

Grid managers can measure voltage, current, temperature, and other variables as demand or conditions change through observability. They can use this information to manage congestion, prevent bottlenecks, increase efficiency, and adjust to renewable energy variabilities. 

Analyzing the collected data allows managers to plan future decisions about maintenance, equipment replacement, or expansion. Many observability tools are enhanced with artificial intelligence for faster, easier analysis.

Resolving Power Outages 

New Relic’s report highlighted observability’s role in quickly resolving power outages. Respondents said observability helped reduce outages and mean time to resolution (MTTR). The quicker the outage could be resolved, the less money lost. Observability, specifically full-stack observability, made a difference, according to those surveyed. 

Number of power outages by year. Image used courtesy of Energy Information Administration

Full-stack observability is considered “end-to-end observability,” Pezaris stated. “It’s based on a complete view of all telemetry data,” he explained.

Components of full-stack observability include monitoring services, security, environment, and customer experiences. About 87% of respondents using full-stack observability reported improving the MTTR, compared to 76% without full-stack observability.

Using Artificial Intelligence

Most grid operators (56%) surveyed use artificial artificial intelligence (AI) and observability tools. That number is growing, with 89% saying they plan to add AI tools by 2026.

AI operations, or AIOps, can process the burgeoning data from decentralized distributed energy resources. Respondents said they used AI to detect and analyze incidents and find root causes.  

“AIOps can support energy and utility organizations by contributing to operational efficiency and enabling faster response times,” Pezaris explained. “It helps teams gather actionable insights and supports decision-makers to uncover learnings regarding system performance, improving incident detection and resolution.”

Cybersecurity

Cyberattack threats are increasing as the grid becomes more complex and decentralized. Observability platforms with AI can detect and stop cyberattacks before they cause widespread harm. Not surprisingly, New Relic’s report found the most common observability use was for security. 

Around 68% of the energy professionals surveyed said their organizations use security monitoring tools. Nearly all (99%) said they would adopt security monitoring by 2026. Security and risk compliance were named as the top trend by 44% of respondents.

“Energy and utilities providers are especially prominent targets for cyberattacks, state-sponsored or otherwise, underscoring the importance of performant network, security, and infrastructure monitoring,” Pezaris stated.

Since the grid is a cyber-physical system, it requires observability tools for physical assets, communication networks, and digital processing. These tools detect intrusions and anomalies and prevent or lessen the impact of cyberattacks.

The grid’s cyber-physical structure. Image used courtesy of the authors

New Relic’s report indicates that most energy and utilities professionals use six tools on average per year. The number of those using a single tool, combining many capabilities into one platform, grew from 2% to 3% in the past year.

A study published in Energy Reports journal identified some observability tools commonly used. They were advanced metering infrastructure (AMI), supervisory control and data acquisition (SCADA), advanced distribution management systems (ADMS), energy management systems (EMI), and wide area measurement protection and control (WAMPAC). In commercially available observability platforms, specific functions may be combined and overlap.

The study also notes that AI and machine learning can create grid simulations to allow operators to test possible scenarios and strengthen cybersecurity strategies.

Future of Observability

Energy and utility companies reported substantial savings by investing in observability tools, with 66% estimating their observability investment value at $1 million or more, or about three times the median annual return on investment.

Power grid operations will become increasingly complex as more renewable and distributed energy resources are added. Digital tools like observability platforms and AI will be crucial in meeting energy demand and securing power sources.

“As pillars of critical infrastructure in communities, energy and utility providers rely on real-time insights into systems and services spread across physical space,” Pezaris explained.

Observability can help grid operators detect and resolve issues quickly and efficiently. Observability is “every engineer’s single source of truth as they troubleshoot, debug, and optimize performance,” Pezaris stated.