Is Your Data AI-Ready? Mapping a Data Quality Strategy for Utilities

I recently co-moderated a half-day workshop at the DTECH Conference for utility professionals who are leading their organizations’ AI initiatives, and it provided some fascinating insights into the state of AI initiatives across the utility industry. Admittedly, the survey I conducted during the seminar was non-scientific, but the takeaways are instructive: more than 80% of the participants reported that the utilities they work for are still at the early stages of AI deployments (either in the planning or pilot phases). Less than 20% reported that they were at intermediate or advanced phases of AI projects, which means that four out of five were not successfully seeing momentum.

Energy utilities face increasing challenges dealing with data. AI image used courtesy of Adobe Stock

I asked participants what was holding back those AI initiatives, and their responses matched what I have heard directly from many utilities: data quality problems are a major obstacle to AI deployments, undermining forward momentum and limiting the business benefits. Other data quality issues attendees raised included:

• Inaccurate data.
• Inadequate precision.
• Incomplete data and lack of fidelity.
• Accessibility issues.
• Problems with data security and compliance.
• Poor tracking of data lineage/provenance.
• High latency or outdated data challenges.

AI runs on data, and data problems like these can bring AI projects to a complete halt — no doubt a factor in why so many seminar participants reported such slow momentum.

The Data Quality Wilderness

Data quality issues can be like being stuck in the wilderness because they take organizations off the path to measurable business benefits, leaving them seemingly stranded. One moment, an organization is excited about AI and charting an optimistic path forward. But then these data issues arise, and the complexity of resolving them feels like being stuck in the wild with no clear markers for how to get back on track.

Ensuring data quality requires a detailed data strategy. AI image used courtesy of Adobe Stock

How do you get out of this wilderness? Utilities need a data strategy map with three key features:

1. A definition of what data quality is and what it means for data to be AI-ready.
2. A process for solving urgent data quality issues that stand in the way of AI today.
3. A game plan for improving data quality, accessibility, and precision over time.

Mapping a Clear Vision for Data Quality

The first step, defining data quality, is critical because there is often no existing consensus in organizations, particularly when it comes to complex data like geospatial information, unstructured data sets, and data that historically resides only in OT systems. It often takes a trained eye to know whether datasets can be considered high-quality data.

For example, a digital map that includes information about infrastructure assets may seem accurate, but only the field crews know firsthand how many inaccuracies it contains. There are often many gaps between that map and the actual situation when they arrive on a work site.

Data quality issues are quite common in our industry for a variety of reasons. One is the enormous volume of data that utilities must manage due to the massive scale of infrastructure and operations. Another reason is that utilities have operated so much longer than most private enterprises, with data that goes back decades. Utility IT and OT systems are also complex, producing a wide range of datasets that originate from very disparate technological systems.

All of these factors can drive data quality issues, making it essential to establish a definition of the accuracy, precision, timeliness, accessibility, compliance, and other qualities that an organization needs. This critical step will not only ensure data is AI-ready but will pay enormous dividends for driving other operational efficiencies.

Leveraging Automation to Increase Data Quality

Having a clear vision for what defines data quality is critical. Putting that into action is just as important. But traditional remedies for data quality issues tend to be slow and costly, relying on manual review of data sets and requiring people with expertise to make manual fixes that remove errors, enrich data, and improve data flows. That approach is impractical for most utilities, whose teams are already fully focused on their primary duties.

Ironically, a cost-effective and resource-effective way to identify and address these data quality problems may be AI itself, which can be trained to do this at scale with automation to achieve remarkable results. AI models can be trained to look at data with the same eye that a utility professional and business analyst would, identifying each of the data quality challenges explained above. After reviewing the assessments, the AI model can fix common quality issues and escalate others for manual resolution. AI can act as a force multiplier that allows a small team to successfully clean and enhance massive datasets that would have been otherwise impractical.

Enhancing Data Quality Over Time

A successful data strategy shouldn’t just end there, though. Utilities should implement steps that steadily increase data quality over time rather than taking a one-time action in preparation for a specific AI project. Enhancing data on a continuous basis will not only drive more business value out of future AI insights, but it will also produce other operational efficiencies and benefits for utilities and ratepayers.

Enhancing data quality is a continuous initiative. Image used courtesy of Adobe Stock

One way to enhance data quality over time is by outfitting mobile work crews with technologies that update, correct, and enhance data as they carry out on-site installations and maintenance. For example, tablets equipped with 3-D computer vision can capture precise visual data about existing equipment that augments and enhances digital information about the utility’s infrastructure assets.

These same tools can also eliminate the “black boxes” where no data exists because data collection has never been performed. Common examples of these black boxes are:

• Substations.
• Underground junction boxes.
• Rarely-visited equipment closets.
• The last-few-meters connections at residential and commercial sites.

Another key way to enhance data quality over time is to move forward with IT/OT convergence initiatives that break down the silos between those systems. Separating those functions reduced risk for decades, but today it is often a liability that creates operational inefficiencies and obstacles for IT initiatives like AI projects. Breaking down the walls between IT/OT with the right strategy for convergence is a critical step for being AI-ready.

Today, too many utilities are stuck in the data wilderness. However, mapping out a strategy that resolves data quality challenges will put them back on the path to achieve AI objectives and gain operational benefits.