Shining a light on data analytics in the energy industry

Posted On

By Shamik Mehta


Data analytics is rapidly changing how utilities operate and has the potential to redefine the current energy provisioning model completely. Utilities are no longer just providers of energy but are becoming receivers and processors of energy production and consumption data. 

Data collection and processing on a massive scale allows utilities to improve user experience, reduce waste, and predictively maintain and upgrade infrastructure to ensure optimal investment strategies.

Digitalization: The next phase of digital maturity

The utility industry is currently in the “digitization” phase of digital maturity, which involves the introduction of data collection and assimilation as a norm. The next phase is “digitalization,” where utilities use deep machine learning and AI to process datasets and locate gaps and inefficiencies that can be exploited. As Matt Schnugg, Senior Director of Data and Analytics for GE Power Digital, told Utility Dive, “data is a linchpin of our collective energy future,” and that future is looking very bright.

Here are some of the ways that utility data is being used to improve network capabilities, energy usage, and customer experience.

Networking Distributed Energy Resources (DERs)

DERs feeding into the grid have the potential to increase participation in the energy market, especially in terms of renewables like solar and wind energy. To function effectively, DERs need to be integrated into the broader grid, which requires advanced networks and analytics that can correctly measure output, load capacity, and demand. This will allow greater consumer choice in terms of purchasing renewable energy and offer DERs a transparent wholesale pricing model.

Billing and usage

By analyzing customer and billing data, utilities are coming to a greater understanding of consumer behavior. The collection of a vast number of customer data points, such as smart meter data, means that they can offer tailored packages to different types of energy users. This can improve the relationship between customer and utility,  while giving both parties the opportunity to make more informed energy decisions.

Predictive asset maintenance

Utility data is not just gathered from producers and consumers; it is also created by the grid infrastructure that’s used to transmit it. Analysis of these grid data points allows utilities to identify which assets are in most urgent need of repair or replacement. 

For example, Duke Energy collected and analyzed more than five million data points to optimize their predictive maintenance and saved $130 million on its network of more than 10,000 transmission substation transformers.

Load capacity

Cloud-based analysis of utility data can give real-time insight into how an energy network is performing and where it may be coming under strain. This is vital for ensuring energy networks can function at full capacity. It can also reduce supply disruption and outages due to risk modeling of factors such as transformer overheating, dramatic usage spikes, or even trees falling on power lines.

Grid upgrades and installation

Innovation in the energy sector is greatly improved through utility data analytics. By having a better understanding of how energy is used and where it’s most needed, grid upgrades and changes can be transparently assessed by regulatory and legislative bodies and delivered cost-effectively. 

An example of this is the EV charging infrastructure necessary to allow the nationwide adoption of electric vehicles. Current proposals, which have support from the United Auto Workers union, lawmakers, and automobile manufacturers, require significant integrated utility data to provide clear costs and requirements to relevant decision-makers.

Challenges of utility data analytics 

Unlocking the benefits of utility data analytics faces a number of challenges. The largest are:

  • Data collection: How it should be done and what data needs to be collected
  • Data security:  Keeping data secure during transmission and authentication
  • Data storage: Either in cloud-based or on-premises systems and how that affects processing
  • Data processing: Where it’s done, what queries are made, and what insights are being looked for
  • Data sharing and integration: How data from companies in different fields and competitors can share and broaden data sets
  • Data privacy: Who has access to customer data and how is it protected

Utility data analytics in action: Westnetz

An example of the challenges posed by the latter two points is Westnetz, the largest distribution service operator (DSO) in Germany. As Germany is undergoing a huge drive to improve the roll-out of EV charging infrastructure, it is incumbent on Westnetz and other collaborators to identify and assess the best locations for EV charging points, taking into account cost, load capacity, and the number of users.

To do this, data needs to be gathered from hundreds of siloed databases, including those owned by regional German municipalities and utility operators, which are in competition with one another. On top of that, there needs to be a way for possible charging point suggestions to be made and collected in one location, as well as another system to judge the viability of these locations in terms of energy infrastructure. At all times, the data also needs to be protected and governed in line with the different regulatory requirements on a state, national, and supranational level (the EU).

With all of these data challenges, the potential for delays and incomplete information is huge. When one considers that this is just one element of upgrading a grid in one country, the challenge for fully incorporating utility data analytics becomes clear. 

The Intertrust utility data analytics solution

It’s clear that for the energy sector to fully reap the benefits of deep machine learning analytics, a bespoke solution for the exchange and protection of utility data is required. To overcome the challenges of large-scale data analytics, Intertrust has developed a secure data exchange platform that, among other benefits, assists utilities in: 

  • Granular control: Secure collaboration between entities and even rivals can be achieved through adjustable access protocols.
  • Data interoperability: Data coming from varied sources, data points, and lakes can often be in different formats. For effective and timely processing, it needs to be unified and interoperable.
  • User authorization: User access is controlled through a strict authorization platform, shifting the burden of access control to a dedicated system.
  • Privacy protection: At all times, data (both PII and company information) is held in accordance with the toughest privacy regulations (such as GDPR).
  • Connectors between data sets: Siloed data sets need to be easily accessed for processing in order to deepen knowledge and improve insights.
  • Audit trails: Data requests are logged to ensure both regulatory compliance and project transparency.

Data analytics has massive potential for utilities to improve service delivery, user experience, and their investment strategies. For this to happen, however, the industry needs to overcome major challenges in relation to the collection, sharing, and processing of utility data. 

The Intertrust Platform is a proven solution for the exchange and protection of data. To find out more, get in touch with us today.


platform CTA Banner

About Shamik Mehta

Shamik Mehta is the Director of Product Marketing for Intertrust's Data Platform. Shamik has almost 25 years of experience in semiconductors, renewable energy, Industrial IoT and data management/data analytics software. Since getting an MSEE from San Jose State University, he’s held roles in chip design, pre-sales engineering and product and strategic marketing for technology products, including software solutions and platforms. He spent 6 years at SunEdison, once the world's largest renewable energy super-major, after spending 17 years in the semiconductor industry. Shamik has experience managing global product marketing, GTM activities, thought leadership content creation and sales enablement for software applications for the Smart Energy, Electrified Transportation and Manufacturing verticals. Shamik is a Silicon Valley native, having lived, studied and worked there since the early 90’s.

Related blog posts


The inherent value of DataOps

Read more


Why secure data collaboration matters

Read more


Energy, data, and the Texas freeze

Read more