Data has been described as the new oil, and there is no doubt that it is an essential ingredient if AI is to work. Clean and properly organised data is required for AI development and implementation, and clear guidelines in terms of sharing data, interoperability and standardisation are crucial. According to analyst firm Cognilytica, over 80% of the time it takes to implement an AI project is spent on data preparation and labelling for use in machine learning projects.

The widespread rollout of smart meters and advanced sensor technology have created a tsunami of data which calls for sophisticated storage and analysis tools. With a sampling rate of four times per hour, 1 million smart meters installed in the smart grid would result in more than 35 billion records (Sagiroglu et al., 2016). There will be a 530% increase in global data volume from 33 zettabytes in 2018 to 175 zettabytes in 2025, according to an EU fact sheet: Artificial Intelligence: Threats and Opportunities.

There is lack of near real-time data in standardised formats across Europe, but efforts are underway to address that problem. Representatives from several countries are involved in the GAIA-X initiative which seeks to create a new data infrastructure for Europe where data can be shared in an environment of trust and digital sovereignty is maintained. Further, Belgium’s grid operator Elia has launched an energy data exchange start-up re.alto which will provide easy access to data for users and providers using standardised energy APIs. Transmission grid operator association ENTSO-E hosts a transparency platform that should help facilitate TSO/DSO communication.

Data sharing is promoted by the Clean Energy Package, introducing a set of rules around data management, data protection and cyber security, including the need for transparency and non-discriminatory access to data, as well as data interoperability. The data strategy released in February Shaping Europe’s Digital Future set out the Commission’s views on data, saying that the way it is collected and used must place the interests of the individual first and fully comply with the EU’s strict data protection rules. At the same time, it notes that the increasing volume of non-personal industrial data and public data in Europe, combined with technological change in how the data is stored and processed, will constitute a potential source of growth and innovation that should be tapped.

Trade association DIGITALEUROPE urged the Commission in a recent call to action to foster a partnership culture to encourage sharing of data between the public and private sectors, and to have assurance that joining a data partnership will not contravene antitrust legislation. It also recommends that the European Single Market should remain connected with the rest of the world and that EU efforts should be based on international standards. Europe’s strict privacy laws under GDPR are already causing headaches for multinational platforms, for example Facebook which has recently been told by Ireland's privacy watchdog, which regulates it in the EU, that it will have to stop transferring its European users' data to the US.

Open data agreements should be explored as governance mechanisms with the principle “as open as needed, as closed as necessary". The UK government recently published a response to its Smart Data Review, which outlines plans for legislation to mandate industry involvement in Smart Data initiatives across multiple industries including energy. Building on this the Energy Data Taskforce has laid out a series of recommendations, such as the presumption of data being open. This is now being picked up by the regulator Ofgem to ensure that the energy sector is opening more and more data. It is essential that data must also be handled in a way that is lawful, secure, fair, ethical, sustainable and accountable. In most European member states the transmission system operators, which are the gatekeepers to a large share of potentially very useful data, are regulated monopolies. This presents a particular challenge to find the right level of regulatory oversight when the data needs to flow between the regulated and commercial sectors.

Utilities should monitor the degree to which customer privacy is respected. In the energy sector, legitimate interest/performance of the contract should be used as the main rule. But a rule is embedded in the GDPR legislation that says automated processes must be explainable, so the explainability of the outcomes of the AI systems and the data used will be looked at, for example why they are turned down for a loan.

The forthcoming E-privacy Regulation also poses challenges for AI in the energy sector, as in the current draft the privacy statement is extended to non-personal data. And while not focussed directly on data or AI, the Digital Services Act expected to be revealed on 2 December 2020 will have implications for the way digital services are allowed to operate that may require a review of AI practices. It will aim to create a framework spanning a wide range of services that will govern liability, competition, employment and advertising using digital tools.

A revised European coordinated plan on AI and draft regulatory framework will be published in the first quarter of 2021.