Feature: US energy transition, extreme weather creating power demand forecasting challenges

US short- and long-term power demand forecasting is becoming increasingly challenging as the power generation fuel mix shifts more toward weather-dependent renewables and energy storage resources, and extreme weather becomes more common, causing power grid operators to adopt new load forecasting approaches.

The increase in renewable energy penetration is making day-ahead load forecasting more dependent on accurate weather forecasting, with extreme weather creating additional challenges.

For example, temperatures across PJM Interconnection territory plummeted beginning on Dec. 23, 2022, and the cold lasted into the morning of Dec. 25, with record lows in some areas as well as record drops in some regions. PJM said it was the most drastic temperature drop in a decade, and power demand during the Christmas weekend was an "extreme outlier" in magnitude and timing.

The weather event put extreme stress on the power grid, and the rapid power demand increase resulted in extremely elevated power prices. Zonal power prices reached as high as around $4,300/MWh Dec. 24, according to PJM. The highest December PJM West Hub daily average real-time on-peak price of $1,111.90/MWh was reached Dec. 23, according to PJM data.

Regarding long-term planning, power system operators have traditionally relied on historical weather patterns to help create power demand forecasts, but with weather becoming more erratic because of climate change, historic weather dynamics are becoming less reliable indicators of future conditions.

Short-term adjustments

Asked how increased weather-dependent renewables penetration is impacting load forecasting day-ahead, an ISO New England spokesperson said in a recent email that the rise in weather dependent renewable penetration, particularly behind the meter solar production, has "introduced a significant level of variability to electricity demand, particularly during daylight hours."

This increased variability presents a challenge for day-ahead and short-term load forecasting and to address this, ISO New England has recently implemented two new load forecasting projects within the last 10 months, the grid operator said.

Artificial Intelligence is also playing a role in short-term power demand forecasting.

In the short-term forecast process, a variety of load forecast models and "model blending algorithms" are employed to inform the human forecasters in their decision-making to produce a final forecast, the spokesperson said, adding that "the models and tools we use are machine learning and AI-based algorithms," and these encompass "neural network models and gradient boosting models that employ tree-based learning algorithms."

The New York Independent System Operator is also working to address these short-term load forecasting challenges.

"In terms of the short-term, day ahead and, real time, the NYISO has for the last several years, developed essentially working with third-party contractors, a pretty robust framework for collecting actual and forecasted output for both wind and solar resources," Tim Duffy, NYISO's manager of demand forecasting and analysis, said in a recent phone interview.

The grid operator is working with contractors that facilitate the data gathering and forecasting process, and then NYISO integrates that into its day-ahead as well as real-time forecasting process, Duffy said.

Extreme weather can also be a challenge for NYISO weather forecasters when storms move through, and the grid operator relies on various services for that data. Duffy said NYISO uses and is looking to expand the use of "probabilistic forecasting."

A deterministic forecast might predict no clouds tomorrow at 3 pm, while a probabilistic forecast provides a more detailed prediction, like there is a 30% chance of cloud cover tomorrow at 3 pm, and that additional level of detail on the forecast uncertainty enables grid operators to better manage costs and risks when dispatching generation resources, according to the US Department of Energy.

Regarding the use of AI, Duffy said NYISO is using neural nets as part of its day-ahead load forecasting, whether that is considered to be AI or machine learning.

Long-term challenges

Given the evolving nature of weather patterns due to climate change, "we are actively investigating methods to incorporate the anticipated impacts of climate change into our supporting data and overall forecasting methodology," the ISO-NE spokesperson said, adding that "we anticipate implementing the selected chosen solution within the next couple of years."

The ISO is also collaborating with the Electric Power Research Institute, an independent nonprofit energy research and development organization, to conduct a probabilistic energy security study to assess energy security risks.

The major challenges in long-term load forecasting center on understanding and modeling emerging technologies such as distributed energy resources like solar PV and batteries as well as technologies involved in electrifying and decarbonizing the heating and transportation sectors, the ISO-NE spokesperson said.

The NYISO's Duffy highlighted that understanding the timing of the energy transition is critical.

"We can understand what the end state will be, whether that's 2030, 2040, 2050, 100% electric, but it's really the path to get there and the timing of the increased penetration of electric vehicles and building electrification," he said. "That really, in my mind, is the biggest challenge."