A lot of smart people spend a lot of time trying to predict how much oil and gas is going to come out of the ground in the future.
Lately, they've been getting it wrong.
"Unpredictability, measured as the frequency of extreme errors in ... projections, has increased in the most recent decade," according to an unusual new study by a team at Carnegie Mellon University that found analysts are getting worse at predicting both how much oil and gas will be produced and how much Americans will need.
That's not good news, according to Ines Azevedo, an engineer at Carnegie Mellon who supervised the study. Oil and gas alone contribute about $250 billion per year to the economy, and fuel-dependent transportation of people and goods add almost a trillion dollars more.
Azevedo's team examined the last six decades of data on energy supply, demand and prices, as well as several decades of projections by the U.S. Energy Information Administration. Part of the federal government, the EIA makes projections each year that lay out how much oil, gas and coal will be produced in the near future, as well as what consumers are likely to demand.
These projections are the gold standard for business and power companies, and anyone else who cares about energy economics. Businesses, state governments and utilities depend on the projections to plan how much fuel to buy, what to charge consumers for electricity and fuel and what kind of new power plants to build.
How much consumers pay at the pump or to your power company depends on how well energy producers plan.
Steven Davis, an energy expert at the University of California, Irvine who was not involved in the study, says predictions have gotten worse for two main reasons. "One was this revolution in how we were able to extract gas and oil through fracking and horizontal drilling, which really shook up the energy sector," he says. Suddenly, oil and gas that was once thought unreachable could be extracted using this new technology. Supplies shot up, and prices dropped.
Economic volatility is also to blame. "The recession that struck around 2007 and 2008," he adds, "really started tanking economies around the world."
The new study shows how prices, production and consumption sometimes exceeded expectations, and sometimes fell below what had been predicted.
Davis says the uncertainty has real consequences beyond making hedge fund managers nervous. It leads businesses to be more conservative.
"You're less inclined to take on big, irreversible investments on things like a gigantic power plant, and instead you may sort of hedge by holding tight with what you have," Davis explains. If "what you have" is an old, polluting power plant, that could be bad for the environment.
One part of the energy business left out of the analysis, published in the journal Nature Energy, is the market for electricity from solar energy and wind power. But past analyses have found the unpredictability holds true there as well; prices have been significantly lower and growth higher than what was projected a decade ago.
Azevedo acknowledges this may not be a permanent situation. Volatility and unpredictability were equally extreme during the 1950s, for example, but in the decades leading up to the early 2000s things were a lot less volatile and more predictable.
But she notes that the energy world looks very different now. It's truly an international business, with oil and gas coming from scores of countries and new technologies changing the energy landscape constantly. Meanwhile, numerous economies that used to be small players, especially China and other Asian countries, now have huge influence over world supply and demand.
Azevedo says the current volatility is a "stark reminder" that "surprises" may be the new norm.