Q&A: Why the EDF's new methane study is seriously flawed

By Staff | June 26, 2018

Last week, the Environmental Defense Fund (EDF) released its study on oil and gas industry methane emissions, which resulted in a Science Magazine article headlined, “Natural gas could warm the planet as much as coal in the short term.”

Seth Whitehead, team lead of Energy In Depth—a project sponsored by the Independent Petroleum Association of America (IPAA)—responded with a blog post on the EID website titled “Five things to know about new EDF methane study.” Others also criticized the methodology used by EDF, which showed U.S. methane emissions to be 60 percent above estimates from the U.S. Environmental Protection Agency (EPA) and other studies.

Whitehead breaks down the problems he has with the EDF’s latest study and why it varies greatly with previous studies on methane emissions. For example, he found that:

- Using facility-scale data leads to exaggerated emissions and goes against recommendations from the National Academy of Sciences (NAS).

- The failure to collaborate with industry is also against NAS recommendations.

- The EDF’s supplemental material includes much lower “alternative” national emissions estimates from past studies—plus some from the EDF itself—but Whitehead emphasized that this data was excluded from the report.

- The EDF study was limited to daytime facility-scale measurements that tend to exaggerate normal emissions because of episodic events, such as liquids unloading.

- The EDF study attempts to portray natural gas use as having the same effect on the climate as coal, even though the environmental benefits of natural gas are well documented.

This industry criticism prompted a response from EDF which noted, “What can be said about the climate impacts of using gas relative to coal for power generation in other parts of the world remains to be determined.” Whitehead discussed his impressions the EDF methane emissions study with North American Shale Magazine.

What’s the key takeaway from your critique of the EDF study?

The biggest issue I—and industry in general—had with this was the study is that it puts natural gas on par with coal as far as climate impacts. That characterization was completely misleading and just flat-out wrong. What was presented in the EDF press release was rather sensational. Once you get into the body of the study, it was pretty clear that EDF claimed they found that natural gas—at least in the short term over a 20-year timeframe—was on par with coal. In response to criticism of the study, EDF has backtracked on that to rewrite what they said last week, which wasn’t conveyed or reported. So they’re rewriting history a bit. Generally speaking, their entire blog post on critiquing industry’s critique is kind of a head-scratcher.

Is this a case of the EDF coming out with a sensational statement to get the headlines it wants and then later downplaying what it said, which doesn’t get as much attention?

Exactly. EDF presented this as a definitive study that was unprecedented. That’s not the case at all. There was a study conducted by the Department of Energy based on EDF’s most comprehensive work. That analysis, which I can’t emphasize enough, included all the National Academy of Science’s recommendations regarding integrating component-level measurements, site-based measurements and top-down measurements. That DOE study found a leakage rate of 1.65 percent. Notably, that is a little higher than some other research that we’ve highlighted on EID. However, it’s not high enough to warrant the media to promote it as evidence that natural gas is really bad and that it’s an inherent threat on the climate front. So EDF decided to do their own study and what you see is the result.

When you looked at the study and the supplemental materials, what were your impressions?

At the very least, it definitely deserves scrutiny and certainly raises some red flags. How they came to these conclusions is buried in the supplemental information. If you take a close look at that, I think an objective observer would look at it and say, “Wait a minute!” What you have here at least raises some suspicions that they worked backwards from a conclusion. They knew what they wanted the narrative to be in the media—that gas is as bad as coal. The public doesn’t have much knowledge about methane emissions or how they’re measured.

Did EDF collaborate with industry while conducting its study?

In EDF’s latest blog post, they characterize their study as a continuing collaboration with industry. That’s just not true at all. This was not an industry collaboration. They say right in the supplemental information that they didn’t give operators a heads-up as they were taking these measurements. If you don’t understand the site where you’re conducting methane measurements, you’re going to lack a fundamental understanding of what you’re collecting.

What do you consider the best sources of information on oil and gas industry methane emissions?

I would encourage everyone to look at the DOE study, which got zero headlines. Unfortunately, if the media doesn’t pick something up—no matter how solid a piece of work it is—then it’s either forgotten about or dismissed. I hope this is an opportunity to highlight that work, which deserves to be compared to the EDF study. There’s also a large body of work that includes many other studies. The latest EDF study is just one piece of a large body of work. It’s an outlier, not only when compared to past reputable methane studies, but also compared to EDF’s own methane studies.