24th May 2017: 'Whitehall's 'Fracking' Science Failure' » 2017 » Mobbsey's Musings » Paul Mobbs/MEI » FRAW

What is that strange man trying to hide?

Whitehall's 'Fracking' Science Failure

How the Government has misled Parliament and the public on the climate change impacts of shale oil and gas development in Britain

Paul Mobbs, Mobbsey's Musings, 24th May 2017

Written for The Ecologist, published 24th May 2017

To download a copy of the Talk Fracking report, Whitehall's 'Fracking' Science Failure, click here

As the Conservative Manifesto portends a planning 'free for all'[1] for shale gas, Talk Fracking[2] launches its new report[3] demonstrating the flaws in the Government's case on fracking and climate change.

The Government's case – detailed in the Mackay-Stone report[4] – has been widely criticized in the past.

Research published over the last 18 months, outlined in Talk Fracking's new report, questions the accuracy of the data used in the Mackay-Stone report. As a result of this new information Whitehall's scientific case has arguably collapsed.

The issue is about science and uncertainty

What is 'science'?

It is a process for how we find, measure and then evaluate the real-world in order to identify 'how it works'.

The problem is, particularly for contentious debates in the media and politics, that we seldom hear the 'confidence' we might have in those results, or their 'uncertainties'. Rarely is the method of how those results were produced ever discussed.

In such an environment it is easy to use 'results' in a way that has no validity to the context in which they were formed.

When we hear the fracking industry and academics argue over leakage figures, we might presume the issue is whether or not one or other set of figures are correct. In fact, the issue here is the method used to make those measurements, and whether that system of measurement produces a 'realistic' result or not.

"Fracking" and Whitehall's energy policy

The Government in Whitehall (distinct from those in Edinburgh or Cardiff, who currently moratoriums on development) has promoted fracking as a means to meet climate change obligations.

As Energy Secretary Ed Davey claimed in 2013[5], shale gas is a "bridge" to[6] a low carbon economy.

That claim rests on the results of one report, written by the Department of Energy and Climate Change's (DECC) Chief Scientist, David Mackay, and the economist, Timothy Stone.

The Mackay-Stone report, published in September 2013, states that,

We have gathered available information on the carbon footprint of shale gas to inform our estimate of the potential impacts of shale gas exploration, extraction and use in the UK on UK climate change objectives… With the right safeguards in place, the net effect on UK GHG emissions from shale gas production in the UK will be relatively small.

The point at issue today is whether that process of evaluation was valid, even when the report was first published in 2013.

'Bottom-up' versus 'Top-down'

How we measure and evaluate the pollution emitted by industrial processes is a compromise between what is technically possible and realistically practicable. Reliably measuring gases emitted from equipment outdoors is difficult, so it require some flexibility.

These historic difficulties mean that regulators have relied on a 'bottom-up' or 'inventory' method to assess the leaks from oil and gas operations.

Small parts of the equipment are tested, either in a laboratory or specially constructed test rigs. The leaks are measured or estimated. Finally the figures are combined in an 'inventory' of the system being monitored to produce a total.

When the climate impacts of oil and gas production were first assessed in the 1990s the assumption was that the effects of leakage were "insignificant"[7].

What has happened since is that the monitoring technology has improved. Today it is possible to equip aircraft or ground vehicles as mobile gas laboratories[8]. These are flown or driven around oil and gas fields to sniff the air.

From that sampling it is possible to produce a 'top-down' estimate of how much gas is leaking in order to create the measured concentrations in the air.

In an ideal world the top-down and bottom-up measurements would, within a reasonable boundary of uncertainty, match.

The difficulty is that they do not.

What consistent studies carried out over the last decade or so have found is that real-world, 'top-down' monitoring exceed the estimated 'bottom-up' measurement of emissions by at least two to four times[9].

It's this mismatch over measuring that is at the heart of the fracking and climate debate.

Howarth and the significance of methane

The research paper which highlighted the significance of this debate over measurement methods was produced by Howarth, Santoro and Ingraffea in June 2011.

The 'Howarth' paper[10] gained prominence because it claimed to show that shale gas was not only worse than conventional gas. Under certain circumstances it could be worse than coal-fired power generation.

The reason why the paper claimed such high climate impacts was due to two main factors:

Firstly, because it was using 'top-down' assessments of leakage from natural gas systems. As noted above, these have consistently produced much higher levels of leakage than 'bottom-up' data.

Secondly, it used a 20- rather than 100-year baseline for the impact of methane on climate change.

The 20-year issue is important as methane has gained prominence as a greenhouse gas. Again, new sampling techniques have been finding far higher concentrations[11] in the environment than were expected.

As we approach climatic tipping points[12], the impact of fast-warming methane is becoming more significant to how we respond to climate change.

The Mackay-Stone review

In Britain, DECC commissioned Mackay and Stone to evaluate the climate impacts of shale gas – although if you read the report, it is clear that it is targeted squarely at the results of the Howarth study.

Very roughly, Mackay and Stone:

That is a fair assessment procedure in order to test the impacts of shale gas against other sources of natural gas for power generation.

The problem with Mackay and Stone's report is not the process, it is the data which they used in their calculations:

Using a figure for leakage which was perhaps a half of what it should have been, and a figure for gas production which was twice what it should have been, the level of impacts which their analysis found is arguably a quarter of what it should be.

Mackay and Stone, while rejecting Howarth's figures, also disregarded other studies produced around that time which had produced similar results to Howarth. Instead they promoted an as yet unpublished study, by Allen et al., which claimed that leakage rates could be minimized using what was called 'reduced emissions completions' (REC).

The Allen study

The 2013 study by Allen et al.[15] was part-funded by the campaign group, the Environmental Defense Fund. It is a 'bottom-up' analysis of leakage from oil and gas operations, and claimed levels of leakage far lower than similar studies.

However, the study ran into problems from the start:

The publishing journal, PNAS, had to issue a correction because the authors had failed to declare their conflicting industry affiliations.

More significantly, the study does not disclose which, and what type of sites were being tested, so it was difficult to relate the results to the industry as a whole.

Most seriously though, the sites were not randomly selected for testing – as acknowledged in the supporting information[16] published alongside the paper. Their industry partners selected which sites they were to test, and so there's no evidence the sample of sites measured were 'representative' of the industry as a whole.

The real problems for the Allen study emerged in 2015.

Research by Howard et al.[17] highlighted that one of the most widely used sensors to measure methane concentrations – which had been used in the Allen study – routinely malfunctioned, under-reporting methane concentrations.

Then the US Argonne National Laboratory[18], which co-ordinates the reporting of US carbon emissions, noted that the sensor might be under-reporting methane levels by three to five times.

Finally, in 2016, the Environmental Defense Fund[19], who had part-funded the Allen study, rejected its results.

Misleading Parliament and the public

From the date of its publication the Mackay-Stone report has been flawed, due to the approach taken to calculate the impacts of shale gas, and in particular due to the selection of data.

DECC and its authors defended this by referencing the Allen study as proof that emissions could be reduced to levels where the impacts would be 'small'. Now that the Allen study has been shown to be flawed, the Mackay-Stone report has been definitively invalidated too.

However, that has not stopped ministers and Parliamentarians quoting it to support the Government's policies on oil and gas extraction.

DECC itself was disbanded in 2016, but in January 2017 the new department – the Department of Business, Energy and Industrial Strategy (BEIS) – issued revised guidance[20] on shale gas. Once again it echoed the results of the Mackay-Stone report.

Distorting the evidence

In March 2016, the Committee on Climate Change (CCC) produced a report on onshore oil and gas production[21]. BEIS did not release it for four months, until Parliament had almost finished for the Summer.

When Environment Secretary Andrea Leadsom announced the report to Parliament[22] she claimed that the CCC said that onshore oil and gas was compatible with the UK's climate targets. This was misleading, as this is not within the context of the CCC's conclusions.

As stated in the recent House of Commons Library briefing on Shale Gas and Fracking[23], the CCC concluded that 'fracking' must pass three tests to be acceptable. The third of those requires that we reduce emissions elsewhere in the economy to accommodate the emissions from onshore oil and gas.

That could be extremely difficult – and might not be possible.

As Climate Change Secretary Nick Hurd stated in evidence[24] to a Select Committee in January 2017, even finding the 50% of savings yet to be identified to meet the UK's climate targets will be, "hard".

Whitehall's fracking policies are completely flawed

The Mackay-Stone report was arguably flawed on the day of its publication.

Today it is wholly discredited.

No minister can quote its conclusions with any certainty without demonstrably misleading MPs and the public as to the current state of the science.

In fact, like the Mackay-Stone report, large parts of the two other reports which the Government rely upon to justify fracking – the Royal Society report[25] from 2012, and the Public Health England report[26] from 2014 – can be similarly invalidated if we look at the weight of evidence now available.

The Mackay-Stone report must be withdrawn, and a moratorium implemented on all 'fracking' operations until we can state their impacts with certainty.

At the same time Whitehall and government ministers must admit to the mistakes in their previous claims, and commit to an open and transparent review of the evidence now available.


  1. Independent On-line: 'Election 2017 – Conservatives back fracking 'revolution' in the party manifesto', 18th May 2017 – http://www.independent.co.uk/news/uk/politics/election-2017-conservatives-fracking-party-manifesto-tory-gas-shale-domestic-enivronment-a7742496.html
  2. Talk Frackinghttp://www.talkfracking.org/
  3. Talk Fracking: 'Whitehall's 'Fracking' Science Failure How the Government has misled Parliament and the public on the climate change impacts of shale oil and gas development in Britain – A report for Talk Fracking', Paul Mobbs, February 2017 – http://bit.ly/2rDa8lw
  4. DECC: 'Potential Greenhouse Gas Emissions Associated with Shale Gas Extraction and Use', David Mackay and Timothy Stone, September 2013 – https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/237330/MacKay_Stone_shale_study_report_09092013.pdf
  5. DECC: 'Edward Davey today made the case for the safe and responsible exploration of shale gas in the UK, in line with UK's climate change targets', 9th September 2013 – https://www.gov.uk/government/news/davey-uk-shale-gas-development-will-not-be-at-expense-of-climate-change-targets
  6. Energy Science and Engineering: 'A bridge to nowhere: methane emissions and the greenhouse gas footprint of natural gas', Robert W. Howarth, vol.2 no.2 pp.47-60, June 2014 – http://onlinelibrary.wiley.com/doi/10.1002/ese3.35/pdf
  7. New York State Department of Environmental Conservation: Section, 'Well Permit Issuance for Horizontal Drilling And High-Volume Hydraulic Fracturing to Develop the Marcellus Shale and Other Low-Permeability Gas Reservoir', September 2009 – ftp://ftp.dec.state.ny.us/dmn/download/OGdSGEISFull.pdf
  8. National Oceanic and Atmospheric Administration: 'Shale Oil and Natural Gas Nexus (SONGNEX) – Studying the Atmospheric Effects of Changing Energy Use in the U.S. at the Nexus of Air Quality and Climate Change', Earth System Research Laboratory, U.S. Department of Commerce, April 2014 – http://www.esrl.noaa.gov/csd/projects/songnex/whitepaper.pdf
  9. Nature: 'Air sampling reveals high emissions from gas field', Jeff Tollefson, vol.482 pp.139-140, 9th February 2012 – http://www.fraw.org.uk/library/extreme/tollefson_2012.pdf
  10. Climatic Change : 'Methane and the greenhouse-gas footprint of natural gas from shale formations', Howarth et al., vol.106 no.4 pp.679-690, June 2011 – http://link.springer.com/content/pdf/10.1007/s10584-011-0061-5.pdf
  11. Geophysical Research Letters: 'A large increase in U.S. methane emissions over the past decade inferred from satellite data and surface observations', Turner et al., Geophysical Research Letters, vol.43, 16th March 2016 – http://onlinelibrary.wiley.com/doi/10.1002/2016GL067987/pdf
  12. Environmental Research Letters : 'Climate forcing growth rates: doubling down on our Faustian bargain', Hansen et al., vol.8 no.1 id.011006, March 2013 – http://iopscience.iop.org/1748-9326/8/1/011006/pdf/1748-9326_8_1_011006.pdf
  13. Cuadrilla Resourceshttps://cuadrillaresources.com/
  14. Paul Mobbs/MEI: '"Behind every picture lies a story" – statistical reality versus PR-hype within the political project of unconventional gas in Britain', 25th July 2013 – http://www.fraw.org.uk/mei/musings/2013/20130725-behind_every_picture_lies_a_story.html
  15. PNAS: 'Measurements of methane emissions at natural gas production sites in the United States', Allen et al., vol.110 no.44 pp.17768-17773, 16th September 2013 – http://www.pnas.org/content/110/44/17768.full.pdf?with-ds=yes
  16. PNAS: 'Measurements of methane emissions at natural gas production sites in the United States – Supporting Information', September 2013 – http://www.pnas.org/content/suppl/2013/09/11/1304880110.DCSupplemental/sapp.pdf
  17. Energy Science and Engineering: 'University of Texas study underestimates national methane emissions at natural gas production sites due to instrument sensor failure', Touché Howard, vol.3 no.5 p.443-455, 4th August 2015 – http://onlinelibrary.wiley.com/doi/10.1002/ese3.81/pdf
  18. Argonne National Laboratory: 'Updated Fugitive Greenhouse Gas Emissions for Natural Gas Pathways in the GREET1_2015 Model', Burnham et al., October 2015 – https://greet.es.anl.gov/files/fugitive-ch4-2015
  19. Environmental Defense Fund: 'Methane Research: The 16 Study Series an unprecedented look at methane from the natural gas system', January 2016 – http://www.edf.org/sites/default/files/methane_studies_fact_sheet.pdf
  20. Department for Business, Energy and Industrial Strategy (BEIS): 'Guidance on fracking: developing shale gas in the UK', 13th January 2017 – https://www.gov.uk/government/publications/about-shale-gas-and-hydraulic-fracturing-fracking/developing-shale-oil-and-gas-in-the-uk
  21. Committee on Climate Change: 'Onshore Petroleum: The compatibility of UK onshore petroleum with meeting the UK's carbon budgets', March 2016 – https://www.theccc.org.uk/wp-content/uploads/2016/07/CCC-Compatibility-of-onshore-petroleum-with-meeting-UK-carbon-budgets.pdf
  22. Hansard: 'Committee on Climate Change: UK's Carbon Budgets', The Minister of State, Department of Energy and Climate Change, Andrea Leadsom, 7th July 2016 – https://hansard.parliament.uk/Commons/2016-07-07/debates/16070737000020/CommitteeOnClimateChangeUKSCarbonBudgets?highlight=shale%20gas%20climate%20change#contribution-16070737000046
  23. House of Commons Library: 'Shale Gas and Fracking' (No.6073), Jeanne Delebarre, Elena Ares, Louise Smith, 4th January 2017 – http://researchbriefings.files.parliament.uk/documents/SN06073/SN06073.pdf
  24. Parliament Live: 'Evidence from Nick Hurd MP, Minister of State for Climate Change and Industry, Business, Energy and Industrial Strategy Committee', Tuesday 10th January 2017 – http://www.parliamentlive.tv/Event/Index/b8d4021c-1011-436e-b2e6-3705f225f7d7
  25. Royal Society/Royal Academy of Engineering: 'Shale gas extraction in the UK', June 2012 – https://royalsociety.org/~/media/policy/projects/shale-gas-extraction/2012-06-28-shale-gas.pdf
  26. Public Health England: 'Review of the potential public health impacts of exposures to chemical and radioactive pollutants as a result of shale gas extraction', June 2014 – https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/332837/PHE-CRCE-009_3-7-14.pdf