The factbombing problem: Why too many statistics can be damaging to the state of public debate

Theresa May’s prime ministership, which is soon to come to an end, will be remembered for many things—cabinet discord, endless Brexit negotiations, and unexpected outbreaks of dancing.

But here at Full Fact, one thing above all else will stick in our mind as we look back on the Theresa May era: really long lists of disconnected facts being reeled off at Prime Minister’s Questions (PMQs).

Record levels of employment; more money for our schools; the biggest cash boost in the NHS’s history. There were times when we could virtually predict her words before she’d even said them.

Mrs May, of course, was not alone in her fondness for sudden outbursts of scattershot statistical claims—she has had a more than willing partner in Jeremy Corbyn. Between the two of them, they made these chains of disconnected claims become so common and repetitive that they will be ingrained into our minds long into the future. And, we suspect, the minds of anybody else who has to watch PMQs for their job.

In fact, it became so common we even came up with a name for it.

Welcome to factbombing

Factbombing is when a politician deploys a flurry of factual claims in quick succession, with no clear overarching point (beyond “things are good” or “things are bad”). Take this example from 3 April, as Jeremy Corbyn steps up to the despatch box to ask his sixth and final question to Theresa May at PMQs. In so doing, he delivers a shocking picture of a country on its knees:

“this Government… has put half a million more children and 400,000 more pensioners into poverty, and doubled homelessness. There is nothing inevitable about rising poverty, homelessness and soaring food-bank use in the fifth richest country on earth… insecure work, low pay and rising pensioner poverty”

On a whole host of issues, the situation seems desperate. Yet, you might feel a lot chirpier all of a sudden, after hearing what Theresa May has to say in response:

“Let us just talk about what is happening under this Government: a record rate of employment; wages growing at their fastest for a decade; debt falling; a long-term plan for the NHS, and the biggest cash boost in the NHS’s history; a skills-based immigration system; more money for police, local councils and schools; the biggest upgrade in workers’ rights for over 20 years; the freeing of councils to build more homes; world-class public services; better jobs; more homes; and a stronger economy”

But frankly, you’re more likely to be downright confused. There is very little clarity to be gained from this exchange.

Are wages up or down? Are jobs getting better, or less secure? Is poverty and homelessness as serious an issue as Mr Corbyn says? Or are we looking at an increasingly robust economy that works for all?

This is death by a thousand statistics. This is factbombing.

Why it’s a problem

The problem with factbombing is not that the information is necessarily inaccurate—in fact it’s pretty much the opposite—the problem is that when too much factual information is thrown around at once, it can make the discussion almost impossible to follow.  

It might sound odd for a factchecking organisation to say, but sometimes “more facts” doesn’t actually improve the quality of the debate.

PMQs is supposed to be an opportunity for debate and scrutiny. Yet instead it’s turned into a competition to see who can list the most talking points quickly enough to clip them up into a social-media friendly video.

Exchanges between Corbyn and May are dominated by convoluted, disconnected soundbites, allowing them to paint completely different pictures of the country, without any of these points actually being challenged or examined.

Here is an example from 16 January this year, as Jeremy Corbyn asks Theresa May his sixth and final “question” at that week’s PMQs. In the course of it, he makes thirteen separate factual claims.

These thirteen claims cover almost as many different topics. He begins—understandably—with a rebuttal of Theresa May’s previous claim that he does not support the police. He then moves on to a new topic: the government’s hostile environment policy and the Windrush scandal. But this is followed by a flurry of eight claims on different topics entirely—homelessness, poverty, care for the elderly, the NHS, crime, and education.

And none of these thirteen claims actually relate to the question he finally asks Theresa May—which itself is more of a comment than a question really—about Brexit and whether she should resign.

There is no meaningful attempt here to hold the government to account on a single issue. How can the Prime Minister even begin to attempt to respond to a “question” covering so many topics?

It’s perhaps not surprising, therefore, that Theresa May often responds with a factbomb in kind. The example at the beginning of the article illustrates this, and how difficult it is for someone watching the debate to make sense of the sheer volume of figures coming their way.

From despatch box to soapbox

Taken in isolation, factbombs might seem like a fairly harmless quirk. Factbombs normally occur on the sixth and final question of the May-Corbyn exchange, and it’s perhaps not surprising that they want to go out on a rousing high, with a mini-campaign speech.

And it’s certainly valid to argue—as Ayesha Hazarika and Tom Hamilton have done in their book Punch and Judy Politics—that there has always been a highly performative tone to PMQs. Part of its purpose is for the leader to rally the troops and put a spring in their MPs’ steps for the rest of the week. Rhetorical flourishes and campaign statements certainly have their place in PMQs.

But the problem is that factbombing is creating a culture where this is becoming overwhelming. The primary purpose of PMQs remains parliamentary scrutiny of government, yet factbombing is turning the despatch box into a soapbox—especially in the Corbyn-May exchanges.

Our job as a factchecking organisation is, fairly obviously, to check facts—but it’s also to understand how and when facts are actually deployed in our political discourse. So naturally, we collected data on a whole year’s worth of PMQs exchanges between Corbyn and May. We’ve looked at how factual claims are spread across the debate (a full explanation of what we consider a factual claim and how we recorded this data can be found here).

Our findings show how common factbombing is—two thirds of PMQs we looked at included a factbomb on question six, and one quarter also included a factbomb elsewhere.

Not only is factbombing pervasive, but it consumes a disproportionately large amount of factual claims in a Corbyn-May debate. Overall, one third of all factual claims made in Corbyn-May exchanges comes on question 6 (i.e. the factbomb question)—if they were evenly distributed, of course, you’d expect that to be a sixth.

Looking beyond just question 6, 12% of all statements made by Corbyn and May during their exchanges have been factbombs.

More importantly, however, those 12% of statements which are factbombs account for 30% of all the claims made during their exchanges at PMQs.

This underlines a culture in which facts are often being used instrumentally, as rhetorical devices, rather than to make clear points in a debate or to scrutinise topics in more depth.

Why is this harmful to public debate?

It’s very difficult to get a meaningful understanding of an issue from a rush of broad-brush statistics that jump across a number of topics and metrics in seconds.

Let’s go back to the exchange we highlighted at the top of the article. Jeremy Corbyn is effectively saying that government’s policies have made large numbers of people, especially the most vulnerable, poorer. Theresa May is saying that the economy is doing better and that government is actively supporting a vast range of sectors.

The reality is that, when politicians make points this broad, they will invariably have enough statistics to back them up. You can isolate the figures that tell the story you want to tell and ignore the inconvenient ones. You can also present figures without context, ignoring awkward caveats.

Jeremy Corbyn says poverty is up: but is that absolute or relative poverty? He refers to insecure work, but how significant is the rise in zero-hours contracts in the wider picture of rising employment?

Theresa May says wages are up: but is that in real terms? Up compared to when? And how does it compare to other, similar economies? She says she’s providing the biggest cash boost in NHS history. But again, is that in real terms? And does it adequately meet the needs of the NHS?

As long as both sides indulge the other in factbombing, these awkward questions won’t be asked. It’s a kind of reverse mutually-assured destruction, with both sides talking past each other safe in the knowledge that they won’t be scrutinised on the important details behind the numbers.

It is these details which matter to the public, which tell the story of their experiences on a daily basis. If politicians fail to interrogate these aspects then they are doing the watching public a disservice.

Why should you care?

For factcheckers, factbombing is pretty annoying to sit through. The party leaders delivering rapid-fire lists of disjointed numbers, week after week, regardless of the topic at hand, can result in us shouting at the TV in despair.

But why should you, as a member of the public, feel as vexed as we do? After all, it’s not your job to follow politicians’ statements as obsessively as us. And PMQs is not watched by large swathes of the general public either. If they’re yelling another incomprehensible array of statistics, you can just switch the TV off. Or, more likely, not bother switch on in the first place.

But that itself is the problem: politicians shouldn’t be driving you to switch off the TV. And it shouldn’t only be full-time politics-watchers who can decipher their statements at PMQs. Even though PMQs will have nothing like the viewing figures of a prime-time TV debate, its content should be informative to anyone who choose to watch.

And perhaps more importantly, the behaviours we see at PMQs are likely to be repeated more widely and on bigger platforms, especially during election campaigns. PMQs may be the factbomb mothership, but it’s also a sneak preview of the claims you’ll be hearing a lot more in other venues.

In the 36 Corbyn-May exchanges we looked at, Theresa May made 36 claims about record levels of employment or unemployment (among either the whole population or younger groups), or about over 3 million more people in jobs since 2010. In that time, Jeremy Corbyn made 40 claims about poverty.

It’s to be expected that politicians repeatedly talk about issues which they find important. This is not a problem. But the issue lies in politicians repeatedly falling back on the same lines to avoid directly answering questions.

This problem is not easy to rectify, especially as factbombing makes for great social media clips for the big parties. But an important first step is to be able to identify the behaviour, and know when statistics are being used to confuse rather than clarify.

As a factchecking organisation, we don’t want people to read our content and take our assessment of a claim as gospel. We want to provide readers with as much evidence as possible, and invite them to read through it and make up their own minds on an issue. We also want to arm them with the ability to critically navigate the vast amounts of information coming their way online, in print, on television or radio.

Helping our readers to be able to identify factbombing is a step in that direction. At the very least, the next time you’re watching a politician reeling off a list of disconnected facts, you’ll know what to call it.