Fact checking doesn’t work (the way you think it does)

Fact checkers need to move from ‘publish and pray’ to ‘publish and act’

This post will appear on the websites of Africa Check (in English and French), Chequeado (in Spanish), and Full Fact (in English). It has now been translated into Finnish and published by Faktabaari.

After another hard election FactCheck.org—the original independent fact checking project in the USA—wrote: “We saw more aggressive fact-checking by journalists in this election than ever before. Unfortunately, as a post-election Annenberg Public Policy Center poll confirms, millions of voters were bamboozled anyway.”

That was 2008.

At the last moment that blog post swerved away from what might have been the conclusion: “are we discouraged that our efforts didn’t prevent this? Not at all. If we hadn’t tried, it might have been worse.”

It should have been discouraging, but it shouldn’t have been surprising. And it’s not surprising that they could have written it again in 2012 or 2016.

The idea that fact checking can work by correcting the public’s inaccurate beliefs on a mass scale alone doesn’t stack up. Nobody should be surprised when, despite fact checkers publishing lots of fact checks, people still believe inaccurate things and politicians still spin and distort. Fact checking can work but not if this is all we do.

It doesn’t work like that because current fact checkers don’t have the reach to get the message through to all those it needs to reach. There is no realistic prospect of anyone funding the communications effort it would take to get that reach. Fact checkers are outspent by campaigns 100 to 1 or more at election times.

And it doesn’t work like that because of how our brains work. Imagine walking late into a party where people have been arguing about politics, announcing that you have a PhD in what they’re talking about, and correcting everyone there. Would you expect them to be grateful?

The most effective campaigns of persuasion engage people’s emotions and not just their heads. Organisations that take sides are better placed to run campaigns of persuasion than fact checkers.

This doesn’t mean that this approach to fact checking is not worth doing. ‘Publish and pray’ is not a completely justified criticism. Simply publishing fact checks has two clear benefits, seen both in experience and in research.

The first benefit is to the audience it does reach, who it can inform and most of the time successfully correct when needed. That audience is often larger and broader than you might imagine, as search helps fact checkers reach millions of people who wouldn’t be regular readers, and now as the increasingly widespread work with Facebook puts fact checking in front of some of its users.

When the fact checker is known and trusted as fair, they can have influential audiences too of other journalists and policy makers using their work, and in that way they can help to inform debate, even when their work is not seen directly by everyone it affects.

The second benefit of publishing fact checks is to the wider public. It’s what we call the ‘they know we check’ effect. We see individuals’ and institutions’ surprise at being asked to justify the facts behind what they say, and we see governments, media outlets, and pressure groups making changes to avoid being vulnerable to factual criticism.

The first generation of fact checking organisations tried fact checking with only this model of pure journalism. Many are still doing so. In our view the benefits are real but they are not nearly enough.

First generation fact checking has a place as part of a wider set of tactics in a fact checking organisation, or as part of the offer of major media outlets that can include fact checking accountability in mainstream interviews or bring a large audience to the fact checks. This is where it has always belonged, at the heart of good journalism. Independent fact checkers can take some credit for inspiring a renewal of interest in fact checking in the mainstream media.

But, in our view, if we want a better answer than “if we hadn’t tried, it might have been worse” then independent fact checking organisations need to do more than just publishing fact checks.

Some fact checkers have already moved on. Fact checkers who came after FactCheck.org got to learn from their open evaluation of their experience and come up with more effective models for tackling misinformation and disinformation. We call this second generation fact checking.

Second generation fact checkers—like Africa Check, Chequeado, and Full Fact—do more than publishing fact checks. Our focus is on what we change not just who we reach. Our work is about power and accountability.

What that looks like varies from place to place. It needs creativity and a sensitive understanding of the political and media environment to see what’s possible. But much of the logic is the same even when the details in different countries can be very different.

First, we move from just publishing to ‘publish and act’. We seek corrections on the record, pressure people not to make the same mistake again, complain where possible to a standards body. In other words, we use whatever forms of moral, public, or where appropriate regulatory pressure are available to stop the spread of specific bits of misinformation.

Secondly, we recognise that our fact checking provides a unique evidence base that gives us important insight into where misleading claims come from in public life and how they are spread.

Fact checkers do the legwork of understanding the problems of misinformation and disinformation in a way that nobody else does. We turn this into decision-ready information to inform our own work, others in civil society, and platforms, regulators, policy makers, and others with the powers to act to tackle distorted information.

Thirdly, we work for system change. Using the evidence from our fact checks we identify patterns and common causes, points where we can intervene to significantly reduce particular kinds or sources of information. The pattern might be who’s publishing something, where it’s published, a particular subject that there’s a lot of false information about, or something else.  The interventions can range from educating children or adults to advocating for policy changes.

Finally, culture. We are trying to create institutions in different societies that can help anchor public debate to reality and to challenge the casual acceptance of deceptive and misleading behaviour. This is a long-term task: it involves earning good trusted reputations and not just getting attention. It needs funders to think long-term as well as fact checkers.

It’s not enough to have an effective approach if it’s badly done. The simple description on paper hides the complexity of doing this in practice, the experiments and the failures and the learning that have got us to where we are. We still have a lot to learn, so now we’re investing more systematically than ever before in research to ground that learning in the best available evidence about all the different aspects of our work and the environments we work in. We know we will never stop misinformation and disinformation completely, but we can aim to reduce the harm it causes.

There is an old slogan among activists: educate, agitate, organise. Fact checkers have concentrated on education for a long time. In the future it will be our job to agitate our engaged regular readers and to organise to challenge those powerful people and organisations that provide and promote misinformation without accountability.

At Full Fact, we’ve been working on this broader second-generation approach to fact checking since we began, inspired by that honest post-election blog from FactCheck.org. We made our first corrections requests in 2010; started working on misinformation policy in the UK in 2011; started working with the authorities to improve official statistics in the UK in 2012. We’ve had successes and we’ve made mistakes, and we still have a long way to go.

At Chequeado, we developed the first live collaborative fact checking model in 2013 that now many others organizations have replicated; created the first Education Programme to teach “critical thinking” for young people; created #LatamChequea, a regional network of factcheckers; and convinced the President’s office to give them in advance all the sources of the President's speech of “State of the Union” in 2018 and 2019.

At Africa Check, we have reached out to political parties and campaign groups to change the way they communicate in South Africa; we are bringing together the health ministry, health practitioners, and community-based groups in Nigeria to tackle health misinformation; and we are developing fact-checking curricula for schools across the continent.

Together we’ve shown that this approach of going beyond just publishing fact checks can work across the world, even though it doesn’t work exactly the same way in different places.

Fact checkers can’t afford to stay stuck on the first generation model, because already we can see that even second generation approaches are no longer enough. The third generation of fact checking will have to address all these issues—but also be able to function at internet scale, be massively collaborative, and work across international borders. We’ve already seen some of the early work that’s getting us closer to that third generation. The work of fact checkers is too important to risk getting left behind.

We’d love to talk to other fact checkers about how we can all have more impact.