Automated Factchecking at Full Fact

Full Fact’s automated factchecking project is central both to the future of our factchecking work and to answering the worldwide need to scale up the work of factcheckers. As the proponents of propaganda and misinformation become more sophisticated in their use of technology, it’s important that factcheckers do not fall behind in our fight against it.

We will make the first automated factchecking engine, and two products built on top, available to factcheckers and journalists around the world in 18 months. This was made possible by prototype funding from Google DNI, followed by a grant from Omidyar and Open Society Foundations. We’re hiring for this project now.

Our factchecking engine has two initial functions:

• The engine will spot claims that have already been factchecked in new places. That allows us to monitor the spread of misinformation and, more importantly, who is spreading it.
• The engine will automatically detect and check new claims using Natural Language Processing (NLP) and structured data. Where possible it will create a snap factcheck using primary source data.

We want to make this available to factcheckers worldwide, starting with two of the leading factcheckers in the world, Africa Check and Argentina’s Chequeado.

As well as building the underlying engine, the project will result in two polished products:

Trends
Trends is a monitoring tool that allows factcheckers to see who is repeating inaccurate information, and allows them to target their work and corrections for maximum impact.

Live
Live is a live factchecking tool which has two modes. It identifies claims that have been factchecked before, in new places, in real time. It provides short verdicts or data for journalists and the public. The more innovative mode identifies and factchecks new claims automatically using structured data, statistical analysis and Natural Language Processing.

These will be based on a robust infrastructure that is:

• Internationally useful without requiring advanced technical teams in each place, by offering tools that can be used ‘as a service’.
• Readily extensible to bring in new developments in automated factchecking. The engine is based on open standards, agreed by factcheckers and others, and has a modular architecture making it possible to ‘plug and play’ new components into the system.
• Capable of running 24/7.  The engine is being professionally developed as a robust production tool.

Automated factchecking is usually oversold. Nobody understands the difficulties and limitations of this effort better than we do. But we also know that nothing else can do more to help serious factcheckers scale, target, and evaluate our work at a time when it is badly needed. For that reason, we believe it is essential that serious factchecking organisations take a lead.

Full Fact’s History

Full Fact is the UK’s independent, nonpartisan, factchecking organisation. It’s our job, with our partners, to anchor public debate to reality.

We have a well-established model of change –

• In the short term, we give people reliable information to make up their own minds on big issues.
• Then, unlike most other factchecking organisations, we seek to get inaccurate claims corrected at source.
• In the medium term, we use the evidence from our factchecking to diagnose systemic problems and get systemic changes.
• In the long run, we are in a fight about the culture of public life.

Full Fact was founded in 2010. We've factchecked the Leveson inquiry, two general elections and three referendums. We go one step further than most factchecking organisations by getting corrections, and building a body of evidence to take to regulators and policy makers to try to improve the ecosystem—for example getting government departments to set up internal factchecking services and newspapers to set up corrections columns.

We are the go-to factchecking partner for national media organisations. In 2015, we factchecked debates for BBC Question Time, The Guardian, LBC Radio, Sky News, ITV and CNN. In 2016 we factchecked the EU referendum live debates for ITV and took part in a live Q&A with Sky News. This year we worked with BBC and ITV live factchecking election debates. We have also worked with or appeared in every national newspaper.

We work with the Office for National Statistics and major research organisations to make statistical and technical information more accessible for the public and media. Most recently we held a conference with the House of Commons Library, Economic and Social Research Council and UK Statistics Authority called Need to Know, where we invited key players who publish data in the UK to discuss how we can better serve the country in the next general election.

Our team is made up of eleven full-time staff. This includes specialised factcheckers with different areas of expertise. We work across the economy, education, health, immigration, crime, Europe and law. We also have a digital and communications team with skills ranging from video production to developing digital products.

We are highly regarded for our statistical expertise. Several of our team sit on advisory boards, our Director has given evidence to MPs on the communication of official statistics, and our team drafted the original official guidance for government statisticians on data presentation.

Full Fact’s roadmap The State of Automated Factchecking, launched in August 2016, set out a plan for making factchecking dramatically more effective using existing technology. In autumn we were one of the first UK organisations to use the “Fact Check” label in Google news. 

In November 2016, we announced support from Google’s Digital News Initiative for the first stages of our automated factchecking work, and we’re grateful for vital support from hosting experts Bytemark and search specialists Flax too. In January 2017 we hosted #FactHack at Facebook, with Flax. This was our first automated factchecking hackday, our second was held at PyData London.

Most recently we have announced a grant from Omidyar and Open Society Foundations which allows us to build on our prototypes. 

Why are we working towards this now?

What we believe about the world affects the decisions we make about the world, whether we are citizens or presidents. Bad information leads to bad decisions and often also to corrosive distrust.

What we believe about the world can be distorted both by the long term effect of repeated misrepresentation, and by the short term effect of campaigns that carelessly, recklessly, wilfully, or indifferently distort the facts. In the last 12 months, we have seen both effects having significant impact around the world, notably in the US election and the UK referendum. It’s been summed up as a ‘post-truth’ world full of ‘fake news’, and whether or not those labels are helpful, the problem is unmistakeable.

The political, economic, and social context of this can’t be dealt with briefly but deserves a brief sketch.

Citizens have an easy answer: to distrust everything. But nothing could do more damage to democracy. We all need information we can trust to place trust wisely. Our alternatives are awful: blind cynicism in everything; blind faith in only those we sympathise with; or complete apathy. All of these options are dangerous and there are those willing to fill the civic vacuum that pervasive distrust leaves.

In the past, the pragmatic response was to decide what sources of information to trust. That made sense when there were only a handful of TV channels, radio channels, and newspapers. Now we live in a time when apparently authoritative sources of information can be quickly generated about anything and, just as it is impossible for people to completely avoid falling for some phishing scam aimed at getting our bank details, it is even more difficult to avoid falling for misinformation—some of which has the resources of nation states behind it.

Modern systems allow for the exploitation of information sharing without them being held to any standards. The platforms that have become central to this are now waking up to a problem that has got too big for them to handle and was in part created by choices they made about, for example, algorithm design and advertising revenue models. We should always be wary about reactions to all this that seek to curb free speech, but factchecking is the free speech response to misinformation. It is essential that sophisticated modern techniques of tracking and responding to new claims are not restricted to those who seek to mislead.

Good journalists have always strived to bring their audiences reliable information: if not perfect information, then the best available version of the truth. That has always involved scrutinising important claims, especially those made by people who have or seek power. The relatively recent advent of freestanding factchecking organisations continues this tradition and makes allies of many other organisations that are vital to informed public debate, from research institutes to government bodies.

At its best, factchecking manages to combine academic rigour with popular communication, but as information sharing speeds up and sources multiply so that your news may not be the same as my news, factcheckers and our allies must aim higher.

The question we have asked ourselves is “how can we use technology to scale and quantify the excellent work already being done?” The answer isn’t to replace human factcheckers but to support them. To build tools that allow them to take what they’re already doing and amplify its impact; to build tools that will help them factcheck more important stories, faster; and to build tools that will aggregate vital data, and create a body of evidence about who spreads misinformation, what spreads most and what interventions were most powerful. We can use new technology to scale, target, and evaluate the work of factcheckers, and so to serve the public better.

Full Fact is in a unique position to contribute. We have a seven-year track record of diagnosing and successfully tackling different components of this problem. We have a strong reputation and excellent connections and we operate in a country where it is easier than most to pioneer both technical innovations and open data innovations. We are ourselves highly unusual in being both expert factcheckers and technically adept. We therefore have the opportunity, and we feel the responsibility, to take a lead and ensure that our work and the work of other factcheckers is strengthened and not weakened by new technology.

Trust is not vertical but horizontal now. It involves interactions with your peers—not just hand me downs from respected organisations. Therefore we have to get better information into those peer-to-peer conversations.

If you want to join us on this mission please look at our job openings.