The first automated factcheck
Giacomo, a Mathematics student from Imperial College London, tells us about his work at Full Fact. He's working on a pioneering project: automated factchecking. We're grateful to Imperial's Charity Insights programme for making this happen.
When I started at Full Fact, I wanted to build a program to carry out their first ever automated factcheck.
Four weeks later, that what’s I finished with: so far, the automated factchecker can check 'Employment is rising'. It reads the word ‘employment’, goes to the Office for National Statistics’ labour force data, and runs a couple of simple tests to get an idea of whether the number of people in work really is rising.
Beyond that one example, the code I wrote shows real promise. It understands what kind of data needs to be looked up when presented with an example sentence.
I spent four weeks on a problem that will take months to solve, so I was only creating a basic framework. Although I did not have enough time to get deep into the part of the software that analyses ONS data, I was able to find a good approach for the part that understands what data to retrieve.
There are still many challenges. When you get into all the different ways sentences are formulated, it seems impossible that there might be one way to process them all. How do you work out that 'since 2010', 'recently' and 'last month' all give you information about time periods, while 'to Europe' and 'in my constituency' are both location-related?
By looking at phrases from different points of view, and discussing them with others, I found and was able to implement a structure for the code that will form a good basis for future work.
It turned out my time at Full Fact was about more than just coding. Along with experience programming and communicating, I learnt about linguistics, an unexpected but interesting field. To make sure that the tool picked up all relevant claims, and excluded irrelevant ones,I had to think about the construction of speech in some detail.
I also learnt it's difficult to get tired of £2.50 falafel wraps at lunchtime.