Excel hell

This week sees the launch of a good practice guide for releasing official statistics in spreadsheets. This might not sound important, but if you look behind the uniform face of gov.uk's statistics hub, you'll find a bewildering variety of spreadsheets.

Take a few examples and open them up.

The latest court statistics spreadsheet opens up on a random tab: “Table 1.3: Trials/small claim hearings, England and Wales, annually 2000-2013, quarterly Q1 2009 — Q2 2014¹.”

EU immigration statistics opens on the explanatory notes page, while NHS workforce statistics plunges straight into headcounts.

After even a short time spent trawling through these inconsistent spreadsheets, HMRC’s betting and gaming bulletin is a relief, with its contents, formatted fonts, and clear contact details.

If you keep clicking through the tabs, you'll see there's a rounding policy, a revisions policy, and information about related statistics.

Variety is not the spice of the researcher’s life when it comes to spreadsheet formats.

Small differences between spreadsheets can make our work significantly harder or easier: we have to spend time working out where the statistician's contact details are, when the statistics were last updated, or whether a number has been rounded. When we're live-factchecking and trying to get answers out within minutes or seconds of a claim being made, inconsistency among spreadsheets is even more frustrating than usual.

In an effort to lower our blood pressure, we talked to the Government Statistical Service (civil service statisticians) about producing a good practice guide for publishing official statistics in spreadsheets.

Around the same time we also launched a secondment scheme for the Government Statistical Service. Junior staff would come to Full Fact for three months at a time and experience how their statistics are used and interpreted in the wider world, while bringing their expertise to our team.

Our first two secondees were Louisa (from the Department of Health) and Emily (from the Ministry of Justice). They both worked on the project, interviewing journalists and researchers and assessing spreadsheets from different departments to work out what good and bad practice looks like. Read what they said about the project here.

Later on, when we were testing out how useful the draft guide was at official statistics events, we discussed the project with the National Statistician's Good Practice Team. (The National Statistician is a top ranking civil servant and statisticians throughout government are professionally accountable to him). They took on the task of turning the project into official guidance.

Last week, the guide was launched at the Government Statistical Service's Northern Conference.

If you regularly handle statistics, you can help by keeping an eye out for changes in the way spreadsheets are published. If you notice they're becoming more consistent over the next six moths (or, equally, if you think nothing's changed), please let us know via team@fullfact.org.

If you'd like to find out more about secondments, please email phoebe@fullfact.org.