Observer Twitter poll data on air monitors is meaningless

“Only 8% of schools have received air monitors that were promised by government”

On 24 October the Observer reported that just 8% of schools had received carbon dioxide monitors promised by the government.

The monitors allow teachers to identify where ventilation might need to be improved, in order to aid the fight against the spread of Covid-19.

But the Observer’s article was based on a Twitter poll carried out by the headteacher of a primary school, which received 1,617 responses.

As we have written before, Twitter polls are meaningless for understanding anything other than the views of the specific people who answered the poll. 

The main reason for doing a poll in the first place is to find out about the views or behaviours of a group of people. As you usually can’t talk to all of them, you choose instead to talk to a sample of them.

That means a poll is only useful if you have some confidence that the people you are surveying are broadly representative of the people you’re interested in.

But with a Twitter poll you can never know this. In this case, you can’t know whether the respondents were people qualified to answer on the subject (teachers, school staff, parents etc). And even if they were, you can’t know if those people can give a representative view of the situation across the country more generally.

After publication, the Observer edited its headline to: “Only 8% of schools in England have received air monitors, headteacher’s Twitter poll suggests.”

This was still inaccurate. The Twitter poll doesn’t suggest that only 8% of schools in England have received air monitors. It tells us only about the responses of whichever Twitter users happened to engage with the poll when it was run, and nothing about the real world.

Also, the poll itself was ambiguously worded, asking: “Anybody seen a CO2 monitor yet? Asking for a school?”

It is possible that some respondents may have answered “no” if their school had received a monitor but they hadn’t personally seen one or had one in their class. 

This is another reason why the data can’t be used to show what proportion of schools have not received a monitor. And of course, it may also be the case that some schools already have monitors which were not provided directly by the Department for Education. 

After publication of this article, the Observer removed the article online. It wrote: "Twitter polls are informal and, if choosing to report them on a significant issue, we should be clear that respondents are self-selecting users of the platform who may or may not be statistically representative. In this case we failed to make that clear.

"The poll question, which was not given in the article, was, “Anybody seen a CO2 monitor yet? Asking for a school?”, and it cannot be said whether all those answering were in a position to confirm school arrangements.

"The online version of the article, which was amended shortly after publication to include reference in its headline to the Twitter poll, has been withdrawn from our website."

So how many schools have received air monitors?

Unfortunately, a Twitter poll on this subject can tell us nothing at all about the scale of the roll-out. But there are no official statistics available showing the latest progress of the roll-out either.

When we asked the Department for Education how many schools actually have received a monitor, a spokesperson told us: “We cannot pre-empt the publication of official Government statistics, but it would be inaccurate to report that the number of monitors being rolled out is equivalent to the number of settings that have confirmed receiving one.

“Our rollout of at least 300,000 carbon dioxide monitors to schools is on track.”

This refers to the target that education secretary Nadhim Zahawi suggested would be met in November, with over 90,000 being delivered by the end of October. For context, there are around 22,000 state or non-independent schools in England.