There are lots of things wrong with this MP’s Brexit poll

Back in August, we wrote a blog on how to spot misleading poll figures. We ran through a range of issues which lead to bad polling, including using unreliable sources, working with unrepresentative samples, and asking leading questions.

We hoped these examples of poor polling practice would help readers spot warning signs for themselves, and better distinguish between reliable and misleading poll figures.

What we didn’t envisage was that a Member of Parliament would produce their own home-made poll which managed to incorporate almost all the examples of bad polling practice which we flagged.

“Astonishing” is a good word to describe this poll, but not for the reasons Mr Elphicke, the MP for Dover, gives.

Facebook is not a representative sample

The first thing to check about any reported polling figures is the source, and to be fair to Mr Elphicke he provides it in full, sharing a screenshot of the poll on which his findings are based. This tells us it was conducted by his own official Facebook account. He lends credibility to this source by describing it as a “10,000 sample open poll on Facebook”.

But to ensure a poll is representative, size is less important than the make-up of respondents. An “open” poll is not normally how reputable pollsters achieve a representative sample: rather they  “weight” the sample to make it reflective of the population they are polling.

In this case, the people who answered this poll won’t be representative of the UK public as a whole. Many of the respondents will “like” or follow Mr Elphicke on Facebook, and given the number of replies it has received, it will have been shared by his followers or appeared on the timelines of people who like similar pages.

Brexit supporters are likely to be significantly overrepresented too. Mr Elphicke is a strong proponent of Brexit and critic of the chequers deal, and his Dover constituency voted in favour of leaving the EU by 62% to 38%.

What about people who don’t want either of these options?

The poll simply tells us which option a non-representative sample prefers when presented with two specific choices, that don’t represent all possible options.

Those who don’t want either option—Chequers or “a free trade deal” (which according to his definition implicitly means leaving the single market and customs union)—aren’t given a voice at all in the poll. Also absent are the pollsters’ good friends, the options “other” and “don’t know”.

There is no option for any other types of deal (e.g. a Norway-style one which keeps the UK in the single market for goods and services), for no deal at all, for a second referendum, or for the government to change tack and remain. These are all options which command relative degrees of popular support.

The question in the poll is a loaded one

The two choices that are presented aren’t on even terms either. Polls shouldn’t include value statements, as these can lead respondents to answer a certain way. Mr Elphicke’s poll is prefaced with a short, selective, summary of the Chequers deal and “a free trade agreement”, painting the free trade agreement in a much better light:

“They say Chequers would mean we would be half in, half out - remaining signed up to EU rules and EU laws. While a free trade agreement, which the EU offered the UK back in March, would enable us to take back control of our laws, money, borders and do trade deals around the world.”

Respondents are being strongly pushed to vote for the free trade deal over Chequers.

Moreover, respondents to his poll are given a choice between a specific deal that’s been laid out, scrutinised and criticised, and a vague one without mention of trade-offs and with its nuances open to interpretation. For all sorts of reasons, the second option is likely to sound more palatable to a wider range of opinion.

We already know that Chequers was quite unpopular with the general public (this finding, like Mr Elphicke’s claim, was reported before the EU rebuffed the Chequers agreement in Salzburg last week). But it’s difficult to get a sense of which alternative option the public might prefer, unless we have much more detail about what the alternatives are.

What is clear is that Mr Elphicke’s poll does nothing to enhance our understanding of public attitudes towards Brexit.