Why don't we know how many migrants are entering and leaving the UK?

About 100 million people enter the UK every year, and about 100 million leave. Net migration—involving those who come here to stay or leave for at least a year—is a tiny fraction of that, estimated at 298,000 last year. So last year there were 298,000 more immigrants than emigrants.

The accuracy of that net migration estimate is limited—it's based on a survey of just 4,000-5,000 migrants interviewed at ports, and that means there's a large grey area. It could very easily be about 40,000 less than that in reality, or 40,000 more. Little wonder, then, that it's been described as "little better than a best guess" by Public Administration Select Committee Chair Bernard Jenkin MP.

To that you might well ask: "why not just count everyone in and count everyone out?" And you wouldn't be alone: it used to be an aspiration shared by government, statisticians and politicians alike. It still is in some cases, but delays, management problems and data issues have made this a more distant prospect.

So how have we got here? Who are we counting now? And can we count people entering and leaving our country in future?

Previous governments abolished paper-based checks

In 1994, the Conservative government of the time partially scrapped exit checks on passengers leaving the UK. In 1998, the Labour government finished the job. Those decisions have provided the background to often-heard criticisms that successive governments stopped 'counting people in and counting people out'.

The justification at the time was that the then paper-based checks amounted to "an inefficient use of resources and that they contribute little to the integrity of the immigration control", according to the Home Office.

In spite of pledges, exit checks still not fully in place

Since then, the prospect of reintroducing exit checks electronically has gained widespread favour. For the past decade, pledges to reintroduce the checks have been made repeatedly, while the timetable for actually doing so has been repeatedly pushed back.

Charles Clarke (then Home Secretary), 2005: "The new borders technology will record people's departure from the country"

Home Office, 2006: "We will progressively reinstate exit—in other words, embarkation—controls in stages, starting with the higher-risk routes and people, identify who overstays and count everyone in and out by 2014."

Gordon Brown, April 2010: "border controls have been brought in and we're counting people out and in from the end of this year"

Coalition Programme for Government: "We support E-borders and will reintroduce exit checks."

Home Office, 2012: "The Government have committed to the reintroduction of exit checks by March 2015?

Nick Clegg, March 2013: "we are reintroducing exit checks."

Nick Clegg, July 2013: [quoted] "I'm not going to pretend to you that exit checks will be restored in full, according to the Home Office's present plans, by the end of this Parliament"

Home Office, October 2014: "The Government is committed to introducing exit checks by April 2015?

David Cameron, November 2014: "We have also brought back vital exit checks at ports and airports"

Ed Miliband, December 2014: "We will introduce those [proper entry and exit] checks"

James Brokenshire (Immigration Minister), February 2015: "in relation to exit checks we are, as the programme is moving forward, seeking to take 100% of those within the scope of the programme. The reason I frame it in those terms is that there are certain exceptions."

Step forward, 'e-borders'—the solution to counting?

The government has rarely been explicit on how exactly it would implement such checks. Initially a programme called 'e-borders' was assumed to be the answer.

E-borders was a project first conceived in 2003 aimed at gathering passenger and travel information electronically and using it to help strengthen the UK's border records and security—similar to a system used in Australia.

Starting in 2008, it was expected to gather an increasing proportion of passenger data, culminating in 95% coverage by the end of 2010 (reflecting Gordon Brown's claim above) and 100% coverage by 2014. In fact, by 2010 only about 60% of passengers were being recorded and as of November last year about 80% were.

As it stated in its 2007 business case, counting was to be one of the key benefits:

"There are also a number of wider socio-economic benefits, for example the ability, for the first time, to comprehensively count all foreign national passengers in and out of the UK, improving public confidence in the integrity of the border and enabling a more accurate count of migrants for future planning and for informing the population count."

This sounds impressive. By scanning passports at check-in and departure and making use of other travel information, details such as people's name, age and nationality can be combined with flight number, times and port of departure or arrival.

A survey of a few thousand migrants versus actual passport data on people entering and leaving the country doesn't sound like much of a contest.

Except it is, because while counting everyone in and out is relatively straightforward (though no simple task, as has been found), counting migrants and filtering out everyone else is hard.

Passports provide names and numbers, but they don't tell stories. Your passport doesn't know why you're travelling, how long you're intending to stay, or whereabouts exactly you're planning to stay. So neither do the authorities—unless you happen to be interviewed on your way in or out.

Visas are another potential source of information, and integrating that into an electronic counting system was recommended by MPs last year. That would, of course, be limited to non-EU nationals because EU citizens don't need a visa to enter the UK.

Using the Census every 10 years to help identify migration flows to local areas is another option, but those figures become obsolete very quickly, and the estimates for local areas produced in between are not without accuracy problems of their own.

Not the whole solution any more

Because of these limitations, and other problems with implementation, the government has backtracked from its original business case, saying in evidence to MPs last year:

"while valuable, this data is by itself insufficient to provide a direct measurement of migration flows. As the information on entries and exits from the UK gets more comprehensive however it will, when combined with other data sources, help improve our statistics in this area."

A similar view has since been expressed by the UK Statistics Authority and—after feasibility testing the data itself—the Office for National Statistics. The Chief Inspector of Borders and Immigration questioned why the government hasn't realised this sooner: "It was not clear why the language of counting in and counting out had survived in each of the business cases."

As things now stand, the e-borders' project has been terminated. The systems are still there, but they're now referred to as 'Semaphore data' and there are separate systems for enforcement purposes. Still, the coverage isn't comprehensive, and as of last month, rail routes into and out of the UK still weren't being counted at all.

The future for counting is uncertain, but it's not necessarily confined to electronic measurement. One recommendation from MPs last year was the possible creation of a new "routine migrant survey", an option the Home Office investigated back in 2011. That could provide more detailed information on migrants' reasons for coming and what they're contributing to the UK.

For now, though, the government thinks a new survey would be bad value for money, and continues to indicate its commitment towards bringing back full exit checks at the border.

Will the government meet its target to reintroduce checks?

At a February 2015 hearing of the Home Affairs Select Committee the immigration minister, James Brokenshire, confirmed the government's plan to introduce "100% exit checks" before the election. He told the Committee that exit checks for flights were "currently at around 96%" while other checks at other exit points (such as the Eurotunnel) were still to be introduced.

Specifically he said:

"We remain confident that we will be delivering exit checks on 8 April."

However the scope of the programme has recently been clarified by the government. The scope of the exit check programme does not include certain routes, such as the Common Travel Area with the Republic of Ireland and those leaving on private boats and flights.

Achieving full monitoring of both incoming and outgoing passenger traffic has also been complicated by the fact that some transportation companies' IT systems do not facilitate data collection, as well as disagreements with some EU countries about whether they should be mandated to collect and share the information.

It's also been confirmed coachloads of under-16s from school won't be included in the checks.

So even if the remainder of exit checks are introduced on 8 April, they might not have as wide a scope as anticipated. The Committee concluded in March:

"We do not see how the Government can fulfil its pledge to deliver 100% exit checks by 8 April 2015. To make it happen will require the Government conceding greater exemptions in addition to coachloads of under 16s."

This article has been updated from a version published in December 2014.