Ten things to know about economic models

There are three kinds of claims being made in the referendum: what’s happened in the past, what’s happening right now, and what will happen in the future.

Claims about the future are much harder to validate – especially since we’ll never know what could have happened in alternative scenarios, had different decisions been made.

Economic models are how economists use data about the past to make claims about the future. The forecasts that come out of them are different from data or statistics about past events.

Models can be very modest in size – not much more than a person with an excel spreadsheet – or they can have hundreds of equations, sub-models and many people working on them. 

Either way, a figure from a forecast is not fairly described as a fact. Often there is simply not enough data, an incomplete understanding of key relationships and an inability to foresee big shocks.

1. There is no ‘correct’ number which tells us the economic impact of leaving the EU.

The range of possible outcomes which have been suggested is very wide.

Most economists think that leaving the EU comes with an overall economic cost. But even economists who basically agree with one another have suggested quite different numbers for exactly how big that cost will be.

We published a guide to some of the pre-referendum predictions here.

2. You are not the UK economy.

Most economic forecasts look at the effect of leaving the EU on the economy as a whole, rather than the effect on individuals or families.

Headline figures like GDP and interest rates do hint at what might happen to individual families, but not all people will be affected in the same way.

Economic forecasts are a bit like weather forecasts.

We (usually) have a decent idea if it will rain later.

This is thanks to mathematical models of how the weather works. Forecasters gather lots of data, look at past patterns, make some assumptions, and solve a set of mathematical equations to tell us something useful about the world.

By gathering data, making key assumptions, and solving a system of equations economic forecasters try to tell us something useful too.

Like weather forecasts, economic models are sometimes very wrong. They have a better record of predicting the impact of small incremental changes in the near-term than large “unpredictable” events that are break with the immediate past. Notoriously, economic forecasters failed to predict the financial crisis in 2008 and were poor at forecasting the recovery.

Like weather forecasts, different models and different institutions make different predictions about what’s going to happen. There’s usually more agreement about what will happen a short way off than a long time in the future.

3. Models are only as good as the data that goes into them.

Models of the economy can be very simple – just a few lines in an excel spreadsheet – or very complex, including thousands of data series and numerous equations and sub-models.

The foundations of any model are still the same: data about the past and assumptions made by the modeller. If the data or assumptions are flawed, it will be a case of “garbage in, garbage out”.

4. Models are only as good as the assumptions that go into them.

Generally, it’s more important to understand the assumptions made by economic models and the general direction they say the economy is heading – rather than believing specific figures.

As an analogy… you’d probably listen if your doctor told you to stop eating junk food, even if she couldn’t tell you exactly how many kilograms you would weigh this time next year.

If you find the assumptions convincing, then you’re more likely to agree with the direction of travel.

Straight after a health scare about obesity, you’d be even more likely to listen to your doctor and agree that eating less junk food would be the best way to lose weight. You might also start to question what factors the doctor might have left out – exercise, for example.

5. Some processes are easier to model than others.  

We don’t know for certain what determines the choices of the many households, companies and governments, here and abroad. All of these will ultimately affect what really happens.

Because the real world is so complicated, economists have to make simplifications. This opens the possibility of making mistakes.

When assessing the impact of the referendum, economists decide which factors to include, how to count variables for which there is no data, like ‘uncertainty’, and how a change in one affects all the others.

6. Models tend to be less good with turning points.

Models look at the past to predict the future.  It follows that they are most likely to be accurate when it is reasonable to think that the future might unfold pretty much like the past.  

For the same reasons, it is no surprise that models are very poor at predicting big shocks and their consequences (such as the 2008 financial crisis).

In the case of the EU referendum, there are a lot of things in the future which could be very different. Models have to make assumptions about these, or ignore them.  

Two of the big assumptions are what might happen to migration and rules would govern trade if we left (trade tariffs, for example). Many models will not have direct data on these variables, which depend on what is happening elsewhere in the world as well as the rules we apply.

Because of this, the modeller will have to make explicit decisions about how trade negotiations might play out, or present a range of possible scenarios. We discuss these more fully in our guide.

7. Check the source.

As well as the technical issues, you may want to make a judgement about the source of the forecast.

The assumptions made in economic models can involve value judgements, and particular economists might simply have a view of the world which leads them to make certain assumptions, which in turn lead to certain conclusions.

There's a question of trust, and a question of transparency. Do you trust them to make assumptions you would accept, and do they make clear what those assumptions are, so you can judge them yourself?

8. Focusing on totals can be misleading.

Take economic growth.

Generally, a growing economy is good for most people. As GDP grows, jobs are created, wages rise, taxes can be cut and some kinds of public spending increase.

But not all prosper. For example, people might work in jobs which face increased competition - or where technologies are changing.  Likewise, many people can prosper individually when the economy is weak.

And there is a that GDP - which measures what people pay or get paid - does not represent a full picture of people’s living standards or their wellbeing.

Take interest rates or exchange rates.  

When interest rates change, different people will experience the impact in varied ways. Higher interest rates can be good for savers but bad for businesses and people with mortgages.  

Similarly, a fall in the value of the pound might benefit export industries, but harm industries who buy things from abroad.

9. There have been similar debates in the past, and they never get settled…

In the early 1990s there was a lively debate across North America about the pros and cons of the proposed North American Free Trade Area (NAFTA).  As with the current referendum debate, the issue then was the impact of changing trade arrangements. James Stanford, a trade union economist, wrote an interesting piece about the use/abuse of standard economic models at the time.

Advocates of NAFTA used economic models to show how free trade would benefit everyone (which the standard conclusion from economic theory). Workers and their representatives feared job losses, pointing to the job losses that had followed previous free trade agreements. The economists dismissed the critics as “economically illiterate or special interest protectionists”.

NAFTA is still debated twenty years later. It is impossible to accurately attribute (with any certainty) the causes of all the varied developments in trade patterns since then. Debates about the conclusions of the models still continue.

10. Models don’t b******t, people do.

None of these are reasons to dismiss economic models, or reject predictions out of hand.

Models force people to be explicit about the assumptions they have made, which factors they think will be most important and how their whole argument adds up. In that sense, models are much better than baseless projections.

Understanding the assumptions that have gone into the models, and their potential limitations, is what makes them a good starting point for debate. They only provide part of the picture, and users should never let them have the final word.