Coffee House

Why British GDP figures are almost ALWAYS wrong.

24 October 2012

4:22 PM

24 October 2012

4:22 PM

Will it be 0.5 per cent? 0.8 per cent? 1 per cent? Whatever figure the ONS tells us GDP grew by in the third quarter of 2012, there’s one thing you can be pretty sure of: it won’t be the actual amount GDP grew by in Q3. In the past 51 years, just 12 of the ONS’s 205 first stabs at quarterly growth have survived later revisions. To be fair, the ONS recognises this, and cautiously labels tomorrow’s figure a ‘preliminary estimate’. But just how wrong is it likely to be? If tomorrow’s figure is +0.5 per cent, does that mean we can be pretty confident that growth was between, say, 0.3 per cent and 0.7 per cent — bad but not disastrous? It turns out we can’t. In fact, a preliminary estimate of +0.5 per cent really means somewhere between +1.2 per cent and –0.2 per cent — anything from encouraging to dire.

Since 1961, the ONS’s first estimates of quarterly GDP growth have been revised by an average of 0.8 percentage points. Here are those revisions for each of the 205 quarters from 1961 Q2 to 2012 Q2:

And here’s that presented another way, with the height of each bar reflecting the frequency of that particular size of revision:


As you can see, it’s not unheard of for the initial estimates to be out by more than three points. In June 1971, for example, the ONS estimated that GDP had fallen by 4.5 per cent in the first quarter of the year — by far the worst drop in any three-month period. The ONS now says that GDP fell by just 0.9 per cent — bad, certainly, but nowhere near as terrible. Or take 1988 Q3: the first estimate of –2.3 per cent has since been revised to +1.4 per cent.

But there is an ONS success story here too. As the top graph shows, preliminary estimates have become a lot more accurate. There hasn’t been a miss of more than 1.3 points since 1988. And in the last 20 years, the average revision has been 0.4 points compared to the one-point average in the previous 30 years.

From 1961 to 1992, the margin of error for first estimates was +/– 2.2 points. That is to say, 90 per cent of the final figures were within 2.2 points of the preliminary estimates. So if the first estimate was +1 per cent we could only say with 90 per cent certainty that growth was between –1.2 per cent and +3.2 per cent.

Since 1992, though, that margin of error has been much smaller, but still +/– 0.7 points. These two graphs, showing the distributions of revisions in the two periods, demonstrate how much the first estimates have improved:

As I say, the ONS does, to its credit, recognise the uncertainty in its estimates. They point out that there’s an inevitable trade-off between timeliness and accuracy. Timely statistics, they say ‘are needed by users so that their decisions better reflect current economic circumstances’. And they proudly boast that, coming just 25 days after the quarter has ended, the ONS provides ‘the fastest official estimate of its type produced by any major industrialised country’. In 2004, Marian Bell, then a member of the Bank of England’s Monetary Policy Committee agreed that ‘we would prefer early imperfect data to late perfect series — it gives us something to work with’.

Fair enough then. The ONS puts out an initial estimate that may well be several tenths of a percentage point out, because it is better than nothing, and considerably better than a guess. But you’ll have to comb through tomorrow’s release carefully to find a mention of the average size of revisions (last quarter it was at the bottom of page three, filed under ‘Supplementary analysis’). And you’ll find no mention at all of the margin of error or confidence interval — a crucial part of any estimate. Instead, the first page will tell you that ‘GDP increased by X per cent in Q3 2012’. Really, it should say ‘GDP increased by X per cent in Q3 2012, plus or minus 0.7 points’. That might muddy the headlines a bit, but it would at least be more honest.

Subscribe to The Spectator today for a quality of argument not found in any other publication. Get more Spectator for less – just £12 for 12 issues.

Show comments
  • Polleetickle of twttr

    beyond the hyperbole; almost all revisions since 1991 have fallen within 0.1-per-cent. Slow News Day?

  • tangent

    Can’t believe you’re arguing over the speed of the economic death spiral. It’s all a sham. It’s just a Bernie Madoff writ large.

  • JonB

    they probably can’t calculate proper confidence intervals, because the survey responses the ONS get in time for the preliminary round are presumably not random, so you can’t appeal to standard probability theory for random samples

  • TomTom

    Wasn’t it Jim O’Neill of Goldman who said British economic statistics had the same quality as those from Communist China ?


    We neither need nor want the fastest figure in the world. The figures impact confidence and thus ACCURACY is vital. Quoting 1 MPC member does not gainsay this.

  • Russell

    It looks to me as though the residuals have a longer correlation length in the more recent quarters – i.e. underestimates (or over-estimates) persist for ~4 consecutive quarters, rather than ~2 in the older data. Seems like ONS have bought smaller short-term fluctuations at expense of longer-term systematics. Somebody (not me!) must have the skills to control for this effect and estimate the *real* improvement in estimates…?

    • alexsandr

      in English, please? Sounds like ‘they make it up’ to me.

      wasnt it Bernard in Yes Minister who said their are lies, dammed lies and government statistics?

  • HJ777

    We don’t know how accurate any of the quarterly GDP figures are – we just know the size of the revisions. The ‘final’ figure may be pretty inaccurate too for all we know, so we can’t say that the preliminary estimates are getting more accurate, just that they are being revised less.

    As for this: “In 2004, Marian Bell, then a member of the Bank of England’s Monetary Policy Committee agreed that ‘we would prefer early imperfect data to late perfect series — it gives us something to work with’.” – it assumes that having this data and being able to act on it earlier is a good thing, but where is the evidence for this? Has our monetary policy been better than other countries as a result? I doubt it.

    I am pretty much of the view of John Cowperthwaite – that it is probably better not to measure these things to avoid the likelihood that the figures tempt government to intervene in the economy.

  • Daniel Maris

    What is clear from the graphs is that the ONS to their credit have become increasingly accurate in their estimates (as one would expect with the computerisation of record keeping) – except for the recession related blip around 2008.

    So don’t hold out too much hope.

    The much more important distortion is that we emphasise total GDP rather than per capita GDP. As the ONS has told us – mass immigration is retarding per capita GDP performance hugely. You need to get to 0.5% before you have even a chance of overall growth.

    Then there is the fact what while we dance around this totem GDP has never been a really good guide to prosperity. Although GDP was racing ahead in WW2 in the UK lives were becoming increasingly frugal. But that in turn was balanced by redistribution, so wealthier people were becoming far poorer in relative terms.

    The reality is that while per capita GDP performance has seen us get poorer by 13% in the recession – the real reduction in disposable income after tax, housing and transport are paid for is much bigger, probably closer to a quarter.

  • Richard Evans

    Ridiculous to use figures from the 60’s and 70’s, any computers used would have less power than a phone today.No one can accurately measure GDP anyway, these are ball park figures, its the trend thats important

  • dalai guevara

    Yes, let’s just print 175 billion quid since 2011:

    1- £75bn in October 2011
    2- £50bn in February 2012
    3- £50bn in July 2012

    …and then report a magical rise of 0.7% in annual GDP. On a £2,43tn
    UK GDP figure, that’s a convenient 170 billion quid rise in GDP!

    Hahahaha, fools are those who take this in as good news. I for one call it ‘levelling the books’.

    Broken and economically illiterate Britain.

  • HooksLaw

    A statistic without some measure of confidence or margin for error is meaningless.
    This cavalier attitude with statistics is never more evident than in the claims for man made global warming. However now that Michael Mann is going to court, and therefore will have to expose his methodologies, that may change.

  • anyfool

    If the figures are wrong every time why did you and your fellow scribes pounce gleefully on the figures showing a double dip in GDP,”i thought your excitement then a bit indecent”. still whatever turns you on.

  • Rhoda Klapp

    Yeah, this is daft. First GDP itself is a pretty rough figure, good for nowt but comparison with itself. Second, it is not expected to be right immediately, and thus each first guess is only fit to be compared with a previous first guess, not with final figures.

    • telemachus

      Whatever the
      figure we will not believe it helps we the voters

      People are said
      to be getting more prosperous if GDP per head increases. But GDP does not measure people’s wellbeing. GDP figures are arrived at by adding together the
      amounts paid for things in a single year. It is not just for things that make
      up living standards – like food, energy, manufactured goods or the provision of
      health and education – but also things like armies, advertising and bankers’

      The Financial
      Times noted “As US GDP rose during the past three decades, incomes
      stagnated or fell in the bottom half of the population” while GNP figures
      include “some production we would be better off if we could do without:
      guns are one example, systemic risk-raising financial products another”.

      Adair Turner
      discarded head of the Financial Services Authority commented that much of what
      finance does is “socially useless”.

      • HooksLaw

        Its a metric, as such it has value provided it is understood for what it is – how it is used is important.
        Take labour’s claims that growth will help us reduce the deficit.
        We had growth as measured by statistics under Labour, but Labour still ran ever growing deficits. Growth is not synonymous with reducing the deficit, tax revenues and spending are.

        • Richard Evans

          I agree, growth. Can be created by borrowing and spending money by a govt, but as Labour showed its how you spend that money is more important than GDP.Fritter it away and growth is a nonsense

        • telemachus

          Growth stimulates growth and in turn ups tax revenues
          Osborne cannotbgetbthis into his thick head
          Ed Balls understands

          • DavidDP

            “Growth stimulates growth”

            Yes. And yet at the same time……no.

            Your returns diminish. You’ll get to the point where every extra pound borrowed and spent results in less than that pound’s worth of growth. And that’s not even beginning to address the effect of the more you borrow, the more you need to spend on interest, leading you to have to spend even more just to stand still.

            We’ve long since reached that point. Unfortunately Balls cannot get it into his thick head.

            • telemachus

              I wrote growth begets growth
              After pump priming you need borrow no more
              You sit back and wait for the tax to come flooding in
              Then you begin to pay back
              Osborne cannot get that into his thick head

              • AdemAljo

                Does your keyboard not have punctuation keys? I’m curious…

                • telemachus

                  Too keen to get the message out

              • TomTom

                Taxation inhibits Growth

              • HJ777

                But this “pump priming” (i.e. huge borrowing and spending) has been going on for over four years now – and spending has increased under the current government. Exactly how long before we can “sit back and wait for the tax to come flooding in”?

  • DavidDP

    Problem is, that everyone will take these figures as gospel. Revisions never get picked up in the same way as teh intiial figures.
    Prompts the question as to why they release the initial figures at all – compile them yes, but then release the revised versions. Yes, there’s an argument that peolpe need to have the data, but if the data is known to be worhtless, then those people won’t be missing much.

    • alexsandr

      its a meaningless number. Just been a piece on Sky news. Seems threy only capture 40% of the stuff, and that will be from larger companies. So the recession may have been a transfer to smaller businesses, And of course it ignores the black economy.
      Then a load of wonks sit round a table and decide the figure.

  • Reconstruct

    There is reason to believe that this year the errors will have been far greater than normal. This is because indicators for industrial sector activity, particularly output and exports, have been quite extraordinarily volatile on a month-on-month basis. Indeed, so volatile has this data been that the ONS is trying to work out what’s been happening. Covered in the British Economic Review here, . It also contains ONS’s response.

    • pilsden

      Thanks well worth the read