Nate Silver has a lot to answer for. Yes, he called the US general election correctly – but he then led the fashionable view that opinion polling technology is now so advanced as to be able to predict what people are feeling to a high degree of accuracy. So confident was Silver in his computers that he thought he could, from the other side of an ocean, predict the UK general election with ‘90 per cent confidence.‘
The BBC lapped this up, and filmed a Panorama with the hubristic title “Who will win the election?”. Richard Bacon joined Silver as he visited the UK ‘to try and forecast the outcome of the most uncertain British election in decades’.
And then, Silver revealed what ‘the data’ said: the Tories would have 283 seats (they ended up with 330) he put Labour at 270 (they ended up with 232) and the Lib Dems at 24 (they won 8).
The Tories won more seats than all of the other parties put together. But as Bacon said at the end: ‘you can be pretty sure that if the most famous broadcaster in the world says it’s too close to call, then it’s too close to call.’ Actually, no: the Tories romped home and Silver was utterly wrong.
Of course, every pollster was wrong. As things turn out, human nature is too complicated to be predicted by computers – an important lesson for us all. Nate Silver has not split the atom of public opinion.
But rather than admit to this, the BBC is trying to cover it up. If you want to watch the Panorama, you find that it has (unlike earlier episodes) been withdrawn from iPlayer.
Could it be that the BBC no longer want viewers to see the episode which predicted the wrong result? If so, perhaps they should have done more research given that there was enough evidence to suggest that Silver is unreliable when it comes to predicting UK elections:
- * During the 2010 election, Silver predicted Labour would only win 214 seats, when actually they won 258 seats. He also predicted the Liberal Democrats would win 101 seats when they actually won 57 seats. At the time, Silver wrote on his FiveThirtyEight blog claiming ‘its very hard to see how [Labour] could thread the needle in such a way that the outcome wouldn’t be devastating to them’. Hard for him to see, maybe.
- * Ahead of the Scottish referendum, Nate Silver predicted that Yes vote had ‘virtually no chance’ of winning. They ended up with 45pc. He later said he regretted saying that ‘it looks like it’s a question of how much the ‘no’ side will win by, not what the outcome might be.’
‘When I was in Scotland last year on a book tour, a reporter asked me an off-handed question about the referendum and I provided an off-handed answer despite not really having spent any time studying it,” Silver told TPM in an email on Thursday. “That’s not usually my style, and I regret the error of having behaved like a television pundit.’
If so, he should have steered clear of the BBC when it came to worship at the shrine of his data machine. Again, Silver was no more wrong than the British pollsters. But can Mr S perhaps suggest that his computers were giving rough guesses, rather than forecasts with 90pc accuracy?
The 2015 polling disaster reminds us that campaigns do matter, that good leaders can win arguments and change minds – and that history does not run on rails. As Karl Popper said we can predict eclipses, but not revolutions. Or, as it turns out, general elections.
P.S For those who do want to see the documentary, not even the BBC can escape the power of YouTube.