Triumph of the 'quants'

This morning we salute an historic election victory. And no, I'm not referring to Barack Obama earning four more years in the presidential hot-seat (a victory now immortalised in the most-popular-tweet-ever). The historic victory that has the twitterati salivating with admiration*, is the triumph of the nerds--and one 'nerd' in particular; Nate Silver of the New York Times blog fivethirtyeight. Silver correctly predicted the outcome in all 50 states--as illustrated in this graphic comparison of his state-by-state prediction map with a map of actual results--beating his 2008 feat of correctly predicting 49 out of 50 (he missed out that time on Indiana, which Obama took by 0.1%).

The day before the election Silver gave Obama a 92% chance of victory, defying the 'gut-instinct' pundits who saw the election as 'too close to call', or those who only weeks ago were talking about the Romney campaign's 'momentum', while Silver still had the probability of a Romney victory at just 25%. The xkcd comic summed things up neatly as follows: "To surprise of pundits, numbers continue to be best system for determining which of two things is larger" (cartoon here).

The financial crisis did much to discredit the value of 'quants' and their stats-based analyses. But the lesson from the crisis was not so much that 'quants-are-bad', but that quantitative (predictive) modelling needs to be applied carefully, and only where the underlying model has valid statistical relevance (I've blogged about this previously here).

So does Silver's triumph herald a new dawn in the public's attitude towards statistics and quantitative analysis? There seems little doubt that the success of the fivethirtyeight formula--both in terms of its predictive power and its ability to attract a big audience--will change the nature (or at least the methods) of political punditry. But what about its potential to improve the image of stats and quantitative, data-based analysis more widely?

I saw a tweet today from a former economics lecturer of mine, who said he had his masters class running election prediction simulations in Stata (a statistical/econometric software programme) this week. Seems to me like a great way to inspire students' interest in these analytical techniques. Economics lecturers regularly lament the difficulty of trying to get undergraduates to engage with stats--many, particularly those who are more politically minded, seem to simply switch off at the sight of an equation. This appears to be almost a form of learned behaviour--evidence of a dysfunctional relationship with numbers--a deep distrust of these seemingly arcane methods of analysis and scepticism about their relevance to the 'real-world'.

The success of movements such as CoderDojo and the RaspberryPi project in promoting computer coding as a hobby for kids (big and small) proves the potential appetite for 'nerdy' pursuits, when presented as engaging, creative--as opposed to mechanistic--activities. Here's hoping the (electoral) triumph of the nerds can do something similar for getting the kids interested in numbers.

____________________________________________

*For example, @monsieurcorway tweeted: 'There's a great quote from a Romney supporter last night at Mitt's 'victory' party... He was asked if he turned up thinking Mitt would win. His response? "I read Nate Silver, that son of a bitch." Says it all. Well done, Nate - You legend.' Or this, from @alanbeattie: "Nate Silver can kill people by shooting lasers from his eyes #FACT".