The dark arts of election predictions

Leave a Comment » | 1,585 Views | 2 Comments » |

• Share
  
• 

John Kearon, Chief Juicer of BrainJuicer®, leads the Marketing Society’s quarterly ‘Makes You Think’ series. These deliberately provocative sessions from the frontiers of marketing are designed to make you think. As an extension of the series, this blog will hopefully provide some tasty brain snacks to keep the synapses firing between sessions.

Election fever is upon us with the televised debates and surprise Lib-Dem surge creating a buzz that makes this election perhaps the most exciting for decades. How many will vote, what percentage of the vote will each party get, how many seats will they gain and who will win? These are the questions in any election and it’s the moment our market research experts turn their attention from predicting marketing success to predicting the election outcomes. So what are the dark arts of our research experts, how accurate are they and have they changed or improved much in recent years? The answers might make you think about the techniques you use every day.

Market research as we know it, was largely invented in the 1930s by Dr George Gallup with his scientific sampling and polling work. Gallup founded the American Institute of Public Opinion in 1935 achieving national recognition by correctly predicting the result of that year’s presidential election from just 5,000 respondents. His prediction was in contradiction to the widely respected Literary Digest magazine whose more extensive poll based on over two million returned questionnaires got the result wrong. The other mainstay of political research, the focus group, was invented in the 1950’s, gaining widespread commercial use before being popularised in UK politics by New Labour.

These mainstays of election prediction and research in general are considered the best the industry has to offer but their prediction record is mixed to say the least. In the 1948 election Gallup wrongly predicted Thomas Dewey would defeat Harry S. Truman in the 1948 election, by five to 15 percentage points. Gallup believed the error was mostly due to ending his polling three weeks before Election Day, which is why today polling continues right up to the moment of the election for improved accuracy. Despite that improvement, the polls have had many notable failures on both sides of the Atlantic; failing to predict John Major’s victory in the UK 1992 election, and failing to predict Barak Obama would win the Democratic nomination over Hilary Clinton.

So eighty years on from Dr Gallup’s scientific sampling innovations and sixty years on from the birth of the focus group, what if any, are the significant improvements in research techniques to predict elections and commercial success?

Not many, if any, seems to be the official answer; a sad indictment on market research’s lack of innovation over the last 30 years, particularly in quantitative research which represents over 80% of all research. Online polling is probably the most widely discussed innovation but despite online’s superior predictive record against the offline pollsters in the last two UK elections, online accuracy is still hotly debated. To be frank, online polling is a marginal improvement of the same basic scientific sampling approach; equally able to find a representative sample but removing the interviewer bias of face-to-face. Despite these improvements online still failed to predict Obama’s success over Hilary, along with the traditional pollsters.

So are there any new, as yet unaccepted techniques, with the potential to improve election and marketing predictions?

Luckily, it seems there are. There’s a generation of breakthrough science in neuroscience, psychology, social science and behavioral economics offering new techniques with the ability to increase our understanding of human behaviour and our prediction of it. This is not about brain scanning or eye-tracking since the interpretative science behind both is still unable to tell us anything more than our current techniques. But this is about the importance of emotions in human decision making. This is where neuroscientists like Damasio, http://tiny.cc/lwr0d, have been able to show we think much less than we think we think and that our decision making is primarily emotional, based on how we feel, to which we add a healthy dollop of post-rationalisation to give the comforting illusion of making higher-order, rational decisions. Despite the central importance of emotions in decision making, until recently, research had no validated way of measuring emotions. Luckily the work of psychologist, Paul Ekman, http://tiny.cc/jeyhd, provided the theoretical framework, showing that core emotions are expressed universally in people’s faces. Researchers can now present these faces, enabling respondents to express how they feel, to what degree and why and accurately measure emotional reactions across countries and cultures. How people say they feel turns out to be more predictive of what they’ll do than asking them what they think they’ll do. I couldn’t find any examples of political researchers using this technique to predict the election, so if anybody knows of any, please share.

Then there are Predictive Markets, http://tiny.cc/0y1d1, popularized by James Surowiecki’s book, The Wisdom of Crowds, http://tiny.cc/yzzct, where he shares the extraordinary performance of the Iowa Electronic Market (IEM) over traditional polling. In over 600 elections, a non representative crowd of 500, mainly white, middle-aged guys, buying and selling shares in who they thought would win the election, were more accurate than the most accurate of the polls, three quarters of the time. Now that’s the sort of improvement in predicting elections the whole research industry should all be interested in. But it breaks the two golden rules of quantitative research; as it uses a crowd of 500 people rather than a randomly sampled representative group and respondents share what they feel other people will do rather than saying what they personally would do. As Surowiecki states in his book, so long as the crowd is large, diverse and faithfully aggregated, they do seem to be uncanny in their ability to predict what other people will do. In addition to the IEM, there are markets like the Hollywood Xchange, which has an excellent record at predicting first week box office takes and Oscar winners.

At BrainJuicer, we have spent the last five years experimenting with Predictive Markets potential for concept screening. In over 300 head-to-heads against gold standard monadic concept testing, we have shown Predictive Markets to be as accurate in identifying those rare top quartile winners but far better at discriminating between good and deeply average concepts, enabling companies not to waste time and money polishing ideas that should never get to market. We are not political pollsters but we did apply the method to the US elections just before the first primary and the crowd predicted that Obama and Hilary would be neck-and-neck and whoever won, they would go on to win the Presidency. This was at a time when no poll suggested Obama would get anywhere near Hilary. This is still a technique in its infancy but as social animals it seems we have a talent for predicting other people’s behaviour better than our own which makes techniques like Predictive Markets very interesting for the future of market research.

Sounds like it’s time to ask the crowd to predict the result and to see how people feel, emotionally about the UK election candidates and parties. We’ll share the results at the next Makes You Think session, I look forward to seeing you there.

Posted: May 4th, 2010 | Author: elen.lewis | Filed under: Customer Champions, Makes You Think | Tags: brainjuicer, election, john kearon, Makes You Think, research | 2 Comments »

---