What does polling teach us about product choices?

Why am I comparing polling with product choice? How can the way we choose a government have any relevance to the way we choose between Coke and Pepsi? Let us look at polling first. Polling is simply what people think about what they see, hear and experience. It is about understanding their choices. The best known form of polling is election polling to track party and candidate standings during federal, provincial and municipal elections. All media channels heavily depend on political polls. So do the politicians. Ultimately, all they are trying to answer is one simple question: Would you choose party A or B.

Sounds very simple, just predict a choice between two options. However, it is not always as simple as it sounds. Political polling is eight decades old in North America but still our predictions are nowhere near perfect. There have been many successes. But there have been many failures as well. Sometimes the predictions have been so spectacularly wrong that real results were completely the opposite of polling predictions. Additionally, it is not uncommon to see two pollsters predicting different results. Even where there is no shortage of data, predicting A vs. B doesn’t seem a simple task.

But if we review the instances where polls get the results wrong (or mostly wrong or find it hard to predict), most of them seem to follow a common pattern. They all contain greater than usual emotional involvement in the outcome: In the Scottish referendum most polls suggested the results were too close to call and yet the “No” side won with a convincing ten percentage point lead. In the 2012 US presidential elections, the results from polls were all over the place and very few predicted a clear and convincing win for Obama. Similar pattern can also be found in the 2014 Alberta election that was held following a scandal. And in India where many people were worried about the leading candidate’s potential communal bias and yet liked his economic policies. But even in these cases, some experts predicted the results correctly. For example, psychologist Robert Cialdini predicted the outcome of the Scottish referendum correctly . Statistician Nate Silver predicted the 2012 presidential election results not only globally but in every single state correctly .

This is where consumer choice comes in. Predicting choice of consumers is complex, because of the emotional component of many purchase decisions. Because consumers use heuristics and shortcuts, they can see $4 as more expensive than $3.99 and buy an item just because it is on sale, even though, at a conscious level, a consumer knows $3.99 is the same as $4 and would admit that it made no sense to buy an item just because it was on sale. All this makes the business of predicting simple choices a complex process. Then how can we really depend on surveys that run for 40 minutes long and ask respondent to predict their choices among 10 products or concepts? The same respondent would behave completely differently in real market than they would in responding to a survey. Indeed, if the same respondent was to retake the survey, the results could be different.

Polling as well as consumer choices become difficult to predict when there is a interplay of two factors: emotions and heuristics (shortcuts that people use to make snap decisions). Emotions make judgments unstable and heuristics can lead to seemingly irrational judgments. Hence, the difference between prediction and reality.

There are ways of working around the effects of emotions and heuristics. We have found some of these guidelines useful.

1. Be careful when there is an emotional component to consumer decisions. When emotions and heuristics take over, no matter how scientifically a sample is designed or how innovative the data collection techniques are, predicting consumer’s choice is not an easy task.
2. Improve the quality of data collection. When it is already difficult to interpret the findings, we don’t want to add to the difficulty by decreasing the quality of responses. A low hanging fruit is the length of the questionnaire. We are unlikely to get a respondent’s considered responses when the questionnaire runs over 30 minutes. Shorten the questionnaire whenever you can.
3. Use approaches that mirror real world decision-making. If I ask a consumer a factual question (such as “Did you have breakfast this morning?”) I am likely to get a truthful answer in most cases. But I am unlikely to get a truthful answer if my direct question is if the consumer bought a car to impress his neighbor even though he could not really afford it. In such cases, use data collection methods that mirror the real world such as the tradeoff technique and use analytic techniques that statistically derive the importance of attributes that drive decision.
4. Where possible, use multiple data sources to validate the results. Triangulation is always better than relying on single data source. This precisely is the technique used by Nate Silver who predicted the results of the Presidential election without himself having done any polls.

Prediction is never simple. It gets more complex when we add emotions and heuristics to the mix. But we can move towards less incorrect predictions by following some of the guidelines here.

ⁱhttps://www.research-live.com/4012273.article?

ⁱⁱfivethirtyeight.com