A few weeks ago, I was having a debate with Darrell Bricker of Ipsos-Reid on Twitter concerning the high degree of variation between poll results for the federal Liberals. Bricker felt that we were mostly looking at the fluctuations caused by using different polling methodologies and the difference between good and bad polls. I felt that there was something going on beyond the methodological differences that was causing such great volatility.
I investigated this question in today's article for The Globe and Mail, which can be found here. Please read it, as it goes into greater detail than I will here, where I'm mostly just going to show some of my work.
I identified the current polling period for the Liberals, stretching back to Justin Trudeau's leadership campaign launch on Oct. 2, as the most volatile since the last election. I calculated this by splitting up the last year-and-a-half into blocks of polls of similar size and calculating the standard deviation from the mean in the results for each party. The result of the calculation was that this current period of polling has shown the most volatility for the Liberals than any party has experienced:
Another thing I looked at was the margin of error. Perhaps this volatility is much ado about nothing, and we're just looking at statistical wobbling. That could be, but too many of the polls' margin of error taken since the beginning of October fall outside of the average over that time.
This chart to the left shows how much the polls have differed, but also how many of them fall outside of each other's margin of error. The EKOS, Léger, Abacus, and Forum polls taken at the end of November and in December are virtually exclusive from one another in terms of their MOE bands (assuming a random sample for argument's sake, of course).
Another factor is methodology. Are live-callers returning different results than polls conducted online or by IVR? Now we might be on to something:
As is usually the case, there appears to be no single factor that is causing such volatility in Liberal polling numbers. Methodology is definitely one factor, as well as the introduction of the leadership race as a wild-card and the events that have occurred over the last three months (by-election, Alberta gaffes, etc.).
It is interesting to me, and hopefully to others, to delve more deeply into the poll numbers from time to time - particularly when something unusual is happening. This degree of volatility makes it difficult to determine exactly where the Liberals are right now, but it can safely be assumed that these trends of volatility are likely to continue straight through to April's convention.