Thursday, December 27, 2012

Measuring Liberal polling volatility

A few weeks ago, I was having a debate with Darrell Bricker of Ipsos-Reid on Twitter concerning the high degree of variation between poll results for the federal Liberals. Bricker felt that we were mostly looking at the fluctuations caused by using different polling methodologies and the difference between good and bad polls. I felt that there was something going on beyond the methodological differences that was causing such great volatility.

I investigated this question in today's article for The Globe and Mail, which can be found here. Please read it, as it goes into greater detail than I will here, where I'm mostly just going to show some of my work.

I identified the current polling period for the Liberals, stretching back to Justin Trudeau's leadership campaign launch on Oct. 2, as the most volatile since the last election. I calculated this by splitting up the last year-and-a-half into blocks of polls of similar size and calculating the standard deviation from the mean in the results for each party. The result of the calculation was that this current period of polling has shown the most volatility for the Liberals than any party has experienced:
You can also see from this chart that the standard deviation for the Liberals has been higher than for the Conservatives and New Democrats in almost every period. This is probably due to the lack of a permanent leader of the Liberal Party: respondents might be answering with a generic Liberal Party in mind, one led by Bob Rae, or one led by any of the candidates for the leadership. That introduces a degree of uncertainty that doesn't exist for the other parties. However, if that was entirely the case, we should expect the same sort of volatility for the NDP in the run-up to their leadership convention in March.

Another thing I looked at was the margin of error. Perhaps this volatility is much ado about nothing, and we're just looking at statistical wobbling. That could be, but too many of the polls' margin of error taken since the beginning of October fall outside of the average over that time.

This chart to the left shows how much the polls have differed, but also how many of them fall outside of each other's margin of error. The EKOS, Léger, Abacus, and Forum polls taken at the end of November and in December are virtually exclusive from one another in terms of their MOE bands (assuming a random sample for argument's sake, of course).

Another factor is methodology. Are live-callers returning different results than polls conducted online or by IVR? Now we might be on to something:
There has been a systemic difference in Liberal results between live-callers, IVR, and online polls. Telephone surveys with live-callers have the highest Liberal results, with online being the lowest. But the differences recorded in this latest period are well outside of the average variation between the methodologies.

As is usually the case, there appears to be no single factor that is causing such volatility in Liberal polling numbers. Methodology is definitely one factor, as well as the introduction of the leadership race as a wild-card and the events that have occurred over the last three months (by-election, Alberta gaffes, etc.).

It is interesting to me, and hopefully to others, to delve more deeply into the poll numbers from time to time - particularly when something unusual is happening. This degree of volatility makes it difficult to determine exactly where the Liberals are right now, but it can safely be assumed that these trends of volatility are likely to continue straight through to April's convention.


  1. Eric, How are the "undecided" handled. Is the undecided # higher than normal over the period. I'd think that also would be an indicator of higher than normal volatility.


    1. That is a good point, but the number of undecideds is hard to measure. Simply because of how the polls are done, the number of respondents who are undecided in IVR and (especially) online polls are incredibly low.

  2. Out of curiosity, for this analysis how did you treat the Ipsos poll from November which was a live-call/internet "hybrid" poll?


    1. I treated it as both a live-call and online poll.

  3. What I take away from this is that there are a lot of "soft-Liberals" or "soft-NDP" out there that are willing to give a revitalized Liberal Party a chance, but that aren't sure that the party has been revitalized yet.

    1. Likewise in 2011, what we saw was not so much a Liberal collapse as all of the soft Liberal/NDP people deciding that this was their golden opportunity to vote NDP since momentum was there way.

      You couldn't believe the number of people I've talked to prior to 2011 who'd say to me "I'd like to vote NDP, but the Liberals are the best bet in keeping Harper out".

      If you look at numbers of people's second choices, there's a clear picture. Conservatives largely have no second choice (less than 50% name a party as a second choice), liberals and NDP largely have each other as their main second choice. The second most probable second choice for Liberals is the conservatives, while for NDP it's the green.

      The fact that there are many NDP supporters who have green as a second choice might explain why the green vote decreased so much in the last election; the NDP momentum convinced many of them to vote for a party that could win rather than their favorite party.

    2. The "bandwagon effect" of wanting your vote to count towards a winner is huge. The Liberals benefitted from it for many years, while voters allowed themselves to be convinced that just because a party hadn't won power before somehow meant that they would never win in the future.

      What happened in 2011 was that voters finally saw the possibility that it could be done -- and thus encouraged, decided to do it en masse. The Liberals will never again be able to use the scare tactic that "only [they] can keep Harper out".

    3. And your position fails when you realize that by 2015 getting rid of Harper/Reform will be foremost on the public mind.

      I'm not saying Liberal or NDP but "anybody but H "!!

    4. But it's worth recalling that a number of long-term surveys reported this year show that Canadians are increasingly divided politically. The norm until recently was for about half of Canadians to say their politics were neither liberal nor conservative... that is no longer the case. Now a plurality (40%) say they are liberal (with a sharp increase in this self-identification since 2008) vs. those who self-identify as conservative (slightly up over the past 4 years years). In other words, the electorate is, for the first time in a long while, polarised (with a significant lean to the left). If people really have turned to the NDP because they felt the party finally represented a viable alternative to the Conservatives and Liberals, then the trend is in their favour... at least until they show they're not substantially different from the Liberals...

  4. Much like Ontario, the Liberal voters as opposed to delegates are trying to explain to the party that they are moderate left of centre voters that generally defalt to the Liberals because they are usually best positioned to defeat the Tories but if they insist on being an austerity Bill 115 type party, these voters feel they have little choice but to decamp to he NDP. A Gerard Kennedy provincially or a Justn federally would keep them sweet but a Sandra Pupatello or a Marc Garneau will send these voters to the NDP. Kitcheer Waterloo and the polls are a warning to the Gr

    its, move left or become the third party, you choose.

    1. Funny how Liberal supporters and voters tend to show themselves to be more centrist and left-of-centre, while the party and its leadership tend towards right-of-centre in actual practice. We may finally be seeing the results of this systemic disconnect.

    2. Anonymous 14:30 & A.S., very astute points.

  5. This is why we poll average.

  6. Anyone consider that the media hype causes this? Really, it is quite ridiculous the amount of coverage Justin gets as opposed to other leadership contenders or other parties.

    1. Back when the NDP leadership race was going on, the media devoted a lot of ink to describing how "boring" it was.

      Brian Topp was rightly hammered by the press for trying to steamroll all of his competitors out of the leadership race right at the start. Somehow, Justin Trudeau is getting praise and a free pass for doing the exact same thing.

    2. The difference is Trudeau has popular support, Topp only had support from the the backroom boys or "establishment". I do not recall much unfavourable press toward Topp regarding his steamroller tactics, however, I do recall the NDP leadership race was categorised as "boring" after a number of debates where all the candidates agreed on policy and principle. It was not the race itself that was boring-it was the manner in which the candidates decided to campaign.


    3. Topp must have had a fair amount of popular support to end up second overall and to take the contest to a fourth ballot. And there was plenty of negative coverage in the media over his "steamroller" tactics, at least until Mulcair and other candidates caught up in organization and momentum.

      I have no doubt that Trudeau has popular support, but how many LPC coronations is that going to be now? Paul Martin, Michael Ignatieff, Bob Rae (interim), Justin Trudeau ...

      It's funny how everyone says they want less partisan and acrimonious politics, but when that is put into practice, things get slammed as "boring". We are addicted to anger and division and that's a big reason why Parliament has been allowed to become as dysfunctional as it now is.

    4. A.S.,

      Topp's strategy was to gain establishment support to garner membership lists and votes. It nearly worked and may have been successful had the NDP not changed their rules mid-campaign to allow more Mulcair supporters from Quebec and elsewhere to join.

      Whether such an outcome is worse than a "coronation" is subjective but, history demonstrates the Liberals had success with appointed leaders; Mackenzie King picked his next two successors St. Laurent who was PM for 10 years and Pearson who won two minority governments. Chretien was long thought heir apparent and he won 3 majority governments even Paul Martin the next Dauphin won a minority government and kept the Tories to a minority, John Turner was the only odd man out and he was dealt the worse hand of any leader in the last 100 years save perhaps the next one. A mixed track record to be sure but, not a disaster. They were able to do so because they had broad based public support.

      I do not wish politics to become less acrimonious. A good deal of cut and thrust is needed to encourage debate and discourse. The NDP leadership race was characterised as boring because among the 7 candidates only 1 difference in policy emerged; Nathan Cullen's strategy for "combined nominations" and collaboration with other parties. There is a certain amount of unintended arrogance and cynicism if a party is only willing to change for electoral gain. Not to say other parties are not interested in finding ways into office but, the lack of ideas does leads one to question why the NDP believes their policies are right, why is there a general consensus within it that improvement, change, reform is not necessary? Senate reform is a very good example. Why did no one propose a course other than abolition? For a party that has pinned its hope on forming Government through a Quebec-West axis they are not giving the West many carrots.

      Parliament works fine. Indeed it has worked in a like manner for the better part of two centuries. We do have a problem at the moment as the NDP is not an effective Official Opposition. They certainly have not demonstrated a willingness for parliamentary collaboration or inclination to be less partisan, acrimonious or angry than former parliaments, especially toward their opposition colleagues. When one looks down South it is difficult to label Parliament as "dysfunctional". Budgets are passed on time (even in minorities), legislation is passed regularly and pork-barreling at least compared to America is far less endemic (though it is certainly not non-existent). Indeed, the F-35 fiasco demonstrates Parliament's effectiveness. The system works as it was designed to do.

      F.R. Scott

    5. Essays aside, it's nice that some of the Liberals' uncontested leaders have been successful, but that still doesn't make it a responsible and democratic way to act. For a party that now professes to be the champions of electoral variety, it seems strange that the Liberals exhibit such a lack of democracy on the inside.

      For the NDP's part, would you really expect leadership candidates to propose drastic changes to the party right after an election campaign in which they won a record number of seats in the Commons? From their perspective, there would not be much policy that needs changing -- rather, the character and qualities of the candidates were what distinguished them from each other in the race.

      Parliament is not working fine and all major parties are to blame for being overly insular, paranoid, petty and hateful. The F-35 fiasco has actually been a shameful moment for our Parliament, as all the efforts of MPs to hold the government to account amounted to nothing as the Conservatives simply ignored it all and whipped their majority to do whatever they pleased without outside input.

    6. The efforts of M.P.s amounted to very little because the 100+ members of the NDP lack the knowledge necessary to be an effective opposition. None of the M.P.s appear to know what a soundbite is or have the ability to pose a question in the Commons capable of garnering media attention. Since, the Hon. Stanley Knowles retired the NDP has experienced a dearth of parliamentary and constitutional procedure knowledge.

      Don't complain about how the system works-out here in BC the last NDP government was not known for its willingness to allow free votes. Poor old Bev Desjarlais was kicked out of caucus by the late Jack Layton because she voted her conscience! So the NDP is quite capable and does on a regular basis whip its caucus both within the House of Commons and Legislatures.

      The Liberals certainly do not need lessons on democracy from a party that within its founding document calls for the running of society and government by appointed commission! Or a party that changes campaign rules mid-stream, or a party (in BC at least) where questions have been raised as to the free will exercised by some "members" in joining.


    7. F.R., I'm impressed that you keep reviving a week-and-a-half-old blog post to keep slamming the NDP. This must matter deeply to you.

      Of course the Liberals (like the Conservatives) think they do not need to learn anything from anyone. This attitude shows up in their actions and in their disdain for anyone who gets in the way of their "natural" governance.

      Today's Liberals have accomplished just as little as the NDP, Bloc or Greens. That is not due to the presence or absence of procedural knowledge; it is due to the fact that the governing Conservatives simply do not care what anyone else thinks, says or does. Past majority governments were more permissive, but Harper and the CPC treat all others as enemies.

      Politics is not purely about soundbites and Question Period and governments of every stripe have been increasingly quick to whip votes and disregard anything that disagrees with their chosen agendas.

      Bev Desjarlais lost her nomination (and resigned from the NDP caucus) due to her stance opposing the legalization of same-sex marriage. It is a major moral and civil rights issue. Nowadays, not even the Conservatives want to touch the subject because it is seen as a fundamental expression of equality and tolerance in this country.

      If the "founding document" you are talking about is the 1933 Regina Manifesto, it has been replaced by the NDP membership twice over: first by the CCF's Winnipeg Declaration (in 1956) and later by the Statement of Principles (in 1983). The original hard-line stances of the Regina Manifesto no longer hold any sway over party policy. Bending the truth is one thing, but surely you would not quote something that is invalid?

  7. Paul Adams has an excellent article at ipolitics regarding why the Liberal Party is moving centre-right notwithstanding a voting base that is centre-left.

    George Orwell.


COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.

Note: Only a member of this blog may post a comment.