Tuesday, May 19, 2015

Methodological note on today's projection update

I've received questions and noticed some discussion concerning the weights applied to the newest polls added to the model. I wanted to take a moment to clarify things for readers.

There are three polls that have been conducted recently that have been added to the projection: EKOS/iPolitics (May 6-12), Forum/Toronto Star (May 12-13), and Nanos/CTV (April 17-May 14). EKOS has the largest sample at 2,177, while Forum surveyed 1,286 Canadians and Nanos a total of 1,000.

In terms of the weighting, EKOS and Forum have a similar weight while Nanos has a heavier weight in the projection. Some readers have wondered why that is, considering that Nanos's poll was taken over a period of four weeks, and so includes a lot of much older data than either the Forum or EKOS polls.

Three things are taken into account when weighting a poll: the sample size, the field dates, and the track record of the polling firm. Nanos has had a very successful track record (though it has participated in fewer elections than either Forum or EKOS, and who knows how Nanos would have done in Alberta in 2012 or British Columbia in 2013 had it participated), and so has a heavier weight.

The model weights each poll on a weekly basis, depending on where a poll lands in the model's weekly blocks of time. If a poll straddles two weekly blocks, it is considered to have been conducted in the block in which the majority of the polling took place. When there are multiple weeks that fit that definition, the most recent one is selected. For that reason, Nanos and EKOS are considered to have been conducted in the same weekly block ending on May 11, while Forum is in the most recent block ending on May 18.

The model is designed this way so as not to put too much importance on differences of a day or two between polls, since the election is so far away and shifts in voting intentions are slow to occur. When the election campaign begins, the model switches over to a daily weighting scheme.

But is it fair to consider the Nanos poll as recent as the EKOS poll, considering its field dates run back to mid-April? On the face of it, it isn't. This is a quirk of the model that is being exposed by Nanos's abnormally long polls (no one else is conducting polls taken over more than a week). During the election campaign, this would not be an issue as no pollster would release such a poll.

So why not consider Nanos an older poll? The guiding principle behind the model is uniformity and objectivity. All polls are handled by the same set of criteria, and no adjustments are made by me. This is what makes it a model in my view - once you stick your thumbs into it based on your gut or intuition, you are just making an educated guess.

I used to weight polls by their median date, rather than by the last day of polling. This was meant to reflect how some polls had older data in them than others, even if they finished polling on the same day. But this had a perverse effect. If I weighted these three polls by their median date, Forum would have been weighted for May 13, EKOS would have been weighted for May 9, and Nanos for May 1. Forum would be considered four days 'newer' than EKOS, despite EKOS leaving the field only one day before Forum did.

It would reward Forum's flash-polls taken over one or two days, and penalize polls conducted over (reasonable) longer periods. The Nanos example is an extreme, but the EKOS example is very relevant. EKOS is polling over a longer period, is less vulnerable to daily blips based on the news-of-the-day, and has the opportunity to call-back people who did not pick-up the first go around. Polls taken over a day or two have no such opportunity, and run a higher risk of being unrepresentative.

Because of this, I abandoned the median-date weighting. But because I apply things uniformly, the Nanos poll is treated as far newer by the model than common sense would dictate.

Five months (to the day) from the next election, this is not a big deal. During the election campaign, no such oddity would be very likely to occur again. In any case, it hasn't had much effect on the projection, which moves slowly this far out from the vote. If the Nanos poll had been weighted for May 1, the median date, the projection would be little different: 31.8% for the Conservatives, 30.3% for the Liberals, and 25% for the NDP.

37 comments:

  1. Clear. Concise. Defensible.

    That's really all we could ask for in an explanation.

    ReplyDelete
  2. It is interesting the minimal impact that adding the good news NDP polls of Forum (30%) and latest EKOS (29%) had on only moving the NDP up to 25%.

    What would the numerical weights of the 3 new polls be?

    ReplyDelete
  3. A shift of 2% is isn't minimal in an aggregate model like the one Eric uses. It's quite telling as the individual data as you've noted shows a much higher rise for the NDP +5% for EKOS and +7% for Forum.

    There's just so much other data with lower scores for the NDP keeping those two polls in check. If more polls begin to show the same as Forum and EKOS the Model will slowly change to reflect that.

    ReplyDelete
  4. What if you did it in a more sophisticated way? Like decay each day separately, and then aggregating those weights together for the entire poll? Does that make sense?

    ReplyDelete
    Replies
    1. The effect would be (more or less) the same as choosing the median date, would it not?

      Delete
    2. Not if your decay is exponential, which I believe it is, no?

      Delete
    3. Actually, even if your decay is linear, the centroid would still be 1/3 from the end of polling, not half way.

      Delete
  5. Eric,

    Is it possible to make a seat projection based on your model that can show seats at different levels of support. If I wanted to see what would happen with the LPC at 20% or the NDP at 35%.

    I'm asking because it look like there is a major efficiency issue with the Liberals and a minor one for the CPC. At 25% the Liberal get 56 seats while the NDP would get 90. At 28% the Cons get 100 seas and at the NDP get 105 at 27%, though it does look to revise at high levels of support for the CPC.

    ReplyDelete
  6. With the new numbers, my model gives:

    138 CPC
    100 LPC
    97 NDP
    2 BQ
    1 GPC

    By region, it is:

    Atlantic
    20 LPC
    7 CPC
    5 NDP

    Québec
    45 NDP
    21 LPC
    10 CPC
    2 BQ

    Ontario
    58 CPC
    41 LPC
    22 NDP

    Prairies
    16 CPC
    6 LPC
    6 NDP

    Alberta
    27 CPC
    4 NDP
    1 LPC

    British Columbia
    19 CPC
    14 NDP
    8 LPC
    1 GPC

    Territories
    1 CPC
    1 LPC
    1 NDP

    ReplyDelete
  7. I don't get the Min/Max seat projection part:
    CPC Max + LPC & NDP Min = 310
    LPC Max + CPC & NDP Min = 300
    NDP Max + LPC & CPC Min = 285
    Where do the extra seats go?

    ReplyDelete
    Replies
    1. They aren't necessarily related. If, for example, the CPC is at its max, the LPC is probably losing a lot of seats to the NDP, so the NDP would not be at its minimum. If the NDP is at its max, the LPC is probably losing a lot of seats to the CPC, so the CPC would not be at its minimum. Etc.

      Delete
  8. Éric, has Nanos made available to you or anyone else what trend, if any, appeared in their month-long poll? In other words, is there any information about how the figures progressed from one week to the next?

    ReplyDelete
    Replies
    1. Not that I know of, but in Nanos's analysis for the CTV hit it sounded like the NDP was doing better at the tail end.

      Delete
    2. Which would accord with other poll results...

      Delete
  9. I may be wrong, but the results for the NDP federally are sounding more and more like the NDP in Alberta. First there was a sudden rise in the NDP popularity which nobody really relieved would last and then there was that huge surge at the end that put the NDP in power. The NDP is rising the same way, there is a similar three way split.
    History may be about to repeat itself federally.

    ReplyDelete
    Replies
    1. Alberta Liberal party 4.3% of the vote and 1 seat.

      So that would mean the end of the federal Liberal Party. I think they will hang on for at least 1 more federal election.

      What also is required for your scenario is a new party on the right that mainly agrees with Harper that splits away half of the Conservative vote.

      Delete
    2. With David Swann, the Alberta Liberals wouldn't have won any. He is the last of their super-incumbents.

      I expect the next Alberta election will be very interesting. Both the Liberals and the PCs are likely to disappear entirely in that one.

      Delete
    3. Yes, there are similarities but many differences; the most obvious; Stephen Harper has not been in power for 44 years! Secondly, the CPC is not "institutionalised" in government, politics, finance and culture to the degree the Alberta PC party was and or is. Glen, history may be about to repeat itself or you may end up losing a number of bets and wagers between now and Oct. 19th.

      Delete
    4. I never gamble. Just observe.

      Delete
  10. Eric,

    I understand the methodological reasoning behind it but, I think it very unlikely the Greens will take Victoria, I think this a quirk of your methodology putting too much weight upon by-elections. This is a riding that in the Provincial wipe-out of 2001 the NDP still retained 37% of the vote! I think you are over-estimating the Green vote, it may be close come election time but, I feel the NDP still has the advantage.

    If the Greens do manage to win Victoria-that will be very bad news for the NDP and I suspect over a 5-15 year period we'll see the Greens replace the NDP as the anti-establishment-protest vote party both in BC and Canada

    ReplyDelete
    Replies
    1. "This is a riding that in the Provincial wipe-out of 2001 the NDP still retained 37% of the vote!"

      2001 was a long time ago. I'd point out it was held by the Liberals federally at the time, and by a very green Liberal. The federal riding actually extends out of Victoria Beacon Hill into Oak Bay Gordon Head which is currently represented by Andrew Weaver, and even in Victoria Beacon Hill the Greens took 33% of the vote their provincially in 2013.

      "I suspect over a 5-15 year period we'll see the Greens replace the NDP as the anti-establishment-protest vote party both in BC and Canada"

      I think that's precisely what's happening on Vancouver Island, though not necessarily anywhere else.

      Delete
    2. The model gives the results it gives. It neither stands nor falls on any single riding.

      As for the Greens replacing the NDP as the "anti-establishment-protest vote party", I'm fairly sure the NDP are OK with that. They don't want to be just an outlet for protest votes (though they'll take them, of course), they want to form the next government.

      Delete
    3. Chris,

      I'm not criticising Eric's methodology but, it seems to me the model is judged by every single riding since, that is its purpose-the creation of a seat projection!

      Well the protest vote/ populist vote is currently with the BC Liberals and the CPC in BC and they keep winning elections!

      If the NDP doesn't want to be a protest party why did they run their last two election campaigns as the party of "No"?

      Delete
    4. The Greens could never hope to replace the NDP, or even compete with them on a serious level. Why? Because they are two very different political parties, with different objectives, and who mostly appeal to different voting constituencies. The NDP is a centre-left, social democratic, labour party with strong institutional links with unions. The Green Party is a sometimes progressive, sometimes conservative environmentalist party whose minimal electoral gains are only a decade old.

      The NDP appeals to labour voters, ideologically left of centre voters, environmentalists, activists, recently Quebec nationalists and occasionally protest voters. There is no way that Canada’s labour unions, who are particularly strong in BC, would ever support the Green Party. If anything, their economic policies (income splitting and the Green Tax Shift) are regressive and right of centre. Also, if you look at Canada’s election results over the past decade, most of the Green Party’s minuscule gains have not come from the NDP. When the Greens began to enjoy a small amount of momentum (after 2004), this newfound support was coming from disenchanted Liberal and PC voters, along with traditional nonvoters. The NDP also began to make more impressive gains at the expense of the scandal plagued Liberals during the same period.

      Besides committed environmentalists (we’re really talking about a tiny sliver of voters), most Green voters tend to be the completely non-ideological types who are just fed up with the status-quo. Really, those voters are fair game for any party. As we have seen before, they can vote for parties of the Hard Right, Hard Left, Centre, whatever.

      The Green Party of Canada is certainly a legitimate party, with some legitimate objectives; however, in all honesty, they face a much lower ceiling than the NDP does. Why? Because as a centre-left party committed to social democracy, the NDP not only has a coherent agenda but can appeal to voter’s pocketbooks. The Greens, on the other hand, not only pander around the political spectrum (income splitting anyone? No, how about child care?), but their platforms always contain measures that will make commodities more expensive for ordinary citizens. There is a very good reason why green parties in Europe never surpass third place. The pinnacle of Green success in Europe has been for these parties to be junior coalition partners to social democrats. In contrast, almost all European countries and most Canadian provinces have had social democratic governments.

      I like Elizabeth May, but it is very likely that the Green Party will be a short-lived experiment in Canadian political history; unless, that is, the voting system is changed within the next decade. In that event, the Greens would have a shot at winning a respectable number of seats (but they’ll never get anywhere near power).

      Delete
    5. I generally perceive the Greens as having the most fiscally conservative economic policies. The CPC did until 2008, but since then they've been spending like mad.

      Delete
  11. I'm not sure, I think the greens will take Victoria. The southern Vancouver Island part of the country is becoming a Green stronghold.

    ReplyDelete
  12. Just general kvetching ...

    The problem with having exciting new developments happen outside of the electoral window: we've got to wait whole *days* between polls!! However will we survive?!?

    ReplyDelete
  13. Here are four more ridings going NDP orange in 2015 federal election, Stephen Harper`s British Columbia support in spiral downward..

    http://www.thetyee.ca/News/2015/05/21/NDP-Dominates-BC-Coast/

    And both Burnaby ridings..

    It`s a new poll Eric..

    Cheers

    ReplyDelete
    Replies
    1. C. E. Kennedy Steawart is in real trouble which is why they moved him to South Burnaby-will the good people of Burnaby vote for someone who doesn't live in the riding?

      Delete
    2. Yes they will...I live in Burnaby and kennedy Stewart is treated like a rock star, pipeline(Kinder Morgan) push by the Harpercons has destroyed any chance they had of winning Burnaby.

      Seymour/Burnaby is gone from the Harpercons for the same reason..

      Those ridings are going orange and or staying orange.

      Cheers

      Delete
  14. When is the next IVR poll due? The anticipation....

    ReplyDelete
  15. Hey Éric, je sais tu fait du bon travail, mais j'ai seulement qu'une petite suggestion pour tes projections.

    Pourrais-tu indiquer la différence de sièges pour les partis d'une mise à jour à l'autre? Mettons par exemple: LPC +4. NDP+2, CPC -6. Comme ça, on pourra vraiment voir la tendance des sondages.

    ReplyDelete
    Replies
    1. That is a good idea Kyle!

      Delete
    2. Thank you! Its nice and all seeing how many seats each party would get but if you don't see the trend then what's the point?

      Delete
    3. There are charts showing the trends on the projection page. Adding in these extra numbers to the main chart would clutter an already cluttered chart.

      Delete
  16. What I would like to see is a FPTP Parliament and a PR Senate. That would be an effective way for the Senate to serve the purpose it was originally intended, being a backup to the Parliament, while avoiding the corruption brought on by being an unelected body.

    ReplyDelete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.