Tuesday, March 30, 2010

Polling Firm Leanings Update

I've updated the pollster leanings chart for all of the national pollsters, except Strategic Counsel, incorporating the polling from the month of February 2010

Quebec has not been updated yet, but will be.

The top pollster for the Conservatives is Ipsos-Reid, which tends to poll them 3.5 points higher than other pollsters. The worst for the Conservatives is Harris-Decima, 2.4 points below average.

The top pollster for the Liberals is Environics, which tends to poll them 3.2 points higher. The worst for the Liberals is Angus-Reid, which polls them 2.1 points lower than the others.

For the NDP, the best pollster is Angus-Reid, 1.7 points higher than average. The worst are Ipsos-Reid and Environics, 2.2 points lower than average.

And for the Greens, the best pollster is the Strategic Counsel, 2.3 points higher than average. The worst is Angus-Reid, 2.2 points lower than average.

Léger Marketing and EKOS seem to be the pollsters with the fewest outliers. Léger Marketing doesn't have a variation of more than one point, while EKOS doesn't have one of more than 1.5 points.

Just for fun, let's take Angus-Reid's poll yesterday and apply these numbers to it:

Conservatives - 35% = 33.8%
Liberals - 29% = 31.1%
New Democrats - 20% = 18.3%
Greens - 7% = 9.2%
Bloc Québécois - 9% = 9%

The chart below tracks how each pollster tends to lean when calculating support levels for the various parties, as compared to the average polling results from other pollsters each month. This does not necessarily equate to a deliberate bias, but instead is more reflective of the polling methods used. This is also not a scientific calculation of any kind, but it does give an indication of how each pollster tends to compare to others.

The following chart shows each pollster's average variation from other polling firms. The numbers are the amount of percentage points a particular pollster favours or disfavours that particular party compared to other pollsters over a similar period of time.

31 comments:

  1. Eric

    Sorta does support my comment earlier that XXXX-Reid does favour the Tories?

    ReplyDelete
  2. Peter this might be helpful for you to look at:

    http://en.wikipedia.org/wiki/Opinion_polling_in_the_Canadian_federal_election,_2008#Polls

    Angus Reid came closest.

    Pollsters don't favour the Tories.

    Reality does.

    ReplyDelete
  3. Eric:
    "It is just a comparison between pollsters. "

    Sure but if the XXXX-Reids are using the same basic methodology then the results should be similar and as you point out they basically are. What it looks like to me is several sort of "front" companies running the same infrastructure.

    ReplyDelete
  4. Shadow
    "Pollsters don't favour the Tories.

    Reality does.

    I'm only going to tell you this once.

    DO NOT ADDRESS ME AGAIN

    I do not talk to LIARS !!

    ReplyDelete
  5. Norman Spector has me feeling a quite a bit better, about my "empowerment of women around the world", fetish.

    ReplyDelete
  6. Peter instead of calling people liars it would be more helpful if you indentified where you disagreed with them.

    Last poll Ipsos took in the 2008 election was actually quite hard on the Tories.

    Because leans are relative to others they only seem pro-CPC.

    Reality is that other pollsters are anti-Tory and Ipsos and AR comes closest to reality.

    ReplyDelete
  7. AJR79 that's a horrible development.

    Reading some of the comment reactions I agree with the notion that this would cost too much money at a time when we're trying to save.

    I also reject the notion that Canada owes Africa anything.

    Europe certainly does, they've made a mess of the place and its their job to fix it.

    We're a new world country, our focus should be on the Americas.

    ReplyDelete
  8. 49

    Sadly the Chief Liar on here doesn't get the message.

    I'm forced to go away. You know where I hang out.

    ReplyDelete
  9. Another good article by Soupcoff, for an HRC complaint, that has mostly slipped under the radar.

    I guess you have to be a bigshot like Macleans, or Ezra, to get much notice.

    Canada needs a George Carlin or a Lenny Bruce, and a repeal of Section 13a.
    May they R.I.P.

    ReplyDelete
  10. for an HRC complaint, that has mostly slipped under the radar.
    I don't think Guy Earle has promoted hatred of any group, only mockery of a specific person. It won't go far.

    ReplyDelete
  11. Regarding the idea that the developed world owes the developing world something, haven't we already given them quite a lot? It was the rational enlightenment in the first world that produced things like the ability to synthesize ammonia, thus allowing the production fo fertilizer. That discovery alone is responsible for millions of people in the developing world being alive today.

    The list of things that we've developed from which they benefit is staggering. What more do they want?

    ReplyDelete
  12. Éric, please excuse if this has been asked before, but what would account for consistent differences between pollsters? Assuming we can discount lying/bias, what would be the cause? One theory was about whether the questioner asks about parties by name or not. That seemed to be mostly to explain Greens differences, but not Liberals vs. CPC.

    Could it be differences in the way they select their samples? Do they publish that? The infamous Dewey defeats Truman comes to mind, where Gallup polled telephone subscribers who tended to be Dewey supporters at the time.

    ReplyDelete
  13. I don't know, it could be any number of reasons, including polling method, questions asked, how they choose their samples... There are almost too many variables.

    ReplyDelete
  14. This comment has been removed by the author.

    ReplyDelete
  15. Here is a story DL will love.

    http://cnews.canoe.ca/CNEWS/Politics/2010/03/30/13407376-qmi.html

    ReplyDelete
  16. "Reality is that other pollsters are anti-Tory and Ipsos and AR comes closest to reality."

    You could replace this with two other phrases:

    "I'm right, and everything else is wrong."

    and

    "If you're not with me, you're against me."

    Both intellectually dishonest and have no place in intelligent conversation.

    ReplyDelete
  17. Just type Guergis into the search.

    Topic:

    Helena Guergis assistant apologizes, for letters praising Guergis.

    Silly hubby Rahim did the same thing a few years ago, on the radio.

    What a pair of coconuts.

    ReplyDelete
  18. Volkov you could do no such thing.

    Those are objective facts and not intellectually dishonest at all.

    If you really want to believe Harris Decima numbers that's your bussiness.

    Reality is reality.

    And on election day the CPC was 3.6% higher than what they said they would be.

    ReplyDelete
  19. Volkov

    "Both intellectually dishonest and have no place in intelligent conversation."

    Describes Shadow to a T !

    ReplyDelete
  20. I'm curious to see how the various pollsters compared to the elections of the last few years. It would be a mistake to only look at the last election to determine if a predictable 'house effect' exists.


    Regarding Intn'l aid: We don't really owe anybody anything. But as a matter humanitarian assistance and economic development we shouldn't turn from Africa towards S. America for a few reasons.

    Sure S. America is closer, For whatever that matters) but they're better off and we don't have much history in development there. Africa --and the Caribbean, don't really need help less and we're more familiar with those regions. We may as well continue there.

    Regarding the Congo, I haven't Bthought all that much about it. but if our efforts in Afghanistan are ending, and the UN's efforts in the Congo are so far effective; it seems like an effective move.

    ReplyDelete
  21. Shadow,

    this is an interesting assumption. The polls that are taken with an actual ballot are all outliers.

    A Poll can be manipulated for any organization that is funding it. The risk is the organization can lose credibility if caught.
    If Peta, Fraser Institute has a study I would look for a potential bias in its design.

    Angus did a recap in 2008 and were most accurate. I have been reading about their investments in technology. Some interesting articles.

    I look at the summary included from each pollster and look for language that may hint at bias. A scathing or unkind comment is usually a dead give away.

    Eric states

    "This is also not a scientific calculation of any kind, but it does give an indication of how each pollster tends to compare to others."

    So why worry?

    ReplyDelete
  22. Correction "without" an actual ballot count.

    ReplyDelete
  23. This comment has been removed by the author.

    ReplyDelete
  24. Angus-Reid has been lauded a few times in comments for coming so close to the actual results in the 2008 election. Indeed, the sum of their errors was 4.4% comparing their last survey to the actual election results. The other pollsters' errors were 7.3% for EKOS, 8.8 for Harris-Decima, 9.0 for Nanos, 9.2 for Asking Canadians and 10.8 for Strategic Counsel. (Source: last poll listed from this Wiki page. I encourage somebody to check my arithmetic.)

    So Angus-Reid is the best hands-down, right? Well, maybe Angus is. For the 2006 election (from this Wiki page, same request for a sanity check) the results are Nanos 1.5%, Strategic Counsel 6.4%, Ekos 7.2% and Ipsos-Reid 8.4%.

    In other words, Ipsos-Reid was the worst of the regular pollsters in 2006. The last shall be first, although the first weren't actually last.

    Everyone should entertain the thought that pollsters may be very good at what they do, or they may just be lucky. To investigate further I invite somebody to make the same calculation for 2004. It's far from clear that either the 2006 or 2008 results will be useful predictors.

    Oh, and to confound another assertion, in 2006 Ipsos-Reid lowballed the Tories just as much as Ekos and Stategic Counsel. Nanos nailed the number. (Errors in those statements are at most 0.1%.)

    ReplyDelete
  25. John,

    I agree with your post, I compare their prediction (poll) to ballots results.

    A pattern in 2008 was everyone underpolled the CPC. Angus had the smallest variance in the between CPC and Liberal too low .6 CPC too high Lib .7 for 1.3.

    I have not tracked their predictions in other countries for four of five elections and many things can account for an improvement.

    Investments in technology and frequency of polling can improve reliability.

    ReplyDelete
  26. John i'll add a further complication.

    Pollsters may have a more accurate record for a certain party, say they get the CPC number right every time, but they get the left wing vote wrong.

    Or maybe they're bang on for the major parties and get the lesser parties wrong. Or other quirks like that.

    Example - in '08 EKOS was pretty close for Liberal support. For NDP support. And for BQ support.

    But they were -3 Tories and +3 Greens from actual result.

    Now the inclusion of "other" makes them a giant joke, basically they inflate Greens and Others at the expense of the CPC while getting the Liberal/NDP/BQ numbers more or less accurate.

    ReplyDelete
  27. 49

    Whole bunch of new stuff. Drop by

    http://forums.delphiforums.com/RelPol/messages/?msg=8330.1

    ReplyDelete
  28. Shadow: Pollsters may have a more accurate record for a certain party, say they get the CPC number right every time, but they get the left wing vote wrong.

    Or maybe they're bang on for the major parties and get the lesser parties wrong. Or other quirks like that.


    Or maybe their errors are just due to statistical variance. I think our society significantly underestimates the amount of noise in a poll. (This is especially true for numbers like Ekos's Saskitoba under-25 demographic which is sometimes based on a single-digit sample; I'd put more faith in a horoscope.)

    This is a less troublesome hypothesis than an alternative one: that the polls are accurate and voters' fickle preferences vary by government-changing amounts from day to day. Fortunately, statistics is a mature science and the margin of error statements are the tip of an iceberg of understanding. That "nineteen times out of twenty" statement really does mean that there's a probability tail out there, not a wall. Five percent of the time the prognostications will be seriously out to lunch.

    The least likely hypothesis is that pollsters have Hidden Agendas.

    I've held forth previously on the difference between voter preferences and predicted election results, so I'll give that dead horse a chance to regrow some skin.

    ReplyDelete
  29. John I don't buy hidden agendas either except maybe from EKOS given my distrust of the CBC and gov't in general.

    However, firms can also "poll shop" in which they select a pollster who's methods inflate their prefered party or issue.

    I also agree that there is a ridiculous amount of noise in polls.

    I don't trust anyone's Atlantic Canada numbers and honestly have no idea what is happening in the region.

    Someone like Angus Reid, who I trust the most, I like their top line numbers but their sub samples are too small for me to trust.

    ReplyDelete
  30. Does anyone have a list of polls just before the 2008 election, and when they were taken? I can't find anything on this.
    I think the Dion blooper reel of October 9 had a strong effect, that was mainly captured on election day October 14. Being the day after Turkey day, I and probably many others, were not getting their usual news sources due to family activities. Plus I recall otherwise reasonable people telling me not to vote for that Frenchman.

    Anyone with info on what polls and when before October 14, 2008?

    ReplyDelete
  31. liberal supporter: Does anyone have a list of polls just before the 2008 election, and when they were taken?

    Yup, the Wiki page that Shadow listed and I linked to earlier. There's a similar page for 2006 and a less useful one for 2004.

    ReplyDelete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.