Wednesday, April 13, 2011

So I heard some polls came out today

Two contradictory polls made a splash today, one by EKOS and the other by COMPAS. A lot of Twitter ink has been spilled over these polls, as COMPAS shows a 21-point gap and EKOS a gap of only five points. But my daily poll summaries cover the polls that have been newly added to the projection. As both EKOS and COMPAS came out late this morning, they were not added to the projection. I will cover them in detail tomorrow.

But I will say a few things now. We're all very familiar with EKOS. They have a good track record, being one of the top five pollsters in my ranking of 24 Canadian firms, and they have been active in this campaign. We can use the polls they have already released in this campaign to take a look at trends and see where the parties are going. COMPAS, on the other hand, does not conduct regular voting intentions polls. In fact, since 2003 I only have records of them being active in two election campaigns within four weeks of election day, and the last one was in the 2005 Alberta provincial election.

The biggest problem is that we have nothing to compare this COMPAS poll to, and are thus unable to identify any trends. If they report again in this campaign and peg the Conservatives at 41% instead of 45%, while other pollsters also show a Tory drop, then we'll have something to chew on. But without something to compare their results to, and with them coming to a very different conclusion than what other pollsters have found, the usefulness of this poll is very limited.

Nanos, on the other hand, continues to be useful. And I'll also take this opportunity to look at how the polls have been moving in battleground Ontario since the beginning of the campaign.
In this poll of eligible Canadian voters conducted for CTV and The Globe and Mail, Nanos finds that not much has changed. The firm did report a Conservative drop for Monday, perhaps a result of the leaked G8 report, but the numbers seem to have held firm since then. They continue to lead with 39.9%, ahead of the Liberals to 30.4%. No statistically significant shifts have occured, and since Nanos's last complete three-day poll ending on April 9 the changes have been infinitessimal.

It's the same thing at the regional level, though a swap of support between the Liberals and the Bloc Québécois is of some interest. In Quebec, the Bloc has gained 5.5 points since yesterday and now leads with 38.6%. That is the best result in any Nanos poll for the Bloc during this campaign - could this be the Tout le monde en parle effect? Gilles Duceppe was on the show on Sunday night, though I thought his performance was only average.

The Liberals, meanwhile, have dropped 4.8 points to 16%, behind both the Tories and the NDP. Not even these shifts are truly statistically significant, but they could be the start of a trend that could be worth watching. Indeed, the Liberals have dropped for three consecutive days in Quebec in Nanos's daily tracking.

The party is on the upswing in Ontario, however, and that might be the story of the campaign so far.

In that province, Nanos puts the Liberals at 41.3% to the Conservatives' 40.1%. And if we look at the daily polling averages tracking chart for Ontario (updated below to include today's Nanos), we can see that the race has been truly getting tighter in the battleground province.

When the campaign started, the Conservatives had a clear edge over the Liberals in Ontario. That edge widened in the first days of the campaign as the Conservatives pushed 45% to the Liberals' 33%. But since the beginning of April the Conservatives have sunk back to around 42%, and have been there since April 2nd. They've even dropped below 42% and have been running around 40-41% in the province since the 7th.

The Liberals, on the other hand, have been steadily gaining to the point where they have overtaken the Conservatives today. Granted, adding COMPAS's poll will probably push the Tory numbers up, but EKOS found the race in Ontario be close as well, so we should expect this trend to continue in the chart.

The New Democrats have suffered the most at the hands of the Liberals, dropping from around 17% from March 29th to April 3rd down to around 13%. They've since recovered a little, but are far from the high of 19-20% they had in the first days of the campaign. They're now at around 15% in Ontario, giving the Liberals the ability to close in on the Tories. The Greens have also made more room for the frontrunners.

It is difficult to estimate where this NDP support sits geographically. I would speculate that the New Democrats are dropping in ridings where they don't stand a good chance of being elected as the anti-Harper vote coalesces around the Liberals. I have no doubt, though, that most NDP incumbents have a very good chance of holding on to most of their support.

To the left is a chart of all of the Ontario polls that have been released since the beginning of the campaign. Unlike the chart at the bottom of the page, they are in order of release. So, you can take a look at how the polls have been moving. You can see immediately how the Liberals have moved from the low-30s to the high-30s since the campaign began.

You can click on the chart to enlarge it.

Time permitting, I hope to add these regional/provincial analyses to my daily poll summaries. I'll take a look at Quebec next, and be sure to check back here tomorrow to see how the new EKOS, COMPAS, and Nanos polls shift the projection.

The questions asked by each of the pollsters are as follows: "For those parties you would consider voting federally, could you please rank your top two current local preferences?" (Nanos).


  1. Ontario and Quebec are key for both Liberals and Conservatives ...

  2. I can buy a tight voter screen creating the large gap shown by COMPASS, (Both for dropping those less committed to voting AND those not committed to a party) but I don't think that would represent an election result very well.

    I guess I never have had a major reason to doubt EKOS, except that the polling and the pollster seem to chase headlines more than other polls. That doesn't mean their polls are wrong... but I'm just suspicious.

  3. Eric, first of all, you can't expect any trends to continue.

    That's not the way polls work. They are snapshots in time. Extrapolating lines on bar graphs assuming a "trend" will continue is a mistake. We can not tell the future with polls, only perhaps know what was going on in the past, within a margin of error.

    Secondly, EKOS has chosen to release a poll of 2 days, April 11 and 12, which is interesting to say the least, since in my knowledge they have never done this before.

    These two days measure the drop in Tory support due to the leak of the auditor general's preliminary report, which one could assume will rebound over time if we are crystal ball gazing, more so than be indicative of any future trend.

    Also , I do agree that polls are always most useful compared to themselves to see how the parties are doing over time, rather than comparing different polling firms.

    So COMPAS has no reference point now, but since EKOS has no other two day poll to compare to, neither does EKOS. We need a full weeks poll to compare to EKOS. It fairly normal to have good nights and bad nights for a party in a weeks worth of polling.

    It's pretty much pure propaganda for EKOS to have released a two day poll particularly at this time. I question the motivation behind that.

  4. Given the limited usefulness of the COMPAS poll you cite, could you run tomorrow's projection model with & without it?

  5. the EKOS polling firm has often been touted as being liberal friendly - even more so then HARRIS DECIMA.i wouldn't discount the COMPAS poll,i have a feeling that these results will be closer to what we see on election night then the other results.if i'm not mistaken, EKOS was one of the polling firms that had G Smitherman and R Ford running neck 'n neck where the final outcome was closer to a landslide.i will be interested in seeing post debate polls as it is widely opined that Steven Harper did well for himself while under attack - he stuck to his message.Michael Ignatieff on the other hand was a disappointment although his supporters won't admit to it.lets say that he was far far from "wiping the floor" with far as any polls go - time will tell....

  6. Hello Eric

    I have some rather pronounced doubt about considering the COMPAS poll as being legitimate - or at least unbiased in its methodology. Here is an example of another poll of theirs:

    I especially enjoyed these completely non-partisan, non-leading questions:

    "After a sharp decline in liberal support in the election [2008] and an increase in Conservative support, the Liberals are wrong to try to gain power by teaming up with the NDP and the Bloc"

    "The Liberals, NDP and Bloc are trying to cheat the voters, who made clear their choices in the recent election."

  7. Anonymous 3:04,

    EKOS has released two-day polls before. I believe they did the same thing last week. It's a sneak peek of their longer-term poll that they will release on Friday.

    Anonymous 3:05,

    COMPAS does not have a high track record rating, so its influence will be minimal. It is a valid data point, however.

    MaHanna Ali,

    No, you are mistaken. EKOS was the only pollster to show Ford with a big lead in Toronto. They were actually very close to the final result.

  8. Skoblin,

    I am trying to find out the wording of the questions, but my emails to COMPAS have bounced. I'm trying to contact QMI.

  9. @MaHanna Ali, I think you must be thinking of a different polling firm. Please see this link, EKOS had 48% to 33% for Ford and it ended up being 47% to 36%, so if anything their polls overestimated the gap.

  10. So I think we need to wait until EKOS standard full week poll release on Friday, (used to be Thursday) to draw any conclusions or comparisons to other EKOS polls.

    The two day poll gives us an indication that over those two nights the AG report leak hurt Conservative support. We know party support can get hit one night and rebound the next.

    I will use my crystal ball to say Liberal numbers go up on Friday in Nanos, not because of the debate, but because they are the least likely voters to watch hockey playoffs. That's why we can't really trust one night or two night polls. The window is too narrow and doesn't allow for call backs thereby skewing the demographic outside of age, and gender.

    I don't think using previous COMPAS poll questions means anything to the current poll. Read the poll questions on the website and pdf. They sound fair.

    Prompting or not, who is a likely voter, calling back or not, who's undecided can explain most of poll variance.

    The EKOS poll is probably loose, prompts for party, not leader, assigns leaners, and doesn't screen for likely voters.

    The Compass poll will have a tighter screen for likely voters, does not assign leaners, may prompt with leader (Harper being most popular will add Con support) etc.

    On election day, the true voter turnout is probably somewhere in between the two estimates, and voters must decide, as well as are prompted with parties and local candidates, not leaders on the ballot.

  11. Interestingly COMPAS doesn't say if they included the leaners or not in their final numbers. At 45-23, (which is about 2:1) I bet they didn't.

    From the COMPAS web site.

    "Interviewees who respond that they are undecided about how they intend to vote are then asked to whom they are “leaning.” This second question is known as the leaner question.
    Among respondents who reveal their vote intention at the first question, Harper leads by more than 2:1 but this falls short of 4:3 among those responding to the leaner question. The dramatically greater Conservative lead in response to the first question reveals a far higher level of enthusiasm and commitment among Conservative than Liberal voters. From this pattern, barring any change in campaign dynamics, a reasonable inference is that the Liberal turnout on election day may be lower than the Conservative turnout.

  12. Iggys little temper tantrum when Layton questioned Liberals' ability to stick with election promises should provide fodder for both the Tories and NDP...if the media decided to talk about it. I come her for an unbiased view on things and ignore the mainstream media with their biased reporting of everything. Even the National Post seems to be routing for Iggy.

    Thanks to you, Eric, for keeping things on an even keel! :o)

  13. There is still a problem with this graph and analysis. Every Nanos poll doesn't have 310 observations in Ontario, as this is a rolling average. In fact, every Nanos poll only has 310/3 new observations for Ontario, which is very little. Therefore there is a high correlation between the results today and yesterday.

    Given the fact that on top of that Nanos is the only pollster putting the Liberals so high in Ontario, I don't believe the race is really that close over there. Closer than 14 days ago? Yes for sure.

  14. Um, I'm completely aware of that and make reference to it often. I also compare three-day samples to older three-day samples that have no overlap. And the graphs show the polling dates. So I'm not sure what you're trying to get at.

  15. And if you are referring to the daily poll averages chart, that incorporates the findings from other pollsters.

  16. I don't even think separate three day polls can really be used for any conclusions. Weekly results are much better.

    Gallup's president has discussed this a lot with regards to Obama's presidential approval numbers, which are a rolling average. He always cautions to go by the weekly average, which tends to even out a lot more.

    It's just not enough days to even out demographic shifts and blips, or to do call backs. Calling new people rather than those who don't answer the first time skews the data. So can a sensational news day.

  17. All one needs to do to understand the lack of credibility of the COMPAS poll is to see that it was done by the Toronto Sun.

    Sun media is the biggest right wing news outlet in the country and I'm guessing they just want something to report on when SunTV launches. I am surprised that threehundredeight would even consider this poll but I guess they have to in order to be fair.

  18. Ah Conservative scandals, the gift that keeps on giving. CBC has a story out today reporting two CPC officials for the Vaughn constituency have resigned over allegations Fantino and Goodyear gave $10 million of public money, in March, to a private health care facility.
    It begs the question, do Canadians really want these bozos renegotiating the health deal with the provinces in 2014?
    Just when the Conservative spinners attempt to influence the polls by arguing Harper won last night's debate -- a four-way tie -- one of their star Ontario candidates steps in a pile of crap.
    And for the record, I'd take Ekos numbers over COMPAS anytime, as they at least have a record of national polling. Likely the true numbers are somewhere between Ekos and Nanos.

  19. The Ekos poll was done on-line, I participated in it.

    I also found it to be a little pushy with some of the questions.

  20. Why not organize ALL the polls chronologically like the recent Ontario polls? One wonders what the trends would reveal . . .

  21. Hey Éric!

    Currently missing greens:

    ON-Etobicoke North
    ON-Hamilton Centre

    Checked the media candidate list, sorted by party, then looked for gaps in numbers that indicated missing ridings. :)

  22. I bet EKOS released the uncharacteristic two day poll because otherwise there wouldn't be a poll post-G8 news and pre-debate. If you're already polling over a week, I suppose (correct me if I'm wrong) there's no reason you couldn't extract a portion of it for a release.

  23. You know, you COULD have gone for the Beatles reference -- and the extra points -- with the header: "I read some polls today, oh boy." Please be more careful in the future.

  24. The Toronto SUN references compass as gospel.

    Enough said.

  25. The EKOS results would be very entertaining, but I can't believe it indicates anything much, at least regionally.
    Factor out the Vancouver CSA and BC is a 4-way split.
    Factor out the Montreal CSA and QC is a 4-way split.

    They've got the NDP up 30% in Vancouver's CSA in a single week (although they show a much more believable 5% shift from Liberals to NDP in the rest of BC).

  26. I really, really hope all you folk out there are watching the French debate ?

    Because it is way more the real world than the English debate of yesterday !!

  27. The COMPAS poll results are heavily based on this statement from the report:

    "Overwhelming Conservative Strength Nationally (after 35% Undecided Removed from Calculations)".

    Of course partisan divides are amplified when you remove the undecided voters, that's the nature of an election; it is equally true that the Conservatives have the most committed voters, as per Ipsos and EKOS.

    This polls should only be included in the projection model with extreme caution.

  28. Eric,

    Why do you think the Liberals are so far ahead in Vancouver South? I think a lot of Liberals will stay home and this is a dead cert for the Tories. I could be wrong of course.


  29. Mathieu Bouchard14 April, 2011 00:33

    The only times I've ever seen COMPAS is to report the highest conservative support we've seen in a long time. For example, this 45 % is an outlier, but also, in the coalition crisis just after the 2008 election, they reported an incredibly large 51 %. In both cases, they give 4 points more to the conservatives than the next highest survey of the same time period.

    Doesn't this look fishy ?

    Are surveys like this being used to influence opinion ?

  30. I have to say I really enjoyed the new format for the debates, both in French and English. But I agree with Peter that the French debate was better.

  31. I wonder, does anyone know if COMPAS rotated the order it prompted for parties in? Might explain the house effect.

    EKOS's house effect is explainable by it prompting for all parties equally, compared to some who don't prompt for any (ie Nanos) or who just don't prompt for Greens. Of course that probably skews things against the Greens as Eric pointed out...

  32. Today's Nanos is very interesting - the Bloc and the Greens are both at their lowest since the daily tracking started (7.5 and 3.1). Oddly enough none of the other parties registered their highest since daily tracking started, though NDP's is best since the first day of daily tracking.

  33. John Wright from ipsos reid had an interesting comment on the radio (AM 1010 in Toronto) this morning. When asked whether he thought the Tories could win a majority he pointed that, amongst the 57 percent of voters who, in their minds, were certain to vote, the Tories held a 44-28 lead over the grits.

    Now, two things stick out about those numbers. First, it's interesting that the percentage of "certain" voters is almost spot-on with what you would expect voter turn-out to be. In contrast, typically 75-85 percent of respondents to surveys identify themselves as "likely" voters - since voter turn-out hasn't been that high in decades (if ever) you know 15-25 percent of those respondents are lying (either to the pollster or to themselves).

    The second point is that those numbers (both in voter support and the percentage of respondents included in the data) aren't far off Compas' numbers. Now, it remains to be seen whether this is the right way to go about measuring ACTUAL voter support, but we'll see.

  34. Anonymous 20:56

    You're right, but all polling firms deal with undecided voters. Eric will know better than I do, but my understanding is that most polling numbers are exclusive of undecided voters (though you sometime see reports of decided and "leaning" voters), so it's not clear why you think doing so skews the Compas data.

  35. I'm sorry, I just don't buy the regional breakdowns in the EKOS poll. The NDP with a ten point lead in BC? I think the last time the NDP had a lead in popular support over the Conservatives in BC was perhaps in 1988. Given the numbers in Vancouver that would mean that they would have to be have a slight lead in rural BC, which is absurd. Moving on to Atlantic Canada, the poll has the Liberals 14 points out in front of the Conservatives in the Atlantic. The level of support for the Conservative based on this poll is actually below 2008 with the ABC movement in Newfoundland. This just doesn't jive with the neck and neck race that most of the other polls are showing. The rest of the regional breakdowns are lowish for the Conservatives but within the margin of error.

    I am wondering if the fact that EKOS is an on-line poll, tends to result in lower than average Conservative numbers as Conservative supporters are less apt to use computers or participate in on-line surveys.

  36. EKOS is not an online pollster.

  37. Eric, yes it is, they do their polling online, I just sent you an e-mail to prove it.

  38. The email you sent me was from Abacus Data. They are an online pollster.


COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.

Note: Only a member of this blog may post a comment.