Tuesday, June 1, 2010

Environics Poll: 6-pt CPC Lead - and the Effects of Prompting

Environics has a new poll, and it shows big change from their last poll at the end of February. Some of that can be chalked up to the time between polls - and some of it can be chalked up to a change in methodology. More on that later. But first...The Conservatives have gained five points and are now at 36%. The Liberals are steady at 30%, while the New Democrats are down one to 15%.

The Bloc Québécois is up one to 10% and the Greens are down six to 7%.

In Ontario, the Tories are up seven points and lead with 40%. The Liberals are up three and follow with 35%. The NDP is down two to 14%.

In Quebec, the Bloc is up four to 41%, while the Liberals are down four to 24%. The Conservatives are up seven to 19% and the NDP is steady at 12%. The Greens fall away to 1%.

In British Columbia, the Conservatives lead with 39%, up two. The Liberals follow with 25% (up six) and the NDP is at 24% (up three).

In Atlantic Canada, the Liberals lead with 46% (up 10). The Conservatives lead in Alberta with 57% (also up 10) and in the Prairies with 47% (down six).

The Conservatives win 69 seats in the West, 54 in Ontario, 8 in Quebec, and 6 in Atlantic Canada for a total of 137.

The Liberals win 16 seats in the West and North, 42 in Ontario, 15 in Quebec, and 23 in Atlantic Canada for a total of 96.

The Bloc wins 51 seats in Quebec.

The NDP wins 10 seats in the West and North, 10 in Ontario, 1 in Quebec, and 3 in Atlantic Canada for a total of 24.

Obviously, bad news for the NDP but a significant gain for the Liberals and a relatively status quo situation for the Conservatives.

Now, about the methodology. When Environics last conducted a national poll in February, they prompted the party names when surveying voting intentions. In other words, they asked the survey-takers "If a federal election were held today, would you vote for the Conservative Party, the Liberal Party, etc." The party names would be listed and rotated each time a new person was asked.

This time, Environics did not prompt. They simply asked respondents for whom they would vote, without listing party names.

The first method gives people their options, and it makes it easier to say for whom they would vote. The second method requires respondents to be more informed and have an actual idea for whom they want to vote. There is an argument for both sides as to which is more accurate, as the voting ballot itself also "prompts".

So, let's compare these two polls. They should give us an indication of how prompting changes a poll, because instead of comparing two different pollsters, we're comparing one pollster who should use the same methods to find their respondents. Thus, the only variable in this comparison is whether respondents were prompted or not.

First, let's look at the undecideds. In that February prompted poll, undecideds were 11% of the population - a rather low number. In this unprompted poll, undecideds are 27%. Now, perhaps the last three months has made it harder for people to decide. Or, prompting makes otherwise undecideds more decisive.

But what about how it affects each party? Well, we can compare that, too. Obviously, we can't compare the results of that February poll to this May poll without any filter. The change in political climate would have a greater role in any voting intention changes than mere methodology.

So, what I have done is estimate the kind of changes the Environics poll should've shown, assuming they had been consistent with their methodology. This estimate was calculated by taking the EKOS, Angus-Reid, and Harris-Decima polls of late February and comparing them to the EKOS, Angus-Reid, and Harris-Decima polls of late May. We're comparing polls taken at the same time as the two Environics polls, using the change between the average of the three February polls and the three May polls as our guide for estimating how much the Environics poll should have changed.

Between February and May, Conservative average growth was 2.7 points (32.3% to 35.0%), or 8.4%. The Liberals have dropped 3.4 points, or 11.3%. The NDP has hardly changed (17.3% to 17.0%), as has the Bloc (8.3% to 8.7%). The Greens also haven't changed (10.3% to 10.3%).

In other words, if prompting plays no role in polling, then we should see the Conservatives up a little, the Liberals down a little, and the NDP, Bloc, and Greens virtually unchanged. Instead, we see the Conservatives higher than they should be, the Liberals, NDP, and Bloc stable, and the Greens way down.

For the Conservatives, they should have gone from 31% in February to 34% in May. Instead, they are at 36%. The Liberals should have gone from 30% to 27%, but instead they are still at 30%.

And, significantly, the Greens should have remained at 13%. Instead, they've dropped to 7% - almost half of their support in February.

What this shows is that prompting gives a boost to the Green Party more than any other, in conjunction with a lower undecided result. The Liberals and Conservatives don't get more support, but they do get a higher proportion of decided support. This indicates that, in a prompted poll, many undecideds choose the Greens as their "parking spot".

The question remains: which is accurate? History argues that Green results in prompted polls are inflated, and inaccurate. On voting day, the Greens have always gotten fewer votes than the pollsters thought they would.

But the Greens have defied history before.

However, until we finally see a Green electoral result matching their polling results, we will have to side with the historical record.


  1. It does not seem to matter if it is a "prompted" or an "unprompted" poll. There is still no majority for the Conservative party.

    The Liberals, do seem to do better in "unprompted" polling, and do have a bigger base of support than in a "prompted poll", such as Ekos, and HD.

    The Liberals, also do better in the Nanos polls, because the respondent must name a preferred party.

    The Greens, do seem to be a little bit of a parking lot, in prompted polls. That is not to say the results are inaccurate, but their support does seem a little inflated.

    Anything can happen in a campaign, and usually does. The governing party tends to lose support in a campaign, so if an election were held at the present time, I would expect a reduced conservative minority, and a stronger Liberal opposition.

    The NDP, will also probably lose some seats, and it is hard to say if the Greens could elect Elizabeth May.

    I expect the Bloc, to do well now, or in the fall if we go to the polls.

  2. It really does look like stagnation.

    No real advance or change anywhere.

    I still contend we need a "none of the above" option as I think there is a lot of discontent out there and no party has really addressed it in any measurable way.

  3. "..Conservative support is ten points higher among men than it is among women, while Liberal support
    is significantly higher among women than it is among men."

    What? there are ahead by 2 points for God's sake amongst women!!!!!!!!!!!! And with the House Effect graph Environics is usually +3 and a bit for the libs and -1 for the cons which could be 37-27 which is in line with most latest polls. Chretien won his second majority at 38.5%!!!!

  4. I like the unprompted polls better, like Nanos. It seems while the Greens get screwed in umprompted polls the Liberals see the biggest gain. This poll and the Nanos poll earlier the month showed Liberal support a good bit higher then other pollsters have this month.

  5. Nice analysis Eric given the data.

    I would suggest that on the open answer polls that a follow-up questions be given :

    Will you vote in the next election?

    Who is the Prime Minister of Canada?

    These simple questions should be included in all polls. We have 40-60% of the voting population that will not vote. To make the political outcome somewhat accurate the first priority of pollsters should be to identify the sample that will actually participate.

  6. Fascinating analysis!

    This gives us a most-probable range of Green support if an election were to be held tomorrow: it's unlikely to be above 13% or below 7%. If the Green Party pumps up mind share during the campaign the vote could be near the high end; if the party hides under a rock, the vote could approach the 2008 levels. Since the top is almost twice the bottom and the mid-point is roughly the pollsters' average, that's a lot of words to say "we don't know".

    However, there is an essential message: brand recognition is everything. This is why the first Green MPs will be so important: the media will seek them out to comment on every issue in the name of balance. Greens will get the media free ride that other parties have enjoyed all along.

    For Greens, the good news is that brand recognition isn't equally vital everywhere. Today, it matters most in a few key ridings. Prompt the voters in those ridings to think Green and the game can change markedly. Be very sure that Greens are concentrating where it matters.

    It would be an interesting exercise to compare prompted and unprompted polling in key Green ridings and some controls. I'd guess that a small sample space and weak demographic corrections would still provide interesting numbers.

  7. My suspicion is that the people who are "undecided" when unprompted are the same people who don't vote. I mean, we all know that the polling data won't perfectly reflect election results, since only 60-odd percent of eligible voters will actually show up to vote. And I'd suggest that a fair number of those non-voters are probably the same people who need a prompt in order to "pick" a party. I mean, really, if you're so clueless that you can't name a political party that you might want to vote for without being told their names, you're probably sufficient uninvolved that you're not going to spend a half hour or so heading out to the local polling station to vote come election day. (This is basically BC's point - if you can't name the prime minister, how likely are you to be a voter?).

    In that sense, the unprompted polls may, indirectly, get more accurate results, because they exclude a greater proportion of non-voters. That's probably also why they more accurately predict green support.

  8. I tend to like unprompted polls more simply because of how realistic the Green results look. I mean, the only time parties explode to the level than Ekos claims the Greens will is is there is another party starting to collapse, case and point in 1993. Otherwise, its a slow, gradual build. Unprompted polls tend to show a tiny jump for the Greens over 2008, yet a jump that would make sense with a party which rarely has any say in the media and is trying to build its organization up.

    11% in BC looks reasonable. 10% in Ontario looks reasonable. All the results look like results that would end up in an election for the Greens. Not the huge 10-12% in Quebec from Ekos, which goes against all common sense, considering that the Greens haven't even reached the 4% threshold in la belle province!

  9. While it is true that the ballot prompts voters, those voters who can't be bothered to have an unprompted opinion aren't likely to take the time out of their day to go to a polling station (or even know an election is taking place).

    This is why I oppose mandatory voting: All it does is reduce the signal to noise ratio in the ballot result.

    The Liberal data point in this Environics poll looks weird. I maintain a scatterplot of all the poll results (I started doing this prior to the 2006 election), and some data point just don't fit the obvious trend line. Most recently, it's been the Liberal number from the Nanos polls, but this number does the same.

  10. Eric:
    In that February prompted poll, undecideds were 11% of the population - a rather low number. In this unprompted poll, undecideds are 27%.

    Which actually supports my comment re "none of the above"? No prompt and the undecideds jump significantly essentially saying they don't really see anything they want.

  11. Have to agree, the GPC support is soft and largely with people who don't vote - namely those under 30.

    How to change that? Who knows? The NDP tried for years to little effect (they used to be the darlings of the sub-30 crowd). Obama did everything right it seemed, but still barely increased youth vote as a percentage overall.

    I think this shows why the big 2 set their policies for seniors (income splitting for seniors only) and not for the sub-30 crowd (lower tuition would be nice, but instead it is always going up).

  12. http://www.thestar.com/news/canada/article/817622--hebert-jack-layton-s-surge-great-news-for-stephen-harper

  13. Peter: Govt Stalling On Detainee Deal

    Now who could have imagined that would happen?

    The skeletons will stay rammed in the closet until after the fall election. The news here is that we now know how the government will fall: the contempt motion after Parliament resumes will be non-confidence.

  14. John:
    the contempt motion after Parliament resumes will be non-confidence.

    Yeah sure looks like they are setting it up that way. Unless the Opposition moves Contempt against only specific ministers ?

  15. Chantal Hebert has it all in reverse. The good news for Stephen Harper is that Ignatieff is a FLOP. Layton is there to pick up the pieces because he is a more appealing "anti-Harper" than is Iggy - but ultimately it still comes back to one simple fact - Iggy is a FLOP. In fact we already know from the ARG poll that the best new of all for Harper would be for Layton not to be there at all and for Ignatieff to lead a united Liberal/NDP - then in a head to head race - Harper would get his majority.

  16. The Liberals need to stop trying to get votes from the NDP and instead try to get them from the Conservatives. There are a lot of fiscal conservatives who are uncomfortable with the Conservatives' foreign and social policies. The Liberals would do well to capitalize on that, leaving true left-wing Canadians to the NDP and right-wing Canadians to the Conservatives.

    Instead, it seems like they are more focused on re-gaining Green and NDP votes. I don't think that is the solution. The NDP at 20% is fine, if the Conservatives are at 30%.

  17. Éric: The Liberals need to stop trying to get votes from the NDP and instead try to get them from the Conservatives.

    And that goes double for Greens.

  18. There are a lot of fiscal conservatives who are uncomfortable with the Conservatives' foreign and social policies.

    Biggest name I can think of is Hugh Segal who still calls himself a Progressive Conservative and battles with the CPC quite often.

    There are lots of the old PC still around who aren't happy with the CPC but have nowhere else to go. The Libs should poach them ! I agree Eric

  19. Peter,

    According to his parliamentary webpage, Hugh Segal is sitting as a Conservative senator, he's listed as a conservative on the conservative party webpage, and he was one of Stephen Harper's advisors in getting him elected. So I'd suggest that you might be overstating his disagreements with the conservative party.

  20. overstating his disagreements with the conservative party.

    I've heard what I quoted from several sources. I'll stand with what I said.

  21. In fact, he's so unhappy with the Conservative Party that he's stopped identifying himself as a Conservative Party Senator. Oops, wait, ...

  22. All,

    Hugh Segal is my political role model. Enough said.

    My, how we would be extremely fortunate if he chose to come across to the Liberals.

    But I suspect he prefers to serve as a moderation brake on the runaway train. I have no choice but to respect that, should that in fact be the case.

    Tried it myself for a few years.

  23. "However, until we finally see a Green electoral result matching their polling results, we will have to side with the historical record."

    Well, we did see that in the Ontario 2007 election, where the Green ballot (8%) precisely matched the steady polling at that level. So perhaps you are only referring to federal elections?

  24. Yes, I'd say that those are different animals.

  25. Great post....imho, polling methodologies do not get enough attention from the mainstream media. There are subtle, and sometimes not so subtle, changes that can occur from poll to poll that can affect responses and should be highlighted. Again, Well done...I don't pay particularly close attention to monthly/weekly polls as the most interesting numbers are never published - information and attitudes of the undecideds and on those have "switched" or changed their mind...phil courterelle, Calgary....


COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.

Note: Only a member of this blog may post a comment.