Tuesday, June 1, 2010

Environics Poll: 6-pt CPC Lead - and the Effects of Prompting

Environics has a new poll, and it shows big change from their last poll at the end of February. Some of that can be chalked up to the time between polls - and some of it can be chalked up to a change in methodology. More on that later. But first...The Conservatives have gained five points and are now at 36%. The Liberals are steady at 30%, while the New Democrats are down one to 15%.

The Bloc Québécois is up one to 10% and the Greens are down six to 7%.

In Ontario, the Tories are up seven points and lead with 40%. The Liberals are up three and follow with 35%. The NDP is down two to 14%.

In Quebec, the Bloc is up four to 41%, while the Liberals are down four to 24%. The Conservatives are up seven to 19% and the NDP is steady at 12%. The Greens fall away to 1%.

In British Columbia, the Conservatives lead with 39%, up two. The Liberals follow with 25% (up six) and the NDP is at 24% (up three).

In Atlantic Canada, the Liberals lead with 46% (up 10). The Conservatives lead in Alberta with 57% (also up 10) and in the Prairies with 47% (down six).

The Conservatives win 69 seats in the West, 54 in Ontario, 8 in Quebec, and 6 in Atlantic Canada for a total of 137.

The Liberals win 16 seats in the West and North, 42 in Ontario, 15 in Quebec, and 23 in Atlantic Canada for a total of 96.

The Bloc wins 51 seats in Quebec.

The NDP wins 10 seats in the West and North, 10 in Ontario, 1 in Quebec, and 3 in Atlantic Canada for a total of 24.

Obviously, bad news for the NDP but a significant gain for the Liberals and a relatively status quo situation for the Conservatives.

Now, about the methodology. When Environics last conducted a national poll in February, they prompted the party names when surveying voting intentions. In other words, they asked the survey-takers "If a federal election were held today, would you vote for the Conservative Party, the Liberal Party, etc." The party names would be listed and rotated each time a new person was asked.

This time, Environics did not prompt. They simply asked respondents for whom they would vote, without listing party names.

The first method gives people their options, and it makes it easier to say for whom they would vote. The second method requires respondents to be more informed and have an actual idea for whom they want to vote. There is an argument for both sides as to which is more accurate, as the voting ballot itself also "prompts".

So, let's compare these two polls. They should give us an indication of how prompting changes a poll, because instead of comparing two different pollsters, we're comparing one pollster who should use the same methods to find their respondents. Thus, the only variable in this comparison is whether respondents were prompted or not.

First, let's look at the undecideds. In that February prompted poll, undecideds were 11% of the population - a rather low number. In this unprompted poll, undecideds are 27%. Now, perhaps the last three months has made it harder for people to decide. Or, prompting makes otherwise undecideds more decisive.

But what about how it affects each party? Well, we can compare that, too. Obviously, we can't compare the results of that February poll to this May poll without any filter. The change in political climate would have a greater role in any voting intention changes than mere methodology.

So, what I have done is estimate the kind of changes the Environics poll should've shown, assuming they had been consistent with their methodology. This estimate was calculated by taking the EKOS, Angus-Reid, and Harris-Decima polls of late February and comparing them to the EKOS, Angus-Reid, and Harris-Decima polls of late May. We're comparing polls taken at the same time as the two Environics polls, using the change between the average of the three February polls and the three May polls as our guide for estimating how much the Environics poll should have changed.

Between February and May, Conservative average growth was 2.7 points (32.3% to 35.0%), or 8.4%. The Liberals have dropped 3.4 points, or 11.3%. The NDP has hardly changed (17.3% to 17.0%), as has the Bloc (8.3% to 8.7%). The Greens also haven't changed (10.3% to 10.3%).

In other words, if prompting plays no role in polling, then we should see the Conservatives up a little, the Liberals down a little, and the NDP, Bloc, and Greens virtually unchanged. Instead, we see the Conservatives higher than they should be, the Liberals, NDP, and Bloc stable, and the Greens way down.

For the Conservatives, they should have gone from 31% in February to 34% in May. Instead, they are at 36%. The Liberals should have gone from 30% to 27%, but instead they are still at 30%.

And, significantly, the Greens should have remained at 13%. Instead, they've dropped to 7% - almost half of their support in February.

What this shows is that prompting gives a boost to the Green Party more than any other, in conjunction with a lower undecided result. The Liberals and Conservatives don't get more support, but they do get a higher proportion of decided support. This indicates that, in a prompted poll, many undecideds choose the Greens as their "parking spot".

The question remains: which is accurate? History argues that Green results in prompted polls are inflated, and inaccurate. On voting day, the Greens have always gotten fewer votes than the pollsters thought they would.

But the Greens have defied history before.

However, until we finally see a Green electoral result matching their polling results, we will have to side with the historical record.