Thursday, April 14, 2011

Pick a poll, any poll

You might as well! The polls that were added to the projection this morning were, in some ways, contradictory. But in other ways they were not, and when we look at the results of the four of them with a closer eye as to why they are different and why they are the same, the polls don't seem as unusual as they might appear at first glance.

No matter, though. Many people will believe the poll they want to believe. And more power to them! There's no way to prove that one poll or another is wrong, especially with more than two weeks to go into the campaign. But I will always default to assuming every pollster takes a professional approach to their work, just as I do with mine. In other words, completely unsubstantiated claims that pollsters cook their numbers will never get any airing here.

So now that that's out of the way, let's get to it. The four polls today are from COMPAS conducted for the Toronto Sun, Innovative Research Group for Maclean's Magazine (and a myriad of radio stations), EKOS Research for iPolitics, and Nanos Research for, of course, CTV and The Globe and Mail. In accordance with the guidelines of Elections Canada (those well-meaning keeners), I should point out that all of these polls sampled eligible Canadian voters, and that Innovative Research uses an online panel.
First off, let's get to the 'controversial' COMPAS poll for the Toronto Sun. As I said yesterday, the biggest problem with this poll is that we have no previous poll from COMPAS for comparison. If we did, and there was a huge swing or no swing it all, we would be better able to judge the validity of their results. This is especially important as the poll is at odds with what others have found. But as far as I can tell, there is nothing in the methodology that easily explains this variation.

Or is there? Although it isn't exactly explained clearly (and my emails to the firm yesterday bounced), it appears that COMPAS's reported results are those of "decided" voters only, excluding "leaners". This explains the huge 35% undecided rate. Virtually every pollster reports on the combined decided and leaning results, and I'm not quite sure why COMPAS would do any different. There doesn't seem to be any compelling evidence that shows that casting aside the leaners brings us to a closer result - a decided voter is not, necessarily, a Canadian who is sure to vote.

Polling firms in the United States have done more work on this "likely voters" issue. We don't really have much to go on up here, and our voting systems are very different (i.e., registered Republican, Democratic, and Independent voters).

COMPAS reports that among decided voters, the Conservatives lead the Liberals by a margin of almost 2:1. This would seem to correspond with the 45% to 24% they report as Canadians' voting intentions. They then go on to say that among leaners the lead is only about 4:3. Well, that's pretty important, isn't it?

So let's try something. Assuming that 65% are decided and 20% are leaners (leaving about 15% undecided, which is what Nanos is reporting), and that the 4:3 lead is the sort of 40% to 30% lead Nanos has found, we narrow down the gap from 21 points to 19, or about 44% to 25%. With rounding (and the fact that I am making a few assumptions), we could even say that the gap is 43% to 26%, or 17 points.

That doesn't look so strange anymore. It is generally where Ipsos Reid has placed the two parties, and is on the outside edges of the margins of error of other recent polls. And when you consider that COMPAS's decided numbers are almost identifical to Ipsos Reid's "absolutely certain to vote" numbers, this poll doesn't look so strange at all.

But the method of reporting the results, and the omission of what questions were asked, is certainly unusual. I doubt that this method of reporting results would meet the MRIA's standards, though more than a few pollsters are truant on that score.

Note that, in the graph above, the regional margins of error have been calculated based on my best estimates of this poll's sample size, as regional breakdowns were not included.

COMPAS's results in Ontario are especially stark, with the Conservatives leading by 49% to only 32% for the Liberals. That would give the Tories 69 seats, with only 27 going to the Liberals and 10 to the New Democrats. But in the Nanos poll, which shows the Liberals leading by 4.4 points, the seat breakdown would be 46 for the Conservatives, 44 for the Liberals, and 16 for the NDP. These two results would create radically different parliaments.

Innovative Research's poll doesn't have anything particularly remarkable in it, and in fact it closely matches my own projection. It does put Michael Ignatieff ahead of Jack Layton on who would make the best Prime Minister, which is new. We last heard from Innovative in a report on Ontario's voting intentions only, taken between April 1 and 3. Since then, the Tories are up three points in the province, which is statistically insignificant.

EKOS's poll, however, is more interesting. It was taken over two days, and those two days will be included in EKOS's weekly report tomorrow for iPolitics. When that poll is released, it will replace this two-day poll in the projection model.

This poll shows a much smaller gap than even Nanos shows: five points. Compared to EKOS's last two-day sneak peek taken between April 4 and 5, the Conservatives have dropped 2.2 points and the Liberals have gained one point. That isn't significant, but the three point gain of the NDP certainly is interesting. Also, note that EKOS has the Liberals in front in the Toronto CMA.

Finally, Nanos. They show a narrowing gap between the Conservatives and Liberals that now stands at 7.8 points, less than it did in 2008. Since their last complete three-day poll, conducted between April 8 and 10, the Conservatives have dropped 2.3 points while the Liberals have gained 0.7. Like EKOS, this isn't significant movement but again the NDP is up, with a 3.1 point gain in three days.

Much of that has come in British Columbia (+9.4) and Ontario (+4.8). In the latter province, the Liberals have gained 5.3 points while the Tories have lost 6.1 points since that last three day poll. Movement definitely seems to be taking place in the battleground province.

In terms of trends, in Atlantic Canada the Conservatives have gained for three consecutive days, as have the Liberals in Ontario and the New Democrats in British Columbia. The Conservatives, however, have lost support for three consecutive days in those two important provinces.

The New Democrats are between 16% and 21% in Quebec in all four of these polls, providing further confirmation of their newfound strength in the province. However, they are also below 20% in Atlantic Canada across the board. That isn't great news for the NDP, but their four MPs in the region are relatively ensconced.

The questions asked by each of the pollsters are as follows: "For those parties you would consider voting federally, could you please rank your top two current local preferences?" (Nanos). "If a federal election were held today, which party would you vote for?" (Innovative). "If a federal election were held tomorrow, which party would you vote for?" (EKOS). The question asked by COMPAS is not available.