PQ rides split to large majority

The last poll from CROP, conducted between May 11 and 16, shows that both the Parti Québécois and the Liberals have lost support since April. But despite being at 34%, the PQ is still in a position to win a large majority of the seats in the National Assembly.

Compared to CROP's last poll taken between April 13 and 20, in the midst of the federal campaign, the Parti Québécois has lost six points but still holds an 11-point lead over the Liberals, with 34% to 23%. That represents a three point loss for the PLQ.

The Action Démocratique du Québec is up three points to 16%, a gain we also saw in Léger's last missive, while Québec Solidaire is up one point to 12%.

This is the second poll showing that the NDP's gains in Quebec have not transferred over to QS.

The Greens are up six points to 11% in this poll, a likely fluke considering that they are now at 25% among non-francophones. Only 69 non-francophones were polled, so the margin of error on this result is rather large.

Both this poll and the recent Léger poll have shown both the PQ and Liberals looking weak. However, neither of these polls have shown that any of the second or third tier parties are taking advantage to any great extent.

Among francophones, who make up the bulk of the population, the PQ is leading with 41%, followed by the Liberals at 18% and the ADQ at 15%.

In and around Montreal, the PQ is ahead of the Liberals with 32% to 23%. Québec Solidaire is running third, with 14%, while in Quebec City the PQ is also ahead with 31%. The ADQ is second with 25% and the Liberals third with 22%.


With the results of this poll only, ThreeHundredEight projects 78 seats for the Parti Québécois, 27 for the Liberals, 15 for the ADQ, and five for Québec Solidaire.

In the past, I would have looked at five seats for QS with skepticism, but the federal campaign in Quebec has shown that elections results are not pre-ordained.

This would be a very large majority for the PQ. Though sovereignty is not a top issue in Quebec at the moment, the option still has the support of 43% of decided voters according to this poll. With the support of Québec Solidaire, also a sovereigntist party, it is quite possible that with this PQ electoral victory a referendum would be on the horizon.

But Quebecers are still unenthusiastic about the prospect of Pauline Marois being premier. She received only 20% support on who is best suited for the job, though that is increased to 38% when we remove the "unsures" and "none of the aboves". Jean Charest follows with 16% or 30% of decideds, while Gérard Deltell comes in third with 10% (19%).

CROP still lists Guy Rainville as the leader of the Greens, despite Claude Sabourin being made leader in November.

Quebec is certainly making the headlines lately, be it because of the NDP's position on the Clarity Act or the province's opposition to the Conservatives' planned reform of the Senate. But we may be as much as two years from the next election, so there is plenty of time for Charest to turn the tide or for Marois to either run away with or scuttle her lead

The new Tory constituency: Far less francophone, far more multicultural

Stephen Harper finally won his majority government, but in the process lost more than half of his party’s seats in Quebec. Gains in the rest of the country made up for these losses, however, and with the Conservatives winning a swathe of new seats in the Greater Toronto Area the ridings represented by Tory MPs has become far less francophone and much more diverse.

You can read the rest of the article on The Globe and Mail website. A condensed version of the article with infographic is also in today's print edition.


As I did with the NDP and the Liberals, here are a few extra tidbits on the new Conservative constituency:

Largest population: Brampton West (170,000 in 2006)
Smallest population: Labrador (26,000)
Oldest median age: Chilliwack - Fraser Canyon (50.6 years)
Youngest median age: Nunavut (23.1 years)
Highest median household income: Halton ($93,000)
Lowest median household income: Dauphin - Swan River - Marquette ($34,000)
Ridings in which French is the largest language group: 7
Ridings in which neither English nor French is the largest language group: 14
Ridings in which immigrants are the majority: 12
Riding with the largest aboriginal population: Desnethé - Missinippi - Churchill River (45,000)

Two maps, one election

Something a little different this morning. I was contacted this week by Sean, the blogger over at the Arbitrary Gopher and a scientist specializing in Human Genetics from Quebec. He put together a very interesting cartogram for the 2011 federal election. It was so interesting that I asked if he would write a guest post for ThreeHundredEight.
---

As a scientist I spend most of my time generating, looking at and graphing data. 

An election is also data. Campaign contributions, vote counts and seat distribution are all data. So how do we graph them? Well, most of the time election results are projected onto a map. A map is indeed a way of communicating data; data regarding distances and areas. Maps are not a very good graph for election results, however, particularly in Canada. 

A graph of an election result should tell us who wins the most seats. So, why do we graph these results on a map? A better graph would be to use a cartogram, which is a map that is transformed to show something other than geography. This cartogram includes all of Canada's current electoral boundaries, but then each riding is scaled according to the size of its population (based on the 2006 census). While it might look strange at first, most of us should be able to find our homes in it pretty quickly.

I think the main difference that this cartogram brings to the table here is in terms of the Liberals and the Greens. While in the first map we see the twin seas of Conservative Blue and NDP Orange, we don't see that the Liberal Party does in fact still exist in the highly populated urban areas of Canada, and we also don't see that the Greens now have a toe-hold out west. Moreover, it's easy to forget looking at the map just how vote and seat rich both southern Ontario and Quebec truly are. Finally, if you know some basic Canadian geography, it's pretty easy to see the results in all of the major Canadian cities. At a glance we can see who won this election, where they won it and by about how much. Fun graph right?

I will be graphing several of the amazing and publicly available national datasets as well as helping to make sense of provincial and even municipal elections in the near future. You can follow new results over here at arbitrarygopher.blogspot.com.

---

Thanks, Sean! As he says, the Liberal Party in the original map is virtually non-existent. You don't see that they won a good deal of their seats in the Toronto and Montreal regions, but in the cartogram you do in fact see the Liberal seats in Montreal and Toronto. And out West, you can see the NDP and Liberal toeholds in Edmonton, Regina, and Winnipeg that you can't really make out on the original map.

Another point worth making is just how distorted the map looks. A similar cartogram for the United States (see here), while also distorted, is at least recognizeable. With Canada's population so concentrated in the thin band close to the American border, we get the cartogram resembling a drop of water. Toronto, Montreal, and Vancouver take up a huge proportion of the cartogram, whereas on the geographic map they are tiny. It's a fascinating way to look at the country.

Relative stability in Ontario, PCs still lead

A new poll by Nanos Research reported Tuesday by The Globe and Mail shows that the Progressive Conservatives still hold a significant lead over Dalton McGuinty's Liberals, though that lead has narrowed.

Compared to Nanos's last poll in March, the Progressive Conservatives are down three points but still lead with 41% support. The Liberals are down one point to 34%. This means the lead has gone from nine to seven points. The New Democrats are up three points to 19%, while the Greens are up one to 5%.

With the small sample size of 503 Ontarians, none of these changes in support are statistically significant. And considering that the New Democrats had 17% support in the 2007 election, it would appear that the federal campaign has had little impact on the provincial NDP's fortunes in Ontario.

With this poll, ThreeHundredEight projects the Progressive Conservatives win 58 seats and form a majority government. That is a gain of 33 seats over their current standing in the Legislative Assembly, but a drop of four seats since the last projection.

The Liberals win 34 seats, a loss of 38 overall but unchanged from the March projection.

The New Democrats pick-up four seats since March and five over their current crop of MPPs, and would win 15 seats.

Nanos looked into the leadership question, and found some contradictory results. In terms of who would make the best premier, Tim Hudak of the Progressive Conservatives topped the list with 31%, up four points since March.

Current Liberal Premier Dalton McGuinty had 25%, up three points, while Andrea Horwath of the NDP jumped six points to 16%. That is a marginally significant real increase for the NDP leader.

But when we remove the "none of them" or "unsures" (that latter group dropped by 11 points), we get Hudak at 42%, McGuinty at 34%, and Horwath at 22%. That is generally where the parties stand, so these leadership numbers would appear to be relatively unimportant.

During the federal campaign, Nanos had the "Leadership Index Score", combining their results on questions of trust, competence, and vision for Canada. Applying that to Ontario gives McGuinty the leg up with a combined score of 85, compared to 71 for Hudak and 37 for Horwath. McGuinty and Hudak were generally at the same level on trust and vision, but McGuinty is considered far more competent.

It would appear that though more Ontarians consider McGuinty to be good at his job, people just don't like him very much. These numbers seem to fit in with the general perception of the Premier that can be sensed in Ontario.

Though a lead is a lead, it isn't the kind of lead a relatively unknown Hudak can take to the bank. Consider that a May 2007 poll by Environics, taken five months before the vote in that year's election, had John Tory's Progressive Conservatives ahead of the Liberals by 38% to 33%. Can McGuinty turn it around again?

The new NDP constituency: Far more francophone, far less multicultural

When New Democrats more than doubled their historic best on election night, the constituency Jack Layton’s party represents was radically transformed. They broke through in Quebec and won 59 seats, more than any single party has taken in the province since Brian Mulroney in 1988 – before the birth of the Bloc Québécois. Accordingly, the NDP constituency has gone from being overwhelmingly English-speaking and more diverse than the national average to mostly French-speaking and less multicultural.

You can read the rest of the article on The Globe and Mail website. There is also a condensed version of the article in the print edition of the newspaper.


Like last week's analysis of the Liberal constituency, this one looks at the New Democrats and their ridings' average demographic profile. Like last week, here are a few extra factoids for the party:

Largest population: Nanaimo - Cowichan (125,000 in 2006)
Smallest population: Western Arctic (41,000)
Oldest median age: British Columbia Southern Interior (47.2 years)
Youngest median age: Churchill (26.4 years)
Highest median household income: Western Arctic ($80,000)
Lowest median household income: Winnipeg Centre ($32,000)
Ridings in which French is the largest language group: 59
Ridings in which neither English nor French is the largest language group: 8
Ridings in which immigrants are the majority: 4
Riding with the largest aboriginal population: Churchill (52,000)

Note that, despite having three times as many seats, the NDP hold as many seats where neither French nor English is the largest language group as the Liberals do. They also hold one fewer seat with an immigrant majority as the Liberals.

Next week, I'll be looking at the Conservatives.

BC Liberals and NDP neck-and-neck, but advantage Clark

A new poll from Ipsos-Reid on the political situation in British Columbia shows that the BC Liberals and BC New Democrats are virtually tied, but if we look deeper into the bowels of the poll we can see that new Premier Christy Clark has the leg-up on her NDP opponent.

This Ipsos-Reid poll used an interesting methodology. While 1,200 British Columbians were polled in all, half were contacted via telephone and half via Ipsos's online panel. It's a sort of hybrid poll, and I wonder if this is something that we might see more of in the future. It could bring together the best of both methods.

According to the poll's findings, the BC Liberals are leading with 41%, closely trailed by the BC New Democrats who are at 39%.

The BC Conservatives come up in third with 10%, with the Greens in fourth at 8%. That is a good result for the Conservatives, and perhaps is an indication of the effect of having John Cummins, former BC MP for the federal Conservatives, as their future leader. His recent comments on homosexuality certainly won't help him increase his party's base, however.

The BC Liberals are leading in the Lower Mainland, the Southern Interior, and the North, while the NDP is ahead on Vancouver Island. With 19%, the Conservatives had their best result in the North, while the best Green result was in the Southern Interior.

Note: An earlier version of this post had mistakenly given the Green Party the same regional support levels as the BC Conservatives. Apologies.

It is a slim, statistically insignificant lead for Mrs. Clark, but a few other questions asked by Ipsos-Reid show why the BC Liberals are in a better position. Asked how likely they were to vote for each party, 48% of respondents said they were somewhat or very likely to vote for the BC Liberals, indicating the party still has some room for growth.

On the other hand, the BC New Democrats are nearer to their ceiling: 43% said they were somewhat or very likely to vote for them.

There is also some potential for growth for the Greens and Conservatives, who had a "likely" response of 25% and 24%, respectively.

In terms of who would make the best premier, 47% said Christy Clark. That makes her more popular than her own party. Adrian Dix of the BC NDP, on the other hand, garnered only 25% on this question. That is not a good sign for the NDP.

Technically, an election is not scheduled until 2013. That would give the NDP plenty of time to solidify their support. But rumour is that Clark may call an election this year, though her recent squeaker of a by-election win may cause her to be more cautious. British Columbians, however, seem game for an election: 46% said that a new one should be held this year, and this in a poll held only one week after the federal election.

The BC Liberal vote being more efficient, ThreeHundredEight projects that with this poll Christy Clark would win 47 seats and hold on to her majority government. That is a drop of two seats from what her party currently holds in the legislature, those gains going to Adrian Dix and the NDP, which would win 37 seats and be returned to the opposition benches.

Nevertheless, this is a very close margin for the BC Liberals. And considering what happened in the federal campaign, this is not the kind of lead that can be taken to the bank. The BC Liberals have a majority government and while it appears that Clark would be more likely than not to win a new mandate on her own, it is most definitely not a sure bet. Whether British Columbians head to the polls for the second time this year may depend on how much of a gambler Mrs. Clark is.

Canadian politics transformed overnight

I was not going to post today. I was instead going to update the chart showing the monthly federal polling averages without any fanfare or mention. March and April needed to be added, and the election results of May 2nd were to take the May slot. I will continue to keep track of federal poll results in this manner, but after inputting the information the image was simply too remarkable to put up on the site without comment.

Below is the average of all federal polls released during each month, stretching back to January 2009. Every poll released is given an equal weighting, so this is a simple average. For May, I've used the election results. And just look at the spikes.

You can click on the image for a magnified view, and it will be stored in the right-hand column and updated with every passing month.

At the national level, things were pretty stable stretching back to January 2010. The Conservatives widened their lead over the Liberals in February and March of this year, and ended up on election night higher than any monthly average of polls has had them. The Liberals sank quickly to 19%, while the New Democrats spiked even more quickly. They had been stagnant at between 15% and 17% for more than two years, but in a period of a few weeks the party went from 18% or so to 31% on election night. Indeed, the April average is really only the half-way point between being in the high-teens at the beginning of the month and the high-20s at the end of it. You can clearly see the Bloc Québécois and Greens falling away as well.

In British Columbia, Alberta and the Prairies, it was much the same story. The Conservatives hit a high on election night, as did the NDP, at the expense of the Liberals.

In Ontario, the 44% of the Conservatives was the culmination of almost a year's worth of growth. Back in the summer of 2010, the Conservatives were around 35% and 36%. The Liberals had been holding pretty steady in the province for more than a year, but on election night their vote collapsed and the New Democrats rose to a historic high. Here again, we see the Greens trailing off.

But it is in Quebec that the NDP gain is most striking. There, the Bloc Québécois had been rolling along nicely between 36% and 41% since February 2009. Out of nowhere, the party drops to 31% in April and bottoms out at 23%, while the New Democrats sky-rocket from an average of 13% in February to 43% on election night. It is simply remarkable - the NDP blew by the Liberals and Conservatives and kept going well past the Bloc.

The Quebec chart also shows the problems of the Liberals here. They were in the low-30s back in early 2009, but had dropped to the low 20s a year later. That decline continued until they ended up at 14% on election night. The Conservatives held firm in the province, though they were still below their 2008 performance and their high of 21% between October and November of 2009.

The shift in Atlantic Canada is not as striking, as the election result generally echoed something we had seen before in December 2009. Still, the Conservatives and New Democrats hit a high while the Liberals hit a low.

How things changed in Canadian politics, almost overnight, is starkly laid out in this chart. The next election is over four years away, but I have to admit I am very curious to see how this chart will look even a year from now. Hopefully the polling firms will report now and then, despite this majority climate.

ADQ gains, PQ slips

A new Léger Marketing poll released over the weekend gives us an indication of how the federal campaign has influenced provincial politics in Quebec. In short, the Parti Québécois appears to have suffered slightly in conjunction with the disastrous federal results for the Bloc Québécois, but neither support for sovereignty nor Quebec's left-wing party, Québec Solidaire, has shifted.

Compared to Léger's last provincial poll taken during the campaign between April 11-14, the Parti Québécois has dropped four points to 34%. That is below the level of support they had in 2008, but the 5% opting for the "Other" option are likely skewing the results.

Between now and the election, probably in 2012, it is unclear whether François Legault will form a new party. He has already more or less refused to join with the ADQ. And if he does form a new party, he is likely to get more than the 5% that the Others are getting in this poll. So it is difficult to know what can be done with this particular result.

Nevertheless, the PQ's 34% is still well above the 23% the Bloc received on May 2nd. And the Liberals, the main federalist party, have not benefitted from the campaign. They've dropped one point to 30%.

Instead, the ADQ (a party quite different from the NDP, to say the least) has made gains, going from 14% to 18%, while Québec Solidaire is unchanged at 9%. Considering that QS should be the natural beneficiary of the NDP's rise in the province, it is quite surprising to see that QS has not moved an inch.

Similarly, support for sovereignty is virtually unchanged. It now stands at 41%, down a statistically insignificant two points since the April poll. That is virtually the same result as the 1980 referendum and where support stood just before the referendum campaign of 1995. As some other polls on why Quebecers voted they way they did have shown, it wasn't because they were turning away from the idea of sovereignty. Despite the downfall of the Bloc, the option is not quite dead yet.

Regionally, there haven't been significant changes. It is interesting, however, that the ADQ has seen its support grow in Montreal and the rest of Quebec while dropping in Quebec City.

But the key shift in this poll is in who Quebecers think would make the best Premier. After removing the undecideds/non-response, Jean Charest tops the list with 36% support, a gain of eight points since mid-April. He is now more popular than his party - incredibly. Many people have thought that Charest's nine lives had run out, but it appears they haven't. Pauline Marois stands at 34%, a drop of eight points, while Amir Khadir and Gérard Deltell are each at 15%. But for Khadir that makes him more popular than his party, while Deltell is less.

With the results of this poll, ThreeHundredEight projects the Parti Québécois would form a majority government with 69 seats. The Liberals win 40, the ADQ 14, and Québec Solidaire two.

This is a slightly better situation for the would-be opposition parties than was the case before the federal election, when the Parti Québécois was in the 75-85 seat range.

There has been much talk about Quebecers' want of change, but at least in this poll it appears that this want did not translate to the provincial level. While this is a low level of support for the Parti Québécois, it is generally where we have seen them since the 2008 election. We will need to see some more polls for confirmation that they are on track to lose support since that last election.

But the question of leadership was a major factor in Quebec in the federal campaign. Still, despite besting Marois by a few points, Charest is unpopular. Jack Layton, on the other hand, is liked by Quebecers. That can motivate people to go out and vote, whereas choosing the least bad option in a public opinion poll may not be an indication that Charest-fever is sweeping the province. Just a hunch.

The new Liberal constituency: less wealthy, less diverse and very East Coast

Two weeks ago, the Liberal Party of Canada had its representation in the House of Commons reduced by more than half. The Grits did not gain any new seats but only managed to hold on to 34 of the 77 ridings they held when the election was called. But in addition to being culled to less than half its size, the constituency that the Liberal Party now represents has also changed. It is poorer, less ethnically diverse, and more concentrated on the Atlantic coast.

You can read the rest of the article on The Globe and Mail website. There is a condensed version of the article in today's print edition, as well.

This piece is an analysis of the demographic profile of all ridings held by the Liberals before and after the election. I will be looking at the New Democrats next week and the Conservatives the week after that. It's based on the 2006 Census, which can be found here.

Here are a few highs and lows for the 34 Liberal ridings:

Largest population: Markham - Unionville (127,000)
Smallest population: Charlottetown (32,000)
Oldest median age: Cape Breton - Canso (45.5 years)
Youngest median age: Etobicoke North (34.8 years)
Highest median household income: Markham - Unionville ($78,500)
Lowest median household income: Papineau ($32,500)
Ridings in which French is the largest language group: 2
Ridings in which neither English nor French is the largest language group: 8
Ridings in which immigrants are the majority: 5
Riding with the largest aboriginal population: Winnipeg North (15,150)

The difference between the New Democratic constituency before and after the election should be remarkable, as the party has gone from having 3% of their seats in Quebec to 57%.

ThreeHundredEight's Track Record

The following is a full record of how ThreeHundredEight has projected elections since 2008. As more elections take place, this list will be expanded. Note that in recording error, I have included any party that was projected to win and/or actually won one seat or at least 3% of the popular vote.

2013 BRITISH COLUMBIA PROVINCIAL ELECTION
Final Projection
Projection vs. Results Analysis

Christy Clark's Liberals were re-elected for the third consecutive time, defeating Adrian Dix's New Democrats, who formed the Official Opposition. One Green MLA was elected for the first time in the province's history, while one independent MLA was re-elected.

Vote projection: 46% New Democrats, 37.7% Liberals, 7.8% Greens, 5.3% Conservatives
Vote result: 44.4% Liberals, 39.5% New Democrats, 8% Greens, 4.8% Conservatives
Error: 13.9 points, or 3.5% per party

As in the Alberta election, the polls showed the opposite of what actually occurred. No survey during the entirety of the B.C. gave the Liberals the lead, while most showed a gap of some 4-9 points in favour of the NDP. With such a high degree of error, the projection model was unable to cope. The projected ranges (44.1%-47.9% for the NDP, 35.8%-39.6% for the Liberals) were also off, though the wider forecasted ranges did capture the Liberal surge.

Seat projection: 49 New Democrats, 35 Liberals, 1 Independent
Seat result: 50 Liberals, 33 New Democrats, 1 Green, 1 Independent
Error: 32 seats, or 8 per party.
Riding projection accuracy: 81.2%
Projected seat range: 44-55 New Democrats, 26-41 Liberals, 0-4 Independents
Seat range error: 21 seats, or 5.3 per party 
Winners called in projected ranges: 85.9%

With the polls having diverged so much from the result, the seat projection model was very wrong. Nevertheless, it still only missed 16 ridings and did not identify the potential winner in 12. This was a much better performance than the Alberta election, where the overall winner was also incorrectly identified. The election was decided in fewer seats.

Seat projection with actual regional vote:  49 Liberals, 36 New Democrats
Error: 6 seats, or 1.5 per party.
Riding projection accuracy: 89.4%
Projected seat range with actual regional vote:  37-57 Liberals, 27-46 New Democrats, 0-2 Independents, 0-1 Greens.
Seat range error: 0 seats, or 0 per party
Winners called in projected ranges: 95.3%

The projection model would have performed very well if the polls had been accurate. It would have only been off by six seats, and would have correctly called the winner in 76 of 85 ridings, identifying the potential winner in 81 of 85. A Green seat would have also been predicted in the ranges. The seat projection model has shown, time and time again, to be an effective way of turning regional vote results into accurate seat numbers. Unfortunately, polling has been missing more often.

2012 QUEBEC PROVINCIAL ELECTION
Final Projection
Projection vs. Results Analysis

Pauline Marois and the Parti Québécois defeated the Liberals under Jean Charest, winning a slim minority government. Charest was defeated in his riding and resigned as leader. François Legault's Coalition Avenir Québec finished third, while Québec Solidaire elected both of its co-leaders to the National Assembly.

Vote projection: 34.1% Parti Québécois, 27.9% Liberals, 26.3% CAQ, 7.1% Québec Solidaire
Vote result: 31.9% Parti Québécois, 31.2% Liberals, 27.1% CAQ, 6.0% Québec Solidaire
Error: 7.4 points, or 1.9% per party

The polls did an adequate job judging the support of the PQ, CAQ, and QS, but under-estimated the Liberals by a significant degree. This was the important "miss" of the campaign, as it made the difference between a PQ majority/minority and a strong/poor finish for the Liberals. Support for the PLQ among francophone Quebecers was particularly underscored. An adjustment made to the model to boost support for the PQ and PLQ and reduce that of the CAQ and QS made the right call on the PLQ and QS, less so for the CAQ, and not for PQ.

Seat projection: 63 Parti Québécois, 33 Liberals, 27 CAQ, 2 Québec Solidaire
Seat result: 54 Parti Québécois, 50 Liberals, 19 CAQ, 2 Québec Solidaire
Error: 34 seats, or 8.5 per party.
Riding projection accuracy: 81.6%
Projected seat range: 57-75 Parti Québécois, 25-39 Liberals, 20-31 CAQ, 1-2 Québec Solidaire
Seat range error: 15 seats, or 3.8 per party 
Winners called in projected ranges: 85.6%
Ridings with all parties' results within projected range: 5.6%

Though the projection model correctly called the PQ as the potentially-minority winner and the Liberals as the Official Opposition, it could not anticipate the very strong showing of the Liberals. Despite the relatively high seat count error, the model's riding projection accuracy was somewhat better. This was not a question of errors cancelling each other out. The uncertainty model identified the correct winner in all but 18 ridings.

Seat projection with actual regional vote:  53 Parti Québécois, 42 Liberals, 28 CAQ, 2 Québec Solidaire
Error: 18 seats, or 4.5 per party.
Riding projection accuracy: 81.6%
Projected seat range with actual regional vote: 41-65 Parti Québécois, 34-55 Liberals, 18-38 CAQ, 1-2 Québec Solidaire
Seat range error: 0 seats, or 0 per party
Winners called in projected ranges: 94.4%

With the actual vote results, the model would have done much better. It would have called a weak minority government but would have still under-estimated Liberal support to the benefit of the CAQ. Liberal incumbents were very resilient, and CAQ support was far more uniform than the support of its predecessor, the ADQ. But the seat ranges would have encompassed the eventual result, so the model would have been able to anticipate the outcome. It would have also identified the potential winner in all but seven ridings, a strong performance.

2012 ALBERTA PROVINCIAL ELECTION
Final Projection
Projection vs. Results Analysis

The Progressive Conservatives under Alison Redford were re-elected to another majority government, with Danielle Smith's Wildrose becoming the Official Opposition. Raj Sherman's Liberals placed third, while Brian Mason's NDP placed fourth.

Vote projection: 38.4% Wildrose, 35.8% Progressive Conservatives, 11.4% New Democrats, 11.1% Liberals
Vote result: 44.0% Progressive Conservatives, 34.3% Wildrose, 9.9% Liberals, 9.8% New Democrats
Error: 15.1 points, or 3.8% per party

Every public poll released in the last week of the campaign gave Wildrose a lead, at times a sizable one. One final poll did show the race tightening, but it appears that in the final 48 hours of the campaign the support for Wildrose swung hard to the PCs, giving them a majority. The story after the election quickly became about how the polls "missed" the call.

Seat projection: 43 Wildrose, 39 Progressive Conservatives, 5 New Democrats
Seat result: 61 Progressive Conservatives, 17 Wildrose, 5 Liberals, 4 New Democrats
Error: 54 seats, or 13.5 per party.
Riding projection accuracy: 58.6%
Projected seat range: 22-62 Wildrose, 20-62 Progressive Conservatives, 3-8 New Democrats, 0-3 Liberals
Seat range error: 7 seats, or 1.8 per party 
Winners called in projected ranges: 77.0%
Ridings with all parties' results within projected range: 3.4%
Average margin of error on riding projections: +/- 5.5%

With the available polling data off by such a significant degree, the seat projection model could not call the election. It did manage to encompass most of the results in the ranges, but even this was off for the Wildrose and Liberals. There was really no way to call this election accurately with the information at hand. The uncertainty model as reflected by the ranges, however, proved worthwhile in its first use as the potential winner was identified in 77% of ridings.

Seat projection with actual regional vote: 65 Progressive Conservatives, 18 Wildrose, 4 New Democrats
Error: 10 seats, or 2.5 per party.
Riding projection accuracy: 71.3%
Projected seat range with actual regional vote: 47-71 Progressive Conservatives, 14-35 Wildrose, 2-5 New Democrats, 0-1 Liberals
Seat range error: 4 seats, or 1.0 per party
Winners called in projected ranges: 78.2% 

With the actual vote results, the model would have closely forecast the results for the Progressive Conservatives and Wildrose, and would have been dead-on for the New Democrats. The Liberals, however, would have still been projected to win no seats and their ranges would have actually been further from the mark. The party's concentration of support in certain ridings in Edmonton and Calgary was remarkable.

2011 SASKATCHEWAN PROVINCIAL ELECTION
Final Projection
Projection vs. Results Analysis

The Saskatchewan Party under Brad Wall increased its majority and won its second consecutive election. The New Democratic opposition under Dwain Lingenfelter was reduced and Lingenfelter resigned as leader of the party after being defeated in his own riding.

Vote projection: 62.4% Saskatchewan Party, 33.6% New Democrats, 3.1% Greens
Vote result: 64.2% Saskatchewan Party, 32.0% New Democrats, 2.9% Greens
Error: 3.6 points, or 1.2% per party

Though a smattering of polls came out a week before the election, only one was released in the final days. But overall, the polls did well and so did the vote projection.

Seat projection: 43 Saskatchewan Party, 15 New Democrats
Seat result: 49 Saskatchewan Party, 9 New Democrats
Error: 12 seats, or 6.0 per party.
Riding projection accuracy: 89.7%
Projected seat range: 40-44 Saskatchewan Party, 14-18 New Democrats
Seat range error: 10 seats, or 5.0 per party
Ridings with all parties at or within 5% of result: 67.2%
Average margin of error on riding projections: +/- 2.8%

The seat projection had a good riding accuracy rating of almost 9 out of every 10 ridings, but they were all wrong in the same way making the error per party relatively high. A regional projection of 47 SP seats to 11 for the NDP made before the election would have been a better result, further indicating the need for a regional projection model.

Seat projection with actual provincial vote: 43 Saskatchewan Party, 15 New Democrats
Error: 12 seats, or 6.0 per party.
Riding projection accuracy: 89.7%
Projected seat range with actual provincial vote: 43-48 Saskatchewan Party, 10-15 New Democrats
Seat range error: 2 seats, 1.0 per party

With the actual vote results, the model still would have given the Saskatchewan Party 43 seats and the NDP 15, but the seat ranges would have been much better, being off by only one seat per party.

2011 NEWFOUNDLAND AND LABRADOR PROVINCIAL ELECTION
Final Projection
Projection vs. Results Analysis

The Progressive Conservatives under Kathy Dunderdale won their third consecutive majority government, and their first with Dunderdale as leader. While the New Democrats under Lorraine Michael placed second in the popular vote, they were beaten out by one seat on the Official Opposition, formed by Kevin Aylward's Liberals. However, Aylward was defeated in his own riding.

Vote projection: 55.2% Progressive Conservatives, 30.5% New Democrats, 14.1% Liberals
Vote result: 56.1% Progressive Conservatives, 24.6% New Democrats, 19.1% Liberals
Error: 11.8 points, or 3.9% per party

There were no polls in the last seven days of the campaign, making this one more difficult to gauge. It would appear that in those seven days about 1/6th of NDP supporters switched over to the Liberals. Or, the NDP vote simply didn't turnout.

Seat projection: 42 Progressive Conservatives, 4 New Democrats, 2 Liberals
Seat result: 37 Progressive Conservatives, 6 Liberals, 5 New Democrats
Error: 10 seats, or 3.3 per party.
Riding projection accuracy: 77.1%
Projected seat range: 40-44 Progressive Conservatives, 3-5 New Democrats, 1-3 Liberals
Seat range error: 6 seats, or 2.0 per party
Ridings with all parties at or within 5% of result: 14.6%
Average margin of error on riding projections: +/- 10.2%

Generally speaking, an error of 3.3 seats per party is not bad at all. But when that means the order of the parties is off, that is a problem. And when it also results in a 77.1% accuracy rating, that is a big problem. The seat projection model did not do very well in this election, though the seat ranges were much better. But the ability of the Liberals to win new seats while incumbents lost was abnormal.

Seat projection with actual provincial vote: 41 Progressive Conservatives, 4 Liberals, 3 New Democrats
Error: 8 seats, or 2.7 per party
Riding projection accuracy: 75.0%
Projected seat range with actual provincial vote: 35-43 Progressive Conservatives, 2-9 Liberals, 3-4 New Democrats
Seat range error: 1 seat, 0.3 per party

While the actual results would have given the correct order of the parties, the riding accuracy would have actually dropped. The need of a regional model was reiterated in this election, as the Liberals won all of their seats in western Newfoundland and in Labrador, while the NDP won all but one of their seats in St. John's. But Newfoundland and Labrador demonstrated again that smaller provinces are much more difficult to project. Newfoundland stumped me - this time.

2011 ONTARIO PROVINCIAL ELECTION
Final Projection
Projection vs. Results Analysis

The Liberals under Dalton McGuinty won a historic third term, but fell just short of securing a majority government. With increases in support and seats for both the Progressive Conservatives under Tim Hudak and the New Democrats under Andrea Horwath, both leaders stayed on to sit in the minority legislature.

Vote projection: 36.6% Liberals, 33.3% Progressive Conservatives, 24.7% New Democrats, 4.1% Greens
Vote result: 37.6% Liberals, 35.4% Progressive Conservatives, 22.7% New Democrats, 2.9% Greens
Error: 6.3 points, or 1.6% per party.

As has become a theme, the polls (and so the projection) under-estimated Tory support and over-estimated New Democratic support. But generally speaking, the vote projection did not perform badly. The projection performed better than four of the seven polling firms that reported in the last week.

Seat projection: 58 Liberals, 29 Progressive Conservatives, 20 New Democrats
Seat result: 53 Liberals, 37 Progressive Conservatives, 17 New Democrats
Error: 16 seats, or 5.3 per party.
Riding projection accuracy: 85.0%
Projected seat range: 51-62 Liberals, 25-35 Progressive Conservatives, 19-22 New Democrats
Seat range error: 4 seats, or 1.3 per party.
Ridings with all parties at or within 5% of result: 44.9%
Average margin of error on riding projections: +/- 3.3%

A discrepancy of five seats for the winning party would, normally, not be a bad performance. But with the election as close as it was, that was just enough to make the call of a majority government wrong. The seat projection model was unable to take into account some of the major regional variations of this election, particularly in southwestern Ontario. The model performed very well in three of the seven regions of the province and acceptably in two others, but only hit 67% accuracy in southwestern Ontario. The seat ranges were more accurate, however, and were only off by two seats for the Tories and NDP. A Liberal minority was envisioned by the ranges.

Seat projection with actual provincial vote: 58 Liberals, 30 Progressive Conservatives, 19 New Democrats
Error: 14 seats, or 4.7 per party.
Riding projection accuracy: 85.0%
Projected seat range with actual provincial vote: 49-60 Liberals, 28-39 Progressive Conservatives, 18-21 New Democrats
Seat range error: 1 seats, 0.3 per party

In this case, the polls were not entirely to blame. I would have still projected a majority Liberal government, but my seat ranges would have been much more accurate. Two problems contributed to the difficulty of this election: regional variations beyond what province-wide trends could have anticipated, and low turnout.

2011 MANITOBA PROVINCIAL ELECTION
Final Projection
Projection vs. Results Analysis

The New Democrats won their fourth consecutive majority, but their first election under Greg Selinger. Though the popular vote was close, the Progressive Conservatives were soundly defeated in the seat count and leader Hugh McFadyen resigned.

Vote projection: 45.6% New Democrats, 42.9% Progressive Conservatives, 8.0% Liberals, 3.1% Greens
Vote result: 46.0% New Democrats, 43.9% Progressive Conservatives, 7.5% Liberals, 2.5% Greens
Error: 2.5 points, or 0.6% per party

With the last two polls forecasting the result almost exactly, the vote projection model worked wonderfully. The only real problem was a slight over-estimation of Liberal and Green support. But at an average error of 0.6% per party, this is the best performance ThreeHundredEight has managed to date.

Seat projection: 36 New Democrats, 20 Progressive Conservatives, 1 Liberal
Seat result: 37 New Democrats, 19 Progressive Conservatives, 1 Liberal
Error: 2 seats, or 0.7 per party.
Riding projection accuracy: 98.2%
Projected seat range: 35-38 New Democrats, 18-21 Progressive Conservatives, 1 Liberal
Seat range error: 0 seats, or 0.0 per party.
Ridings with all parties at or within 5% of result: 70.2%
Average margin of error on riding projections: +/- 2.7%

The seat projection model in this case demonstrated what can be done when the polls are right. The seat projection called 56 of 57 ridings correctly, with the incorrect riding being decided by little more than 100 votes. Though the projection ended up being off by only one seat, the seat range encompassed the actual result. If it wasn't for the one riding (St. Norbert, in Winnipeg), this election would have 100% correctly called, and not just the seat totals.

Seat projection with actual provincial vote: 36 New Democrats, 20 Progressive Conservatives, 1 Liberal
Error: 2 seats, or 0.7 per party.
Riding projection accuracy: 98.2%
Projected seat range with actual provincial vote: 35-38 New Democrats, 18-22 Progressive Conservatives, 0-1 Liberals.
Seat range error: 0 seats, 0.0 per party.

The Manitoba election was a complete success for ThreeHundredEight, and far beyond expectations. It was also the perfect election to show the usefulness of seat projections - the polls called for a tight finish, but the seat projections showed a comfortable New Democratic majority.

2011 PRINCE EDWARD ISLAND PROVINCIAL ELECTION
Final Projection 
Projection vs. Results Analysis 

Prince Edward Island gave the Liberals under Robert Ghiz their second mandate, as expected, but were far from the sweep that many thought possible.

Vote projection: 52.9% Liberals, 34.2% Progressive Conservatives, 8.1% New Democrats, 4.1% Greens
Vote result: 51.4% Liberals, 40.2% Progressive Conservatives, 4.4% Greens, 3.2% New Democrats
Error: 12.7 points, or 3.2% per party

There were two problems with getting the correct vote projection. Firstly, there were only two polls during the campaign, and the last one was completed six days before the vote. Secondly, the polls over-estimated NDP support and under-estimated Tory support. Were the polls off, or did something happen in those last six days?

Seat projection: 26 Liberals, 1 Progressive Conservative
Seat result: 22 Liberals, 5 Progressive Conservatives
Error: 8 seats, or 4 per party. 
Riding projection accuracy: 85.2%
Projected seat range: 25-26 Liberals, 1-2 Progressive Conservatives
Seat range error: 6 seats, or 3 per party.
Ridings with all parties at or within 5% of result: 29.6%
Average margin of error on riding projections: +/- 4.6%

While the under-estimation of PC support was a problem, the real source of the error was the small size of PEI's ridings. Many seats were decided by a few handfuls of votes, and the model can only do so much with ridings that are so small where the tiniest swings can make the difference.

Seat projection with actual provincial vote: 25 Liberals, 2 Progressive Conservatives
Error: 6 seats, or 3 per party.
Riding projection accuracy: 88.9%
Projected seat range with actual provincial vote: 24-26 Liberals, 1-3 Progressive Conservatives
Seat range error: 4 seats, 2 per party.

The model hit its limitations with such a small province. All politics is local, but that is especially the case for Prince Edward Island.

2011 CANADIAN FEDERAL ELECTION
Projection vs. Results Analysis

The 2011 Canadian federal election was a historic one. Stephen Harper won the first majority government in seven years, and the Liberal Party was replaced as the Official Opposition by the New Democrats. The Bloc Québécois was also reduced to a rump.

Vote projection: 36.4% Conservatives, 27.3% New Democrats, 22.8% Liberals, 6.7% Bloc Québécois, 5.6% Greens
Vote result: 39.6% Conservatives, 30.6% New Democrats, 18.9% Liberals, 6.0% Bloc Québécois, 3.9% Greens
Error: 12.8 points, or 2.6% per party

As described in the post-mortem, the problem here was the amount of weight reduction over time each poll received. In a more stable election this would not have been a problem (the weighting was tested for the 2006 election, which itself wasn't the most stable), but in an election with a last minute swing the model was not nimble enough. 

Seat projection: 143 Conservatives, 78 New Democrats, 60 Liberals, 27 Bloc Québécois, 0 Greens
Seat result: 166 Conservatives, 103 New Democrats, 34 Liberals, 4 Bloc Québécois, 1 Green
Error: 98 seats, or 19.6 seats per party
Riding projection accuracy: 76.0%

As the popular vote input into the model was inaccurate, the seat projection was inaccurate. But the seat projection model wasn't the problem.

Projection with actual regional vote: 161 Conservatives, 106 New Democrats, 37 Liberals, 4 Bloc Québécois, 0 Greens
Error: 12 seats, or 2.4 seats per party
Riding projection accuracy: 87.7%

The seat projection model would have worked pretty well with the right popular vote projection. But with the polls underestimating Conservative support by two points or more, it would have been difficult to project a Conservative majority correctly.

2010 NEW BRUNSWICK PROVINCIAL ELECTION
Projection vs. Results Analysis

New Brunswick doesn't tend to boot-out a government after only one term, but in 2010 they did. Shawn Graham's Liberals were crushed under the heel of David Alward's Progressive Conservatives, while the New Democrats again were unable to elect a single MLA.

Vote projection: 43.6% Progressive Conservatives, 40.1% Liberals, 10.4% New Democrats, 4.3% Greens
Vote result: 48.8% Progressive Conservatives, 34.4% Liberals, 10.4% New Democrats, 4.5% Greens
Error: 11.1 points, or 2.8% per party

The issue in this election was the lack of polls. Corporate Research Associates did daily polling only until 10 days before the vote, while Abacus Data did a poll mid-campaign. With so little data, a correct projection would have been difficult to achieve, though the results for the NDP and Greens was quite close.

Seat projection: 31 Progressive Conservatives, 23 Liberals, 1 New Democrat
Seat result: 42 Progressive Conservatives, 13 Liberals, 0 New Democrats
Error: 22 seats, or 7.3 seats per party
Riding projection accuracy: N/A

Projecting a Progressive Conservative majority was a bit of a gamble, considering the last polls appeared to be showing a narrowing gap. But nevertheless the projection was off in showing the size of the PC win. As in the 2011 federal election, however, it wasn't the seat projection model that failed. Riding-level projections were not made.

Projection with actual provincial vote: 44 Progressive Conservatives, 10 Liberals, 1 New Democrat
Error: 6 seats, or 2.0 seats per party

A decent result, and just like in 2011 it would have been crazy to have projected something like this. But elections can still surprise us.   

2008 QUEBEC PROVINCIAL ELECTION
Projection vs. Results Analysis

After almost being defeated by the ADQ in 2007, Jean Charest took advantage of some good polling numbers and called an election only a year later. It was a good gamble, and the Liberals were returned to power with a majority government. The Parti Québécois recovered from its lacklustre 2007 performance, while the ADQ returned to its third-tier status.

Vote projection: 42.9% Liberals, 33.2% Parti Québécois, 15.5% ADQ, 4.0% Québec Solidaire, 3.8% Greens
Vote result: 42.1% Liberals, 35.2% Parti Québécois, 16.4% ADQ, 3.8% Québec Solidaire, 2.2% Greens
Error: 5.5 points, or 1.1% per party

The vote projection did well here, though the PQ was slightly underestimated. The Green projection was off, but this was primarily because the Greens did not present candidates in all 125 ridings. A correction for this fact would have been helpful.

Seat projection: 68 Liberals, 53 Parti Québécois, 4 ADQ, 0 Québec Solidaire
Seat result: 66 Liberals, 51 Parti Québécois, 7 ADQ, 1 Québec Solidaire
Error: 8 seats, or 2.0 per party
Riding projection accuracy: N/A
Projection with actual provincial vote: N/A

ThreeHundredEight's first projection was a successful one, being with two seats for the PLQ and the PQ and within three seats for the ADQ. It correctly projected a slim Liberal majority and a strong PQ opposition, which most other projectors did not predict at the time.

ELECTION PROJECTION RANKINGS
VOTE PROJECTION RANKING
(2.4% average error per party per election)

1) 0.6% error per party - 2011 Manitoba
2) 1.1% error per party - 2008 Quebec
3) 1.2% error per party - 2011 Saskatchewan
4) 1.6% error per party - 2011 Ontario
5) 1.9% error per party - 2012 Quebec
6) 2.6% error per party - 2011 Canada
7) 2.8% error per party - 2010 New Brunswick
8) 3.2% error per party - 2011 Prince Edward Island
9) 3.5% error per party - 2013 British Columbia
10) 3.8% error per party - 2012 Alberta
11) 3.9% error per party - 2011 Newfoundland and Labrador

SEAT PROJECTION RANKING
(7.1 average seat error per party per election)

1) 0.7 seat error per party - 2011 Manitoba
2) 2.0 seat error per party - 2008 Quebec
3) 3.3 seat error per party - 2011 Newfoundland and Labrador
4) 4.0 seat error per party - 2011 Prince Edward Island
5) 5.3 seat error per party - 2011 Ontario
6) 6.0 seat error per party - 2011 Saskatchewan
7) 7.3 seat error per party - 2010 New Brunswick
8) 8.0 seat error per party - 2013 British Columbia
9) 8.5 seat error per party - 2012 Quebec
10) 13.5 seat error per party - 2012 Alberta
11) 19.6 seat error per party - 2011 Canada

RIDING PROJECTION ACCURACY RANKING
(79.3% of ridings correctly called - 715/902)

1) 98.2% of ridings correctly called - 2011 Manitoba (56/57)
2) 89.7% of ridings correctly called - 2011 Saskatchewan (52/58)
3) 85.2% of ridings correctly called - 2011 Prince Edward Is. (23/27)
4) 85.0% of ridings correctly called - 2011 Ontario (91/107)
5) 81.6% of ridings correctly called - 2012 Quebec (102/125)
6) 81.2% of ridings correctly called - 2013 British Columbia (69/85)
7) 77.1% of ridings correctly called - 2011 Newfoundland (37/48)
8) 76.0% of ridings correctly called - 2011 Canada (234/308)
9) 58.6% of ridings correctly called - 2012 Alberta (51/87)

POTENTIAL WINNERS IDENTIFIED IN RANGES
(83.2% of ridings - 247/297)

1) 85.9% of ridings - 2013 British Columbia (73/85)
2) 85.6% of ridings - 2012 Quebec (107/125)
3) 77.0% of ridings - 2012 Alberta (67/87)

WITH ACTUAL VOTE RESULTS
SEAT PROJECTION RANKING
(3.0 average seat error per party per election)

1) 0.7 seat error per party - 2011 Manitoba
2) 1.5 seat error per party - 2013 British Columbia
3) 2.0 seat error per party - 2010 New Brunswick
4) 2.4 seat error per party - 2011 Canada
5) 2.5 seat error per party - 2012 Alberta
6) 2.7 seat error per party - 2011 Newfoundland and Labrador
7) 3.0 seat error per party - 2011 Prince Edward Island
8) 4.5 seat error per party - 2012 Quebec
9) 4.7 seat error per party - 2011 Ontario
10) 6.0 seat error per party - 2011 Saskatchewan

RIDING PROJECTION ACCURACY RANKING
(85.3% of ridings correct - 769/902)

1) 98.2% of ridings correct - 2011 Manitoba (56/57)
2) 89.7% of ridings correct - 2011 Saskatchewan (52/58)
3) 89.4% of ridings correct - 2013 British Columbia (76/85)
4) 88.9% of ridings correct - 2011 Prince Edward Island (24/27)
5) 87.7% of ridings correct - 2011 Canada (270/308)
6) 85.0% of ridings correct - 2011 Ontario (91/107)
7) 81.6% of ridings correct - 2012 Quebec (102/125)
8) 75.0% of ridings correct - 2011 Newfoundland and Labrador (36/48)
9) 71.3% of ridings correct - 2012 Alberta (62/87)

POTENTIAL WINNERS IDENTIFIED IN RANGES
(89.9% of ridings - 267/297)

1) 95.3% of ridings - 2013 British Columbia (81/85)
2) 94.4% of ridings - 2012 Quebec (118/125)
3) 78.2% of ridings - 2012 Alberta (68/87)

Turnover to provincial coverage

In the coming days and weeks, ThreeHundredEight will be changing over to coverage of provincial elections. The graphic on the top of the page makes that clear. Mixed in with post-election analysis of the May 2nd federal election will be updates here and there to prepare the website for provincial coverage, with a few administrative changes thrown in as well.

Prince Edward Island, Manitoba, Ontario, and Newfoundland and Labrador (in that order) will be having their provincial elections between October 3 and October 11. Saskatchewan will be having theirs on November 7. As provincial campaigns generally last a month, we should be in a campaign from the beginning of September until the vote in Saskatchewan, barring any surprises that may come from a province like British Columbia. But for all intents and purposes, the campaigns in these provinces are already under way.

The graphic on the top of the page will be covering the four provincial elections scheduled for October. Saskatchewan's projections will be kept in the right-hand column until the four October elections are completed.

Of course, ThreeHundredEight will also be keeping an eye on the provinces that are not having elections this year, as well as the federal scene. However, with the next federal election not scheduled until October 2015, I will not be maintaining federal projections. And once the five provincial elections are completed, ThreeHundredEight will turn to the elections likely to occur in 2012, such as that of Alberta and Quebec.

I have not yet developed any riding-level models for the five provinces having elections this year. I will work to complete them as soon as possible, using the same projection system that was used in this past federal elections (with a few tweaks here and there based on the new data from the federal campaign). In the meantime, I will keep the graphics up-to-date with the latest polls and a seat projection based on those polls, using the old-style abstract projection model.

The five elections this year should be interesting. Those in Newfoundland & Labrador and Prince Edward Island are expected to be cakewalks for the governing parties, but this past federal election has demonstrated how things can change. Ontario and Manitoba should have some hotly contested elections, while the campaign in Saskatchewan will probably hand another majority to Premier Brad Wall.

Not all of these elections will be polled to any great extent. As far as I can tell, only one polling firm was active in the 2007 elections in PEI and Manitoba, but five were active in the Ontario campaign. So we should have a steady, but not overwhelming, stream of polls from each of the campaigns.

Suggestions, ideas, and comments are more than welcome as the site changes over to provincial coverage. 

Ranking the Pollsters (Updated)

Now that the dust has settled, we can take a look at how each of the pollsters did and assess their performances in the past election.
While it is true that polls are only a snapshot in time, and so are limited in their ability to predict into the future, polls are judged by how closely they align with election results. There is no other way to judge them. Certainly, it is possible that these polls were tracking voting intentions on their field dates accurately, and so cannot be judged for missing out on changing intentions over the last days or hours of the election campaign. While that is a valid argument, a means of assessing pollsters is required and comparing their final poll results to the actual results of the election is the only measuring stick we have. And pollsters themselves use that measuring stick, so we can certainly hold them to their own standards.

All polling firms have been assessed by their final poll results. Nanos Research has been assessed according to their final two-day report that was featured by CTV and The Globe and Mail. Crop, Innovative Research, and Environics are being assessed by their final polls, though they were all taken one week or more before the day of the vote. That may seem unfair, but these are the final numbers we have from these firms and they need to be assessed by some measure. Consider it a penalty for not releasing data closer to the end of the campaign.

Pollsters are assessed by their average error per party. In other words, being off by a total of 20 points for the five national parties combined would be the equivalent of having an average error of 4.0 per party. The party who performed best is highlighted in white. The polling firms are placed in the order of the date of their final poll. We'll start at the national level.

Angus-Reid had the best result for the second consecutive election, with an average error of only 1.0 per party. Nanos Research, Ipsos-Reid, Harris-Decima, and Léger Marketing also performed well, all with an average error of 1.4 points or lower. Of those who reported close to election day, EKOS and Compas performed worst.

See the bottom of this post for a discussion of how the margin of error could be taken into account in this assessment.

Note that only Compas, who was nevertheless very far off, was the only polling firm to over-estimate Conservative support. All others under-estimated their support by almost two points or more. The New Democrats were relatively well-tracked, but only Abacus and Ipsos-Reid had the Liberals lower than their actual result.

Disregarding the results from Environics and Innovative, the best polling method turned out to be the online panel, with an average error of 1.4 points per party. Traditional telephone polling scored 1.6 points, while IVR stood at 2.0 points' worth of error per party.

Also note that those pollsters who do not prompt for the Green Party (Ipsos-Reid) or any party (Nanos Research) generally predicted the Green Party's eventual tally better than those who prompted for the Greens.

Now let's move to the regional assessments, going west-to-east and so starting with British Columbia.

Here, Léger Marketing and Compas scored best with an average error of two points per party. Harris-Decima, at an error of 2.5 points, also did well in this province.

Nanos Research and Ipsos-Reid, who both put the Liberals in the mid-20s, did worst here.

As at the national level, only Compas over-estimated Conservative support. All of the others under-estimated their support, mostly to the benefit of the Liberals and New Democrats. But results varied widely, with Liberal support being pegged at between 10% and 26%, while the NDP was scored at between 25% and 40%. Small sample sizes are partly to blame. Green support, on the other hand, was well tracked.

In Alberta, Angus-Reid did best with an average error of only 1.3 points. The next best was Abacus Data, at 2.1 points. Harris-Decima did worst.

The pollsters had an easier time discerning Conservative support in this province, with two of the pollsters being exactly right. Others (Ipsos-Reid) inflated Tory support while yet others under-estimated them (Harris-Decima, EKOS, Forum). The NDP and Liberals were well tracked, though only Abacus had them in single-digits. All-in-all, the pollsters did well in Alberta.

Compas and Nanos, however, grouped Alberta with Saskatchewan and Manitoba. In these three provinces, Nanos bested Compas by an average of 0.7 points. Compas appeared to give some of the Liberal support to the Tories, while Nanos gave some of the Tory support to the NDP.

In the more usual grouping of Saskatchewan and Manitoba, Angus-Reid did best with an average error of 1.5 points, closely followed by Ipsos-Reid. EKOS struggled here.

Generally, the pollsters had an easier time pinpointing Conservative support here, with three of the pollsters within about a point of their final result. The New Democrats were also well represented, with the biggest error being only of 3.1 points among those polling firms active in the final days of the campaign. The Liberals were also well tracked, while only Ipsos-Reid correctly had the Greens at 3% in these two provinces.

Ontario was the most important province to poll correctly. Angus-Reid did the best here, with an average error of only 1.7 points. They were closely followed by Harris-Decima, while Ipsos-Reid did the worst among those who polled in the final days of the campaign.

But the problem here, as elsewhere, was in recording Conservative support. Again, only Compas over-estimated the Tories while all others had them at 41% or lower. That error had great consequences in determining whether they would win a majority or minority government.

The pollsters, except Compas, over-estimated NDP support, but only by a little. Some (EKOS, Harris-Decima, and Angus-Reid) were very close to accurately predicting the NDP's support, but others had them well over their actual result. The Liberals were generally well polled, however, two pollsters being on the money and another three being within a point.

Quebec provided the biggest surprise on election night, but amazingly the pollsters did very well in the province. They did better than they did in Ontario, which usually has larger sample sizes.

Ipsos-Reid takes the crown in Quebec, but was closely followed by Nanos, Forum, and Léger. Compas and Crop did the worst, though they were still relatively close.

Three pollsters were almost exactly right in predicting the NDP's result, while four others had the NDP over 40%. The pollsters had a bit more trouble with the Bloc, with only Nanos and Forum indicating that they would end up in the low 20s. The pollsters did an excellent job in recording Liberal support, but did a little worse with the Conservatives. All in all, though, the pollsters did an excellent job in Quebec.

That was not the case in Atlantic Canada, which was the worst polled region in Canada. Granted, it usually is polled in very small numbers but the same amount of people are usually surveyed in the Prairies, and there the pollsters did much better.

Harris-Decima was closest with a very good average error of 1.3 points, while Léger was close as well. But EKOS, Angus-Reid, Compas, and Ipsos-Reid were all off by six points or more, with seven pollsters putting the NDP in first in the region.

Results varied wildly, with the Conservatives pegged at between 26% and 44%, the NDP between 28% and 46%, and the Liberals between 11% and 30%, among pollsters active at the end of the campaign. Only Harris-Decima had the NDP below 30%, while most under-estimated Liberal support. Atlantic Canada was a wash.

And that brings us to our final ranking. The pollsters have been ranked on two scores: their national average error (recorded in the first chart at the top of this post) and their average regional error. Combining the two rankings anoints Angus-Reid as the best pollster in 2011, but also awards Harris-Decima, Léger Marketing, and Nanos Research with honourable mentions.

Nationally, Angus-Reid comes out on top but was only marginally better than Nanos, Ipsos-Reid, Harris-Decima, and Léger Marketing.

Forum Research and Abacus Data, in their first federal campaigns, did very respectably.

EKOS struggled while Compas was the worst pollster active in the final days of the campaign. Environics and Innovative Research might have done better had they polled closer to election day.

However, national results can be close merely because all of the regional errors cancel each other out. That was the case with Ipsos-Reid, which drops to 9th place in the average regional error ranking.

Instead, Léger Marketing takes the top spot on its regional results with an average error of 2.4 points. Harris-Decima and Angus-Reid were close behind with an average error of 2.7 and 3.0 points, respectively, while Abacus Data placed 4th on the regional ranking.

Regionally, of those active in the final days of the campaign the online pollsters were off by an average of 2.9 points. Traditional telephone surveys were off by 3.7 points, while IVR surveys were off by 3.9 points. Though it could have been blind luck, the much-maligned online surveys performed best in the 2011 federal election.

Margin of Error Update

Some pollsters and commenters have brought up the issue of the margin of error, and whether that should be taken into account to judge the performances of the polling firms.

From a statistical standpoint, polling firms should be able to accurately predict the result of each party within the poll's margin of error. Though it is hardly the focus of any press release or media report, the margin of error is (or should be) always included in any poll report, and thus if a poll gets the result correct within the margin of error it is a technically accurate poll. But this benefits polling firms with smaller samples, who have larger margins of error within which to work.

If we use this standard, at the national level we would have to eliminate all but Ipsos-Reid, Abacus Data (assuming a random sample for this online pollster), and Nanos Research. These are the only three firms whose national poll findings were within the margin of error of the sample.

If we take it to the next level, assessing each polling firm by the margin of error for each individual party (i.e., the margin of error for the Green Party is not the same as the margin of error for the Conservative Party because of their different levels of support), we have to eliminate all but Nanos Research from this assessment. Ipsos-Reid would fail to pin down NDP support taking this margin of error into consideration, while Abacus would be wrong for the Green Party.

But why stop at the national level? We do not elect presidents - regional data is as important, if not more important, than the national horserace numbers. If we bring it down to the regional level, then even Nanos has to be eliminated as their result of 23.6% for the Liberals in British Columbia (rather than their actual 13.4%) is outside of this particular sample's 7.7% margin of error. No pollster would survive this assessment, as no pollster would fit this criteria even taking into account the 95% confidence.

Just as problematic methodology will not be corrected by over-sampling, it can also be masked by the large margin of error of smaller samples. In the end, Nanos should be commended for having its national poll results within the margin of error, with honourable mentions also going to Ipsos-Reid and Abacus Data. But from my perspective, polling firms that conduct surveys with large samples, thereby giving us more reliable regional results, should not be thrown under the bus. Polls are reported for the consumption of the general public, and the general public is interested in how accurately the polls predict actual outcomes, both nationally and regionally. While I agree that some consideration should be taken for the margin of error in assessing the performance of the pollsters, which I have now done, I believe that my original assessment stands.

A multimillion-vote sea change for federal parties

On election night, over 1.7 million fewer Canadians voted for the Liberals, Bloc Québécois, and Green Party than in 2008. Along with 890,000 new votes, the lost ballots were divided up between the Conservatives and New Democrats, who increased their vote count by more than 620,000 and nearly two million votes from the last election, respectively. But these gains and losses were not spread out evenly across the country. 

You can read the rest of the article on The Globe and Mail website, here. A condensed version of the article along with the infographic can also be found in today's print edition of The Globe and Mail.

The piece isn't a simple vote count, it is meant to also focus on the regional weight of votes gained or lost for each of the parties. For example, almost 60% of all NDP vote gains came in Quebec, despite the province's voters representing 25.8% of the national total.

To give you a base of comparison as you read the article and check out the graphic, here are the proportions each region of the country represented out of the 14.7 million Canadians who voted:

Ontario: 37.6%
Quebec: 25.8%
British Columbia: 12.7%
Alberta: 9.5%
Atlantic Canada 7.7%
Prairies 6.4%

You can immediately see that much of the change that took place happened in Ontario and Quebec. These regions were over-represented in vote gains and losses for all parties - except for the New Democrats in Ontario, demonstrating in part just how important the NDP's performance in Quebec was in propelling them to Official Opposition status.

Weighing the polls

On Monday night, ThreeHundredEight's seat projection model failed for two reasons. First, the 7% reduction in weight given to polls for each day that passed was not nimble enough to capture the huge shift in voting intentions in the last two weeks. Secondly, the polls under-estimated Conservative support by over two points nationally, three points in Ontario, and four points in British Columbia. As the seat projection is only as good as the polls that are put into it, this is part of the reason why the seat projection failed.

So what steps can be taken to fix the problem? This fall, a flurry of provincial elections will occur and ThreeHundredEight will be endeavouring to do a much better job in accurately projecting their outcome. Tuesday's post demonstrates that the projection model can produce good results, and tweaks will be made to make it even better. But clearly the vote projection model has to be overhauled.

The first step was to throw out the weightings given to the last three elections. The idea was to use these elections as an "anchor", ensuring that the projection would not be thrown off too much by the fluctuations in the polls. For example, using this anchor was an effective way to keep the Green vote below the unrealistic support levels at which various polls put them.

But in a wild campaign like that of 2011, the anchor was unnecessarily weighing down the projection. Indeed, instead of being off by a cumulative 12.8 points, the projection would have been off by 12.2 points without the anchor. That means it's gone.

Next, I played around with the vote projection model and changed only the rate of depreciation of each poll's weight by age. The following chart shows the cumulative error between the vote projection and the actual result for the five main parties at various reduction rates (i.e., 0.75 means a daily reduction of 25%).

As you can see, the best weighting was between a daily reduction of 20% and 30% (0.80 to 0.70). Delving more deeply, I found that a reduction of 23% per day had the best result, bringing the cumulative error down to 6.8 points for the five parties, an average of 1.4% per party.

Rates of 21% and 22% also resulted in a cumulative error of 6.8 points, but after looking at how it would have done at the regional level for Ontario, Quebec, and British Columbia, I settled on 23% being the closest.

Nevertheless, the projection would have still been off. Though it would have been very close for the New Democrats and Bloc (on the money for the Bloc and off by 0.2 points for the NDP), it would have been further off for the Conservatives (36% rather than the actual 39.6%) and the Liberals (20.5% instead of 18.9%). In fact, for the Tories it was impossible to get closer than 36.3% using a weighted average of all polls.

The pollsters are partly to blame for that. If I dropped the weighted projection completely and simply used the average of the final poll from each polling firm released within the last five days of the campaign, I would still have only gotten 37.4% for the Conservatives, off by 2.2 points. Most significantly, the Conservative average would have low by 4.6 points in British Columbia and 3.5 points in Ontario - that is where the Tory majority was won.

Nevertheless, abandoning the weighted vote projection for a simple average would have given a better result, with a cumulative error of 5.8 points. The average cumulative error in British Columbia, Ontario, and Quebec would have been 8.5 points, rather than the 9.6 points when using a reduction rate of 23% in the weighted average.

At this point, one would suggest that I move to a polling average of only the most recent polls and be done with it.

However, the 2006 election tells a different story. There are some parallels between the 2006 and 2011 elections, as they both featured a shift in public opinion during the campaign. Using a weighted average of all polls with a reduction rate of 10% per day would bring about a very good result, with a cumulative error of 3.3 points at the national level. Any other reduction rate pushes the error to 3.9 or 4.0 points, which is still very acceptable over five parties.

But using only the final polls of the campaign would result in a cumulative error of 5.8 points. In other words, while a simple poll average would have been more accurate in 2011, the weighted poll averaging system that I use in my popular vote projection would have been more accurate in 2006. And of all these scenarios, using the average of the last five days of polling in the 2008 election is worst of all at a total error of 6.9 points.

If ThreeHundredEight is still around for the 2015 election, some sort of ballot box effect might have to be taken into account. Obviously, a ballot box effect at the federal level cannot be translated to the provincial level - the parties just aren't the same. But this daily weighting of 23% appears to give good results, and so will be used for the provincial projections this fall. Adjustments for polling error may also be investigated for each of the provincial elections and included in the model, while the track records of each of the pollsters will also be adjusted in the wake of this election.

On that note, I will be taking a detailed look at how the pollsters did (briefly hinted at here) next week.

UPDATE 06/05/11: Running the projection at 23% daily reduction, I would have projected 143 Conservatives, 102 New Democrats, 49 Liberals, and 14 Bloc seats. While it would have been a much better projection, it still would not have called the Conservative majority - just like the polls did not.

How vote-splitting gave the Tories Ontario - and a majority

The Conservative Party can finally lay claim to Ontario – and the majority bragging rights that come with it – but it was the NDP that may have made the decisive blow to the former Liberal fortress.

You can read the rest of the article on The Globe and Mail website here.

I believe a little further explanation is needed for this article. Clearly, if we add the Liberal and New Democratic votes together the Conservatives would have lost many ridings. In an email to me, a professor from the Université de Montréal noted that doing so would have reduced the Tories to only 137 seats. While this might be true, this isn't what I mean by vote-splitting. The Liberals and New Democrats are two separate parties, and their votes can't simply be added together.

What I was looking at was how New Democratic gains contributed to Liberal defeats. The best example of this is the riding of Moncton - Riverview - Dieppe in New Brunswick.

In 2008, Brian Murphy of the Liberals received 17,797 votes. The Conservative candidate had 16,297 votes while the NDP candidate finished third with 7,394 votes (source).

Using Elections Canada's preliminary results, we see that on Monday night Robert Goguen of the Conservatives received 17,408 votes, an increase of 1,111 votes. Brian Murphy dropped 2,553 votes to 15,244, while Shawna Gagné of the NDP gained 6,659 votes for a total of 14,053.

One could argue that by adding the NDP and Liberal votes, the anti-Conservative option would have almost doubled Goguen's total. But that is not the argument my article makes.

The gap between Goguen and Murphy was 2,164 votes, meaning that those who switched over from the Liberals (primarily, we can't know exactly how the vote split) to the New Democrats helped elect Goguen. Only 2,165 of the 6,659 new NDP voters in this riding (32.5%) needed to vote Liberal to avoid voting in a Conservative.

It was the same situation in many ridings in Ontario, where NDP gains were greater than the gaps between Liberal incumbents and Conservative challengers. Certainly, people need to vote for the candidate they support but there may have been some unintended consequences.

This is not to put the blame on the New Democrats, nor does blame need to be assigned. With their level of support, the Conservatives earned their majority. And in fact, in many of the ridings won by the Conservatives from the Liberals, the Liberal candidate was simply beat. The Conservatives made significant gains in many ridings, something that Liberal-to-NDP vote switchers could not have stopped. And it was the failings of the Liberal Party that pushed many of their supporters to the New Democrats and the Conservatives. Nevertheless, this vote shift played an important role in electing a Conservative majority government.

2011 Federal Projection vs. the Results

Well, at least I got the order of the parties right.

Clearly, the final projection was wrong. It under-estimated both the Conservatives and the New Democrats and over-estimated the Liberals and the Bloc Québécois. While I was not alone in making this error, I humbly recognize that of all the projections mine was among the worst.

Of the 308 ridings in the country, ThreeHundredEight.com correctly called 234 of them. That's an accuracy rate of 76.0%, which is absolutely unacceptable.

However, it was not the seat projection model that failed. The seat projection model actually performed very well - or would have had the popular vote projection model not missed the mark so completely.

In fact, inputting the actual provincial-level vote results into the model, treating them as the results of a poll, churns out a very accurate result.

With the actual popular vote results in the 10 provinces, the projection model projects 161 seats for the Conservatives, 106 for the New Democrats, 37 for the Liberals, and four for the Bloc Québécois.

In other words, the model would have been off by five seats for the Tories, three for the New Democrats, three for the Liberals, and one for the Greens. It would have had the shocking four-seat result for the Bloc correct. In total, it would have correctly called 270 ridings for an accuracy rating of 87.7%. That could be better, but considering the wild changes in support throughout the country that is not a bad result at all.

In short, the seat projection model was able to turn regional results into reliably accurate seat results. Claims that seat projection models cannot capture huge, historic changes are false.

Accurately capturing how people will behave at the polls, however, is a different matter. The vote projection model failed irrevocably. It was partly the fault of its slow progression and partly because of the inaccuracies of the polls.

I should be clear that the polls were not terribly inaccurate. They were actually quite good in pegging the support levels of the Liberals, New Democrats and Bloc Québécois. But they under-estimated the Conservatives, arguing for a strong minority rather than a relatively strong majority government. This was especially problematic in British Columbia, Ontario, and Atlantic Canada. On the other hand, full marks go to the pollsters in unpredictable Quebec. They were all very close to the actual result.

ThreeHundredEight.com projected two things: the popular vote and how that would transform into seats. It failed at the first, which means it failed at the second. Work will be done to get the popular vote projection model up to snuff in time for the autumn provincial elections. However, I am very confident that my seat projection model is an accurate model, and will accurately project elections as long as the right numbers are put into it. Had I simply trusted the latest polls, rather than including weighted polls stretching back to the beginning of the campaign, my projection would have been much closer to the actual result.

I do apologize for the inaccuracy of my projections. But in the aftermath of such a historic election result the projection model has proven that it can be used effectively. I will work on ensuring that in the next election I will use it properly.