The 2011 Ontario election was a tricky one. Throughout the campaign, the polls showed a neck-and-neck race between the Liberals and the Progressive Conservatives. Some of the polls showed a widening gap between the two parties in the final days, but the end result wound up being just as close as the campaign had promised it would be.
The Liberals under Dalton McGuinty won their third consecutive election, but were reduced to a minority government by a margin of one seat. Tim Hudak's Progressive Conservatives formed the Official Opposition while Andrea Horwath's New Democrats remained in third place. Both opposition parties, however, improved their standings in the legislature and all three leaders stayed on.
This was a total error of 16 seats, or 5.3 per party.
Using the actual provincial vote result improved things only slightly, to an error of 4.7 seats per party. The Liberals were still erroneously projected to win a majority.
Whether using the projected vote or the actual vote, the riding level accuracy was 85.0%, with 91 of 107 ridings being accurately called.
The chart below shows a few quick facts about the projection at the riding level.
The New Democrats and Progressive Conservatives were generally well called, as between 70% and 75% of their riding level projections were within 5%. This dropped to about 59% for the Liberals, and overall 45% of ridings, or 48 out of 107, had every party at or within 5% of their actual result.
This was the real problem. The Progressive Conservatives were under-estimated in 61 of 107 ridings, and only over-estimated in 36 ridings. Ten were called exactly.
This contrasts greatly with the Liberals and NDP, who generally split evenly. The New Democrats were under-estimated in 50 ridings and over-estimated in 48, with nine being called exactly, while the Liberals were over-estimated in 53 ridings and under-estimated in 46, with eight being called exactly.
In all, the margin of error at the riding level was +/- 3.3% per party.
These are the top three ridings in terms of accuracy:
But why did the projection get 16 ridings wrong? It wasn't just the polls under-estimating the Progressive Conservatives, as the model would have still gotten 16 ridings wrong with the actual province-wide numbers.
The problem is that province-wide numbers are difficult to use in such a regionally diverse province. If we look at the results more closely, we see where things failed regionally. The bulk of the incorrect ridings were in southwestern Ontario, where the Tories were under-estimated in 17 out of 21 ridings. They were also under-estimated in every riding in northern Ontario.
For the three major parties, the riding level projections had an average error of +/- 4% or less in central Ontario, the Hamilton/Niagara region, and the GTA. But that error was more than 5% in Toronto, southwestern Ontario, and the north, where more than two-thirds of the errors took place. This demonstrates why a regional model is necessary, despite the lack of uniformity in how the pollsters divvy up the province.
This was a big reason for a lot of the errors, but there were also some specific factors at play. For example, Rocco Rossi was not a "star candidate" in Eglinton-Lawrence, or at least that bonus made that riding an inaccurate call.
Overall, however, it was not a bad projection. The 1.6% popular vote error was acceptable, as was the +/- 3.3% error at the riding level. Being off by 5.3 seats per party is less acceptable, but had the Liberals eked out their majority it would not have been as consequential an error as it turned out to be.