Thursday, October 13, 2011

PEI: Projection vs. Results

It's time to return to the island, and take an indepth look at how the projection stacked up against the results in Prince Edward Island, the first election in the series of five to take place in 2011.

In what is most important, the projection made the right call. Robert Ghiz's Liberals were given a majority government, with Olive Crane's Progressive Conservatives forming the Official Opposition. Neither the Greens nor the New Democrats entered the legislature.

But with the polls showing a strong lead for Ghiz, his re-election with a majority was a foregone conclusion.

ThreeHundredEight projected 26 Liberals and one Progressive Conservative, when instead 22 Liberals and five Progressive Conservatives were elected. With the exact provincial vote totals input into the model, the projection gave the Liberals 25 seats and the Tories two. While better, it was still off.
The projected seat range of between 25 and 26 Liberals and between one and two Progressive Conservatives was as good as the projection with the actual results. I'll return to why the projection made the wrong calls in several ridings later.
The vote projection for the Liberals and the Greens was quite good, missing the Liberals by only 1.5 points and the Greens by 0.3. But the Progressive Conservatives took six more points than expected while the NDP took almost five points less.

There are a few reasons for this. Both polls released at the end of the campaign under-estimated the Tories and over-estimated the New Democrats, and both polls were released almost a week before the vote was held. Opinion may have shifted in those last days of the campaign, and the inability of the New Democrats to run a full slate undoubtedly contributed to them garnering less of the vote.

In the end, the projection had an accuracy rating of 85.2%, making the correct call in 23 of the 27 ridings in Prince Edward Island. The projection with the actual vote totals would have made the correct call in 24 of 27 ridings, for an accuracy rating of 88.9%.
As you can see from the list of facts above, in the four ridings where the wrong calls were made the average margin of victory was 9.0%. This is somewhat skewed, as in two ridings the margin was less than 1.5%, while in two others it was over 12%.

The biggest problem was the over-estimation of Liberal strength, which was done in almost 82% of ridings. The same was done for the NDP in 73% of the ridings in which they ran candidates, while the Tories were under-estimated in 85% of ridings.

On average, Liberal support was projected to be 5.3% higher than it actually was in each riding, while it was projected to be 5% lower than the actual result for the Tories. The Green and NDP estimates were much better.

But that does not mean the riding projections weren't accurate. In roughly half of ridings the Liberal and/or PC support was estimated at within 5% of the actual result, while that increases to over 85% for the New Democrats and Greens (not surprisingly, since in most ridings they did not get much more than 5%). In all, 29.6% of party support projections in each riding were at or within 5% of the final result.

Taken together, the margin of error for each and every riding projection was, on average, +/- 4.6%.
The chart above shows the three closest riding projections, in Alberton-Roseville, Kensington-Malpeque, and Vernon River-Stratford.

The performance of the Island Party in Alberton-Roseville prevented that projection from being absolutely perfect.

While the projection in Prince Edward Island was adequate, what prevented it from being more accurate?

In addition to the local factors at play, which will always be amplified in a smaller province, there were a few other problems.

The most important was the unexpected strength of the Progressive Conservatives in eastern PEI. It was expected to be their best region, but they did much better there than the provincial swing should have indicated.

Does Prince Edward Island need a regional model? It probably would have helped. The last poll from Corporate Research Associates did have a regional breakdown, but it wasn't reported in full in The Guardian. Constructing a regional model for PEI would require more information in future elections. In any case, both CRA and MQO Research (especially) under-estimated Progressive Conservative support, and even in the regional breakdowns CRA had the Tories and Liberals "neck-and-neck" in eastern PEI, when the result was actually a seven point edge for the Tories.

But what of the four ridings in which the wrong calls were made? Souris-Elmira and Tignish-Palmer Road went Tory for no reason that the projection model could have predicted. Removing the incumbency bonuses and giving the two PC candidates the "star" bonus still would have shown them leaning Liberal.

Stratford-Kinlock would have been projected to vote PC if James Aylward had been considered a star candidate, but Aylward does not fit the profile.

Georgetown-St. Peters is a little more interesting. With the correct province-wide vote, the projection would have given this riding to the Tories. But it seems that the lack of a PC incumbent did not play a big role in this riding - removing this factor would have given the riding to the Tories in the pre-election projection.

What this boils down to is that two of the four ridings that were wrongly called could not have been correctly called by this projection model. The ridings simply went their own way and bucked province-wide trends. Another would have been correctly called had the vote projection been accurate, while the last riding could have only been projected correctly by assigning the Tory candidate a star bonus he (to be harsh) did not deserve. A regional model which would have had the Progressive Conservatives doing better in the eastern part of the island might have done a better job getting Stratford-Kinlock and Souris-Elmira correct, meaning only Tignish-Palmer Road would have still voted unexpectedly in a perfect projection storm.

The projection for Prince Edward Island was a middling performance, ranking fourth in the seven elections that ThreeHundredEight has made projections up to this point.