Wednesday, September 5, 2012

Quebec election aftermath

Last night's election results in Quebec were surprising, but the event that took place in Montreal after the results were known overshadowed to a great degree what happened in the voting booths. The attack perpetrated by an undoubtedly deranged individual was an assault not only on his victims, but on all Quebecers - and all Canadians - and upon our democratic values. My thoughts are with the families of the two victims. Tragically one has passed away.

This sort of heinous act puts a lot of things in perspective, but it is important not to let the crime of a single man carry more weight than the act that millions of Quebecers took part in yesterday, an act that hundreds of millions of people worldwide are forbidden from doing and for which they have given their lives.

I will have some analyses of last night's results on the websites of The Globe and Mail and The Huffington Post Canada later today and I will update this post with links. UPDATE: The Globe article can be found here. The article for The Huffington Post Canada can be found here.

It will take a little while to pore over how the projection performed, especially as I will have to calculate the regional results of last night's vote in order to see how the model would have done with the correct numbers. That was a big problem last night. No poll was very accurate, especially in terms of the Liberals. The highest number they had in a late-campaign poll was 29%, and that was in Forum's election-eve survey that also had the PQ at 36%. The poll that had the best result for the Liberals also had the worst result for the PQ. In other words, it was not very helpful.

The projection did make a few good bets, forecasting that the polls would under-estimate the Liberals and over-estimate the support of the CAQ (edit: at least in terms of Léger and CROP), Québec Solidaire, Option Nationale, and the Greens. It was wrong to believe that the polls would also under-estimate the Parti Québécois and that was the difference between projecting a bare-minimum majority government and the minority government the party actually won.

A preliminary analysis suggests that the projection made the right call in about 82% of ridings (with the projected ranges, that is bumped up to 86%), a decent performance considering the faulty inputs of the polls. There were a few surprises, and like in the Alberta election the resilience of the Liberal incumbents was extraordinary.

Once I plug the actual regional results of the election into the model, I'm very confident that the projection will provide a close result. But that just proves once again that the challenge is being able to forecast results with the polling data that is available. More and more, that data is looking less and less useful. The two polls conducted on the eve of the vote were both fatally flawed (one suggested a clear PQ majority, the other a Liberal catastrophe). It is unfortunate that Léger and CROP were not in the field later in the campaign, but it is impossible to know if they would have performed any better.

In the end, ThreeHundredEight's vote projection turned out to be closer than any of the polls, with a total error of 8.1 points (compared to 8.4 for Léger and 10.4 for Forum/CROP). That is not quite good enough, and more work will have to be done to determine how to best forecast the difference between voting intentions and voting behaviour. Wider confidence intervals and the recognition that there is only so much that polls can tell us about what will happen tomorrow may be in order.