Monday, November 2, 2015

A different approach to seat projections

One lesson learned from the 2015 federal election is in the difficulty in projecting outcomes when the voting base of a party has dramatically shifted from one election to the next. In this case, it was the Liberal Party that attracted new and lapsed voters to the polls. Many of these voters appeared in unexpected numbers and in unexpected places, which is why virtually all seat projectors estimated that the Liberals were on track to win only a minority government, with a majority government being possible but unlikely.

Turnout in most elections does not dramatically shift from one campaign to the next, making it much easier to project outcomes at the seat level based on regional or provincial changes in support. It is no accident that the model has performed best when changes in turnout were lowest, while it has struggled more in elections with big changes in turnout.

But this is not all about new voters. More people voted in 2006 than in 2008 or 2011, meaning that a lot of people who voted in the 2006 election stayed home in the next two elections as the Liberals plummeted in public support. The Liberals did attract a lot of first-time voters in this campaign, but they were also successful in bringing back a lot of voters that just stayed home in 2008 and 2011.

Though there were a few surprises, the vast majority of the ridings the Liberals won in this campaign were ridings they had held in the past. Many of their pick-ups in Quebec, for instance, were won by the Liberals in 2000.

Taken together, this suggests that looking further back into a riding's history than just the most recent election can tell us a lot about how likely a riding is to go one way or another in the next election. If the basic principles of the swing model can work when looking at just one election, then they should be able to work when applied to elections further back in time, swinging the results from two or three elections ago according to where the polls are today.

This is what I tested with the 2015 results, swinging the results in each riding over the last three elections. I tried various weightings (looking at just the last two elections, giving all three elections equal weighting, trying proportions of 4/2/1, etc.), but the one that worked best was also the one that was most intuitive. It weighted the three elections with a proportion of 3/2/1, or 50% for 2011, 33.3% for 2008, and 16.7% for 2006.

As shown below, the results were very positive.

Using the actual provincial and regional vote shares (which is the real test of the projection model), the old model would have delivered the Liberals 154 seats, the Conservatives 120 seats, and the NDP 56 seats. The actual results for the Liberals and Conservatives would have fallen outside of the likely ranges, and into the maximum and minimum ranges.

But with the new model, the Liberals would have been projected to take 166 seats, the Conservatives 116, and the New Democrats 45. It would have been almost right on the mark for the NDP, and exactly on the mark for the Bloc Québécois. Most importantly, the Conservatives and Liberal results would have fallen within the likely ranges — the Liberal result falling well within that mark.

The regional projections would have been improved significantly, particularly in Quebec. The actual result there was 40 seats for the Liberals, 16 for the NDP, 12 for the Conservatives, and 10 for the Bloc. With the new model, it would have given 38 seats to the Liberals in Quebec, with 18 going to the NDP and 12 to the Tories. Only in Atlantic Canada would the results have not fallen within the maximum and minimum ranges, with the Liberals topping out at 30 seats and the NDP bottoming out at two. Peter Stoffer and Jack Harris would have still been projected to survive.

Even considering where the vote projection model stood on the eve of the election, this new seat projection model would have given the Liberals a range of between 128 and 177 seats — meaning their final tally of 184 seats would have fallen outside the likely range, but the potential for a Liberal majority victory would have been a big part of the final analysis (rather than the marginal part it was actually given).

Accuracy would have been higher at the riding level, increasing from 81.4% to 82.5%. But most significantly, the potential winners would have been identified in the high-low ranges in 91.4% of cases, meaning in only 29 ridings would the potential winner have been missed, compared to 43 with the old model.

This new model requires a few other changes, including some judgement calls in terms of how to handle special cases. This involves ignoring the results of some past elections, such as the 2006 and 2008 elections in Saanich–Gulf Islands, where Elizabeth May did not run, or cases where an independent took an out-sized portion of the vote (as in 2006 in Portneuf–Jacques-Cartier, when the Conservatives ran a candidate against independent André Arthur, or in 2008 when Bill Casey ran as an independent in what is now Cumberland–Colchester).

In addition, the incumbency effect built into the model is now a redundancy, as the results of the previous elections already take into account an incumbent's ability to withstand wider trends. Losing an incumbent, however, is still applicable.

Note that I do intend to run this new model through more tests as time permits to ensure that its improved performance isn't due to a fluke related just to the 2015 federal election. But time is an issue, because the next provincial election is just weeks away.

The next test: Newfoundland and Labrador

This new model may never be tested for real at the federal level, as the next election may not be decided according to the first-past-the-post system. Elements of it could be part of a model designed to project the outcome of a ranked ballot election, however.

The real tests will come in the provincial elections that will still use the first-past-the-post electoral system. And the next one is coming very soon: Newfoundlanders and Labradorians are heading to the polls on November 30.

How would the new model have performed had it been used back in 2011? Again, it would have done a better job.

The actual results of that election delivered 37 seats to the Progressive Conservatives, six seats to the Liberals, and five to the New Democrats. This despite the NDP finishing ahead of the Liberals in the popular vote by over five percentage points.

With the actual results plugged into the 2011 model, it awarded 41 seats to the PCs, four to the Liberals, and three to the NDP. Not a bad showing, considering it still would have given the Official Opposition nod to the Liberals. But the riding-level accuracy of just 75%, or 36 out of 48 ridings, left a lot to be desired.

The new model would not have improved upon the 2011 performance, at least in the top-line numbers: it still would have been 41 Tories, four Liberals, and three New Democrats. But the riding-level accuracy would have increased to 81.3%, with correct calls in 39 out of 48 ridings.

The election in Newfoundland and Labrador will pose a few problems, in that the number of seats has been reduced from 48 to 40. There have also been a large number of floor-crossings and by-elections since 2011, further complicating matters. These are the sorts of things that can throw any seat projection model for a loop.

But I will put the new model to the test nevertheless and see how it does. The same principles behind what has been, I believe, a very effective model (which has been used in 17 provincial and federal elections) are still in place, so I consider this a refinement rather than a wholesale change. We'll see how it does soon.