Sunday, November 29, 2015

For the record

At an event put on by the MRIA last week, four pollsters went over what they saw happening in the last federal election campaign.

According to this report by iPolitics, Darrell Bricker of Ipsos Reid was quite critical of the work done by polling aggregators, such as myself and Barry Kay of the Laurier Institute for the Study of Public Opinion and Policy (who was partnered with Global News, Ipsos Reid's media partner). He said:

“I would argue, quite frankly, that these models are as likely to maximize error by putting polls together as they are to minimize it…There’s work to do here and there’s value associated with doing this. But it’s not happening well right now…Sorry, CBC.”

Bricker may very well have a point. Aggregation can certainly be done better, and I and others are always working towards improving it.

But for the record, the error in Ipsos Reid's final poll totaled 1.2 points per party, compared to 1.3 points per party for and the CBC Poll Tracker. Hardly a case of maximizing error. And the aggregation out-performed Ipsos Reid at the regional level in British Columbia, the Prairies, Ontario, Quebec, and Atlantic Canada. Ipsos Reid only did better than in Alberta.

Ipsos Reid did out-perform in the 2011 federal election campaign. But since then, when changes were made to the aggregation model, has out-performed Ipsos Reid in the provincial campaigns in Ontario in 2011 and 2014, in British Columbia in 2013, in Quebec in 2014, and in Alberta in 2015, as well as in the municipal election in Toronto in 2014.

More broadly,'s aggregation has out-performed the polls in most elections. This site has had a smaller average per party error than the average error of all the pollsters in the field for 15 consecutive federal, provincial, and municipal campaigns, and 16 of 18 campaigns overall.

Aggregation is a way to minimize error in the vast majority of cases.

In terms of seat projections, there is certainly more work to be done and I have already taken some steps to address some of the issues with the model. But I should also take this opportunity to point out that the model itself, divorced from how the polls do, has identified the potential winners in 90% of ridings, and its error in the more narrow likely ranges has been four seats or less — in total, all parties combined — in 12 of 14 elections where ranges were given.

Bricker is right to point out the amount of polling data that Nate Silver has to work with. Having that much data would certainly make things a lot easier. But our first-past-the-post system and multi-party democracy is also much more complicated. This was shown in 2010 when Silver had just as much trouble as everyone else in projecting the outcome of the election in the U.K. It happened again this year when FiveThirtyEight affiliated itself with a British outfit.

American elections, with their two parties and, at the presidential and Senate levels, very big jurisdictions, are much more predictable. In this past election, the Liberals jumped 20 points and the Conservatives dropped eight from the previous vote. The last time there was a change of government in the United States, the swing between the Republicans and the Democrats was just five points. Almost half of the seats on offer changed hands in the 2015 Canadian federal election. In the 2012 U.S. presidential election, only two states out of 50 changed hands.

There will always be elections that for one reason or another are very difficult to predict without making unreasonable assumptions or leaps, and the 2015 federal election campaign was one of them. Rather than see that as a reason to abandon everything, this unpredictability is something fascinating that tells us a lot about what happened. That's part of the process — experiments that go wrong can be just as informative as those that go right.

Thursday, November 26, 2015

Riding results with a preferential ballot

In my column for the CBC today, I looked at what impact a preferential ballot might have had on the 2015 federal election. Take a look at the full article.

The calculation was based on second choice polling data at the end of the campaign, which had been broken down to the region. So, with that information I was able to estimate which party a Conservative voter in British Columbia, for instance, would mark as his second choice. With that data, I was able to re-run the 2015 federal election in every riding.

Of course, this assumes that all else is equal, including how the campaigns would have unfolded. It also assumes that the regional-level second choice preferences apply in each riding, which of course would not always be the case. So a few assumptions have had to be made. But I think the exercise is nevertheless indicative of what impact a preferential ballot might have had, and what impact it would have in the future if the parties do not change their electoral strategies.

Since I ran this exercise down to the riding level and had the numbers on hand, I decided I might as well share the data with readers here. In the tables below, I've shown the results after going through the exercise. Blank results for a party mean it was dropped at an earlier ballot and its support redistributed to other parties. 

Thursday, November 19, 2015

How the cities went in the federal election

In many ways, Justin Trudeau's victory on Oct. 19 was an urban one, with the Liberals virtually sweeping the urban ridings and winning just enough rural ones to put them over the top for a majority government.

The Elections Canada results page, in addition to showing the breakdown of the results by province, also has the option to show the results in major centres. I'm not quite sure how those are defined (likely any riding within a CMA), and these results are the preliminary results, but they do give us an indication of how each of these major centres voted in the federal election.

UPDATE: The definitions for each major centre can be found here. As you can see, Elections Canada used a wide brush to define these.

In the chart below, I've included all of the major centres that Election Canada defined as having at least three ridings. None of this information is original — but considering the way that Elections Canada has the data organized (you can only see four at a time) I thought it would be useful to compile it all in one place.

The Liberals won big in each of the largest three cities, beating out the Conservatives by 14 points in and around Vancouver and 16 points in and around Toronto, while also edging out the New Democrats by 15 points in and around Montreal. 

But the Liberals also did very well in other big centres, topping 40% in Winnipeg, Hamilton, Kingston, Kitchener-Waterloo, Ottawa, Gatineau, Halifax, Moncton, and St. John's. They also managed to eke out victories in Sherbrooke and Trois-Rivières, the last two some real signals of how the Liberals broke through in francophone Quebec.

Outside of Atlantic Canada, the Ottawa-Gatineau region was the major centre that voted in the largest numbers for the Liberals.

The party also did quite well in the southern Ontario cities like London, Oshawa, and St. Catharines-Niagara, centres they lost to the Conservatives by narrow margins. 

Surprisingly, the Liberals placed second in Quebec City and Regina. Only in Victoria, Saskatoon, and Windsor did the Liberals finish in third place. And it is something to see the party sitting at just under 30% in Calgary.

The Conservatives won the cities in Alberta and Saskatchewan by very wide margins, and managed to squeeze out the Liberals in London, Oshawa, and St. Catharines-Niagara. The party did very well in Quebec City, winning it by 15 points.

The Tories also had strong second-place showings in Winnipeg, Kingston, and Kitchener-Waterloo. Its numbers in Montreal and Gatineau were unsurprisingly low, but the party also put up poor numbers in Atlantic Canada and in both Vancouver and Victoria, though their Vancouver numbers were enough to place them second.

And that is because the New Democrats were pushed out of the urban centres in big ways. The only city they won was Windsor, with just under 39 per cent, and in only St. John's and Victoria (which the NDP narrowly lost to the Greens) did the party manage more than 30% support. 

In addition to those two cities, the NDP also finished second in Saskatoon, Gatineau, Montreal, Sherbrooke, Trois-Rivières, and Halifax. 

The Bloc had poor numbers in the five major centres in Quebec, putting up their best results in Trois-Rivières and Montreal, where they took over 20% support.

The Greens won Victoria by the skin of their teeth (and thanks to Elizabeth May), but nowhere else did better than 4.5% support.