Friday, October 7, 2011

Ontario election - the morning after

Ontario elected a Liberal government last night, but it took until the wee hours of the morning before it could be definitively called a minority government. The Liberals won 53 seats, one short of a majority. It doesn't get any closer than that.

I called a Liberal majority. The polls suggested that would be the case, with the gap being 3.3 points in my weighted aggregation and 4.8 points in the average of the last six polling firms to report. Instead, the gap was 2.2 points. Some pollsters did very well (Forum and Abacus, for example), others did less well. That is how it goes.

And just like the pollsters, ThreeHundredEight will learn from this election and move on. It was, by no means, a failure like that of the 2011 federal election. And it was only a few days ago that I called 98.2% of ridings, 56 out of 57, in Manitoba. Like the pollsters, you lose some and you win some. In this election, my vote projection beat out four of the seven polling firms to report in the last week of the campaign.

Before moving on to how the projection performed, I invite you to read my articles today on The Globe and Mail website about how the pollsters performed, and at the Huffington Post Canada, on where the election was won and lost. Also, a full post-mortem of the Ontario projection will follow in the coming weeks.

The projection model performed adequately, with an 85% accuracy rating on the riding calls and an average error of 5.3 seats per party. In a more clear cut election, that would have been acceptable. In a close election like last night's, that was the difference between a minority and a majority government. But it compares quite favourably to the 13.3 seats per party error in Ontario that I had in the abysmal federal election projection.

The seat ranges, dipping to 51 for the Liberals, did envision this sort of minority result.

But the polls are not to blame for the errors in the precise projection. With the actual results, I would have still projected a Liberal majority of 58 seats, with 30 going to the Progressive Conservatives and 19 to the New Democrats. Why?

There are two reasons for the discrepancy. The first is turnout. At 48%, a drop from 53% in 2007, the question of who votes becomes paramount. Polls showing between 70% and 90% of respondents as decided are going to have problems and seat projections based on past results are going to have problems. Whether this was actually one of the reasons for the errors in the projection is difficult to say, but it would seem to be one likely suspect.

The second is the regional variations in Ontario. As the polls don't divide Ontario up uniformly, I did not have a regional model for the province. In other provinces this might not be a big issue, but in Ontario it is a big problem. I will work towards developing a regional model for Ontario for the next election, whenever it comes.

That it was the regional breakdown that caused the most problems is apparent by the projection's accuracy on a region-by-region basis. The projection called every seat correctly in the Hamilton/Niagara region and 91% of the seats correctly in central Ontario and Toronto. The accuracy rating was 89% in the GTA, still acceptable, while it was 86% in eastern Ontario. In northern Ontario, 9 out of 11 were called correctly.

In southwestern Ontario, however, the projection was only 67% accurate. This is where the projection failed.

Had the polls been dead-on, my projected seat range would have been much better, at 49-60 Liberals, 28-39 Progressive Conservatives, and 18-21 New Democrats. I only would have been off for the NDP by one seat in the ranges. This is where local factors come into play - the projection (except in rare cases, as in Manitoba) cannot get everything right. The ranges are just as important, but just as a pollsters can't report only using the MOEs, I need to make a precise call.

The Electoral Track Record has been updated with these details, and you can see how it stacks up to past performances. Let no one say that I cover up past errors, they are there for everyone to see - as are my successes.

Going forward, I believe that a regional model is necessary for some provinces, particularly Quebec and Ontario. They will be developed. I also believe that ThreeHundredEight needs to be able to predict the vote and not just aggregate the polls. An aggregation needs to be maintained to give a clear picture of what the polls are saying, but a prediction should be more useful in the future. With the Conservatives being under-estimated in every election so far this year, it seems that there is a systemic problem in how people respond to polls. Is the "shy Tory" factor for real in Canada?

I don't consider it too likely that Ontario will be heading back to the polls soon, so there is time to work on these things. Newfoundland and Labrador votes Tuesday and Saskatchewan goes to the polls on November 7. There is still a lot to do! The Newfoundland and Labrador projection will be updated over the Thanksgiving weekend, but in the meantime the Campaign Polling Trends chart has been updated for the province.

24 comments:

  1. Rural Ontario was out to get the Liberals. Most in southwestern Ontario felt abandoned by McGuinty & Co, hence the dumping of two ministers. Can't shove wind turbines down our throats and close up our jails and expect to not pay the price at the ballot box.

    But, since we were largely ignored during the campaign, nobody south of the 401 really knew what was about to happen.

    ReplyDelete
  2. When the voter turn out is no where close to the level of responses in the polls, there is a significant danger that the results of the election could be rather different than what one would expect.

    Based on using data some of the pollsters provided on how likely people were to vote and how firm they were in their choice, I took the NDP level of support to be too high and the PC support to be too low. As it turned out I was still too high for the Liberals and marginally too high for the NDP.

    What would be interesting to know is how much variation there was in turn out in the ridings, and who actually voted versus the census demographics for Ontario.

    In an ideal world all the pollster would release their raw data after the election for polling geeks to go over and try to understand what the poll was saying versus what happened on election day.

    ReplyDelete
  3. As always, I appreciate the tremendous amount of work you put into this project. The model (and the polls!) may never be perfect but your overview and analysis are informative and helpful.

    ReplyDelete
  4. Somebody else on here posited the "rural discontent" factor.

    I think they have a solid point. Driving around Eastern Ontario I'm seeing more and more of the "Back Off Government" signs. It seems the movement is growing and will become more prominent with time.

    So this isn't being polled as it is "nebulous" right now but that influence is out there and active. May explain some of the surprise PC rural vote?

    ReplyDelete
  5. To be a bit of a pedant, Ontario did not elect a Liberal government as Canadians do not elect governments. Ontario elected its 40th parliament, the seat distribution in that parliament allowed for the formation of a Liberal minority government. Given the level of ignorance out there re: how our system works, people need to phrase things more accurately.

    ReplyDelete
  6. Eric - thanks for all the work you have done. I really enjoyed watching the election coverage with a print out of your riding forecast in hand. Overall, some fantastic work.

    Thanks again - I'll certainly be reading your blog for a long time.

    ReplyDelete
  7. Once again, I appreciate the tremendous amount of work you put into the site, as well as your transparency it reporting your methodology and mistakes. I find it interesting how electionprediction.org came away with stunningly accurate seat totals this time (although the specific seats weren't nearly as accurate), coming at the problem from a completely different angle. I think you are right to try to break Ontario down into a more regional model. If we can't imagine how the ridings break down, the overall polls mean nothing.

    Also, I predicted a low general interest in this election, and that proved true with the low turnout. And low turnouts in Ontario would seem to favour the conservative parties.

    ReplyDelete
  8. Even smaller provinces like Nova Scotia and Saskatchewan have a pronounced and significant 'regional' factor to their voting trends - the existence of 'regions' in Ontario is, and has always been, a very significant factor in its elections. I've already seen buzz today about regionalism in Ontario, and I imagine a good amount of the dialogue we'll see in post-mortems will revolve around this dynamic.

    If pollsters had been able to agree on definitions of the varying regions, you would have been able to input regional data into your projections. And if you had been able to do that, I'm sure you would have had much better numbers.

    I don't doubt that next time around everyone involved will bear the regional dimension in mind more than this time.

    ReplyDelete
  9. "With the Conservatives being under-estimated in every election so far this year, it seems that there is a systemic problem in how people respond to polls. Is the 'shy Tory' factor for real in Canada?"

    I want to echo Bernard von Schulmann and say no. This is a pure turnout effect. Thus, public opinion diverges systematically from voter opinion. What is needed is much more explicit information from pollsters about the intentions of likely - probably *certain* - voters, as distinct from eligible voters.

    ReplyDelete
  10. Eric thanks for your tremendous work. Made the election much more interesting. Thanks again.

    ReplyDelete
  11. I'm a 33yr old independent business owner and I did not vote.

    I did not feel like I understood what each party actually stood for vs. what they said they stood for vs. what they will actually, realistically be able to do if/ when elected.

    ReplyDelete
  12. There is a poll out for the Yukon election. Are you going to do a seat projection there!?

    http://www.datapathsystems.net/
    It is far too close to call:
    NDP 35%
    Yukon Party 35%
    Liberals 26%
    Greens 2%
    First Nations 2%
    Based on sample size, the margin of error is +/- 5.1%.

    ReplyDelete
  13. Eric,

    One thing I noticed with my projection was the big divide between rural and urban projections; I found that I got quite a few of the urban ridings correct, even margin-wise, as compared to the rural ridings.

    Is there a way for you to divide up urban and rural ridings, and tweak them accordingly, like you would a regional? That would go a long way to getting more accurate results than we did last night.

    ReplyDelete
  14. Just wanted to say that this is some great work you're doing. FYI I think your riding accuracy is better than 85.0% in Ontario...you were off by a total of 16 seats, and I believe normally the riding error is that total divided by 2. That means your riding error is only 8, and your accuracy would be 99/107 = 92.5%

    ReplyDelete
  15. I also wouldn't be looking for "shy Tories"; they may exist, but probably not in statistically significant numbers.

    This is simply the result of polls polling people who say they're going to vote, and then don't.

    I think in Ontario the turnout would have been higher if there hadn't been a (correct) perception at the end of another Liberal victory. Ontarians in the past few federal and provincial elections are more easily motivated to vote "against" than "for", and I think if there had been a perceived "danger" of Hudak being premier, that people would have turned out in greater numbers. If you think the guy who you're mostly okay with (which is probably the best way to describe the feelings of most Ontarians towards McGuinty) is going to win anyway, why bother with the hassle of voting?

    Following that's a very well-established feature of North American political demographics that elderly white people, particularly in rural (note that Northern Ontario isn't "rural", it's just remote) areas, have extremely strong partisan affiliations (even though they may not self-identify) and vote. Every single time.

    At this point, those people are mostly pretty hard-right, so as turnout goes down, their proportional influence increases, more so in areas where the population is old and white (like, um, southwestern Ontario.)

    This is why the Republican party in the US has an, er, interesting history over the past two decades of proactively trying to depress voter turnout and prevent voter registration on a large scale, as opposed to just the occasional being-dicks-at-the-polling-station that happens in Canada.

    -MikeAbroad

    ReplyDelete
  16. Ed,

    Thanks. I called 16 ridings incorrectly, so it isn't a question of the seat count.

    ReplyDelete
  17. The big question IMO is "How do we get them to vote??"

    ReplyDelete
  18. good work here Eric!
    I think that pundits need to give more weight to the #s who are most likely to vote.
    Also you should project each of the last 5-6 polls on their own as well as your own prediction.
    I think that Forum and Abacus would have projected the minority - although the Liberal vote was very efficient in the GTA.

    ReplyDelete
  19. When it comes to Southwestern Ontario (which we both got completely wrong), I don't think a regional approach would have helped. The swings across the region varied by large margins and I can't find (at least to start) an explanation for it, be it local/rural, incumbency, or anything else.

    For example, in London North Centre, the PCs went up 5.2%, while in next-door London-Fanshawe they went down 1.0%. Similarly, they were up 17.4% in Elgin-Middlesex-London while down 0.1% in next-door Haldimand-Norfolk. Overall, they were up 6.6% in SWO.

    ReplyDelete
  20. DL, that poll would result in 8 NDP, 7 YP, 3 Libs.

    ReplyDelete
  21. Rural issues will only get the PCs so far. If they want a majority some day, they will have to ditch the Tea-Party like social conservative fringe (Hillyer and MacLaren) and reach out to people outside the rural/small town Anglo-German Protestant base.
    If a mainstream centre-right party cannot win prosperous suburban seats like Oakville and Ancaster-Dundas they will have ZERO chance of coming to power, unless they drop the leader and change the ideology.
    As I have stated before, conservatives win when they focus on economy and security; they lose when ugly social issues (women's rights, religious bigotry, abortion, so-called ruralist stuff, homophobia) raise their head.
    Congrats to Ontario PC Party for snatching defeat from the jaws of victory !!!

    ReplyDelete
  22. The exaggerated regional differences in Ontario have hit a new high. Canadians are, sadly, used to seeing exaggerated regional differences in our federal elections. Now Ontario has joined the parade of stronghold politics.

    In two-thirds of Ontario’s electoral districts, Liberal strongholds -- the GTA, Ottawa, Hamilton-Niagara and the North -- it took only 26,000 Liberal voters to elect an MPP. Elsewhere it took 53,156 to do so. In the rest of Ontario, 36 of the 107 districts, Progressive Conservative strongholds, it took 25,667 PC voters to elect an MPP while elsewhere it took 73,858 to do so. In the Liberal stronghold regions it took 44,818 NDP voters to elect an MPP, while in the PC stronghold regions it took 153,966 NDP voters to elect an MPP.

    In the 23 ridings west of the GTA and Hamilton-Niagara, 388,961 PC voters elected 14 MPPs (27,782 votes per MPP), while 325,177 Liberal voters elected 7 MPPs (46,454 votes per MPP), and 215,730 NDP voters elected 2 MPPs (107,865 votes per MPP)

    In the 13 ridings outside the GTA from Barrie to Cornwall and Pembroke, 268,649 PC voters elected 11 MPPS (24,423 votes per MPP), 167,148 Liberal voters elected 2 MPPS (83,572 votes per MPP), and 102,616 NDP voters elected no one.

    In Ontario’s outgoing cabinet of 28 members, nine were from those 36 districts. In last week’s election only nine Liberal MPPs were elected from those districts. Will all nine be in cabinet, including rookie Teresa Piruzza?

    Ontario Liberals need proportional representation more than ever. John Gerretsen has not given up. On Sept. 21, 2011, at an all-candidates meeting in Kingston he expressed renewed support for proportional representation. He was eloquent about the benefits of coalitions, debate, and the fact that his party with 42% should not have been allowed to rule with a majority. (He acknowledged that not all in his party share this view.) This is like what he said in 2007: “Nobody is ever 100-per-cent right and nobody is ever 100-per-cent wrong. Governing is the art of compromise. There’s nothing wrong with having the governing party take into account smaller parties.” Gerretsen is a lawyer, former President of the Association of Municipalities of Ontario (AMO), and the longest serving Mayor in Kingston’s history. He’s not afraid to go after what he wants.

    ReplyDelete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. Please keep discussion on topic.