Monday, September 10, 2012

Quebec election: expectations vs. results

Less than a week ago, the results of the Quebec election came as a bit of a surprise to pollsters and pundits. But the surprise was not in the grand strokes of the outcome itself: as expected, the Parti Québécois won with the Liberals forming the Official Opposition, the Coalition Avenir Québec coming third, and Québec Solidaire winning two seats. The surprise instead came in the details.

A minority result for the PQ was always a distinct possibility, but it turned out to be a lot smaller than expected. The Liberals were best positioned to finish second, but they out-performed expectations to a significant degree. And the CAQ, despite a good haul of votes, did not manage to win as many seats as they might have and a few of their star candidates did not win in their ridings.

ThreeHundredEight's projection gave a slight edge to a PQ majority, but this overall result was envisioned. The problem with the projection was that the Liberals were under-estimated in the polls, which meant that the projected results over-estimated both the PQ and the CAQ.

The Quebec election was not a repeat of the Alberta election. Equating the two would be a silly mistake. The polls in Quebec were generally within the margin of error for the PQ, CAQ, and QS and they only under-estimated the Liberals. The polls from the Alberta election gave Wildrose a 10-point lead when the result was a 10-point win for the Progressive Conservatives. Nothing close to that occurred last Tuesday.

In terms of the Alberta election, the question of "what went wrong?" can certainly be asked. The public polls did not capture a swing in the last week that internal polls did manage to identify. The question that should be asked in Quebec is "why were the results different?"

Polls are often erroneously seen as simply a way to predict an outcome. They are, of course, a snapshot and things can change - but voting intentions rarely shift to a significant degree in a matter of days. That is why results usually match the last polls of a campaign. But polls are most useful as a way to understand and contextualize what is going on during an election campaign. After the election is over, they can be a very good tool to understand what happened. And projections based on polling data do the same.

Of course, polls and projections will always have a degree of error and I will take a look at the errors the projection made below. But working from the idea (which has been proven over time, notwithstanding the few exceptions) that in a normal election polls can track the voting intentions of the population accurately and that projections can turn those numbers into accurate seat counts, a comparison of polls/projections to results paints a very good picture, not of how polls and projections failed, but of what worked and what didn't for the parties themselves.

For a full statistical breakdown of the projection's performance, I invite you to take a look at the Electoral Track Record.

How would the seat projection model have worked had the polls been bang-on in each of the model's six regions? Polls for three of them (Montreal island, Montreal suburbs, and Quebec City) were available throughout the campaign while polls for the other three (Eastern, Central, and Western Quebec) were available for some of the polls.

The model would have awarded the Parti Québécois 53 seats, the Liberals 42, the CAQ 28, and two to Québec Solidaire. In other words, a strong result for the PQ and QS and an under-estimation of Liberal strength to the benefit of the CAQ.

The riding accuracy, at 81.6%, would not have changed but the potential winners would have been identified in 94.4% of ridings when using the seat projection ranges.

Those ranges (using the polling volatility in the last week of the campaign as a guide, as the model normally does) would have encompassed the final result: 41-65 for the PQ, 34-55 for the Liberals, 18-38 for the CAQ, and 1-2 for QS. The projection would have called for a PQ minority, a strong Liberal opposition, and a third place finish for the CAQ, while the odds would have been considered very unlikely for a PQ majority or a CAQ second-place finish. This would have been a good call.
So the projection model would have been able to forecast the election properly with accurate polling - but nevertheless the most likely outcome would have still under-estimated the Liberals to a significant degree. Why?

This is where the results of the election and how they differ from polls and projections can tell us something about what went right for the Liberal campaign. In the Alberta election, to take one example, the projection model would have done very well with accurate polls with one exception - the extraordinary resilience of Liberal incumbents in Edmonton and (especially) Calgary. With the Liberal vote dropping so steeply in Alberta as a whole, no projection model that wasn't based on a hunch could have forecast that the Liberal incumbents in these two cities would prove so difficult to topple. While that was an example of the projection model being wrong, it said more about the Liberals in those ridings than it did about the model itself.

Region vote projections vs. results
The chart to the left shows the difference between the projection and the results in each of the six regions. We can see that the Liberals did significantly better than expected in Montreal (though it did them no good in terms of seats), in Quebec City, and central Quebec.

The performance of the Liberals in central Quebec was quite remarkable. Polling data that was available for the region in the mid-point of the campaign, when the Liberals were far from out of the race, showed that central Quebec was not a strong part of the province for the party. It put them at 27%, behind the PQ and the CAQ. As their vote tanked in the polls in the regions of Quebec and among francophones (not to mention the riding polls that showed the Liberals with little support in Nicolet-Bécancour, Saint-François, Saint-Maurice, Sherbrooke, and Trois-Rivières), the projection put the Liberals at between 23% and 29% in the region. They ended up with 32.2% of the vote, beating out the CAQ (30.9%) and the PQ (28.4%).

As a result, this was the region with the most amount of errors and the region that skewed the projection against the Liberals most importantly. Their vote in this highly francophone region turned out in great numbers, voting for their incumbent MNAs in particular. Along with a surprising degree of resilience in the Montreal suburbs, particularly Laval, this was the region of Quebec that 'saved' the Liberals from a far more significant defeat.

Seat projections vs. results
But what happened to the Parti Québécois? They were over-estimated almost across the board, with eastern Quebec being the only region where they out-performed expectations. The seat ranges were only off by three seats for the PQ, however, and that can mostly be attributed to the strong Liberal performance in central Quebec. But it can also be blamed on the PQ's under-achieving results in the suburbs around Montreal.

The PQ was projected to take between 38% and 42% of the vote in this region, but ended up with 35.9%. Neither the Liberals nor the CAQ were outside of their projected ranges in the suburbs, but both were at the higher end of expectations. The 450 was considered the make-or-break region for both the PQ and the CAQ, and in the end this was correct. The PQ won 15 seats instead of the projected 19, putting them at the very bottom of their expected results. They could have won as many as 24 seats in the region, enough alone to have given them a majority government.

For the CAQ, they did as well as expected in and around Montreal and Quebec City. The projection gave them seven seats in Quebec City, which they indeed won, and seven in the suburbs. They took six.

But it was in francophone Quebec outside of the two main centres where the CAQ vote did not materialize. The party won no seats in eastern Quebec instead of the projected one, one seat in western Quebec instead of the projected three, and five seats in central Quebec instead of the projected nine. But, generally speaking, the projection was quite good in estimating the CAQ's vote - and the projection model would have still given them nine more seats than they actually won if the polls had been completely accurate.

Why? The CAQ's under-performance is one case where the error that the projection made tells us something important about the CAQ. The CAQ's vote share aligned quite closely with the polls and the projection, but their seat result did not.

The reason for this is that the CAQ's vote was far more uniform than the ADQ's. If the ADQ had gotten 27.1% of the vote in this election, they very likely would have won around 28 seats. But despite the CAQ having swallowed up the ADQ and its MNAs, the party is not the same. It does use the same populist and vaguely nationalist language of Mario Dumont, but the party and its leader take a more centrist approach. That changed the party's voter profile somewhat, enough to give the CAQ relatively even support - particularly in central and western Quebec. The model expected the CAQ's vote to be more heavily concentrated in central Quebec to the detriment of the western part of the province - instead, the CAQ's vote was more uniform and that meant a lot of second place finishes.

When the next election occurs, which considering the minority government in place could not be very long, the model will undoubtedly perform better because it will be able to use the CAQ's vote as a baseline, rather than the ADQ's.

Another factor which made the projection over-estimate the CAQ was the adjustment made for the floor-crossers in Blainville, Deux-Montagnes, and Sanguinet. While the adjustment would have worked in most elections, for whatever reason it did not work in this one. It over-estimated the support of Daniel Ratthé, Benoit Charette, and François Rebello to a significant degree. Without the adjustment in place, the model would have correctly given Blainville to Ratthé but Deux-Montagnes and Sanguinet to the PQ. The results for each party would have been quite accurate - indicating that the floor-crossers did not personally bring any large number of  votes with them.

If we look closely at the results, the projection did well on some of the minor details. Québec Solidaire's results were projected very well, with the result of 12% on the island of Montreal (their most important region) being off of the projection by 0.1 point. The model also did quite well with Option Nationale, the Greens, and the other parties.

The projected ranges were correct for every party in some tough-to-call ridings as well: Jean-Lesage (correctly forecasting a close three-way race), Groulx, and Gouin, among others. Generally speaking, the model did quite well on an individual basis for each party in the ridings where the winner was called correctly. The ridings that were not called correctly often had very large errors, suggesting that fiercely local circumstances were at play in many of them.

But the turnout adjustments may not have helped matters. While it did correctly assume that QS would be over-estimated and the PLQ under-estimated, and the adjustment did not effect the projection for the CAQ significantly, it was wrong to consider the PQ under-estimated in the polls. The last surveys by CROP and Léger were excellent for the PQ, and the last surveys by Forum and EKOS over-estimated the PQ greatly.

The model made a bet that, in most elections, would have been correct. Going forward, I am not sure if this is a bet that is worthwhile to make. The adjustment seems to have caused a lot of confusion during the campaign, with readers not understanding why I had the CAQ so low and the PQ and PLQ so high, no matter how many times I explained it. Perhaps it would be best for ThreeHundredEight to limit itself to poll aggregation and what those polls would deliver in terms of seats, with the high and low ranges being used to estimate how the polls are expected to be wrong. I welcome suggestions from readers.

With the surprising performance of Jean Charest's Liberals, the Quebec election threw almost everyone for a loop. But the results might have had more to do with the Liberals' stronger organization and the mythical prime à l'urne playing a role than the errors of polls and projectors. Quebec's distinct society does seem to come with distinct elections - no one will be surprised if next time the result is, well, a surprise.

34 comments:

  1. Thanks for all the hard work, Eric. A new party rising will always cause problems, especially when the party swallows up, but does not become, a former party as was the case with the CAQ/ADQ.

    I do think some of the pollsters are going to have some 'splainin to do with regard to how their numbers were so wildly off. I'm thinking particularly of EKOS here, who completely missed the boat.

    ReplyDelete
  2. Solid analysis, Éric.

    With respect to your comments regarding the model's assumption that the incumbent and official opposition are underestimated while all others are overestimated, I admit I felt a bit skeptical about it throughout the campaign. While I have no doubt that you thoroughly analyzed plenty of provincial and federal elections and found that more often than not that's what happens, I think the unique circumstances surrounding any given election are such that it simply may not be safe to apply it as a "universal" rule. I find it frankly impressive how many factors you take into consideration in your model (e.g. floor crossings, star candidates, record of pollster accuracy, etc.), but perhaps this one crosses the subtle boundary into overanalysis.

    One thought I had was that perhaps the "penalty" for a third party should cease to be applied in cases when an important surge propels such a party into second place in the polls. For example, such a rule might have have helped you better project the results for the NDP and Liberals last federal election. But then again, just how much of a surge is required before the formula gets changed is debatable: in the QC election most polls near the end showed the CAQ ahead of the Liberals, though never by more than 2 points, yet the Liberals ended up decisively ahead of the CAQ on election night. So again, maybe it's simply too risky to try crafting any sort of universal rules pertaining to this phenomenon.

    Cheers,

    Dom

    ReplyDelete
    Replies
    1. Thanks Dom. I think you might be right. I may go with the polling aggregate as the official vote projection, but use a different system for determining the ranges based on what has happened in other elections, rather than polling volatility.

      I like the idea of using polling volatility as a guide, though. Perhaps I should investigate it more deeply and see if I can come up with something a little more robust.

      Delete
    2. That seems like a pretty good idea, though I think it's worth noting that there have been notable cases recently where polls were in agreement and still off. The PLQ support is a good example here...

      How familiar are you with Monte Carlo simulations? I believe that's how 538 approaches its ranges.

      Delete
    3. All of your sensible readers understand that you're ultimately restricted by the accuracy of the polls, so in that sense the very safest way to go would be to stick strictly to averaging the polls and using volatility for the ranges. That would leave just about no reasonable opportunity for anyone to criticize your work. That being said, I do think many of the extra assumptions you make are canny and worthwhile, and after all where would be the fun and intrigue if you completely refrained from adding any insight into your methods? I guess the challenge is to maintain a cautious balance, knowing that the more "twists" you incorporate, the more you open yourself up to criticism whenever the projections turn out wrong (not that I don't think you can handle it!).

      Dom

      Delete
  3. As said, that a nice analysis and good critical thinking, you didn't hide behind some excuse.

    Quebec Province is not one of the easiest political ground to analyze. I think effectively you should drop the "big party" get a bonus and "small party" are overestimated and you should if you want to keep some precision try to go with the history of party. The PLQ have a tendency to be underestimated, the PQ doesn't not. I would have bet that the PQ would end up a bit lower in absolute votes, but still, my point hold. You generalized a rule that might be true in 90% of the case, but wasn't true for one of the major actor this time.

    Or if you don't want to do such thing, you can try and correct slightly your predictions with what neutral journalist report from region to region. Anyways, I expect you to fallow elections with some attention and maybe interest.

    It's not "pure" science, but it certainly help. Like our old relative that can quite accurately predict weather, that ain't fool proof science but if you listen to them and to the meteorologist you can get a clearer picture.

    Such information, it might give an edge in fierce race, push up a bit the top range for seats, or lower the bottom, nothing major, but such small adjustment would had made your model correct on the election night, on the extreme, but you would have probably catch the small nuance.

    Thanks again and keep up the good job.

    ReplyDelete
  4. The problem is the polls just aren't accurate enough, and I don't see that changing in the future. No matter how much you tweak your model, there's little you can do if the main input is crap. Garbage in, garbage out.

    Polling companies are having an increasingly hard time getting representative samples. Cellphones, do-not-call registries, and a reluctance to answer polls honestly or at all are contributing to less accurate polls. I've never once been polled, and I don't think that's an uncommon phenomenon with young people who have never had a landline.

    I enjoy reading your projections, but it scares me when MSM reports it as fact, because the error rates are so high.

    ReplyDelete
  5. Regarding the confusion caused by the prediction of the outcome, I think you have to be more explicit. If you take the pool as is, your model is always going to be wrong. But when you made your projection, you should indicate how much is add or subtract for each party based on what.

    For all the campaign, I did not understand why you where giving that much of a bonus to the PQ. I think it was because it was the runner and you where ignoring previous election.


    I think you should develop a model that consider size and position compare to equivalent party in other election as factor 1 and difference in previous election for this party as factor 2. By having the pool result, the value for each factor and the result in a complementary, it would clear the confusion and improve your model.

    ReplyDelete
  6. With regards to limiting the scope of the site to poll aggregation and a range of seats, I think that's a good idea. The focus on attempting to project individual seat off faulty data has, I think, detracted from the site's accuracy, and it is, in any case, fundamentally impossible to predict individual seat results from regional, provincial, or national polls alone. I've been uncomfortable for a while how people use this site and others to inform their strategic voting decisions (often citing the riding projections as "polls"), and the focus on forecasting individual seats probably leaves your readers less informed.

    If you take a bird's eye view of the kind of work you've been doing and the projections you've been making, some of them are outright silly in how authoritative they've been. Your Rothesay by-election projection, for example, nailed down the PC vote within a margin of +/- 1%, or your PEI projection that projected the results to +/- 1 seat. I'm sure that, if you had been asked for your personal opinion on the day prior to the election, you wouldn't have bet much money on the results actually falling within those ranges, but those are the ranges you published as projections on the site. You've generally been pretty good at adding caveats to those projections in the text, but you've still been releasing projections that you yourself could not possibly believe were entirely reliable.

    I think there's a very legitimate function for a site to analyse polls and regional shifts in support and how those could translate to seat numbers, and you've shown that you're pretty well equipped to do that. I'd really like you to move away from the microlevel riding projections that are impossible to get right with any regularity.

    ReplyDelete
    Replies
    1. I, too, am uncomfortable with how the projections for individual ridings by this and other sites have been used in the media and for strategic voting.

      But I do not focus on individual riding projections. I only mention them when analyzing a riding poll. I present the individual riding projections for two reasons: 1) people want to see them, and if I didn't post them I would be inundated with requests, and 2) for transparency. I have to show my work, in effect. I disagree that they are a focus of the site.

      And I think you are wrong to suggest that individual riding results are impossible to predict. The projected ranges before the election identified the potential winner in all but 18 ridings. With accurate polls, it would have done so in all but seven. And in the Manitoba election, the projection called 56 of 57 ridings correctly. That a model can predict 85% or more of seats correctly is no accident, there is a degree of predictably to electoral results.

      I always include a lot of caveats and I always explain that there is an element of uncertainty to the projections - but there is only so much I can do. Many people will ignore those caveats and explanations no matter what efforts I make.

      And I think you are assigning a level of certainty to what I forecast that I myself have never expressed. I am honest with my readers by giving the results of the model, and expressing when I think they might be wrong.

      Perhaps there are changes I can make to address your concerns, however. I could post simply the ranges for each party in each riding and identify the party considered the winner by the model. Or I could just identify the projected winner and any party within range of winning the seat.

      I would be interested to hear what others think. But if I were reading a site like this, I would always want more information instead of less. I have never run this site with an eye towards the LCD, and I think it would be diminished if I did.

      Delete
    2. Oh, I definitely agree that a model like the one you're using can identify and predict with some accuracy in which seats a party is more likely to win, and I think that the "potential winner within projected ranges" standard is exactly the one to use. It's telling that you use the Manitoba example, because it's the only one of the last 9 elections in which the results matched your projected seat ranges. The other 8 were wrong. I contend that this isn't out of any sort of malpractice from your part, but simply because elections are in fact hundreds of small-scale contests and the factors at play - candidate quality, campaign quality, particular local issues - disappear when the only data you have is an aggregate of twenty of more seats. Past election results + current polls are a good predictor of future results, but not good enough to make seat-by-seat projections with confidence.

      Now, people were curious as to your seat-by-seat projections back when they were hidden, and I'm well aware of that - I'm pretty certain I was one of the people harassing you once in a while to get a particular result or another. I did it knowing the limitations of what you're trying to do, though, and knowing that it was basically a bit of fun. We both know that people are taking them more seriously than that, though, including journalists looking to them to know how Charest was doing in his riding from poll to poll. I don't know what the answer to that problem is, though. Maybe releasing the riding by riding numbers your model projects outside writ periods, but keeping them to yourself during the writ?


      You wrote that you thought I was assigning a level of certainty to your forecasts that you had never expressed, but I think that's wrong. You certainly always nuance your projections in your posts, but how you present your data matters, and influences how people think of it. When you presented your final Québec project, the top of the page - the number you're presenting, the one people will come away with - wasn't a range with a top end and a low end and a confidence interval. It was a pie chart with fixed number - the PQ is projected to win 63 seats, boom. Your headline wasn't "PQ most likely to win more seats", it was "Parti Québécois victory". You do moderate the language and present more nuance later on, but this stuff matters. If you want your readers to understand your projections results are in fact a range of possibilities and not a fixed number, you want to lead with the nuance, not make them dig for it.

      When I go to 538.com, the second number I see, and it's right up top, is that Obama is an 80% favourite to win reelection - that tells me there's uncertainty in the prediction, how much of it Nate Silver believes there is, and that things can change. The first number I see is a number with the average number of EVs Obama would be expected to win, and the fact that it's not a whole number tells me that there are several possible scenarios. It's impossible to look at state-by-state predictions on without intuiting that Nate Silver is projection probabilities, not results. I think you could make a better choice of what data you present and how you present it to better convey that there are a range of possible outcomes, not just the one outcome at the top of the page.

      I'd essentially just like to see more concessions to uncertainty and the limitations of the model in the way the data is presented.

      Delete
    3. Joffré,

      You make a good point about how the data is presented, and I will take greater care in emphasising the nuance going forward. It will undoubtedly save me some headaches after-the-fact. But I disagree with you somewhat on how I handled this election - the headline of my final projection was "Parti Québécois victory" because that is what the numbers said would happen. I didn't say "PQ majority" because they didn't say that the PQ would absolutely win a majority. I said the PQ would win and they did. I don't think, when my projections show a clear result, that I should be afraid to make a clear call. It would have been silly in the NL or SK elections not to forecast a PC/SP majorities because 'anything can happen'.

      As to 538.com, I think you'll see that when the election approaches that degree of certainty will decrease. On the eve of the 2008 vote, he had a 100% chance of Obama winning, if I recall correctly. He is forecasting the result on Nov. 6, whereas I forecast the result if an election were held immediately. And you'll see that he still does project the popular vote in each state - sometimes based on nothing but his own model's "fundamentals". His model is based very strongly on probability. If I had a similar model, I would take similar steps.

      But I take your point well, and will try to emphasis the uncertainty to an even greater degree than I have in the past. I made a lot of changes to that effect for the two 2012 elections already.

      Delete
    4. 538 is lucky too in that there is much more frequent and broad polling in the US, and has been for a while. He can look back at decades of elections (that are held every two years) and build a statistical model from a huge number of polls and results.

      Delete
    5. They had a 98.9% chance of an Obama win, with 1.1% for McCain btw. Just dug it up. So yah, very high certainty.

      Delete
    6. And 538 is lucky in that there are only two parties (Bernie Sanders excepted)...

      Delete
    7. Yah that certainly simplifies the mathematics.

      Delete
    8. I'm sketching out a new way to calculate the ranges that, if I decide to go with, I will present soon.

      Generally speaking, the projection would have two tiers of ranges. The projection itself would be mildly adjusted when it is deemed necessary (i.e., an adjustment for parties that do not have seats in the legislature is one).

      The first tier would be based on the margin of error of the polls used in the projection. In other words, it would be a reflection of the likely results if the polls are right (and the results are within the MOE).

      The second tier would be a forecast of future events based on how support for a party has been shifting in the past, not dissimilar to what is current in place but with a greater degree of uncertainty.

      I think that such a model would be a good recognition of the fact that there are a lot of unknowns and intangibles in elections, but all of it based on some actual data. I'll have more soon.

      Delete
  7. Long-time reader, first-time writer--I'd be sorry to see this site cut back on content in any way, because I think all of it is valid information to put out--and you're reliably careful about the value of what you're analyzing.

    It still seems like we're at a loss explaining those quirks in polling accuracy, most famously "shy Tory" syndrome, that slightly upset the applecart. It seems to me (and I might be wrong) that there seem to have been regular retreats to safe, and usually incumbent, choices right at the end of campaigns, most obviously in Alberta. I know you account for incumbency in your projection, but it seems a certain percentage of voters, when it comes down to it, never quite feel ready for the change they'd thought attractive, and told pollsters about.

    Reno

    ReplyDelete
  8. Why not publish two models--one adjusted for turnout expectations and one unadjusted? I understand that would require extra work, but it would, in effect, provide two possible takes on the results. It would also help a lot of people understand what the turnout/mobilization adjustment actually does relative to the polls.

    As for the strategic voting issues...while I understand your discomfort, people interested in voting strategically or advocating strategic voting are always going to find some standards for assessing which ridings are competitive and which parties to vote for to stop other ones. If no one did any sort of riding estimates, people would make arguments from the last election results, for example. Others might use the number of signs up for a given party as a cue. So I wouldn't stop posting riding projections. I consider transparency worth the price of people (ab)using your projections for strategic voting purposes, particularly when people who want them or want to broadcast them are going to find strategic voting cues one way or another.

    ReplyDelete
  9. Eric -- don't beat yourself up too much over this last election result. I listened to dozens of commentator, pollsters, and unaffiliated voters throughout this campaign and not one of them said they thought, for instance, that the PQ and the PLQ would end up nearly tied come election night. There is no way to predict the unpredictable.

    After pretty solid results from the provincial votes in 2011, I think your models are becoming better and more sophisticated just at the same time as the polls are becoming worse (hopefully just temporarily). So my advice, stick with the vote projections -- don't move over into just poll aggregation. Otherwise, keep up the great work.

    ReplyDelete
  10. I believe for future Quebec elections it would we wise to formulate a new model that incorporates the language makeup of the ridings and the distincting between anglophones and allophones. (the former being much less likley to vote for anybody other than the liberals)

    ReplyDelete
  11. Hi Eric:

    Well done! Thanks for your analysis of both the Quebec election and the two provincial by-elections in ON.

    Any chance you'll try and do some work on the US Presidential election?

    Thanks muchly,

    Earl

    ReplyDelete
  12. If Eric will take a tiny piece of advice?

    Stay well away from US Presidential election !!

    For one thing polls don't matter as the actual voting polls are run by the political parties not some non-political commission as here !

    And the aroma just gets worse as you dig deeper !

    ReplyDelete
  13. I won't be touching the US presidential election. I couldn't hope to come even close to the kind of analysis Nate Silver does, so I won't try!

    http://fivethirtyeight.blogs.nytimes.com/

    ReplyDelete
  14. Eric,

    Presumably when you did your adjustment for the popular vote numbers you did a regression and got an R-squared and standard error value. Have you thought of using that as a basis to estimating the range?

    ReplyDelete
  15. You guess. And you are trying to become a media hotshot who if taken seriously, like Nanos, will have far too much influence on the narratives that circulate during elections. I wish you would go away.

    Arthur Cramer, Winnipeg

    ReplyDelete
    Replies
    1. Thanks Arthur! But I'm not going anywhere. The narratives that circulate are merely based on what the polls say - so if you have a beef with what polls reveal you should take it up with them.

      Delete
    2. Here's one person hoping you stay just where you are.

      Delete
    3. I don't think pollsters like Nanos has, or wants that much of an influence on narratives. I think it's because the media is abusing their power, trying to make a poll sound like such a major news story because they have nothing better to report. In the age of the internet, the roles of journalists are being dumbed down. Whereas before journalists would meet and connect with people to get the real story, now most of them just go online and google. This has created a disturbing decrease in quality of reporting, and generates more untruths and semi-truths. Unlike what the media dictates, polls should never be viewed in absolute certainty because of the fact that real people can change their minds instantly.

      Delete
  16. Well, that is simply not the case. For example, whenever the NDP is leading the Tories by the margin of error, the narrative is Tories and NDP in statisical tie. When its the other way around, the narrative is Tories lead NDP. Public opinon polling is being conducted with the intent of misleading the public and to create a band wagon effect. They contribute nothing to the public discourse, and simply servie to confuse the populace as the real core of many issues simply is ignored in the rush to get people like Nanos on the air. I think polling is undermining the democratic process as it is being manipulated without the MSM calling it as it occurs. By the way, I would feel this way if, as a NDPer, the shoe was on the other foot. Public polling is undermines democracy. You were way out on your call in Quebec, I can't see how you can refute that. Your use of this venue undermines the public discourse. I stand by my comments, I wish you, Nanos and the rest of your bunch would go away.

    Arthur Cramer, Winnipeg

    ReplyDelete
  17. Yes, the MSM, Nik Nanos, and myself are colluding in a vast conspiratorial plot to undermine the public discourse to the benefit of the Conservatives.

    I think you'll see that, when a poll puts the NDP in the lead, I have always said just that.

    Like here:

    http://www.threehundredeight.com/2012/06/majority-support-for-federal-ndp-in.html

    Or here:

    http://www.threehundredeight.com/2012/06/three-federal-polls-25-ndp-leads.html

    And here:

    http://www.threehundredeight.com/2012/06/130-seats-for-ndp.html

    Not to mention here:

    http://www.threehundredeight.com/2012/05/ndp-leads-could-win-plurality-of-seats.html

    And also here:

    http://www.threehundredeight.com/2012/05/ndp-leads-conservatives-by-three.html

    But I'm afraid these links might disagree with the narrative you yourself would like to construct, so I imagine you will ignore them.

    ReplyDelete
    Replies
    1. It's kind of sad that after 6 years of Tory rule, public cynicism of the political process has reached an all time peak, where it has even reached pollsters.

      Delete
  18. Frankly, I'm just surprised that people are so bothered - even panicky - about the imprecision of these polls and of projections based on them. Did I miss an article in Nature or some other science journal where it was announced that human behaviour is conclusively quantifiable and predictable? We're talking about statistical analyses - and not of variables that are in themselves uncomplicatedly quantifiable and measureable, like distribution of blood types or something, but of attitudes and preferences – phenomena that are the product of myriad and often conflicting variables, phenomena that are changeable – often radically so. I have the impression that some people here have a sort of religious devotion to polls and to Éric’s projection model. It’s unjustified. Éric’s work is interesting, and it provides a kind of reference point, as do the polls themselves, but you’re fooling yourself if you think they’re genuinely predictive. Even if Éric projection for the Quebec election had been right to the individual riding – or vote! – it would still not be predictive because people vote the way they do (or don’t vote at all) for far too many reasons that are not and cannot be statistically represented. If Éric applied this hypothetical 100% projection again, it would not perform perfectly a second time. That is why there are university departments of statistics/demographics and of political studies. Understanding what is happening with human behaviour is unlikely to be a genuine science for a very long time.

    ReplyDelete
  19. Unless we switch over some kind of true proportionnal voting system, sites like these will be immensely useful to get a grasp on the impact of fluctuating public opinion. I don't think the track record of this election is so wrong. Factors may need tweaking, some to be retired, other added, but it is part of the game and you're building something.

    I think you should stick with a vote prediction model. In this election, I can recall at least three other sites doing election projections. If you limit yourself to polling agregates, how 308.com will be different from these ? Your site have done enough coverage to build a sound prediction model. One that cannot be always right on the money, but still, don't let that expertise withered because some people disagree with your practices. This said, you should be open to insight that can help perfect the model.

    As long that you stay transparent on the factors taken in account and the fact that the numbers are simply projections loaded with uncertainty, I think this site will remain great.

    ReplyDelete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. Please keep discussion on topic.