Monday, February 28, 2011

From pre-writ polls to election day, history is not on Ignatieff’s side

Whenever a party is confronted with a yawning gap between themselves and their main rivals in a pre-election poll, it is usually waved off because “campaigns matter.” And they do. Campaigns give parties the opportunity to present a platform to Canadians and give voters a chance to get to know the leaders and their local candidates better. Campaigns also tend to shift pre-election polling results a great deal.

You can read the rest of the article on The Globe and Mail website.

Ontario and Quebec the keys to Tory gains

You can also read my column for The Hill Times here. A subscription is required, so why not subscribe?

It's a comparison of the four polls in early February (from Nanos, Ipsos-Reid, EKOS, and Harris-Decima) to the last time all four polling firms were in the field at the same time in early December.

I will be posting later today about Angus-Reid's new monster poll, so stay tuned!

17 comments:

  1. Interesting piece by Chantal Hebert in today's Star on this subject,.

    ReplyDelete
  2. Eric,

    Usually you are pretty careful about confidence intervals, but I don't think you did it this time. Based only on nine events (elections) your analysis produced probably as many conclusions - which statistically speaking makes no sense. Looking at it differently, you are probably right in only a few of your conclusions, given the large degeneracies in the analysis.

    With such a small sample, I wouldn't dare making more than one (may be two), most obvious conclusions. I think the only robust one is:

    - Campaigns do matter

    Going into smaller particularities (like - "campaigns matter more for Conservatives than for Liberals") pushes the analysis toward total unreliability).

    ReplyDelete
  3. First,

    This wasn't a statistical analysis. It was more of a overview of what has happened in the past. That's why the accompanying graphic focused on the historical record, rather than any "trends".

    I didn't argue that "campaigns matter more for Conservatives than for Liberals", I merely pointed out that the Liberals have tended to lose support during campaigns, while the Conservatives have gained over the last nine elections. I'm not the first to identify this tendency.

    And that shifts tend to take place between the Tories and the Liberals is there in the history, but also in common sense. I just added some numbers to what is commonly believed.

    I would have been able to drill further back than 1980, however I didn't want to go back further than a generation or two.

    But perhaps more of an emphasis on the history would have been better.

    ReplyDelete
  4. Eric,

    This may have been you intention, but given that you normally write articles dealing with reasonably robust statistics (polls, projections), the article (full of numbers) does create an impression that all these numbers are statistically sound, which they are not. But may be it is true only for political junkies like myself :)

    ReplyDelete
  5. They may not be statistically significant, but they are true!

    ReplyDelete
  6. If something is not statistically significant, it is probably _not_true_ (or may be it's true; actually, nobody knows).

    ReplyDelete
  7. My anecdotal observation - both at the federal and the provincial level - is that the incumbent party has a tendency to have its support erode over the course of an election campaign - and since most of the time in Canada, the Liberals have been the incumbent party - they are the ones whose support has eroded. There are exceptions to be sure.

    I think the reason why governing parties tend to lose support during a campaign is that an election campaign levels the playing field. Before the writ is dropped, the party in power has complete control of the agenda and tends to always be making news and vast sums of tax-dollars promoting itself. Once a campaign begins - spending limits come into play and sudden;y all the parties are spending about the same amount and also you have three opposition parties all attacking the government, night after night after night.

    ReplyDelete
  8. I think one thing that is not being taken into account the fact that the election period is also greatly influenced by situational factors. The breakdown of Mulroney's conservative coalition, coupled with the bad campaigning of Campbell's team and the Chretien "bells palsy ad" all contributed to their disastrous 1993 loss. The 1997 result can be explained in the same way. The government lost support in MB because of the timing of the election (after the Red River Flood), and in Atlantic Canada because of their cuts to EI. Reform and the NDP were able to attack the government's record during the campaign and gain in these areas.

    I think this has more to do with situational factors and the opposition making gains during the campaign period at the expense of the ruling government than it does anything else.

    Something else to consider, however, is that in political polling in British elections, pollsters have coined the term the "Shy Tory Factor" (http://en.wikipedia.org/wiki/Shy_Tory_Factor), in which Conservative numbers were understated in the polls, because people didn't want to admit to supporting the Tories, but these numbers came out in the election, increasing the level of Conservative support. That could also be a factor in the Conservative numbers.

    ReplyDelete
  9. FHS Political Forum,

    Regarding the shy tory factor: it may be true for straightforward polls, but Eric has done a significant work in building a prediction model which was fine-tuned to match real measurements (past electoral results), which should remove biases like "tory shy" etc.

    Eric, did I get it right?

    ReplyDelete
  10. If something is not statistically significant, it is probably _not_true_ (or may be it's true; actually, nobody knows).

    Not quite. It is true that, over the past 30 years, the Conservatives have improved over their pre-writ numbers while the Liberals have not. Statistical significance has no meaning there, as those are not probabilistic result - they did happen. The question is whether those results are part of a underlying pattern or whether or not they're random. The sample size may not be large enough to draw statistically meaningful conclusions, but that isn't likely to be much comfort to Liberals looking at those numbers, which I think is Eric's point.

    It is possible to make a fetish about statistical significance, as opposed to real significance. You see this often going the other way, where people report results that are statistically significant, but practically meaningness (saying "Drug A has a statistically significant impact on life expectancy of cancer patients" doesn't mean much when that increase is one week). Similarly, if every couple of years you were mugged walking down street A, you'll probably stop walking down street A, notwitstanding that your mugging may well be a random event (i.e., you're just as likely, statistically, to be mugged walking down street B).

    ReplyDelete
  11. I have to lean in the direction that First is going here.

    Whenever you say something akin to 'in the past 9 times, the result was ABC 6 of those 9 times', it implies predictive power.


    That's why people cite them; indeed that's the suggestion in the headline 'not on Ignatieff's side'.

    Suppose the Liberals have historically improved their poll numbers over a campaign if the campaign began on or near a full moon. That might be a true fact.

    But that doesn't make it meaningful, much less useful as a predictive tool.

    ReplyDelete
  12. First,

    Yes, that is right. My model accounts for polling error in past elections.

    Carl is correct, as it is an irrefutable fact that the Liberals lost support in 8 out of 9 elections compared to pre-writ polling, and that the Conservatives gained in every election since 1980 except for the 1993 election.

    That is what I mean when I say everything in my article is true.

    ReplyDelete
  13. Regarding the "shy Tory factor" in the UK. In last year's election it was replaced by the "shy Labour factor". It was the Labour party that did better than any of the polls predicted - and the speculation was that because Gordon Brown ran such a dreadful campaign and because Labour was so unpopular after 13 years in power a lot of traditional Labour supporters were embarrassed to say they would still vote Labour. There was a "shy Tory factor" in previous British elections because in those elections it was the Tories who had really bad leaders who ran really embarrassingly bad campaigns that no one wanted to admit they supported.

    Right now in Canada, I suspect that if anyone is "shy" about who they support, its probably people who usually vote Liberal but are afraid to admit to it because Ignatieff is such an object of ridicule.

    ReplyDelete
  14. "Something else to consider, however, is that in political polling in British elections, pollsters have coined the term the "Shy Tory Factor" (http://en.wikipedia.org/wiki/Shy_Tory_Factor), in which Conservative numbers were understated in the polls, because people didn't want to admit to supporting the Tories, but these numbers came out in the election, increasing the level of Conservative support."

    No, that is because older people & those with higher incomes vote are more likely to vote than younger people & those in the low income bracket.

    ReplyDelete
  15. "Right now in Canada, I suspect that if anyone is "shy" about who they support, its probably people who usually vote Liberal but are afraid to admit to it because Ignatieff is such an object of ridicule."

    That's sort of hard to buy. I mean, are potentially Liberal supporter really more embarassed to admit they support Iggy than they were of Dion (who's polling numbers from most pollsters at the end of the disastrous 2008 campaign were almost spot on with his actual numbers)? Iggy hasn't done a great job (so far) but I don't think he's been quite a bumbling as his predecessor (so far).

    In any event, it's one thing not to admit you're a tory because others are embarassed by your party, it's another all together to not want to admit your a Liberal because YOU'RE embarrased by your party. It's a lot easier to vote for someone others dont like than it is to get out and vote for someone YOU don't like.

    ReplyDelete
  16. "Suppose the Liberals have historically improved their poll numbers over a campaign if the campaign began on or near a full moon. That might be a true fact."

    That would be a true fact, but only if the only time a campaign began on or near a full moon over the last 30 years was in 1993.

    Again, you have to be cautious about making a fetish out of statistical significance. With large datasets, statistical significance is meaningful, with small datasets it really isn't. Big datasets is for statistics, small datasets if for historians.

    ReplyDelete
  17. 01 March, 2011

    "Sometimes horse races are racesbetween horses" (cicblog.com/comments.htm)

    Posted: 10:38 AM on March 1, 2011 to the Globe and Mail . . .

    I don't agree with this analysis.

    You can* simply look at the numbers in past elections and pre-elections to say the odds of a dramatic change are such-and-such.

    Polls, and elections results, are an expression of the underlying general circumstance impacted by significant events and leadership (sometimes a particular 'policy' has a significant impact, like the Ontario PC's in their last election, but I would attribute that to personalities and leadership)

    Polls, and their fluctuations, before an election are a good illustration of this.

    If you look at the elections cited, '84, 93,'06,'08 one need only look at the 'Turner effect' which was accentuated by the 'Mulroney effect', the 'Kim Campbell effect', the sponsorship scandal and the 'Dion effect', which was moderated by the chilling 'Harper effect' (in other words, if the Con's had had a better leader - i.e., not a right-wing extremist ideologue, they may have obtained that majority).

    One need look at what is happening to the numbers in absolute terms as well.

    E.g., is the swing due to more voters going towards a party or going away from a party. In the last election there were 800,000 - 900,000 less people voting Liberal, this fits in with 'leadership effect'. The same thing occurred in the recent by-election in Vaughan, Liberals had 10,000 less people voting for them whereas the Con's had approx the same number, hence Liberals lost and Con's won. Voter turn-out was key in the Vaughan by-election and that is the real indicator as far as I am concerned.

    So, for one thing, if a party's number are down in the pre-election polls, there are significant fluctuations in the polls and a large undecided. Then, if it is due to leadership, perhaps during an election people, getting to know that leader better, decide that they are so bad after all. This is, of course, what the Con's hoped for Harper in previous elections (and the upcoming as well) and what the Liberals are hoping for Ignatieff.

    Sometimes horse races are races between horses.

    excerpt: Lloyd MacILquham cicblog.com/comments.html

    ___

    * Posted: 10:46 AM on March 1, 2011 Globe and Mail

    In my post: 10:38 AM on March 1, 2011

    I wrote:

    "You can simply look at the numbers in past elections and pre-elections to say the odds of a dramatic change are such-and-such."

    It shoud read "You can ^not simply look at the numbers . . . "

    (now that's a real 'Oda'-rian slip)

    ReplyDelete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.