Friday, March 21, 2014

No wait, it's a landslide

For the second time in the election campaign in Quebec, a poll by Forum Research (this time reported by the Toronto Star, after the Montreal Gazette was scolded by the MRIA for the way the last Forum poll was contemptuously reported) tells us that all assumptions made about the state of the race in Quebec are wrong. Rather than a close race, the Liberals have soared ahead with a double digit lead. Philippe Couillard is, thus, on track to bettering Jean Charest's majority victory of 2008.

Accordingly, the projection has lurched violently to the advantage of the Liberals. The party now leads with 43.1% (or between 41% and 47%, assuming normal polling errors) against just 32.8% for the Parti Québécois (or between 32% and 36%). The Coalition Avenir Québec remains well behind with 13.3% (or between 12% and 14%) while Québec Solidaire has slipped to 7.8% (or between 7% and 8%).

These sorts of numbers give the Liberals a large majority government of 71 seats, with the PQ reduced to 50 and the CAQ to just two, putting them in a tie with QS. The likely ranges now point only to a Liberal victory, with between 61 and 76 seats while the PQ could win only between 46 and 59 seats.

Has the campaign really swung so heavily towards Couillard and the Liberals? The last time Forum reported, they gave the Liberals a two-point edge when both CROP and Léger saw the race as tied or leaning towards the PQ. Léger and CROP both subsequently recorded Liberal gains, so perhaps Forum will be a precursor to further Liberal gains again. But these numbers for the Liberals are rather extraordinary. The party has not been recorded at 45% support in any poll for many, many years. Has the PQ campaign really imploded to such a huge extent?

Considering the unusual nature of this poll in the context of other surveys, I will update the projection this weekend (something I would normally not do) if new surveys from CROP and/or Léger emerge that portray a different picture.

Forum was last in the field on the day of the campaign launch, on March 5. Since then, they have recorded a five point jump for the Liberals to 45% and a six point drop for the PQ to 32%. The CAQ was up one point to 13%, while Québec Solidaire was unchanged at 7%.

Since Forum does not report correctly weighted support levels by language (instead, they report the raw, unweighted, disproportionately distributed numbers) I have left them off the chart.

The poll was conducted on March 19 only. In my view, this is a problem and is one that Forum could easily avoid but continues to choose not to. The firm conducts its polls in one evening over a period of a few hours. What this means is that Forum polled 1,650 Quebecers who happened to be home between, say, 7 PM and 10 PM on Wednesday night.

The question must then be asked. Are Quebecers who are home between 7 and 10 on a Wednesday night representative of Quebecers as a whole? Are they markedly different from Quebecers home on Thursday nights or Tuesday nights, or Quebecers who were not home on this particular Wednesday? Luckily, the Montreal Canadiens were not playing on Wednesday, otherwise the poll might have been conducted among non-hockey fans only. The by-election miss in Brandon-Souris in November 2013 might have been partly caused by the Grey Cup taking place on the same night of the final poll.

That something as minor as this could even be a factor is an example of why polling should not be done on a single day. And with Forum conducting its poll in such a narrow time frame, call-backs are impossible. Telephone pollsters carry out call-backs to try to reach the people they missed when they first called in order to maintain the random nature of the sampling. Otherwise, no attempt is made to reach a representative sample of the population, just a sample of the people home on a particular night. And as Forum uses interactive voice response methodology, which has a response rate of between 1% and 3%, the need to conduct call backs is even greater.

So why not run the poll over three days, for example, instead of three hours? What is being sacrificed to get 'fresher' numbers for no particular reason? I can see the benefits of a flash poll after a big event, such as a debate, but not during an unremarkable period of the campaign.

The poll showed little real movement in Quebec City or the regions, but recorded a big swing in the Montreal area. There, the Liberals jumped seven points to 49%, while the PQ fell eight points to 30%. This was responsible for almost all of the shift in voting intentions province wide.

It wasn't the only big swing recorded. Pauline Marois's approval rating was virtually unchanged at 32%, but Couillard's increased by 10 points to 44%. François Legault's surged by 14 points to 48%, with his disapproval rating plummeting 17 points to 32%. That seems a little extreme.

I can't help but look at this poll with a good deal of skepticism, at least in terms of the precise numbers. The trendline seems consistent with other polls. But we can't definitively say whether the poll is on the mark or not until we get some corroborating numbers. Hopefully, we'll get those soon.

23 comments:

  1. "the Montreal Gazette was scolded by the MRIA for the way the last Forum poll was contemptuously reported"

    Hahaha! I had totally noticed that! I've never seen a news article report on a poll with as much as reservation as that one. It basically read:

    "We obtained these polling results from Forum Research, and last election they put out this one poll that was sorta, kinda, somewhat close to the actual result, so there's a chance these numbers might be worth something, but generally speaking everyone should be skeptical of polls".

    It was hysterical!

    Dom

    ReplyDelete
    Replies
    1. I like critical reporting, like the reporting that was done on the final Brandon-Souris poll by the Winnipeg Free Press. But that article in the Gazette was more along the lines of "here's a poll, but polls totally blow, amirite?"

      I agree with the MRIA that if the author felt polls were so horrible, why was the article written at all?

      Delete
  2. Sir, you made me happy and excited until i read your comments... Why do you bother changing your projections on the basis of polls which you tell us to take with a grain of salt?

    ReplyDelete
    Replies
    1. The aim of aggregating the polls together is so that the flaws in each of them individually can be cancelled out. This poll raises alarm bells, but is not necessarily inaccurate. The aim is to give readers all of the information and let them decide for themselves.

      Delete
  3. Perhaps the big swing in the Montreal area exaggerates Liberal support but accurately records PQ support. After all, the Liberals historically receive between 40-45% on the island of Montreal whilst the PQ has been below 30% for several election cycles - am I correct?

    ReplyDelete
  4. I do find it hard to believe that outside the two big metropolitan areas the Liberals still show a significant lead. Sorry but that flies in the face of past history !

    ReplyDelete
  5. Forum has poll on Trinity Spadina byelection:
    http://www.forumresearch.com/forms/News%20Archives/News%20Releases/87825_Trinity-Spadina_News_Release_%282014.03.16%29_Forum_Research.pdf

    ReplyDelete
    Replies
    1. The poll doesn't show anything unexpected a 14% NDP lead over the Liberals. By-elections are often unpredictable and usually have low turnout. I think the end result will likely be closer than a 14% spread.

      Delete
    2. The NDP should be able to hold Trinity-Spadina. Joe Cressy is a strong candidate and they are well organized in this riding, despite changing demographics due to gentrification favouring the Liberals.

      On the other hand, I don't think the nomination controversy will affect the Liberals. Most voters probably won't care. And right now it seems more of a "he said", "she said" thing.

      Delete
  6. Dunno about their Quebec work, but when they poll in Toronto my reaction is ... "Wake me whenever the new orgs find enough change under the cushions to pay for an Ipsos one"

    ReplyDelete
    Replies
    1. Actually Forum's polling in the Toronto Centre byelection turned out to be very accurate - it was rural Manitoba where they totally screwed up

      Delete
  7. I don't live in Quebec, but I have been following this election closely. I am surprised how sloppy and unfocused the PQ campaign has been.

    Now with the polls favouring the Liberals again, Pauline Marois is claiming that English speaking students from other parts of the country are going to come to Quebec and illegally vote in the election!

    I would have expected more shrewdness from a veteran politician like Marois.

    ReplyDelete
    Replies
    1. Once again the PQ can not win fairly so they blame the "ethnic vote" to quote Jacques Parizeau for their loss.

      Most students under the law will be able to vote. The law requires one to be resident in Quebec for 6 months. Most universities begin in September roughly 7 months before the election. The other part of eligibility is a question; "do you intend to live in Quebec"? Of course many students may not have this intention over the long term but, the question is worded without a specific timeline and so becomes open to interpretation by the voter. Most voters are unlikely to move from Quebec the day of the election or the day after. So on the face of it while the second part appears to be a barrier to voting it is broadly worded enough that most voters will be able to answer "yes" truthfully.

      Delete
  8. The PQ always had, and seems like always will, have bad strategists. Compared to the PLQ's very focused and very smooth campaign, the PQ is all over the place and nowhere at the same time. One point being: if seperation is to be made, considering the changing demographics with the massive immigrant population Québec is accepting, you'll need to convince them of the well being of the project. Even English-speaking Quebecers should be involved in this and made to feel like what we are building is better, but no. It's always a "us against them" sort of project, take the Charter for example. "They" do not conform, so we'll force them into it, we'll try to assimilate them the way the British tried to assimilate us 300 years ago. It doesn't make any sense, especially considering our own past history, but for the few enlighted at the PQ, seems like it does.

    All that to say, there is nothing surprising about this sloppy campaign when you take into consideration the whole political organization of the PQ. Even now, they are acting as if Pauline Marois was still in the opposition trying to win government with the constant attacks against the Liberals, but no, she was in power, she should be defending her accomplishments and present her plan for the next 4 years and leave the competition to the attack mechanism, to lower themselves there. When the population sees a person that would make a good opposition leader because of their fight and bite, that's generally where they'll place such person. Again, seems the PQ doesn't know that. Politics 101...

    ReplyDelete
    Replies
    1. ...and as far as I know, the British did not, in fact, attempt to assimilate Quebec. Language was kept, religion was kept, legal system was kept.

      Delete
  9. I was excited about the new Leger poll until I found out it was an online non-probability poll. Really Leger? A non-probability poll? How the heck are they able to publish something like that and say it's representative? They can't even give a margin of error

    ReplyDelete
  10. New Leger/ Journal de Montreal poll has the Liberals up 40% to PQ 33%. Not as big a lead as Forum suggests, but pretty close.

    No way Greens are going to get 2% of the vote -- they are only running in 1/3 of the ridings so will likely get closer to 0.7% of the vote.

    ReplyDelete
  11. Odd, the Globe and Mail is reporting it as a non-probability poll, but other news outlets give it 1.9 percent margin of error 19 out of 20".

    ReplyDelete
  12. Léger has been conducting its polls online throughout this election, and for the last few elections as well. CROP also does its polling online.

    ReplyDelete
    Replies
    1. I don't take issue with it being online. I take issue with it be a non-probability poll.

      Delete
  13. Online polls are non-probability polls. All the polls Léger has done in this campaign and in recent campaigns have been non-probability polls, as have been the polls by CROP in this cycle.

    ReplyDelete
  14. Globe and Mail: "Because it was an online non-probability survey, the margin of error cannot be determined, Leger said." Why is there no margin of error here, whereas there are in most other polls?

    ReplyDelete
    Replies
    1. Not in other online polls, or at least one shouldn't be given. Often, firms will say what the MOE would be with a probability poll of equal size, and that number is usually repeated without the caveat by the media.

      Delete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. Please keep discussion on topic.