Friday, March 7, 2014

Everything we thought we knew was wrong?

It just wouldn't be an election in Canada anymore without an unusual poll from Forum Research raising eyebrows and over-turning the weight of evidence collected by other polling firms. Last night, Forum did it again in a new poll first reported by the Montreal Gazette.

The poll gave the Liberals 40% support against 38% for the Parti Québécois and 12% for the Coalition Avenir Québec. In the larger scheme of things, those results are not too far from the latest findings from CROP and Léger. But underneath the top line results, the numbers mark a serious departure from what the two Quebec-based firms have been finding to be the case for months. Or do they? More on that below.

The vote projection has moved rather dramatically as a result of this poll being added to the aggregate. The PQ has been bumped up just 0.8 points to 37.8%, but the Liberals have jumped 2.7 points to 37.6%. Much of that came from the CAQ, which fell 1.8 points to 13.6%.

It hasn't shifted the seat projection very much, however. The PQ has slipped one seat to 68, with the Liberals at 50 seats. The ranges have changed a bit more, with the PQ dropping from 62-81 seats to 61-78 and the Liberals increasing from 36-56 to 42-60. The likely ranges still give the PQ the victory, but a majority is less assured than it was on March 5. The CAQ's range has dropped from 5-7 seats to 1-5.

Now to the Forum poll, which was conducted on the first day of the campaign via interactive voice response. On the one hand, it is good to have a different methodology in the field. Both CROP and Léger use online panels, though CROP reverted to live-callers in the 2012 election campaign. We will have to see if they do so again. On the other hand, Forum's one-night flash polls are not the best way to go about collecting a sample. There's no time for call-backs, so instead of a poll of eligible voters in Quebec it is a poll of eligible voters in Quebec who happened to be home on Wednesday night. Are they different than those Quebecers home on Thursday nights?
As mentioned, the top line results are not too unusual. Though the Liberals are certainly higher than the 34% to 35% recently recorded, perhaps the start of the campaign has boosted them. The 38% for the PQ is well in line with other results, as is the 7% for Québec Solidaire. The 12% for the CAQ is low (Forum always seems to have the CAQ lower than other firms), but it is not much different from the 15% and 16% registered in the last polls by CROP and Léger.

But then we get to the linguistic breakdowns. Forum has the PQ at 41% among francophones, low compared to CROP and Léger but not unreasonable (that is where they were in polls conducted between October and January). But Forum has the Liberals at 36%. That is an incredible number - though almost exactly where Forum has the party in their last polls in September and October of last year. But just as those results were at odds with what CROP and Léger had at the time, they are again. According to those two firms, the Liberals have been at or below 25% among this demographic for the last two months. The last time either firm had the Liberals as high as 36% was years ago.

The non-francophone score is also quite high for the Liberals: 89%. But that is not an implausible number, as CROP has had the party that high in the last year.

It would seem that these numbers from Forum are unweighted, because otherwise they make no sense. If the Liberals are at 36% among francophones (who make up roughly 80% of the population) and 89% among non-francophones (who make up the remaining 20%), they would be at roughly 47% support province wide. The PQ's score would translate into about 34% support. This is likely due to the over-sampling of Quebec City, where the Liberals are at 46% support. This would have been done to build a useable sample. But it would seem that Forum did not re-weight their reported francophone numbers to take that into account (they would have to in order to calculate the provincial numbers, however). In other words, the number for the Liberals among francophones is not weighted correctly - that is the raw number, meaning the Liberals are NOT at 36% support among francophones.

Instead, the Liberals would seem to need to be around 28% for the overall numbers to make any sense, with the PQ around 47% and the CAQ around 16%. This would put Forum's numbers more in line with what other firms are showing.

The big difference, then, comes primarily in Quebec City. The numbers in Montreal and the regions are not dissimilar from those of the other firms, but the Liberals have not been put higher than 34% in any polls by CROP and Léger since December. The margin of error for a sample of 144, which is what Forum reports for the Quebec City region, is +/- 8%. So, it is possible that the Liberals are pushing the limits of that MOE in the provincial capital.

So, perhaps the poll is not as absurd as it appears on first glance. But these are things to keep in mind going forward: Forum's reported support for francophones seems unweighted, meaning the numbers are worthless if they are not regionally weighted. Focus must thus remain only on the regional and overall numbers if we are to take anything useful from the Forum polls.

27 comments:

  1. Using only the new Forum, I get:

    60 PQ
    58 PLQ
    5 CAQ
    2 QS

    Using the aggregated numbers, I get:

    64 PQ
    54 PLQ
    5 CAQ
    2 QS

    I have a hard time believing this poll, it is so outside what we have seen recently that it sounds like a statistical anomaly. On the other hand, I think it may be a precursor to the election if no major event shatters current PQ and PLQ voters and the CAQ voters feel they have a better chance at ousting the PQ if they shift to the PLQ.

    ReplyDelete
    Replies
    1. Forum's outlier poll in the previous Quebec election turned out to be correct IIRC. I believe they came closest in BC and Alberta as well? We'll see.

      Delete
    2. Forum had the 2012 election as 36% PQ, 29% PLQ, and 25% CAQ. Not horrible, but it was less reflective of the result because it implied a PQ majority. CROP and Leger had the PLQ in third, but did not imply that the PQ would win a majority, so it is a bit of a wash.

      Yes, they were closest in BC and Alberta. The BC result was a bit by default, though, since their final poll was done about a week before the vote. And in AB, they still had WR up on the PCs by two points, rather than being behind by 10.

      Delete
    3. Yah, not great, but best of the worst at least.

      Delete
  2. I think we can expect a wild ride in the polls for the next few weeks. The Charte des valeurs was the only thing being talked about for nearly a year. But with the budget and the election call, it has evaporated from the headlines, and the CROP poll suggested only 8% thought identity politics was the #1 issue. If public debate shifts to the economy, on which the distinctions between the parties are still ill-defined among voters, then expect lots of volatility in public opinion.

    ReplyDelete
  3. I suspect as the campaign goes on both the Charte des Valeurs and the seperation thing will become less important and the economy more important. Should that happen the PQ record is pretty dismal and we can expect a minority govt.

    ReplyDelete
    Replies
    1. I hope you are right that the campaign does turn on economic issues because it puts the PQ at a disadvantage. Both separation and the charter of values are bad for the economy. Recently there has been talk about oil exploration in the Gulf of St. Lawrence but, that will get scrapped if the PQ gets a majority as will other investments. Much will hinge on suburbanites in Laval and the South Shore if the PLQ can convince them that a PQ victory will be detrimental to their economic well being they may just have a shot at winning a majority.

      Delete
    2. Chantal Hebert has a different take on this !!

      http://www.thestar.com/news/canada/2014/03/08/pqs_sovereignty_ambitions_shouldnt_distract_from_larger_realities_in_quebec.html

      Delete
    3. Carbonear why do you think Marois is SO happy to get Peladeau on board. Because the PQ has no support for it's economy performance and Peladea is a real BIG wheel !!

      Delete
    4. I agree, Peladeau gives Marois an air of economic respectability. Of course Peladeau's success is heavily influenced by the regulated telecommunications sector of Quebec. It is easy to be successful when you are essentially the only game in town. Just like Danny Millions on the Rock

      Delete
  4. Judging by the articles I've seen in The Globe and Mail the last few days, it seems like the PQ has already won a majority.

    The election campaign was always going to be tight. The PQ is never ahead in the polls by large margins. Sure the PQ's support is better spread out that they can win a majority government while their popular vote is tied to the Liberals, but that does not signal any confidence in a referendum.

    My personal opinion, I think the PQ will be back in government with another shaky minority government.

    ReplyDelete
  5. New CROP poll:
    http://ici.radio-canada.ca/sujet/elections-quebec-2014/2014/03/09/016-sondage-intentions-de-vote-majorite.shtml

    ReplyDelete
    Replies
    1. Yes, I am waiting on the detailed regional results before including it in the projection.

      Delete
  6. I wonder if Marois realizes exactly what she has brought on board ?? Peladeau brings some serious baggage with him including a real hatred of unions and the labour groups can be really powerful if they need to be. Secondly I hope Marois doesn't think Peladeau will be happy with a mere Cabinet seat ?? Given his past history he is a mega-manager and won't want less than the Premiership !!

    ReplyDelete
    Replies
    1. Peladeau will want finance and will settle for nothing less. He has the potential to become Marois' biggest headache.

      Delete
    2. Peter,

      Wasn't the last guy interested in the premiership called Gilles Duceppe???

      Delete
    3. Chantal Hebert in today's Toronto Star really agrees with me !!

      http://www.thestar.com/news/canada/2014/03/11/pierre_karl_pladeaus_candidacy_marks_beginning_of_covert_battle_for_pauline_marois_succession.html

      Delete
    4. And the CBC agrees

      http://www.cbc.ca/news/politics/for-the-pq-the-two-edged-sword-that-is-pierre-karl-p%C3%A9ladeau-1.2567512

      Delete
    5. This is not a place to post links.

      Delete
    6. And do check out the Andrew Coyne's piece in today's National Post. Confirms what I and others have said.

      Delete
  7. Éric,

    I hope Drainville and Lisée remember to put Pauline on their Christmas card list again this year.

    ReplyDelete
  8. Eric,

    Is there a bell-weather riding in Quebec that always votes for the government? If sowhich one and how are they leaning as of today?

    Thanks

    ReplyDelete
    Replies
    1. I actually wrote about this for the G&M in the run-up to the 2012 vote:

      http://www.theglobeandmail.com/news/politics/charests-fate-in-quebec-election-comes-down-to-these-six-ridings/article4435182

      I identified six such bellwether ridings.

      Only two of them, Laval-des-Rapides and Sherbrooke, went with the PQ in 2012, so it would seem they are the only remaining bellwether ridings.

      I currently have the Liberals up by one point in Laval-des-Rapides and the PQ ahead by 12 in Sherbrooke.

      Delete
  9. Sherbrooke is not a very good bellwether since it was Charest own riding and that inflated the PLQ vote there a lot in the last few elections...

    ReplyDelete
    Replies
    1. Probably. But maybe Charest inflated the PLQ's vote everywhere else as well...

      Delete
  10. There is a "leader effect" when a major party leader is the candidate in a riding that i think throws off many models.

    ReplyDelete
    Replies
    1. I've been working on a way to model that, but hadn't finished it in time for this campaign. Hope to have it done for 2015, because Outremont is one of the first seats to swing over to the LPC in the model, and it really probably shouldn't be.

      Delete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.