Monday, September 24, 2012

Partisan language, and interpreting polls

If you think that the tone of debate on Parliament Hill has become more partisan in recent years, it’s not just you. An analysis of House of Commons’ transcripts shows how one portion of the daily routine in the House has been increasingly used for partisan ends since the Conservatives came to power in 2006.

You can read the rest of the article on The Globe and Mail website here.

I also have a column in The Hill Times this morning dealing with the importance of interpreting a poll within proper context, and how interpreting it falsely can lead to false narratives. You can read my column here, but you'll need a subscription to The Hill Times (a smart investment).

In the column, I mention the recent Angus-Reid poll in British Columbia that showed a statistically insignificant increase for the B.C. Liberals, prompting headlines that the party was on the upswing. The column was written before this weekend's Ipsos-Reid poll, but the reaction to that poll is another perfect example of how polls need to be put into proper context.

I will look into the entrails of the poll in more detail later this week, but the important numbers are 32% for the B.C. Liberals and 12% for the B.C. Conservatives. That 32% was heralded as terrific news by Liberals in the province, and added to a narrative that formed over the weekend - that the B.C. Liberals are not quite dead and that the B.C. Conservatives are not a legitimate alternative.

John Van Dongen, the party's only sitting MLA (he crossed the floor from the Liberals) decided to quit the party over the weekend, justifying his departure with John Cummins' inability to make gains in the polls. Would he have felt the same way if the only numbers he had to go on were the ones from Angus-Reid and Forum, showing only a small gap between the two parties? Or was he basing that decision in part on the new Ipsos-Reid poll, that put the B.C. Liberals 20 points ahead of the Conservatives?

Trumpeting the Ipsos-Reid poll as terrific news for the B.C. Liberals is a bit of a problem, as it must assume that the polling firm is acting in a vacuum. This is the kind of poll reporting that Nate Silver criticized last night.

As you can see from the chart above, the B.C. Liberals have consistently over-performed in polls by Ipsos-Reid - on average by about six points. Both Forum and Angus-Reid have had the Liberals at much lower numbers. The good result from Ipsos-Reid is not, then, a sign that the Liberals are on a real upswing from the mid-20s of recent months. They have merely experienced a statistically insignificant increase from a polling firm that generally gives them better results.

That is not to say that Ipsos-Reid is wrong. There is every chance that Ipsos-Reid is closer to the mark and that Angus-Reid and Forum are under-scoring the B.C. Liberals. But placing Ipsos-Reid's numbers within context is absolutely essential in interpreting this poll correctly. Otherwise, the headlines accompanying the next poll from Forum or Angus-Reid have a good chance of falsely giving the impression that the B.C. Liberals are slipping once again.

7 comments:

  1. I choose to only believe the polls that feed the narrative closest to my own beliefs.

    ReplyDelete
  2. In BC there have been two fairly different stories in polling over the last year. On the one side Forum and Angus Reid with the BC Conservatives and BC Liberals in the same range in the low 20s. The other being NRG, Mustel and Ipsos which have the BC Liberals closer to 30% and the BC Conservatives at half that amount.

    Based on how the parties did in the by-elections and how they have been doing with fundraising and organizing, I tend to think the second set of numbers are closer to what the mood of the public is in BC.

    ReplyDelete
    Replies
    1. IMHO this dissonance in the polls was an indication of a lot of "soft" support for the BC Conservatives. So both polls were "right" in a sense - there were just a large number of voters out there that would choose BC Liberal or BC Conservative depending on how the question was asked.

      Delete
  3. RE: your article in today's Globe on Members' Statements - there was an interestingly similar article in the Canadian Parliamentary Review's 2009 fall issue.

    Anyway, will you be posting more details online from your almost 1,000 speeches analyzed?

    ReplyDelete
    Replies
    1. Yes, Aaron Wherry posted a link to it on his blog at Maclean's today. I had not seen it before then - but it is an interesting study, looking at the 38th and 39th parliaments in detail (i.e. 2004 to 2008).

      My analysis stretched from 1994 to 2012, which I think gives a bit of a better picture of how things have changed over the last 18 years, though perhaps a little more superficially since I only sampled three days from each year. They seem to have looked at every speech over their period.

      I wasn't planning on posting any more details, as all of the interesting bits were mentioned in the article and the accompanying graphic showed the rest. Was there something in particular you were interested in?

      Delete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.