Thursday, July 25, 2013

Interview with Don Mills of Corporate Research Associates

Today's column for The Globe and Mail is the first of three that will be running over the next few weeks, looking at polling in Canada. Each piece will be focusing on the modes of contact used by polling firms today: live-caller telephone, IVR, and online. 

For this series, I interviewed a number of Canada's leading pollsters via email. I'll be posting the transcripts of these email interviews over the next few weeks here to accompany the Globe articles. Today, we talk to Don Mills, Chairman and CEO of the Corporate Research Associates. This Halifax-based firm conducts its political polling over the telephone with live-callers - though in a growing trend, they have just recently announced the launch of a new online panel. Firms that stick to only one mode to do their polling are becoming rarer and rarer.

308: While other firms have moved to online panels and IVR polling, CRA continues to use live-callers. Why have you stuck with this methodology?

DM: The methodology has stood the test of time. Our industry standards do not allow the use of margin of error for online research which are considered samples of convenience. Many companies violate industry standards in this regard. We agree with that standard.

308: What do you consider the strengths of telephone polling with live-callers compared to other methodologies?

DM: Its ability to produce random representative samples is still the biggest strength in my opinion.

308: What are its limitations?

DM: Very costly relative to other methodologies in terms of data collection and also hard to find those in the 18-24 year old age group to interview,

308: Generally speaking, how does telephone polling compare to other methodologies in terms of costs and effort?

DM: Generally it depends on the population to be surveyed and the length of the survey. Hard to reach segments are particularly expensive for telephone research. For general population type research, telephone research is likely 25%  or more expensive than online on average.

308: What challenges do you face in building a representative sample, considering falling response rates and increased use of cell phones over landlines?

DM: Response rates for public opinion type research have not actually changed dramatically in the last number of years. It has always been difficult, but not impossible, to find those aged 18-24 years old. Interestingly enough, although those 18-24 are more likely to use cell phones personally, there is also a high proportion who still live at home with their parents who have landlines and who can still be reached on landlines as a consequence.

308: What role does weighting play in producing accurate results?

DM: Weighting is still important in random telephone research to properly reflect the known distribution of the population. It is nearly impossible to have proportional representation in random telephone research.

308: CRA has been in the business for a very long time. How has political polling changed over the years? 

DM: Surprisingly little in my opinion. Young people continue to be disengaged in the political process for the most part as in the past. Older people continue to believe in the process and to be the most likely to vote. I think the challenge today is mainly among those in the child rearing years who have very busy lives with their children with little time to think about anything but their jobs and their families. This is the group that is likely now less engaged in the political process as a result.

308: How has the business of polling in general changed?

DM: There was a time when media invested in polling that was not just about the horse race, delving more into the issues to better understand the basis for supporting parties. This is no longer the case. Political polling is much more superficial than it was in the past.

308: Do you face any particular challenges as a pollster based in Atlantic Canada?

DM: Not particularly. Our company has a well established reputation which helps us when collecting public opinion. We have always been a non-partisan company and have never conducted polling for any political party. I think that enhances our creditability with both the political parties and the media. I believe our track record has been outstanding as well.

308: Your polls tend to be conducted over several weeks, instead of over a few days. Why is that, and what are the benefits or problems associated with this?

DM: Most of our political polling is done on a predictable quarterly basis, the same period each year. The consistency of our process provides a larger time frame than we would if we were working with media. In that case, doing survey over a few weeks rather than a few days is not really an issue.  Our media work is normally over shorter time periods, although the rolling polling we did for the Telegraph Journal in the last New Brunswick election was pretty much a gold standard for tracking change over the course of the election.

308: CRA also tends to have some of the largest proportions of undecideds of any polls produced in Canada. Why?

DM: Most of the time we do not try to force an opinion. We also report those undecided, don’t plan to vote and prefer not to say together in the our numbers while other companies do not. This looks like the percentage of undecided is higher than is actually the case. Near elections, we typically ask a leaning towards question which reduces the undecided percentage considerably. I have never personally worried about those undecided because my experience is that a lot of those people are not actively engaged and probably don’t vote and those that do vote along the line of the decided group.

308: What changes, if any, need to be made to ensure that telephone polling produces good results in the future?

DM: We have strict standards for telephone research such as assuring that interviewers are well trained and respectful, that the proper number of callbacks are done properly. Adherence to the standards necessary to conduct valid telephone research remains the key to achieving reliable results. There needs to be some industry effort to promote the value of participating in survey research as a benefit for citizens as well. In a free society, how much value is associated with the ability to express your opinions in the hope of changing life for the better?

5 comments:

  1. An interesting interview overall. Thanks for conducting and sharing it, Éric.

    I absolutely agree with Mr. Mills on the issue of public polling becoming increasingly focused on "the horse race". However, is the source of the problem the media (who may be relying on the "horse race" perspective to appear unbiased) or its consumers and their demand? As political parties have watered down their policy and their principles to capture larger swathes of the undecided public with shiny partisanship and image politics, perhaps even the politically-literate public, once interested in policy and the genuine principles that differentiated political parties, have become caught up in the horse race themselves.

    ReplyDelete
    Replies
    1. I would attribute this effect to an entirely different cause, the rise in dominance of a sensationalist media. As people have less time to invest in details, they demand their information in ever-more-easily-digestible chunks - newsbytes. The most easily digested and spun results are always the headline results, and hence the attention shift to "The Horse Race" at the expense of greater depth.

      Delete
    2. Well, as people live increasingly busy lives with more focus on work, distant to family and community life, I would agree that people come home from work, put their feet up and seek out the most easily digestible forms of news media.

      But if this shift in how the media presents information was the primary reason for The Horse Race effect, we should simply have the issues that we once discussed in greater depth, watered down -- we've seen this, but the even greater concern is that political issues seem to have taken a backseat in favor of the ongoing saga of political fortunes.

      I would argue that politicians and analysts have seen just how many of us tune issues and substance out, just as the media has and to attract the swathes of the middle, the undecided, they've need to reduce their policy and double-down on their image which in turn has media focusing on images, brands and of course, voter trends to fill the political vacuum once occupied by policy debates.

      Delete
  2. As we no longer have a census I cannot help but wonder how we will know when a sample represents the population.

    ReplyDelete
    Replies
    1. We do still have a census for the major things, such as age and gender. Those remain on the mandatory survey. I'm not sure about questions related to income or education, though, which are also routinely used for the weighting schemes.

      Delete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. Please keep discussion on topic.