For this series, I interviewed a number of Canada's leading pollsters via email. I'll be posting the transcripts of these email interviews over the next few weeks here to accompany the Globe articles. Today, we talk to Don Mills, Chairman and CEO of the Corporate Research Associates. This Halifax-based firm conducts its political polling over the telephone with live-callers - though in a growing trend, they have just recently announced the launch of a new online panel. Firms that stick to only one mode to do their polling are becoming rarer and rarer.
308: While other firms have moved to online panels and IVR polling, CRA continues to use live-callers. Why have you stuck with this methodology?
DM: The methodology has stood the test of time. Our industry standards do not allow the use of margin of error for online research which are considered samples of convenience. Many companies violate industry standards in this regard. We agree with that standard.
308: What do you consider the strengths of telephone polling with live-callers compared to other methodologies?
DM: Its ability to produce random representative samples is still the biggest strength in my opinion.
308: What are its limitations?
DM: Very costly relative to other methodologies in terms of data collection and also hard to find those in the 18-24 year old age group to interview,
308: Generally speaking, how does telephone polling compare to other methodologies in terms of costs and effort?
DM: Generally it depends on the population to be surveyed and the length of the survey. Hard to reach segments are particularly expensive for telephone research. For general population type research, telephone research is likely 25% or more expensive than online on average.
308: What challenges do you face in building a representative sample, considering falling response rates and increased use of cell phones over landlines?
DM: Response rates for public opinion type research have not actually changed dramatically in the last number of years. It has always been difficult, but not impossible, to find those aged 18-24 years old. Interestingly enough, although those 18-24 are more likely to use cell phones personally, there is also a high proportion who still live at home with their parents who have landlines and who can still be reached on landlines as a consequence.
308: What role does weighting play in producing accurate results?
DM: Weighting is still important in random telephone research to properly reflect the known distribution of the population. It is nearly impossible to have proportional representation in random telephone research.
308: CRA has been in the business for a very long time. How has political polling changed over the years?
DM: Surprisingly little in my opinion. Young people continue to be disengaged in the political process for the most part as in the past. Older people continue to believe in the process and to be the most likely to vote. I think the challenge today is mainly among those in the child rearing years who have very busy lives with their children with little time to think about anything but their jobs and their families. This is the group that is likely now less engaged in the political process as a result.
308: How has the business of polling in general changed?
DM: There was a time when media invested in polling that was not just about the horse race, delving more into the issues to better understand the basis for supporting parties. This is no longer the case. Political polling is much more superficial than it was in the past.
308: Do you face any particular challenges as a pollster based in Atlantic Canada?
DM: Not particularly. Our company has a well established reputation which helps us when collecting public opinion. We have always been a non-partisan company and have never conducted polling for any political party. I think that enhances our creditability with both the political parties and the media. I believe our track record has been outstanding as well.
308: Your polls tend to be conducted over several weeks, instead of over a few days. Why is that, and what are the benefits or problems associated with this?
DM: Most of our political polling is done on a predictable quarterly basis, the same period each year. The consistency of our process provides a larger time frame than we would if we were working with media. In that case, doing survey over a few weeks rather than a few days is not really an issue. Our media work is normally over shorter time periods, although the rolling polling we did for the Telegraph Journal in the last New Brunswick election was pretty much a gold standard for tracking change over the course of the election.
308: CRA also tends to have some of the largest proportions of undecideds of any polls produced in Canada. Why?
DM: Most of the time we do not try to force an opinion. We also report those undecided, don’t plan to vote and prefer not to say together in the our numbers while other companies do not. This looks like the percentage of undecided is higher than is actually the case. Near elections, we typically ask a leaning towards question which reduces the undecided percentage considerably. I have never personally worried about those undecided because my experience is that a lot of those people are not actively engaged and probably don’t vote and those that do vote along the line of the decided group.
308: What changes, if any, need to be made to ensure that telephone polling produces good results in the future?
DM: We have strict standards for telephone research such as assuring that interviewers are well trained and respectful, that the proper number of callbacks are done properly. Adherence to the standards necessary to conduct valid telephone research remains the key to achieving reliable results. There needs to be some industry effort to promote the value of participating in survey research as a benefit for citizens as well. In a free society, how much value is associated with the ability to express your opinions in the hope of changing life for the better?