Friday, November 26, 2010

Pollster House Effects Update

Time for an update on the house effects chart. This month, I'm updating Harris-Decima and Nanos Research. See below for a full explanation of what these house effects measure.

For Harris-Decima, they did not show any huge disparity at the national level, though their largest was in polling 1.6 points lower than average for the Conservatives. In Quebec, they polled the Conservatives 1.9 points lower than average, while polling the Bloc Québécois and Greens 1.9 points higher than average.

Adding these findings to the chart, Harris-Decima is the worst pollster for the Conservatives at the national level and the New Democrats in Quebec. They are the second worst for the Liberals nationally and the Conservatives in Quebec. They are second best for the Greens nationally, and best for the Bloc in Quebec.

On to Nanos. They did have some large disparities in October. They polled the Liberals 4.3 points higher than the average and the Conservatives 3.3 points higher. This was at the expense of the Greens, who they polled 5.2 points lower than average. Undoubtedly this is due to their method of not prompting party names in their questions.

In Quebec, they polled the Conservatives 6.0 points higher and the Liberals 4.9 points higher, while they polled the Greens 6.3 points lower.

This makes them the worst pollster, both nationally and in Quebec, for the Green Party. They are tied for being the best pollster for the Conservatives in Quebec, and are the second best pollster for the Liberals both nationally and in Quebec. They are the second worst pollster for the Bloc.

So, just for giggles, let's apply these house effects to Harris-Decima's and Nanos' most recent national polls. The first number is the actual poll result, the second is what it becomes with the house effects.


Conservatives - 33.0% = 35.3%
Liberals - 28.0% = 28.7%
New Democrats - 17.0% = 17.2%
Bloc Québécois - 9.0% = 9.0%
Greens - 10.0% = 8.2%


Conservatives - 37.1% = 37.8%
Liberals - 31.6% = 28.6%
New Democrats - 15.4% = 15.0%
Bloc Québécois - 10.8% = 10.8%
Greens - 5.2% = 7.6%

Interestingly, the Liberals and Greens end up at similar results, with the disparity seeming to come between the Conservative and New Democratic results.

Now Quebec:


Bloc Québécois - 39.0% = 36.3%
Liberals - 21.0% = 21.6%
Conservatives - 13.0% = 14.8%
New Democrats - 11.0% = 12.9%
Greens - 11.0% = 11.1%


Bloc Québécois - 42.8% = 46.4%
Liberals - 26.2% = 21.8%
Conservatives - 19.3% = 17.1%
New Democrats - 10.5% = 10.7%
Greens - 1.3% = 4.9%

Much more of a variation here, though again the Liberal numbers are very similar.

The chart below tracks how each pollster tends to lean when calculating support levels for the various parties, as compared to the average polling results from other pollsters each month. This does not necessarily equate to a deliberate bias, but instead is more reflective of the polling methods used - the "house effects". This is also not a scientific calculation of any kind. Methodological differences, field dates, margins of error, and polls released vs. polls unreleased all play a role in these calculations. But it does give a little bit of an indication of how each pollster tends to compare to others - and only to each other.

The following chart shows each pollster's average variation from other polling firms. The numbers are the amount of percentage points a particular pollster favours or disfavours that particular party compared to other pollsters over a similar period of time.