Thursday, July 15, 2010

The Ballot Box Factor

Polling voting intentions is not the same as polling actual voting behaviour. Answering a poll is very different from voting, something that you do only 10 or 20 times in your life. While that last news clip might have convinced you not to vote for that guy, when election day comes you may decide that you actually prefer that guy's party to the other guy's (or gal's).

I've been thinking about how to reflect this factor in the projection model. Originally, I had thought that my inclusion of the last three election results would act as a bit of a guide wire for my projection. But seeing the Green Party at over 10% leads me to believe that I may need to look into other methods of predicting voting behaviour based solely on polls.

So, I took a look at the last three elections, and how the polling compared to the actual results. I did something like this back in October 2009, and the conclusion I drew was that there wasn't enough data. While I'm still not happy with the amount of data I have to use, I've reversed my decision not to include some method to reflect this in the projection model.

The following chart was done-up for that post, and it is still relevant.This shows how things generally even out for the Liberals and Conservatives, but also that the NDP and Greens tend to over-perform in polls.

First, I looked at the last week of polling in the three last elections. In 2008, the average result of that last week was 34.2% for the Conservatives, 26.6% for the Liberals, 19.5% for the NDP, 9.6% for the Bloc Québécois, and 9.6% for the Greens. The actual election results were 37.7%, 26.3%, 18.2%, 10.0%, and 6.8%, respectively.

In other words, on election day the Tories had 110% of their average polling result, compared to 104% for the Bloc, 99% for the Liberals, 93% for the NDP, and 71% for the Greens.

But, you say, people change their minds in a week. How about three days? The difference is negligible: 110% for the Conservatives, 101% for the Bloc, 99% for the Liberals, 94% for the NDP, and 72% for the Greens. However, we can reasonably assume those last three days to be more accurate, so these are the numbers I will be using. Nevertheless, I think it interesting that the average polling results over the last seven days is more or less the same as the results over the last three days. It makes you wonder when is the point that voters make up their minds.

In the last three days of the 2006 election, the polling average for the Conservatives was 37.1%. It was 27.5% for the Liberals, 18.5% for the NDP, 11.3% for the Bloc, and 5.5% for the Greens. The actual result was 36.3% for the Conservatives, 30.2% for the Liberals, 17.5% for the NDP, 10.5% for the Bloc, and 4.5% for the Greens.

So now the Liberals performed best on voting day, with 110% of their polling average. The Conservatives had 98% of their polling average, the NDP 95%, the Bloc 93%, and the Greens 82%. You might be noticing a trend.

In 2004 there were fewer polls, but nevertheless the result was 111% for the Liberals, 103% for the Bloc, 97% for the Conservatives, 86% for the Greens, and 85% for the NDP.

After weighting the three elections (2008 being worth three times as much as 2004, for example) the average I worked out was to increase the Liberal vote by 1.046 times and the Conservative vote by 1.037 times. I also needed to decrease the Bloc vote by a factor of 0.987, the NDP by 0.928, and the Greens by 0.778.

But why punish these parties for what the pollsters have done in the past, much of it explained by the margin of error? After all, the next election might be completely different. Polling methods could change or the pollsters could simply be luckier. So, I've halved those factors to come up with a number that I am comfortable with.

The popular vote projection, then, will now include this voting-day factor (though I haven't added it yet to the charts at the top of the page yet).

Note, this factor will not be added to the regional results, as the margins of error are much larger and the inclusion of past electoral results minimizes the importance of this voting day effect.

With this new factor weighed into the projections, the popular vote you currently see at the top of the page turns into:

Conservatives - 33.7%
Liberals - 28.3%
New Democrats - 16.3%
Bloc Québécois - 9.6%
Greens - 9.3%

I am far happier with those numbers, but I am open to suggestions as to how I can best deal with this issue.


  1. I don't think you are ever going to be happy with such a predictor.

    In 04,06 the liberals were able to use the hidden adgenda to greater use than in 08 when the tories had been in government for 2 years.

    And likewise the liberals have used scandal mongering more in the most recent election against the tories.

    On the flip side. The liberals had a record to run on in the first 2,... and more recently the scare about coalition and green shift in the last.

    In other words there is a whole host of factors surrounding incumbency and non incumbency for the 2 main parties. And they will not remain as constants election to election.

    Beyond that there is changes in behind the scenes stuff. The tories for example had a much weaker war room in the 04 election, but the gotv machine is much stronger recently. Who the current leader is can be a large factor too. Dion for example was someone who excited very few people into a frenzy to come out and vote on election day.

    I think it would be more useful to datamine demographics, and use them to split up voters into likelyness to vote. however there is even less data on that in Canada :/

  2. Eric i'd just add that the ballot box factor includes a key measure beyond voter intention, which is a party's organizational strength.

    How much money they've spent on GOTV efforts, microtargeting, voter contacts, that sort of thing.

    I would suggest that in 2004 and 2006 the Liberals out performed their polls because they still had money, power, and organization. By 2008 they no longer did because their party infrastructure was wrecked due to poor fundraising.

    The Conservatives on the other hand have been building organizational strength.

    By every measure their overwhelming advantage will continue and even INCREASE in the next election.

    However, I don't know enough about the smaller parties organizations/fundraising to see if this theory holds up for them.

    Perhaps Alice over at pundits guide will have an opinion to this question:

    Can one identify the ballot box factor based on the comparative fundraising ability of various parties over several cycles ?

    Another possibility = higher number of incumbents increases ballot box factor.

    If either or both cases are true then your adjustments could potentially be making your projections WORSE by increasing the Liberals at a greater factor than the Conservatives.

    Considering Liberal fundraising has been a joke for some time now they might actually stand to be DECREASED by a factor similiar to 2008 and their past, faded glories ignored altogether.

  3. I propose a trick to make your analysis more accurate: Canadians will vote for any party running the government by about 10% over what the polling shows.

    Note how between 2006 and 2008, the Liberals and Conservatives basically switched places, while all of the others stayed the same (except for the Green vote getting a bit softer).

  4. Couple of minor quibbles.

    First no allowance for Independents although I believe the current House contains two?

    Second the Greens, having never elected anyone are over weighted IMO

    Just thoughts

  5. ...cont

    As to how to fix it, presumably if you have a large enough number of observations, the election specific factors should net out, so taking an average of the differences between polls and results over time should leave you with a rough approximation of the systmetic differences between polls and results. As you suggest, though, 3 observations just aren't enough to get you there. So trying to do that at this stage, may end up distorting the likely Liberal/Tory vote.

    As an ad hoc fix, at least until you get enough observations (which, if we keep having minority governments, may not take long), I'd suggest that you make the correction you observe for the Green vote (since their underperformance has been consistent over the past 3 elections) and maybe for the NDP as well, but don't adjust the Tory, Liberal or Bloc numbers (except for consequential adjustment resulting from the changes to the NDP/Green numbers, since if their share of the vote fall, the share of the other parties will increase slightly). I don't pretend that that is an elegant fix, but it may be a practical one in the short-term. In practice, this is what you've actually done (since the reweighting of the Liberal and Tory vote is quite small, and of the same order of magnitude for each), but I think this is as much a matter of luck (and your practice of discounting older elections - which, as an aside, is not appropriate when you're trying to identify systematic differences in voting results. By giving greater weight to recent election, you also give greater weight to election-specific factors in that election.) than anything else. For now, it works, but the same formula might not work after the next election (until you get more datapoints).

    Alternatively, you might be able to see if you can use older polling data for older elections to try and build up your dataset, but given that that would involve elections with different parties, using results from different polling companies and using different methodologies, I don't know if that would be worthwhile (as an aside, it's worth noting that there appear to have been significant changes in the polling companies over 2004-2008 as well, so it may be worth thinking about whether that has changed anything).

    I feel kind of bad about saying all this, because I've previously suggested that you try to make these sort of systematic corrections, but I've come around to your way of thinking (if that's any comfort).

    On a related point though, I think you're right that including older election results in your dataset doesnt't work to control for systematic differences in election results and given that, I think you should drop them entirely because they just distort your dataset. It's hard to see how the way people voted in 2004 is a helpful indicator of how they'd vote in 2010 or 2011.

  6. Eric,

    I understand what you're doing (and have previously advocated doing something like this), but I now think that your initial thought, that you don't have enough data, may have been the right.

    THe problem is that there just isn't enough datapoints to distinguish between systematic differences between polling results and election results and ideosynchratic (election specific) differences between the two. You should try to control for the former (since, being systematic, they will presumably occur again in a future election) but not the latter.

    For example, I suspect that there will be a systematic difference between poll results for the Greens. Their polling results (at least from some firms, other seem to get it right) will always exceed their ballot box results. This may be because people choose them in protest in polls (but don't actually vote for them) or because their supporters are consistently less likely to vote at all (i.e., if they're younger or more disengaged, on average, than the supporters of other parties). You could probably tell a similar story, to a lesser degree about the NDP.

    On the other hand, the differences between ballot box results and polls results for the Liberals and Tories dont' appear to be systematic (at least based on the 2004-2008 elections), but rather can probably be largely explained by elections specific factors (although there may be a systematic element buried in their - "shy tory syndrome", perhaps). In 2008 the over/under performance of the Tories/Liberals was probably due to low voter turnout amongst Liberals. The Liberal campaign was so uninspiring that a lot of Liberal voters sat on their hands. Conversely, the over/under performance of the Liberals/Tories in 2004 was likely due to Tory bumbling in the last week of the campaign which discouraged soft Tory support or caused them to shift back to the Liberals. In each case, though, those are election specific factors which, a priori, can't be expected to be repeated in the next election. So, to the extent that the over/under performance of a given party observed in 2004-2008 was due to election specific factors, I'm not sure they should be included in the adjustment.

  7. Good points Carl, but as you say, reducing the Green and NDP vote needs to be counter-balanced by increasing the Conservative and Liberal vote, which I've done.

    It isn't perfect, and I agree that are big problems, but some leaps need to be made for these predictions. We only have three elections to use as a basis for comparison, which isn't enough.

    As to including past elections in the projection, I have to disagree. Including them is one of the ways I incorporate the incumbency factor. The way people voted in 2004 helps us know how they will vote in 2010 or 2011 because people tend to vote for the same parties. Using the past as our guide, we know the Tories and Liberals won't get 50% or 15%, and that the Greens are more likely to remain at 8% or under rather than increase their vote to 12%.

  8. ERic,

    I agree that incumbancy may be a relevant factor, although that wouldn't explain why youd' include the 2004 or 2006 election results.

    And while many people will vote the same in 2010, as they did in prior elections, we know that many won't (that's why we change governments every now and then). Recent polls reflect those changes, 6 year old election results don't. By using old election data, you're distorting your dataset. Put it this way, the same argument could be made about 6 year old polling data, but you wouldnt use that (or if you did, you wouldn't give it the weight you give to past election results).

    And, yes, we know that the Tories and the Liberals aren't going to get 50% of the vote, but we don't need old election results to tell us that, current polls say the same thing.

  9. First, I think the ballot box bump the two larger parties get is the result of their being covered more and people with little attachment to the political system having more knowledge about them: greater familiarity, in other words.

    The incorrect assumption that's being made here, in my view, is comparing polls taken in the last week of the election to polls being taken now, when there's even less attention being paid to politics, and also probably less emphasis being put by the pollsters themselves on getting large, super-representative samples. Those guys know they are being judged on their results in the last week, not the ones they're releasing now. I just don't see the two as being reliably comparable in terms of their predictive power.

    Try and take the average of the polling results 6 months out and 8 weeks out and compare them with the final results of the parties, and see if that shows significant changes, because in addition to the polling booth bump at the end of the campaign (usually for the Liberals), there is another common bump at the beginning of the campaign, which is the "equal coverage bump" that the NDP always gets once the news organizations know they're being numerically monitored on that score. You might find that the effect close to polling day for the NDP is partially offset by the other effect going in to the campaign.

    The guy who ran showed all his polling on a timeline expressed as the number of days out from an election. It's a bit easier to do with a fixed election date I grant you (which we are supposed to have now, but for all practical purposes don't). But try and assume an election day of October 25, and see what that looks like.

    As to Shadow's question, I think there's something to that, but I wouldn't only look nationally. The Liberals have a lot of residual strength in their riding associations' bank accounts, and that may explain some of the ballot box bump for them.

    Barcs, if you're interesting in demographic predictors, I hope you've signed the petition on the Long Form Census, so we can continue to have them available.

    Keep up the good work.

  10. Shadow: Can one identify the ballot box factor based on the comparative fundraising ability of various parties over several cycles ?

    Good thinking. It's a nice hypothesis and the data is there to be analyzed. Éric, I see hours of fun in your future. I'd suggest making that the second line of investigation.

    The first is to split the vote by age groupings (as EKOS does) and multiply by demographic voting rates. This addresses the young, non-voting Green and the old, voting Tory. The nice thing is that no empirical parameters are involved. For extra marks, take that a step further and adjust using voting rates by sex and by region.

    Splitting each survey is of course unwieldy. However, if voting rate ratios between demographics are relatively stable over a number of surveys (which sounds plausible) and support for each party across demographics doesn't change much (which might be more of a stretch) then a single adjustment factor can be generated for each party. Sounds like a good first kick at the can, anyway.

    The remaining Tory premium and Green discount are presumably due to the GOTV (Get Out the Vote) machinery. The Green machine has matured greatly since the last election, which is why Shadow's proposal is so appealing. It links performance to a current number. The only disadvantage is the empirical relationship: what exactly is the function mapping fundraising to GOTV effectiveness?

    The challenge for the Green Party is to mess up the demographic adjustment by re-engaging the disenchanted and self-disenfranchised electorate. Wish us luck for the good of the country.

  11. Eric,

    I agree that if you reduce the NDP and greens numbers , you have to adjust the Liberal and Tory numbers. And I think, your results are probably the same numbers you'd get if you used my ad hoc method, at least for now.

    But what you've actually done is adjusted the Liberal and Tory numbers, based on their previous election results, not just based on the changes you make to the NDP and Greens. In this case, it works out because the Liberal and Tory numbers from the previous elections net out (though only because of your re-weighting). In this case, you got lucky, but it might not work out that way after the next election. Just something to think about for the future.

  12. I would go back to not trying to model this.

    3 data points is a ludicrously small sample to attempt to draw any conclusions from.

  13. PG,

    I think you're definitely right about comparing polls relative to similar polls at simiar time periods relative to the expected date of the next election. That's a point I've made before, and I think its an important one.

    I think voter behaviour changes when an election is imminent. Disengaged voters who may just name a default party when asked before a campaign is imminent, because they just can't be bothered to think about who they will actually vote for. It's always been suggested that the Liberals are that default party for most Canadians, and I know that Lispop had (thought I can't find it anymore) a long series of polling data that shows that the Liberals generally do worse in elections than pre-election period polls would indicate, and certainly we've seen that in 2008, 2006 and 2004 (and, we saw what happened last fall when the prospect of an imminent election arose - the Grits dropped like a brick).

  14. I'll add an interesting point to this discussion. In the UK, the conventional wisdom was that polls always underestimated the Tory vote (ie: the "Shy Tory" factor). In the 1992, 1997, 2001 and 2005 elections the Tories did better than the final polls indicated and Labour did a bit worse. However, in this year's election, there was a bit of a new "shy Labourite" phenomenon and it was Labour who did a bit better than the polls predicted and the Lib Dems did a lot worse.

    My conclusion is that polls tend to underestimate support for big parties that are polling well below their historic norms that have leaders who are objects of ridicule and who the press has decided have run a bad campaign. This causes people who traditionally support the party in question to be embarrassed to admit that they would vote for it.

  15. Well, well, lookie what Aaron Wherry is blogging about today...

  16. Interesting stuff.

    When I went about looking at this for my seat projects, I took at look at the regional breakdowns of the last two elections plus a flury of provincial elections. So, this gave 12-15 data points.

    My view was that the results weren't systematic except in the case of the Greens (and I came up with a very similar correction factor as you do here).

  17. TPG,

    I am completely on the fence on the long form census. (and the mandatoriness of the shorter one)

    I agree that there is value in having the data to the government and to the country as a whole.

    I am however leary about some of the consumer and advertising groups expressing support and wanting (continued) use of the data. They might not get my name... but it does seem like an abuse of my privacy still.

    One of my neighbors was almost charged last time it went around because he didn't want to participate. I agree that it kind of limits the usefulness of the data if he doesn't..... But are we about forcing people to do things in this country? He doesn't much like giving the government his data (and I can understand why,... I have problems every time I am forced to deal with a government agency... crop insurance, revenue canada, GST... did you know that the GST department can alter your numbers, the 5 or 10 you give them without any information that the numbers are wrong?, payroll, gun registry, etc etc. If I had customer service at any private business like I get from the government they would soon be out of business.

    But even that is beside the point.... partly because a portion of the data is false. I have a couple of other neighbors that refuse to fill out the census because they don't want to give the government and others such information. Only since it is illegal to not fill it out... they do fill it out with gobbledy gook. Apple trees, strawberries, chickens, llamas, one even put forestry on his.... on bald prairie.

    You tell me which does more harm? The guy who just refuses to fill it out? Or the ones that actively lie to keep the government out of their business....

    So even tho I think that the dataset that can come out of a mandatory census can be extremely important... I also have to question its use, and its accuracy.

    So on the petition to keep it.... I am completely on the fence :/

  18. Éric - I'm very happy with your method, here. You've weighted more recent elections more heavily (because if there's been a change in a party's organisational strength, for example, the newer values will be more indicative of future results). I see no reason to complain about this at all.

    The biggest improvement, I think, is that you'll now be predicting much more realistic vote numbers for the Green Party, which prevents then from splitting the vote and giving the Conservatives an unrealistically optimistic projection.

  19. There was certainly a "Shy Tory" effect for the Reform vote in 1993 and 1997, where they outperformed their poll numbers by about 80% on election day.

  20. TPG, I have been trying to do some more learning about the census.

    Can someone explain to me how changing the long form 20% mandatory to 33% voluntary does such damage?

    I am aware that the differing method of tracking taints data somewhat. But given the problems I posted earlier about falsified data and refusal to give data. Wouldn't this produce a better dataset in some ways?

    And why aren't people talking about this in the context of change?? I spent quite some time reading to learn that it was a change and not a removal.... Lazy journalism,.... coupled by lazy spinmasters and bloggers. Usually I can count on atleast a few bloggers and real journaists to fact check the "journalists who are perfectly unbiased and completely without an agenda".

  21. "There was certainly a "Shy Tory" effect for the Reform vote in 1993 and 1997, where they outperformed their poll numbers by about 80% on election day."

    That's not true at all. The final polls in the 1993 and 1997 elections had the Reform Party at 18% or so and that is exactly what they got when the votes were counted.


    Good argument for the usefulness and need of a census.

    But I am still looking for a good read on why it should stay the same as it is rather than the change from 20% mandatory to 33% voluntary.... the only good reason I have found so far... is that it will be a little difficult to compare the previous data to the future data collected.

    In all the stuff I have been able to find there has been no real discussion on the merits of each collection method..... Which I think is kind of required to make an informed decision.....

    The requirement for real actual journalists... has never been greater, nor the current prospects bleaker.

  23. Hi Barcs I think it was Stephen Gordon who got the census ball rolling and then leftist blogger Aaron Wherry over at macleans who really whipped things into a frenzy.

    The issue Gordon raised is that by making the census optional certain kinds of people more than others will choose not to do it.

    The theory is that the illiterate, the poor, minorities, people with health problems, and many other groups will be less likely to fill out their long form than the average Canadian.

    So the picture that's being painted of Canada will be inaccurate. More white, more middle class, more educated, more stereotypical Canadian than it actually is.

    How big a deal its going to be isn't known.

    I'd suggest it won't be as big a problem as everyone suggests because

    a) many people don't fill out their census already. Hardly anyone is fined or charged, usually they just get an exemption.

    b) people who would skip the census are already giving lousy data. Making things up or answering randomly is common. Its not like anyone double checks your answers!

  24. Wells seems to think that there is a solid 10% shift from Liberal to CPC in the last 3 elections.

    Wells has been an anti-Harper pit bull for the last year or so. It seems that Liberal President Apps has ticked him off more with his calling out of the anti-Ignatieff media and poll analysis.

    from the article:
    "As a rule of thumb, the Harper-era Conservative writ-period bounce seems to be about five percentage points or a little more. The Liberal writ-period decline is comparable. Which means if the two parties are tied in voter support on the day a campaign begins, the Liberals should, as a rule of thumb, expect to be 10 points behind when people actually vote. Right now the two parties are not tied."


    "So that’s five federal elections in a row, 1997 to 2008, where the trend is for the Liberals to bottom out and the (assorted conservative, then Conservative Party) opponent to have better results than recent polls had indicated."

  25. Another point that should be made is because of this bizarre decision - it is actually going to cost the taxpayer way MORE money. It has been estimated that mailing out the long form to a third of all households and making it non-mandatory as opposed to mailing it to 20% of household and making it mandatory will cost about an extra $30 million dollars.

    Oh well, what's $30 million from the folks who just wasted a billion of G20 "security" and are now about to flush $16 billion down the toilet of fighter jets and that we don't need.

    The Harper Conservatives are with out a doubt the most money wasting and fiscally irresponsible government of all times.

  26. OT! Green Party Leadership Race On:

  27. Interesting Paul Wells observations on polling trends and elections:

    Very good news for Conservatives!

  28. Earl, BCVOR what Wells is talking about is interesting with regards to what Eric does here at 308.

    Right now the CPC is at around 33, the Liberals at 28. By historical standards that translates to 38% slim CPC majority and 23% all time Liberal low.

    Of course, campaigns matter and history doesn't always repeat itself. But its a pretty robust pattern.

    Whether Eric should apply the +5% CPC, -5% LPC modification is up to him. (I'd add a -3% Green, +3% NDP modification too.)

    In the end it doesn't matter because once a campaign gets underway the polls correct themselves and his projection will be made accurate.

  29. Barcs apparently 21,000 Canadians registered Jedi knight as a religion in the 2001 census:

    I'm guessing exactly 0% of those people were fined or jailed for answering incorrectly.

    It kind of bolsters the case that people put a bunch of random stuff down for their answers.

    Not forcing people to do the census would weed out people with bad attitudes and provide more accurate data.

  30. Here's the chart I was thinking of from Lispop, which compares pre-election polls with election results going back to 1962.

    The striking result is that, with two exceptions (1974 and 1993) the Liberals have always done worse than pre-election polls would suggest. There isn't as obvious a trend for the Tories, although they generally do as well as pre-election polls suggest or a bit better (only 4 times over that period have they done worse than pre-election polls suggest, atlhough on two occasions the underperformance was nominal, 1%. The other two occasions was the 1993 fisco and the '79 election).

  31. The issue with a voluntary vs mandatory census...

    The census is what allows you to tell what the 'real' numbers are. If you make it voluntary how do you know how much to adjust results by to match the public at large? IE: what percentage of the population is first generation Canadian? A voluntary census (or poll) will not provide a very accurate figure as the percentage will be low, and the margin of error high enough to make it impossible to know for certain if it is growing or shrinking.

    Now, we could mix together the numbers from Revenue Canada with figures from hospitals (births/deaths) and with stats from immigration and other departments and probably get a very accurate picture but then it would all be joined together via SIN numbers or something like that thus easy to tell who makes how much plus remove all privacy between government departments which I suspect would be a far bigger issue.

    Is the census a pain? Yes. Does it feel intrusive? Yes. Is it necessary if we wish to have effective government? Yes. If you feel government should not exist, then the CPC has the right idea. If you feel we need roads, hospitals, schools, etc. and not have to guess as to trends that lead to decisions on them (plus blow a fortune on polling companies) then we need to have a full census.

    FYI: just imagine if a certain group decides to push filling out the voluntary long form - suddenly that group could end up looking many times stronger than they are and get far more gov't goodies.

  32. Peter... That opinion article is just another arm waving hand wringing exercise.

    It does nothing to discuss the merits of either method and only makes dire predictions if it is changed.

    Further to the article it uses the AGW-like line "Statisticians are unanimous:".... Which can only make me believe that the rest of the article is as badly researched.

    Like all the other articles I have been able to find it is disappointingly devoid of facts on the issue. Yes, even the NP has failed in its journalistic duty.

  33. I'm amazed people think the census is going to be an issue.

    Remember the hot topic of overseas abortion funding ?

    Everybody has forgotten it. Totally manufactured issue. The G8/G20 were a giant success for Harper. Canadians LOVED having them.

    If something as debate provoking as abortion had zero political traction imagine the effect of something as boring as the minor differences in sample composition and data quality between a mandatory and voluntary survey.

    HINT: Discuss things voters actually care about like the ECONOMY instead of things you wished voters cared about.

  34. John:

    If you feel we need roads, hospitals, schools, etc. and not have to guess as to trends that lead to decisions on them (plus blow a fortune on polling companies) then we need to have a full census.

    Plus the use of the voluntary means two additional things. First the long form can't be targeted, and secondly with the voluntary there simply isn't any way to correlate what has gone before with what you get now.

    Two big factors against the voluntary IMO.

  35. Barcs:

    Yes, even the NP has failed in its journalistic duty.

    That isn't really an "Opinion" piece at all but rather an Editorial. That's why I put it up. More weight than just an Op Ed

  36. Barcs,

    There's no questions that, statistically, you need a mandatory census if you want to be able to get a representative sample of the Canadian population. So, on that front, the statisticians are right. If the census is voluntary, that sample won't be representative.

    But while the stasticians are unquestionably right on that front, that doesn't resolve the issue. First, it's an open question as to just how skewed the dataset will be if the census is voluntary. In other words, even if the sample is not representative of the Canadian population, it may be "close enough" (for government work, as they used to say). Moreover, the assumption being made is that the current census is represenative of the Canadian population. That may be true, but if the people who would otherwise opt-out of the long-form census are people who would otherwise fill out their census with gibberish, that's an assumption that's worth questioning.

    But, even if the statisticians are right, and the new census will not be as useful as the old census, that doesn't settle the issue either, because it's a fair argument to suggest that Canadians shouldn't be compelled to provided detailed information into their private lives. That's just a weighing exercise between the value of private rights and public goods, and it's not clear that Statisticians neccesarily have any edge in that assessment.

  37. The Fraser Institute routinely cites StatsCan data in their studies, and yet they appear to be the only group against the mandatory census.

    Fraser Institute's stance

    I find thi argument from privacy and fairness pretty compelling.

  38. Shadow: "I'm guessing exactly 0% of those people were fined or jailed for answering incorrectly".

    Who says the answered the question incorrectly? Maybe they are Jedis (or is it Jedii?).

    May the force be with you.

  39. I've filled out the long form census. I didn't like filling it out and found it intrusive, however I do think that it is necessary. Harper's decision is the wrong one. As for his reasons for doing so - to satisfy the base - I'd think getting rid of a silly and expensive pleasure boat test would make a lot more sense. Where exactly is his base going to go if not to the CPC?

  40. Carl I must object to your utilitarian framing of the census issue, I certainly wouldn't call it a "weighting exercise".

    People work half their lives for the government, if necessary stand for conscription, perform jury duty, register for identification, and obey a myriad of rules and restrictions on their activities.

    It may seem small and petty given the above but sometimes its the little things.

    Canadians would rather spend one hour with their kids than filling out a long form census to subsidize the non-essential activities of business lobbyists, social activists, and academics.

    Sometimes the scales just break and the "weighting exercise" becomes a situation where the ends, no matter how lofty, just don't justify the means.

    It reminds me of the following scene from The Brothers Karamazov where the modern intellectual Ivan says the following to the moral paragon Alyosha:

    Imagine that you are creating a fabric of human destiny with the object of making men happy in the end, giving them peace and rest at last, but that it was essential and inevitable to torture to death only one tiny creature—that baby beating its breast with its fist, for instance—and to found that edifice on its unavenged tears, would you consent to be the architect on those conditions? Tell me, and tell the truth.”

    “No, I wouldn't consent,” said Alyosha softly.

  41. Changing a census into a voluntary one from a mandatory one does indeed add self-selecting bias into a survey that did not previously have it. I also do not agree with the suggestion that no longer requiring people to fill out the census would somehow prevent or encourage people to not intentional provide incorrect information. A prank like faking adherence to the Force to get it into the census isn't prevented by making the census voluntary.

    Barcs, just because the media does not advance a theory you would prefer to believe does not mean they are not providing facts.

    I had read that the census was being expanded as it was being made voluntary, but if no statistician came forward to back the government's claim that this was just fine, it doesn't mean that the media wasn't providing facts. If may just mean that there were no statisticians who agreed with that arguement. Certainly the users of the data provided by the census don't seem to agree with that argument.

    Shadow; here we go again I see.

    Definition of manufactured issue:
    1. Government does something or doesn't do something which has genuine cosnequences.
    2. Other parties, interest groups, or the public oppose them.
    3. Polls show a majority agrees with Opposition or an inquiry is assigned to investigate.
    4. Government remembers they only have to care what 35% of voters think.
    5. Someone insists that since the Conservatives are still in government/have pissed people off about something new by now/are still polling above 30% that the issue does not exist, is manufactured, or that Canadians don't really care.
    6. Someone on this blog asks why the Harper government keeps having its legitimacy questioned so often.

    People advance issues regardless of political traction precisely because they do care and there are real issues at stake.

    Ira; I think the Fraser Institute is just generally in love with the idea of the government no longer doing a service and instead paying some private body to do a worse job.

  42. "That isn't really an "Opinion" piece at all but rather an Editorial. That's why I put it up. More weight than just an Op Ed"

    BS.... A paid guy to the newspaper has no more knowledge on the subject (unless he researches it) then some angry guy on the street. He is there because he can write, not for his expertise on the subject.

    And secondly... given that it is in the Editorial section.... That is the definition of Opinion. The only difference between the Editorial and the Opinion section is that a guy working for the paper wrote it. And I do stand by my assessment that it is a complete fluff piece with little or no knowledge or facts.

    Carl, the short form is still mandatory... its the long form they are dickering about. And as I said way above this post... I do think some of the data is important... Which is why I am on the fence on this long form issue.

    I am not sure why the mandatory short form cannot be used to weight the voluntary long form data as it is now on for the mandatory long form....

    I don't accept that "this group might not fill it out as much" as an argument.... government funding of programs and infrastructure may not happen (will be reduced) for you if you don't.... That seems like a pretty good incentive to me to fill it out. And if you don't care??? Well,... I don't really either... I and the government I pay my taxes to are not here to protect you from stupidity or from not caring about yourself.

    So once again I am left with "The past data won't perfectly match up with the data going forward" vs "I don't think I have the need to give the government and others this kind of information"

    I haven't seen a real concrete argument outside of those two.... Nor have I really seen someone trying to create a real argument outside of those two, the rest like the "groups might be underrepresented" argument is full of chains of ifs and maybes and topped off with a bunch of garbage.

  43. Barcs,

    The short-form is still mandatory, but the contents of the short-form are far less than that of the long-form. Basically, it's basically, number of people, age sex, relationship language. So you can use it for weighing for those purposes (how many people live where, gender, age, etc.). You can't use it to weight out factors (education, ethnicity, disabilty, income, etc.) which may be relevant. The statisticians do have valid arguments.


    I don't think I was making a utilitarian argument, I was simply pointing out it's a trade-off between public goods and private rights. How you weight those trade-off depends on your perspective. If you're a libertarian, the private rights are paramount. If you're a utilitarian (and believe that voluntary data is materially less valuable than compelled data), maybe you side with the public good. I'm not taking a stand one where or the other, I'm just framing the question.

    I think Tasha whatherface in today's Post had a valid point, that maybe the solution is a compromise. Weed-out some of the more intrusive questions (especially those where the public benefit isn't obvious: Is there really a compelling public need to know how many rooms there are in my house or how many hours I spent washing my car) but keep the compulsory element.

  44. Earl,

    It's worth remembering that the CPC base has been known to say "F-this" and to set up a new party (as they've done only recently in Alberta). As someone who played a major role in setting up the new party the last time they did that, Harper is probably pretty sensitive to that risk.

    OF course, the other reason you play to your base is to keep them fired up and to get volunteers and donations down the road. I'm not saying that's what's happening here (because I don't see this as being a real big issue with Joe Q Public), but that's why you do it.

  45. The police should of made the Black Bloc and the G20 protesters fill in the long forum census so the government would have an accurate picture of their background and root causes and adjust social programs to accommodate them and make their lives fuller.

  46. Is the census any more accurate than the results of surveys done by private polling firms?

    If so, why is that?

    And if not, why is the government doing the census at all? Why not rely on private data collection?

  47. Well, at least now we know what happens to data quality when you move to a voluntary survey.

    Apparently our Yankee cousins tested doing that in 2003 only to find that the data was far less useful and it cost far more money (because they sent out more surveys). So, it seems the statistics have real concerns, rather than just statistical ones.

  48. Ira, I suppose the infornation might be the case if the data collection were voluntary. But certainly no private company could eliminate the bias inherent in allowing non-responses.

    I'm not sure if the breadth of information accumulated by StatsCan is replicated by any private agency at the moment.

    But StatsCan is pretty well respected apparently, and there's no reason to assume that some private source (Which may not even exist) must be more cost effective or more accurate. Or that we'd be better off trying to create a private market of indeterminate value for something by creating a shortage, and screwing with something that already worked.

    Incidentally, the Globe is reporting that a 2003 effort to create a voluntary census in the USA was canned because plummeting reponse rates made the info useless:

  49. Carl i've suggested what I think would be a reasonable compromise.

    Extend the 8 questions on the short form to 15 questions. Have four rotating versions with different questions 9 through 15 which will be sent out at random.

    Make completion of your census mandatory but include as a possible answer for questions 9 through 15 "decline to answer due to privacy concerns".

    Remove jail sentence as a penalty. Stop harrasment tactics for people who don't fill their census out.

    And only include questions the GOVERNMENT needs to conduct its operations.

    Forget bussiness lobbyists, social activists, and academic researchers.

    They can hire a polling firm!

  50. "Barcs, just because the media does not advance a theory you would prefer to believe does not mean they are not providing facts."

    Huh?? Sorry if that's the vibe you got Kevin, but I am not asking for an adoption of my opinion.... I am asking for people (like journalists, policy wonks, etc) to do their job and research the facts and present them to the public. To date all I have seen is a bunch of hand-wringing....

    "Certainly the users of the data provided by the census don't seem to agree with that argument."

    ... of course not... who in the information business who doesn't like easy access to very cheap data??

    "If may just mean that there were no statisticians who agreed with that argument."

    Zero? Nada? Zilch?.... Bull. It doesn't matter how stupid an idea you put forth, there is always someone to support it. And if there isn't... How did it come up as an option? If no one anywhere supported it,... how did it become an idea,... and then an idea that got passed on??

    Ira,... That is a good question. And I think you have hit upon the reason many of the mentioned stakeholders want to keep it... Why pay pollsters when the government will force people to give them the information and then sell it to you cheap.

  51. Carl.... cost more money?

    How does sending out a few extra pieces of paper cost more money??

    Answer, sending a $5-10 envelope is cheap, data entry by a government employee is the time consuming expensive part....

    The bulk of the cost is in getting them back. Which means... many people seem to have sent them back in.... more than the smaller mandatory survey. So how much is the worry that people won't do a voluntary survey?

    Atleast the story does provide some facts, (finally). And it is an interesting read.

    Although... I am somewhat amused that it seems to hit and cover every hand-wringing question so far... and follow it up with quotes and condemnation of the evil rightwing republicans and tea party who oppose the current system..... With an obvious bias against changing the census, and against the right... is the rest of the story any good? Or is this really balanced journalism?

  52. "Incidentally, the Globe is reporting that a 2003 effort to create a voluntary census in the USA was canned because plummeting reponse rates made the info useless:"

    1 in 5 = 20% mandatory
    1 in 3 = 33% voluntary

    "When a percentage of Americans were given the choice of filling out the national survey, the mail-back response rate dropped by a third."

    ...... 33% dropped by 1/3... = 22%

    In other words... It seems to have been taken into account.

  53. Carl according to wiki the Americans scrapped the long form census:

    Apparently because many found it to be burdensome and intrusive. They've replaced it with a rolling, yearly survey:

    Supposedly its mandatory (although no jail time) but nobody has ever been fined and its of dubious legality whether they could.

    Canada seems to be more or less adopting the American position of the US Democrats and Barack Obama by eliminating the long form census and replacing it with a voluntary household survey.

    Since the US version is effectively voluntary anyways I fail to see the difference.

    I'm not sure why the American experience is being used as an example to bolster the case of the pro-census folks when the facts seem to be more complicated.

  54. G8/G20 Opposition

  55. Barcs,

    They may get the same number of respondents with a voluntary survey if they send it out to 1 in 3 vs. 1 in 5 people, but the data isn't likely to be useful if, as you suggest 2/3rds of the recipients don't respond. So even on your account, they'd be spending the same amount of money on a far less useful collection of data. In that case, they might as well just scrap the long-form census. In any event, the estimate that the voluntary survey will cost more isn't something that I've made up, it came from the government. It anticipates that it'll cost an extra $5 million to send the extra long-form surveys and a further $25 million to try to encourage non-respondents to complete them. I don't know, paying more for crappier data seems to me to be a lousy investment.


    I'm not sure why you say the US scrapped the the long-form census. They simply moved from a survey every 5 years to a rolling survey every year. They changed the form and name of the long-form census, but they kept its function.

    And note, it is still mandatory. You say it's "effectively voluntary" because no one has ever been fined. Of course, you don't know that for a fact, but in any event, the fact that the response rate changed so dramatically when the the survey was explicitly made voluntary suggests that the fines do have the effect of compelling people who would otherwise not complete the survey to complete it.

    The American experience is telling because they tried to do eactly what the government proposes to do and it didn't work. That's why it's being brought up. What's the line about insanity being doing the same thing over and over and expecting different results?

  56. " Peter said...
    The $16-billion fighter jet boondoggle"

    Good job peter.... you picked a guy that is somewhat more vested in the liberal party than Chretien, Martin, Dion and Iggy all put together.

    (But atleast you pointed out that the NP doesn't lean as far to the right as you claim)

    He argues that its a boondoggle, then dismisses the other options as "not really worse, but not better for Canada"

    Then argues that the sole-source contract is bad.... (but from my point of view it would seem to be required given the number of companies currently producing top of the line fighter jets that also have the plans to build this plane. You might be able to argue that there is other planes. Jeff did, but decided they weren't better)

    He finishes off stating that we do need to do a deal of this magnitude to replace our aging fighterjet fleet.

    His sole problem seems to be in the sole sourcing, with a jab at the conservatives for pointing out that it was the liberals who got us involved in the program. (evil conservatives no less)

    Do you even read past the headlines of the stories you post? Perhaps you would like to inform yourself on a few of the issues and not make your decisions on "Harper bad, tories evil, right wing = any and all bad things...."

  57. "The bureau followed up by phone,.... and found less cooperation when people were told it was a voluntary survey"

    .... hmmmmm you mean to tell me that some people think it is an issue??

    "The bureau saw a “dramatic decrease” in response rates when they tested out a voluntary version of the ACS, with mail-in replies dropping by about one-third. Only 22% of black households and 20% of Hispanic households mailed back a completed survey when it was voluntary, compared to 43% of white households and 42% of non-Hispanic households."


    Did anyone else notice that even the NP didn't compare the after numbers with the before numbers? How many of each group returned the mandatory one?

  58. "I'm not sure why you say the US scrapped the the long-form census. They simply moved from a survey every 5 years to a rolling survey every year. They changed the form and name of the long-form census, but they kept its function."

    So you say that in the US (during the middles of the Clinton Democrat reign), it is possible to change both the questions; and the delivery method.... without irrevocably tainting the data like the major argument against changing the Canadian census???


    I am still on the fence Carl, but you and peter are presenting a compelling argument for change.

  59. One thing getting lost in this census debate is wbether we should be allowed to force people to answer questions they don't want to answer.

    The CBC went out on the street to ask Canadians if they found the questions too intrusive, but that missed the point. The point is, should I be allowed to use the threat of legal sanction to compel people to do answer questions they don't want to answer? Whether I'd be willing to answer the questions is irrelevant.

    If you're charged with a crime, you're not required to cooperate with or talk to the government about it. So why do we not offer ordinary citizens the same courtesy?

    This isn't about effectiveness or costs. This is about forcing people to do things.

    It's refreshing to see this government finally take a principled stand on something.

  60. without irrevocably tainting the data like the major argument against changing the Canadian census???

    Because even though the format and delivery changed it remains a compulsory survey. Don't know what the US penalties are but there are some.

  61. But at least you pointed out that the NP doesn't lean as far to the right as you claim

    Oddly I've noticed that under its new ownership it is slowly shifting left to.

    Why?? Maybe that's what the market wants ??

  62. "Don't know what the US penalties are but there are some."

    Not quite sure when it changed.... but it was up to $100,.. and is now up to $5000 for non compliance.

  63. "Oddly I've noticed that under its new ownership it is slowly shifting left to.
    Why?? Maybe that's what the market wants ??"

    A good theory.... but not the first time Jedras has been published there.... Unlike say the CBC or the star, the NP DOES show some balance.


COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.

Note: Only a member of this blog may post a comment.