Friday, September 19, 2014

Scotland's decision and the polls

In the end, the referendum result on Scottish independence was not as close as the polls made it out to be. Granted, it was still a relatively near-run thing. But the 55% to 45% result was not the 52% to 48% the polls suggested would be the case. Why?

Put away your pitchforks for a moment, and let's look at what might have occurred. Undoubtedly, the pollsters that were in the field in Scotland will be doing some work of their own. It will be interesting to see what they come up with in the coming days.

But first, let's look at what the polls said would happen.

If we do a simple average of all the polls that were in the field to September 16 or 17, we see that, after the removal of undecideds, the Yes side was expected to take 47.8% of the vote, with the No side at 52.2%. The Yes side was thus over-estimated by 3.1 points, and the No side under-estimated by the same amount.

By that measure, it was a bit of a miss. All but one of these polls had the Yes side at either 47% or 48% after the removal of undecideds. Missing by two or three points is not a horrible result, but all of the polls missed in the same direction. This suggests it was not a question of bad polls exactly - if they were simply done incompetently, some should have had the Yes side lower than 45%. Instead, there was an issue with what people were telling the pollsters.

The average support for the Yes side before the removal of the undecideds was 44.1%, or 0.6 points lower than the result. In fact, virtually all of the polls had the raw Yes support at or lower than 45%. Could it be that the undecideds, who averaged 7.4%, swung to the No side?

If that was the case, it means that roughly 90% of undecideds voted No, with just 10% or so voting Yes. That is a rather big number. In 1995, support for the Yes averaged 47% in the final polls with support for No at 42%. Portioning out the 11% of undecideds suggests that about 20% of them would have voted Yes, and 80% of them No. So perhaps the lopsided result in Scotland is not outlandish.

But the undecideds may have just not voted. And an exit poll done by Lord Ashcroft suggests that the last-minute deciders swung to the Yes side, not the No side. His poll suggested that 15% of Yes supporters made up their minds in the last few days of the campaign or on voting day itself, compared to 6% of No voters. This means that roughly two-in-three voters who decided how to vote at the last moment swung to the Yes side.

Assuming that is accurate, that blows the theory that it was a disproportionate swing to the No side among undecideds that messed the polls up.

Another finding from the Ashcroft poll, however, points to something else.

The poll found that 14% of No voters would be 'reluctant in any way to tell your friends, family, or colleagues how you voted', compared to 11% of Yes voters. That is not a huge difference, but it could explain some of the error if we assume that people who would be reluctant to tell their friends how they voted would be reluctant to tell pollsters how they would vote.

Turnout might be another contributor to the miss.

In the four councils that voted for independence, turnout was 79.1% (Glasgow was the main culprit). In the remaining 28 councils that voted against independence, turnout was 86%. If we use that as a rough measure of how much more likely unionists were to vote than secessionists, then the polls would have instead had the Yes at 45.7% and the No at 54.3% - a much closer prognostication than their actual tally.

Put these two factors together, and virtually all the error is accounted for. It means a combination of being less likely to vote on the Yes side, and being more likely not to reveal voting intentions on the No side. So the question then is - was it ever going to be close?

20 comments:

  1. The polls overestimated "yes" in our 1995 referendum too.

    ReplyDelete
    Replies
    1. You wound me, sir, with your non-reading of my post.

      Delete
    2. I don't see any mention of Quebec?

      Delete
    3. I guess it was subtle:

      If that was the case, it means that roughly 90% of undecideds voted No, with just 10% or so voting Yes. That is a rather big number. In 1995, support for the Yes averaged 47% in the final polls with support for No at 42%. Portioning out the 11% of undecideds suggests that about 20% of them would have voted Yes, and 80% of them No. So perhaps the lopsided result in Scotland is not outlandish.

      Delete
    4. Or perhaps people just answered one way and then voted another due to social stigma. Classic Shy Tories.

      Delete
  2. "Was it ever going to be close?"

    I would argue that it was close. 44.7% is a massive chunk of the electorate. And given the voter turnout, I'd say the Yes side likely got more votes than any political party ever in the history of Scotland.

    There is massive support for independence in Scotland. It just isn't majority support.

    65-35 isn't close. 55-45 is.

    ReplyDelete
    Replies
    1. Hopefully the British government takes this seriously and continues with devolution.

      Delete
    2. Totally arbitrary what is and is not a close result. Unfortunately there are probably enough nationalist who agree with your reading Ira and Scotland much like Quebec will be placed in a neverendum situation. Salmond sensibly stated that he believed the result should last for a "generation" but, with his departure the new leader may hold a different view.

      Delete
  3. "Was it ever going to be close?"

    In retrospect No. The biggest challenge for referendums such as this is fear of the unknown. This fear can be overcome, but retorts such as the PQs favoured "economic terrorism" or the SNP "negative fear-based campaign" do not cut the mustard with sufficiently many voters to sway the vote to the Yes side.

    The SNP would have done better if it (1) had a currency plan and (2) had admitted to separation pains to be followed only later with gains. The electorate can smell a rat when promised rainbows in a separate Scotland. Lastly the national identity wasn't strong enough to overcome the fears. This was very much an open question going into the referendum, but now we have an answer.

    The last point to make in this rather long post is that people sometimes lie to the pollsters. Few would admit to voting for Thatcher in the 80s or Mike Harris in the 90s yet they won several elections. People will sometimes answer what is expected of them on the phone, and then do something else when the moment of truth comes.

    This is where voter sentiment analysis plays a role and the basis of my prediction.

    ReplyDelete
    Replies
    1. Some people also simply change their mind (without lying).

      Delete
    2. "the national identity wasn't strong enough to overcome the fears".

      That is an insulting statement to Scots the world over. You should apologise or retract your statement lest people start criticising your nationality!




      Delete
    3. Polstats,

      "the national identity wasn't strong enough to overcome the fears".

      As someone who is Scottish I find that statement very insulting. If someone said the same about Quebeckers some people would be outraged. Fundamentally it is a poor comment; 3.6 million people voted and you simply can not know the reasoning behind all those 3.6 million votes. In any case you have brought forward no evidence to prove your point.

      Perhaps Scottish people like the UK or like their jobs a BAE or HMNB Clyde. Perhaps they have doubts about the amount of oil remaining. Asking pertinent questions about what currency an independent Scotland would use, or government expenditures or its relationship with the UK is not fear-they are legitimate questions. The fact both the Scottish and Quebec nationalists had difficulty answering these questions demonstrates weak planning on their part.

      More fundamentally the idea of sharing the Pound or the Canadian dollar is flawed. Not simply because of the dangers and lack of flexibility such monetary unions can entail (look at Greece) but, from a philosophical perspective. A country can not be independent if it does not set its monetary and fiscal policies. If monetary and fiscal policy is set in London or Ottawa foreign countries will have undue influence on Scotland or Quebec, interest rates may adversely affect economies. Scotland and Quebec would have less influence to manage and persuade decision makers than they do today. That is not fear it's realpolitik.

      Delete
    4. @Bede

      I'm trying, rightly or wrongly, to ascertain the causes of the vote. This is not about being or not being offended. It is about what did or didn't happen.

      3.6 million people voted and you simply can not know the reasoning behind all those 3.6 million votes.

      You are wrong about this. That is what after the fact polls are for and Lord Ashcroft polls found that people were afraid of the economic consequences. This is a very valid and rational fear that is many instances overcome by nationalism. It didn't happen in this case.

      Please try to keep sentiment outside the evaluation of figures. If you don't you end up with "unskewed" polls like Romney.

      Delete
    5. Polstats,

      You are not ascertaining anything you are making statements! Such as: "If you don't you end up with "unskewed" polls like Romney"; "This is not about being or not being offended"; "the national identity wasn't strong enough to overcome the fears".

      You don't speak for Scots and you certainly don't speak for me! How can you possibly know if I am, offended or not? You assume and that is your problem!

      You also forget the nuances; you don't say some Scots or a per centage of Scots cast a No vote due to economic concerns you say "the national identity". You have done the same with Lord Ashcroft's poll that clearly identified 52% of No voters had reasons other than economic concerns. Once again however, you forget the nuance and state; "people were afraid"! By doing so you imply something about Scottish people. It is deeply offensive. You should apologise and leave it at that.

      I understand your English language skills may not be perfect and that the impact of your words is an unintended consequence of this imperfection not a malicious act but, I would advise writing more carefully on controversial or sensitive topics and I think you would be well advised to buy an OED. .

      Delete
  4. I think it is simply another example of "brains" overcoming "heart" !!

    ReplyDelete
    Replies
    1. That is a bad omen for the federal Liberals.

      Delete
  5. Did any of the Scottish polls reapportion the undecideds? I know Canadian polls do this all the time and seem to be quite good at it.

    ReplyDelete
    Replies
    1. Many polling firms did reappropriate undecided voters. From my reading of them it appears the usual method was to split undecideds 50/50.

      Delete
  6. The MB seat of The Pas is currently vacant but it does not seem to be on your byelection barometer.

    ReplyDelete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.