Wednesday, February 23, 2011

The Incumbency Factor

Recently, I've been looking at the issue of incumbency and how to factor its effect into my upcoming riding-level projection model. The investigation resulted in an article for The Globe and Mail, but I had focused on one aspect of incumbency in that piece: the re-election rate.

I've continued my analysis and have now gone through all 308 ridings for the 2006 and 2008 elections. Rather than focus on re-election, I have focused on an incumbent's ability to out-perform, or under-perform, their party. This is very useful for projecting electoral outcomes.

For example, in British Columbia in 2006 the New Democrats increased their support as compared to the 2004 election by a factor of 7.5%. That isn't to say they increased their support by 7.5 points, but rather their support increased 1.075 times. However, in 2008, their support stood at only 91.3% of their 2006 total.

For NDP incumbents in British Columbia, then, I looked at whether they under-performed their party in 2006 (i.e., increased their support at a lower rate than their party did) and over-performed their party in 2008 (i.e., lost support at a lower rate than their party did). This would seem to be logical - incumbents should be less prone to losing support than, say, a random candidate in a riding in which the party stands no chance.

However, I didn't stop there. I broke it down by party and the number of times an incumbent had been re-elected before. Looking for patterns and trends, I found that in some cases there is a statistically significant incumbency effect.When an incumbent's party loses support in a province or region (region being the Prairies and Atlantic Canada), the incumbent loses support at a lower rate than their party in 75.1% of cases (333 such cases in 2006 and 2008). In only 24.9% of cases does an incumbent lose a greater proportion of support than the party as a whole. That's significant, and on average an incumbent out-performs a sinking party by 8%.

However, when a party gains support in a province or region, an incumbent performs worse in 53.8% of cases. Assuming that the incumbent vote has a higher degree of stability, we would assume that they would tend to increase their support at a lower rate than their party, but 53.8% of 195 cases is not very significant. It is even less so when you average out the "incumbency effect" when a party gains: 1.00. If you extend it to a few more decimal places you get a slightly negative effect, but the incumbency effect when a party is gaining seems to be virtually zero. If a party has a gain in support, incumbents tend to increase their support at the same rate in their riding.

Breaking down an incumbent's performance by party when that party gains support in a particular region did not show any significant variations. Liberal incumbents did much worse than their colleagues, but there was not a large enough sample size (14) to determine anything significant from it.

That was not the case when an incumbent's party loses support in a particular region. Breaking that down by party did show a large degree of variation, with New Democratic incumbents appearing to be the most secure. This would, on the face of it, make some sense. Voting for the NDP is very different from voting for the Liberals or the Conservatives - it is almost certainly a vote for an opposition party. This means that voters in NDP ridings tend to like their NDP MPs quite a bit, and/or believe strongly in the role of the NDP in Parliament. Voting Liberal or Conservative, on the other hand, seems to be a coin toss far more often.

After the NDP, Liberal incumbents performed best, followed by the Conservatives. There isn't a huge amount of variation between the three parties, but enough that it equates to roughly three points for a Conservative incumbent and five points for an NDP incumbent. The one party that showed a great degree of difference from the others was the Bloc Québécois. Their incumbents tend to see their vote shift at almost the same rate as their party as a whole.

There was very little variation, on the other hand, when breaking down incumbents (in cases where the party loses support) by the number of times they have been elected in the past. The factor varies only from 1.06 for 4th-time incumbents to 1.10 for 3rd-time incumbents, with 1st-time, 2nd-time, and 5th+-time incumbents falling in between.

But when a party gains support in a region, there does seem to be a difference between sophomore MPs and more veteran MPs. Sophomore incumbents (i.e., incumbents who were elected for the first time in the previous election) did better than their party (when their party increased its support in the region) in 60% of cases. This makes sense - when an MP is first elected voters are less sure of them. When they stand for election for a second time, there is a greater chance that more voters will flock to them, even at a greater rate than voters switch over to that party in a given province or region.

However, veteran incumbents (i.e., MPs who have been elected two or more times in the past in their riding) tend to do worse than their party as a whole, when their party is gaining in a region: 58.6% of veteran incumbents did worse than their party.

The last aspect of incumbency I looked at was when there was no incumbent. Whether by retirement, death, or disgraceful resignation, I looked at what happened in a riding after an incumbent MP did not stand for re-election. The result was that only in 13.4% of cases did the incumbent party do better in the open seat than their party did in the region or province as a whole. In 86.6% of cases, the rookie candidate for the incumbent party did worse than the party did in the province or region as a whole, and there was no significant variation between whether a party increased or decreased its support in the region. And the drop in support was significant, enough to reduce a healthy 40% to a risky 35%.

Incumbency is the last major factor I wanted to look at before building my new projection model. The results of this analysis have been added to the model with success, bumping the accuracy rating up by 10%.

I've completed the projection model for British Columbia, taking into account a variety of different factors. With the next projection update, the new model will be used for British Columbia, and other provinces will be added as they are completed.

I tested the model for both the 2006 and 2008 elections, and it performed at a high degree of accuracy. I will present the results of the tests later this week or next.

I am currently working on expanding the new projection model into the other provinces. I'm working on Alberta at the moment, and hope to have it all up and running in the next few weeks. Every factor input into the model is verifiable, testable, and evidence-based. The tests of the model have, so far, proven its validity. When the test results are presented, I will provide detailed methodology as well. The new projection model will provide projections for all 308 ridings, taking into account their individual characteristics where possible.

I'll leave you with one little tidbit on what the projections for British Columbia are now showing: it's beneficial to the NDP. But the new model still shows that the Liberals are poised for gains in the province.

24 comments:

  1. so an MP's voter support in their second election will generally be the highest they can achieve, but at that point it tends to be relatively stable.

    makes sense.

    ReplyDelete
  2. Very interesting to see the difference of the incubency effect whether the party gains or losses support in a province. I admit I didn't think of that (I have an incubemcy effect of course, but it's the same for gain/loss). I'm interested to see what it will give.

    By the way, you can't really be harder than you're currently are on the NDP ;-) So I'm not surprised by this change.

    But I have one criticism: your model seems to take into account only the incubency effect and provincial changes. There isn't any within-province effect, right? By that, I mean, if the Liberals loses 10% in Ontario, you're assuming the same swing for all regions of Ontario, am I right? Therefore, what you attribute to the incubency effect could simply be a regional effect. For instance, when the Conservatives drop in Quebec, they drop less in the region of Quebec city. But is it because of an incubency effect or is it because of a regional (within province) effect? I would say both.

    10% gain in accuracy is great and very similar to the gain I was getting when I added the incubency effect.

    Bryan Breguet
    2closetocall.blogspot.com

    ReplyDelete
  3. Yes, you may be right that it is a combination of a regional effect with an incumbency effect. In the end, it should give the same results, as MPs generally tend to be elected in geographic bunches.

    And yes, I'm looking at province-wide trends. Unfortunately, only EKOS (and Leger for Quebec) breaks any of their numbers down by anything lower than province/region, so we don't have any other information to work from.

    ReplyDelete
  4. No we don't have information, but we can do the same exercise as you did for the incubency effect. We can find that when the Tories gain 2 points in Quebec, they actually increase much more in the region of Quebec city. This is what DemocraticSpace was the first to do and allowed him to make projections much more accurate than any uniform-swing model. I'm surprised to hear that you won't take that into account, especially for big provinces such as Quebec or Ontario.

    ReplyDelete
  5. The model is a proportional swing model, so yes, if the Conservatives gain two points in Quebec they will actually gain more support in regions in which they are stronger, such as Quebec City.

    ReplyDelete
  6. Ok but this is not what I meant. The swing would be proportionally the same. I used the term "uniform" to simplify, that's all. The fact is that your model doesn't allow any regional effects. So if the Conservatives drop 10% in Quebec, they will drop 10% in every region of Quebec (well, less the incubency and your other riding-level effects). But we all know this isn't the case in real life. The data clearly shows that the swing is not the same (uniform or proportional) within a province. But maybe you're right and your incubency effect will actually capture both the incubency and regional effects. Again, I was simply surprised.

    ReplyDelete
  7. Adding both an incumbency and regional effect would just be piling on. Where would the incumbency effect end and the regional effect begin?

    Perhaps there would be some merit in gauging a regional effect in ridings where the party does not hold the seat, but most "regions" in Canada have only a handful of seats. For Quebec City, if you're using a sample base of five or seven ridings, are you catching a regional effect or just a coincidence?

    You'll have more to chew on when I present the tests of my projection model.

    ReplyDelete
  8. Is it just me, or does the incumbent effect look to be more a matter of obvious calculations by political parties than an independent variable? It makes sense that incumbents would do best for parties with falling shares of the vote since the party will naturally worry more about keeping its existing seats under those circumstances, while incumbents for a party whose support is growing wouldn't be perceived as facing as much danger (thus allowing the party to focus more on flipping other seats in its favour and eliminating the perceived incumbent advantage).

    ReplyDelete
  9. That certainly could be the reason for this effect, but I think you are under-estimating the local role that is played in elections.

    ReplyDelete
  10. "I think you are under-estimating the local role that is played in elections."

    Money probably matters here.

    Incumbents can raise more money locally, where as challengers rely on the central party to fund their campaign.

    If things are going south than obviously challengers are ignored and incumbents still have their own $$$.

    ReplyDelete
  11. Really looking forward to seeing the new model!

    I'm somewhat surprised that you have to do it separately for each region (i.e. rolling it out for BC first, etc.). Hopefully you'll let us know when you lay it all out how much difference there is between them!

    ReplyDelete
  12. jbailin,

    I will, that is a good idea.

    It's being rolled out one province at a time simply because I'm impatient and it is a very long process. In addition to entering the data for each riding, I have to look at the history of each riding to add in any other factors. Plus I'm a little finicky with the graphics and how I am setting it all up, so it takes a little time.

    I'm taking great care to ensure there are no wonky results.

    As to why British Columbia was first, it is because I was testing it out in great detail. The tests are too big of a job to do for the whole country, so I just started with BC. I looked at the average margin of victory, whether I was correct on the seats, whether I was correct on the order of the parties, and what MOE I had for each individual party. That is a large test to do for the entire country.

    Once the other provinces are completed, I will test them for seat-call accuracy only, which will still take some time but is less onerous.

    ReplyDelete
  13. I read an interesting academic study a year or two ago which suggested that much of the small incumbency factor that exists in Canada is highly qualified.

    I'm going from memory here, but the suggestion was that the true incumbency advantage (i.e. influenced by the individual incumbent) that is there is mostly accounted for by:
    - the incumbent having name recognition where the challenger may not
    - the perception of an incumbency advantage means challenging parties have more difficulty recruiting high-calibre candidates

    The study examined the background of challengers and found that, to varying degrees, much of the sitting MP's incumbency advantage could be eliminated if the challenging candidate was (for examples):
    - a former MP
    - a former provincial MLA, MPP, etc.
    - a former mayor
    - a former city councillor

    ReplyDelete
  14. Strategic communications just (attempted) to poll me.

    Probably a BC provincial poll coming out soon ?

    I think the NDP uses their services.

    ReplyDelete
  15. "Where would the incumbency effect end and the regional effect begin"

    Well this is exactly why using actual statistical methods such as econometics helps. This allows me to estimate both effects. And yes most of my "regions" are composed of 10 ridings. But since we can use more than one election, that provides me with enough data points. On top of that, except in Alberta, you sually don't have one party winning the entire region. Thus we have the necessary source of variation.

    And by the way, I do explain much more of the swing when I add both (regional and incumbency). Have you heard of the R-squared coefficient? I'm asking cause you seem to know what statistical significance means. For Quebec and the Conservatives, if I use only the provincial swing as well as the incubency effect, I explain 58% of the riding-level swing. When I add all my other variables (regional effects, the level of support of the other parties, etc), I explain as much as 88%.

    What I find weird in your new model is that it almost feels like a step backward from DemocraticSpace. He was the first to see the importance of regional effects. So you are adding incumbency but dropping this big part. I find it really hard to justify. It doesn't mean your model won't provide a good approximation (hey, a simple linear uniform swing already provides a good one). When I decided to build a model, I started with what DS had accomplished and built on it.

    At the end, it's gonna be really interesting to compare our models because I trully believe that you are doing a much better job than me at finding some riding effects (such as having a cabinet minister, etc). So it'll really provide two points of view.

    Bryan Breguet
    2closetocall.blogspot.com

    ReplyDelete
  16. Oh and by the way, yes the incumbency and regional effects could be mixed together, but only for the party who actually won the ridings! What about the other 3 or 4 parties? For them, having regional effects is crucial. This is what allowed DS to predict the Tories gain in the region of Quebec city, before they even had MPs there.

    By the way, you don't need to enter the results for each riding. You can simply download this information from the website of Election Canada. They provide everything you need (and then you add who is the incumbent in excel).

    ReplyDelete
  17. @Henry: wow, a lot of work for Eric if he has to look at which candicate was former city councillor! lol But you might be onto something there.

    ReplyDelete
  18. While Greg Morrow has done terrific work at his site, I'm not convinced Democratic Space is the standard you think it is. ThreeHundredEight had a better projection than Democratic Space for the 2008 Quebec election, and my projection model would have out-performed Democratic Space in British Columbia (the only province tested so far) in the 2008 federal election, with the correct province-wide voting results input into it. I don't know how DS would have performed with an accurate vote projection, but there you have it.

    ReplyDelete
  19. Meanwhile Ekos latest poll on CBC says things have returned to normal and the CPC is back at 34% instead of the crazy numbers last week showed !!

    ReplyDelete
  20. 3% other and 12% greens in new EKOS poll ? 15% really ?

    Other + Green in '08 = 8%

    Of course EKOS fell back. There's no room for large CPC numbers when the Greens/Others are that high.

    Probably the MOE was at work two weeks ago.

    Wasn't that what Harper's team said would happen all along ?

    ReplyDelete
  21. Extremely, extremely unlikely (read: impossible) that all four major pollsters were showing high-end-of-MOE results two weeks ago.

    ReplyDelete
  22. Well I'll post soon strong evidence that proportional swing is just wrong and I really don't understand why people keep using it. I already spent countless e-mails with HKDP to tell them how crappy their model was lol (they hate me now, seriously and they even remove their old websites because of me - I was accusing them of lying on the number of correctly predicted ridings). Proportional swing just doesn't fit the data. It's even worse for small parties, such as we've seen with the ADQ in 2008 (where a proportional swing model was predicting a loss of Mario Dumont...)

    As far as 2008 in Quebec is concerned, you "beat" 2008 overall? Or riding-by-riding? Cause it's one thing to correctly predict the overall results cause you have a lot of mistakes canceling each other, but it's another thing to correctly predict the most ridings. And if you look at the 2008 federal election, the proportional swing did well overall, but had way more mistakes than my model, DS or a simple linear uniform swing model. Again, it just turned out that a lot of mistakes were canceling each other out. And for Quebec in 2008, since you were using a proportional swing model, it means you were greatly underestimating the ADQ.

    And by the way, with the correct percentages, my model was correct (overall, I didn't look at the riding level yet) except for one seat (and this was the semi-surprising win of QS. Here is the "proof":

    http://geloso-breguet.blogspot.com/2008/12/post-mortem-des-projections.html

    (btw, I was also projecting the same in my revised finale projections. But you missed them since I posted them very late the day before the election. In those, I was estimating the percentages with comitted voters only).

    I would like to know how many mistakes your model had in 2008 in Quebec, at the riding level. Then we could compare apples-to-apples

    As for BC and DS, as you said, it's unfair to compare projections with the actual votes to the one estimated. I wish would do post-mortem as you and I do (i.e: using the actual results of the election). Cause as we've seen with your projections way off in NB (or NS, not sure which one), having the wrong percentages will necessarily biase the projections.

    How many mistakes would your model have made in 2008 in BC by the way?

    ReplyDelete
  23. Bryan,

    Again, please wait until I report on the test results of my projection model. I'm not going to spoil the results now.

    Proportional swing, left alone, is not accurate. My test results will show that. But with the added factors they can be made to be accurate, precisely because the added factors are based on correcting the problems with proportional swing.

    In 2008, my overall call was more accurate than Democratic Space's.

    I didn't make full riding projections, but would have been at about the same level of riding accuracy as Democratic Space. I only "called" close races, not calling the more predictable races (he did the opposite). I didn't calculate my final rating, however, but it would have been around 90%. I was at 82% on the closest races (66 in all), whereas Democratic Space was 63% on the "too close to call" races (24 in all).

    ReplyDelete
  24. "Extremely, extremely unlikely (read: impossible) that all four major pollsters were showing high-end-of-MOE results two weeks ago."

    Nobody said anything about the other 3 pollsters.

    Everybody got it right two weeks ago.

    EKOS just got it right by accident.

    They're now returning to having things wrong, as per usual.

    ReplyDelete

COMMENT MODERATION POLICY - Please be respectful when commenting. If choosing to remain anonymous, please sign your comment with some sort of pseudonym to avoid confusion. Please do not use any derogatory terms for fellow commenters, parties, or politicians. Inflammatory and overly partisan comments will not be posted. PLEASE KEEP DISCUSSION ON TOPIC.