Forum tried to poll PC party members. What happened next was strange.

Polling for a party's leadership campaign is fraught with hazard. While it can reveal how the general population and party supporters feel about the leadership candidates - and that is useful information - they can't tell us anything specific about the actual outcome. Only a very small proportion of party supporters actually vote in a leadership race, and that proportion is even smaller when we are talking about the general population.

There are two ways to go about polling a party's membership. One way is to obtain a list of party members and call them, which has happened in the past. The less common route is to do a standard poll and ask people if they are party members.

That's what Forum did in its latest poll concerning the Ontario Progressive Conservative leadership race. Good for them for trying, but logistically it should be a nightmare. According to the PCs, they have over 80,000 members, representing 0.6% of Ontario's adult population. You'd have to do a poll of 50,000 people to get a decent sample size of 300 party members, if the poll is done properly. With the response rates that IVR gets, you'd have to call about 2.5 million households.

Instead, Forum polled 881 Ontarians and found 65 paid-up party members. This is revealing, for several reasons.

Those 65 paid-up party members represent 7.4% of the sample, quite a bit more than what it should have been if the poll was completely random. Forum found 12 times as many PC party members in its poll than it should have.

This raises a few issues. First is the question of whether we can believe these 65 respondents. If the poll was done correctly, some of them would have been lying or misunderstood the question.

Second is the question of accuracy. If Forum could find 12 times as many PC members as it should have, what does that say about its sampling methods?

The third question is, perhaps, the most interesting one. What if active political partisans are just so much more likely to participate in a poll than the rest of the population, that being over-represented by a factor of 12 is actually not a sign of a problem with the poll, but a problem with the people who take part in a poll in the first place?

Response rates are tiny these days, and particularly with IVR polling. Nova Scotia's electoral laws require pollsters to report their response rates, and when Forum did in 2013 it showed that rate to be about 2% to 3%. That is a large proportion of potential voters not bothering to pick up the phone or complete the survey.

If that 2% to 3% is largely made up of people who are active partisans, does that skew the poll? Other polling has shown that poll-takers are more likely to be politically active and do volunteer work. The important question, then, is whether these people are politically different than the rest of the population. If they are, than the poll will be led astray. If they aren't, than the poll will be effective.

Putting aside these existential questions, the poll itself had a counter-intuitive result. PC party members broadly lined-up with the views of the general population and of PC supporters.

Of those 65 people who claimed to be party members, 51% said they preferred Christine Elliott. Another 13% said they preferred Monte McNaughton, while 10% opted for Patrick Brown. In the chart above, each box represents one of those 65 respondents.

Another 14% said none of these three contestants were their preference, while 11% were undecided.

Among PC supporters, a larger group, Elliott had 35% to 15% for McNaughton and 11% for Brown. Among all Ontarians, Elliott had 24% to 7% for McNaughton and 6% for Brown.

There are many caveats with these sorts of numbers, some of which were spelled out above. But the most important is, of course, sample size because the margin of error of a sample of 65 people is +/- 12.2%.

When we look at the poll with these margins of error, we get a much less clear picture.

(This is also a good example of how the margin of error applies differently depending on each candidate's support. Brown, at 10%, cannot be as low as -2%.)

Now we see Elliott at between 39% and 63%, McNaughton between 5% and 21%, and Brown between 3% and 17%.

If we look at just the decided PC party members, the margin of error increases to +/- 14.1%, and Elliott's support ranges between 55% and 83%, compared to 7% to 29% for McNaughton and 4% to 24% for Brown.

By these numbers, we would consider Elliott a lock to win on the first ballot. McNaughton and Brown are fighting for second place. However, we also have a pretty good idea that this is not the actual contours of the race.

We won't know the final tally of each candidate's membership drive until the end of April, when the PCs release the numbers. But Brown has claimed about 40,000 members, with Elliott claiming 34,000 and McNaughton 20,000. One oft-quoted source puts McNaughton closer to 6,000, which is a more intuitive result. We just don't know for sure, though any misleading candidate will be embarrassed once the numbers are announced.

So the numbers are probably not too far off, particularly if McNaughton is at 6,000 instead of 20,000 (the three claims add up to 94,000, but 80,000 if the source is correct).

At those rates, Brown should be somewhere between 43% and 50% of newly signed-up members, compared to 36% to 43% for Elliott and 8% to 21% for McNaughton.

This ignores the voters that were already with the party when the race started (some 10,000) and those signed up by Vic Fedeli and Lisa MacLeod (both of whom endorsed Elliott).

So now we get to the question of accuracy. Considering the limitations of Forum's poll, their Elliott numbers are, perhaps, plausible. Those undecideds could be from the ranks of the original party members, who should represent some 13% of the total. But to have Brown and McNaughton within the same error bands with at most half of Elliott's support, and McNaughton narrowly favoured among the two, does not quite align with what we know about the race.

This really puts the poll in question, and highlights the limitations of this attempt at getting at the voting intentions of party members. Does Elliott really have the kind of advantage that Forum gave her? It doesn't seem so, but we won't know for sure until the votes are counted in May.

A tale of two leadership races

The two largest opposition parties in Canada's two largest provinces will be choosing new leaders in May. The front runner in the campaign to replace Pauline Marois as leader of the Parti Québécois is clear, and all metrics agree on who it is. But the race to replace Tim Hudak as leader of the Ontario Progressive Conservatives should be a much easier call than what it is turning out to be.

I wrote about some other leaders today in my column for the CBC, breaking down the approval ratings of Stephen Harper, Thomas Mulcair, and Justin Trudeau. Check it out here.

There are three candidates in the PC leadership contest. Christine Elliot is an MPP in the GTA and a former leadership contestant, the best known of the three in part because she is the widow to the late Jim Flaherty. Patrick Brown is an MP from Barrie, and before this contest very few people had ever heard of him. The same goes for Monte McNaughton, an MPP from southwestern Ontario.

Since the beginning of the race, I have been tracking endorsements. It is a way to determine who is the favourite of the party establishment, and more often than not it tracks the outcome relatively closely. By this measure, Christine Elliott is the consensus choice.

She has garnered 81% of the endorsement points my system awards, giving her an insurmountable lead. Her list of endorsers includes former premier Bill Davis, two leadership candidates who dropped out of the race (Lisa MacLeod and Vic Fedeli), and long-time MPPs like Ted Arnott, Ernie Hardeman, Norm Miller, Julia Munro, and Laurie Scott. She also has some of the higher profile Ontario MPs on her side, including Chris Alexander, John Baird, Peter Kent, Kellie Leitch, Rob Nicholson, and Lisa Raitt.

Running second in the endorsement rankings is Brown, with 14% of available points. He has only a few caucus endorsements, the longest serving being Toby Barrett. He has some Ontario MPs on board, including Mark Adler, Paul Calandra, Rick Dykstra, and Bal Gosal. Jean Charest, former Quebec Premier and, more importantly in terms of the endorsement system, former leader of the federal Progressive Conservatives, has also endorsed him. Brown boasts the endorsements of some famous hockey players, including Wayne Gretzky, but the endorsement system does not take those into account, as they have no role within the party.

In third is Monte McNaughton, with 5% of the endorsement points. His list of endorsements is quite short, the only other caucus member being Bob Bailey. McNaughton also has the endorsement of former Toronto mayor Rob Ford, though again that is not one considered by the system.

It would seem to be a very cut-and-dry race, then. Elliott is the candidate the party wants to win, and normally we would expect the membership to follow suit. This was the case when Justin Trudeau won the federal leadership of the Liberals in 2013.

But this marks out Elliott as the establishment candidate, and Brown is explicitly running an anti-establishment campaign.

And it appears to be a very successful one. The period for signing up new members just recently came to a close, and the Brown campaign claims to have signed up 40,000 new members. That is a huge number, considering that at the start of the campaign the PCs had just 10,000 paid-up members.

Reports of the number of members signed up by Elliott and McNaughton differ. One source told the media that Elliott signed up just 13,000 new members, though the Elliott campaign denies this and claims that number represents less than half of the total.

(Just speculating, but 13 and 30 sound a lot alike. Perhaps the source merely misheard?)

McNaughton claimed some 20,000 new members, though that same source put the number instead at around 6,000.

What does this mean for the leadership? Quite a bit. Let's assume that Brown has signed up 40,000 members, Elliott has signed up 30,000, and McNaughton about 6,000. Add to that the 7,000 signed up by Fedeli and MacLeod, as I saw reported in one column, and the 10,000 original members. That puts us at about 93,000. Let's round that up to 100,000 for good measure, assuming that another 7,000 signed up on their own.

If all of those members vote (or at least all vote in the same proportions), and if the 17,000 original and non-aligned members vote according to the breakdown of the endorsement rankings, we would end up with Elliot at 51% on the first ballot (assuming Fedeli's and MacLeod's members go with her), Brown at 42%, and McNaughton with 7%.

While that would point to a win by Elliott, there are a lot of assumptions being made to give her that razor-thin victory. If she falls short of the 50% mark, she and Brown would be in a near tie. It follows that McNaughton's supporters would go to Brown, who appears to be campaign to the right of Elliott (and McNaughton to the right of both).

There's the problem for Elliott. In a three-way race, she might have difficulty winning if she doesn't get to 50% on the first ballot, or at least very close. A lot depends on how effective Brown's campaign really is.

But numbers are not the only factor in this race. Members need to be distributed efficiently across the province, since each riding is awarded points according to how many members are in it. So, it all depends on how those members are spread out. It makes it a very difficult race to call.

That leaves us with the inevitable conclusion that the endorsement rankings system will not be a good reflection of the eventual outcome. All it can do, at this point, is give an indication of who the party establishment wants to win. If Brown pulls it off, he will have done so against the wishes of the party's elite.

A mercifully easier campaign to call in Quebec

The race in Quebec is much more straightforward.

There are five candidates in the PQ's contest. Topping the list is Pierre-Karl Péladeau, newly elected MNA in 2014 and a media mogul. Alexandre Cloutier, Bernard Drainville, and Martine Ouellet are all MNAs as well, and sat in Marois's short-lived cabinet. Pierre Céré is in the bottom tier. He ran for the PQ in the last election and was defeated.

In this race, the endorsement rankings follow nicely with the general perception of the campaign.

Péladeau leads with 56% of endorsement points. His list of endorsers includes caucus members like Claude Cousineau, Nicole Léger, Nicolas Marceau, and Pascal Bérubé. He has former premier Bernard Landry, as well as former MNAs like Daniel Turp, Serge Cardin, Diane De Courcy, and Scott McKay.

Second in the ranking is Cloutier with 22% of the endorsement points. His most important caucus support is François Gendron, who was first elected in 1976. Véronique Hivon, who was considered a potentially strong leadership candidate before the race started, is also with him. Well-known former MNAs like Louise Beaudoin, Léo Bureau-Blouin, and Louise Harel have endorsed Cloutier.

Not far behind with 17% is Drainville, who has a number of caucus members including Guy Leclair, Carole Poirier, and Mathieu Traversy.

Ouellet has 5% of the endorsement points, all coming from former MNAs. Céré has 1%, and all of his endorsers come from the Bloc Québécois.

But whereas the PC race's endorsements do not seem to reflect the reality on the ground, the rankings for the PQ contest do seem to be closer to the mark. Take, for example, fundraising.

Péladeau has so far raised about $114,000 (consider that fundraising rules in Quebec are far more restrictive than in Ontario, where Elliott apparently has raised over half-a-million dollars), roughly half of all of the money raised by the contestants still in the race.

Drainville and Cloutier form the second tier in fundraising with about $40,000 apiece, followed by Ouellet at $29,000 and Céré at $12,000.

If those totals represented votes, Péladeau would take about 49%, followed by Drainville and Cloutier at 17% each, Ouellet at 12%, and Céré at 5%. Those numbers are not too different from the current endorsement rankings, though it would suggest Ouellet is somewhat under-estimated.

There are still some two months left to go, but it would seem Péladeau will win. If he doesn't take more than 50% on the first ballot, he is likely to come close enough to get over the mark by the third ballot (Céré, dropping off first, would likely not have enough votes to get him over the mark on the second). That the party has not settled on the consensus alternative helps his case, but it could be a closer result than one would expect. Ouellet and Céré, campaigning on the left, are unlikely to see a lot of their supporters going to Péladeau, boosting either Drainville or (more likely) Cloutier. If Péladeau ends up with less than 45% on the first ballot, I could see it getting to a head-to-head finish.

In the end, the details of the outcome looks more like trivia. The real contest to watch will be in Ontario.

Elliott, Drainville lead first PC and PQ endorsement rankings

With the two largest opposition parties in Canada's two largest provinces looking for new leaders, it is time to launch ThreeHundredEight.com's leadership endorsement rankings. As the contests just get going, Christine Elliott and Bernard Drainville start as the leaders in the Ontario PC and Parti Québécois leadership races.

Long-time readers may remember the leadership endorsement system, which was employed with some success to gauge the 2011 Bloc Québécois, 2012 Liberal, and 2013 NDP leadership races.

The system uses endorsements to try to estimate the support each candidate has within the 'establishment' of the party they are trying to lead. Each endorsement is given a value based on the endorser's importance within the party. Though it is not necessarily a real gauge of support in a leadership race, past contests have shown that 'establishment' support is a decent proxy for membership support. A full explanation of how the system works can be found here, where the tables and charts for the PC and PQ leadership races will be maintained.

How has the system worked in the past? It was developed by trying to reverse-forecast a series of races, with points being awarded in a way that would get a close estimate to the final result. When it was first deployed in the 2011 BQ race, the system correctly pegged Daniel Paillé to be the eventual winner. In the 2012 NDP race, it accurately put Thomas Mulcair and Brian Topp as the two front-runners, giving Mulcair 29% of the endorsement points (he got 30% support on the first ballot).

The system's best performance was in the 2013 Liberal leadership race. Justin Trudeau was awarded 82% of the provincially-weighted endorsement points (to match the system being used to elect the party's new leader), and in the end received 80% of the points. Joyce Murray was given 8% of the endorsement points, and took 10% of the vote. Martha Hall Findlay was given 10% of the points and took 6% of the vote. 

In provincial races, when the system was applied, it accurately picked Dwight Ball to beat Paul Antle in the Newfoundland and Labrador Liberal race. Philippe Couillard was correctly identified as the eventual winner of the Quebec Liberal race, and the system also put Raymond Bachand and Pierre Moreau in a near-tie, as actually happened. And if I had applied the system to the recent Alberta NDP race, it would have gotten Rachel Notley's share of the vote almost exactly right.

So, let's get to the endorsement rankings, starting with the Ontario Progressive Conservatives.

Elliott starts as the favourite to win within the PC establishment, with 47.3% of the currently available endorsement points. She has, by far, the largest number of endorsers and has wide support within caucus (in addition to herself, seven caucus members are backing her). She also has the support of a few Conservative MPs and former MPPs.

The only other candidate with a good number of endorsements is Lisa MacLeod, who comes second with 32.5% of the endorsement points. She has four caucus members supporting her, including long-time MPPs Garfield Dunlop and Julia Munro.

After these two, there is a clear second tier. Monte McNaughton is narrowly in third in the endorsement rankings with 8.6%, thanks to the support of caucus member Bob Bailey, as well as a former MPP.

Patrick Brown, with 8.2% of the points, gets most of his support from Ottawa. He has the support of four fellow MPs, more than Elliott's three. He has one caucus member, Rick Nicholls, in his camp.

Rounding out the list is Vic Fedeli, who currently has no endorsements but has 3.4% of the points due to the points he himself brings to the table.

This seems intuitive. Elliott is widely seen as the frontrunner, and a recent Forum Research poll suggested a plurality of Ontarians think she'll win. The wildcard in this race will be Doug Ford, if he chooses to enter. I imagine he could get a lot of support from the membership without acquiring many endorsements.

Now to the leadership race to take over the Parti Québécois. Here again, a political heavyweight has yet to throw his hat in the ring.

But of those who are in the race, Drainville is well ahead. He has 56.2% of the currently available endorsement points. He is the only candidate currently to boast any endorsements from within caucus. He has three, as well as the support of a former MNA.

After him are Alexandre Cloutier, Martine Ouellet, and Jean-François Lisée. All three of them only have as many points as their own 'value' gives them. It will be interesting to see which of the three can land some endorsements in the early stages of the race.

The last is Pierre Céré, who is not a sitting PQ MNA (nor has he ever been one). He does have the support of a former BQ MP, Yves Lessard, so he is on the board.

The big question is, of course, Pierre-Karl Péladeau. A bid for the leadership seems inevitable, and he already has three MNAs who would support him if he runs. That gives him a total of 30 points, meaning he would be narrowly in front of Drainville with 36.4% to 35.7%. It seems only a matter of time before Péladeau tops the list.

Both of these races will come to a close in May, so there is still plenty of road ahead of us. I will be updating the rankings every Wednesday, though I may not always post about them on the front page. Follow this section of the site for all the latest numbers.