Prediction is Difficult, Especially About the Future

With apologies to Niels Bohr, I just wanted to take a moment and point out that the art or science of predicting Canadian election outcomes is actually far less advanced and accurate than American readers might be used to.  I have written of my fear that the NDP boom could lead to the obscene outcome of a Conservative majority on a scant 37-38% of the national popular vote, but I could easily be quite wrong, and I really hope I am.

Partly that is because the situation is still developing and if the NDP "orange crush" continues to grow until election day itself, then it will quite likely go past the second tipping point I discussed, where the NDP actually wins significant numbers of currently Liberal, BQ and (most importantly) Conservative seats where they up until now are not believed to lead, rather than just splitting the non-conservative vote and letting the Conservatives win a number of normally Liberal or BQ ridings without increasing their actual support.

The problem for the NDP is that they started so far behind the Liberals and Conservatives that even a historic 5-10% increase in national support doesn't necessarily make them competitive in nearly enough ridings to guarantee a Tory minority.  I don't rule out the possibility of a super-duper history changing increase in support of 15% or more that actually puts the NDP into real competition with the Conservatives to form the government.  After all, this actually happened in Ontario in 1990, which shocked the nation (and the NDP) by delivering an NDP majority government to Canada's most populous province.

But putting aside the dynamic nature of the situation very much still in motion, just in general Canadian elections are harder to predict so you should discount significantly anyone's predictions (including mine):

• 308 elections versus 51.  First up, predicting the outcome of a US presidential election is easier because there are a lot less constituent parts that decide the outcome. 
• Less polling.  There's plenty of national and regional polls, but these are really of rather limited value in predicting the individual riding outcomes which actually determine the real outcome.   We're a smaller country, a smaller media market, there's just a lot less overall money flowing in the system to pay for the level of polling we see in the US, and polling a state is just much easier for pollsters than any 1 riding (hard to find 1000 adults willing to pick up the phone living within a much smaller geography).
• More competitive races.  The US Presidential is 51 seperate elections, but really only a relative handful of these are truly competitive.  I haven't crunched the numbers in any meaningful way, but Canada's riding boundaries are not really gerrymandered.  The term doesn't even get used up here.  There are certainly safe seats for each party, but compared to the US congress, a far higher proportion of MPs in Parliament actually have to compete for re-election.  Aside from the lack of gerrymandering, the stronger party system here means that things like the incumbentcy factor and name recognition matter much less.  If your party is out of favour, you're in trouble.  In 1993, the ruling Progressive Conservatives lost all but 2 seats.  
• More competitive parties.   Even at their peak in 2000 under Nader, the US Greens were not even a threat to win a state or take a seat in Congress.  Sure there was Perot but in general, it's almost always a 2 candidate race.  Canada has five parties that could win seats in Parliament.  Plus a couple independents.  There are a significant number of competitive 3 way races for ridings, particularly in BC and Quebec, and even a four-way race or two.   It's just damn difficult to predict the winner in these ridings and even if you are lucky enough to get a riding poll, it's hard to predict how events will affect the support as the poll ages.  
• First past the post.  Americans have the same system, but it's worth calling this out to avoid confusion say with proportional representation based European multi-party systems, where a 5% increase in support for some smaller party means they get 5% more seats. 

As an empirical sign of all this, one of the election prognostication sites, electionprediction.org still has 60 seats listed as "too close."  Their track record in the past 3 Federal elections (2008, 2006, 2004) has been a respectable near-90% right call average, and less than a week from this one they don't want to call 20% of the seats.  Harper only needs to gain 12 for a majority, and they have 5 times that many as toss-ups.  So yeah, I'm worried.

Still, I can't fault the NDP die hards for their enthusiasm.  I know they've been hoping for (and, ahem, repeatedly predicting) a break through like this for a very long time.   My negativity is just my read on the likely outcome, not some cynical attempt to deflate them.  I was against Ignatieff becoming Liberal leader and now he might well preside over the death of the Liberal party as the governing alternative or maybe even as a party at all.  Given the choice between another Harper term and some kind of Layton/Ignatieff coalition I will of course hope that Layton can complete this budding miracle.   Eric at 308 is skeptical that the NDP can go from fourth to government and offers some solid reasoning why, and I find that more intellectually persuasive than anything I am hearing from the hopeful Orange crowd.  But there is a very real chance we're just off the map as far as empirical based models go, and into a realm where polling and past experience offer very little predictive value.

That said, I'm very glad for our non-partisan and rock solid voting system.  Because the realm of possible outcomes is quite wide so it's nice to know in advance I can have strong confidence that whatever result we get is real.  If it is Jack, I will eat my orange crow with some gusto.