Site Administrator Of:

Supporter Of:


Another Test For Forum Research & Its Polling Methods (in byelections anyhow)

For a minute, let’s forget what partisan side of the fence we’re all on, and turn into poll analysts. Forum Research is the only pollster doing any poll tracking of the four byelections, which are to commence tomorrow. Forum’s probably last poll of the cycle before Monday lists the following findings:

Bourassa: Liberals 43, NDP 31, BQ 15 (Margin of Error for poll: +/- 6%)
Toronto-Centre: Liberals 48, NDP 35, CPC 13 (MOE: +/- 4%)
Brandon-Souris: Liberal 50%, CPC 36%, NDP 8% (MOE: +/- 5%)
Provencher: CPC 48%, LPC 37%, Green 8%, NDP 6% (MOE: +/- 7%)

Forum has oft been criticized for recent polls in other byelections/elections where their final #’s seemed way off from what actually happened in the final result. They use Interactive Voice Response (IVR) for their polling, which is basically an automated call asking folks to press buttons to indicate who they support. This is our latest instalment of checking up on how this particular method works in byelections.

Keep an eye on election night when all the votes are counted and whether Forum comes anywhere within their Margin Of Error for the different political party vote. As I understand it (though if I’m wrong, I’ll happily be corrected) a Margin of Error indicates the % either higher or lower that a particular political party’s percentage could be; ie. in Brandon Souris, a MOE of +/- 5% I believe means that the LPC candidate of Rolf Dinsdale at 50% states his vote % could be as high as 55% or as low as 45%, and so on for the other political parties in the riding. Apply other MOE +/- %’s to the other ridings, and add or subtract against the final vote of each political party’s result accordingly, and let’s see how they do. (Again, if I am misreading what the MOE actually means, please feel free to let me know, and I’ll happily update the post).

It’s unfortunate other pollsters chose not to participate in this for a comparison of accuracy as well as methodology, but apparently, if I read some of the folks who poll on Twitter accurately, they’re afraid of high costs to run polls here, as well as concerns for a good enough sample rate, but it is what it is.

I’ve been a skeptic of Forum polling (though as a Liberal supporter, I’ll be very happy if they’re right in the 3 ridings we lead), but I’ll be just as hard on them as anyone if their results are inaccurate on this, whether they’re completely wrong, or even if a Liberal candidate wins by more or less then the MOE % that was listed by them.

UPDATE: I should also mention Forum gives the usual pollster stat that poll accuracy with this is considered accurate 19 times out of 20.

UPDATE 2: Frank Graves of Ekos, who I correspond with on Twitter, sends the following (starting here, if you wish to follow the Twitter tweet convo thread): “Your post is fine, but a few points: The MOE refers to the known population of all eligible voters. Only 30% or so will vote…so estimates for that sub-population only guessed at until after vote…estimate can be within MOE for the pop’n + be off the vote..whether by live int(erviewer) or robot, one should have call backs to reflect those at home less and a sample of cell phone and land lines…cell only very difficult to obtain at riding level because no geocode for those phones.. tend to be younger less affluent…overall methodology more important than mode of contact, + you need a good likely voter model… hard to construct at riding level…all of which is why we (Ekos) steer clear of riding level contests for the most part..”

Update 3 @ 1:59 pm: I’ve read on FB that one of the folks I know who lives in that riding got a call from Forum today; so they may be trying to get in 1 more poll before Monday. If they publish it, I’ll post the updated #’s here for the most accurate comparison we can get from results vs their polling.


2 comments to Another Test For Forum Research & Its Polling Methods (in byelections anyhow)

  • Skeena Sage Williamson

    Hi Scott,

    Thanks so much for this blog post — I’ve been looking for information on these Forum polls.

    You might want to take a look at the “By-Election Barometer” on . Éric Grenier seems to have a pretty good record when it comes to forecasting by-election results. He takes Forum riding-specific polls into account, but other factors (including trends from regional polls) are fed into his projection model too.


  • David Anderson

    I wish polls could be made illegal. They are almost always perverse in that they could (and are often intended to) influence voters to support the “winner”, a thoroughly stupid and undemocratic idea. In the process they reduce the likelihood of thoughtful debate amongst citizens and the candidates themselves.
    Because I never know the parameters of the pollster who call me, I always refuse to participate. They could be easily part of a larger manipulation. No thanks.

unique visitors since the change to this site domain on Nov 12, 2008.