Our chances run low: calibration on 378 calls
Our called sides won 218 of 378 settled calls when our own chances expected 188, and the prices 207. What that gap says about our model, and its limits.
On this page
Our published chances have been too low on the sides we call. Across the 378 calls settled on our record, with games from 4 to 31 October, our chances for the called side added up to about 188 expected wins. The calls won 218. A forecaster that is well calibrated should land near its own expected total; we finished 30 wins above it. This is a weakness in the model, and we would rather say so than have a reader find it.
The check, band by band
We grouped every settled call by the chance we published for the called side, then counted how often those sides won.
| Our chance | Calls | Won | We expected | Win rate | Our average chance |
|---|---|---|---|---|---|
| Under 40% | 59 | 25 | 22.4 | 42.4% | 38.0% |
| 40% to 50% | 149 | 83 | 66.5 | 55.7% | 44.6% |
| 50% to 60% | 118 | 71 | 64.2 | 60.2% | 54.4% |
| 60% to 70% | 39 | 30 | 25.0 | 76.9% | 64.0% |
| 70% to 80% | 11 | 7 | 8.0 | 63.6% | 73.1% |
| 80% and over | 2 | 2 | 1.7 | 100% | 86.3% |
| All | 378 | 218 | 187.8 | 57.7% | 49.7% |
The 40% to 50% band carries most of the gap. Those 149 sides won 55.7% of the time against an average published chance of 44.6%. This is the plain version of a reliability diagram, described in what a calibration curve shows.
Is it luck
If every call were independent and our chances were right, the total would usually land within about 9 or 10 wins of 188. A 30-win gap is about three times that. Calls are not fully independent, though: many share a weekend, a league and the same kind of game, so the real spread is wider than that sum suggests. It is still hard to read the gap as noise alone.
The market did better
On the 374 calls with a price, the best prices we saw implied about 207 wins. The calls won 215. The prices were nearer the truth than our chances were, which is the opposite of what a model with an edge should show. The prices also fell short of the real total, by 8 wins, but by far less than our 30.
What it means for the edges on our cards
An edge is our chance minus the price's implied chance. If our chances on called sides run low, the edges we print are, if anything, understated on average, and many cards marked negative may not be. But the bands are uneven: the 70% to 80% band won less than we said, on only 11 calls. We are not going to adjust published numbers by hand to fix this. Cards stay as published and are settled that way, as explained in how a BAI8 call gets settled on the record.
What this does not tell you
This checks only the side we called, not every outcome on every card, and the side we call is not chosen at random. It covers four weeks, mostly football. A model can be well calibrated and still not beat the market, and the reverse. For how much a record of this size can prove, read how many bets before a record means anything.
18+. Bet only what you can afford to lose.
Get our call on the next match
Thirteen passes of research before the start: who to back, the price worth taking and why. Run it in the app on any match.
Free plan: 5 assessments a month. No card.
Common questions
Are BAI8's probabilities well calibrated?
Not yet, on the side we call. Across 378 settled calls from 4 to 31 October, our chances added up to about 188 expected wins and the calls won 218. We have been too cautious about our own picks.
Is that gap just luck?
Probably not all of it. The gap is about 30 wins, and chance alone would usually move the total by about 9 or 10 either way. But the calls are not fully independent, so treat that as a rough guide.
Did the bookmakers do better?
Yes. On the 374 calls with a price, the prices' implied chances expected about 207 wins against 215 actual. They were closer than we were.
Does BAI8 change a card's chance after publishing?
No. A published chance stays as it is and is settled as published. That is what makes this check possible.