TL;DR: You do not make decisions with certainty. You make them with incomplete information about a future that has not happened yet, which means every meaningful choice is a bet. The people who decide best in this condition do two things ordinary decision-makers do not: they judge a decision by its quality at the moment it was made, not by how it happened to turn out, and they translate their vague sense of “probably” into actual numbers they can be held to. This piece gives you the reframe, a six-dimension scorecard for grading a decision before you know the result, and a calibration habit borrowed from the best forecasters on record. In a tournament run by the University of Pennsylvania, trained probabilistic thinkers beat professional intelligence analysts with access to classified information by about 30 percent. Decide like a CEO placing capital at risk. Update like a student who assumes the last result taught them something.
Here is the trap almost everyone lives in. A friend drives home after a few drinks and arrives safely, so they conclude it was fine. Another buys a lottery ticket, wins, and calls it a smart move. A founder ships an untested product on a hunch, it succeeds, and the hunch becomes “vision.” In each case the person judged the decision by the outcome. And in each case the judgment is wrong, because a good outcome can follow a terrible decision and a bad outcome can follow an excellent one. Poker players and professional forecasters have a name for this error, and unlearning it is the first move toward deciding well.
This is a companion to our Personal Decision Stack, which lays out the full system for choosing under abundance. Here we go deep on one layer of that stack: how to think probabilistically when certainty is off the table.
Every decision is a bet
The reframe is simple and it changes everything downstream. A bet is a decision about an uncertain future in which you commit something you value to an outcome you cannot control. By that definition, choosing a job, a city, a business model, or how to spend a Saturday are all bets. You are wagering time, money, attention, or reputation on a future state of the world, and you are doing it with a probability of being right that is almost never 100 percent.
Annie Duke, the former professional poker player who popularized this framing in her 2018 book Thinking in Bets, points out what this reframe forces you to admit: “I’m not sure” is not a weakness to hide but the accurate description of every forecast you will ever make. Once you accept that you are always betting, three questions replace the false comfort of certainty. What am I actually wagering? What are the odds I am right? And what does the payoff look like across the range of things that could happen?
The CEO lens sharpens this. No competent operator commits capital by asking only “will this work?” They ask what the expected value is: the size of the prize multiplied by the probability of getting it, weighed against the size and probability of the loss. You can run your own decisions the same way, and you should, because the alternative is what most people do, which is to decide on gut feeling and then let the result decide whether the gut was smart.
Resulting: the error that keeps you stuck
Duke’s term for judging a decision by its outcome is resulting, and it is the single most expensive habit in everyday reasoning. Resulting corrupts learning in both directions. When a bad process produces a good result, resulting rewards the bad process and you repeat it. When a good process produces a bad result, resulting punishes the good process and you abandon it. Either way you are training on noise.
The reason resulting is so sticky is that outcomes are loud and processes are quiet. The result is a fact you can see; the quality of the decision is a judgment you have to reconstruct. So the mind takes the shortcut. This is close cousin to hindsight bias, the well-documented tendency to see an outcome as more predictable after the fact than it ever was before it. The antidote is not to ignore outcomes. It is to grade the decision separately from the result, on the information you actually had at the time.
The discipline sounds like this: after any decision that mattered, ask “was that a good bet?” before you ask “did it work out?” A good bet that loses is still a good bet. A bad bet that wins is still a bad bet, and if you cannot say that out loud about your own wins, resulting still owns you.
The evidence that probabilistic thinkers decide better
This is not a philosophy. It is a measured effect. Between 2011 and 2015, the U.S. intelligence community funded a forecasting tournament through IARPA, its research arm, to find out whether anyone could actually predict geopolitical events better than chance. The Good Judgment Project, led by psychologists Philip Tetlock and Barbara Mellers at the University of Pennsylvania, entered and won so decisively that the other university teams were dropped.
The findings below are the reason to take probabilistic thinking seriously rather than treat it as a poker-table party trick.
Table 1 — What the forecasting research actually found (verified public data)
| Finding | Source | What it means for you |
|---|---|---|
| Trained “superforecasters” beat professional intelligence analysts with classified access by roughly 30 percent on accuracy | Good Judgment Project, Tetlock & Mellers, University of Pennsylvania (IARPA tournament, 2011-2015) | Method beat privileged information. How you think can outweigh what you know. |
| The tournament spanned about 500 questions and over one million individual forecasts across four years | Good Judgment Project reporting | The effect is robust, not a lucky streak on a handful of calls. |
| Accuracy was scored with the Brier score, where lower is better and every forecast is stated as a numeric probability | Brier (1950), as applied by the Good Judgment Project | Vague predictions cannot be scored or improved. Numbers can. |
| The best forecasters updated their estimates frequently in small increments as new information arrived | Tetlock & Gardner, Superforecasting (2015) | Deciding well is a loop, not a one-time verdict. |
The headline is worth sitting with. In a domain as hard as forecasting world events, disciplined amateurs beat credentialed experts who had secrets the amateurs did not. The edge did not come from access. It came from a way of thinking: state probabilities in numbers, keep score, and update.
The Bet Framing Method: a scorecard for grading a decision before you know the result
Knowing you should separate decision quality from outcome quality is not the same as being able to do it. You need a way to grade a decision at the moment you make it, using only what you know then. The scorecard below is our attempt to make that concrete. Score each dimension from 0 to 2 and total it out of 12; anything under 8 is a signal to slow down before you commit.
Table 2 — The Decision Quality Scorecard (CEOtudent editorial framework)
| Dimension | 0 points | 1 point | 2 points | Why it matters |
|---|---|---|---|---|
| Stake clarity | I have not named what I am risking | I know roughly what is at risk | I can state the exact time, money, or reputation on the line | You cannot size a bet you have not measured |
| Probability estimate | “It’ll probably be fine” | A rough range (say 50-70 percent) | A single number I would defend | Numbers can be scored and corrected; feelings cannot |
| Base rate check | I ignored how often this works for others | I glanced at the typical outcome | I anchored my estimate on the real base rate first | Most bad forecasts ignore how the same bet usually ends |
| Downside survivability | I have not asked what happens if I lose | I could absorb the loss with strain | A total loss is recoverable and non-catastrophic | Never take a bet that can end the game, however good the odds |
| Disconfirming evidence | I only looked for reasons it works | I noted one objection | I actively sought the strongest case against | Seeking only supporting evidence is how confident people stay wrong |
| Reversibility | Irreversible and I treated it lightly | Costly to undo | Cheap to reverse, or I built in an exit | Reversible bets deserve speed; irreversible ones deserve the scorecard |
The scorecard does two useful things at once. It slows down the irreversible, expensive, low-survivability bets that deserve friction, and it gives fast permission to the small reversible ones that do not. It also creates a written record of your reasoning that you can revisit later and grade honestly, which is the only way to tell a good bet that lost from a bad bet you got away with.
Calibration: turning “probably” into a number
The habit that separates the Good Judgment Project’s best forecasters from everyone else is calibration: the match between your stated confidence and your real-world hit rate. You are well calibrated when the things you call 70 percent likely happen about 70 percent of the time. Most people are badly calibrated because they never attach numbers, so they never find out.
The fix is a practice, not a talent. Start stating probabilities out loud on ordinary predictions: the meeting will run long, the launch will slip, this hire will still be here in a year. Write the number down. Then, crucially, keep a decision journal and go back to check. Over a few dozen scored predictions you will discover your personal bias: most people are systematically overconfident, calling things 90 percent that land 60 percent of the time. Once you see the gap, you can correct it, and calibration research consistently shows it improves with feedback and practice. This is the student half of the CEO-and-student pairing made literal: you are running experiments on your own judgment and updating on the results.
Two guardrails keep calibration honest. First, always check the base rate before you check your gut, because the outside view (how often this kind of bet works for people in general) is a better starting anchor than your inside story about why you are different. Base-rate neglect, first documented by Daniel Kahneman and Amos Tversky, is the reliable engine of overconfidence. Second, decide your exit before you enter. A pre-committed “I will stop if X” protects a good decision from becoming a bad one when a losing outcome tempts you to chase.
How to build the habit
You do not adopt probabilistic thinking by agreeing with it. You adopt it by installing three small routines and letting them compound.
Run a weekly bet review. Pick the two or three decisions that mattered most this week and grade them on the scorecard, out loud or in writing, before you let yourself think about how they turned out. This is where you catch resulting in the act.
Keep a one-line prediction log. For any forecast you care about, write the claim, your probability, and the date you will know. Reviewing it monthly is the single fastest way to calibrate, and it pairs naturally with our guide on how to develop good judgment, which covers where reliable intuition comes from in the first place.
Reframe your losses on purpose. When a decision goes badly, force the question “was the bet still correct given what I knew?” Sometimes the answer is no and you learn something. Often the answer is yes, the variance simply landed against you, and the correct move is to make the same bet again. Knowing the difference is the entire skill, and it connects directly to the wider toolkit in our index of mental models that actually matter.
FAQ
Is thinking in bets just gambling with fancier words?
The opposite. Gambling usually means taking bad bets for excitement. Thinking in bets means refusing bets whose downside can ruin you and sizing the rest to the real odds. It is the discipline casinos rely on, not the impulse gamblers act on.
How do I estimate a probability when I have almost no data?
Start with the base rate for the category, then adjust for what makes your case genuinely different, not just what makes it feel special. Even a rough number you will commit to beats a confident “it’ll work,” because a number can be checked and improved and a feeling cannot.
Does this slow every decision down?
No, and that is the point of the reversibility test. Cheap, reversible choices should be made fast; the scorecard is for the expensive, hard-to-undo bets where a few minutes of friction saves you from a large loss.
What is the single highest-leverage habit to start with?
The prediction log. Writing down “I think this is 70 percent likely, I will know by Friday” and later checking it is the fastest known route to better calibration, and calibration is what makes every future bet sharper.
Sources
- Annie Duke, Thinking in Bets: Making Smarter Decisions When You Don’t Have All the Facts (2018)
- Philip Tetlock and Dan Gardner, Superforecasting: The Art and Science of Prediction (2015)
- Good Judgment Project, led by Philip Tetlock and Barbara Mellers, University of Pennsylvania, IARPA forecasting tournament (2011-2015)
- Glenn W. Brier, “Verification of Forecasts Expressed in Terms of Probability,” Monthly Weather Review (1950)
- Daniel Kahneman and Amos Tversky, foundational work on base-rate neglect and judgment under uncertainty
- Daniel Kahneman, Thinking, Fast and Slow (2011)
- Baruch Fischhoff, foundational research on hindsight bias
This content was compiled with the support of AI following in-depth research, then written and prepared for publication by the CEOtudent editorial team.
This post is also available in:
