The Calibrated Mind: Master Note

I. Operating Principles

1. You run on testimony, not verification. You have never audited the earnings report, the drug trial, or the age of the Earth. Your accuracy therefore depends on judging institutions, incentives, and processes, a learnable skill, more than on raw analysis.

2. Human error is systematic, and that is good news. Random error washes out; systematic error compounds, but a predictable failure is a correctable one. The entire toolkit below exists because the failure modes repeat.

3. Intelligence is an amplifier, not a defence. Myside Bias shows almost no correlation with intelligence (Stanovich, West & Toplak, 2013), and a larger bias blind spot associates with higher cognitive ability (West, Meserve & Stanovich, 2012). More horsepower means better rationalizations. (Contested at the edges: a 2025 twin study challenges whether rationality is separable from g at all.)

4. The master bug is asymmetric scrutiny. Congenial claims face "can I believe this?", one supporting reason passes. Threatening claims face "must I believe this?", one flaw dismisses. Both feel like honest inquiry; you never experience yourself choosing to be unfair.

5. Published evidence is a filtered sample. The largest replication audit to date (SCORE, Nature 2026) found 55.1% of social-science claims replicate, with effect sizes shrinking ~80%. Standard journals publish positive results 96% of the time; Registered Reports, accepted before results exist, only 44%. The gap is Publication Bias, not discovery.

6. Fix the process, not the person. Individual debiasing training yields g ≈ 0.26 on a weak literature, with poor real-world transfer. What works is structural: change the representation of the problem, add adversarial review, and keep score.

II. The Decision Protocol

graph TD
    A["Claim or decision arrives"] --> B{"Base rate known?"}
    B -->|"No"| B1["Find the reference class<br/>before touching the narrative"]
    B1 --> C
    B -->|"Yes"| C{"Severe test?<br/>Could this evidence have<br/>come out against the claim?"}
    C -->|"No"| C1["Treat as ~no evidence.<br/>Check selection effects<br/>and incentives instead"]
    C1 --> D
    C -->|"Yes"| D{"Do I want this<br/>to be true?"}
    D -->|"Yes"| D1["Raise the bar:<br/>write the strongest<br/>opposing case first"]
    D1 --> E
    D -->|"No"| E{"Is any branch ruinous?"}
    E -->|"Yes"| E1["Ergodicity rule:<br/>cap the downside first.<br/>Expected value is the wrong tool"]
    E1 --> F
    E -->|"No"| F["Commit: explicit probability<br/>+ resolution date"]
    F --> G["Log it → score it → review in bands.<br/>Judge the process, not the outcome"]

III. The Model Toolkit

Seeing the data

Testing the claim

Sizing the bet

Reading the humans

IV. The Evidence Ledger

What the replication era did to the famous findings. Cite from the left column freely, the middle column with caveats, the right column never.

What forecasting research actually supports: experts beat chance only at short horizons (near-chance 3–5 years out); foxes beat hedgehogs; simple extrapolation beat both. The strongest robust predictor of accuracy is update style, frequent, small revisions (superforecasters: 5.1 forecasts per question, mean update 0.11, vs 2.0 and 0.20). Reference Class Forecasting was the highest-value tagged technique: Brier 0.17 vs 0.26.

V. Actionable Leverage

Finance & risk

• Base rate before thesis. Open every pitch with the outcome distribution for comparable firms; the vivid story about this one is a sample of one.
• Severity-test every backtest. A strategy tuned on the data that evaluates it would look good regardless: BENT, no information.
• Size for the time average. Any position with a live path to ruin is mispriced by EV logic. Survive first, optimise second.
• Decompose the thesis. List the 4–5 legs that must all hold and multiply rough probabilities. The product is always lower than the gut number, because conjunctions are less probable than they feel.
• Model second-order exposures, don't footnote them. FX translation on cross-listed holdings reprices continuously, not just at conversion; withholding tax drags every dividend. The tidy map omits these; the territory doesn't.
• Budget for invisible wins. Prevented losses never show up as outcomes, so risk management always looks like waste after it works. Say this out loud before anyone cuts the hedge.

Academics

• Test by production, not recognition. Fluency from reading feels like mastery and predicts almost nothing. Close the book and rebuild the derivation.
• Process-grade your own answers. A right answer from wrong reasoning is a Gettier pass, it will fail you on the exam variant. Log why you were right, not just that you were.
• Translate every stats problem into counts out of 1,000 until it is automatic.

Career & life

• Discount survivor advice. Every "how I got the offer" story comes from a sample with the rejections removed.
• Build intuition only where it can form. Expert intuition requires regular cues plus fast, unambiguous feedback (Kahneman & Klein, 2009). Choose co-op work with tight feedback loops; distrust confident gut-feel, including yours, everywhere else.
• Read the strongest opposing case before committing publicly. After public commitment, the identity machinery turns reading into ammunition-gathering.
• Audit institutions with one question: what does admitting an error cost here? Where the answer is "your reputation," errors are hidden until they are catastrophic.

The practice system

Weekly drills, pick two: a Fermi estimate checked against reality · one conditional probability converted to natural frequencies · one held belief traced to its original source (expect a press release) · a Pre-Mortem on any commitment over your threshold · a Steelmanning pass an opponent would endorse as accurate.

VI. Vault Map

Foundations: Severe Testing · Falsifiability · Bayesian Updating · Base Rates · Likelihood Ratio · Replication Crisis

Risk: Ergodicity · Expected Value · Antifragility · Opportunity Cost · Goodhart's Law

Self: Motivated Reasoning · Myside Bias · Scout Mindset · Calibration · Prediction Log

Next actions: (1) create Prediction Log and enter five dated predictions today; (2) stub Base Rates, Selection Effects, and Ergodicity, the three highest-traffic nodes for a finance vault; (3) run one source-archaeology drill this week.