# Federico Carrone: Complete Content site: https://federicocarrone.com updated: 2026-07-05 scope: full text export for articles, series episodes, talks, about, keywords, and curated recommendations > Full text export for articles, series episodes, talks, about, keywords, and curated recommendations. See llms.txt for navigation. ## Articles ### Solving Futbol Like Chess *Published: 2026-07-05* > European futbol optimized what is measurable and converged on a monoculture. On variance, the duel, deep blocks, and the danger of optimizing proxies. URL: https://federicocarrone.com/articles/solving-futbol-like-chess/ European futbol has spent the past fifteen years solving the game like chess. A generation of coaches optimized for pass completion, pressing triggers, territorial control, rest defense, and positional occupation. The problem is that they optimized for what is measurable. Attacking space early, attempting the difficult pass, dribbling past a defender, deliberately creating chaos: these are high-variance plays. They fail more often than they succeed. If you evaluate players by completion rate, ball retention, or positional discipline, those actions look like mistakes. So they get coached out. Eventually, everyone converges toward the same local optimum. The analysts eventually noticed, and modern models now reward the risky forward actions the old metrics punished. But the correction arrived too far up the pipeline. Academies still select for legibility: the kid who holds his position and completes his passes gets promoted, the one who loses the ball five times trying the pass that breaks the match open gets corrected. By the time a player is old enough to be measured properly, the variance is gone. The game becomes increasingly legible. Every team occupies similar spaces, presses in similar ways, builds from the back with similar patterns, and minimizes the same risks. Systems become better at defeating other systems, but worse at dealing with players who refuse to behave like systems. The convergence is also enforced by moral language. Possession is virtuous, attacking is brave, defending is a confession of inferiority: they parked the bus, they played anti-futbol, they didn't deserve to win. This is nonsense. Every tactic is a rational answer to a problem, not a moral position. The problem is your players, the opponent, the score, the stakes. Playing open against a superior team means competing on exactly the terms where you are weakest. That is stupid. The intelligent answer is to compress the space, defend as a unit, and threaten in transition. And it is hard: keeping eleven players organized for ninety-plus minutes, shifting as a block hundreds of times, staying concentrated when one lapse ends the match. Most teams can't do it. The defensive masterclass looks passive precisely because its excellence is invisible. The dribbler and the deep block look like opposites, but they are the same bet. A low block compresses the match into a handful of chances, and fewer chances means more variance in the result. The favorite wants many events so that quality wins. The underdog wants few so that luck matters. The duel is the same wager inside a single play: fail five times for the one action nobody can plan against. Europe's best attackers right now are duel players: Yamal, Vinicius, Doku, Musiala. The system isolates them high and wide against a fullback because it cannot generate what they do, only the conditions for it. Positional futbol industrialized the deployment of the 1v1; it never learned to produce it. South America still does, not because its coaches are romantics but because formation there runs through street futbol, futsal, and late systematization, where the only way to survive is to beat the man in front of you. Europe buys back the variance its academies coached out. Argentina in 2022 is the proof. Scaloni's team was structurally modern, and it kept everything the dogma discourages: the tactical foul, the provocation, the willingness to let a match turn ugly when ugliness favored them, and a player whose entire game is the refusal to behave like a system. They won the World Cup. Italy may have been the first futbol culture to lose its identity this way, which is ironic because the systematization began there: Sacchi's Milan is a direct ancestor of modern pressing. But Italy's advantage was never positional perfection. It was asymmetry, unpredictability, an instinct for making matches uncomfortable. Even catenaccio was less a style than a weapon. Converging on a model partly descended from its own export, Italy surrendered what made it different, and missed two World Cups. Since Guardiola, elite futbol has one template: possession as a first principle, patient build-up, endless circulation until a gap appears. In the right hands it is magnificent and it wins everything. But it has hardened into dogma. Futbol is not won by completing the most passes. It is won by scoring more goals than the other team. Sixty-eight percent possession, six hundred passes, sideways and backwards and sideways again, and nothing at the end. Possession without penetration is decoration. The team recycling the ball for its own sake is avoiding risk, prioritizing not losing over winning, and calling it a philosophy. Sterile possession is risk aversion in a nicer suit. Meanwhile the low block that beats it every season gets called cowardice. Futbol's greatest eras were defined by variety, not conformity: catenaccio, total futbol, direct play, gegenpressing, the deep block, each an answer to a real problem. This is the danger of optimizing proxies. When a measure becomes a target, it stops being a good measure. When everyone optimizes the same measurements, they stop optimizing for victory and start optimizing for looking efficient. Metrics become targets. Proxies replace objectives. Variance is mistaken for error. The outliers capable of breaking the system disappear because the system itself learns to eliminate them. --- ### A Proof Is Only as Good as Its Spec *Published: 2026-06-06* > Formal verification doesn't eliminate risk. It relocates it into the spec, the model, and the trusted base. Five runnable Lean 4 proofs that compile cleanly and still sit on real bugs. URL: https://federicocarrone.com/articles/formal-verification-moves-trust/ I want Ethereum to have more formal verification, not less. That's why I'm writing this. There's a reliable way to discredit the whole formal-methods effort, and it's to oversell it. Let "formally verified" start to mean "safe," and the first time something wearing that label fails in production, everyone walks away with the wrong lesson at the worst possible moment. And the label is going on far more than smart contracts now: ZK proving circuits, consensus clients, the virtual machine itself. I'd like to head that off, because the tool is genuinely one of the best we have. It just deserves a sharper claim than the one it usually travels under. The people building Lean models of the protocol are doing some of the most valuable work in the ecosystem, and I want to strengthen the claim their work supports, not chip at it. So take it as given throughout that the team is competent and the methods are mature. The gaps I'm about to point at aren't rookie mistakes. They're what's left after a good team has done everything right. Here's the claim I'll defend: > Formal verification shrinks one surface, the gap between an implementation and its specification, almost to zero. What it doesn't do is remove risk. It moves risk elsewhere: into the specification, the model, and the trusted base. The failure mode is to mistake that move for an elimination. A machine-checked theorem doesn't say "the code is correct." Written out in full it says something more guarded: > the implementation satisfies the specification, inside a model, modulo a trusted base, for the properties someone thought to state. The first clause is the part verification actually delivers, and it delivers it completely. Tests and fuzzers sample the space of behaviors; a proof covers all of it. Everything after that first clause is human judgment, and the checkmark doesn't touch any of it. The examples below are each a complete, `sorry`-free Lean 4 proof. (The trusted-base one carries a `sorry` on purpose, which I'll flag when we get there.) Each shows the shape of a real bug living in one of those later clauses. The code compiles, and you can run it yourself. One objection deserves to be raised straight away, because it runs under the whole piece: *"Every one of these is just the spec or the model or the axioms being wrong, and getting those right is the entire point of the verification program. So you're arguing for the program, not against it."* That's true, and it isn't a rebuttal. Getting the spec, model, and trusted base right is the work. My point is that the work isn't obviously easier than writing correct code in the first place, and that some of it can't be audited from inside the proof at all. The danger is cultural. A team that reads the checkmark as having *finished* that work, rather than relocated it, gets bitten by a bug that technically was never there. I'm not claiming any of this is new. Each of the four gaps is well understood in the formal-methods literature, and a specialist will recognize all of them on sight. What I want to add is emphasis, plus three concrete things: a small runnable demonstration of each gap, a map onto where this ecosystem actually lost the most money, and the discipline that follows. If there's a novel claim, it's only this: don't let a green checkmark quietly stand in for that work. {% toolkit(label="Key takeaways") %} - Formal verification shrinks one surface, the gap between an implementation and its specification, almost to zero. It does not remove risk; it moves risk into the spec, the model, and the trusted base. - Written out in full, a machine-checked theorem says: the implementation satisfies the specification, inside a model, modulo a trusted base, for the properties someone thought to state. Only the first clause is what verification delivers. - Each gap is shown with a complete, runnable Lean 4 proof that compiles cleanly and still sits on a real bug: a "sorting" function that returns the empty list, a transfer that quietly pays the deployer, an overflow that a wrong model cannot see. - The danger is cultural. A team that reads the checkmark as having finished the work, rather than relocated it, gets bitten by a bug that technically was never there, and the failures map onto where this ecosystem actually lost money. - This is an argument for more verification stated accurately, not against it. {% end %} ## First, what verification genuinely closes The gaps only mean something measured against the power, so start with the power. This is the thing a proof does that no test suite can. Model the machine word, state the property you care about, and the kind of overflow that wrecks a naïve model (the one in "a proof is about a model," below) stops being a risk and becomes impossible: ```lean4 def checkedAdd (a b : UInt8) : Option UInt8 := if a.toNat + b.toNat < 256 then some (a + b) else none -- whenever checkedAdd succeeds, the result is the true sum, no silent wrap, ever theorem checkedAdd_never_wraps (a b r : UInt8) (h : checkedAdd a b = some r) : r.toNat = a.toNat + b.toNat := by unfold checkedAdd at h split at h · rename_i hlt; simp only [Option.some.injEq] at h; subst h; rw [UInt8.toNat_add]; omega · contradiction ``` The quantifier is doing all the work. That `∀ a b` covers all 256 × 256 inputs here, and all 2²⁵⁶ × 2²⁵⁶ at full width, and the proof settles every one of them at once. A fuzzer can only sample that space. The exact bug we'll hit below, a balance dropping when you deposit into it, is now unreachable, and provably so: ```lean4 theorem checkedAdd_never_loses (a b r : UInt8) (h : checkedAdd a b = some r) : a ≤ r := by have := checkedAdd_never_wraps a b r h rw [UInt8.le_iff_toNat_le]; omega ``` That's not a small thing. A whole class of bug, gone across every possible input, with a certainty testing can't reach. It's why formal verification is worth the trouble, and why it's worth describing accurately. Keep it in mind; the rest of this is about where its reach stops. ## A proof only constrains what you thought to say Start with the simplest version. We say what it means to sort, "the output is sorted," and prove an implementation meets it: ```lean4 inductive Sorted : List Nat → Prop | nil : Sorted [] | one : ∀ a, Sorted [a] | cons : ∀ a b l, a ≤ b → Sorted (b :: l) → Sorted (a :: b :: l) def IsSortingSpec (f : List Nat → List Nat) : Prop := ∀ l, Sorted (f l) def sortBad (_ : List Nat) : List Nat := [] theorem sortBad_correct : IsSortingSpec sortBad := by intro _; exact Sorted.nil ``` `sortBad` discards your input and returns the empty list. The empty list is sorted, so the proof goes through. What's missing is the requirement that the output be a permutation of the input, and since nobody wrote that down, nothing rules out the cheat. At this size the gap is obvious. The trouble is that the same kind of gap doesn't get any more visible as the system grows. Here it is in a transfer: ```lean4 def TransferSpec (f : Nat → State → State) : Prop := ∀ amt s, amt ≤ s.alice → (f amt s).bob = s.bob + amt ∧ (f amt s).alice = s.alice - amt def transferEvil (amt : Nat) (s : State) : State := { alice := s.alice - amt, bob := s.bob + amt, deployer := s.deployer + amt } theorem transferEvil_correct : TransferSpec transferEvil := by intro amt s _; exact ⟨rfl, rfl⟩ ``` `transferEvil` debits Alice and credits Bob exactly as the spec demands. It also quietly credits the deployer on every call. The spec pinned down what happens to Alice and Bob and said nothing about anyone else, so the theft is invisible to it. To be fair, the stronger property catches the cheat at once: ```lean4 def CompleteTransferSpec (f : Nat → State → State) : Prop := ∀ amt s, amt ≤ s.alice → (f amt s).bob = s.bob + amt ∧ (f amt s).alice = s.alice - amt ∧ (f amt s).deployer = s.deployer -- the clause that was missing theorem transferEvil_not_complete : ¬ CompleteTransferSpec transferEvil := by intro h have hd := (h 1 { alice := 1, bob := 0, deployer := 0 } (by decide)).2.2 exact Nat.succ_ne_zero 0 hd ``` "So write the stronger spec." Right, and good methodology pushes you to. Frame conditions turn "what doesn't change" into an explicit obligation; a full functional-correctness spec tries to pin behavior down completely. That's the correct discipline, and serious teams follow it. But look at what it asks for, and what it can't give back. A spec is often far smaller and clearer than the code it governs, and that's much of the reason to write one. Its completeness is a different question, and that's the one you can't discharge. Listing every property an adversary might reach for, every account that has to stay fixed, every invariant that has to survive every interleaving, is open-ended work. No proof hands you that list, and there's no theorem that says "you've now named everything that matters." Verification stays silent about every property you didn't write down, and that's almost exactly where security bugs live. ## A proof is about a model, and you don't ship the model This is the sharpest of the four, because a better spec won't fix it. The property is stated, the proof is real, and it's still a true theorem about the wrong universe. Take an invariant nobody would argue with: depositing into your account never lowers your balance. Over the natural numbers it's a theorem: ```lean4 def depositℕ (balance amount : Nat) : Nat := balance + amount theorem deposit_never_loses_funds_ℕ (balance amount : Nat) : balance ≤ depositℕ balance amount := Nat.le_add_right balance amount ``` The machine you deploy to doesn't have natural numbers. It has fixed-width words, and they wrap around. The same statement, word for word, is now false, and you can prove it false: ```lean4 def depositWord (balance amount : UInt8) : UInt8 := balance + amount theorem deposit_CAN_lose_funds_word : ¬ (∀ balance amount : UInt8, balance ≤ depositWord balance amount) := by intro h exact absurd (h 255 1) (by decide) -- 255 + 1 wraps to 0 example : depositWord 255 1 = 0 := by decide ``` A maxed-out account that receives one more unit ends up holding nothing. The proof over ℕ "ruled that out," in a universe where it couldn't happen in the first place. This isn't academic. In 2018 the `batchOverflow` bug (CVE-2018-10299) drained the BeautyChain ERC-20 token through exactly this arithmetic: two transfers of 2²⁵⁵ added up to 2²⁵⁶ and wrapped a 256-bit balance back to zero. A conservation proof over ℕ would have signed off on the vulnerable contract. Now the obvious objection, and it's fair: no competent team models token arithmetic over ℕ. They use bitvector reasoning that captures the machine word exactly, and this particular bug doesn't survive it. Agreed. But the lesson isn't "pick a better integer type." It's that a proof's reach stops at the edge of its model, and you can't see that edge from inside the proof. `UInt8` here is just a stand-in. Swap in a flawless 256-bit word model and you've only moved the edge somewhere else. The model still leaves something out: the gas schedule, the lowering from source to bytecode, the scheduler, the hardware, the bytes that actually get deployed. A loop you proved terminates can still run out of gas and revert, because cost was never in the model. Two clients that each provably refine the same abstract spec can still split the chain, if the spec left the byte encoding open and they filled it in differently. There's another gap that no integer type touches, and it's concrete rather than philosophical. A proof about a specification is not a proof about the implementation that runs it. You can verify a protocol in Lean and still have said nothing about the client codebases that execute it, because they aren't extracted from the proof. Good teams know this and work at it, either by verifying clients directly or by running the formal spec as a differential-testing oracle against them. Nobody's unaware of the gap. The point is that closing it is a second effort about as large as the first, and the proof about the spec doesn't do it for you. The bit-precise model closes the arithmetic gap and leaves this one wide open. This isn't a story about careless people. It's the shape of the field's biggest successes. seL4 and CompCert are verified down to assumptions they state openly, about the compiler, the hardware model, and what's simply out of scope, and the risk that remains sits at those boundaries rather than in the verified core. seL4's documentation is refreshingly direct about it: the non-leakage result holds only for the information channels the hardware model represents, so timing side channels outside that model are out of scope, full stop. CompCert is the encouraging version of the same story. Years of fuzzing turned up no bugs in its verified optimizer, only in the unverified code around it. In both cases the proof did its job, and the boundary was where the attention was owed. The step from "the model I proved things about" to "the system that actually runs" is itself an assumption. You can make it smaller and write it down explicitly, and good practice does, but you can't turn it into a theorem from inside the proof, because the real machine isn't a mathematical object the proof can range over. ## "The theorem checks" is not "the system is verified" A proof is only as sound as the kernel, the axioms in scope, and any shortcuts you took. Here's the fact that carries this section: the kernel checks your proof, but it has no way to check whether your axioms are true. That part is left to people. And a necessary axiom and a ruinous one look identical, same keyword, same green checkmark. You can't verify anything that uses cryptography without assuming properties you can't prove. That's normal and correct: ```lean4 axiom Hash : Nat → Nat axiom hash_collision_resistant : ∀ a b, Hash a = Hash b → a = b theorem ids_are_unique (a b : Nat) (h : Hash a = Hash b) : a = b := hash_collision_resistant a b h ``` Now a false axiom, except it doesn't look false. Over the integers or the reals, `a * b / b = a` is just true, and anyone who learned algebra there will nod it through. Over a fixed-width word it's false, because the multiplication overflows: ```lean4 axiom mul_div_cancel : ∀ (a b : UInt8), b ≠ 0 → a * b / b = a theorem fee_recoverable (price qty : UInt8) (h : qty ≠ 0) : price * qty / qty = price := mul_div_cancel price qty h ``` Lean accepts the axiom, and the plausible "the fee is always recoverable" theorem sitting on top of it. It will also prove the axiom false, using nothing but its own standard logic, with no help from the bogus assumption: ```lean4 theorem mul_div_cancel_is_false : ¬ ∀ (a b : UInt8), b ≠ 0 → a * b / b = a := by intro h exact absurd (h 200 2 (by decide)) (by decide) -- 200*2 = 144 (mod 256); 144/2 = 72 ≠ 200 ``` So the kernel accepted the axiom and a refutation of that same axiom, side by side, without a word of complaint. It checked the proofs. It never formed an opinion about whether the axiom was true. And that's the realistic danger, not some flagrant `0 = 7` that review would catch on the first pass. It's an axiom that quietly imports intuition from the wrong number system, or describes the environment, a memory model, a cost or timing assumption, and gets it *almost* right. From inside the proof it's indistinguishable from one that's exactly right, and it gets easier to miss as the spec grows. A third hole is quieter still: a proof left unfinished and shipped green. ```lean4 theorem solvency_preserved (assets liabilities : Nat) (h : liabilities ≤ assets) : liabilities ≤ assets + 1 := by sorry ``` Lean doesn't lie about that one either. It prints a warning and records `sorryAx`. Whether the warning fails your build is a CI policy decision, not a fact about the proof. "The pipeline is green" can quietly mean "the proof checks, and CI was configured not to reject the thing that would have caught this." The only check that tells the three apart is to ask what each theorem actually rests on: ```lean4 #print axioms ids_are_unique -- [Hash, hash_collision_resistant] ← necessary; must be reviewed by humans #print axioms fee_recoverable -- [propext, mul_div_cancel] ← rests on the false axiom; the kernel raised no objection #print axioms mul_div_cancel_is_false -- [propext] ← refuted using only Lean's standard logic; the axiom was just wrong #print axioms solvency_preserved -- [sorryAx] ← unproven; a warning CI may have ignored ``` "That's malpractice, not a limitation of FV." For the false axiom, sure. But the necessary axiom isn't malpractice. It's unavoidable, and it's every bit as load-bearing and every bit as unprovable. Every verified system rests on a trusted base: the kernel, the elaborator, any use of `native_decide` (which swaps the kernel out for the compiler), and a set of honest assumptions about cryptography and hardware that might be subtly wrong. You don't get to remove the trusted base. You only get to audit it. So the honest statement is an operational one. A verified project is exactly as strong as its axiom policy, its CI policy, and its dependency audit, and the checkmark vouches for none of those. ## Verification can't save you from a wrong requirement, only state it precisely This is the deepest case, and the one that should worry the program most, because the proof is real and the spec looks perfectly reasonable. Take a withdrawal, broken into its two real effects: paying the user, which in a live system is the external call that hands control to a possibly hostile caller, and updating the internal record. ```lean4 def pay (amt : Nat) (w : World) : World := { w with pocketed := w.pocketed + amt } def debit (amt : Nat) (w : World) : World := { w with recorded := w.recorded - amt } def WithdrawSpec (before after : World) : Prop := after.pocketed = before.pocketed + before.recorded ∧ after.recorded = 0 ``` The spec reads: after withdrawing the full balance, the user has pocketed that balance and is owed nothing. Reasonable enough. Now two implementations, one that pays before it debits and one that debits before it pays: ```lean4 def withdrawUnsafe (w : World) : World := let amt := w.recorded; debit amt (pay amt w) def withdrawSafe (w : World) : World := let amt := w.recorded; pay amt (debit amt w) theorem unsafe_meets_spec (w : World) : WithdrawSpec w (withdrawUnsafe w) := by refine ⟨?_, ?_⟩ <;> simp [withdrawUnsafe, pay, debit] theorem safe_meets_spec (w : World) : WithdrawSpec w (withdrawSafe w) := by refine ⟨?_, ?_⟩ <;> simp [withdrawSafe, pay, debit] ``` Both satisfy the spec. The verifier is equally happy with either, and "verified" tells you nothing about which one you'd rather deploy. Run them under reentrancy, though, where the attacker re-enters during `pay`, before the record is updated, and they come apart. The property that separates them is the one nobody wrote down: ```lean4 -- The attacker re-enters during `pay`, before the record is updated. def unsafeUnderReentrancy (w : World) : World := -- trace: pay; pay; debit; debit let amt := w.recorded -- both calls see the same balance debit amt (debit amt (pay amt (pay amt w))) def safeUnderReentrancy (w : World) : World := -- debit first ⇒ re-entry sees 0 let amt := w.recorded let w1 := debit amt w pay amt (pay w1.recorded (debit w1.recorded w1)) def NoOverWithdrawal (run : World → World) : Prop := ∀ w, (run w).pocketed ≤ w.pocketed + w.recorded theorem safe_reentrancy_no_overwithdrawal : NoOverWithdrawal safeUnderReentrancy := by intro w; simp [safeUnderReentrancy, pay, debit] theorem unsafe_reentrancy_overwithdraws : ¬ NoOverWithdrawal unsafeUnderReentrancy := by intro h exact absurd (h { recorded := 100, pocketed := 0 }) (by decide) -- pockets 200, not 100 ``` The safe order provably never overpays. The unsafe one provably does: start it with a recorded balance of 100 and it pays out 200. Both carry a clean proof of the same spec. The whole distance between "fine" and "drained" sat in one unstated assumption, that the external call can't re-enter before the state update, which everyone held in their heads and nobody put in the spec. This is The DAO (June 2016) in miniature, where a reentrant call pulled out roughly 3.6 million ETH by re-entering during the payout, before the balance was written down. The proof wasn't fake. The requirement was incomplete, and verification reproduced it faithfully, hole and all. "A capable team wouldn't stop at a final-state relation. A trace property, an effect-typed spec, or an explicit adversary allowed to re-enter would all catch this." Granted, and that's the right instinct; the tools exist. But every one of them needs you to decide, up front, to model the external call as a re-entry point an attacker controls. That decision is exactly the knowledge the whole ecosystem was missing in 2016. The property had to be known before it could be written, and the richer the formalism, the more of these choices it asks you to get right. You can't prove your property set is complete. You find out it wasn't, usually after the exploit. Verification turns "is this code correct?" into "have we stated every property that matters?", and the second question doesn't come with a checkmark. ## Where this leaves us I'm not arguing against formal verification. I'm arguing against one way of describing what it buys you. It does something real and rare. It eliminates the bug class "the code doesn't do what the spec says," completely, across every input the model allows. That class is large and dangerous, and clearing it is worth a great deal of effort. seL4 and CompCert are landmarks precisely because they cleared it for real systems. What it does is clear that one surface and concentrate the rest of the risk into three places the checkmark doesn't reach: - the specification, which can be incomplete or faithfully wrong; - the model, where you can have a true theorem about the wrong universe, with a refinement gap to the real machine you can't see from inside the proof; - the trusted base, the axioms the kernel never judges and the CI policy it never sets. The losses this ecosystem actually suffered line up with those three, not with the implementation-versus-spec gap a proof closes. The DAO was a requirement nobody had finished. The overflow drains would have slipped past a proof done over the wrong arithmetic model, and would have been caught by one done over the right model, which is the whole point about model choice. Consensus splits live on the boundary between a spec and the independent clients that implement it. A green checkmark feels like it has answered all of these. It hasn't, and in the overflow case it only answers if you happened to pick the model that lets it. So the slogan shouldn't be "formally verified, therefore safe." Closer to the truth: > Verification turns an open-ended search for bugs into a precise, bounded list of things you still have to get right: the spec, the model, the trusted base. That list is where the real work now lives, and it deserves more scrutiny once the checkmark is green, not less. In practice that means treating the gaps as part of the verification work, not as footnotes to it: - Pair every functional spec with its completeness properties: conservation, "nothing else changes," totality, permutation. Keep asking what the spec leaves unconstrained. - Verify over the type you actually deploy, and put the costs in the model. Failing that, write the refinement to the real machine down as an explicit assumption, and treat that assumption as attack surface. - Don't let a proof about the spec stand in for a guarantee about the implementation. Verify the client too, or differential-test it against the executable spec. The distance between "the protocol is correct" and "this node is correct" is real work. - Run `#print axioms` in CI over every theorem you ship, and make the axiom set and the `sorry` policy things a human signs off on. - Write the unstated assumptions down (atomicity, non-reentrancy, ordering) and prove the properties that depend on them, instead of stopping at the final-state spec. Do that, and verification delivers what it promises. Treat the checkmark as the finish line and you've traded a real audit for the feeling of one. The tool is excellent. The finish line just sits further out than the checkmark suggests, and saying so plainly is, I think, the best way to help an effort that deserves to work. --- *All five examples, the positive demonstration above plus the four gaps, are complete, compiling Lean 4 source at [github.com/unbalancedparentheses/verified-still-broken](https://github.com/unbalancedparentheses/verified-still-broken), and check under `nix run`. The only warning is the intentional `sorry` in the trusted-base example, whose presence is the point.* ## References - **The DAO (June 2016), reentrancy, ~3.6M ETH.** Gemini Cryptopedia, [*The DAO Hack*](https://www.gemini.com/cryptopedia/the-dao-hack-makerdao); Chainlink, [*Reentrancy Attacks and The DAO Hack*](https://blog.chain.link/reentrancy-attacks-and-the-dao-hack/). - **`batchOverflow` / BeautyChain (BEC), April 2018.** NVD, [CVE-2018-10299](https://nvd.nist.gov/vuln/detail/CVE-2018-10299). - **seL4, what the proofs assume (incl. side channels out of model scope).** [*What the Proofs Assume*](https://sel4.systems/Verification/assumptions.html). - **CompCert, the trusted computing base.** Monniaux & Boulmé, [*The Trusted Computing Base of the CompCert Verified Compiler*](https://arxiv.org/pdf/2201.10280); fuzzing result: Yang et al., *Finding and Understanding Bugs in C Compilers* (PLDI 2011). --- ### Discipline Without Love Optimizes for the Wrong Variable *Published: 2026-05-21* > On pain, building, capitalism, discipline, and realizing I had treated love as something that came after the work. URL: https://federicocarrone.com/articles/discipline-without-love/ The last year and a half, but particularly the last six months, were incredible but also very very tough. I went through difficult personal problems and had to expand my tolerance for pain to extremes I didn't know existed. You don't fight pain by toughening up. You fight it by controlling what you let near you. I left alcohol, lowered my consumption of caffeine to bare minimums, deleted my instagram and removed most of the people I didn't care about from my life and I doubled down on living with friends family and partners I love. My father told me multiple times that I have lived multiple lives in one. It's very likely that he has some responsibility for this because I became obsessed with adventures thanks to Jules Verne when I was a kid. I wanted to explore. I did trips to places and met people that you see only in nightmares. After getting burnt and almost dying multiple times I created [LambdaClass](https://x.com/class_lambda) and now [Ergodic Group](https://ergodicgroup.com). Those near deaths weren't only pain. Each one was teaching me the same thing: turn this into something that outlasts you, or it was just damage. I'm trying to do the same right now with what happened to me the last few weeks. I remember like if it was yesterday the day I got lucky and saved my ass. I remember having lunch in Madrid with my uncle, who was equal parts admired and horrified by how far I'd taken things, and telling him: "I'm gonna change my life and build a small empire of companies". That sentence, spoken across a table to someone who actually saw me, is the hinge my entire life now swings on. If I'm not mistaken I was around 25 years old that day. I had to transform my adventures into companies and a family of people that wanted to live building things for society rather than just pushing myself to my physical and psychological limit to prove myself. I learned something building that family: when aligned minds work together toward the same thing, they produce a third thing that none of them could have reached alone. This is also when I became more capitalistic. Capitalism enables adventure seekers like myself to build value for others. Some of us get pleasure only by building and experiencing things, expanding from our comfort zone every day, getting punched every day by reality. Capitalism gives a path for these people to express themselves like artists, building companies freely, while everyone else wins in exchange great products and experiences built by the crazy ones. The day I decided to create Lambda and then Ergodic is the day I realized I could turn my tendency to look for new experiences into value for society. It's the day I realized I'm a builder. For years I thought building was enough. That if you constructed something real, companies, systems, value, the rest would follow. It doesn't. Every book I read, every mentor I had, taught me to be disciplined, to have a clear goal, to build relentlessly. I was a lone warrior with friends, a sword and a purpose. Nobody taught me that the purpose itself was wrong. You can build everything you said you would and still realize you optimized for the wrong variable. Yesterday somebody showed me something about themselves that deeply broke my heart and opened it at the same time. That's why I write this. Today is another special day. It's the day I fully understood that life is not only about building but about loving. I always loved friends, partners, family but I didn't fully understand how important unconditional love is. I had been treating it as something that would arrive once I'd built enough, and I had it backwards. It was the point the whole time. It's the only thing that can make people heal their problems, their traumas, their addictions, their sadness. Loving everyone, including your adversaries, is the most therapeutic thing you can do. Some people believe that just believing in things is enough and they will manifest it. Others, and I think I was one of them, are too rational and believe everything is fixed with routines and discipline. What I learned is that you need both. You need a stupid belief in yourself and the people around you, and you also need the routines and work to transform those beliefs into reality. That's the correction I'm making: discipline without love optimizes for the wrong variable. I'm going to love unconditionally everyone around me and double down on my almost militaristic routines, exercise while also stopping every once in a while to analyze myself and my life. Why do I write this here? Writing is therapeutic for me. It's a way of processing feelings and growing. Why make it public? Because maybe it's useful for somebody. And because it exposes me. That exposure makes me feel I can't lie to myself. Keep creating, keep building, but love the people around you while you do it. That's the part I almost missed. To the person that taught me this: thank you. You showed me something about love that I couldn't have learned any other way. Now stop running from the things that hurt. I hope you double down on the path you took. Don't just patch the problem, fix it and grow. You can't escape from yourself. Love yourself, heal, and trust that you are worth the work. I do. --- ### CommitLLM: How to Verify an LLM Inference *Published: 2026-04-02* > LLM APIs ask you to trust that the provider ran the model and settings they advertise. CommitLLM adds cryptographic receipts and audits without zero-knowledge prover costs. URL: https://federicocarrone.com/articles/commitllm/ You send a prompt to an LLM API. The provider says it ran Llama 70B. Maybe it did. Maybe it served a smaller model to save money, changed the quantization, altered the decode settings, or patched the answer after generation. Today you usually cannot tell. You get text back, an invoice, and a promise. For casual use, a promise is often enough. For enterprise procurement, regulated systems, benchmark evaluation, or agent workflows making consequential decisions, it is not. If the model behind the answer matters, "trust us" is not a satisfying interface. Today you get two unsatisfying options. Statistical fingerprinting can give you evidence, but not exact per-response verification. Zero-knowledge proofs can give you much stronger guarantees, but the prover cost is still too high for production serving. You either get weak signals or strong proofs you cannot afford. We built [CommitLLM](https://github.com/lambdaclass/commitllm) to sit in that gap. The provider keeps the normal GPU serving path. No proving circuit. No per-response proof generation. The model answers normally, returns a compact receipt, and only opens internal trace data if the client asks for an audit. {% toolkit(label="Key takeaways") %} - LLM APIs ask you to trust that the provider ran the model and settings it advertised. Today you usually cannot tell, and for procurement, regulated systems, or consequential agents that is not enough. - CommitLLM binds each response to a committed deployment surface (weights, quantization, tokenizer, chat template, decode policy, post-processing) with a compact receipt, and opens internal trace data only when the client challenges it. - It is cheap enough to use in production: about 12 to 14 percent generation overhead, CPU-side verification (roughly 1.3 ms per challenged token, 10 ms for a full audit), so the normal serving path stays normal. - It is honest about guarantee strength: the matmul shell is exact via Freivalds, the attention interior is only approximate and confined to a tight corridor, and anything unreplayable is rejected fail-closed. - It is not zero-knowledge: an interactive audit with a client-held key, not a transferable succinct proof, but the economics work today where ZK does not. {% end %} ## How one audited response works At a high level, the protocol is simple: 1. The provider commits to the deployment surface: model weights, quantization, tokenizer, chat template, decode policy, and post-processing. 2. The client sends a prompt and gets back both the model output and a receipt binding that response to the committed deployment. 3. Most of the time, that is enough. The normal path stays fast. 4. If the response matters, the client challenges specific tokens and internal states. 5. The provider opens the requested trace data, and the verifier checks it on CPU against the committed model and configuration. The point is not to prove every inference up front. The point is to make cheating risky and cheap to detect without forcing the provider to run a cryptographic proving farm. ## Why this is cheap enough to matter The practical insight is that transformers spend most of their time doing matrix multiplication. If you can check those multiplies cheaply, the rest becomes manageable. The trick that makes this work is old. Freivalds published it in 1977. It gives you a way to test whether a matrix multiplication was done correctly without fully recomputing it. Suppose the provider claims it computed `z = W @ x` for some public weight matrix `W`. Recomputing `W @ x` directly is expensive. But if the verifier has a secret random vector `r` and has precomputed `v = r^T @ W`, then checking whether `v . x == r^T . z` costs only a dot product. If the provider used the wrong weights or produced the wrong output, the check fails with overwhelming probability. That covers the expensive shell of the transformer: `W_q`, `W_k`, `W_v`, `W_o`, `W_gate`, `W_up`, `W_down`, and `LM_head`. The remaining operations, RMSNorm, RoPE, SiLU, and the quantization bridges, are small enough to replay exactly. ## What the receipt binds The receipt does not just bind "some model ran." It binds the full surface that changes what comes out: - Model identity: a Merkle root over the checkpoint - Quantization scheme and configuration - Tokenizer, chat template, preprocessing - Decode policy: temperature, top-k, top-p, penalties, stop conditions - Output post-processing Change any of these and the receipt changes. The provider commits before learning which tokens and layers the verifier will challenge. ## Where the guarantees are exact and where they are not We are honest about what the protocol can and cannot do. **Exact.** Shell matmuls (Freivalds), quantization bridges, embedding lookup, the final-token tail from a captured boundary state, LM-head binding, logits, decode replay, output-policy replay. Algebraic verification or canonical recomputation. If it is wrong, the check fails. **Approximate.** The attention interior. Native FP16/BF16 attention is not bit-reproducible across GPUs. We constrain attention from both sides (shell-verified Q/K/V going in, committed post-attention output coming out) but we do not pretend it is exact. **Statistical.** Prefix/KV provenance in routine audit mode. The commitment binding is exact, but unopened positions are covered by challenge sampling. Deep audit upgrades this to exact. **Fail-closed.** Anything the verifier does not know how to replay is rejected. No silent best-effort fallbacks. ## Numbers The prototype adds roughly 12-14% overhead during generation. That is the first important number because it means the normal serving path still looks like normal serving. You are not replacing inference with a proof system. You are adding auditability to it. For Llama 70B, routine audit costs about 1.3 ms per challenged token, while a full single-token audit costs about 10 ms. Verification runs on CPU. No client-side GPU is required. That is the second important number: the verifier can be lightweight even when the model is not. On the corrected replay path for Qwen2.5-7B-W8A8 and Llama-3.1-8B-W8A8, the attention mismatch beyond 1k tokens is already narrow: worst-case `L_inf` of 8 and 9, with more than 99.8% of elements staying within one quantization bucket. In plain English, the only part we do not claim as exact is already confined to a tight corridor. ## Why not ZK ZK proofs give you a transferable proof object that anyone can verify. That is a stronger property than what CommitLLM provides. The cost is that ZK prover overhead is still orders of magnitude too high for production LLM serving. CommitLLM makes a different bet: interactive audit, client-held verifier key, small normal-path overhead. A fully disclosed audit transcript can be re-checked by third parties, but the receipt itself is not a succinct proof. For enterprise, regulated deployments, and decentralized compute, the interactive model is enough and the economics work today. ## Open work We need more model families beyond Qwen and Llama, tighter analysis of adversarial freedom after the attention corridor, stronger KV provenance, and [Lean formalization](@/series/concrete/2026-03-12-proving-systems-code-in-lean/index.md) of the core protocol properties. LLM infrastructure has made a strange peace with unverifiability. A provider puts a model name on a dashboard and the customer accepts it because there is no practical alternative. I do not think that equilibrium lasts. If model provenance matters, the interface should not be a logo and a promise. It should be a receipt and the ability to audit it. The [paper](https://github.com/lambdaclass/commitllm/blob/main/paper/main.pdf), [code](https://github.com/lambdaclass/commitllm), and [roadmap](https://github.com/lambdaclass/commitllm/blob/main/roadmap.md) are public. --- ### Self-Replicating Programs Emerge from Random Noise *Published: 2026-03-18* > Turing completeness is a shallow pit you fall into. Self-replication is an even shallower one. A recent paper shows that self-replicating programs spontaneously emerge from soups of random code, no design required. URL: https://federicocarrone.com/articles/computational-life/ Most programmers think Turing completeness is the interesting threshold for a computational system. It gets all the attention. But a lower, stranger threshold matters more for the origin of complex behavior: self-replication. A [recent paper](https://arxiv.org/abs/2406.19108) by Agüera y Arcas et al. shows that self-replicating programs spontaneously emerge from soups of random code. No one designs them. No fitness function selects for them. They assemble themselves from noise, take over the soup, and keep evolving. I reproduced the core result in about 300 lines of code. {% toolkit(label="Key takeaways") %} - Turing completeness is treated as the interesting threshold, but it is a low bar that almost any flexible system falls into by accident. Self-replication is a lower and stranger threshold, and it matters more for the origin of complex behavior. - Self-replication is not the same as a quine. It needs self-modification: programs that rewrite their own code as they run. - In Agüera y Arcas et al., self-replicators emerge spontaneously from a soup of random code, with no designer and no fitness function. They assemble from noise, take over the soup, and keep evolving. - The emergence is real, not an artifact of initialization: seeded random soups still cross from no replicators to replicators dominating. - The core result is simple enough to reproduce in about 300 lines. {% end %} ## What Is Turing Completeness In the 1930s, three people independently formalized what "computation" means: - **Alan Turing** defined the Turing machine: a tape, a head, and a set of rules for moving and writing - **Alonzo Church** defined the lambda calculus: a tiny language where everything is a function - **Kurt Gödel** defined recursive functions: a way to build computable functions from a few basic operations All three systems turned out to define exactly the same class of computable functions. This equivalence is a proven theorem. The Church-Turing thesis goes further: it conjectures that these formalisms capture *all* of what is effectively computable, not just that they agree with each other. A system is **Turing-complete** if it can simulate a Turing machine. This sounds like a high bar; it is not. ## The Bar Is Absurdly Low You need exactly two things for Turing completeness: - **A way to branch.** If/else, any conditional, any mechanism that chooses between two paths based on some state. - **A way to loop with unbounded state.** Recursion, a counter that can grow without limit, an infinite tape, any form of open-ended repetition with memory. Any system with both is Turing-complete. The problem is that both of these show up naturally in almost any system designed to be even slightly flexible. You add conditionals because users want "if this then that." You add repetition because users want to do things more than once. You add variables or cells or registers because users want to store intermediate results. At some point, without anyone planning it, those features combine into a general-purpose computer. Turing completeness keeps appearing in systems never intended for general computation: - **CSS** (animations + selectors + calc; debated, depends on the interaction model) - **Excel** (formulas + circular references or LAMBDA) - **PowerPoint** (animations with conditional triggers) - **Magic: The Gathering** (card interactions form a Turing machine) - **Conway's Game of Life** (gliders and logic gates) - **SQL** (recursive CTEs) - **The C preprocessor** (macro expansion with recursion tricks) - **TypeScript's type system** (conditional types + recursion) - **Minecraft redstone** (logic gates + repeaters) Nobody sat down and said "let's make CSS Turing-complete." They added features for styling, and those features accidentally crossed the threshold. Turing completeness is not a high peak you climb but a shallow pit you fall into. ## The Halting Problem as a Tell There is an ironic way people sometimes discover accidental Turing completeness. Someone notices that a certain class of configurations can loop forever and there is no general way to predict which ones will terminate. That is the halting problem. Alan Turing proved in 1936 that no algorithm can decide, for every possible program, whether it halts or runs forever. This is not a practical limitation but a mathematical impossibility. Once a system is Turing-complete, the halting problem applies to it. If you discover that your type checker, your template engine, or your build system can loop forever in ways you cannot predict, you have probably built a Turing-complete system by accident. ## What Is Self-Replication Self-replication is a different kind of threshold entirely. A self-replicating program is one that produces a copy of itself somewhere. John von Neumann studied this in the 1940s using cellular automata. He wanted to understand the minimum requirements for a machine that could build a copy of itself. His self-replicating automaton was enormously complex, involving hundreds of thousands of cells. But the concept was clear: a system that reads its own description and writes that description into a new location. The requirements for self-replication are more minimal than for Turing completeness. You do not need conditionals, arithmetic, or unbounded memory. You need: - A way to **read your own code** - A way to **write it somewhere else** - A way to **repeat** until the copy is complete A copy loop is sufficient. ## Quines Are Not What We Mean A quine is a program that outputs its own source code. It is a self-description trick and a beloved puzzle in programming culture. But quines are not the kind of self-replication that matters here. A quine runs once, prints itself to stdout, and stops. It does not spread. It does not compete. It does not modify its environment. The replicators in the paper we are about to discuss do something different. They overwrite their neighbors. When program A executes alongside program B, A writes its own bytes over B's bytes. Now there are two copies of A. Next epoch, both copies can overwrite two more neighbors. The difference is between self-description and self-propagation: quines describe themselves, replicators spread. ## Self-Replication Is a Lower Bar Than Turing Completeness Self-replication requires less than Turing completeness. Turing completeness requires conditionals and unbounded state. Self-replication requires only a copy loop. In the Forth variant studied in the paper, a 6-byte program is a complete self-replicator. In certain conditions, a single byte (`0C`) copies itself onto the neighbor's tape. One byte. If systems as narrow as CSS and Magic: The Gathering accidentally cross the Turing completeness threshold, then self-replication, a strictly easier feat, should be even harder to avoid. Any substrate where programs can read and write their own code is a candidate for spontaneous self-replication. The paper puts this to the test. ## Previous Work: Tierra and Avida Artificial life researchers have studied self-replicating programs since the early 1990s. Tom Ray's **Tierra** (1991) created a virtual machine populated with self-replicating programs. The programs competed for CPU time and memory. Over time, they evolved: parasites appeared that hijacked other programs' replication machinery, then hyper-parasites that resisted parasites, then an entire ecosystem of strategies. It was a landmark result. Charles Ofria's **Avida** (2004) extended the idea into a full research platform. Programs could evolve to perform computational tasks in exchange for faster replication. Avida demonstrated the evolution of complex behaviors from simple replicators. Both started with hand-crafted self-replicators. Neither asked whether one could assemble from nothing. ## The Paper ["Computational Life: How Well-formed, Self-replicating Programs Emerge from Simple Interaction"](https://arxiv.org/abs/2406.19108) by Blaise Agüera y Arcas, Jyrki Alakuijala, James Evans, Ben Laurie, Alexander Mordvintsev, Eyvind Niklasson, Ettore Randazzo, and Luca Versari answers that question. Yes. Self-replicators emerge spontaneously from random noise. The paper uses a language called BFF, an extension of Brainfuck. BFF has 10 instructions and operates on a tape with two heads: one for reading, one for writing. The key property is that the program itself lives on the tape. There is no separation between code and data. Programs are self-modifying. ## Why Self-Modification Is the Key Ingredient The experimental setup is not just "run random programs." It is "concatenate two programs onto a shared tape and run them together." This means program A's instructions can write over program B's bytes. This is essential. Without it, programs would execute in isolation and nothing would change. Each program would do its thing, produce no lasting effect on any other program, and the soup would stay random forever. The shared tape is the analog of chemistry. Molecules do not just exist side by side. They react. They break and form bonds. They change each other's structure. In the same way, programs on a shared tape change each other's code. That is where the dynamics come from. ## How the Experiment Works The "primordial soup" algorithm: 1. Create a pool of 2^17 programs, each 64 bytes long, initialized with random bytes 2. Each epoch, randomly pair programs up 3. For each pair, concatenate them into a single 128-byte tape and execute for up to 2^13 steps 4. After execution, split the tape back into two 64-byte halves 5. The modified halves replace the original programs 6. Apply a small background mutation rate (0.024%, random bit flips) 7. Repeat There is no selection pressure. There is no reward signal. There is no fitness function. Programs just run, modify each other's tapes, and get put back into the pool. ## What Emergence Looks Like For the first few thousand epochs, the soup looks like noise. Complexity, measured by high-order entropy approximated via brotli compression, stays low. Programs modify each other randomly and the distribution of bytes drifts toward a stationary state biased by the BFF instruction set. Then, suddenly, something changes. The paper calls it a "state transition." Complexity spikes. The number of unique programs in the soup drops sharply. A self-replicator has emerged. The self-replicator is a program that, when concatenated with any other program, copies itself over the other program's half of the tape. Once one appears, it spreads exponentially. Within a few hundred epochs after the transition, most of the soup is copies of the replicator or close variants. The paper traces the exact moment of emergence. In one case study, the first replicator appears at epoch 2355. Before that, there is a "pre-replicator" loop that copies bytes but is not yet a complete copier. Through a specific sequence of interactions with neighboring programs, the loop acquires the right structure to become a full self-replicator. ## It Is Not Just Random Initialization A natural objection: maybe the self-replicators were already present in the random initial soup and just needed time to take over. The paper rules this out carefully. They compare four experimental conditions across 1,000 runs each: - **Short runs** (128 epochs, random init): self-replicators appear only 3 times out of 1,000 - **Long runs** (16k epochs, random init): self-replicators appear about 40% of the time - **No-noise runs** (16k epochs, zero mutation, fixed shuffle): self-replicators appear about 50% of the time - **Seeded runs** (128 epochs, one hand-crafted replicator injected): self-replicators take over about 22% of the time The short vs. long comparison shows that self-replicators are being *created* by the dynamics, not just discovered in the initial noise. The no-noise variant is particularly striking: even with zero background mutation and deterministic pairing, self-replicators still emerge at a similar rate. The main driver is self-modification through program interaction, not random noise. ## Replicator Generations The first replicator to emerge is often fragile. In the case study, the initial self-replicator is a palindrome that copies itself in reverse. The reversed copy then copies itself back in the original direction. This works, but it has a flaw: it cannot handle zeros well. When the replicator encounters programs full of zero bytes, it floods the soup with zeros, a "zero-poisoning" phase where complexity stagnates and degrades. Then something happens again. A more robust replicator appears somewhere in the soup. This one has a `|{<,}|` structure that can overwrite zeros. It overtakes the first replicator and its zero-poisoned debris, and complexity starts climbing again. There are generations of replicators. The first one is not the last. They compete, and more robust variants replace fragile ones. Programs that copy themselves more reliably spread faster, a form of differential survival without an explicit fitness function. ## The 2D Grid The paper also runs simulations on a 2D grid of 240 x 135 programs, where each program can only interact with neighbors within distance 2. Self-replicators still emerge, but now you can watch them spread spatially. A replicator appears somewhere on the grid and expands outward like a wave, overwriting the random pre-life soup. The difference from the well-mixed soup is propagation speed. In the 0D soup, a replicator takes over in O(log n) epochs. On a 2D grid, it takes O(sqrt(n)) epochs because it spreads by contact with neighbors. The 2D setting also provides fertile ground for multiple replicator variants to coexist and compete, since spatial separation allows different lineages to persist. ## Beyond BFF The paper tests the same primordial soup setup with other computational substrates: **Forth.** A stack-based language with a restricted instruction set. Self-replicators emerge even faster and more consistently than in BFF. Almost all 1,000 runs show a state transition within 1,000 epochs (compared to 40% within 16,000 for BFF). The relative simplicity of self-replication in Forth explains this: a complete self-replicator can be just 6 bytes. In certain conditions, a single byte (`0C`) executing on an empty stack copies itself over. **Z80 CPU emulation.** A 2D grid of 16-byte programs executed on an emulated real-world Z80 processor. Self-replicators emerge here too, but the dynamics are richer. First, a wave of stack-based replicators sweeps the grid, using the fact that PUSH instructions write to memory via the stack pointer. These form a coexisting ecosystem. Then a second wave appears: replicators that exploit the Z80's LDIR and LDDR instructions, which are dedicated block-copy operations. These are more efficient and overtake the stack-based variants. Multiple generations of increasingly capable replicators, on a real CPU architecture. **Intel 8080.** Produces simple two-byte non-looping replicators in long-tape settings. **SUBLEQ.** The counterexample. SUBLEQ is one of the simplest Turing-complete languages, with just a single instruction. Despite being Turing-complete, self-replicators never spontaneously emerged in SUBLEQ soups, even after billions of iterations. The shortest known SUBLEQ self-replicator is 60 bytes (25 bytes in a 4-operand variant called RSUBLEQ4). Too long to assemble by accident from random interactions. ## What Determines Fertility The SUBLEQ counterexample is the sharpest result in the paper. It shows that Turing completeness is not what matters: SUBLEQ can compute anything, yet it cannot spontaneously produce self-replicators, because the shortest one is too long. What determines whether a computational substrate will spontaneously generate life is not its theoretical power. It is the **length of the shortest self-replicator** the language admits. Short replicators mean a short distance from random noise to a functioning copier. Long replicators mean the soup stays dead forever. BFF: short replicators, fertile. Forth: even shorter replicators, even more fertile. Z80: medium-length replicators, fertile but slower. SUBLEQ: long replicators, barren. This is a concrete, measurable property of a computational substrate. It is not about expressiveness or elegance or mathematical power. It is about how easy it is to accidentally build a copier. ## My Reproduction I reproduced the 2D BFF soup in about 300 lines of code. The setup is a 240 x 135 grid of 64-instruction BFF programs with a maximum of 2^13 execution steps per interaction. Each program is visualized as an 8x8 pixel square, colored by its content. The BFF interpreter is the bulk of the code. The rest is the soup logic (random pairing within a radius-2 neighborhood, execution, splitting, mutation) and visualization. What you see when you run it: a grid of colored static. For a while, nothing seems to happen. The colors shift slowly as programs modify each other. Then, somewhere on the grid, a patch of uniform color appears. It grows. Within a few hundred more epochs, the entire grid is dominated by one or two colors, with small patches of variation at the boundaries where different replicator lineages meet. That is a self-replicator taking over. No one wrote it. No one selected for it. It assembled itself from random interactions and then outcompeted everything else by the simple fact that it copies itself. ## Open Questions Self-replication is just the first step. In biology, replication was the spark, but evolution was the fire. Once replicators compete for resources, selection pressure creates increasingly complex strategies: metabolism, signaling, cooperation, parasitism, immune systems. Can any of that happen in these computational soups? The Z80 experiments hint at it: different replicator families coexist and compete, using different instruction strategies. But nothing remotely like metabolism or cooperation has been observed yet. What would a substrate need to support behaviors beyond replication? The paper does not answer this, but it frames the question precisely. We now know that self-replication is easy. The hard question is: what comes after? ## Structure Without Design The other things I have been writing about run in the opposite direction. In the [type systems article](@/articles/2026-01-01-type-systems/index.md), the whole point is that humans design constraints and the compiler enforces them. Types are structure we impose on programs to prevent mistakes. In the [theorem proving series](@/series/theorem-proving/_index.md), we go further: types become logical propositions, and the compiler verifies mathematical truth. Both are stories about humans carefully building formal structure and machines checking it. This paper is the opposite. Nobody designed the self-replicators. Nobody wrote a type system or a proof checker. The structure emerged from noise through simple interaction rules. The programs in the soup have no types, no correctness properties, no formal guarantees. They just copy and mutate and compete. And yet the result is recognizably structured. The replicators have internal logic. They use loops, conditionals, pointer arithmetic. Later generations are more robust than earlier ones. On the Z80, successive waves exploit increasingly powerful instructions. Something that looks like engineering appears without an engineer. The type systems article and the theorem proving series are about what happens when you start with structure and push it as far as it can go. This paper is about what happens when you start with nothing and structure shows up anyway. Both directions are worth understanding. Formal verification and spontaneous emergence both deal in programs, types, and computational substrates. They are the same landscape seen from opposite ends. ## Further Reading - [The paper](https://arxiv.org/abs/2406.19108) (Agüera y Arcas et al., 2024) - [Official code repository (cubff)](https://github.com/paradigms-of-intelligence/cubff), with CUDA and CPU implementations - [Z80 self-replicator explorer](https://github.com/znah/zff) by Alexander Mordvintsev - [Tom Ray's Tierra](https://en.wikipedia.org/wiki/Tierra_(computer_simulation)) (1991), the classic predecessor with hand-crafted initial replicators - [Avida](https://en.wikipedia.org/wiki/Avida) (Ofria and Wilke, 2004), artificial life platform for studying evolution of computational organisms - [Von Neumann's self-reproducing automata](https://en.wikipedia.org/wiki/Von_Neumann_universal_constructor) (1966), the theoretical foundation --- ### Fede's Guide to a Healthier Life *Published: 2026-03-13* > An ordered, evidence-based health guide for nerds who used to think the body didn't matter. URL: https://federicocarrone.com/articles/health-guide/ When I was young I loved science and engineering. Like most nerds, I thought thinking was the only thing that mattered. Working out seemed like a vanity project, something for people who cared about how they looked and not much else. I didn't understand the body-mind connection at all. I was a skinny kid who spent all day reading, tinkering with computers, and hanging out with friends. The idea that physical health could affect how well I think would have sounded like nonsense to me. It took me a long time to figure out how wrong I was. This guide is what I wish someone had handed me back then. Most of it is stuff with good research behind it. Some of it is things that seem useful but where I would be more cautious. The studies are linked at the end if you want to check them yourself. One thing before you start: order matters. The biggest mistake is working on the wrong layer. People buy supplements while sleeping six hours a night. They track every calorie while sitting still all day. Fix the foundation first. What you do most days matters much more than the occasional big push. This is practical guidance, not personal medical advice. If you are pregnant, have a chronic condition, or take medication, do not treat supplement doses or treatment-style suggestions here as defaults. Talk to a clinician first. --- ## The short version If you read nothing else, read this. These eleven habits capture most of what the research says actually matters. 1. Sleep 7-9 hours at consistent times 2. Walk 7,000-10,000 steps every day 3. Lift weights three times a week 4. Avoid ultra-processed food 5. Eat 1.6-2.2g of protein per kg of bodyweight 6. Get daylight daily, and regular sun exposure without burning 7. Maintain close relationships, in person 8. Delete social media, or at least delete the feed 9. Minimize alcohol 10. Do hard cardio at least once a week 11. Build and maintain muscle and VO2max across your life Do these consistently and you are already ahead of most people. Everything below explains why. --- ## The hierarchy 1. **Sleep** -- everything else breaks without it 2. **Movement** -- walk and lift, consistently 3. **Food** -- real ingredients, enough protein 4. **Metabolic health** -- a central driver of modern chronic disease 5. **People** -- social isolation raises mortality risk in a real way 6. **Purpose** -- having something worth being healthy for 7. **Mind** -- stress does physical damage 8. **Breathing** -- small changes can have surprisingly large effects 9. **Light & environment** -- what surrounds you every day matters 10. **Brain health** -- keep learning, protect your hearing 11. **Heat & cold** -- potentially useful, but much less foundational than the basics 12. **Supplements** -- conditional tools, only once the rest is solid 13. **After 35** -- testosterone and a few age-specific extras --- ## Sleep **Not just enough hours. Consistently.** Irregular sleep schedules predict mortality more strongly than sleep duration itself. Even after controlling for how long people sleep, the most regular sleepers have substantially lower risk of dying from any cause. **Pick a wake time. Keep it every day, including weekends.** That one decision improves almost everything else downstream. A lot of people think they function fine on five or six hours. Most of them don't. The problem with chronic short sleep is that people adapt to feeling bad. They stop feeling acutely sleepy and mistake that adaptation for functioning normally. When researchers restricted people to six hours a night for two weeks, their cognitive performance fell to the level of someone who had been awake for 24 hours straight, but they stopped noticing. Performance kept getting worse while they kept thinking they were fine. You cannot trust your own judgment about whether you are sleeping enough. **Morning light anchors the whole system.** Spend 10-15 minutes outside in natural light within an hour of waking, no sunglasses. This tells your brain what time it is. That signal sets when you get tired, when melatonin releases, and when you fall asleep that night. **Your bedroom should be cool, dark, and quiet.** Around 18-20C (64-68F). Even faint light through closed eyelids disrupts sleep quality. Keep the phone out of the room. **Caffeine lasts longer than you feel it does.** Its half-life is 5-7 hours. A coffee at 3pm still has half its effect at 9pm. People who sleep badly often drink more coffee to compensate, which makes the sleep worse, which increases the coffee. That loop can run for years without anyone noticing. Try cutting it off at noon for two weeks. Most people are surprised by the difference. If caffeine makes you crash later in the day, try delaying your first coffee 60-90 minutes after waking. The evidence for an exact timing window is weaker than internet advice makes it sound, but for some people it genuinely helps smooth energy across the day. Worth experimenting with. Before bed, some people benefit from magnesium glycinate (300-400mg) or inositol (1-2g), taken 30-60 minutes before sleep. These are optional, not foundational. Reasonable experiments if sleep is shaky, but not substitutes for fixing light, caffeine, schedule, and stress. Afternoon dip: 10-20 minutes of NSDR (non-sleep deep rest) or yoga nidra can restore alertness without the grogginess of a long nap. If you snore, wake unrefreshed, or feel exhausted despite adequate hours: get screened for sleep apnea. It is far more common than most people realize, and many cases go undiagnosed. Treatment, if needed, can produce dramatic improvements in energy, mood, and cognitive function. --- ## Movement People hear "exercise" and think gym membership, running plans, or complicated routines. But the people who walk the most have roughly half the mortality risk of those who walk the least. That is walking. No gym. No equipment. No special skill. **Aim for 7,000-10,000 steps a day.** The benefit plateaus at different levels depending on age. Roughly 6,000-8,000 for adults over 60, and 8,000-10,000 for younger adults. There is no need to obsess past that. The point is daily movement. **Lift weights three times a week.** Muscle is not just cosmetic. It is metabolically active tissue. It regulates blood sugar, produces anti-inflammatory signals, and improves how you feel and perform every day. Squats, deadlifts, pressing, rowing, and other compound movements done consistently capture most of the benefit. **Train balance on purpose.** Good balance prevents injuries, improves athletic performance, and makes everything from hiking to playing sports safer. Five to ten minutes before a workout is enough. Single-leg stands, step-ups, hip stability work. Address mobility and posture. Sitting for 8-10 hours a day shortens the hip flexors, weakens the posterior chain, and contributes to stiffness, weak hips, and bad spinal mechanics. All of that feeds chronic back pain. Five minutes a day goes a long way. Hip flexor stretches, thoracic rotation, hamstring work, basic mobility. The goal is to maintain full range of motion in every joint across your life. **Do 1-2 hard cardio sessions a week.** VO2max is one of the strongest predictors of lifespan we have. The gap between low and moderate fitness is one of the largest mortality differences of any modifiable factor. Hard cardio also helps mood and depression comparably to medication, partly because it raises BDNF, a protein that helps the brain grow and repair itself. Here is the good news if you are starting from zero: even 15 minutes a week of vigorous effort (jogging, fast cycling, hiking uphill) seems to get you a surprisingly large share of the total benefit. The curve is steep at the bottom. A lot of the gain appears to come from going from nothing to something. You do not need an hour. You need to get your heart rate up a few times a week and actually push. Protocol: 4-8 minutes of genuinely hard effort, rest, repeat 2-4 times. Run, bike, row, swim, anything that forces you to work. Add 150 minutes of easy movement per week on top of that. Walking counts. So does cycling, swimming, or anything you can do while still holding a conversation. Hard sessions build capacity. Easy movement keeps everything working between them. --- ## Food When people are given ultra-processed food instead of real food, they eat about 500 extra calories a day without noticing. Not because they are weaker or less disciplined. Because processed food is engineered to bypass normal satiety signals. **Cook real food** and most of the calorie problem becomes much easier. Drink enough water. Mild dehydration often shows up as fatigue, headaches, and poor concentration, not thirst. Aim for 2-3 liters a day, more if you train, use a sauna, or live in heat. Drink before you feel thirsty. Thirst is a late signal. **Eat enough protein.** Aim for 1.6-2.2g per kilogram of bodyweight per day, split across at least three meals. Meat, fish, eggs, dairy, legumes all count. Whole food beats powder most of the time. Protein shakes are a backup, not a foundation. If it suits you, eat in a roughly 8-10 hour window. This mostly helps with portion control and is often easier to sustain than calorie counting, but it is optional. Broccoli sprouts are a reasonable optional add-on. A small handful, about 30-60g raw, is enough. Crush them slightly before eating to release sulforaphane. Biomarker evidence is good. Long-term disease prevention is promising but not settled. **Eat fermented food every day.** Yogurt, kefir, kimchi, sauerkraut, kombucha. Any of these. Clinical evidence shows improvements in microbiome diversity and inflammatory markers. Fermented foods are cheap, low-risk, and the broader literature points in the same direction. **Minimize alcohol.** It disrupts sleep architecture, suppresses muscle protein synthesis, and is classified by the WHO as a Group 1 carcinogen. Any amount increases cancer risk. The U.S. Surgeon General's 2025 advisory reinforced this. There is no safe amount. The same basic rule applies to recreational drugs: do not build your life around them, and do not pretend weed is some harmless exception. Cannabis is lower-risk than many other drugs, but lower-risk is not the same as good for you. Regular use can impair attention, memory, motivation, sleep quality, and in some people worsen anxiety, paranoia, or psychosis risk. Smoked cannabis is also still smoke. The bigger problem is that most people have a stupid picture of what a drug problem looks like. They picture someone homeless, jobless, or obviously falling apart. That is usually not what it looks like at first. A person can still work, train, socialize, and look fine from the outside while the substance quietly becomes non-optional. And from the inside, it rarely feels like dependence. It just feels useful, social, deserved, or temporary. By the time someone can clearly see that they need it, it has usually been running more of their life than they wanted to admit. If you care about long-term health, the default should be minimization, not rationalization. **Take care of your mouth.** Gum disease is strongly linked to cardiovascular disease, diabetes, and dementia. The bacteria responsible enter the bloodstream and drive systemic inflammation. Brush and floss daily, scrape your tongue, get professional cleanings at least once a year. --- ## Metabolic health Type 2 diabetes, cardiovascular disease, fatty liver disease, many cancers, and a significant portion of cognitive decline all trace upstream to the same cluster: insulin resistance, chronic inflammation, and visceral fat accumulation. These processes develop slowly and silently, often for decades. By the time something shows up on a standard blood test, the process has usually been underway for years. Insulin resistance means your cells stop responding well to insulin, so your body has to produce more and more of it to keep blood sugar normal. That can go on for years before fasting glucose looks obviously bad. During that time, insulin stays high, fat storage gets easier, hunger regulation gets worse, energy gets less stable, and the whole system shifts toward inflammation and visceral fat gain. That is why fasting insulin often tells you something earlier than glucose alone. Metabolic health is highly responsive to behavior. Resistance training, real food, adequate protein, good sleep, and less ultra-processed food address most of it. Track these markers. These are practical targets, not universally agreed clinical cutoffs, but they are useful benchmarks: - **Fasting insulin:** below 10 uIU/mL is reasonable; below 5 is aspirational (common in lean, active people, but not a universal clinical standard). - **Triglyceride-to-HDL ratio:** below 2 is good. Above 3 deserves attention. - **Fasting glucose:** below 90 mg/dL is associated with good metabolic function. - **HbA1c:** below 5.4% usually reflects good glucose regulation. - **Waist-to-height ratio:** below 0.5. If all five are in range, you are usually in a much better place metabolically. If they are not, go back to the basics: lift, walk, sleep, eat real food, get leaner. --- ## Know your numbers Most health problems develop quietly. You feel fine until you don't. Get bloodwork once a year if you can. It is often cheap relative to the damage it can prevent. These are useful markers, not universal targets. Some are broadly accepted clinical measures, some are practical heuristics, and some are more about optimization than diagnosis. **Blood:** - **Vitamin D:** 40-60 ng/mL is a reasonable optimization range, not a universal medical target. Many people are low, especially in winter. - **Omega-3 index:** 8%+ is a common optimization target, not something everyone needs to test or chase. - **Ferritin, B12, zinc:** common silent deficiencies. - **Fasting glucose + fasting insulin + HbA1c:** these tell you how your body is handling sugar and how hard it is working to do it. - **Cholesterol panel:** LDL, HDL, triglycerides. Ask for ApoB if possible. It predicts cardiovascular risk better than LDL alone. - **hsCRP:** a simple inflammation marker. If it is elevated, something upstream is wrong. - **Triglyceride-to-HDL ratio:** under 2 is a useful heuristic for metabolic health, not a formal diagnostic cutoff by itself. **Metabolic:** - **Waist-to-height ratio:** aim for below 0.5. Waist size is one of the simplest proxies for visceral fat, and visceral fat drives disease. **Physical:** - **Grip strength:** one of the best predictors of how you will age. - **Single-leg balance:** stand on one leg, eyes open, for 10 seconds. If you cannot, pay attention. - **VO2max:** if you can test it, do. It is one of the strongest longevity predictors available. **Daily:** - **HRV** (heart rate variability) via a Whoop or Garmin: useful, but noisy. Ignore single readings. Watch the trend. --- ## People Social isolation is associated with substantially higher mortality risk, even after adjusting for the usual suspects like smoking, weight, and drinking. It is not just that lonely people eat worse or move less. Isolated people have higher cortisol, sleep worse, and get sick more often. The body notices when nobody is around. **In-person contact is different from digital contact.** Texting, screens, following people online. None of it fully replicates the biological and psychological effects of physical presence. Real time with real people matters. The phone is part of the problem. Social media platforms are engineered to be hard to put down. Variable reward, compulsive checking, fractured attention. Heavy use is consistently linked to worse sleep, higher anxiety, and shorter attention spans. **My recommendation is to delete your social media accounts, or at least run a real deactivation trial.** The evidence is not just correlational. In the largest randomized experiment on this, about 35,000 people before the 2020 U.S. election, deactivating Facebook for six weeks improved a combined emotional-state index covering happiness, depression, and anxiety by 0.060 standard deviations. Earlier randomized studies found that quitting Facebook for a week improved life satisfaction and positive emotions, and that taking a one-week break from major platforms improved well-being while reducing depression and anxiety. The literature is not perfectly one-sided, but the better experiments point in the same direction often enough that I think the default should be to stop, not to negotiate with the feed. The usual advice is to set limits. Use it less, turn off notifications, take breaks. For some people that is enough. For many it is not. The architecture of these products is designed to override your intentions. If you keep failing at moderation, stop trying to manage it. Remove the thing that keeps hijacking your attention. Keep a way to message people you care about. Delete the feed. Rules for the phone itself: no phone in the first hour after waking, no phone in the bedroom, no screens at meals, leave it behind on walks. Do a longer break sometimes. Even a weekend changes things. The first few hours feel uncomfortable. Then your attention starts to return. --- ## Purpose People with low life purpose have roughly double the mortality risk of those with high purpose. When you have purpose, you sleep better, train more consistently, recover faster from illness, and carry lower baseline inflammation. Purpose does not make you healthy on its own; it makes you do the things that make you healthy. That is why it belongs this high on the list. Purpose is not the same thing as happiness. Happiness comes and goes. Purpose is the thing that still asks something from you on bad days. People with strong purpose often report more short-term stress. They also tend to live longer. It makes discipline automatic. Health behaviors that require sustained willpower are hard when they feel like arbitrary obligations. They become nearly effortless when they're in service of something real. Where it comes from: meaningful work, caring for people, building something, getting genuinely skilled at something hard. Not consuming. Not accumulating. Purpose built on contribution and growth tends to last. Purpose built on status collapses the moment the achievements stop arriving. A question worth sitting with: what would you regret not having done? If the answer is vague, that probably means something. --- ## Mind Sustained stress raises cortisol. Elevated cortisol over time suppresses the immune system, impairs memory, and accelerates cardiovascular disease. The body interprets prolonged stress as ongoing threat and responds accordingly, indefinitely, if the signal keeps coming. There are three ways out: remove the source when you can, calm the body directly, or change how you are reading the situation. Exercise, sleep, sauna, and breathwork calm the body. Therapy can help with the interpretation part. The mistake is treating stress as something noble to endure instead of something to solve. **The specific problem to interrupt is rumination.** Most of the damage doesn't come from the event itself but from replaying it. What breaks the loop: intense physical exercise, deep absorption in something difficult, or writing to understand rather than just vent. If the loop persists, CBT is one of the fastest and most effective tools available. **Meditate.** Even 10 minutes a day reduces cortisol, improves attention, and makes you less reactive. You don't need an app or a philosophy. Sit still, breathe through your nose, and notice when your mind wanders. That's it. The benefit comes from the practice of noticing, not from achieving some special state. Let yourself be bored. The brain does its best organizational work when it has nothing to do. People who fill every gap tend to feel more anxious and less directed than people who leave space. Sit with boredom. It's uncomfortable for a few minutes. Then it becomes something else. Schedule play. Unstructured time with no output attached to it. In a structured life it won't happen by itself. Put it in the calendar. --- ## Breathing Most people breathe badly. Habitual mouth breathing and chronic overbreathing are common patterns with real downstream effects, and because breathing is automatic, almost nobody notices. Chronic mouth breathing is associated with dry mouth, snoring, disrupted sleep, and elevated blood pressure. Breathing too fast can keep the nervous system in a low-level stress state and reduce how efficiently oxygen gets delivered to tissue. **Nasal breathing is generally preferable, especially at rest and during sleep.** The nose filters and humidifies air, slows the breath naturally, and produces nitric oxide, a molecule that opens blood vessels and improves circulation. Mouth breathing tends to engage the upper chest rather than the diaphragm, which is associated with shallower, faster breaths and a more activated nervous system. During hard exercise mouth breathing is often necessary, but defaulting to nasal breathing at rest is worth cultivating. **Breathe less, not more.** Your blood is already 95-99% saturated with oxygen. Bigger breaths do not raise that. Carbon dioxide is part of what helps blood vessels open and oxygen move from red blood cells into tissue. When you breathe too fast, you blow off too much CO2 and things work worse, not better. Try slowing to around 5-6 breaths per minute through the nose. You should feel a mild air hunger, not distress. Just the sense that you could breathe more if you wanted to. That feeling usually settles as the pattern changes. Check your mechanics. One hand on the chest, one on the belly. On every inhale, the belly should move first and further. If your chest rises and your belly stays flat, you have defaulted to a chest-breathing pattern. Many adults have. It can be retrained with a few weeks of deliberate practice. If you suspect mouth breathing at night: micropore tape over the lips before bed is a low-cost experiment. If you have nasal obstruction, sleep apnea, or any breathing difficulties, check with a doctor first. --- ## Light & environment Morning sun helps set the circadian clock. Midday sun does something different. UV exposure on skin contributes to vitamin D production and nitric oxide release, and daylight is linked to mood and circadian regulation more broadly. People who actively avoid sun exposure have significantly higher all-cause mortality, even after controlling for vitamin D levels. The benefits of sunlight appear broader than any single molecule. Aim for regular daylight and some moderate sun exposure on skin when practical. Midday sun can be useful, but skin type, latitude, season, and skin-cancer risk matter. Morning light and moderate daylight exposure are the higher-confidence recommendations. Protect yourself from burning. Regular moderate exposure is useful. Repeated sunburn is not. Cover up or find shade before your skin starts to redden. Your indoor air is probably worse than you think. Indoor air is often 2-5 times more polluted than outdoor air because of cooking fumes, synthetic materials, cleaning products, and mold. A HEPA air purifier in the bedroom is one of the better environmental upgrades most people can make. Open windows when you can. A simple air-quality monitor for CO2 and PM2.5 helps because otherwise you are guessing. **Don't heat food in plastic.** Heating plastic releases phthalates and BPA, chemicals that interfere with the body's hormone system. Use glass or stainless steel. Protect quiet. Background noise, traffic, open offices, and constant notifications keep the stress system slightly activated all day. The body adapts to it consciously. The cortisol response does not adapt. --- ## Brain health A large share of dementia cases trace back to things you can actually change: not moving enough, bad sleep, cardiovascular problems, social isolation, hearing loss, depression, obesity, untreated vision loss, high LDL cholesterol. Most of these are already on this list. If you are following the rest of this document, you are already protecting your brain. **Learn something genuinely new, continuously.** The brain gets sharper through effortful use. Language, music, craft, sport. Anything hard enough to make you make mistakes and adjust builds cognitive reserve. Passive consumption does not do the same thing. If it feels easy all the time, it is probably not doing much. **Read.** We are not born with a reading circuit. The brain repurposes neurons that were doing other visual work and wires a new system through practice. Intensive reading strengthens white matter in language regions and can temporarily increase connectivity in language and sensorimotor networks. Those circuits do not just sit there untouched if you stop using them. Reading is not passive consumption. It makes you build meaning, hold context, simulate experience, and keep your attention on one thing for a while. Very few habits train all of that at once. If you used to read and stopped, the infrastructure is still there but it is degrading. Pick up a book. **Protect your hearing.** This is not an old-person problem. Noise damage starts whenever you keep exposing yourself to loud environments. Hearing loss pulls people out of conversation, out of social life, and over time out of cognitive stimulation. Get your hearing tested if something feels off. Wear earplugs anywhere that leaves your ears ringing afterward. That ringing is damage. --- ## Heat & cold Frequent sauna use is associated with much lower cardiovascular mortality. The data is observational and mostly from Finnish men, and the control group already used saunas once a week. Still, the mechanism story is plausible: lower blood pressure, better vascular function, lower inflammation. If you have access to a sauna, it is worth using. Cold exposure is useful too, but method and timing matter. Cold showers produce a genuine norepinephrine spike that improves alertness and mood. Full immersion goes further: larger surface area, longer exposure, stronger cardiovascular and anti-inflammatory adaptations. The research on cardiovascular benefit specifically comes from immersion studies, not showers. The main caveat: avoid cold exposure right after lifting. It can blunt the adaptation you are trying to get from strength training. Use it after cardio or on rest days. --- ## Supplements Supplements matter much less than sleep, movement, food, and metabolic health. Get the basics right first. No pill fixes a broken foundation. That said, a few are evidence-backed enough to be worth considering. None of these should be read as personal medical advice, and several matter only in the presence of deficiency, symptoms, or specific goals. ### Reasonable defaults for many people | Supplement | What it does | Dose and timing | |---|---|---| | **Creatine** | Increases strength and muscle mass. Also shows modest cognitive benefits, particularly memory and processing speed. One of the most studied supplements in existence, with an exceptional safety record. | 3-5g per day. Timing doesn't matter; just take it daily. | | **Omega-3 (EPA+DHA)** | Supports cardiovascular health and reduces inflammation. Aim for 1g/day for most people. Doses above 2g/day have been associated with elevated atrial fibrillation risk in multiple clinical trials. Test your omega-3 index to know your baseline. | 1g per day with your largest meal as a starting point. Adjust until your blood index reaches 8%. Do not exceed 2g/day without medical advice. | | **Vitamin D3 + K2** | Most useful in people who are actually deficient. Test before assuming. Magnesium is required for D3 activation. | 2,000-5,000 IU D3 + 100-200mcg K2 (MK-7 form), with food. Retest rather than guessing long-term. | | **Psyllium husk** | Lowers LDL cholesterol by around 13 mg/dL on average, improves blood sugar control, and significantly reduces hunger. One of the most effective cardiovascular interventions available without a prescription. | 10-15g per day in 2-3 doses, 15-30 minutes before meals, in a full glass of water. | | **Magnesium glycinate** | Required for over 300 biological processes. Improves sleep quality, reduces blood pressure, and supports muscle recovery. | 300-400mg, 30-60 minutes before bed. | | **Inositol** | Can improve sleep onset and quality in some people. Also supports insulin sensitivity, though most of that evidence comes from studies in women with PCOS. Pairs well with magnesium. | 1-2g, 30-60 minutes before bed. Avoid treating it as a default if you already sleep well. | ### Consider depending on your situation | Supplement | What it does | Dose and timing | |---|---|---| | **Berberine** | Activates AMPK, improving insulin sensitivity and reducing blood glucose, HbA1c, and triglycerides. Most useful for people with elevated metabolic markers. Interacts with many medications (statins, metformin, blood pressure drugs, and others) because it inhibits CYP enzymes. | 500mg, 2-3 times per day with meals. This is not a casual supplement if you take medication. | | **Arabinoxylan** | Lowers LDL cholesterol through a different mechanism than psyllium (converts cholesterol to bile acids via gut bacteria). Also a prebiotic. Less clinical evidence than psyllium but promising results from a Stanford multiomic trial. | 5-15g per day, with meals. Start low; high doses can cause GI discomfort. | | **Prebiotic fiber (inulin)** | Feeds beneficial gut bacteria, especially Bifidobacterium. **Keep doses moderate.** A Stanford trial found that 30g/day triggered systemic inflammation and liver stress markers in some participants. Low doses from food (garlic, onions, bananas) or modest supplementation are fine. High-dose supplementation is not. | 2-5g/day with meals. Do not exceed 10g/day as a supplement. | | **Zinc** | Useful only if bloodwork confirms deficiency. Excess zinc blocks copper absorption. | 15-30mg with food. | | **Collagen + Vitamin C** | Supports tendon and connective tissue repair. Take before exercise; blood flow to tendons spikes during training, which is when collagen is incorporated. | 10-15g collagen + 50mg vitamin C, 30-60 minutes before training. | | **Ashwagandha** | Reliably reduces cortisol during high-stress periods. Cycle off after 6-8 weeks. Avoid if pregnant, breastfeeding, or if you have thyroid, autoimmune, psychiatric, or hormone-sensitive conditions. | 300-600mg KSM-66 extract, with food. | | **Ginger** | Anti-inflammatory, digestive support, modest metabolic benefits. Low priority relative to everything else, but among the most robustly studied botanicals. | 1g with meals. | --- ## After 35 Everything above applies at any age. The items below become more relevant as you get older. **Track testosterone.** Total and free testosterone are worth discussing with a doctor if you notice declining energy, mood, recovery, or libido. Not a universal screening default, but worth knowing your numbers if something feels off. **Consider NMN or NR.** These raise NAD+, a molecule involved in cellular energy and repair that declines with age. The mechanism is real and confirmed in humans. Long-term longevity outcomes are not yet established. 500-1,000mg NMN or 300-500mg NR, in the morning. Interesting, but firmly optional. **Consider Tongkat Ali.** May support testosterone in some men with lab-confirmed low levels. This is not a default wellness supplement. See a doctor before starting. 400mg in the morning, retest at 8 weeks. --- ## Final principle Sleep enough. Move daily. Lift. Eat real food. Stay metabolically healthy. Keep close relationships. Protect your attention. Avoid obvious self-destruction. --- ## Research appendix **Biomarker targets** - Holick et al. (2011). Vitamin D deficiency. *NEJM.* [pubmed.ncbi.nlm.nih.gov/21675913](https://pubmed.ncbi.nlm.nih.gov/21675913/) - Harris et al. (2017). Omega-3 Index as cardiovascular risk factor. *AJCN.* [pubmed.ncbi.nlm.nih.gov/28404576](https://pubmed.ncbi.nlm.nih.gov/28404576/) **Sleep regularity** - Windred et al. (2024). Sleep regularity is a stronger predictor of mortality risk than sleep duration -- 60,977 UK Biobank participants. *Sleep.* [academic.oup.com/sleep/article/47/1/zsad253/7280269](https://academic.oup.com/sleep/article/47/1/zsad253/7280269) - Huang et al. (2024). Sleep patterns and risk of chronic disease as measured by long-term monitoring with commercial wearable devices -- All of Us cohort. *Nature Medicine.* [nature.com/articles/s41591-024-03155-8](https://www.nature.com/articles/s41591-024-03155-8) - Van Dongen et al. (2003). Chronic restriction to 6h of sleep per night for 14 days produced cognitive deficits equivalent to one night of total sleep deprivation; subjective sleepiness ratings stabilized while performance continued to decline, indicating participants were largely unaware of their impairment. *Sleep.* [pubmed.ncbi.nlm.nih.gov/12683469](https://pubmed.ncbi.nlm.nih.gov/12683469/) **Sleep duration** - Shen et al. (2024). Sleep duration and all-cause mortality. *GeroScience.* [link.springer.com/article/10.1007/s11357-024-01469-8](https://link.springer.com/article/10.1007/s11357-024-01469-8) **Walking** - Paluch et al. (2022). Daily steps and mortality -- 15 cohorts. *Lancet Public Health.* [thelancet.com/journals/lanpub/article/PIIS2468-2667(21)00302-9/fulltext](https://www.thelancet.com/journals/lanpub/article/PIIS2468-2667(21)00302-9/fulltext) **Resistance training and muscle** - Paluch et al. (2024). Resistance exercise training and cardiovascular health -- AHA scientific statement. *Circulation.* [ahajournals.org/doi/10.1161/CIR.0000000000001189](https://www.ahajournals.org/doi/10.1161/CIR.0000000000001189) - Andersen et al. (2025). Muscle power vs strength as a mortality predictor -- 3,889 adults followed 10.8 years; lowest vs highest muscle power: HR 5.88 (men), 6.90 (women). Power (force x velocity) predicted mortality more strongly than strength alone. *Mayo Clinic Proceedings.* [mayoclinicproceedings.org/article/S0025-6196(25)00100-4/abstract](https://www.mayoclinicproceedings.org/article/S0025-6196(25)00100-4/abstract) **VO2max & exercise for mental health** - Ahmadi et al. (2022). Vigorous physical activity, incident heart disease, and cancer -- 71,893 UK Biobank adults with accelerometers; 15 min/week of vigorous activity (6+ METs) was associated with about 18% lower all-cause mortality; optimal dose ~54 min/week for 36% reduction; steep dose-response at the bottom. *European Heart Journal.* [pubmed.ncbi.nlm.nih.gov/36302460](https://pubmed.ncbi.nlm.nih.gov/36302460/) - Kokkinos et al. (2022). Cardiorespiratory fitness and mortality risk -- 750,302 participants. *JACC.* [jacc.org/doi/10.1016/j.jacc.2022.05.031](https://www.jacc.org/doi/10.1016/j.jacc.2022.05.031) - Noetel et al. (2024). Exercise for depression -- network meta-analysis of 218 RCTs; exercise comparable to antidepressants for non-severe depression; walking, jogging, yoga, and strength training all showed significant effects. *BMJ.* [pubmed.ncbi.nlm.nih.gov/38355154](https://pubmed.ncbi.nlm.nih.gov/38355154/) - Blumenthal et al. (1999). Exercise vs sertraline for depression -- comparable outcomes. *Archives of Internal Medicine.* [pubmed.ncbi.nlm.nih.gov/10547175](https://pubmed.ncbi.nlm.nih.gov/10547175/) - Blumenthal et al. (2007). Exercise vs medication for major depression -- RCT. *Psychosomatic Medicine.* [pubmed.ncbi.nlm.nih.gov/17846259](https://pubmed.ncbi.nlm.nih.gov/17846259/) - Carek et al. (2011). Exercise for mental health disorders. *Primary Care.* [pubmed.ncbi.nlm.nih.gov/21628294](https://pubmed.ncbi.nlm.nih.gov/21628294/) - Szuhany et al. (2015). Meta-analytic review of BDNF levels after exercise. *Journal of Psychiatric Research.* [pubmed.ncbi.nlm.nih.gov/25455510](https://pubmed.ncbi.nlm.nih.gov/25455510/) **Social connection** - Wang et al. (2023). Social isolation and mortality -- 90 cohort studies, 2.2 million participants; social isolation associated with 32% higher all-cause mortality. *Nature Human Behaviour.* [nature.com/articles/s41562-023-01617-6](https://www.nature.com/articles/s41562-023-01617-6) - WHO Commission on Social Connection (2025). Loneliness affects 1 in 6 people globally; accounts for roughly 871,000 deaths per year; doubles depression risk; significantly raises stroke, heart disease, diabetes, and cognitive decline risk. [who.int/groups/commission-on-social-connection](https://www.who.int/groups/commission-on-social-connection) **Social media deactivation** - Allcott et al. (forthcoming). Effect of deactivating Facebook and Instagram on emotional state -- about 35K participants in two RCTs before the 2020 election; Facebook deactivation improved an emotional-state index by 0.060 SD; Instagram by 0.041 SD overall, with exploratory subgroup evidence of larger effects for women under 25. *American Economic Journal: Economic Policy.* [aeaweb.org/articles?id=10.1257/pol.20240806](https://www.aeaweb.org/articles?from=f&id=10.1257%2Fpol.20240806) - Allcott et al. (2020). The welfare effects of social media -- randomized four-week Facebook deactivation increased subjective well-being, reduced political polarization, and increased offline activities including socializing with family and friends. *American Economic Review.* [aeaweb.org/articles?id=10.1257/aer.20190658](https://www.aeaweb.org/articles?id=10.1257%2Faer.20190658) - Tromholt (2016). Quitting Facebook leads to higher levels of well-being -- one-week randomized Facebook break in Denmark improved life satisfaction and positive emotions, with larger effects for heavy and passive users. *Cyberpsychology, Behavior, and Social Networking.* [doi.org/10.1089/cyber.2016.0259](https://doi.org/10.1089/cyber.2016.0259) - Lambert et al. (2022). Taking a one-week break from social media improves well-being, depression, and anxiety -- randomized one-week cessation of Facebook, Instagram, Twitter, and TikTok improved well-being and reduced depression and anxiety. *Cyberpsychology, Behavior, and Social Networking.* [doi.org/10.1089/cyber.2021.0324](https://doi.org/10.1089/cyber.2021.0324) **Chronic stress** - Kivimaki et al. (2012). Job strain and coronary heart disease -- 197K participants. *Lancet.* [pubmed.ncbi.nlm.nih.gov/22981903](https://pubmed.ncbi.nlm.nih.gov/22981903/) **Rumination** - Nolen-Hoeksema et al. (2008). Rethinking rumination. *Perspectives on Psychological Science.* [pubmed.ncbi.nlm.nih.gov/26158958](https://pubmed.ncbi.nlm.nih.gov/26158958/) **Sauna** - Laukkanen et al. (2015). Sauna and cardiovascular mortality. *JAMA Internal Medicine.* [jamanetwork.com/journals/jamainternalmedicine/fullarticle/2130724](https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/2130724) - Laukkanen et al. (2018). Sauna and all-cause mortality. *BMC Medicine.* [bmcmedicine.biomedcentral.com/articles/10.1186/s12916-018-1077-z](https://bmcmedicine.biomedcentral.com/articles/10.1186/s12916-018-1077-z) **Cold water immersion** - Grassi et al. (2024). Cold immersion and hypertrophy -- meta-analysis. *European Journal of Sport Science.* [pubmed.ncbi.nlm.nih.gov/38581211](https://pubmed.ncbi.nlm.nih.gov/38581211/) **Ultra-processed food** - Hall et al. (2019). Ultra-processed diet RCT. *Cell Metabolism.* [cell.com/cell-metabolism/fulltext/S1550-4131(19)30248-7](https://www.cell.com/cell-metabolism/fulltext/S1550-4131(19)30248-7) - Diaz et al. (2025). Premature mortality attributable to ultra-processed food across 8 countries -- 4-14% of premature deaths attributable to UPF, linear dose-response of 3% higher all-cause mortality per 10% increase in UPF consumption. *American Journal of Preventive Medicine.* [ajpmonline.org/article/S0749-3797(25)00072-8/abstract](https://www.ajpmonline.org/article/S0749-3797(25)00072-8/abstract) - Sheringham et al. (2025). Minimally processed vs ultra-processed diets -- randomized crossover trial of 55 adults; minimally processed diets produced greater weight loss even when both diets followed healthy eating guidelines, suggesting processing itself matters beyond nutrient content. *Nature Medicine.* [nature.com/articles/s41591-025-03842-0](https://www.nature.com/articles/s41591-025-03842-0) **Time-restricted eating** - Sutton et al. (2018). Early TRE and insulin sensitivity. *Cell Metabolism.* [cell.com/cell-metabolism/fulltext/S1550-4131(18)30253-5](https://www.cell.com/cell-metabolism/fulltext/S1550-4131(18)30253-5) - Lowe et al. (2022). TRE vs caloric restriction -- 12-month RCT. *NEJM.* [nejm.org/doi/10.1056/NEJMoa2114833](https://www.nejm.org/doi/10.1056/NEJMoa2114833) **Protein** - Morton et al. (2018). Protein dose-response for muscle. *BJSM.* [bjsm.bmj.com/content/52/6/376](https://bjsm.bmj.com/content/52/6/376) **Fermented foods** - Wastyk et al. (2021). Fermented food vs high-fiber diet RCT. *Cell.* [pubmed.ncbi.nlm.nih.gov/34256014](https://pubmed.ncbi.nlm.nih.gov/34256014/) **Broccoli sprouts / sulforaphane** - Egner et al. (2014). Sulforaphane and detoxification enzyme induction. *Cancer Prevention Research.* [pubmed.ncbi.nlm.nih.gov/24913818](https://pubmed.ncbi.nlm.nih.gov/24913818/) **Creatine** - Wang et al. (2024). Creatine and strength -- meta-analysis of 23 RCTs. *JISSN.* [pubmed.ncbi.nlm.nih.gov/38589767](https://pubmed.ncbi.nlm.nih.gov/38589767/) - Prokopidis et al. (2023). Creatine and memory in healthy adults -- meta-analysis of RCTs; strongest effects in adults aged 66-76. *Nutrition Reviews.* [pubmed.ncbi.nlm.nih.gov/35984306](https://pubmed.ncbi.nlm.nih.gov/35984306/) - Xu et al. (2024). Creatine and cognitive function -- 16 RCTs, 492 participants; significant effects on memory (SMD=0.31) and processing speed. *Frontiers in Nutrition.* [pubmed.ncbi.nlm.nih.gov/39070254](https://pubmed.ncbi.nlm.nih.gov/39070254/) **Omega-3** - Mattumpuram et al. (2024). Omega-3 and cardiovascular outcomes -- 176K participants. *Clinical Cardiology.* [pubmed.ncbi.nlm.nih.gov/39501985](https://pubmed.ncbi.nlm.nih.gov/39501985/) - Kim et al. (2024). Omega-3 and sudden cardiac death. *JACC.* [pubmed.ncbi.nlm.nih.gov/39753168](https://pubmed.ncbi.nlm.nih.gov/39753168/) **Vitamin D3** - Ruiz-Garcia et al. (2023). Vitamin D and all-cause mortality -- 80 RCTs. *Nutrients.* [pubmed.ncbi.nlm.nih.gov/37111028](https://pubmed.ncbi.nlm.nih.gov/37111028/) - Petrelli et al. (2024). Vitamin D and cancer mortality. *European Journal of Cancer.* [pubmed.ncbi.nlm.nih.gov/38368684](https://pubmed.ncbi.nlm.nih.gov/38368684/) **Psyllium husk** - Anderson et al. (2000). Psyllium and LDL reduction -- meta-analysis of 8 controlled trials. *AJCN.* [pubmed.ncbi.nlm.nih.gov/10648260](https://pubmed.ncbi.nlm.nih.gov/10648260/) - Jovanovski et al. (2018). Psyllium and LDL, non-HDL, and apoB -- systematic review and meta-analysis of 28 RCTs; median dose of 10.2g/day significantly reduced LDL. *AJCN.* [pubmed.ncbi.nlm.nih.gov/30239559](https://pubmed.ncbi.nlm.nih.gov/30239559/) **Arabinoxylan and inulin** - Lancaster, Snyder et al. (2022). Arabinoxylan vs inulin in 18 healthy adults -- multiomic trial with escalating doses (10-30g/day); arabinoxylan reduced LDL via bile acid conversion; inulin at 30g/day triggered systemic inflammation and liver stress markers (elevated ALT) in some participants. Individual responses varied widely. *Cell Host & Microbe.* [pubmed.ncbi.nlm.nih.gov/35483363](https://pubmed.ncbi.nlm.nih.gov/35483363/) **Magnesium** - Zhang et al. (2022). Magnesium and sleep quality -- meta-analysis. *Nutrients.* [pubmed.ncbi.nlm.nih.gov/35889031](https://pubmed.ncbi.nlm.nih.gov/35889031/) **NMN / NR** - Yoshino et al. (2021). NMN and insulin sensitivity. *Science.* [pubmed.ncbi.nlm.nih.gov/34878335](https://pubmed.ncbi.nlm.nih.gov/34878335/) - Igarashi et al. (2022). NMN and muscle function. *NPJ Aging.* [pubmed.ncbi.nlm.nih.gov/35173189](https://pubmed.ncbi.nlm.nih.gov/35173189/) **Inositol** - Unfer et al. (2016). Inositol and insulin sensitivity -- primarily studied in PCOS populations. *Gynecological Endocrinology.* [pubmed.ncbi.nlm.nih.gov/27351220](https://pubmed.ncbi.nlm.nih.gov/27351220/) **Tongkat Ali** - Leisegang et al. (2022). Tongkat Ali and testosterone -- meta-analysis of 5 RCTs. *Medicina.* [pubmed.ncbi.nlm.nih.gov/36013514](https://pubmed.ncbi.nlm.nih.gov/36013514/) **Ashwagandha** - Arumugam et al. (2024). Ashwagandha and cortisol -- meta-analysis of 9 RCTs. *Explore.* [pubmed.ncbi.nlm.nih.gov/39348746](https://pubmed.ncbi.nlm.nih.gov/39348746/) **Ginger** - Anh et al. (2020). Ginger and human health -- 109 RCTs. *Nutrients.* [pmc.ncbi.nlm.nih.gov/articles/PMC7019938](https://pmc.ncbi.nlm.nih.gov/articles/PMC7019938/) **Collagen + Vitamin C** - Shaw et al. (2017). Gelatin, vitamin C, and collagen synthesis. *AJCN.* [pubmed.ncbi.nlm.nih.gov/28174772](https://pubmed.ncbi.nlm.nih.gov/28174772/) **Breathing / nasal breathing** - Nestor, J. (2020). *Breath: The New Science of a Lost Art.* Riverhead Books. *(Accessible synthesis of breathing research; not a peer-reviewed source.)* - McKeown, P. (2015). *The Oxygen Advantage.* William Morrow. *(Popular science; covers CO2 tolerance and nasal breathing; not peer-reviewed.)* - Yang et al. (2024). Mouth closure and airflow in patients with obstructive sleep apnea -- nonrandomized clinical trial; mouth closure helped some patients but worsened airflow in heavy mouth-breathers with velopharyngeal obstruction, which is why mouth-taping should be treated as an experiment, not a default. *JAMA Otolaryngology-Head & Neck Surgery.* [pubmed.ncbi.nlm.nih.gov/39361293](https://pubmed.ncbi.nlm.nih.gov/39361293/) **Purpose and mortality** - Alimujiang et al. (2019). Association between life purpose and mortality among US adults -- 6,985 participants, HR 2.43 comparing lowest to highest purpose category. *JAMA Network Open.* [pubmed.ncbi.nlm.nih.gov/31125099](https://pubmed.ncbi.nlm.nih.gov/31125099/) - Shiba et al. (2021). Associations between purpose in life and mortality by socioeconomic status -- 13,159 U.S. adults older than 50; the highest purpose category was associated with lower 8-year mortality across SES strata. *American Journal of Preventive Medicine.* [pubmed.ncbi.nlm.nih.gov/34020851](https://pubmed.ncbi.nlm.nih.gov/34020851/) **Light / sun exposure** - Lindqvist et al. (2016). Avoidance of sun exposure as a risk factor for major causes of death. *Journal of Internal Medicine.* [pubmed.ncbi.nlm.nih.gov/26992108](https://pubmed.ncbi.nlm.nih.gov/26992108/) **Air quality and cognition** - Allen et al. (2016). Associations of cognitive function scores with carbon dioxide, ventilation, and volatile organic compound exposures. *Environmental Health Perspectives.* [pubmed.ncbi.nlm.nih.gov/27285588](https://pubmed.ncbi.nlm.nih.gov/27285588/) **Endocrine disruptors** - Trasande et al. (2015). Estimating burden and disease costs of exposure to endocrine-disrupting chemicals in the European Union. *Journal of Clinical Endocrinology & Metabolism.* [pubmed.ncbi.nlm.nih.gov/25742516](https://pubmed.ncbi.nlm.nih.gov/25742516/) **Dementia / modifiable risk factors** - Livingston et al. (2024). Dementia prevention, intervention, and care -- Lancet Commission update. *Lancet.* [pubmed.ncbi.nlm.nih.gov/38849039](https://pubmed.ncbi.nlm.nih.gov/38849039/) **Hearing loss and dementia** - Thomson et al. (2017). Hearing loss as a risk factor for dementia -- systematic review. *Laryngoscope Investigative Otolaryngology.* [pubmed.ncbi.nlm.nih.gov/28894825](https://pubmed.ncbi.nlm.nih.gov/28894825/) **Cognitive reserve / learning** - Stern et al. (2020). Cognitive reserve and lifestyle factors. *Lancet Neurology.* [pubmed.ncbi.nlm.nih.gov/31704171](https://pubmed.ncbi.nlm.nih.gov/31704171/) **Reading and the brain** - Keller & Just (2009). Intensive remedial reading instruction increased white matter integrity in language tracts in children -- diffusion tensor imaging study. *Nature Neuroscience.* [pubmed.ncbi.nlm.nih.gov/19234458](https://pubmed.ncbi.nlm.nih.gov/19234458/) - Berns et al. (2013). Reading a novel increased functional connectivity in language and sensorimotor brain regions, persisting for days after reading. *Brain Connectivity.* [pubmed.ncbi.nlm.nih.gov/24111882](https://pubmed.ncbi.nlm.nih.gov/24111882/) - Dehaene, S. (2009). *Reading in the Brain.* Viking. *(How the brain repurposes visual neurons for reading; foundational neuroscience.)* - Wolf, M. (2007). *Proust and the Squid.* Harper. *(History and science of the reading brain; covers circuit formation and degradation.)* **Metabolic health / insulin resistance** - Kraft, J.R. (1975). Detection of diabetes mellitus in situ. *Postgraduate Medicine.* (foundational fasting insulin work) - Reaven, G.M. (1988). Role of insulin resistance in human disease. *Diabetes.* [pubmed.ncbi.nlm.nih.gov/3056758](https://pubmed.ncbi.nlm.nih.gov/3056758/) - Magkos et al. (2024). Metabolically healthy obesity -- from epidemiology and mechanisms to clinical implications. *Nature Reviews Endocrinology.* [nature.com/articles/s41574-024-01008-5](https://www.nature.com/articles/s41574-024-01008-5) - Despres, J.P. & Lemieux, I. (2006). Abdominal obesity and metabolic syndrome. *Nature.* [pubmed.ncbi.nlm.nih.gov/17167475](https://pubmed.ncbi.nlm.nih.gov/17167475/) - Sethi et al. (2024). Ketogenic diet for bipolar disorder and schizophrenia -- pilot trial of 21 adults on antipsychotics; after 4 months, 79% showed clinically meaningful psychiatric improvement, none still had metabolic syndrome. First US clinical trial on metabolic psychiatry since 1965. Small, uncontrolled, but notable effect size. *Psychiatry Research.* [pubmed.ncbi.nlm.nih.gov/38547601](https://pubmed.ncbi.nlm.nih.gov/38547601/) --- ### China is trying to commoditize the complement *Published: 2026-01-22* > What happens to the West's services advantage when strong AI models are free, portable, and running on every laptop? URL: https://federicocarrone.com/articles/china-commoditizing-the-complement/ China is trying to win by commoditizing the complement. The strategy is working, though not without friction. This is a structural challenge the West should take seriously instead of dismissing. For the last two decades, the West exported cognition because it owned the platforms, the cloud, the software distribution, and the talent concentration. If the cognitive engine becomes cheap, portable, and good enough, that asymmetry weakens. A small country can buy or download the same cognitive machinery, then apply it to its own bureaucracy, its own companies, its own language, its own domain problems. The West has dominated the thinking and services world. Software, finance, media, research, management layers, and the export of expertise. The US is the clearest example. In 2024, US services exports were about 1.1 trillion dollars, the highest on record. The US and the West sell thinking at scale. AI threatens to flatten that advantage because AI turns thinking into infrastructure. China dominates the atoms world. Industrial capacity, manufacturing throughput, physical supply chains, cost curves. In 2023 China produced about 28 percent of global manufacturing value added. If you can make the layer next to you cheap and abundant, you drain its pricing power and force value to move somewhere else. In AI, the complement is model access. For a lot of Western companies, the business is still basically gated intelligence sold as an API. China has every incentive to make that layer feel like electricity: available everywhere, cheap, hard to monopolize. Open weight releases are part of that play: DeepSeek, Qwen, Kimi, and MiniMax are only a few of the Chinese open-source models. Once strong models are common, model access stops being a moat. It becomes a commodity input. A huge fraction of what we call services is legible work: reading, writing, coding, summarizing, translating, drafting, answering, generating variations, searching a space of options. That layer is now replicable and it is getting local. Apple is publishing technical reports about on-device foundation models, including aggressive quantization aimed at making serious inference run on consumer hardware. When strong models run on a laptop, countries stop importing thinking as a service. They import weights, or they distill, fine-tune, and deploy inside their own borders. The commodity play is working, but it is not frictionless. China faces real constraints, and they shape how far the strategy can go. Capital controls limit how freely Chinese companies can operate globally. The state can redirect investment at a scale nobody else can match, but centralized allocation tends to overshoot. Solar panel overcapacity, steel oversupply, and the EV price war all follow the same pattern: massive subsidized buildout that ends up compressing margins for everyone, including the Chinese firms themselves. Top talent still flows toward open research environments. By MacroPolo's Global AI Talent Tracker, China is the single largest source of the world's top-tier AI researchers, yet a large share of them end up doing that work in the United States. Tightening political control over universities and private firms can speed up execution on defined goals, but it makes the open-ended, high-risk research that produces real breakthroughs harder to sustain. Predictability matters for long-term innovation, and the last few years dented it. The abrupt suspension of Ant Group's record $34 billion IPO in November 2020, the 2021 crackdowns that erased the for-profit tutoring sector overnight and froze new game approvals for months, and Didi's forced retreat from US markets after its 2021 listing all sent the same signal: any company can become a target without warning. Foreign firms recalibrated their exposure and some domestic founders turned cautious. Centralized coordination buys speed, but it also shrinks the appetite for bets that do not match current state priorities. The West still has one advantage that is hard to replicate: it is where most of the world's ambitious talent wants to live, work, and build. It is a compound effect of open institutions, freedom of movement, and decades of accumulated trust. As long as that holds, the West keeps attracting the talent and the capital that turn ideas into new industries. None of these constraints cancel out the commodity play. They set its ceiling. China can drive the price of model access toward zero faster than anyone, but the open-ended research and the institutional trust that turn a cheap commodity into new industries are much harder to subsidize into existence. China stays strong in atoms because it already has the scale advantage. The West still leads in areas that require deep institutions and long accumulated competence, frontier research and high trust services in particular. But AI compresses the services premium by making a large portion of cognition cheap and replicable. That is why open models matter. They attack the margin structure of the thinking economy. If you sell intelligence, this is bad news. If you own distribution, hardware, data, or a workflow people cannot easily leave, you survive. If you own atoms and you get thinking for free, you get a scary combination, because the services premium that sustained Western economic leadership for decades can be undercut by a player with industrial dominance and access to the same cognitive tools. --- ### Fede's Guide to Type Systems: From Generics to Dependent Types *Published: 2026-01-01* > A practical guide to type systems, from everyday generics to dependent types that prove correctness, with examples in Rust, Scala, and Idris URL: https://federicocarrone.com/articles/type-systems/ Every type error you've ever cursed at was a bug caught before production. Type systems reject nonsense at compile time so you don't discover it at 3 AM. But they vary wildly in what they can express and what guarantees they provide. If you learn nothing else: ADTs + pattern matching + generics. These three concepts will improve your code in any language and take days to learn. The concepts here progress from generics (reusable code) through traits (shared behavior) to linear types (resource safety) to dependent types (proving correctness). Each step buys you more compile-time guarantees at the cost of more work satisfying the type checker. ## Structure Concepts are organized into tiers: | Tier | What's Here | You Should Know If... | |------|-------------|----------------------| | 1: Foundational | Generics, ADTs, pattern matching | You write code | | 2: Mainstream Advanced | Traits, GADTs, flow typing, existentials | You design libraries | | 3: Serious Complexity | HKT, linear/ownership types, effects | You want deep FP or systems programming | | 4: Research Level | Dependent types, session types | You work on PLs or verification | | 5: Cutting Edge | HoTT, QTT, graded modalities | You do research | You don't need to read linearly. Jump to what interests you. But concepts build on each other: if GADTs confuse you, make sure you understand [ADTs](#algebraic-data-types) first. ## Tier 1: Foundational Every modern statically-typed language supports them. If you use a typed language, you're already using these. ## Parametric Polymorphism (Generics) You write a function to get the first element of a list of integers. Then you need it for strings. Then for custom types. You end up with `first_int`, `first_string`, `first_user`, duplicated code that differs only in types. The alternative, using a universal type like `Object` or `any`, throws away type safety entirely. You're back to hoping you don't pass the wrong thing. Abstract over the type itself. Write the function *once* with a type parameter, and it works for *any* type. The crucial property is **parametricity**: the function must behave the same way regardless of what type you plug in. It can't inspect the type or behave differently for integers versus strings. That constraint is exactly what makes generics powerful. When a function is parametric in `T`, it can only shuffle `T` values around. It can't create new `T`s out of thin air, can't compare them, can't print them. This means generic functions come with "theorems for free": guarantees about their behavior that follow purely from their type signature. For example, a function with signature `fn mystery(x: T) -> T` can *only* return `x`. There's nothing else it could possibly return. The type signature alone proves the implementation. In languages where values are freely copyable (like Haskell), `pair :: a -> (a, a)` must return `(x, x)`. The parametricity constraint eliminates every other possibility. (In Rust, move semantics add a wrinkle: `fn pair(x: T) -> (T, T)` won't compile without `T: Clone`, because `x` can only be used once.) What this gives you: - Write once, use with any type - No code duplication - Compiler verifies each usage with concrete types - Parametricity guarantees: a function `fn id(x: T) -> T` can *only* return `x` Fair warning: the syntax gets ugly. You will eventually write `fn process` and question your life choices. This is the price of expressiveness. It's still better than duplicating code. ```rust // Rust: One function works for any type T fn first(slice: &[T]) -> Option<&T> { slice.first() } first(&[1, 2, 3]); // Option<&i32> first(&["a", "b"]); // Option<&&str> first(&[User::new("Ada")]); // Option<&User> // The implementation is identical for all types // Parametricity: we can't inspect T, so we can only shuffle values around ``` ```rust // What can this function possibly do? fn mystery(x: T) -> T { // We can't: // - Print x (we don't know it implements Display) // - Compare x (we don't know it implements Eq) // - Clone x (we don't know it implements Clone) // We can ONLY return x x } ``` --- ## Algebraic Data Types You're modeling a user who can be either anonymous or logged in. In a typical OOP language, you might write: ```java class User { String name; // null if anonymous boolean isLoggedIn; } ``` Tony Hoare calls null references his "billion dollar mistake," but the problem runs deeper than null. This type allows four states: anonymous with no name, anonymous with a name (!), logged in with a name, logged in without a name (!). Two of these are nonsense, but your type permits them. Every function must check for and handle impossible states. Types should describe *exactly* the valid states. We need two tools: - **Sum types** (enums, tagged unions): "this OR that", a value is one of several variants - **Product types** (structs, records): "this AND that", a value contains all fields Combined, these are **algebraic data types** (ADTs). The "algebra" comes from how you calculate possible values: products multiply (struct with 2 bools = 2 × 2 = 4 states), sums add (enum with 3 variants = 3 states). Here's the algebra in action. Consider: - `bool` has 2 values: `true`, `false` - `(bool, bool)` has 2 × 2 = 4 values: `(true, true)`, `(true, false)`, `(false, true)`, `(false, false)` - `enum Either { Left(bool), Right(bool) }` has 2 + 2 = 4 values: `Left(true)`, `Left(false)`, `Right(true)`, `Right(false)` The power comes from combining them. You model your domain with exactly the states that make sense. If a user is either anonymous (no data) or logged in (with name and email), you write that directly. The type system then enforces that you can't access a name for an anonymous user, because that field doesn't exist in that variant. - **Make illegal states unrepresentable**: if your type can't hold invalid data, you can't have bugs from invalid data - No null checks for "impossible" cases - Self-documenting domain models - Exhaustive [pattern matching](#pattern-matching) (covered next) ```rust // Rust: This type CANNOT represent an invalid state enum User { Anonymous, LoggedIn { name: String, email: String }, } // There is no way to construct: // - "Logged in with no name" (LoggedIn requires name) // - "Anonymous with a name" (Anonymous has no fields) fn greet(user: &User) -> String { match user { User::Anonymous => "Hello, guest".to_string(), User::LoggedIn { name, .. } => format!("Hello, {}", name), } } ``` ```rust // Model a payment result: each variant has exactly the data it needs enum PaymentResult { Success { transaction_id: String, amount: f64 }, Declined { reason: String }, NetworkError { retry_after_seconds: u32 }, } // No nulls. No "reason" field that's only valid sometimes. // Each variant is self-contained. ``` ```rust // The classic: Option replaces null enum Option { None, Some(T), } // Result replaces exceptions enum Result { Ok(T), Err(E), } // These are ADTs! Sum types with generic parameters. ``` If you come from OOP, ADTs require rethinking how you model data. Instead of class hierarchies with methods, you define data structures and functions that pattern match on them. Available in Rust, Haskell, OCaml, F#, Scala, Swift, and Kotlin. --- ## Pattern Matching Given an algebraic data type, you need to branch on its variants and extract data. With OOP, you'd use `instanceof` checks or the visitor pattern, both verbose and error-prone. Worse: when you add a new variant, the compiler doesn't tell you about all the places that need updating. Pattern matching is the natural counterpart to [ADTs](#algebraic-data-types). If constructors *build* sum types, pattern matching *deconstructs* them. They're two sides of the same coin. The compiler knows every possible variant of your sum type. When you write a `match`, it checks that you've covered them all. Forget a case? Compile error. Add a new variant to your enum? Every `match` in your codebase that doesn't handle it becomes a compile error. This is **exhaustiveness checking**. The comparison to `if-else` or `switch` is instructive. In most languages, `switch` doesn't warn you about missing cases. Pattern matching does. And unlike the visitor pattern (OOP's answer to this problem), pattern matching is concise and doesn't require boilerplate classes. - **Exhaustiveness checking**: forget a case, get a compile error - **Refactoring safety**: add a variant, compiler shows everywhere to update - **Destructuring built-in**: extract fields while matching - Cleaner than if-else chains or visitor patterns ```rust // Rust: Compiler ensures all cases handled enum Message { Quit, Move { x: i32, y: i32 }, Write(String), ChangeColor(u8, u8, u8), } fn process(msg: Message) -> String { match msg { Message::Quit => "Goodbye".to_string(), Message::Move { x, y } => format!("Moving to ({}, {})", x, y), Message::Write(text) => format!("Writing: {}", text), Message::ChangeColor(r, g, b) => format!("Color: #{:02x}{:02x}{:02x}", r, g, b), } } // If you forget a case: // error[E0004]: non-exhaustive patterns: `Message::ChangeColor(_, _, _)` not covered ``` ```rust // Guards add conditions fn describe(n: i32) -> &'static str { match n { 0 => "zero", n if n < 0 => "negative", n if n % 2 == 0 => "positive even", _ => "positive odd", } } // Nested patterns fn first_two(items: &[T]) -> Option<(T, T)> { match items { [a, b, ..] => Some((a.clone(), b.clone())), _ => None, } } ``` Pattern matching is now in C# 8+, Python 3.10+, and most functional languages. Once you use it, you won't go back. --- ## Subtyping You have a function that logs any HTTP response. You've also defined `JsonResponse` and `XmlResponse` types with extra fields. Without some way to express "a JsonResponse *is* an HttpResponse," you'd need separate logging functions for each, or abandon type safety. If type `B` has everything type `A` has (and possibly more), you can use a `B` anywhere an `A` is expected. This is subtyping: `JsonResponse <: HttpResponse` means JsonResponse is a subtype of HttpResponse. Think of it as a contract. An `HttpResponse` promises certain capabilities: it has a status code and body. A `JsonResponse` fulfills that contract and adds more: it also has a parsed object and content type. Anywhere the code expects "something with status and body," a JsonResponse works fine. The extra fields are ignored but don't cause problems. This is the Liskov Substitution Principle encoded in the type system: if `JsonResponse <: HttpResponse`, then any property that holds for HttpResponse should hold for JsonResponse. ### Nominal vs Structural: Two Philosophies This is a fundamental classification of type systems, not just a detail of subtyping: | Aspect | Nominal | Structural | |--------|---------|------------| | Type equality | Based on declared name | Based on shape/structure | | Subtyping | Explicit declaration required | Implicit if structure matches | | Philosophy | "What it's called" | "What it can do" | | Abstraction | Strong boundaries | Flexible composition | | Refactoring | Rename breaks compatibility | Structure changes break compatibility | **Nominal typing** requires explicit declarations. Even if two types have identical fields, they're different types unless related by declaration: ```java // Java: nominal typing class Meters { double value; } class Feet { double value; } // These are DIFFERENT types despite identical structure Meters m = new Meters(); Feet f = m; // ERROR: incompatible types ``` **Structural typing** cares only about shape. If it has the right fields and methods, it fits: ```typescript // TypeScript: structural typing interface Point { x: number; y: number; } // Any object with x and y is a Point const p: Point = { x: 1, y: 2 }; // OK const q: Point = { x: 1, y: 2, z: 3 }; // OK (extra field allowed) class Coordinate { x: number; y: number; } const r: Point = new Coordinate(); // OK (same structure) ``` **Go's approach** is interesting: nominal for defined types, but interfaces are structural. A type implements an interface if it has the right methods, no declaration needed. ```go // Go: structural interfaces type Reader interface { Read(p []byte) (n int, err error) } // MyFile implements Reader without declaring it type MyFile struct { ... } func (f MyFile) Read(p []byte) (int, error) { ... } // Works: MyFile has the right method func process(r Reader) { ... } process(MyFile{}) // OK ``` ```typescript // TypeScript: Structural subtyping interface HttpResponse { status: number; body: string; } interface JsonResponse { status: number; body: string; contentType: "application/json"; parsed: object; } function logResponse(res: HttpResponse): void { console.log(`${res.status}: ${res.body}`); } const jsonRes: JsonResponse = { status: 200, body: '{"ok": true}', contentType: "application/json", parsed: { ok: true } }; logResponse(jsonRes); // OK! JsonResponse has everything HttpResponse needs ``` The downside: subtyping complicates type inference and introduces variance questions. When `JsonResponse <: HttpResponse`, is `List` a subtype of `List`? It depends on whether the list is read-only (covariant), write-only (contravariant), or mutable (invariant). See [Variance](#variance) for details. Rust sidesteps this by using traits instead of subtyping for polymorphism. --- ## Tier 2: Mainstream Advanced These features appear in modern production languages but require more sophistication to use well. They're essential for library authors and for writing highly generic code. ## Traits / Typeclasses You want to sort a list. Sorting requires comparison. How does the generic sort function know how to compare your custom `User` type? Approaches without traits: - **Inheritance**: `User extends Comparable`, but what if User comes from a library you don't control? - **Pass a comparator every time**: verbose, easy to forget - **Duck typing**: no compile-time safety, crashes at runtime if method missing Separate the *interface* from the *type*. Define `Ord` (ordering), `Eq` (equality), `Display` (printing) as standalone interfaces called traits (Rust) or typeclasses (Haskell). Then declare that `User` implements them. In some languages you can do this for external types too; Rust is more restrictive because of orphan rules. This goes a long way toward the "expression problem": how do you add both new types and new operations without modifying existing code? With OOP inheritance, adding new types is easy (new subclass), but adding new operations is hard (modify every class). With traits, you can add new operations (new trait) and implement them for existing types. In practice, coherence rules and orphan restrictions limit how far this goes, but it covers a lot of real cases. When used with static dispatch, the implementation is resolved at compile time. When you call `user.cmp(&other)`, the compiler knows exactly which comparison function to use because it knows the concrete type. No vtable lookup. This is called **monomorphization**: the compiler generates specialized code for each type you use. (If you use trait objects or dynamic dispatch instead, you pay for a vtable lookup at runtime, but gain flexibility.) In Rust, the "coherence" rule prevents chaos: there can be at most one implementation of a trait for a given type. You can't have two different ways to compare Users. This means you can always predict which implementation will be used. Haskell's typeclasses have a similar expectation but enforce it differently, and some languages are more permissive. - **Ad-hoc polymorphism**: different behavior for different types, resolved at compile time - **Retroactive implementation**: add interfaces to types you don't own - **Coherence**: at most one implementation per type (no ambiguity) - **Trait bounds**: require capabilities, not inheritance ```rust // Rust: Define a trait trait Summary { fn summarize(&self) -> String; } // Implement for your type struct Article { title: String, author: String, content: String, } impl Summary for Article { fn summarize(&self) -> String { format!("{} by {}", self.title, self.author) } } // Implement for another local type struct Number(i32); impl Summary for Number { fn summarize(&self) -> String { format!("The number {}", self.0) } } // Use as a bound: T must implement Summary fn notify(item: &T) { println!("Breaking news! {}", item.summarize()); } // Or with impl Trait syntax fn notify_short(item: &impl Summary) { println!("Breaking news! {}", item.summarize()); } ``` ```rust // Standard library traits use std::fmt::Display; use std::cmp::Ord; // Multiple bounds fn print_sorted(mut items: Vec) { items.sort(); for item in items { println!("{}", item); } } // Default implementations trait Greet { fn name(&self) -> &str; fn greet(&self) -> String { format!("Hello, {}!", self.name()) // default impl } } ``` Rust's orphan rules restrict where you can implement traits to prevent conflicting implementations. This is sometimes frustrating but maintains coherence. --- ## Associated Types You're defining an `Iterator` trait. Each iterator produces items of some type. With regular generics, you'd write `Iterator`. But this makes `Iterator` and `Iterator` *different traits*, and a type could implement both, creating ambiguity about which one applies. What you want: the item type should be *determined by* the implementing type, not chosen by the user. Some type parameters are *outputs* (determined by the implementation), not *inputs* (chosen by the caller). Associated types express this: "when you implement this trait, you must specify what Item is." The distinction matters. With a regular type parameter like `Iterator`, you're saying "this is an iterator that could work with any T." But that's not how iterators work. A `VecIterator` always produces the type that the Vec contains. The type is determined by the iterator, not chosen by the user. Think of it as a type-level function. Given a type that implements `Iterator`, you can ask "what does it produce?" and get back the associated `Item` type. `Vec`'s iterator has `Item = i32`. `HashMap`'s `.iter()` produces `Item = (&K, &V)`. The implementing type determines the associated type. - **Cleaner APIs**: one trait, not a family of traits - **Type-level functions**: the implementing type determines the associated type - **Better error messages**: "Item not found" vs. "Iterator not satisfied" ```rust // Rust: The standard Iterator trait trait Iterator { type Item; // Associated type: implementor decides fn next(&mut self) -> Option; } // Implementing: specify what Item is struct Counter { count: u32, max: u32, } impl Iterator for Counter { type Item = u32; // Counter produces u32s fn next(&mut self) -> Option { if self.count < self.max { self.count += 1; Some(self.count) } else { None } } } // Using: the Item type is known from the iterator type fn sum_all>(iter: I) -> i32 { iter.fold(0, |acc, x| acc + x) } ``` ```rust // Without associated types (what you'd have to write) trait BadIterator { fn next(&mut self) -> Option; } // Problem: impl BadIterator and impl BadIterator // are different traits! A type could implement both! ``` Associated types are less flexible than type parameters when you need the same type to implement a trait multiple ways. But for most cases, they make APIs cleaner. --- ## Flow-Sensitive Typing You check if a value is null before using it. You know it's not null inside the `if` block. But does the type system know? ```java // Java: limited type narrowing Object x = maybeNull(); if (x != null) { // Java lets you call x.toString() here without complaint. // But the type is still Object. The compiler doesn't narrow it // to a more specific non-null type you can branch on further. // Compare this to TypeScript, where the type actually changes. x.toString(); } ``` **Flow-sensitive typing** (also called **occurrence typing** or **type narrowing**) refines types based on control flow. After a type check, the type system narrows the variable's type in branches where the check succeeded. Type information *changes* as you move through code. The type of `x` isn't fixed at its declaration. It evolves based on what the program has learned. After `if (x !== null)`, the type of `x` in the `then` branch is narrower than at the start. This bridges static and dynamic typing philosophies. Dynamic languages always know the runtime type. Static languages traditionally fix types at declaration. Flow-sensitive typing lets static types benefit from runtime checks without losing static guarantees. ```typescript // TypeScript: flow-sensitive typing function process(value: string | number | null) { // Here: value is string | number | null if (value === null) { return; // value is null in this branch } // Here: value is string | number (null eliminated) if (typeof value === "string") { // Here: value is string console.log(value.toUpperCase()); // OK: string method } else { // Here: value is number console.log(value.toFixed(2)); // OK: number method } } // Works with user-defined type guards too interface Success { data: object; } interface Failure { error: string; code: number; } function isSuccess(result: Success | Failure): result is Success { return (result as Success).data !== undefined; } function handle(result: Success | Failure) { if (isSuccess(result)) { console.log(result.data); // TypeScript knows result is Success } else { console.error(result.error); // TypeScript knows result is Failure } } ``` ```kotlin // Kotlin: smart casts fun process(x: Any) { if (x is String) { // x is automatically cast to String here println(x.length) // No explicit cast needed } // Works with null checks too val name: String? = getName() if (name != null) { // name is String here, not String? println(name.length) } } ``` - **Eliminates redundant casts**: The compiler tracks what you've already checked - **Catches impossible branches**: If a branch can never execute, the compiler warns - **Natural null handling**: Null checks automatically narrow types - **Type guards**: User-defined functions can narrow types Flow-sensitive typing complicates the type system. The type of a variable depends on *where* you are in the code, not just its declaration. This makes type checking more complex and can lead to surprising behavior when variables are reassigned or captured in closures. Available in TypeScript, Kotlin, Ceylon, Flow (JavaScript), Rust (with pattern matching), Swift, and increasingly other modern languages. --- ## Intersection and Union Types You have a value that could be one of several types. Or a value that must satisfy multiple interfaces simultaneously. Regular generics and subtyping don't express these relationships cleanly. ```typescript // How do you type a function that accepts string OR number? // How do you require an object to be BOTH Serializable AND Comparable? ``` **Union types** (`A | B`) represent "this OR that." A value of type `A | B` is either an `A` or a `B`. You must handle both possibilities before using type-specific operations. **Intersection types** (`A & B`) represent "this AND that." A value of type `A & B` has all properties of both `A` and `B`. It satisfies both interfaces simultaneously. These correspond to logical OR (union) and AND (intersection). ```typescript // TypeScript: Union types type StringOrNumber = string | number; function process(value: StringOrNumber) { // Must narrow before using type-specific operations if (typeof value === "string") { console.log(value.toUpperCase()); // OK: string method } else { console.log(value.toFixed(2)); // OK: number method } } // Discriminated unions: tagged sum types type Result = | { kind: "ok"; value: T } | { kind: "error"; error: E }; function handle(result: Result) { switch (result.kind) { case "ok": return result.value; // TypeScript knows value exists case "error": throw result.error; // TypeScript knows error exists } } ``` ```typescript // TypeScript: Intersection types interface Named { name: string; } interface Aged { age: number; } type Person = Named & Aged; // Must have both name AND age const person: Person = { name: "Ada", age: 36 }; // Intersection for mixin-style composition interface Loggable { log(): void; } interface Serializable { serialize(): string; } type LoggableAndSerializable = Loggable & Serializable; function process(obj: LoggableAndSerializable) { obj.log(); // OK: has Loggable obj.serialize(); // OK: has Serializable } ``` ```scala // Scala 3: Union and intersection types def process(value: String | Int): String = value match case s: String => s.toUpperCase case i: Int => i.toString // Intersection: must satisfy both traits trait Runnable { def run(): Unit } trait Stoppable { def stop(): Unit } def manage(service: Runnable & Stoppable): Unit = service.run() service.stop() ``` - **Precise typing for heterogeneous data**: JSON, configs, APIs with variant responses - **Mixin composition**: Combine interfaces without inheritance hierarchies - **Discriminated unions**: Type-safe pattern matching on tagged variants - **Subtyping relationships**: `A` is subtype of `A | B`; `A & B` is subtype of `A` ### Intersection Types in Type Theory In formal type theory, intersection types have deeper significance. The **intersection type discipline** can type more programs than simple types: some programs untypable in System F become typable with intersections. This is because intersections allow giving a term multiple types simultaneously. ``` // The identity function can have type: λx.x : Int → Int // for integers λx.x : String → String // for strings λx.x : (Int → Int) ∧ (String → String) // BOTH at once with intersection ``` This enables **principal typings** for some systems and is used in program analysis and partial evaluation. TypeScript (extensive), Scala 3, Flow, Ceylon, Pike, CDuce, and research languages. Java has limited intersection types in generics (``). Haskell achieves similar effects through typeclasses. --- ## Generalized Algebraic Data Types (GADTs) You're building a type-safe expression language. You have `Add(expr, expr)` and `Equal(expr, expr)`. `Add` should return an integer; `Equal` should return a boolean. But with regular ADTs, the `Expr` type has no way to track what type of value each expression produces. Your `eval` function either: - Returns `Object` and requires downcasting (unsafe) - Returns a sum type like `Value::Int | Value::Bool` and requires checking (verbose) Let each constructor specify its own, more precise return type. `Add` constructs an `Expr`; `Equal` constructs an `Expr`. The type parameter tracks what the expression evaluates to. With regular ADTs, all constructors return the same type. `Some(x)` and `None` both return `Option` for the same `T`. But with GADTs, different constructors can return *different* type instantiations. `LitInt(5)` returns `Expr`. `LitBool(true)` returns `Expr`. The "generalized" means this flexibility. Pattern matching reveals the payoff. If you match on an `Expr` and see a `LitInt`, the compiler knows the type parameter is `Int`. It can use this knowledge to type-check the branch correctly. You can return an `Int` directly, not a wrapped type. This information flow from patterns to type checking is what makes type-safe evaluators possible. The cost: type inference breaks. The compiler can't always figure out what type an expression should have, because it depends on which constructor was used. You need explicit type annotations at GADT match sites. - **Type-safe interpreters and DSLs**: the type tracks the expression's result type - **Eliminates impossible patterns**: if you match on `Expr`, you know it's not `LitBool` - **More precise types**: information flows from patterns to the type checker Rust doesn't support GADTs directly. Scala 3 has clean syntax: ```scala // Scala 3: GADT syntax enum Expr[A]: case LitInt(value: Int) extends Expr[Int] case LitBool(value: Boolean) extends Expr[Boolean] case Add(left: Expr[Int], right: Expr[Int]) extends Expr[Int] case Equal(left: Expr[Int], right: Expr[Int]) extends Expr[Boolean] case If[T](cond: Expr[Boolean], thenBr: Expr[T], elseBr: Expr[T]) extends Expr[T] // Type-safe eval: return type matches expression type def eval[A](expr: Expr[A]): A = expr match case Expr.LitInt(n) => n // here A = Int, return Int ✓ case Expr.LitBool(b) => b // here A = Boolean, return Boolean ✓ case Expr.Add(l, r) => eval(l) + eval(r) case Expr.Equal(l, r) => eval(l) == eval(r) case Expr.If(c, t, e) => if eval(c) then eval(t) else eval(e) // This WON'T compile: // Expr.Add(Expr.LitBool(true), Expr.LitInt(1)) // Error: expected Expr[Int], got Expr[Boolean] // Usage val expr: Expr[Int] = Expr.Add(Expr.LitInt(1), Expr.LitInt(2)) val result: Int = eval(expr) // Type-safe: result is Int, not Object ``` GADTs are available in Haskell, OCaml, and Scala 3. TypeScript has limited support through type guards. --- ## Existential Types You want a collection of things that share a trait, but they're different concrete types: `Vec` containing integers, strings, and custom structs. But `Vec` requires one specific `T`. Hide the concrete type behind an interface. An existential type says: "there *exists* some type `T` implementing this trait, but I won't tell you which." You can only use operations from the trait, nothing type-specific. The duality with generics: - **Generics (universal)**: caller picks the type, "for *all* types T, this works" - **Existentials**: callee picks the type, "there *exists* some type T, but you don't know which" Why is this useful? Consider a plugin system. Each plugin is a different type, but they all implement `Plugin`. You want a `Vec` containing all your plugins. With generics alone, you'd need `Vec`. With existentials, you get `Vec>`: a collection of "things that are some type implementing Plugin." The concrete types are hidden (existentially quantified), but you can still call Plugin methods on them. - **Heterogeneous collections**: mix different types with shared interfaces - **Information hiding**: callers can't depend on the concrete type - **Dynamic dispatch**: select implementation at runtime ```rust // Rust: dyn Trait is an existential type use std::fmt::Display; fn make_displayables() -> Vec> { vec![ Box::new(42), Box::new("hello"), Box::new(3.14), ] } fn print_all(items: Vec>) { for item in items { println!("{}", item); // Can only call Display methods } } // You don't know the concrete types, but you can display them all ``` ```rust // impl Trait in return position is also existential fn make_iterator() -> impl Iterator { // Caller doesn't know this is specifically a Range // They only know it's "some iterator of i32" 0..10 } // Useful for hiding complex iterator adapter chains fn complex_iter() -> impl Iterator { (0..100) .filter(|x| x % 2 == 0) .map(|x| x * x) .take(10) } ``` The cost: `dyn Trait` has runtime overhead (vtable lookup) and you can't recover the concrete type. Use generics when you know the type statically. --- ## Rank-N Polymorphism Normally, the *caller* of a generic function chooses the type parameter. But sometimes you want the *callee* to choose. Consider a function that applies a transformation to both elements of a pair, but the elements have different types. ```rust // This doesn't work in Rust fn apply_to_both(f: impl Fn(T) -> T, pair: (i32, String)) -> (i32, String) { (f(pair.0), f(pair.1)) // Error! T can't be both i32 and String } ``` In Rank-1 polymorphism (normal generics), `forall` is at the outside: the caller picks one `T` for the whole function. In Rank-2+, `forall` appears inside argument types: "the argument must be a function that works for *any* type." The "rank" refers to how deeply `forall` can be nested: - **Rank 0**: No polymorphism. $int \to int$. - **Rank 1**: $\forall$ at the top. $\forall T.\; T \to T$. Caller picks $T$. - **Rank 2**: $\forall$ in argument position. $(\forall T.\; T \to T) \to int$. The *argument* must be polymorphic. - **Rank N**: Arbitrary nesting. Why would you want this? Consider the ST monad trick in Haskell. `runST` has type $(\forall s.\; ST\; s\; a) \to a$. The $s$ type variable is universally quantified *inside* the argument. This means `runST` picks $s$, not the caller. Since $s$ is chosen by `runST` and immediately goes out of scope, no reference tagged with $s$ can escape. This is how Haskell provides safe, in-place mutation: the type system guarantees mutable references can't leak outside `runST`. The cost is severe: type inference becomes undecidable at Rank-3 and above (Rank-2 inference is decidable but already impractical for most use cases). You must annotate everything. Most languages avoid this complexity. - **More precise types**: "must work for all types" is a strong requirement - **Encapsulation**: ST monad uses Rank-2 types to ensure references can't escape - **Enable patterns impossible with Rank-1** Rust can't express full Rank-N types directly, though higher-ranked trait bounds (`for<'a>`) give you a limited form of Rank-2 over lifetimes. OCaml can go further: ```ocaml (* OCaml: Rank-2 polymorphism via record types *) (* Rank-1: caller chooses 'a *) let id : 'a -> 'a = fun x -> x (* Rank-2 requires a record with polymorphic field *) type poly_fn = { f : 'a. 'a -> 'a } let apply_to_both (p : poly_fn) (x, y) = (p.f x, p.f y) (* This works: id is polymorphic *) let result = apply_to_both { f = id } (42, "hello") (* result = (42, "hello") *) (* This FAILS: (+1) only works on int, not any type *) (* let bad = apply_to_both { f = fun x -> x + 1 } (42, "hello") *) (* Error: This field value has type int -> int which is less general than 'a. 'a -> 'a *) ``` ```haskell -- Haskell: cleaner Rank-2 syntax with RankNTypes extension {-# LANGUAGE RankNTypes #-} -- runST : (forall s. ST s a) -> a -- The 's' type variable is chosen by runST, not the caller. -- This makes it impossible to return an STRef outside runST, -- because the 's' won't match anything outside. ``` Rank-N types are rare outside Haskell. Most languages don't support them, and you can usually work around their absence. --- ## Tier 3: Serious Complexity These features require significant learning investment but let you write abstractions impossible in simpler type systems. They're common in functional programming languages and increasingly appearing in mainstream languages. ## Higher-Kinded Types (HKT) `Vec`, `Option`, `Result`: they're all "containers" you can map a function over. You write `map` for `Vec`. Then for `Option`. Then for `Result`. The implementations look structurally identical: ```rust fn map_vec(items: Vec, f: impl Fn(A) -> B) -> Vec fn map_option(item: Option, f: impl Fn(A) -> B) -> Option fn map_result(item: Result, f: impl Fn(A) -> B) -> Result ``` Can't we abstract over the *container itself*? Types have **kinds**, just as values have types: ``` Int : Type -- a plain type Vec : Type -> Type -- takes a type, returns a type Result : Type -> Type -> Type -- takes two types, returns a type ``` `Int` is a complete type. But `Vec` by itself is not a type. You can't have a variable of type `Vec`. You need `Vec` or `Vec`. `Vec` is a *type constructor*: give it a type, get back a type. HKT lets you abstract over type constructors like `Vec` and `Option`, instead of only types like `Int`. You can define `Functor` as a trait for *any* type constructor, then implement it once for each container. The pattern `Functor`, `Applicative`, `Monad` from functional programming all require HKT. They describe properties of *containers*, not specific types. "Functor" means "you can map over this container." That applies to `Vec`, `Option`, `Result`, `Future`, `IO`, and infinitely many other type constructors. Without HKT, you'd write `map_vec`, `map_option`, `map_result` separately. With HKT, you write one `map` that works for any `Functor`. - **Functor, Monad, Applicative**: abstract patterns over any container - **Write code once**: works for `Option`, `Result`, `Vec`, `Future`, `IO`, ... - **Foundation of functional programming abstractions** Type inference becomes significantly harder. Combined with features like impredicative polymorphism, it can become undecidable. Languages with HKT typically require explicit annotations. Rust deliberately avoids full HKT (using GATs as a workaround for some cases). Rust doesn't have HKT. Scala 3 does: ```scala // Scala 3: F[_] is a type constructor (kind: Type -> Type) trait Functor[F[_]]: def map[A, B](fa: F[A])(f: A => B): F[B] // Implement for List given Functor[List] with def map[A, B](fa: List[A])(f: A => B): List[B] = fa.map(f) // Implement for Option given Functor[Option] with def map[A, B](fa: Option[A])(f: A => B): Option[B] = fa.map(f) // Now we can write generic code over ANY functor def double[F[_]: Functor](fa: F[Int]): F[Int] = summon[Functor[F]].map(fa)(_ * 2) double(List(1, 2, 3)) // List(2, 4, 6) double(Option(5)) // Some(10) double(Option.empty[Int]) // None // Monad builds on Functor trait Monad[M[_]] extends Functor[M]: def pure[A](a: A): M[A] def flatMap[A, B](ma: M[A])(f: A => M[B]): M[B] // map can be derived from flatMap def map[A, B](fa: M[A])(f: A => B): M[B] = flatMap(fa)(a => pure(f(a))) ``` HKT is standard in Haskell, Scala, and PureScript. Rust avoids full HKT but added GATs (Generic Associated Types) as a partial workaround. If your language doesn't support HKT, don't fight it. Three similar functions are fine if they're short. --- ## Linear and Affine Types Resources must be managed: files closed, memory freed, locks released. Forget to close a file? Leak. Close it twice? Crash. Use it after closing? Undefined behavior. I didn't understand why affine types mattered until I spent three days debugging a double-free in a C++ codebase. The ownership was "obvious" to whoever wrote it six months earlier. Rust would have rejected the code instantly. Garbage collectors handle memory but not files, sockets, or locks. Manual management is error-prone. Microsoft reports that 70% of their security vulnerabilities are memory safety issues, and use-after-free remains a top exploit vector. Can the type system track resource usage? Most type systems only track *what* a value is. Linear types also track *how many times* it's used. This is the **substructural** family, named because they restrict the structural rules of logic (weakening, contraction, exchange): | Type | Rule | Structural Rule Restricted | Use Case | |------|------|---------------------------|----------| | Unrestricted | Any number of times | None | Normal values | | Affine | At most once | Contraction (no duplication) | Rust ownership, can drop unused | | Linear | Exactly once | Contraction + Weakening | Must handle, can't forget | | Relevant | At least once | Weakening (no discarding) | Must use, can duplicate | | Ordered | Exactly once, in order | Contraction + Weakening + Exchange | Stack disciplines | **Ordered types** are the most restrictive: values must be used exactly once and in LIFO order. They model stack-based resources where you can't reorder operations. Rust uses **affine types**: values are used at most once (moved), but you can drop them without using them. True **linear types** require using values exactly once. You can't forget to handle something. "Use" includes transferring ownership. When you pass a `String` to a function that takes it by value, you've "used" the String. It's gone from your scope. You can't use it again. The borrow checker tracks ownership and prevents use-after-move. Borrowing (`&T` and `&mut T`) is how Rust escapes the "use once" restriction when you need it. A borrow doesn't consume the value; it temporarily lends access. The original owner keeps ownership and can use the value after the borrow ends. The borrow checker ensures borrows don't outlive the owner. - **Memory safety without GC**: no runtime overhead, no pauses - **Resource safety**: can't forget to close files - **Prevent use-after-free**: type system rejects it - **No data races**: ownership prevents shared mutable state ```rust // Rust: Affine types (values used at most once) fn consume(s: String) { println!("{}", s); } // s dropped here fn main() { let s = String::from("hello"); consume(s); // s moved into consume // println!("{}", s); // ERROR: borrow of moved value: `s` } // File handles: RAII through ownership use std::fs::File; use std::io::Read; fn read_file() -> std::io::Result { let mut file = File::open("data.txt")?; let mut contents = String::new(); file.read_to_string(&mut contents)?; Ok(contents) } // file automatically closed here (Drop trait) // Can't use file after it's moved/dropped // Can't forget to close (happens automatically) // Can't close twice (Drop runs exactly once) ``` ```rust // Borrowing: temporarily use without consuming fn print_length(s: &String) { // borrows s println!("Length: {}", s.len()); } // borrow ends, s still valid fn main() { let s = String::from("hello"); print_length(&s); // lend s print_length(&s); // can lend again println!("{}", s); // s still valid } // Mutable borrows: exclusive access fn append_world(s: &mut String) { s.push_str(" world"); } fn main() { let mut s = String::from("hello"); append_world(&mut s); // Only ONE mutable borrow at a time (prevents data races) } ``` The borrow checker takes practice. Some patterns (graphs, doubly-linked lists) fight against it. But once you internalize ownership thinking, most code just works. ### The Broader Family: Ownership, Regions, and Capabilities Linear/affine types are part of a broader family of resource-tracking type systems: | System | What It Tracks | Example | |--------|----------------|---------| | **Linear/Affine** | Usage count (exactly/at most once) | Move semantics | | **Ownership** | Who owns a value | Rust's ownership model | | **Region/Lifetime** | How long a reference is valid | Rust lifetimes (`'a`) | | **Capability** | What permissions a value grants | Object-capability languages | **Ownership types** make the owner explicit in the type. Rust combines ownership with affine types: the owner is responsible for cleanup, and ownership can transfer exactly once. This is more than tracking usage; it's tracking *responsibility*. **Region types** (or **lifetime types**) track the *scope* where a reference is valid. Rust's lifetime annotations (`&'a T`) are region types: they prove references don't outlive the data they point to. ```rust // Rust: lifetimes are region types fn longest<'a>(x: &'a str, y: &'a str) -> &'a str { if x.len() > y.len() { x } else { y } } // The 'a says: the returned reference is valid as long as // BOTH input references are valid. The compiler checks this. ``` **Capability types** encode *permissions*, not just structure. A `ReadCapability` lets you read, while `WriteCapability` lets you write. The type system ensures you can only perform operations you have capabilities for. This is object-capability security expressed in types. These ideas originated in research (region inference in MLKit, capability calculus, Cyclone's safe C) but reached mainstream through Rust. Languages like Vale and Austral explore different points in this design space. --- ## Effect Systems Does this function do I/O? Throw exceptions? Modify global state? In most languages, you can't tell from the signature. A function that *looks* pure might read from the network, crash your program, or modify a global variable. ```java // What does this do? You have to read the implementation. String process(String input) ``` Track what **effects** a function can perform in its type. Pure functions have no effects. `readFile` has an `IO` effect. `throw` has an `Exception` effect. A function `String -> Int` with no effects can only compute on its input. A function `String -> IO Int` might read files, hit the network, or launch missiles. Effects propagate: call `readFile` inside your function, your function now has `IO` too. The compiler tracks this automatically. Some systems also provide **effect handlers**: intercept an effect and provide custom behavior. Instead of performing I/O, you could log what I/O *would* happen. Instead of throwing an exception, you could collect errors. This is like dependency injection, but for effects. You write code using abstract effects, then "handle" them differently in tests versus production. - **Effects visible in signatures**: see at a glance what a function can do - **Purity is provable**: no-effect functions are guaranteed pure - **Effect polymorphism**: generic over what effects are used - **Effect handlers**: programmable control flow, algebraic effects ```haskell // Koka: Effects are part of the type // Pure function: no effects fun pureAdd(x: int, y: int): int x + y // Function with IO and exception effects fun readConfig(path: string): string val contents = read-text-file(path) // io effect if contents.is-empty then throw("Config file is empty") // exn effect contents // Effect polymorphism: map preserves whatever effects f has fun map(xs: list, f: (a) -> e b): e list match xs Nil -> Nil Cons(x, rest) -> Cons(f(x), map(rest, f)) // If f is pure, map is pure // If f has io effect, map has io effect ``` ```haskell // Effect handlers: provide custom interpretations of effects effect ask ctl ask(): a fun program(): ask int val x = ask() val y = ask() x + y // Handle by providing values fun main(): io () // Handle 'ask' by returning 10 each time with handler ctl ask() resume(10) val result = program() // 20 println(result.show) ``` Effect systems are in Koka, Eff, Frank, and Unison. Haskell uses monads as a workaround. Most mainstream languages don't have them, so you can use discipline instead: pure functions in the core, effects at the edges. --- ## Refinement Types Your function divides two numbers. The divisor can't be zero. You add a runtime check: ```rust fn divide(x: i32, y: i32) -> i32 { if y == 0 { panic!("division by zero"); } x / y } ``` But the caller might *know* y is non-zero because it's from a non-empty list length. You're checking unnecessarily. And what if you forget the check somewhere? Attach logical predicates to types. Instead of `Int`, write $\{x : Int \mid x > 0\}$. A refinement type is a base type plus a predicate that values must satisfy. This is a sweet spot between regular types and full dependent types. Regular types distinguish "integer" from "string" but can't distinguish "positive integer" from "negative integer." Dependent types can express almost anything but require proofs. Refinement types let you express common properties (non-null, positive, in bounds) and use automated solvers to verify them. The compiler uses an **SMT solver** (Satisfiability Modulo Theories) to verify predicates at compile time. SMT solvers are automated theorem provers that can handle arithmetic, bit vectors, arrays, and more. When you write `divide(x, y)` where `y` must be positive, the solver checks whether `y > 0` is provable from the context. If `y` came from a list length, and lists are non-empty, the solver can prove this automatically. Division by zero becomes a *type error*, caught before running. Buffer overflows too. Array index out of bounds. Integer overflow. These become compile-time checks when you add the right refinements. - **Prove properties at compile time**: non-zero, positive, in bounds - **Eliminate runtime checks**: when the compiler can prove safety - **Catch errors earlier**: type checker finds the bug, not production - **Lightweight verification**: more than types, less than full proofs ```haskell // F*: Refinement types with dependent types // Natural numbers: ints >= 0 type nat = x:int{x >= 0} // Positive numbers: ints > 0 type pos = x:int{x > 0} // Division requires positive divisor (not just non-zero!) val divide : int -> pos -> int let divide x y = x / y // This compiles: 5 is provably positive let result = divide 10 5 // This FAILS at compile time: // let bad = divide 10 0 // Error: expected pos, got int literal 0 // This also fails without more info: // let risky (y: int) = divide 10 y // Error: can't prove y > 0 ``` ```haskell // Vectors with length in the type (simple dependent types) val head : #a:Type -> l:list a{length l > 0} -> a let head #a l = List.hd l // This compiles: let first = head [1; 2; 3] // This fails: // let bad = head [] // Error: can't prove length [] > 0 // Safe indexing: index must be less than length val nth : #a:Type -> l:list a -> i:nat{i < length l} -> a // The refinement i < length l guarantees bounds safety ``` The SMT solver can fail or timeout on complex predicates. When it works, it's like magic. When it doesn't, you're debugging why the solver can't prove something you know is true. F*, Dafny, Liquid Haskell, and Ada/SPARK all use this approach. ### When Refinement Types Aren't Enough Refinement types work well for predicates on values: $\{x : Int \mid x > 0\}$, bounds checks, non-nullity, arithmetic constraints. SMT solvers handle these automatically. But they hit walls. The first wall is type-level computation. You want `printf "%d + %d = %d"` to have type `Int -> Int -> Int -> String`. The format string determines the type. This isn't a predicate on a value. It's computing a type from a value. Refinement types can't express this. The second wall is state. Session types need types that change based on what operations you've performed. Refinement types constrain values but can't express "after calling `open()`, the handle is in state Open." For this you need dependent types or linear types. The third wall is induction. SMT solvers are decision procedures for specific theories: linear arithmetic, bit vectors, arrays. They don't do induction. Refinement types can say "this list has length > 0" but struggle with "this vector has length n + m." You can write $\{v : Vec \mid len(v) = len(a) + len(b)\}$, and for simple cases SMT solvers can verify it. But proving it across recursive calls, showing each step preserves the invariant, requires induction the solver can't do. F* sits on the boundary. It has both refinement types (SMT-backed) and full dependent types (proof-backed). You start with refinements and escalate to manual proofs when the solver fails. This is a reasonable mental model: refinement types are dependent types where an automated prover handles the easy cases. If an SMT solver can verify your property in a few seconds, refinement types work. If you need type computation, state tracking, or induction, you've crossed into dependent type territory. --- ## Tier 4: Research Level These concepts are primarily found in research languages and proof assistants. They provide the strongest guarantees but require significant expertise. Understanding them helps even if you never use them directly. ## Dependent Types You want a function that appends two vectors. The result should have length `n + m`. With regular types, you can express "returns a vector" but not "returns a vector whose length is the sum of the inputs." ```rust // Regular types: can't express the length relationship fn append(a: Vec, b: Vec) -> Vec ``` Refinement types help with predicates, but what if types could *compute*? Types can depend on values. `Vector<3, Int>` (a vector of 3 integers) is a different type than `Vector<5, Int>`. These aren't the same type with the same length checked at runtime. They're *different types*. A function expecting a 3-element vector won't accept a 5-element vector, just like a function expecting a String won't accept an Int. Function types can express relationships between inputs and outputs: ``` append : Vector -> Vector -> Vector ``` The return type *computes* from the input types. If you append a 3-element vector to a 5-element vector, you get an 8-element vector. The `n + m` is evaluated at the type level. Types and terms live in the same world. This is the Curry-Howard correspondence in full force. Types are propositions. Programs are proofs. `Vector` is a proposition: "there exists a vector of n elements of type a." Constructing such a vector proves the proposition. A function type `Vector -> Vector` is an implication: "if you give me a proof of n-vector, I'll give you back a proof of n-vector." The payoff: matrix multiplication that's dimensionally checked at compile time. $Matrix\langle n, m \rangle \times Matrix\langle m, p \rangle \to Matrix\langle n, p \rangle$. If dimensions don't match, the code doesn't compile. - **Type checking requires evaluation**: undecidable in general - **Termination checking required**: non-terminating functions break type checking - **Proving is different from programming**: you need to think about why code is correct, not just that it works - **Verbose proofs**: sometimes more proof code than actual code ```haskell -- Idris 2: Dependent types -- Vector indexed by its length data Vect : Nat -> Type -> Type where Nil : Vect 0 a (::) : a -> Vect n a -> Vect (S n) a -- head: ONLY works on non-empty vectors -- Not a runtime check. The TYPE prevents calling on empty. head : Vect (S n) a -> a head (x :: xs) = x -- No case for Nil needed! Vect (S n) can't be Nil. -- The S n pattern means "at least 1" -- append: the type PROVES lengths add append : Vect n a -> Vect m a -> Vect (n + m) a append Nil ys = ys append (x :: xs) ys = x :: append xs ys -- Type-safe matrix multiplication Matrix : Nat -> Nat -> Type -> Type Matrix rows cols a = Vect rows (Vect cols a) -- Dimensions must match, checked at COMPILE TIME matMul : Num a => Matrix n m a -> Matrix m p a -> Matrix n p a -- This won't compile: -- matMul (2x3 matrix) (5x2 matrix) -- Error: expected Matrix 3 p, got Matrix 5 2 ``` ```haskell -- Type-safe printf! -- The format string determines the function's type printf : (fmt : String) -> PrintfType fmt -- printf "%s is %d years old" -- has type: String -> Int -> String -- printf "%d + %d = %d" -- has type: Int -> Int -> Int -> String -- Wrong number/type of arguments = compile error ``` Dependent types are in Idris 2, Agda, Coq, Lean 4, and F*. For most application code, they're overkill. [Refinement types](#refinement-types) or [phantom types](#phantom-types) often suffice. --- ## Communication and Protocol Typing Concurrency introduces problems that go beyond sequential code. Functions have types, but what about *interactions*? Type systems for communication ensure that distributed components agree on protocols, preventing deadlocks and message mismatches at compile time. ### Why Concurrency Needs Types Beyond Functions In sequential code, a function type `A -> B` tells you everything: give an `A`, get a `B`. But concurrent systems have: - **Ordering constraints**: Must send request before receiving response - **Protocol states**: What you can do depends on what happened before - **Multiple parties**: Client, server, and maybe others must agree - **Failure modes**: Deadlock, livelock, message type mismatch Regular function types can't express "after you send X, you must receive Y before sending Z." Protocol violations compile fine but fail at runtime. ### Session Types **Session types** encode communication protocols in channel types. The channel's type *changes* as you use it, tracking protocol state. Distributed systems communicate over channels. Client sends `Request`, server responds with `Response`. But what if the client sends two requests without waiting? Or expects a response that never comes? Protocol violations cause deadlocks or silent failures, discovered only in production. Session types fix this by making channels typed state machines. Start with `!Request.?Response.End`. After sending a request, you have `?Response.End`. After receiving the response, you have `End`. Each operation transforms the type. Using the wrong operation is a type error. Key concept: **duality**. The client's view is the *dual* of the server's view: sends become receives and vice versa. If the client has `!Request.?Response.End`, the server has `?Request.!Response.End`. The types are symmetric. This ensures both sides agree on the protocol, verified at compile time. Well-typed programs can't deadlock. ``` // Session types: Types encode protocols // Notation: // !T = send value of type T // ?T = receive value of type T // . = sequencing // End = session finished // Client's protocol view type BuyerProtocol = !String. // send book title ?Price. // receive price !Bool. // send accept/reject End // Server's view: the DUAL (swap ! and ?) type SellerProtocol = ?String. // receive title !Price. // send price ?Bool. // receive decision End // Implementation (pseudocode) buyer(channel: BuyerProtocol) { send(channel, "Types and Programming Languages"); // channel now has type ?Price.!Bool.End let price = receive(channel); // channel now has type !Bool.End send(channel, price < 100); // channel now has type End close(channel); } // Multiparty session: Three-way protocol global protocol Purchase(Buyer, Seller, Shipper) { item(String) from Buyer to Seller; price(Int) from Seller to Buyer; choice at Buyer { accept: payment(Int) from Buyer to Seller; address(String) from Buyer to Shipper; delivery(Date) from Shipper to Buyer; reject: cancel() from Buyer to Seller; cancel() from Buyer to Shipper; } } ``` Session types are mostly in research: Links, Scribble, and various academic implementations. Few production systems use them directly, but the ideas influence API design. ### Actor Message Typing **Actor systems** (Erlang, Akka, Orleans) use message passing instead of shared memory. Each actor has a mailbox and processes messages sequentially. But what messages can an actor receive? Without typing, any message can be sent to any actor. Typos in message names, wrong payload types, or protocol violations surface only at runtime. **Typed actors** constrain what messages an actor can receive: ```scala // Akka Typed: Actor's message type is explicit object Counter { sealed trait Command case class Increment(replyTo: ActorRef[Int]) extends Command case class GetValue(replyTo: ActorRef[Int]) extends Command } // The actor can ONLY receive Counter.Command messages def counter(value: Int): Behavior[Counter.Command] = Behaviors.receive { (context, message) => message match { case Increment(replyTo) => replyTo ! (value + 1) counter(value + 1) case GetValue(replyTo) => replyTo ! value Behaviors.same } } // Sending wrong message type = compile error // counterRef ! "hello" // ERROR: String is not Counter.Command ``` ```erlang %% Erlang: Dialyzer can check message types via specs -spec loop(state()) -> no_return(). loop(State) -> receive {increment, From} -> From ! {ok, State + 1}, loop(State + 1); {get, From} -> From ! {ok, State}, loop(State) end. ``` ### Comparing Approaches | Approach | What's Typed | Guarantees | Examples | |----------|-------------|------------|----------| | **Untyped channels** | Nothing | None | Raw sockets, most languages | | **Typed messages** | Message payload types | No wrong payloads | Go channels, Rust mpsc | | **Actor behavior types** | What actor accepts | No invalid messages | Akka Typed, Pony | | **Session types** | Protocol state machine | No protocol violations | Links, research | | **Multiparty session** | N-party protocols | Global protocol safety | Scribble, research | ### Practical Adoption Rust's `Send` and `Sync` traits are a lightweight form of concurrency typing: they mark which types can safely cross thread boundaries. This isn't protocol typing, but it prevents data races at compile time. Go's typed channels (`chan int`, `chan Message`) ensure payload types match but don't track protocol state. Full session types remain mostly academic, but the ideas are seeping into practice. TypeScript's discriminated unions with exhaustive matching approximate protocol states. Rust's typestate pattern uses the type system to enforce valid sequences of operations. --- ## Quantitative Type Theory (QTT) Linear types track usage (use exactly once). Dependent types need to inspect values at the type level. But inspecting a value for typing shouldn't count as "using" it at runtime! ```haskell -- We want the length n to be: -- - Available at compile time (for type checking) -- - Erased at runtime (zero cost) data Vect : Nat -> Type -> Type ``` How do you combine linear/affine types with dependent types cleanly? Annotate each variable with a **quantity** from a semiring: - **0**: compile-time only (erased at runtime) - **1**: exactly once (linear) - **ω**: unlimited The key problem this solves: in dependent types, type-checking might *use* a value to determine a type, but that "use" shouldn't count at runtime. The length `n` in `Vect n a` is used at the type level to ensure vectors have the right size. But at runtime, you don't want to pass `n` around. It should be erased. With QTT, you write `(0 n : Nat)` to say "n exists for type-checking but has zero runtime representation." The `0` quantity means "used zero times at runtime." The type checker uses it. The compiled code doesn't include it. This also cleanly handles linear resources. A file handle has quantity 1: use it exactly once. A normal integer has quantity ω: use it as many times as you want. The quantities form a semiring, which makes them compose correctly when you combine functions. ```haskell -- Idris 2 uses QTT natively -- The 'n' has quantity 0: erased at runtime! data Vect : (0 n : Nat) -> Type -> Type where Nil : Vect 0 a (::) : a -> Vect n a -> Vect (S n) a -- n is available for type checking but has zero runtime cost -- Linear function: use x exactly once dup : (1 x : a) -> (a, a) -- ERROR: can't use x twice! -- Valid linear function consume : (1 x : File) -> IO () -- Unrestricted normal : (x : Int) -> Int normal x = x + x -- Fine, x is unrestricted (quantity ω) -- Mixing: erased type, linear value id : (0 a : Type) -> (1 x : a) -> a id _ x = x -- a exists only at compile time -- x is used exactly once at runtime ``` Idris 2 uses QTT. Granule is a research language exploring graded types more generally. --- ## Cubical Type Theory Homotopy Type Theory (HoTT) introduced revolutionary ideas: types as spaces, equality as paths. The **univalence axiom** says equivalent types are equal. But it was just an axiom that didn't compute. Asking "are these two proofs of equality the same?" got no answer. Make equality *computational*. In standard type theory, you can prove two things are equal, but you can't always *compute* with that equality. Univalence (equivalent types are equal) was an axiom: you could assert it, but it didn't reduce to anything. Asking "is this proof of equality the same as that one?" might not give an answer. Cubical type theory fixes this by taking homotopy seriously. A proof of equality `a = b` is literally a path from `a` to `b`. Formally, it's a function from the interval type `I` (representing [0,1]) to the type, where the function maps 0 to `a` and 1 to `b`. You can walk along the path. You can reverse it (symmetry). You can concatenate paths (transitivity). This geometric intuition makes equality computational. Univalence becomes a theorem: given an equivalence between types, you can construct a path between them. And crucially, transporting values along this path actually *applies* the equivalence. Everything reduces. Everything computes. You also get functional extensionality (functions equal if they agree on all inputs) and higher inductive types (quotients, circles, spheres as types) for free. ```haskell -- Cubical Agda {-# OPTIONS --cubical #-} open import Cubical.Core.Everything -- I is the interval type: points from 0 to 1 -- A path from a to b is a function I → A -- where i0 ↦ a and i1 ↦ b -- Reflexivity: constant path refl : ∀ {A : Type} {a : A} → a ≡ a refl {a = a} = λ i → a -- For all points, return a -- Symmetry: reverse the path sym : ∀ {A : Type} {a b : A} → a ≡ b → b ≡ a sym p = λ i → p (~ i) -- ~ negates interval points -- Function extensionality: just works! -- If f x ≡ g x for all x, then f ≡ g funExt : ∀ {A B : Type} {f g : A → B} → (∀ x → f x ≡ g x) → f ≡ g funExt p = λ i x → p x i -- Univalence: equivalences give paths between types ua : ∀ {A B : Type} → A ≃ B → A ≡ B -- And this COMPUTES: transporting along ua -- actually applies the equivalence! ``` Cubical Agda, redtt, cooltt, and Arend implement cubical type theory. Unless you're doing research in type theory or formalizing mathematics, you won't need this. --- ## Separation Logic Types You're writing code with pointers. How do you know two pointers don't alias? That modifying `*x` won't affect `*y`? In C, you don't. It's undefined behavior waiting to happen. ```c void swap(int *x, int *y) { int tmp = *x; *x = *y; *y = tmp; } // If x == y this becomes a no-op. // The deeper problem is that aliasing makes pointer-manipulating code // much harder to reason about in general. ``` Reason about **ownership of heap regions**. The key operator is **separating conjunction** (`*`): `P * Q` means "P holds for some heap region, Q holds for a *separate* region." If you prove you own separate regions, they can't alias. Classical logic has conjunction (∧): "P and Q are both true." Separation logic adds a new conjunction (*): "P holds for part of memory, Q holds for a *different* part of memory, and these parts don't overlap." This is the missing piece for reasoning about pointers. When you write `{x ↦ 5 * y ↦ 10}`, you're asserting: x points to 5, y points to 10, *and x and y are different locations*. The separating conjunction makes non-aliasing explicit. Without it, modifying `*x` might affect `*y`. With it, you know they're independent. The **frame rule** makes proofs modular. If you prove `{P} code {Q}` (running code in state P yields state Q), then `{P * R} code {Q * R}` for any R. Whatever R describes is *framed out*, untouched by code. You can reason about each piece of memory independently. Rust's borrow checker embodies these ideas. Mutable borrows are exclusive ownership of a memory region. The guarantee that you can't have two `&mut` to the same location is the separating conjunction at work. Concurrent separation logic extends this to reason about shared-memory concurrency. ``` // Separation logic specifications (pseudocode) // Points-to assertion: x points to value v x ↦ v // Separating conjunction: DISJOINT ownership // x ↦ a * y ↦ b means x and y are different locations {x ↦ a * y ↦ b} // precondition: x points to a, y points to b, SEPARATELY swap(x, y) {x ↦ b * y ↦ a} // postcondition: values swapped // The * GUARANTEES x ≠ y // Without separation: aliasing can invalidate the proof obligation // Frame rule: what you don't touch, stays the same // If: {P} code {Q} // Then: {P * R} code {Q * R} // R is "framed out", untouched by code // Linked list segment from head to tail lseg(head, tail) = (head = tail ∧ emp) // empty segment ∨ (∃v, next. head ↦ (v, next) * lseg(next, tail)) // node + rest ``` ```rust // Rust's borrow checker encodes similar ideas fn swap(x: &mut i32, y: &mut i32) { // Rust GUARANTEES x and y don't alias // Can't have two &mut to the same location! let tmp = *x; *x = *y; *y = tmp; } // This won't compile: // let mut n = 5; // swap(&mut n, &mut n); // Error: can't borrow n mutably twice ``` You get separation logic ideas implicitly through Rust's borrow checker. For explicit proofs, tools like Iris (Coq), Viper, and VeriFast let you verify pointer-manipulating code. --- ## Sized Types Dependent type systems need to know all functions terminate. Otherwise type checking could loop forever. Typically they require **structural recursion**: arguments must get smaller in a syntactic sense. But this rejects valid programs: ``` merge : Stream → Stream → Stream merge (x:xs) (y:ys) = x : y : merge xs ys ``` Neither `xs` nor `ys` is structurally smaller than both original arguments! Track *sizes* abstractly in types. A `Stream` has "size" `i`. Operations might not be syntactically smaller but are *semantically* smaller in size. The type checker tracks sizes symbolically. The problem is termination checking. Dependent type checkers must ensure all functions terminate, otherwise type-checking could loop forever. Simple structural recursion ("the argument gets smaller") works for many cases but rejects valid programs. Consider merging two streams. At each step, you take one element from each stream. Neither stream is "structurally smaller" than both inputs. But semantically, you're making progress: you're consuming both streams. Sized types capture this. Each stream has an abstract size. After taking an element, the remaining stream has a smaller size. The type checker sees sizes decreasing and accepts the function. For coinductive data (infinite structures like streams), you need **productivity checking**: you must produce output in finite time. Sized types handle this too. The output stream's size depends on the input sizes in a way that guarantees you always make progress. ```haskell {-# OPTIONS --sized-types #-} open import Size -- Stream indexed by size data Stream (i : Size) (A : Set) : Set where _∷_ : A → Thunk (Stream i) A → Stream (↑ i) A -- ↑ i means "larger than i" -- Thunk delays evaluation (coinduction) -- take: consume part of a sized stream take : ∀ {i A} → Nat → Stream i A → List A take zero _ = [] take (suc n) (x ∷ xs) = x ∷ take n (force xs) -- map preserves size map : ∀ {i A B} → (A → B) → Stream i A → Stream i B map f (x ∷ xs) = f x ∷ λ where .force → map f (force xs) -- merge: interleave two streams -- Both streams get "used", sizes track this correctly zipWith : ∀ {i A B C} → (A → B → C) → Stream i A → Stream i B → Stream i C zipWith f (x ∷ xs) (y ∷ ys) = f x y ∷ λ where .force → zipWith f (force xs) (force ys) -- Without sized types, the termination checker might reject these -- because it can't see that streams are being consumed productively ``` Agda supports sized types. They're useful when the termination checker is too strict, particularly for coinductive definitions. --- ## Pure Type Systems There are many typed lambda calculi: simply typed, System F, System Fω, the Calculus of Constructions, Martin-Löf type theory. Each has its own rules for what can depend on what. Is there a unified framework? **Pure Type Systems** (PTS) provide a single parameterized framework that encompasses most typed lambda calculi. A PTS is defined by three sets: - **Sorts** ($\mathcal{S}$): The "types of types." Typically $*$ (the type of ordinary types) and $\square$ (the type of $*$ itself) - **Axioms** ($\mathcal{A}$): Which sorts have which sorts as their type (e.g., $* : \square$) - **Rules** ($\mathcal{R}$): Triples $(s_1, s_2, s_3)$ specifying that functions from $s_1$ to $s_2$ live in $s_3$ By varying these parameters, you recover different type systems: | System | Rules | What It Expresses | |--------|-------|-------------------| | Simply Typed $\lambda$-calculus | $(*, *, *)$ | Terms depending on terms | | System F | $(*, *, *), (\square, *, *)$ | Types depending on types (polymorphism) | | System F$\omega$ | $(*, *, *), (\square, *, *), (\square, \square, \square)$ | Higher-kinded types | | $\lambda P$ (LF) | $(*, *, *), (*, \square, \square)$ | Types depending on terms (dependent types) | | Calculus of Constructions | All four rule combinations | Full dependent types + polymorphism | The **Lambda Cube** visualizes this: three axes representing term-to-term, type-to-type, and term-to-type abstraction. Each corner is a different type system. ``` λC (CoC) /| / | / | λPω λP2 /| /| / | / | / | / | λω λP System F | | / | | / | | / λ→ (Simply Typed) ``` ### Why It Matters PTS provides: - **Unified theory**: Understand all these systems as instances of one framework - **Metatheoretic results**: Prove properties (normalization, type preservation) once, apply everywhere - **Design guidance**: When designing a type system, you're choosing a point in this space - **Implementation reuse**: Type checkers can be parameterized by PTS specification The Calculus of Constructions (top corner) is the basis for Coq. Martin-Löf Type Theory (related but distinct) underlies Agda. Understanding PTS clarifies what dependent types *are*: the ability to form types that depend on terms, placed on equal footing with other forms of abstraction. ### Connection to Practice When you write `Vector` in a dependently typed language, you're using term-to-type dependency: the type `Vector` depends on the term `n`. This is the λP axis of the Lambda Cube. When you write `forall T. T -> T`, you're using type-to-term polymorphism: the System F corner. Modern dependently typed languages live near the CoC corner, with various additions (universes, inductive types, effects) that go beyond the pure PTS framework but are still understood through it. ### Further Reading - "Lambda Calculi with Types" by Henk Barendregt (the definitive reference) - "Type Theory and Formal Proof" by Rob Nederpelt and Herman Geuvers --- ## Tier 5: Cutting Edge Research These concepts are at the research frontier. They haven't reached mainstream languages yet, but they influence future designs. Brief coverage for completeness: | Concept | What It Explores | Why It Matters | |---------|------------------|----------------| | **Graded Modal Types** | Unify effects + linearity in one framework | Single system for many features | | **Call-by-Push-Value** | Unify call-by-name and call-by-value | Cleaner operational semantics | | **Polarized Types** | Positive (data) vs. negative (codata) types | Better duality understanding | | **Ornaments** | Systematically derive related types | Auto-generate `List` from `Nat` | | **Type-Level Generic Programming** | Reflect on type structure | Auto-derive instances | | **Logical Relations** | Prove program equivalence | Foundation for verification | | **Realizability** | Extract programs from proofs | Programs from math automatically | | **Observational Type Theory** | Equality without axioms | Computation + extensionality | | **Two-Level Type Theory** | Separate meta from object level | Clean staging/metaprogramming | | **Multimodal Type Theory** | Multiple modalities (necessity, etc.) | Generalize many features | ### Graded Modal Types (Brief Example) ```haskell -- Granule: grades unify linearity and effects id : forall {a : Type} . a [1] -> a -- use exactly once id [x] = x dup : forall {a : Type} . a [2] -> (a, a) -- use exactly twice dup [x] = (x, x) -- Grades form a semiring, combining naturally -- One system handles linearity, privacy, information flow... ``` --- ## Practical Concepts A few concepts that don't fit the tier structure but are practically important: ## Variance When `JsonResponse <: HttpResponse`, what's the relationship between `List` and `List`? It depends on how the container uses its type parameter. This question matters for every generic type. You might expect `List` to be a subtype of `List` always. But that's wrong in general, and understanding why is key to writing correct generic code. The intuition: if you can only *read* from a container (produce), then `List` can substitute for `List`. You asked for HTTP responses, I give you JSON responses, JSON responses are HTTP responses, everyone's happy. But if you can *write* to a container (consume), it's the reverse. A container that accepts any HTTP response can accept JSON responses. But a container that only accepts JSON responses can't substitute for one that accepts any HTTP response, because someone might try to put an XML response in it. Mutable containers are the problem case. You can both read and write. Neither subtyping direction is safe. Java's decision to make arrays covariant was a mistake we're still paying for. You can put an Integer into a Number[] that's actually a Double[] at runtime, and it explodes. ```typescript // TypeScript: variance annotations // Covariant (out): Producer <: Producer interface ResponseSource { fetch(): T; } // If it produces JsonResponses, it produces HttpResponses // Contravariant (in): Handler <: Handler interface ResponseHandler { handle(x: T): void; } // If it handles any HttpResponse, it can handle JsonResponses // Invariant: no subtyping relationship interface ResponseCache { get(): T; // covariant use store(x: T): void; // contravariant use } // Both uses = invariant (no safe subtyping) ``` --- ## Phantom Types Type parameters that appear in the type but not in the data. Used for compile-time distinctions. At first, this sounds pointless. Why have a type parameter that doesn't affect the data? The answer: to carry information at the type level that the compiler checks, even though the runtime doesn't need it. Consider a `UserId` and a `ProductId`. Both are just integers at runtime. But mixing them up is a bug. With phantom types, `Id` and `Id` are different types, even though both hold a single integer. The phantom parameter (`User` or `Product`) exists only for the type checker. Zero runtime cost. Full compile-time safety. The Mars Climate Orbiter (1999) was lost because one team used metric units while another used imperial, and 327 million dollars burned up in the Martian atmosphere. Phantom types turn unit mismatches into compile errors: `Distance` and `Distance` can't be mixed. ```rust use std::marker::PhantomData; // Unit types (no data, just type-level tags) struct Meters; struct Feet; // Distance carries a unit, but only at type level struct Distance { value: f64, _unit: PhantomData, // zero runtime cost } impl Distance { fn new(value: f64) -> Self { Distance { value, _unit: PhantomData } } } // Can only add distances with the same unit fn add(a: Distance, b: Distance) -> Distance { Distance::new(a.value + b.value) } let meters: Distance = Distance::new(100.0); let feet: Distance = Distance::new(50.0); // add(meters, feet); // ERROR: expected Meters, got Feet add(meters, Distance::new(50.0)); // OK: both Meters ``` --- ## Row Polymorphism Functions that work on records with "at least these fields," preserving other fields. Regular generics abstract over types. Row polymorphism abstracts over *record structure*. A function `getName` needs records with a `name` field. It shouldn't care about other fields. Row polymorphism lets you write this: "give me any record with at least a `name: String` field, and I'll return the name." Extra fields pass through unchanged. If you have `{ name: "Ada", age: 36, title: "Countess" }` and call `getName`, you get "Ada" back. The function ignores `age` and `title`, but doesn't require you to strip them first. More flexible than structural subtyping because it's parametric: works uniformly for any extra fields. This is common in functional languages with records (PureScript, Elm, OCaml) and solves the problem of writing functions that operate on "records with certain fields" without committing to a specific record type. ```purescript -- PureScript: Row polymorphism -- Works on ANY record with a name field -- The | r means "and possibly other fields" getName :: forall r. { name :: String | r } -> String getName rec = rec.name -- Preserves extra fields! getName { name: "Ada", age: 36 } -- "Ada" getName { name: "Alan", email: "a@b.c" } -- "Alan" -- Can require multiple fields greet :: forall r. { name :: String, title :: String | r } -> String greet rec = rec.title <> " " <> rec.name greet { name: "Lovelace", title: "Countess", birth: 1815 } -- "Countess Lovelace" -- The 'birth' field passes through, ignored but preserved ``` --- ## Languages Compared Rather than ranking languages linearly, this section maps popular languages across the taxonomy axes. Real languages are bundles of trade-offs. ## Comparison Tables ### Core Type System | Language | Checking | Discipline | Polymorphism | |----------|----------|------------|--------------| | **Rust** | Static | Nominal | Parametric + traits | | **Haskell** | Static | Nominal | Parametric + typeclasses | | **OCaml** | Static | Nominal + structural | Parametric + modules | | **Scala** | Static | Nominal | Parametric + implicits | | **TypeScript** | Gradual | Structural | Parametric + unions | | **Python** | Dynamic | Nominal + protocols | Runtime ad-hoc | | **Java** | Static | Nominal | Parametric (erased) | | **C#** | Static | Nominal | Parametric | | **Go** | Static | Structural | Parametric + interfaces | | **Kotlin** | Static | Nominal | Parametric + reified | | **C++** | Static | Nominal | Templates | | **Lean/Coq** | Static | Dependent | Full dependent | ### Advanced Features | Language | Inference | Linearity | Effects | Soundness | |----------|-----------|-----------|---------|-----------| | **Rust** | Bidirectional | Affine + lifetimes | Via types | Sound | | **Haskell** | HM extended | Optional linear | Monads | Mostly sound | | **OCaml** | HM | None | Algebraic | Sound | | **Scala** | Bidirectional | None | Library | Edges unsound | | **TypeScript** | Constraint | None | None | Unsound* | | **Python** | Minimal | None | None | Unsound | | **Java** | Local | None | None | Mostly sound | | **C#** | Local | None | None | Sound | | **Go** | Local | None | None | Sound | | **Kotlin** | Local | None | None | Sound | | **C++** | Minimal | Manual/move | None | Easy to break | | **Lean/Coq** | Bidirectional | None | Pure | Sound | *TypeScript is intentionally unsound for pragmatic reasons. ## Language Profiles ### Rust Ownership and affine typing for systems safety. Rust's type system is built around *resource management*. Affine types (values used at most once) combine with the borrow checker to eliminate use-after-free, data races, and resource leaks at compile time. Lifetimes are region types that prove references don't outlive their referents. Trade-offs: No garbage collector means some patterns (cyclic structures) require workarounds. Expect to fight the borrow checker for a few weeks before it clicks. But for systems code, the safety guarantees are unmatched outside research languages. Best for: Systems programming, performance-critical applications, anywhere memory safety matters. --- ### Haskell Parametric polymorphism plus effect encoding. Haskell pioneered typeclasses (ad-hoc polymorphism without inheritance) and proved that effect tracking via monads works at scale. The type system supports higher-kinded types, GADTs, type families, and with extensions, approaches dependent types. Trade-offs: Complexity accumulates. Extensions interact in surprising ways. Lazy evaluation complicates reasoning about performance. Productive Haskell requires internalizing concepts that don't transfer from imperative languages. Best for: Compilers, financial systems, anywhere correctness matters more than onboarding speed. --- ### OCaml Pragmatic functional programming with sound foundations. OCaml keeps Hindley-Milner inference simple while adding modules with structural typing. The module system enables abstraction and separate compilation. OCaml 5 added algebraic effects, bringing first-class effect handling. Trade-offs: Less expressive than Haskell, fewer libraries than mainstream languages. But the simplicity is intentional: the type system stays predictable. Best for: Compilers (including Rust's original), theorem provers, DSL implementation. --- ### Scala Maximum expressiveness on the JVM. Scala pushes the boundaries of what's expressible in a statically typed language: path-dependent types, implicits for type-level computation, union and intersection types. Scala 3 cleans up the syntax while adding match types and explicit term inference. Trade-offs: The expressiveness creates complexity. Compile times suffer. Some corners are unsound. The type system can be "too powerful" for teams that don't need it. Best for: Complex domain modeling, big data (Spark), anywhere you need JVM compatibility with advanced types. --- ### TypeScript Structural gradual typing with strong flow sensitivity. TypeScript chose structural typing to model JavaScript's duck typing, and gradual typing to enable incremental adoption. Its flow-sensitive type narrowing is among the best: the type of a variable changes based on control flow. Union types and discriminated unions bring algebraic data types to JavaScript. Trade-offs: Intentionally unsound in several places (bivariant function parameters, type assertions). The goal is usability and tooling, not proofs. `any` is always an escape hatch. Best for: Large JavaScript codebases, teams migrating from untyped to typed, frontend development. --- ### Python Runtime flexibility with optional static hints. Python's type system is bolted on: the runtime ignores type hints entirely. Tools like mypy and pyright check them statically. This enables gradual adoption but means types are advisory, not enforced. Trade-offs: No runtime guarantees. Type coverage varies across the ecosystem. But the flexibility is intentional: Python prioritizes "getting things done" over proving correctness. Best for: Scripting, data science, rapid prototyping, anywhere development speed trumps runtime safety. --- ### Java Nominal enterprise typing with conservative evolution. Java's generics use type erasure for backward compatibility, limiting what's expressible. The type system is nominal: explicit declarations define relationships. Evolution is slow and deliberate. Trade-offs: Verbose. Limited inference. No value types (until Valhalla). But stability and backward compatibility matter for enterprise software. Code written in 2004 still compiles. Best for: Enterprise systems, Android development, anywhere long-term stability matters. --- ### C# Pragmatic nominal typing with steady evolution. C# evolves faster than Java, adding features like nullable reference types (flow-sensitive null tracking), pattern matching, and records. The type system is nominal but increasingly expressive. Trade-offs: Windows-centric history (though .NET Core is cross-platform). Less expressive than Scala or Haskell. But the evolution is pragmatic: features that work in enterprise settings. Best for: Windows development, game development (Unity), enterprise .NET systems. --- ### Go Structural minimalism. Go deliberately limits the type system. Interfaces are structural (implement by having the methods), generics were added reluctantly. The philosophy: simple tools for simple problems. Trade-offs: Lack of expressiveness means repetitive code. No sum types means error handling via multiple returns. But the simplicity aids onboarding and tooling. Best for: Cloud infrastructure, CLI tools, services where simplicity aids maintenance. --- ### Kotlin Null safety and smart casts baked into the type system. Kotlin treats nullable types as first-class: `String` is non-null, `String?` is nullable, and the compiler forces you to handle the difference. Smart casts narrow types after checks automatically. If you wrote `if (x is String)`, the compiler knows `x` is a `String` inside that branch without a cast. Combined with sealed classes (sum types), data classes, and coroutines, Kotlin is what Java would look like if designed today. Trade-offs: Still JVM-bound for most use cases (Kotlin/Native and Kotlin/JS exist but are less mature). The language surface keeps growing. But for Android and server-side JVM work, it's a strict upgrade over Java's type system. Best for: Android development, JVM server-side code, anywhere you want Java interop with a modern type system. --- ### C++ Unchecked power. C++ templates are Turing-complete, enabling extreme metaprogramming. Move semantics approximate affine types but aren't enforced. The type system can express almost anything but guarantees almost nothing. Trade-offs: Easy to write undefined behavior. Compile errors are notorious. But when you need zero-overhead abstraction with full control, nothing else competes. Best for: Game engines, embedded systems, performance-critical code where control matters more than safety. --- ### Lean and Coq Types are proofs. These are proof assistants first, programming languages second. Full dependent types mean types can express any mathematical proposition, and programs are proofs of those propositions. Type checking is theorem proving. Trade-offs: Writing proofs is hard. Libraries are limited. But for verified software (CompCert, seL4), they're the gold standard. Best for: Formal verification, mathematics formalization, critical systems requiring proofs. --- ## One-Sentence Summaries | Language | Core Type System Identity | |----------|---------------------------| | Rust | Ownership and affine typing for memory safety | | Haskell | Parametric polymorphism plus monadic effects | | OCaml | Sound HM inference with structural modules | | Scala | Maximum expressiveness on the JVM | | TypeScript | Structural gradual typing with flow sensitivity | | Python | Runtime flexibility with optional static hints | | Java | Conservative nominal enterprise typing | | C# | Pragmatic nominal typing with steady evolution | | Go | Structural minimalism by design | | Kotlin | Null safety and smart casts on the JVM | | C++ | Unchecked power via templates | | Lean/Coq | Dependent types where programs are proofs | --- ## Synthesis ## What Makes Type Systems Hard ### Decidability The more expressive, the harder to check automatically: | Feature | Type Checking | |---------|---------------| | Simply typed | Decidable, linear time | | Hindley-Milner | Decidable, exponential worst case | | System F (rank-N) | Checking decidable, *inference* undecidable | | Dependent types | Undecidable in general (needs termination checking) | ### Inference How much can the compiler figure out without annotations? | Feature | Inference | |---------|-----------| | Local types | Full | | Generics (HM) | Full | | GADTs | Partial (needs annotations at GADT matches) | | Higher-rank | None (requires explicit foralls) | | Dependent | Almost none (proving needs guidance) | ### Type Equality When are two types "the same"? | System | Equality | |--------|----------| | Simple | Syntactic: `Int = Int` | | With aliases | Structural: `type Age = Int`, then `Age = Int` | | Dependent | Computational: must evaluate to compare | | HoTT | Homotopical: paths between types | ### Feature Interaction Features often compose poorly: - **Subtyping + inference**: makes inference much harder - **Dependent types + effects**: need special care (effects in types) - **Linear types + higher-order functions**: subtle ownership tracking - **GADTs + type families**: can make inference unpredictable --- ## Practical Evidence: Do Types Actually Help? Anecdotes claim types catch bugs. But what does the evidence say? ### Empirical Studies | Study | Finding | |-------|---------| | **Hanenberg et al. (2014)** | Static types improved development time for larger tasks but not small ones | | **Mayer et al. (2012)** | Type annotations aided code comprehension, especially for unfamiliar code | | **Gao et al. (2017)** | ~15% of JavaScript bugs in studied projects would have been caught by TypeScript/Flow | | **Ray et al. (2014)** | Languages with stronger type systems correlated with fewer bug-fix commits (GitHub study of 729 projects, though the methodology and effect sizes have been widely debated) | | **Microsoft (2019)** | 70% of security vulnerabilities in their C/C++ code were memory safety issues (addressable by Rust-style types) | The evidence is **mixed but generally positive**: - Types help most for **larger codebases** and **unfamiliar code** - Types help less for **small scripts** where overhead exceeds benefit - **Memory safety types** (Rust) show clearest wins for security-critical code - **Gradual adoption** (TypeScript) shows measurable bug reduction even with partial coverage ### Tooling Impact Type systems enable tooling that untyped languages can't match: | Capability | Enabled By | Example | |------------|-----------|---------| | **Accurate autocomplete** | Type information | IDE knows methods on a variable | | **Safe refactoring** | Type checking | Rename symbol across codebase | | **Go to definition** | Type resolution | Jump to actual implementation | | **Inline documentation** | Type signatures | See parameter/return types | | **Dead code detection** | Exhaustiveness | Unreachable branches flagged | | **Compile-time errors** | Type checking | Catch mistakes before running | Languages like TypeScript transformed JavaScript development primarily through **tooling**, not runtime safety. The types exist largely to power the IDE experience. The sweet spot varies by project. A weekend script doesn't need Rust's borrow checker. A database engine does. --- ## Verification in Practice Dependent types and proof assistants blur the line between programming and mathematics. How are they actually used? ### Real Verified Systems | System | What It Proves | Language/Tool | |--------|---------------|---------------| | **CompCert** | C compiler preserves program semantics | Coq | | **seL4** | Microkernel has no bugs (full functional correctness) | Isabelle/HOL | | **HACL*** | Cryptographic library is correct and side-channel resistant | F* | | **Everest** | Verified HTTPS stack (TLS 1.3) | F*, Dafny, Vale | | **CertiKOS** | Concurrent OS kernel isolation | Coq | | **Iris** | Concurrent separation logic framework | Coq | | **Lean's mathlib** | 200,000+ mathematical declarations | Lean 4 | These are **production systems**, not toys. CompCert is used in aerospace. seL4 runs in military helicopters. HACL* is in Firefox and Linux. ### The Verification Workflow Writing verified code differs from normal programming: ``` 1. SPECIFICATION Write a formal spec of what the code should do (This is often harder than writing the code) 2. IMPLEMENTATION Write the code that implements the spec 3. PROOF Prove the implementation satisfies the spec (Interactive: you guide the prover) (Automated: SMT solver finds proof or fails) 4. EXTRACTION Generate executable code from the verified artifact (Coq → OCaml/Haskell, F* → C/WASM) ``` ### Proof Burden The ratio of proof code to implementation code is sobering: | Project | Implementation | Proof | Ratio | |---------|---------------|-------|-------| | seL4 | ~10K lines C | ~200K lines proof | 20:1 | | CompCert | ~20K lines Coq | ~100K lines Coq | 5:1 | | Typical F* | varies | 2-10x implementation | 2-10:1 | This is why verification is reserved for **critical infrastructure**, not business logic. But the ratio is improving as tools mature. ### Lightweight Verification Full proofs are expensive. Lighter-weight approaches offer partial guarantees: | Approach | What You Get | Cost | |----------|-------------|------| | **Refinement types** (Liquid Haskell) | Prove properties via SMT | Low annotations | | **Property-based testing** (QuickCheck) | Find counterexamples | Write properties | | **Fuzzing** | Find crashes/bugs | CPU time | | **Model checking** | Explore state space | Build model | | **Design by contract** | Runtime checks from specs | Write contracts | Refinement types are the sweet spot for many applications: you get meaningful guarantees (array bounds, non-null, positive) without full proofs. Liquid Haskell and F* make this practical. ### When to Verify | Verify When... | Skip Verification When... | |----------------|---------------------------| | Security-critical (crypto, auth) | Prototype/MVP | | Safety-critical (medical, aerospace) | Business logic | | High-assurance infrastructure | UI code | | Correctness matters more than ship date | Deadline-driven | | Bugs are catastrophically expensive | Bugs are cheap to fix | Most code doesn't need formal verification. But for the code that does, types that can express and check proofs are invaluable. --- ## The Complexity Ranking | Rank | Concept | Learning | Implementing | Worth It For | |------|---------|----------|--------------|--------------| | 1 | ADTs + Pattern Matching | Low | Low | Everyone | | 2 | Generics | Low | Medium | Everyone | | 3 | Traits/Typeclasses | Medium | Medium | Library authors | | 4 | Affine Types (Rust) | Medium | Medium | Systems programmers | | 5 | GADTs | Hard | Medium | DSL/compiler writers | | 6 | HKT | Hard | Hard | FP enthusiasts | | 7 | Effect Systems | Hard | Hard | Language designers | | 8 | Refinement Types | Hard | Hard | Verified software | | 9 | Dependent Types | Very Hard | Very Hard | Researchers, proof engineers | | 10 | Session Types | Very Hard | Very Hard | Protocol verification | | 11 | Cubical/HoTT | Extreme | Extreme | Mathematics, foundations | --- ## What to Learn Based on Your Goals | Your Goal | Focus On | |-----------|----------| | Write better code in any language | ADTs, pattern matching, generics, traits | | Systems programming | Affine types (learn Rust) | | Library design | Generics, traits, associated types | | Functional programming | HKT, typeclasses, effects | | Build compilers/interpreters | GADTs, dependent types basics | | Formal verification | Refinement types, dependent types | | PL research | Everything, including HoTT | --- ## The Future Several trends are reshaping how we think about types: 1. **Effect systems going mainstream**: Unison, Koka showing the way. Expect more languages to track effects. 2. **Refinement types in practical languages**: Lightweight verification becoming accessible. 3. **Linear types spreading**: Rust proved affine types work at scale. Others will follow. 4. **Gradual dependent types**: Getting dependent types into mainstream languages incrementally. 5. **Better tooling**: Type errors becoming clearer. IDE support improving. The UX gap is closing. --- ## Conclusion Type systems exist on a spectrum from "helpful autocomplete" to "machine-checked mathematical proofs." Where you should be on that spectrum depends on what you're building. For most code, Tier 1-2 concepts (ADTs, generics, traits, pattern matching) kill the bugs that waste the most debugging time: null pointer exceptions, forgotten enum cases, type mismatches. They're available in Rust, Scala, Swift, Kotlin, and even TypeScript. Tier 3 concepts (HKT, linear types, effects) require more investment but let you abstract over containers, track resources, and prove purity. Rust's ownership model shows that "hard" concepts can become mainstream when the tooling is right. Tier 4+ concepts (dependent types, session types, HoTT) are mostly for researchers and specialists, but they're where tomorrow's mainstream features come from. Linear types were "research" until Rust. Effect systems might be next. The best investment is understanding the *ideas* over the syntax. Once you grok "make illegal states unrepresentable," you'll apply it in any language. Once you understand why linear types matter, you'll appreciate Rust's borrow checker instead of fighting it. --- ## Appendix: Type System Taxonomy This appendix provides a reference taxonomy of type system dimensions. These concepts are useful for understanding how languages differ, but aren't prerequisites for the main content. ## Hindley-Milner Type Inference This section covers how type inference works under the hood - useful for understanding compiler behavior but not required for using type systems effectively. Static typing traditionally meant annotating everything. Java's infamous: ```java Map> map = new HashMap>(); ``` This verbosity is why many developers fled to dynamic languages. But dynamic typing means discovering type mismatches at runtime, often in production. What if the compiler could *figure out* the types? In 1969, Roger Hindley discovered (and Robin Milner independently rediscovered in 1978) an algorithm that can infer the most general type for any expression in a certain class of type systems, without any annotations. The key observation: even without annotations, code contains type information. If you write `x + 1`, the compiler knows `x` must be a number because `+` requires numbers. If you write `x.len()`, `x` must be something with a `len` method. These constraints propagate through your program. The algorithm works by: 1. Assigning fresh type variables to unknown types (like algebra: let `x` be unknown) 2. Collecting constraints from how values are used (`x + 1` means `x` must be numeric) 3. Unifying constraints to find the most general solution (solving the equations) The "most general" part matters. If you write a function that works on any list, the algorithm infers "list of anything," not "list of integers." You get maximum reusability automatically. - The brevity of Python with the safety of static typing - Write code without type annotations; the compiler figures them out - Catch type errors at compile time, not runtime - The inferred type is always the *most general*, so your function works for all types that fit ```rust // Rust: The compiler infers all types here fn compose(f: impl Fn(B) -> C, g: impl Fn(A) -> B) -> impl Fn(A) -> C { move |x| f(g(x)) } let add_one = |x| x + 1; // inferred: i32 -> i32 let double = |x| x * 2; // inferred: i32 -> i32 let add_one_then_double = compose(double, add_one); // No type annotations needed, compiler infers everything let result = add_one_then_double(5); // 12 ``` ```rust // Even complex generic code needs minimal annotations fn map(items: Vec, f: impl Fn(T) -> U) -> Vec { items.into_iter().map(f).collect() } let numbers = vec![1, 2, 3]; let strings = map(numbers, |n| n.to_string()); // Compiler infers: T = i32, U = String ``` The trade-off: some advanced features ([GADTs](#generalized-algebraic-data-types-gadts), [higher-rank types](#rank-n-polymorphism)) break inference and require annotations. But for everyday code, you get static typing's safety without its traditional verbosity. Available in ML, OCaml, Haskell, Rust, F#, Elm, and Scala. ### Beyond HM: Other Inference Strategies Hindley-Milner is the gold standard for inference in purely functional languages, but other strategies exist: | Strategy | How It Works | Used In | |----------|--------------|---------| | **Bidirectional** | Types flow both up (inference) and down (checking) | Rust, Scala, Agda | | **Constraint-based** | Collect constraints, solve with SMT/unification | Gradual typing, refinement types | | **Local** | Infer within expressions, require declarations at boundaries | Java (var), C++ (auto) | **Bidirectional typing** is particularly important for modern languages. Instead of pure inference (bottom-up) or pure checking (top-down), types flow both ways. When you write `let x: Vec = vec![1, 2, 3]`, the expected type `Vec` flows *down* to help infer the element type. When you write `let x = vec![1, 2, 3]`, the literal types flow *up* to infer `Vec`. This scales better than pure HM to richer type systems. GADTs, higher-rank types, and dependent types all work well with bidirectional typing because explicit annotations guide inference where needed. --- ## Orthogonal Dimensions Type systems are not a linear progression. They combine orthogonal axes independently: ``` CHECKING: Static ← Gradual → Dynamic EQUALITY: Nominal ← → Structural POLYMORPHISM: None → Parametric → Bounded → Higher-Kinded INFERENCE: Explicit → Local → Bidirectional → Full (HM) RESOURCES: Unrestricted → Affine → Linear EFFECTS: Implicit → Monadic → Algebraic ``` Real languages pick a point on each axis. Rust is static + nominal + affine. TypeScript is gradual + structural. Haskell is static + nominal + higher-kinded + monadic. The combinations create the design space. ### 1. Time of Checking | Approach | When Types Checked | Examples | |----------|-------------------|----------| | **Static** | Compile time | Rust, Haskell, Java | | **Dynamic** | Runtime | Python, Ruby, JavaScript | | **Gradual** | Both, with boundaries | TypeScript, Python+mypy | ### 2. Type Equality | Approach | Types Equal When... | Examples | |----------|---------------------|----------| | **Nominal** | Same declared name | Java, Rust, C# | | **Structural** | Same shape/fields | TypeScript, Go interfaces | ### 3. Polymorphism | Kind | What It Abstracts | Examples | |------|-------------------|----------| | **Parametric** | Type variables (`T`) | Generics in all typed languages | | **Ad-hoc** | Different impls per type | Overloading, typeclasses, traits | | **Subtype** | Substitutability | OOP inheritance, structural subtyping | | **Bounded** | Constrained type variables | `T: Ord`, `T extends Comparable` | ### 4. Type Inference | Strategy | How Types Inferred | Examples | |----------|-------------------|----------| | **Hindley-Milner** | Global, principal types | ML, Haskell, OCaml | | **Bidirectional** | Up and down the AST | Rust, Scala, Agda | | **Local** | Within expressions only | Java `var`, C++ `auto` | | **Constraint-based** | Solve constraint systems | TypeScript, gradual systems | ### 5. Predicate Refinement | Level | What Types Express | Examples | |-------|-------------------|----------| | **Simple** | Base types only | Most languages | | **Refinement** | Types + predicates (`{x: Int \| x > 0}`) | Liquid Haskell, F* | | **Dependent** | Types compute from values | Idris, Agda, Lean | ### 6. Substructural / Resource Tracking | Discipline | Usage Rule | Examples | |------------|-----------|----------| | **Unrestricted** | Any number of times | Most languages | | **Affine** | At most once | Rust ownership | | **Linear** | Exactly once | Linear Haskell, research calculi | | **Relevant** | At least once | Research systems | | **Ordered** | Once, in order | Stack disciplines | ### 7. Effect Tracking | Approach | What's Tracked | Examples | |----------|---------------|----------| | **None** | Effects implicit | Java, Python, Go | | **Monadic** | Effects in type wrappers | Haskell IO | | **Algebraic** | First-class effect handlers | Koka, OCaml 5 | ### 8. Flow Sensitivity | Approach | Type Changes With... | Examples | |----------|---------------------|----------| | **Insensitive** | Fixed at declaration | Java, C | | **Sensitive** | Control flow | TypeScript, Kotlin, Rust | ### 9. Concurrency / Communication | Approach | What's Typed | Examples | |----------|-------------|----------| | **Untyped** | No protocol checking | Most languages | | **Marker traits** | Send/Sync capabilities | Rust | | **Session types** | Protocol state machines | Research, Links | --- **Real languages combine these axes.** Rust is static + nominal + parametric + bidirectional + affine + flow-sensitive + marker traits. TypeScript is gradual + structural + parametric + constraint-based + flow-sensitive. There's no single "best" combination; each serves different goals. ## The Expressiveness Map How type systems relate in terms of expressiveness versus annotation burden: ``` EXPRESSIVENESS Low ──────────────────► High │ Simple │ ML Haskell+Exts Inference │ │ │ │ ▼ ▼ │ Rust ────► Rust+GATs │ │ │ │ │ Scala 3 │ │ │ │ │ │ ▼ ▼ ▼ │ OCaml+Mods │ │ │ ▼ Needs │ F*/Lean ◄── Refinements Annotations│ │ │ ▼ │ Idris/Agda ◄── Full Dependent │ │ │ ▼ Proof │ Coq/Lean4 ◄── Proof Assistant Required │ │ │ ▼ │ Cubical ◄── HoTT │ ▼ ANNOTATION BURDEN ``` The further right you go, the more you can express in types. The further down you go, the more work you must do to satisfy the type checker. Where you land depends on what you're building and how much pain you're willing to trade for guarantees. --- ## Dynamic Type Systems In dynamic languages, types exist and are checked, just at runtime rather than compile time. Values carry type tags at runtime. Operations check these tags before executing: ```python ## Python: types checked at runtime def add(a, b): return a + b add(1, 2) # Works: both ints add("a", "b") # Works: both strings add(1, "b") # TypeError at runtime! ``` The type error still happens. It just happens when you run the code, not when you compile it. This trades earlier error detection for flexibility and development speed. Dynamic typing works well for: - **Prototyping and exploration**: When you don't yet know what shape your data will take - **Scripts and glue code**: Short-lived code where development speed matters more than maintenance - **REPLs and interactive development**: Immediate feedback without compilation - **Highly dynamic domains**: Serialization, ORMs, and metaprogramming where static types fight the problem Dynamic typing is "types checked later." The question isn't "static vs dynamic" but "how much static?" Python with type hints, TypeScript with strict mode, Rust with full ownership tracking: these represent different points on a spectrum. Pick the point that matches your problem. Python, Ruby, JavaScript, Lisp, Clojure, Erlang, Elixir. Most have optional type systems now (Python's type hints, TypeScript for JavaScript). ## Gradual Typing Gradual typing blends static and dynamic checking within the same language. You can add types incrementally, and the system inserts runtime checks at the boundaries between typed and untyped code. In a gradually typed system, you can leave parts of your code untyped (using `any` or equivalent) while fully typing other parts. The type checker verifies the typed portions statically. At runtime, checks are inserted where typed code interacts with untyped code. ```typescript // TypeScript: gradual typing in action function greet(name: string): string { return `Hello, ${name}`; } // Fully typed: checked statically greet("Ada"); // OK at compile time // Escape hatch: 'any' bypasses static checking function processUnknown(data: any): void { // No compile-time checking on 'data' console.log(data.someProperty); // Could fail at runtime } // The boundary: where typed meets untyped function fromExternal(json: any): User { // Runtime validation needed here return json as User; // Risky! No guarantee json matches User } ``` The **gradual guarantee** is the formal property that makes this work: adding type annotations should not change program behavior (unless there's a type error). You can migrate from untyped to typed code one function at a time without breaking anything. This enables incremental adoption: 1. Start with a dynamically typed codebase 2. Add types to critical paths first 3. Gradually expand type coverage 4. Runtime checks catch boundary violations ### Blame Tracking When a type error occurs at a boundary, who's at fault? **Blame tracking** attributes errors to the untyped side of the boundary. If typed code calls untyped code and gets a wrong type back, blame falls on the untyped code. ```python ## Python with type hints def typed_function(x: int) -> int: return x + 1 def untyped_function(y): return "not an int" # Bug here ## At runtime, the error is blamed on untyped_function result: int = untyped_function(5) # Runtime TypeError ``` TypeScript, Python (with mypy/pyright), PHP (with Hack), Racket (Typed Racket), Dart (before null safety), C# (with nullable reference types). ## Further Reading **Books:** - "Types and Programming Languages" by Benjamin Pierce, the textbook - "Software Foundations", free online, interactive proof-based introduction - "Programming Language Foundations in Agda", dependent types for programmers - "Type-Driven Development with Idris" by Edwin Brady, the most practical intro to dependent types - "The Little Typer" by Friedman and Christiansen, a gentle Socratic walk through dependent types **Languages to try:** - **Rust**: Best practical introduction to affine types - **Haskell**: HKT, typeclasses, GADTs, the functional programming standard - **Idris 2**: Most accessible dependent types - **Koka**: Clean effect system design **Papers:** - "Propositions as Types" by Philip Wadler, covers the Curry-Howard correspondence - "Theorems for Free" by Philip Wadler, what parametricity guarantees - "Linear Types Can Change the World", why linearity matters --- ### Thank You to the Argentine Senate *Published: 2025-11-12* > The Argentine Senate gave me a Mención de Honor al Valor Científico. Thank you, and a note about who actually earned it. URL: https://federicocarrone.com/articles/senate-scientific-merit/ On 12 November 2025 I received a Mención de Honor al Valor Científico from the Comisión de Ciencia y Tecnología of the Argentine Senate. Thank you. The ceremony was in the Salón Azul of the Congreso Nacional. Most of the time I was sitting there thinking that the other people had done work much heavier than mine. The [full list is on the Senate's site](https://www.senado.gob.ar/micrositios/nota/23423/noticias). Ergodic also wrote [something about my part](https://ergodicgroup.com/blog/federico-carrone-senate-scientific-merit/). Senator Anabel Fernández Sagasti proposed my name. The diploma says: > por su destacada labor, que se distingue por combinar ciencia, ingeniería, seguridad informática, educación y comunidad, transformando investigación de frontera en soluciones concretas que fortalecen el desarrollo nacional In English: > for his outstanding work combining science, engineering, information security, education, and community, transforming frontier research into concrete solutions that strengthen national development Thank you to her. Also to Senator Silvina García Larraburu, president of the commission, and Senator Víctor Zimmermann, its vice president, who handed out the diplomas. [LambdaClass](https://x.com/class_lambda) is not me. [Ergodic](https://ergodicgroup.com) is not me. They are groups of people who decided to spend their lives building things together. Most of them will never receive a diploma with their name on it. The diploma has one name on it. That is not how any of this actually happened. If at some point you built something with me, this one is yours too. Gracias. --- ### Crypto doctrine *Published: 2025-09-25* > Crypto found product-market fit where trust is weakest: inflationary or censored economies, and internet-native communities that need programmable coordination and markets. URL: https://federicocarrone.com/articles/crypto-doctrine/ # Crypto and the accelerated and chaotic 21st Century Crypto has been most useful where trust is weakest. In practice, it has found product-market fit in two places: - In countries where inflation, capital controls, or censorship are ordinary constraints, crypto gives people and companies tools they actually need. - In internet-native communities, crypto provides a financial layer that lets people coordinate, speculate, and build markets at a scale the web did not support before. People that don't live in a developing country or that didn't grow up with the internet have enormous difficulties understanding crypto because they don't have skin in its game. They believe crypto doesn't have any "real" use case or that is not serious enough. They are right. The thing is that we are living in a world that's becoming more absurd. Memes do not only make you laugh anymore, memes are now winning elections. These use cases will grow with time and probably new ones will be found. The world is becoming more chaotic and more divided each day. Crypto benefits from that kind of environment because it reduces the number of places where trust has to be taken on faith. One of crypto's prime advantages is that it [kills many of the middlemen and allows us to coordinate](@/series/ethereum/2025-12-02-the-missing-institution-of-the-internet/index.md) even in the harshest environments. Trust assumptions fall because more of the system is enforced by incentives, compilers, distributed systems, and cryptography. That does not remove politics or disagreement; it just narrows the set of things people need to argue about. Most of us are internet natives. We grew up on IRC, 4chan, Reddit, Hacker News, Twitter, Bitcoin, and Ethereum, and we also have roots in unstable countries. We are the Fremen of crypto, raised in a harsh environment. We know what chaotic societies feel like from the inside, and we know what it takes to build inside them. At the same time, we are builders who like working at the frontier of engineering and scientific change. Open source and decentralization are not just philosophical preferences for crypto. They are practical conditions for the ecosystem to work. Building in the open, helping other people onboard, and creating systems larger than the original project are part of how crypto survives long term. This can look irrational if you assume the only goal is short-term extraction. It is more legible if you assume the goal is to help a new financial and coordination layer persist. Our main objective is to help these new internet highways get built in sustainable ways. Economic sustainability matters, but so do resilience, openness, and the ability to resist the usual drift toward centralization. Centralization is almost always easier in the short run. If pure money were the only objective, there would be simpler ways to make it. We treat money as a tool, not the final point. Whether crypto is useful is already settled. It is, in places most critics never look. The open question is whether the systems being built today will be resilient enough to matter when the next round of chaos arrives. --- ## Series: Concrete Concrete is a systems language built so the compiler can say more than pass or fail. Linear types, explicit capabilities, visible cleanup, source contracts, and Lean-checked proof evidence turn hidden program behavior into facts the compiler can expose. ### Nutrition Labels for Trust *Published: 2026-06-13* > Vitalik Buterin wants trust nutrition labels for software. Concrete shows what the machine-and-math half looks like when the compiler produces it instead of a vendor writing prose. URL: https://federicocarrone.com/series/concrete/nutrition-labels-for-trust/ The man in the painting at the top of this page is doing the oldest verification job there is. He is weighing each coin on a balance, one at a time, because the face stamped on a coin is a claim and its weight is the evidence, and a money changer who confused the two went broke. He does not trust the mint. He trusts the scale. Five hundred years later, almost all of our software asks us to trust the stamp. It ships with a name, a logo, a reassuring sentence about security, and no scale anywhere. The bill for trusting the stamp comes due in supply-chain backdoors, dependencies nobody audited, and "verified" badges that turn out to have meant a marketing review, and it usually arrives late and all at once. Vitalik Buterin put the missing scale into one sentence: > In an ideal world all software and hardware would have "nutrition labels" that provide a full list of trust dependencies - what math and which actors' honest behavior (and on what time scale) the system is relying on to provide its core functionality and implied guarantees. > > [Vitalik Buterin (@VitalikButerin)](https://x.com/VitalikButerin/status/2059622765364207619) binji replied with the obvious objection, and then with something more interesting: > even if this was available, it could still prove to create a cognitive load that is ignored by many, so they'd still opt into systems that preselect for them. you see this in nutrition labels and dieting etc, where most people prefer the convenience of being given a basket of "things to eat" from a verified source (doctors, dieticians…influencers). > > but here's where the agentic world gets interesting, as ai becomes the new ui, the necessity of privacy preserving agents personalized to a users preferences that can handle the cognitive load to supplement their decisions while proving verified logic on how they come to that conclusion will be key. > > feels like a strong intersection opportunity at large here and is a mixture of verification, privacy, agent-assisted decision making, and overall web hygiene > > [binji (@binji_x)](https://x.com/binji_x/status/2059625420828344645) That exchange splits the problem cleanly. Vitalik is asking for the artifact: what does this system depend on? binji is asking who is supposed to read it without turning every user into a security engineer. [Concrete](@/series/concrete/2025-12-26-the-concrete-programming-language-systems-programming-for-formal-reasoning/index.md), the programming language I have been writing about, sits between those two questions. It does not solve the whole thing. It builds the part that should never have been prose. {% toolkit(label="Key takeaways") %} - A trust nutrition label should be a compiler artifact, not vendor prose. Concrete already produces the math-and-code half of one as a byproduct of compiling. - The label has real fields: the capabilities a program uses, every trusted and unsafe boundary, and a per-obligation evidence class (proved, assumed, or trusted) down to a named trusted computing base and the axioms the proofs rest on. - It is verifiable, not asserted. The reports are deterministic and `concrete diff` fails closed when trust weakens between versions. - It deliberately stops at one half. It says nothing about which actors you trust or for how long, and the verification is partial: the backend and final binary stay trusted. The output is "here is what is proven, by what, and what is not." - For agents, that means trust should come from the kernel-checked artifact, not the agent's say-so. {% end %} ## Two Different Kinds Of Trust Vitalik's label is really asking for two lists. One list is technical: which proofs, components, foreign calls, permissions, and machines does this system rely on? The other is social: which people or institutions must behave honestly, and for how long? Those are not the same problem. Concrete works on the first list. It says nothing deep about incentives, collusion, governance, or whether some actor stays honest for six months. But the technical list is already enough work, and it is the part a compiler can actually produce. ## A label generated by the compiler Concrete is built around a simple habit: when the program makes a claim, the compiler should keep the receipt. A contract, a capability, a trusted boundary, a runtime-safety check: each one becomes something the tool can report on. Compilation should leave behind more than a binary. It should leave behind a record of what the program used and what evidence backs each claim. That ledger has real fields, not slogans. A small slice of it might look like this: ```text function: parse_config capabilities: File trusted_boundaries: - trusted extern fn os_read obligations: O1 array_bounds buffer[i] evidence: proved_by_kernel_decision engine: omega O2 ensures result_is_valid_config evidence: assumed O3 proof link Config.Proofs.parse_config_shape evidence: stale reason: body fingerprint changed tcb: - Concrete checker - Lean kernel - proof attachment and fingerprint machinery - LLVM/backend/runtime/OS/hardware ``` The syntax above is illustrative, but the categories are real: authority, trusted boundary, obligation, evidence class, stale proof, trusted base. The label comes from the program and the proof artifacts. Nobody writes it afterward as a compliance paragraph. **Capabilities.** Concrete tracks effects as a visible capability vocabulary: the concrete permissions `File`, `Network`, `Process`, `Console`, `Clock`, `Random`, `Env`, `Alloc`, and `Unsafe`, plus a `Std` macro that expands to the standard set and user-defined aliases that expand at parse time. A function with no annotation is pure. A function that allocates on the heap must say `with(Alloc)`. A function that touches the network must say `with(Network)`, and so must everything that transitively calls it. The label cannot under-report, because a program that uses an effect it did not declare does not compile. **Trusted boundaries.** Every place the program steps outside what the checker can guarantee is marked and locatable: `trusted fn`, `trusted impl`, `trusted extern fn`, and functions or calls carrying `with(Unsafe)`. You can ask the compiler to enumerate them, and it answers with a list and source spans rather than a shrug. "Does pointer tricks internally" and "can call arbitrary foreign code" are different risks, and they get different markers. **Evidence classes.** My favorite detail is the absence of a single green checkmark. Every obligation says how it is justified: `proved_by_lean` for a kernel-checked theorem, `proved_by_kernel_decision` for a decision procedure, `solver_trusted` for an external solver result, `tested_by_oracle`, `assumed`, `trusted`, `stale`, `unproven`, and more. "This type-checks," "this is proven," and "we are trusting the author here" are different statements. Concrete keeps them separate. That is the money changer's distinction restored to software: the coin you weighed, the coin you did not, and the coin you are choosing to take on faith. None of that is pseudocode, so here is the authority half in real Concrete. A foreign call is a named, audited boundary; a function that touches the console has to say so; and the requirement climbs the call graph on its own: ```rust trusted extern fn putchar(c: i32) -> i32; // foreign boundary, audited fn print_int(n: i64) with(Console) { /* ... */ } // needs Console fn greet() with(Console) { print_int(42); } // inherits it from print_int fn main() with(Std) -> Int { greet(); return 0; } ``` Delete `with(Console)` from `greet` and the program stops compiling, because `greet` calls something that needs it. Authority is not a comment that can drift out of date. It is part of the type, rechecked on every edit. ## The label carries proofs, not just declarations A permissions screen says an app can use the network. A software bill of materials says a binary contains some library at some version. Concrete's label can say something a list of dependencies cannot: that a specific property of a specific function has been mechanically proven, and by what. The mechanism is ordinary design by contract, pointed at verification. A function carries `#[requires]` and `#[ensures]` clauses and loop invariants. Each becomes a proof obligation with a stable identifier. A precondition is assumed at the function's entry and surfaced at every caller, so it cannot be quietly dropped; depending on the active policy, the caller either discharges it or carries an explicit unproven obligation. How it gets discharged is where the marketing version would usually start lying. Concrete is kernel-first. The decision procedures `omega`, for linear integer arithmetic, and `bv_decide`, for bitvectors, produce certificates that are checked inside the toolchain, so using them adds no external solver to the trusted base. But the caveats stay visible. `bv_decide` relies on a compiled LRAT checker, which brings a named tier of native-code trust; it is not pure kernel reduction, and the axiom inventory says so. Concrete can also hand a condition to an external SMT solver. When it does, the result is labeled `solver_trusted`, and that solver binary becomes part of the trusted base for that obligation. Trust has not vanished. The label tells you which kind of trust you just used. `concrete prove` is the workflow that makes this usable. It generates a Lean proof workspace for a function, links registered theorems back to their obligations, and supports replay so a proof stays bound to the exact source it was written against. A fingerprint, now a truncated SHA-256 over the function's structure, detects when the code drifts out from under its proof, and the evidence class flips to `stale`. The label cannot keep claiming "proven" about a function that has since changed. Here is the proof half in real code, just as small. A bit rotation whose precondition says the shift must stay in range: ```rust #[requires(0 <= n && n < 32)] fn rotr(x: u32, n: u32) -> u32 { return (x >> n) | (x << (32 - n)); } ``` That `#[requires]` is not a comment and not a runtime assert. The compiler turns it into an obligation, pushes it onto every caller, and then reports, one call site at a time, how each call discharges it: ```text call rotr(x, 13) requires 0 <= n && n < 32 -> proved_at_callsite call rotr(x, n) [n=7] requires 0 <= n && n < 32 -> proved_by_kernel_decision (bv_decide) call rotr(x, 40) requires 0 <= n && n < 32 -> failed_at_callsite call rotr(x, k) requires 0 <= n && n < 32 -> unproven_at_callsite ``` Four calls, four honest verdicts. A constant in range folds to proved. A value fixed earlier by a `let` is handed to a decision procedure and closed with checked evidence. A constant out of range is reported as a violation, and policy can make that a hard failure. An argument the compiler cannot pin down stays `unproven`, labeled exactly that, never quietly rounded up to fine. The last line is the one that matters most: the label would rather tell you it does not know than tell you a comforting lie. That is the money changer setting a coin aside because the scale was inconclusive, instead of waving it through. Return to the config example. The useful fact is not that the program is "verified." It is that its `File` authority is visible, its absence of `Network` and `Alloc` authority is checked, its operating-system boundary is named, one bounds obligation is discharged by `omega`, one semantic parsing claim is only `assumed`, and one old proof has gone `stale`. Six facts, six different kinds of trust, none of them collapsed into a checkmark. You learn more from that than from a green "verified" badge, precisely because it shows you where the badge would have been lying. The word "verified" has to stay disciplined. Capabilities and boundaries are enforced or reported by the type system. That is useful, but it is not the same thing as a proof. A contract that reaches `proved_by_lean` or `proved_by_kernel_decision` has a machine-checked argument behind it. The label keeps those cases apart so the enforced does not pretend to be proven. ## The label includes itself The best part is that Concrete applies the same suspicion to itself. It prints the trusted computing base: the layers you must trust for any proof to mean anything. The checker and compiler. The Lean kernel. The proof-attachment and fingerprint machinery. The LLVM backend. The runtime, the operating system, the hardware. And the foreign code behind every `extern fn`. Most systems hide this list. Concrete prints it. Ken Thompson gave the reason in his 1984 Turing Award lecture. You cannot fully trust code you did not write yourself, and the rot can reach all the way down to the compiler: a compiler can carry a backdoor that survives even after its own source is scrubbed clean, by recognizing when it is compiling itself and quietly reinserting the trick. That does not make trust hopeless. It means "trust me, the compiler is clean" is not an answer. You have to name what trusting the compiler commits you to. A compiler that prints its own trusted base and the axioms its proofs stand on is Thompson's question answered out loud instead of waved away. It even prints the axioms. An axiom-inventory gate runs over every theorem and fails the build on anything undocumented. The mathematical assumptions the proofs are allowed to lean on are named: `propext`, `Classical.choice`, `Quot.sound`, and the flagged native-code trust tier for compiled certificate checking. That is the literal answer to Vitalik's "what math are you relying on," extracted automatically rather than asserted in a README. The label can also be regenerated. Same source, same reports. `concrete diff` compares two versions and flags when trust weakens, when a proof goes stale, when authority escalates, when a boundary erodes. A label you can regenerate and diff is evidence. A label you cannot is marketing. ## What it does not cover Here is the part I would rather say myself than have you catch me on. Concrete does not model the second half of Vitalik's label at all. There is no notion of actors, incentives, collusion, honesty-until-some-time, or social trust anywhere in it. Its accounting is static, about which layers and which math to trust, not dynamic, about which humans behave well and for how long. The actor-and-time-scale half is a real and separate problem, and it belongs to mechanism design and economics, not to a systems language. The verification is also partial, and the label says so. Proofs attach at the contract and proof-model level, over an intermediate representation and an idealized integer model. The chain from there through the backend down to the final binary is trusted, not verified, and binary correctness sits openly among Concrete's explicit non-claims. Many obligations are still `missing` or end in a hand-written Lean proof rather than automatic discharge. So the claim is not "Concrete proves your program correct end to end." It is smaller and more useful: Concrete proves selected claims over its proof model, then tells you which properties are proven, by what, which trusted base they rely on, and which properties are not proven at all. That is worth more than a green badge. ## Who reads the label binji's objection is correct and survives even a perfect label. Labels create cognitive load, most people ignore them, and they fall back on a basket curated by someone they trust. This happens with food labels and diets, and it would happen with trust labels too. A manifest nobody reads is decoration. But his answer points at the kind of artifact Concrete produces. He wants agents that carry the cognitive load while showing verified logic for their conclusions. For that to work, the agent needs structured facts it did not invent. Concrete's label is machine-consumable. It has identifiers, source spans, dependencies, and evidence classes. An agent can read it directly. The cognitive load binji worries about is a problem for a human staring at a wall of facts, not for software filtering those facts against a user's policy. And Concrete's strongest conclusions arrive with proofs the kernel already checked, or with explicitly weaker labels when they do not. binji wants the agent to prove the logic behind its recommendation. With Concrete, the load-bearing proof evidence was checked independently of any agent. The agent does not have to be trusted to produce that evidence. It only has to point at evidence that already exists and that it cannot forge without changing the artifact. ## Trust should come from the artifact, not the agent This changes the role of the agent. The usual story makes the agent the thing you must trust: align it, audit it, believe it. Concrete pushes some trust downward into the artifact. The agent's job is then smaller. It reads facts it cannot easily fake and applies the user's policy to them. The split is simple. Concrete produces the verified input. The agent applies the user's preferences. The Lean kernel anchors the strongest evidence. The remaining trusted layers are named instead of hidden. The agent is still not magic, but at least it is reading facts grounded outside itself. One more line so I do not oversell it. Concrete answers the input problem: trustworthy facts about an artifact. It does not answer the alignment problem: whether the agent faithfully serves the user. It can make the agent's inputs harder to fake. It does not make the agent good. ## The ingredients, not just the dish Everything above labels a program you wrote. But the trust dependency that actually bites is the one you did not write: the parser that quietly starts logging to disk, the hash helper that adds a network call "for telemetry," the dependency whose proof silently downgrades between versions. Vitalik's phrase is "a full list of trust dependencies," and in practice your dependencies are your imports. binji's basket of ingredients is the import list. Concrete's design notes take the next step, and I want to be exact: this part is written down as a direction, not yet shipped. The principle is that an import should not silently grant power. It should say what it brings in and what it is forbidden to bring in. So an import carries a ceiling and a floor: ```rust import std.parse requires(no File, no Network, no Unsafe) import hmac.compute requires(proved_by_lean) import crypto.compare requires(constant_time, no secret_sink) ``` and a manifest sets a whole-project authority budget: ```toml [authority] allowed = ["Alloc"] forbidden = ["File", "Network", "Process", "Unsafe"] ``` Now drift fails closed. If the parser grows `File` authority, or the hash helper's evidence downgrades from `proved_by_lean` to `assumed`, the build stops and demands an explicit change to the constraint. That is the supply-chain backdoor from the top of this post, caught at compile time instead of explained in a postmortem. Capabilities, contracts, and evidence classes exist today; bounded imports and authority budgets are a design on paper, not a feature you can run. But the direction is the whole point, because it is where the label stops describing one program and starts describing the entire dependency tree, which is the only level at which "a full list of trust dependencies" is actually true. ## Labels should be compiler artifacts Software trust labels should not be vendor prose. They should be compiler artifacts: deterministic, diffable, and backed by machine-checked evidence wherever the strong claims are made. Concrete shows what that looks like for capabilities, contracts, proof obligations, evidence classes, axioms, and trusted boundaries. This is the same argument as [a fact-producing compiler](@/series/concrete/2026-04-09-a-fact-producing-compiler/index.md) and [when the compiler is the oracle](@/series/concrete/2026-03-20-when-the-compiler-is-the-oracle/index.md), pointed at a live conversation elsewhere. The systems-language world and the crypto-trust world are circling the same object from opposite sides. One wants software to emit a verifiable manifest of what it depends on. The other wants a language where that manifest falls out of compilation. Concrete does not solve the honest-actor half, and it does not pretend to. It takes the half that can be mechanized and mechanizes it: the part CI can reject, a reviewer can diff, and an agent can stand on without asking to be believed. --- ### A Fact-Producing Compiler *Published: 2026-04-09* > Concrete already knows a lot about what a program relies on: authority, allocation, recursion, trust, safety obligations, and proof evidence. The next step is making those facts easy for agents, CI, and reviewers to use. URL: https://federicocarrone.com/series/concrete/a-fact-producing-compiler/ > Series note: this article is part of the Concrete series and responds to Dmitri Sotnikov's [Giving LLMs a Formal Reasoning Engine for Code Analysis](https://yogthos.net/posts/2026-04-08-neurosymbolic-mcp.html). > Related: [When the Compiler Is the Oracle](@/series/concrete/2026-03-20-when-the-compiler-is-the-oracle/index.md) and [Why Concrete Exists](@/series/concrete/2025-12-26-the-concrete-programming-language-systems-programming-for-formal-reasoning/index.md). When an AI agent explores a codebase, it usually greps for names, reads a few matches, searches for callers, reads those, and tries to piece together a mental model of the program from text fragments. This works about as well as you would expect. The agent is asking structural questions about a program, things like "can user input reach this SQL query?" or "what changes if I touch this function?", but the only tool it has is text search. Yesterday I read Dmitri Sotnikov's [article](https://yogthos.net/posts/2026-04-08-neurosymbolic-mcp.html) about giving LLMs a symbolic reasoning engine for code analysis. His tool, [Chiasmus](https://github.com/yogthos/chiasmus), parses source code with tree-sitter (a syntax parser), turns definitions and calls into logic facts, and lets an LLM run graph queries instead of grepping through files. That is a much better interface: the agent asks a structural question and gets a structural answer. Reading the post gave me a better phrase for part of what we are building with Concrete: a **fact-producing compiler**. Concrete is the systems programming language we are building for programs that need auditability. It compiles code into an executable and into checked statements about what that executable can do. ## From syntax to semantics Chiasmus works by recovering structure from source code after the fact: it parses files, extracts which functions exist and what calls what, and turns that into queryable facts. For existing languages that were never designed to expose this information, that is the practical approach. Concrete can go further because the compiler already knows more than syntax. Tree-sitter can see that `foo` calls `bar`, but the Concrete compiler also knows that `bar` requires `Network` authority, that `foo` carries that authority in its signature, and that the call chain crosses a trusted FFI boundary. By authority I mean what code is allowed to do: allocate memory, read files, touch the network, call unsafe code, cross into foreign code. In Concrete, those permissions are part of the program the compiler checks. They are not comments and they are not recovered later by a scanner. Today the compiler exposes this as a human report. An authority report on our JSON parser shows where allocation comes from: ``` capability Alloc (13 functions): pub store <- store -> vec_push parse_string <- parse_string -> store pub parse_value <- parse_value -> parse_string parse_array <- parse_array -> vec_push ``` Read one line like this: `parse_value` needs allocation because it calls `parse_string`, which eventually stores bytes in a vector. The compiler tracks capabilities (`File`, `Network`, `Alloc`, etc.) in function signatures, enforces them transitively, and can report the path that explains why a function needs some authority. It also tracks execution shape: direct recursion, mutual call cycles, and loop boundedness. The `predictable` profile is the stricter direction: today Concrete reports and checks parts of bounded execution, while the fully enforced profile is still being tightened. For proofs, the same rule applies. A report can say whether a claim was merely reported by the compiler, enforced by a compiler check, proved in Lean (a proof assistant), or accepted because of a trusted assumption. Proof evidence is attached to the function name and body fingerprint, so changed code cannot silently keep stale proofs. All of this started as human-readable reports. Since then, the project has moved closer to the thing this article is arguing for: one artifact that review tools, CI, and agents can all read. The reports now have proof bundles, source-contract obligations, VC ledgers, traceability output, audit summaries, and better drift detection. The remaining work is not to invent the facts. It is to make the whole surface pleasant to query. ## What querying looks like Consider a routine scenario. A reviewer opens a PR that bumps a dependency. The new version reads environment variables three layers deep in a helper. In Rust, nothing in the function signatures changes and the reviewer has to diff the dependency source or hope the changelog mentions it. In Concrete, the function that calls into that dependency must have declared `Env` authority or the build breaks. I want the same kind of interface for every compiler fact, capabilities included. Here is what that could look like. An agent or tool asks a question, and the compiler returns a checked answer: "Can the packet parser core touch the network?" ```json { "reachable": false, "from": "main.decode_header", "to_capability": "Network", "evidence": "compiler-checked call graph and capability facts" } ``` "Why is main not predictable?" ```json { "violations": [ { "gate": "no_blocking", "capability": "File", "path": ["main.main", "std.fs.read_to_string"] } ] } ``` "Which dependency widened authority since yesterday?" Because the compiler tracks the authority chain for every function, it can return the path. The model can explain the result. The compiler should supply it. ## What we are making queryable First, we should expose the facts Concrete is already built around: which functions can allocate, read files, touch the network, call unsafe code, or cross FFI; why each authority is required; which functions recurse or enter call cycles; which loops are bounded; which runtime-safety obligations exist; which proof claims are current, proved, stale, missing, assumed, enforced, reported, or trusted. Then it should answer the questions people ask in review. Did this dependency add a path to `File`, `Network`, or `Env`? Did this module become less predictable? Did a proof go stale? Did a trusted boundary move? Did authority widen between yesterday's build and today's? The fact artifact comes first. A query CLI, MCP, CI checks, review tools, and agent integration should all read the same checked facts. ## The compiler should say what is true A compiler should say what is true about the program, beyond "type check passed." What can this function touch? Is it recursive? Are its loops bounded? Does it cross FFI? Is the proof current? Which path explains the authority? Those are program facts. Concrete produces some, enforces some, proves some, and labels the rest. The direction is not just better reports. It is a compiler artifact that other tools can safely build on. --- ### What Concrete Makes Worse *Published: 2026-03-24* > Concrete's constraints have real costs. Linear cleanup is verbose, hidden-capture closures are gone, and the ecosystem is still early. Here is what the language actually makes harder. URL: https://federicocarrone.com/series/concrete/what-concrete-makes-worse/ > Series note: this is the tradeoffs entry in the Concrete series. > For the foundation, start with [Why Concrete Exists](@/series/concrete/2025-12-26-the-concrete-programming-language-systems-programming-for-formal-reasoning/index.md). For the most practical demo, read [When the Compiler Is the Oracle](@/series/concrete/2026-03-20-when-the-compiler-is-the-oracle/index.md). The previous articles in this series argued that Concrete's design constraints are worth it. Explicit capabilities make code auditable. Linear types prevent resource leaks at compile time. No hidden behavior means the compiler can report what your program actually does. I believe all of that. But I have been writing Concrete code for long enough to know where the constraints bite, and I have not been honest enough about that in public. This article is about what Concrete makes worse. Not in theory, not as an abstract "it's stricter." Specific code that is uglier, longer, or more painful to write in Concrete than in Rust or Zig. If you are considering whether these tradeoffs are worth it for your domain, you deserve to see the cost up front. ## Linear cleanup is verbose and repetitive In Rust, RAII handles resource cleanup. You open a file, use it, and when the scope ends the compiler inserts a Drop call. You never write the cleanup. Three resources, zero cleanup lines: ```rust fn process(path: &str) -> Result { let config = File::open("config.toml")?; let input = File::open(path)?; let mut output = File::create("report.txt")?; let settings = parse_config(config); let data = read_all(input); let report = analyze(&settings, &data); write!(output, "{}", report)?; Ok(report) } ``` In Concrete, every resource needs explicit cleanup. Owned values must be consumed exactly once. If you forget, the program does not compile. That is the point, but here is what it looks like: ``` fn process(path: &String) with(File, Alloc) -> Result { let config = open("config.toml")? defer destroy(config) let input = open(path)? defer destroy(input) let output = create("report.txt")? defer destroy(output) let settings = parse_config(&config) defer destroy(settings) let data = read_all(&input) defer destroy(data) let report = analyze(&settings, &data) write(&mut output, &report)? Ok(report) } ``` Six `defer destroy` lines. The function's logic is the same, but half the lines are cleanup ceremony. Real Concrete code looks like this when you work with multiple resources. The ratio gets worse as functions get more complex. It is tempting to say "well, at least you can see every cleanup site." That is true. It is the reason the alloc report works, the reason auditors can trace resource lifetimes without reading the implementation of every type, the reason the oracle experiment could identify unnecessary allocations mechanically. But when you are writing the code, you feel the weight. In Rust, you trust that Drop runs at scope exit and you move on. In Concrete, you think about destruction order, you type `defer destroy` for every owning binding, and you occasionally stare at a function wondering if there is a way to factor out the ceremony. There usually is not. The worst case is error paths. If a function opens resource A, then tries to open resource B and fails, the error propagation with `?` runs the deferred cleanup for A. That part works. But if you need conditional cleanup, different paths owning different subsets of resources, the linearity checker forces you to handle every case explicitly. Rust's Drop handles this invisibly. Concrete makes you write it out. I think the tradeoff is correct for the domains Concrete targets. But I no longer describe it as "more annoying to write" as if it were a minor inconvenience. It is a substantial ergonomic cost that you pay on every function that manages resources. ## No hidden-capture closures hurts composition In Rust, filtering a list is one line: ```rust let active: Vec<_> = users.iter().filter(|u| u.is_active()).collect(); ``` In Concrete, there are no closures in the Rust or JavaScript sense. No lambdas with invisible captures. You write a named function and pass it: ``` fn is_active(user: &User) -> Bool { return user.active } let active: Vec = filter(&users, is_active) with(Alloc) ``` This is fine for `is_active`. It is a meaningful predicate that deserves a name. But what about filtering by a threshold that changes? In Rust: ```rust let expensive: Vec<_> = items.iter().filter(|i| i.price > threshold).collect(); ``` The closure captures `threshold` from the enclosing scope. One line, obvious what it does. In Concrete, you cannot capture implicitly. The function you pass to `filter` can only use its arguments. If it needs the threshold, you have to say so: pass it as another argument, write a specialized helper, or pass an explicit function pointer with an explicit context value. Recent callable-value work makes that last option much more usable than it was when this article was first written. It still does not feel like a Rust closure, and that is the point. The context is visible. The callback's capabilities are visible. The compiler can see the shape of the thing you are doing. The rationale is real. Closures are hidden captures. A closure that captures a mutable reference is implicit aliasing. A closure that captures an owned value is an implicit move. A closure that captures by clone is an implicit allocation. In Concrete, all data flow is visible: function arguments go in, return values come out. Nothing is smuggled through a captured environment. But expressiveness has a floor. Below that floor, code stops being clear and starts being bureaucratic. Simple data transformations, map/filter/reduce chains, callback patterns, event handlers, all of these are natural with closures and heavier when callback state has to be explicit. Concrete is still below the ergonomic floor for this class of problems. The answer Concrete has settled on is not to sneak closures back in. It is bound callbacks: an explicit function pointer plus explicit context, with capabilities carried on the callback type and scoped borrows prevented from escaping. That is less pleasant while you are writing the code, but much easier to audit later. It preserves the thing Concrete cares about most: no hidden data flow. ## The missing ecosystem If you try Concrete today, you will hit walls that have nothing to do with the language's design. There is no package manager. Dependencies are manual. The formatter exists now, but it is still young compared with `rustfmt` or `zig fmt`. There is no LSP, so your editor gives you little: no mature autocomplete, no inline semantic errors, no go-to-definition. The standard library has more than 30 modules, which sounds like a lot until you need something it does not cover and realize you are writing it from scratch or calling C through FFI. Rust has crates.io, cargo, rustfmt, rust-analyzer, and a library for nearly anything. Zig has a package manager and a growing ecosystem. Concrete has a compiler, a test runner, a formatter, and early audit/proof tooling. This is a maturity problem, not a design one. The compiler works. The language is real. But the surrounding infrastructure that makes a language livable for daily work is early. If you pick Concrete for a project today, you are signing up to build some of that infrastructure yourself, or to wait. I am not going to pretend this does not matter. Tooling is not secondary to language design. An immature formatter still leaves style and editor integration feeling rough. A language without an LSP means slower feedback loops. A language without a package manager means dependency management is manual labor. These are not luxuries. They are the difference between a language you can advocate for and one you use alone. The plan is to build the rest of it. The LSP is the next major quality-of-life gap. A package manager is further out. But plans are not tools, and I would rather be honest about what exists today than let someone discover the gaps after committing. ## The cost and the payoff come from the same source Every pain point in this article traces back to the same design decisions that make Concrete's strengths possible. Linear cleanup is verbose because every resource lifetime is explicit. That explicitness is why the alloc report works, why the compiler can tell you exactly where allocation happens and through which call chain. No hidden-capture closures is painful because it eliminates a natural composition pattern. That elimination is why all data flow is visible, why callback capabilities are trackable through the call graph, why the proof surface is not contaminated by invisible captures. The missing ecosystem is the cost of building a new language instead of extending an existing one. That independence is why the grammar is LL(1), why capabilities are built in from the start, why the compiler can be an oracle instead of a gatekeeper. These are not separate tradeoffs; they are the same one viewed from different angles. You cannot have the reports without the verbosity. You cannot have the trackable call graph without making callback state explicit. You cannot have a language designed for machine reasoning without starting from scratch. For firmware, security boundaries, cryptographic policy, safety-critical components, I still think the tradeoff is right. The previous articles in this series explain why. This one explains what it costs. Both are true at the same time. --- ### When the Compiler Is the Oracle *Published: 2026-03-20* > I ran an autoresearch-style loop on a Concrete program. The compiler told an agent where authority, allocation, and proof surface could improve and confirmed when those properties changed. No profiler, no benchmark noise. Your compiler can answer questions instead of saying pass/fail. URL: https://federicocarrone.com/series/concrete/when-the-compiler-is-the-oracle/ > Series note: this is the most practical entry point in the Concrete series. > If you want the shorter manifesto first, read [Why Concrete Exists](@/series/concrete/2025-12-26-the-concrete-programming-language-systems-programming-for-formal-reasoning/index.md). If you want the language reference behind this article, use [Concrete Spec](@/series/concrete/spec/_index.md). I have been building [Concrete](https://github.com/lambdaclass/concrete) for a while now. Something happened this week that I did not plan for, and it may end up mattering more than the things I set out to build on purpose. I let an AI agent improve a Concrete program using only the compiler's reports as feedback. No profiler. No benchmarks. The agent read what the compiler knew about the program, tried refactorings, checked if the compiler's answers improved, and kept or reverted. It worked better than I expected. The compiler had made the search space clean enough that the agent did not have to wander through benchmark fog. That points at why Concrete is useful in the first place. A language that makes authority, allocation, trust boundaries, and proof surface explicit is easier to audit, easier to optimize, and easier to automate against. You do not have to reconstruct the truth from profiler traces, stale docs, and reviewer intuition. The compiler can tell you what is true about the program, and that changes how you build systems software. To explain why, I need to start with what Concrete is and what makes it different. {% toolkit(label="Key takeaways") %} - An AI agent improved a Concrete program using only the compiler's reports as feedback: no profiler, no benchmarks. It read what the compiler knew, tried refactorings, kept the ones that improved the reports, and reverted the rest. - It worked because Concrete makes authority, allocation, trust boundaries, and proof surface explicit, so the compiler can state what is true about a program instead of leaving it to be reconstructed from profiler traces, stale docs, and reviewer intuition. - That is the general point: a language whose compiler emits real semantic facts is easier to audit, to optimize, and to automate against. - The agent did better than expected because the search space was clean: the compiler removed the benchmark fog it would otherwise have had to wander through. {% end %} ## What Concrete is Concrete is a systems language. It compiles to native code through LLVM, has explicit memory layout, manual ownership, FFI to C, and no garbage collector. It sits in the same space as Rust and Zig. But it makes a set of design bets that those languages do not make, and those bets are what made this week's experiment possible. **The grammar is LL(1).** The entire language parses with one token of lookahead. No ambiguous constructs, no context-dependent parsing. I chose this constraint on purpose. It means every program has one parse, every tool sees the same structure, and the syntactic surface is small. You can fit the grammar in your head. **Ownership is linear, not affine.** In Rust, ownership is affine: values can be used at most once, but you can also forget about them and the compiler will insert a Drop call behind the scenes. In Concrete, owned values must be consumed once. If you forget to clean up a resource, the program does not compile. Cleanup is explicit with `destroy` and `defer`. This is stricter than Rust and more annoying to write, but it means the compiler can reason about resource lifetimes in full. No invisible cleanup code executes behind your back. **Capabilities replace the ambient authority model.** This is the big one. In most systems languages, a function can do anything the process is allowed to do. A function with no special annotations can open files, make network connections, allocate unbounded memory, and call into C. You find out what a function does by reading its implementation, or by trusting its documentation. In Concrete, every function declares what it is allowed to do: ``` fn sha256(data: &Bytes) with(Alloc) -> String { ... } fn serve(port: u16) with(Network, Alloc, Console) -> Int { ... } fn allow_command(cmd: Command, policy: Policy) -> Decision { ... } ``` `sha256` can allocate but cannot touch the filesystem or network. The compiler enforces this from the type system. `serve` can use the network, allocate, and write to the console, but cannot read files. `allow_command` has no capability annotation at all. It is pure. A function can only call functions whose capabilities are a subset of its own. The concrete capability vocabulary includes `File`, `Network`, `Clock`, `Env`, `Random`, `Process`, `Console`, `Alloc`, and `Unsafe`, with `Std` and user-defined aliases expanding into those concrete names. If a function does not declare one, it cannot transitively reach any function that uses it. The type system enforces this, not convention. **Trust is split three ways.** Rust has one keyword for everything the compiler cannot check: `unsafe`. In Concrete, that splits into three distinct mechanisms: - **Capabilities** (`with(File, Network)`): semantic effects visible to callers. "This function does I/O." In the function signature. - **`trusted`**: containment of pointer-level tricks behind a safe API. Permits pointer arithmetic, raw dereference, raw assignment. Does NOT permit FFI, does NOT suppress capabilities, does NOT relax linearity. - **`with(Unsafe)`**: authority to cross foreign boundaries. FFI, transmute. Required even inside `trusted` code. When you see `trusted fn load_policy(path: &String) with(File, Alloc)`, you know: it does pointer work internally, it reads files and allocates, it cannot touch the network. In Rust you would see `unsafe` and have to read the body to learn any of that. **The compiler is written in Lean 4.** I chose Lean because it is a good language, and it created a structural opportunity: every Concrete program becomes Lean data during compilation. That data can feed proof tooling without crossing into a separate parser and model. The thing I hoped would happen has started to happen: you can write source contracts, turn them into proof obligations, attach Lean-checked evidence, and see when that evidence goes stale. This is still not end-to-end binary verification. It is a working bridge from ordinary systems code toward machine-checked claims about that code. **No hidden behavior.** No implicit destructors at scope exit. No exception unwinding. No operator overloading. No hidden-capture closures. No trait objects. No Deref coercions. No macro system. The code you read is the code that executes. This makes the language less convenient than Rust in many ways, and I think that is the right trade for the domain Concrete targets: firmware, security boundaries, cryptographic policy, and safety-critical components. Rust is excellent at enforcing safety properties. Concrete is better at turning the compiler into an instrument panel instead of a check-engine light. That is the practical claim behind the language. If you are building firmware, security-sensitive infrastructure, cryptographic software, or anything else where hidden behavior is expensive, Concrete gives you a compiler that can answer higher-level questions directly: which functions can allocate, which modules gained authority, which code is pure enough to prove, which trust boundaries moved. In most languages, you piece those answers together from code review, profiling, documentation, and static-analysis tools that only partially agree. In Concrete, they come from the language semantics themselves. ## The compiler as an oracle All of these design choices produce the same structural consequence: the compiler knows a great deal about what every function in your program does, and it knows it from the type system, not from heuristics. Most compilers throw this knowledge away after producing a binary. Concrete keeps it and exposes it. The exact report surface has grown, but the important modes include: - `--report eligibility`: which functions are pure enough to prove, and which proof gates they fail - `--report proof-status`: which claims are proved, stale, missing, blocked, ineligible, or trusted - `--report proof-bundle`: a JSON evidence bundle with proof state, assumptions, registry entries, and dependency facts - `--report contracts`, `--report vcs`, and `--report obligation-ledger`: source contracts, generated verification conditions, and obligation status - `--report audit` and `--report traceability`: evidence classes, trusted boundaries, and source-to-backend facts - `--report authority`: for each capability, which functions require it and through which transitive call chain - `--report alloc`: where allocation, cleanup, and defer happen - `--report unsafe`: trust boundaries, extern functions, unsafe crossings - `--report caps`: per-function capability sets - `--report layout`: struct sizes, alignment, field offsets - `--report interface`: public API surface - `--report mono`: monomorphized generic code These are structured facts derived from the same semantic analysis that type-checks the code. Same code, same report, same result. Deterministic. The proof-eligibility report on our JSON parser looked like this: ``` === Proof Eligibility Report === module Types: ✓ mk_null ✓ mk_bool ✓ mk_number ✓ mk_string ✓ mk_array ✓ mk_object ✓ mk_error ✓ is_error module StringPool: ✗ store (requires capabilities: Alloc) module Lex: ✓ is_ws ✓ is_digit ✓ skip_ws ✓ match_keyword module Parser: ✓ err ✗ parse_string (requires capabilities: Alloc) ✓ parse_number ✗ parse_value (requires capabilities: Alloc) ✗ parse_array (requires capabilities: Alloc) ✗ parse_object (requires capabilities: Alloc) ✗ parse_kv (requires capabilities: Alloc) Totals: 27 functions, 14 eligible for ProofCore, 13 excluded ``` Every function gets a verdict and a reason. The authority report shows the transitive call chains: ``` capability Alloc (13 functions): pub store <- store -> vec_push parse_string <- parse_string -> store pub parse_value <- parse_value -> parse_string parse_array <- parse_array -> vec_push parse_object <- parse_object -> vec_push parse_kv <- parse_kv -> parse_string ``` `parse_value` needs Alloc because it calls `parse_string`, which calls `store`, which calls `vec_push`. The compiler tells you the full path. A human reviewer reads this and knows which dependency to cut. An agent reads this and has a target. ## The experiment I pointed an AI agent at Concrete's JSON parser. 719 lines, recursive-descent, handles the full JSON spec minus floats and unicode escapes. Real code that has been pressure-tested. The agent's only tools were the compiler's reports and the ability to edit code and run the program. Starting point: **14/27 functions proof-eligible (51.9%)**. The question: can the agent, guided only by structured compiler reports, improve this program step by step? ### Round 1: extracting pure logic The agent read the proof report and noticed that several excluded functions had pure decision logic buried inside them. `parse_string` handles escape sequences. The mapping from `\n` to character code 10, `\"` to 34, `\\` to 92, and so on is pure decision logic. It does not allocate. It does not do I/O. But it was inlined inside an allocating function, so the compiler could not see it as a separate proof target. The agent extracted it into its own function: ``` pub fn map_escape(esc: i32) -> i32 { if esc == 34 { return 34; } // \" if esc == 92 { return 92; } // \\ if esc == 47 { return 47; } // \/ if esc == 110 { return 10; } // \n if esc == 116 { return 9; } // \t if esc == 114 { return 13; } // \r if esc == 98 { return 8; } // \b if esc == 102 { return 12; } // \f return 0 - 1; } ``` Same pattern for character classification in `parse_value` ("is this character the start of a string, number, array, or object?"), value comparison in the test harness, and trailing-content validation in `parse_json`. The loop after each extraction was simple: 1. Run the eligibility/proof-status reports. Did the proof-eligible count go up? 2. Run the program. Does it still pass? 3. Yes to both? Keep the change. After this round: **19/32 proof-eligible (59.4%)**. Five new pure functions, all extracted from effectful code without changing behavior. The agent did not need to understand what the JSON parser does. It did not need to know what JSON is. It read a structured report, identified functions excluded for a specific reason, found pure logic mixed with effectful logic inside those functions, and separated them. The report was the guide. The compiler confirmed the result. ### Round 2: eliminating unnecessary allocation Then the agent looked at `--report alloc`. `parse_value` is the recursive core of the parser. Each JSON value passes through it. And each call was heap-allocating three strings to check keywords: ``` let kw_true: String = "true"; defer drop_string(kw_true); if match_keyword(s, p, &kw_true) { return ParseResult { val: mk_bool(1), pos: p + 4 }; } let kw_false: String = "false"; defer drop_string(kw_false); // ...same for "null" ``` Three malloc/free pairs per call to `parse_value`, to compare a handful of characters. On nested JSON, `parse_value` is recursive. An object like `{"a": {"b": {"c": true}}}` calls it four times, producing twelve unnecessary heap allocations for one small document. On a large JSON file with thousands of values, that is thousands of wasted malloc/free cycles. The fix was obvious once the report pointed it out: match keywords by comparing characters directly, no heap allocation needed: ``` pub fn match_true(s: &String, pos: i32) -> bool { let slen: i32 = string_length(s) as i32; if pos + 4 > slen { return false; } return string_char_at(s, pos as Int) as i32 == 116 // t && string_char_at(s, (pos + 1) as Int) as i32 == 114 // r && string_char_at(s, (pos + 2) as Int) as i32 == 117 // u && string_char_at(s, (pos + 3) as Int) as i32 == 101;// e } ``` After this change, `parse_value` dropped out of the alloc report. Final result: **22/35 proof-eligible (62.9%)**, and `parse_value` no longer allocates to check keywords. That was the point where this stopped feeling like a parlor trick. ## Doing less is doing more Both improvements came from the same refactoring. Extracting pure logic and eliminating unnecessary allocation were the same move. The allocation-free keyword matchers are also pure functions. They show up in both reports: gone from the alloc report, added to the proof report. Better architecture and better performance, confirmed by two different compiler reports, from one structural change. In most codebases, performance optimization and code quality feel like separate concerns. You optimize for speed in one pass, refactor for clarity in another, and sometimes they conflict. The faster version is harder to read, the cleaner version is slower. The compiler made visible something that was always true but hard to see: the simplest version of a function, the one that does only what it needs to do with no incidental allocation and no mixed concerns, is the fastest, the most auditable, and the most provable at the same time. This follows from how Concrete's reports work. A function that does less needs fewer capabilities. Fewer capabilities means more likely pure. Pure means proof-eligible. No incidental allocation means faster. These are the same property, simplicity, measured from different angles. The reports make it visible. An agent optimizing for any one of these axes tends to improve the others. That is a useful property for an automated loop. The search space is not adversarial. You are not trading off proof eligibility against performance or auditability against speed. You push in one direction and get improvements across the board. ## Why this does not work in other languages In Rust, finding unnecessary allocations means profiling. You run benchmarks, fire up a heap profiler, stare at flamegraphs, try to figure out which allocations are incidental versus essential, refactor, re-profile, and hope the numbers improved. The compiler tells you nothing about allocation patterns. It says pass or fail. An AI agent doing this in Rust would have to generate benchmark code, run benchmarks (noisy, depends on system load), parse profiler output (tool-specific, often visual), guess which allocations are unnecessary, refactor, re-benchmark, and hope the noise did not mask the signal. In Concrete, the agent read `--report alloc`, saw three cleanup sites in `parse_value`, read `--report authority` to understand the call chain, replaced the allocations with direct character comparisons, and confirmed with `--report alloc` that `parse_value` dropped out of the report. No profiler. No benchmark noise. Deterministic signal. Rust's compiler does not model these properties the same way. Rust does not track allocation as a first-class semantic property. It has no capability system. It does not distinguish between "this function allocates because it needs to" and "this function allocates because of a convenience pattern." The information is not there for the compiler to report. This is a language design question. You cannot bolt this onto an existing language after the fact. The tracking has to be in the type system and the function signatures. If `with(Alloc)` is not in the language, the compiler cannot compute the transitive authority chain for allocation. If capabilities are not in function signatures, the compiler cannot determine which functions are pure. The reports exist because the language was designed to make them possible. ## The feedback signal problem Andrej Karpathy has been talking about "autoresearch": automated loops where an agent tries changes and uses some signal to decide whether to keep them. The bottleneck is always the fitness function. Benchmark numbers are noisy. A 2% improvement might be measurement error. A 5% regression might be system load. Test suites are binary: pass or fail, no gradient. Static analysis tools produce warnings that may or may not matter. None of these give an agent a clean signal to optimize against. Most of the time you are navigating by weather. Compiler reports are deterministic. "Is `parse_value` in the alloc report?" has one answer. "How many functions are proof-eligible?" is a number that does not change between runs. "The transitive authority chain for this capability" is a structured fact the agent can parse and reason about. These are exact properties of the program's semantic structure. This turns program optimization into a search problem with a reliable fitness function. That is what makes it tractable for automated agents. ## The research directions, and why they all become fitness functions The experiment used three reports. But we have been developing research directions for Concrete that each add new axes: **Authority budgets.** Capabilities today are real per-function facts. Authority budgets extend that idea to modules and packages. Imagine you depend on a JSON parsing library. You declare: ``` #[authority(Alloc)] import json_parser; ``` That is a contract: `json_parser` may only use `Alloc`. If the maintainer's next release adds a logging call, even buried three layers deep in a helper function, your build breaks. The dependency violated its authority budget. You did not need to read the changelog or audit the source. The compiler checked the transitive capability set, which it already computes, and found it exceeded the budget. This is the supply-chain direction: make authority drift fail closed instead of relying on a changelog or manual audit. In Rust or Go, a dependency can add network access, file system reads, or environment variable lookups without you knowing unless you audit the code or catch it in review. In Concrete, the underlying capability facts already exist; the budget layer is the policy gate that would turn a widened authority set into a compile-time failure. For an agent: restructure code until a module's capability set fits its declared budget. **Allocation budgets.** `with(Alloc)` is currently binary. The proposal classifies functions as NoAlloc, Bounded (allocates but provably bounded), or Unbounded. The compiler walks the call graph to classify. In safety-critical code for medical devices, avionics, or industrial control, you often need to prove that a function cannot allocate unboundedly. Today that is a manual audit. With allocation budgets, the compiler classifies it. For an agent: push functions from Unbounded to Bounded to NoAlloc, with the compiler confirming each step. **Execution cost tracking.** Ask the questions a reviewer actually cares about: does this function loop, is the loop bounded, is there recursion, how deep can the static call chain go? Concrete makes those questions more tractable because there is no dynamic dispatch, no hidden-capture closures, no hidden allocation, and the SSA CFG is clean. Explicit callbacks still exist, but the function, context, and capabilities are all visible. For bounded functions, you can compute abstract instruction counts. Combined with allocation budgets, "this function runs in bounded time with bounded allocation" becomes the kind of sentence you can take to a safety certifier. For an agent: identify unbounded loops and try to make them bounded, with the compiler verifying the classification changed. **Semantic diff and trust drift.** Today, code review means reading source diffs. You see that someone changed 200 lines across four files. You try to figure out if anything important changed about the program's behavior. You might miss that a helper function now transitively reaches the network, or that a previously pure function started allocating. Semantic diff replaces this with structured comparison of the compiler's reports between two versions. The output would look something like: ``` authority changes: + process_request now requires Network (via log_to_server) - validate_input no longer requires File proof surface: - parse_header dropped from ProofCore (now requires Alloc) trust boundaries: + new trusted function: fast_copy ``` The output is not "the lines that changed" but "the trust properties that changed." A reviewer reads this and knows what to scrutinize. A CI gate reads this and blocks a PR that adds `Network` authority to a module that had none before, because a semantic policy was violated. For an agent: flag trust regressions and try to fix them before they reach code review. **Proof addon architecture.** The compiler produces stable semantic artifacts and evidence reports (`Core`, `ProofCore`, proof status, proof bundles). Proof tooling, including Lean, kernel decision procedures, and optional external solvers, consumes those facts instead of being fused into ordinary compilation. Proof failure is not automatically compile failure unless a profile or policy says it is. You can ship code that compiles and grow proof coverage over time. `ProofCore` is the pure subset: functions with no capabilities, not trusted, no extern calls. The autoresearch experiment, extracting pure logic to grow ProofCore, is the workflow this architecture enables. Each of these was designed for human consumption. Each also produces a fitness function an agent can optimize against. If the compiler can state a fact clearly enough for a human to act on, it is clear enough for an agent to act on too. ## Auditability and optimizability are the same thing I designed the report system for human auditors who need to understand what a program does without reading every line. I wanted a security reviewer to ask "what authority does this module have?" and get an answer from the compiler, not from documentation that might be stale. I built the reports for auditing. I did not expect them to become an interface for optimization. Auditability and machine optimization are the same property viewed from two angles. The properties that make a program auditable (explicit capabilities, visible trust boundaries, trackable allocation) are the same properties that give an agent a reliable fitness function. Making the language honest about what code does, for the benefit of human reviewers, made it optimizable by machines as a side effect. An auditable program is one where the compiler can state facts about behavior without running the code. An optimizable program is one where the compiler can confirm that a change improved some property without running benchmarks. These are close to the same requirement. If the compiler can say "this function needs Network because of this call chain," a human can audit it and an agent can try to eliminate it. I was thinking about a human reviewing a firmware update tool and wanting to know "can this function access the network?" The fact that the same answer is useful to an automated optimizer says something about what compilers should do. ## The bet The JSON parser was one program. Concrete's MAL interpreter has 60 functions with 24 excluded from ProofCore. The same loop should work there, and on every program the compiler can report on. The technique generalizes because the reports generalize. The vision is a compiler that participates in the development loop as more than a gatekeeper. Most compilers say pass or fail. With structured reports, the compiler says what is true about your program. With authority budgets, it says what you promised would be true, and where reality diverges. With semantic diff, it says what changed about your program's trust properties since the last version. Each layer gives both humans and agents more to work with, and the layers build on each other. The compiler becomes an oracle: a source of structured knowledge about program behavior. The agent becomes an optimizer that tries thousands of refactorings and keeps the ones that move those answers in the right direction. The programmer sets the goals, picks which properties matter and what budgets to enforce, and reviews the results. The payoff is concrete, not abstract elegance: tighter audits, clearer security boundaries, less performance guesswork, and a development loop where both humans and machines steer from the same map. A language the compiler can explain, and that machines can improve, because it was designed to be honest about what code does. --- ### Can I prove Concrete programs in Lean? *Published: 2026-03-12* > The original roadmap for proving Concrete programs in Lean, updated now that part of that bridge exists: source contracts, proof obligations, Lean-checked evidence, stale detection, and an explicit trusted base. URL: https://federicocarrone.com/series/concrete/proving-systems-code-in-lean/ > Series note: this is the proof-roadmap entry in the Concrete series. > For the language overview, start with [Why Concrete Exists](@/series/concrete/2025-12-26-the-concrete-programming-language-systems-programming-for-formal-reasoning/index.md) and [Concrete Spec](@/series/concrete/spec/_index.md). For the most practical compiler-report demo, read [When the Compiler Is the Oracle](@/series/concrete/2026-03-20-when-the-compiler-is-the-oracle/index.md). I wrote the first version of this essay when proving [Concrete](https://github.com/lambdaclass/concrete) programs in Lean was still mostly a plan. The question was simple and a little risky: can we take a function written in Concrete, connect it to something inside the compiler, and prove a real property about it in Lean? Part of that answer now exists. Not for the whole language, and not for the final binary, but for a real subset. Concrete can take a source contract, turn it into an obligation, attach a Lean-checked proof or a decision-procedure result, and tell you when that evidence no longer matches the code. The important part is not that every claim turns green. It is that the tool refuses to mix up "proved," "assumed," "trusted," and "not done yet." ## Why I think there is an opening Verified systems code is expensive. The seL4 microkernel took roughly 20 person-years to verify: about 200,000 lines of Isabelle proof for roughly 10,000 lines of C. Fiat Cryptography generates verified primitives that ship in real browsers. Mathlib has formalized an enormous amount of mathematics in Lean. The proof side is fine. The languages are the problem. They were never designed as proof targets, so most of the effort goes into bridging two unrelated worlds. That bridging cost is what I want to attack. With VST you write C, a separate tool parses it into Coq, and you hope the tool's model of C matches what your compiler actually does. Verus stays in Rust but has to model unsafe code, implicit Drop, Deref coercions, and everything else Rust was not designed to make verifiable. F* and Dafny start from verification and try to extract systems code, but you end up in a proof assistant. ATS tried to be both, but never caught on. It always ends the same way: bolt verification onto a language that fights you, or extract systems code from a language that was not built for it. I think Concrete might be in a different position because of a decision I made early on: writing the compiler in Lean 4. ## What the compiler already gives me Concrete's compiler is written in Lean 4. I did not do this because I wanted to prove things about programs. I did it because Lean is a good language. But it created a structural opportunity I did not fully appreciate at first. After parsing and elaboration, every Concrete program becomes Core IR, a small, explicit, fully typed intermediate representation defined as Lean inductive types: ``` inductive CExpr where | intLit (val : Int) (ty : Ty) | boolLit (val : Bool) | binOp (op : BinOp) (lhs rhs : CExpr) (ty : Ty) | call (fn : String) (typeArgs : List Ty) (args : List CExpr) (ty : Ty) | match_ (scrutinee : CExpr) (arms : List CMatchArm) (ty : Ty) | borrow (inner : CExpr) (ty : Ty) | borrowMut (inner : CExpr) (ty : Ty) | deref (inner : CExpr) (ty : Ty) -- ... ~40 constructors total across CExpr, CStmt, CMatchArm ``` This is native Lean data. Not a foreign object imported through FFI. Not a serialized AST that some other tool has to parse. The same types the compiler manipulates during elaboration, type checking, and lowering are the types I could write proofs about. There is no translation between two separate tools, though there are still internal boundaries (elaboration, the Core-to-math connection, lowering) where meaning could drift. Those boundaries live inside one codebase rather than across two ecosystems, and that matters. All surface sugar is gone by the time code reaches Core. Method calls become plain function calls. `?` becomes explicit match-and-return. `->` becomes dereference-then-field-access. Borrows, moves, and destroys are explicit operations. The compiler pipeline has hard boundaries: ``` Parse → Resolve → Check → Elaborate → CoreCheck → Mono → Lower → SSA → Emit ``` Core is the semantic authority. Everything before it is surface convenience; everything after it is lowering toward machine code. When I started thinking about where a proof boundary would go, Core was the obvious answer. It was already there. ## The original idea Here is what I want to do. Take a Concrete function, say one that reverses a list: ``` fn reverse(xs: List) -> List { let mut acc: List = List::Nil for x in xs { acc = List::Cons(x, acc) } return acc } ``` The compiler already elaborates this into a `CExpr` value in Lean. I want to connect that `CExpr` to Lean's existing list library and prove something about it, like that it preserves length. When I wrote this, I thought I needed three pieces that did not exist yet. **Piece 1: formal evaluation semantics for the proof representation.** A Lean definition that says what extracted expressions mean: ``` -- Sketch, not real code yet inductive Eval : Env → CExpr → Value → Prop where | intLit : Eval env (.intLit n ty) (.int n) | binOp : Eval env lhs (Value.int a) → Eval env rhs (Value.int b) → Eval env (.binOp .add lhs rhs ty) (.int (a + b)) -- ... ``` **Piece 2: a connection between extracted code and Lean mathematics.** I need to show that "this extracted term, when evaluated" means the same thing as "this Lean function over Lean lists." This is the hardest part, because you are connecting running code to the math you actually care about. **Piece 3: the proof itself.** Once the first two pieces exist, I can state and prove: ``` theorem reverse_length (xs : List α) : length (concreteFn_reverse xs) = length xs := by -- proof using Lean's standard list lemmas -- concreteFn_reverse is the Lean-level interpretation -- of the Concrete function's Core representation ``` That basic shape survived, but the implementation became more honest. The proof boundary is not "the whole compiler is now a theorem." It is an extracted proof representation, a source-linked theorem, a fingerprint that ties the theorem to the current body, and an evidence class that says exactly what was proved and what remains trusted. ## Why I think Concrete's design makes this feasible Writing the compiler in Lean is not enough on its own. If Concrete had the same feature surface as Rust or C++, the Core IR would be enormous and formalization would be just as painful. The reason I think this can actually work is that I designed Concrete to be small, and several of the design choices I made for other reasons turn out to help here. In Rust, reasoning about ownership requires modeling Drop: implicit destructors that run at scope exit, invisible control flow the programmer never wrote. In Concrete, owned values must be consumed exactly once and cleanup is explicit with `destroy` and `defer`. When you formalize Core, what you see is what executes. Capabilities make effects visible in the type system. If a function has no capability annotations, it is pure; the compiler enforces that. `with(File)` means file I/O. I can mechanically carve out the pure fragment of a codebase and reason about it without dragging the operating system into the proof. The trust boundaries double as proof boundaries. Safe code is compiler-checked. `trusted` code hides pointer-level implementation behind a safe API. `Unsafe` covers foreign calls and raw system access. The language already marks where each one is, so I do not have to figure out where trust enters the picture. No operator overloading, no implicit conversions, no closures with hidden captures, no exceptions, no trait objects, no Deref coercions. I originally left these out because I think they make code harder to read and audit. It turns out they also make formalization harder. Not having them keeps the proof surface small. ## How I compare this to other projects Nobody gets this for free. Verus gets Rust's ecosystem but has to model a language that fights verification at every turn. Fiat Cryptography generates verified code that no human wrote. RefinedC takes on all of C, and it shows. What I want is something different: write real low-level code in Concrete, with explicit ownership that compiles to native executables, and then prove properties about it in Lean using Lean's existing libraries. If the bridge works, the code you prove would be the code you ship, not a model that gets translated. And you would not need a bespoke verification ecosystem because Lean already has one. Concrete does not have Rust's ecosystem or C's installed base. I am betting that a small, explicit language can make the proof part cheap enough to be worth it anyway. ## What changed since the roadmap Here is where this actually stands now. What exists now is the first working version of the bridge I was imagining. A Concrete function can carry a contract. The compiler can turn that contract, or a runtime-safety concern like an array bound, into an obligation. A proof can be linked back to the source. A fingerprint ties that proof to the current body. If the body changes, the proof stops being current instead of silently remaining trusted. The examples matter because they show the shape of the promise and the boundary at the same time. `constant_time_tag` proves value correctness, but it does not pretend to prove machine-level timing. `hmac_sha256` carries a deeper refinement story. Bounds, division, and overflow obligations can be discharged by `omega` or `bv_decide`. The negative examples are just as important: assumptions remain assumptions, fake proof names get caught by `concrete prove --check`, stale proofs go stale, and vacuous contracts do not get to dress up as real proofs. What still does not exist is just as important. There is no formal proof of the whole checker. There is no verified path from Core through SSA and LLVM to the final binary. There is no whole-program verification story. The proof model is cleaner than the machine that eventually runs the code, and that gap has to stay visible. Concrete now proves real selected claims, but it does not prove the whole program correct end to end. ## Where I expect problems Pure functions over algebraic data types are the easiest proof target: arithmetic, structural transformations, parsers over bounded inputs. The first proofs should live here. FFI, `Unsafe`, and `trusted` code all sit outside the proof boundary. Foreign functions are black boxes. `trusted` and `Unsafe` code do things the type system cannot fully track. You model their contracts as axioms and you do not pretend otherwise. I prefer it this way: the proof tells you exactly where you are trusting something you have not verified, instead of pretending everything is covered. Mutable heap code is where things start getting expensive again. Separation logic can model it. I would rather earn the right to worry about that later. Concurrency is last. Real concurrency proofs need explicit models of scheduling, memory ordering, and synchronization, and I do not want to formalize any of that before the simpler cases work well. Concrete defers concurrency to later phases of the language for this reason. There is another problem I have not mentioned yet: even after a proof representation has clean evaluation semantics, those semantics can still diverge from what the compiler's lowering passes actually do. But this is a single boundary inside a single codebase, not two separate tools that have to agree on what C or Rust means. Closing that gap, proving lowering preserves semantics, is still a well-defined proof target for later. ## What comes next The next step is no longer "one proof of one real function." That happened. The next step is scale and honesty: grow the provable subset, keep evidence classes separate, make proof authoring less painful, tighten the soundness connection between extracted proof terms and the compiler pipeline, and keep the trusted computing base visible enough that nobody mistakes a local proof for a universal guarantee. --- ### Designing a Programming Language for the AI Era *Published: 2026-03-11* > Edgar Luque is right that AI creates a new barrier for programming languages. He is wrong that the barrier is universal. Languages designed for machine generation and machine verification invert the problem entirely. URL: https://federicocarrone.com/series/concrete/ai-wont-save-your-language-unless-you-design-for-it/ > Series note: this article assumes the basic Concrete frame is already in place and asks a narrower question about AI-era language adoption. > For the series foundation, read [Why Concrete Exists](@/series/concrete/2025-12-26-the-concrete-programming-language-systems-programming-for-formal-reasoning/index.md). For the main Rust comparison, read [The Rust Effects Debate and Concrete's Case for a Smaller Language](@/series/concrete/2026-03-09-rusts-grand-vision-and-concretes-answer/index.md). Edgar Luque recently wrote about how [AI creates a new adoption barrier for programming languages](https://edgl.dev/blog/ai-language-adoption/). His claim is that AI coding assistants need training data, training data only exists for popular languages, and so new languages get bad AI support, which prevents adoption, which in turn prevents training data from accumulating. A self-reinforcing loop that locks in whatever is already dominant. If you are building a new general-purpose language that competes with Python, Go, or Rust on roughly the same terms, Luque's analysis is devastating. What makes it worse than previous adoption barriers is that you cannot community-effort your way out of it. The AI training pipelines belong to a handful of companies, and those companies will always prioritize the languages where the most data already exists. But there is a blind spot in the argument. ## Where I Think Luque Is Wrong Luque assumes that languages are passive consumers of AI support. But what if a language is designed so that machines can reason about it *better* than they reason about established languages, because it is simple and explicit enough that an LLM can work largely from the specification? Established languages carry enormous amounts of tacit knowledge: idioms, conventions, workarounds, and unwritten rules that live only in the collective practice of millions of developers. Think about Python's `for/else` and the fact that most experienced developers avoid it entirely, or Go's error handling conventions that are nowhere in the language spec, or the subtle difference in Rust between when you should use `unwrap` and when you should propagate with `?`. An LLM needs a massive corpus precisely because it needs to soak up all of this unwritten knowledge through sheer exposure. As I wrote in [Legibility Kills What It Measures](@/series/les-circuits-longs/2026-03-03-legibility-kills-what-it-measures/index.md), Michael Polanyi called this the tacit dimension: "we know more than we can tell." The bigger a language's tacit dimension, the more training data you need before an AI can navigate it. A language that strips out enough tacit knowledge changes the loop. When more of what matters is already in the grammar, the types, and the capability annotations, the specification carries much more of the burden. [Concrete](https://github.com/lambdaclass/concrete) is designed around this principle. ## What Concrete Does Differently Every design choice points in the same direction: minimize ambiguity, maximize what a tool can figure out just by reading the code. **LL(1) grammar.** The entire language parses with one token of lookahead. No ambiguous constructs, no context-dependent parsing rules. The syntactic surface is genuinely small. **Explicit control flow.** What you read is what executes. No implicit destructors at scope exit, no exception unwinding through invisible paths, no operator overloading quietly changing what `+` does. **Explicit capabilities.** If a function reads a file, allocates memory, or touches the network, the signature says so: `with(File)`, `with(Network)`, `with(Alloc)`. You do not need to trace the call graph to know what a function might do. **Linear types.** Owned values must be consumed exactly once. The compiler rejects code that forgets to clean up a resource or uses one after it was moved. This is the kind of bug that LLMs are particularly bad at catching, because it requires tracking state across an entire function body. **One way to do things.** No closures and lambdas with hidden captures, no exceptions and result types overlapping, no five different iteration styles. Concrete has been growing explicit callbacks, but they are deliberately plain: the function, the context, and the capabilities are visible. Less surface area means fewer opportunities to pick the wrong approach. Rust is the closest existing language to this list, which is exactly why it is the right comparison. But Rust still carries a large tacit layer around the core language: implicit destructors via `Drop` that run at scope exit, no capability system, operator overloading through traits, lifetime elision rules, `Deref` coercions, macro-heavy APIs, and ecosystem conventions around async, error handling, and trait patterns. Rust reduced tacit knowledge compared to C++, but a substantial amount still lives in practice rather than in the spec. Concrete pushes further in the same direction. None of these features were designed for AI. I designed them because I think they make a better language for humans. They make behavior easier to see, APIs easier to review, and bugs easier to catch before runtime. Those same properties also make the language easier for a machine to generate correctly. ## Why This Changes the AI Problem When an LLM generates Python, it leans heavily on patterns absorbed from millions of files. It has to, because no specification captures how experienced Python developers actually write Python. A language whose grammar fits in a few pages and whose type and capability system encodes most of the rules changes the economics. You can paste the entire language spec into a context window. The model does not need to have seen a million Concrete programs to know what is legal; it can read the rules and apply them. That matters more now than it would have three years ago. Context windows are long enough to hold a full language spec alongside the code being generated. Tool use lets the model call the compiler mid-generation and read the errors back. Iterative repair workflows are standard. All of these trends help every language, but they help a small, explicit language disproportionately, because the spec fits in context and the compiler errors are precise enough to actually drive the fix loop. The bottleneck shifts from "how much code exists in this language" to "how much of the language can be recovered from the spec and the compiler." New languages will lose if the contest is raw corpus size. They can still compete if the contest is whether a model can generate valid code from explicit rules. ## Errors Have To Be Legible In Python or JavaScript, many wrong programs still make it past generation and into execution. The LLM generates a function that forgets to close a file handle, or swallows an exception, or mutates shared state in a way that only breaks under concurrency. The bug surfaces later, often outside the moment when the code was written. In Concrete, the compiler catches it. Forgot to consume a linear value? Compile error. Called a function that does I/O without the right capability? Compile error. Used a value after it was moved? Compile error. The error messages tell the LLM exactly what to fix. The useful question is whether the generate-check-fix loop converges quickly. First attempts matter less than repair speed. Precise compiler errors help. Silent runtime failures do not. Concrete goes further. It is written in Lean 4, and the proof path is now real rather than merely aspirational. You can put a contract next to a function, turn that contract into obligations, and attach evidence to the result. Sometimes that evidence is a Lean theorem. Sometimes it is a decision procedure. Sometimes it is only an assumption or an unfinished proof, and the report says so. That is not a proof of the whole compiler, and it is not end-to-end binary verification. It is the useful beginning of a proof-backed toolchain. As the volume of machine-generated code grows, tests and human review will fall further behind. If you want strong correctness guarantees at that scale, formal verification is the endgame. ## What Follows From This Luque suggests that new languages can survive by retreating to niches where AI matters less. Concrete's position is the opposite: target a world where AI matters *more*, where most code is machine-generated, and build the language so it works with that reality instead of hiding from it. The AI adoption barrier is real. For most new languages, it makes adoption harder in a way they cannot easily fix. My view is that languages designed for machine generation and machine verification can compete on a different axis entirely. That is the case I am making for Concrete. --- ### The Rust Effects Debate and Concrete's Case for a Smaller Language *Published: 2026-03-09* > Wuyts is right about effects and ownership. The Hacker News skeptics are right about complexity. Concrete accepts both and says no to refinement types. URL: https://federicocarrone.com/series/concrete/rusts-grand-vision-and-concretes-answer/ > Series note: this is the main Rust-comparison entry in the Concrete series. > If you are new here, start with [Why Concrete Exists](@/series/concrete/2025-12-26-the-concrete-programming-language-systems-programming-for-formal-reasoning/index.md). If you want the language reference, use [Concrete Spec](@/series/concrete/spec/_index.md). Yosh Wuyts recently wrote about his ["grand vision" for Rust](https://blog.yoshuawuyts.com/a-grand-vision-for-rust/), outlining three directions he thinks the language should pursue: effects, stronger substructural types, and refinement types. The [Hacker News thread](https://news.ycombinator.com/item?id=47256376) that followed split predictably: one camp saw a safer, more principled systems language taking shape while the other saw echoes of Scala, C++, and a language that becomes harder to read than the software it is meant to clarify. Both camps are seeing something real, and I think resolving the tension between them requires something other than adding more features to Rust. That is where [Concrete](https://github.com/lambdaclass/concrete) comes in. ## The Diagnosis Is Correct Forget the specific Rust syntax Wuyts proposes. The strongest part of his post is the problem he identifies. Rust already has a growing collection of "function colors": `async`, `const`, fallibility, generators, and many other properties that low-level software wants to track. The more of these pile up, the more the language feels like a collection of special cases rather than a single model of what functions can and cannot do. Systems code needs to express things like: - this function cannot do I/O - this function cannot unwind - this function cannot allocate - this function cannot call into the host - this function is deterministic These matter in practice. Embedded code, kernels, runtimes, critical infrastructure, formal verification, and any codebase where audits are part of the job. Wuyts is right about all of this. Where I disagree is with the solution. You do not fix this by adding more "colors" to functions inside Rust. You fix it by designing a language where effects are built in from the start. To make that more concrete, imagine a low-level function that parses input, allocates a buffer, and writes to a file. In Rust, the relevant properties end up spread across different mechanisms: `async` if it yields, `Result` if it fails, allocator behavior in the implementation, I/O permissions in convention rather than in the type, and possibly more markers later if the language grows in that direction. In Concrete, the same question is stated in one place: this function is pure unless declared otherwise, and if it allocates and writes a file its signature says `with(Alloc, File)`. The exact syntax does not matter here. What matters is that effect tracking is one model, not a pile of special cases. The language gets simpler when the effect model is consistent from the start. ## The Complexity Objection Is Also Correct The skeptics on Hacker News are also right in that languages do not become complex just because they have powerful ideas but because those ideas pile up, and the interactions between them multiply in ways nobody planned for. Effects interact with async. Async interacts with traits. Traits interact with generics. Generics interact with inference. Inference interacts with macros. Macros interact with diagnostics. Each feature makes sense on its own, but together they can make a language exhausting to read. The comment that stuck with me was not "I don't want safety." It was something closer to: I want to read business logic without feeling that every line is a proof obligation. This objection goes more directly to the tradeoffs of the language, because a language can become "safer" in one technical sense while becoming harder to understand in practice. I take this concern as a design constraint for Concrete. If a feature makes the language bigger without making ordinary code easier to read, it should not ship. This is why Concrete says no to things that mainstream languages keep around for convenience: - no hidden control flow - no implicit destruction - no hidden allocation - no exceptions - no hidden-capture closures - no trait objects - no operator overloading - no user-defined effects - no general attribute system There is a common pattern behind that list: each item either hides work from the reader, creates more than one way to express the same behavior, or forces the compiler and tooling to reason about more implicit interactions. Concrete removes those pressures at the language boundary instead of managing them after the fact. This minimalism is payment towards a bet that stronger guarantees and readability can coexist, but only if the language stays small. ## Effects and Ownership: Yes Concrete agrees with Wuyts on two of three axes. Nobody disputes that effects belong in a systems language. The real question is whether you design one model for all of them up front or let them accumulate as special cases. Rust will eventually need a real solution here if it keeps moving in this direction, and this kind of thing is much easier to get right in a new language than in one with a large installed base and a large library ecosystem. What matters is one effect model that handles all of them. On ownership, Rust is affine: values can be used at most once. That prevents use-after-free in safe code, but it still allows you to forget about values entirely. Concrete goes further. Owned values are linear by default, meaning they must be used exactly once. If you want to prevent resource leaks at compile time instead of hoping for the best at runtime, "at most once" is not enough. In Concrete, the compiler does not silently drop resources for you. Cleanup is explicit with `destroy` and `defer`, and if a linear value is not consumed, the program does not compile. This is stricter than Rust, and closer in spirit to Austral. It also costs more. Programmers have to write cleanup paths more explicitly, APIs have less room to paper over ownership decisions, and some common patterns become more ceremonial. I think that is an acceptable price, but it is a real price. ## Refinement Types: No The main disagreement is refinement types, and this is where I side with the skeptics. Refinement types are powerful. Pattern types and view types are clever. They can remove runtime checks, express partial knowledge, and make more borrows legal. All true. But they push the language from "the compiler checks resource and effect rules" toward "the compiler expects programmers to encode facts about their programs into types." That is a very different kind of language from what Concrete is trying to be. To be precise, I mean Concrete should not put this machinery in the core language. That is different from saying refinement techniques are useless. They may still belong in external proof tooling, verification-oriented libraries, or generated obligations that sit above the language proper. But they do not belong in the center of a language whose main job is to stay small, explicit, and reviewable. There are three reasons Concrete does not put them in the core language. First, refinement types dramatically increase what the type checker has to prove. They change the nature of what the compiler is expected to verify, and that change affects the whole language. Second, it makes code harder to read. Every constraint embedded in a type signature competes for attention with what the code actually does. Programmers can handle refinement types fine. But attention is limited, and type signatures should help you understand a function, not become puzzles on their own. Third, it works against one of Concrete's goals: being friendly to machine generation and machine review. Refinement systems are where generated code, human review, and proof tooling start pulling in different directions. So the breakdown is: - explicit effects: yes - stronger ownership: yes - refinement types in the core language: no Refinement types are useful in the right context. They are also expensive, and I do not think that cost belongs in the core of a language built around the idea that readability and formal reasoning should help each other instead of competing. ## Smaller, Not Smarter If you read Wuyts's post and think "this is Rust becoming more academic," Concrete might look like it goes even further. It is written in Lean, it talks about kernels and soundness and linearity, and it uses formal language on purpose. But Concrete is going in the opposite direction: a smaller language with stricter rules. That difference matters. A language becomes hard to read when its theory gets deeper, when there are too many ways to write the same thing, when implicit behaviors pile up, and when too many styles are all technically valid. Concrete gets stronger guarantees by taking away options instead of adding them: - one effect model - one ownership model - no invisible work - a small kernel - a grammar simple enough to parse with one token of lookahead If Concrete fails, it will fail for other reasons. Turning into C++ through feature creep is the failure mode the language is built to avoid. ## The Dividing Line This debate is not really about "type theory versus pragmatism." It comes down to something more specific: should a systems language get stronger guarantees by becoming more expressive, or by becoming narrower and more explicit? Rust tries to stay expressive while adding more guarantees. That makes sense for a mainstream language with a large ecosystem. Concrete takes the other path. If a guarantee needs more hidden machinery, more feature interactions, or more inference, I would rather skip it. If a guarantee comes from making the language more explicit, more uniform, and smaller, then I want it. Systems programming needs better tools for tracking effects and managing resources. The skeptics are right that language complexity is a risk too, and that type-system features become counterproductive when they make the language harder to hold in your head. Concrete accepts the first point and treats the second as a hard constraint. If Rust is asking how to become safer without becoming unreadable, my view is that this gets much easier when the language is willing to say no to more things. Concrete is not trying to out-express Rust. It is trying to out-constrain it. ## Further watching - Rich Hickey, [*Simple Made Easy*](https://www.youtube.com/watch?v=SxdOUGdseq4) (Strange Loop). The canonical case that simple, meaning unentangled, beats easy, which is the whole argument for a smaller language said in another vocabulary. --- ### Why Concrete Exists *Published: 2025-12-26* > Concrete is a systems language designed so the compiler can reason about what code does: authority, allocation, resource lifetimes, and proof surface. URL: https://federicocarrone.com/series/concrete/the-concrete-programming-language-systems-programming-for-formal-reasoning/ > This is the foundation piece for the Concrete series. > If you want the most practical demonstration first, start with [When the Compiler Is the Oracle](@/series/concrete/2026-03-20-when-the-compiler-is-the-oracle/index.md). If you want the living language reference, use [Concrete Spec](@/series/concrete/spec/_index.md). Systems programming has a recurring problem. We want to write code close to the machine, but we also want to make strong claims about what that code does. Does it allocate? Does it touch the network? Does it leak resources? Can it be audited without tracing twenty helper functions and three layers of library convention? Most languages answer those questions indirectly. You read the implementation. You profile. You infer from style. You trust `unsafe` blocks, docs, and review discipline. Even in strong languages, much of what matters about a program lives outside the type system. Concrete exists because I think that is the wrong place to stop. Concrete is a systems language built around a single organizing principle: **every important property the compiler can know about a program should be explicit enough for humans and machines to act on directly**. ## The problem The usual systems-language tradeoff is framed as performance versus safety. That is real, but it is not the only tension that matters. There is another one: - languages can be expressive enough to build serious software - or they can be simple enough that the compiler can explain what the software is doing When a language gets more implicit, more inferential, and more feature-rich, the gap between "what the code does" and "what the compiler can state plainly" gets wider. Destructors run invisibly. Allocation hides inside convenience patterns. Effectful behavior is ambient. Trust boundaries collapse into one broad `unsafe` bucket. Code remains writable, but it gets harder to audit, harder to prove, and harder to optimize from semantic facts rather than benchmark noise. Concrete is an attempt to push in the other direction: a narrower language that gives up convenience so the compiler can expose more truth. ## The core bets Concrete makes five bets. **1. Effects belong in function signatures.** If a function reads a file, allocates, touches the clock, or calls the network, that should be visible in its type via capabilities such as `with(File)` or `with(Alloc)`. No ambient authority, no guessing from implementation detail. **2. Ownership should be linear by default.** Rust's affine ownership prevents use-after-free, but it still lets you forget a value and let Drop clean up behind your back. Concrete is stricter. Owned values must be consumed exactly once. Cleanup is explicit with `destroy` and `defer`. That is more ceremonial, but it means resource lifetimes are part of the visible program rather than a hidden compiler action. **3. Hidden behavior is the enemy.** No implicit destructors, no hidden allocation, no operator overloading, no closures with invisible captures, no exceptions unwinding through code you cannot see. When you read a function, the goal is that you are looking at what actually executes. **4. Smaller beats cleverer.** I do not want a language that keeps absorbing more machinery in exchange for stronger guarantees. I want one effect model, one ownership model, one small grammar, and a core language that still fits in a human head. **5. The compiler should produce structured knowledge, not just pass/fail.** If the language is explicit enough, the compiler can report things like capability usage, trust boundaries, allocation sites, public interface surface, and proof-eligible subsets. That changes how humans review code and how automated agents improve it. ## What that looks like in practice This design leads to a specific kind of language. A pure function is pure because it declares no capabilities and the compiler proves that claim through the call graph. A function that allocates says `with(Alloc)`. A function that can read files says `with(File)`. A function that crosses a foreign or semantically dangerous boundary says `with(Unsafe)`. A function that manages a resource shows its cleanup in the source with `defer destroy(x)`. A piece of low-level implementation trickery can be marked `trusted`, separating internal pointer-level unsafety from externally visible semantic authority. The result goes beyond "safer C" or "stricter Rust." It is a language whose semantics are meant to be legible enough that the compiler can answer higher-level questions directly. That is why the later posts in this series focus on things like: - why Rust's effect debate points toward a smaller language, not a larger one - why explicitness matters for AI-assisted generation - why a Lean-based compiler creates an opening for proving real systems code - why compiler reports can become a practical optimization loop - what Concrete makes worse in exchange for those properties ## What Concrete is for Concrete is not trying to be the best language for everything. It is aimed at code where hidden behavior is expensive: - firmware - security boundaries - cryptographic software - policy engines - safety-critical components - systems code that may eventually need formal verification These are domains where "what can this function do?" is not a stylistic question but an audit question, sometimes a certification question. For those domains, I think a smaller and more explicit language is a better trade than a more ergonomic and more magical one. ## What Concrete is not for Concrete is not a convenience-first language. It is worse than Rust or Zig at many pleasant things: - resource-heavy code is more verbose because cleanup is explicit - higher-order composition is more explicit because there are no closures with hidden captures - the ecosystem is early - the tooling stack is still immature compared with established languages That cost comes from the design. A later tradeoffs post will make it explicit in more detail. I think those costs are worth paying for some domains and not worth paying for others. Concrete's upsides and downsides come from the same constraints. ## Why Lean matters Concrete's compiler is written in Lean 4. That started as a practical implementation decision, but it has architectural consequences. Every Concrete program becomes native Lean data as it moves through the compiler pipeline. That creates a path, still incomplete but very real, from systems code to proof tooling without stitching together two unrelated ecosystems. It also encourages a language shape that can actually be formalized: small core semantics, explicit effects, explicit ownership, limited implicit machinery. I do not think "written in Lean" is the main reason to care about Concrete. The main reason is the language design itself. But Lean makes the long-term verification story more plausible than it would be otherwise. ## What exists today Concrete is no longer just a sketch. The compiler is a real Lean 4 codebase with a staged pipeline, a substantial test suite, an expanding standard library, contracts in source, proof reports, and audit-oriented report modes. What does not yet exist is just as important: - no fully verified end-to-end kernel connected to the implemented compiler - no mature formatter, package manager, or LSP - no broad adoption story yet The biggest change since this foundation piece was first written is that the proof story has stopped being just a direction. Some functions now carry proof links in the source. If the function changes, the proof can go stale instead of quietly staying green. Reports distinguish a Lean-checked theorem from a compiler-enforced fact, a solver result, an assumption, or a missing proof. The honest claim is still not "the compiler is verified." It is narrower and more useful: selected claims about selected functions can now be proved, reported, diffed, and audited without pretending they are all the same kind of guarantee. So the right way to read this series is not "finished language announcement." It is "a case for a particular design direction, with some parts already working and others still being built." ## Where to go next If you want the rest of the argument, read the series in roughly this order: 1. [Why Concrete Exists](@/series/concrete/2025-12-26-the-concrete-programming-language-systems-programming-for-formal-reasoning/index.md) 2. [The Rust Effects Debate and Concrete's Case for a Smaller Language](@/series/concrete/2026-03-09-rusts-grand-vision-and-concretes-answer/index.md) 3. [The AI Training Data Trap for Programming Languages Has an Exit](@/series/concrete/2026-03-11-ai-wont-save-your-language-unless-you-design-for-it/index.md) 4. [Can I prove Concrete programs in Lean?](@/series/concrete/2026-03-12-proving-systems-code-in-lean/index.md) 5. [When the Compiler Is the Oracle](@/series/concrete/2026-03-20-when-the-compiler-is-the-oracle/index.md) 6. [What Concrete Makes Worse](@/series/concrete/2026-03-24-what-concrete-makes-worse/index.md) 7. [A Fact-Producing Compiler](@/series/concrete/2026-04-09-a-fact-producing-compiler/index.md) If you want the language details rather than the essay version, use [Concrete Spec](@/series/concrete/spec/_index.md). That is the bet behind Concrete: not a more expressive systems language, but a more legible one. A language that constrains the programmer so the compiler can say more that is actually useful. --- ## Series: The Edge of Chaos Complex systems made of many copying, interacting parts can sit near a critical edge, the narrow band between order and chaos. There, small shocks cascade, large events need no large cause, and the calm itself can be a warning. This series builds that idea step by step from simple models, then turns it on markets, flows, and risk management. ### When Risk Models Create Risk *Published: 2025-06-06* > The last essay said the tail is too hard to measure. Jón Daníelsson goes one step further: in finance, the measurement itself changes the thing measured. When everyone uses the same risk model, the model becomes a synchronization device. The answer is a system designed to survive without everyone trusting the same number. URL: https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/ The last essay ended with Taleb's objection. The tail is where the risk is, and the tail is exactly the part of the distribution the data refuses to pin down. If you cannot know the probability of ruin, stop pretending you can. Change your exposure instead. Jón Daníelsson adds a colder institutional version of the same argument. He says something stronger than "risk models are inaccurate": in finance, the act of measuring risk changes the risk. A model works less like a thermometer held up to the weather and more like a rule handed to people who then trade, hedge, deleverage, report, and regulate according to it. The number enters the system it tries to describe. That is the final turn in this series. If everyone copies each other, small shocks cascade. If everyone uses the same risk model, the model itself becomes one of the things they copy. ## I. Risk is not weather Daníelsson's core distinction is simple. Some risks are mostly exogenous. A storm is coming whether you measure it or not. Your forecast can be wrong, but the forecast does not usually create the storm. Financial risk is different. It is endogenous. It is generated by the interaction of people who watch the same prices, use similar models, face similar constraints, and react to one another. A small price move changes measured volatility. The change in measured volatility changes risk limits. The changed risk limits force sales. The forced sales move prices. The price move confirms the model's warning and forces the next sale. The inelastic-markets essay earlier in this series showed flows moving prices against a steep demand curve. This loop has a different trigger: the measured risk that every desk is watching and acting on at the same time. ![A circular feedback diagram. A price fall raises measured risk, rising measured risk makes risk limits bind, binding limits force sales, sales thin liquidity, and thin liquidity feeds back into another price fall.](/images/charts/risk-1-feedback-loop.png) *The risk model sits inside the market. Its output changes behavior, and that behavior changes the market the model is trying to measure.* Run the loop directly. Raise leverage or shared-rule monoculture and the same first shock turns into a larger forced-selling cascade; lower liquidity and each sale hits the price harder. Cranking up monoculture is also the portfolio-insurance lesson from 1987: many balance sheets following the same sell-on-decline rule. {{ risk_feedback() }} This is the Millennium Bridge in market form. When the London bridge opened in 2000, pedestrians felt a tiny sideways sway and unconsciously adjusted their steps. Those synchronized steps amplified the sway, which synchronized the pedestrians further, which amplified it again. No one planned the movement. No outside shock had to keep pushing. The crowd and the bridge made the instability together. Markets do the same thing with balance sheets. ## II. The model works until it matters This is why Daníelsson's 2002 paper, "The Emperor has no Clothes," cuts deeper than an ordinary complaint about Value-at-Risk. The usual criticism is that VaR misses the tails. True, but familiar. Daníelsson's sharper point is that the data used to estimate risk comes mostly from ordinary times, when the system is not yet feeding back on itself. In those times, risk looks measurable. Volatility is low. Correlations behave. Historical losses seem bounded. The model passes its tests. Then the crisis starts, and the object being measured changes character. Risk stops looking like a property of an asset and starts looking like a property of a crowd. People who were diversified discover they all own the same trade. People who thought they had risk limits discover everyone else's risk limits trigger at the same time. The model that looked conservative in the calm becomes a lever in the panic. So the failure is performative as well as statistical. A shared risk model can make the market more fragile by making everyone respond to the same signal in the same way. ## III. The hidden danger is uniformity This is Daníelsson's most useful addition to the story. Leverage matters, but monoculture is the deeper danger. Regulators like common rules because common rules look fair. Same assets, same risk weights, same capital formulas, same stress tests. No one gets special treatment. Everyone is comparable. The system becomes legible. But a legible system can be a synchronized system. If every bank measures risk the same way, every bank sees risk fall in the boom. If every bank sees risk fall, every bank is allowed to expand. If every bank expands, prices rise, losses look even less likely, and the model approves still more balance-sheet growth. Then the sign flips. Measured risk rises, capital becomes scarce, and everyone tries to shrink at once. *Before you look: the same shock hits a hundred institutions running one shared model, then a hundred running different ones. Sketch the system-wide selling in each world.* ![A plot of system-wide selling pressure through time. A red spike shows monoculture, where institutions sell at the same moment. A green, lower, wider curve shows diverse models and constraints spreading the response over time.](/images/charts/risk-2-monoculture.png) *A common model turns many balance sheets into one reflex. Heterogeneous rules spread the same shock across time and reduce the peak pressure.* That is one giant common error, copied across institutions. August 2007 is the clean example. In the second week of the month, quantitative equity funds running similar statistical-arbitrage models all hit the wall at once. As one large fund deleveraged, it pushed down exactly the stocks the others held, whose models then flagged the same risk and forced them to sell too, a domino none of them could see from inside. Goldman's Global Equity Opportunities fund lost nearly a third of its value, and its CFO explained the week as "25 standard deviation moves, several days in a row," events the models said should not happen across the lifetime of the universe. They were not freak draws. They were the signature of many institutions running one kind of model and selling into each other. Amir Khandani and Andrew Lo later reconstructed the mechanism and called it the unwind. The lesson is uncomfortable because it runs against the administrative instinct. A financial system made of heterogeneous institutions, different models, different constraints, different time horizons, different kinds of wrongness, looks messy. It is harder to supervise. It is harder to compare. But the mess is doing work. Diversity breaks synchronization. A market where everyone disagrees is annoying. A market where everyone agrees is dangerous. ## IV. Microprudential is not macroprudential The distinction matters. Microprudential regulation asks whether each institution is safe on its own. Does this bank have enough capital? Is this portfolio inside its VaR limit? Did this desk pass its stress test? Macroprudential regulation asks what happens when all those individually sensible actions are taken together. If every bank sells the risky asset to reduce risk, who buys it? If every model raises margin at once, who supplies liquidity? If every institution tries to become safer by shrinking the balance sheet, does the system become safer or does it crash the price of the thing everyone is selling? This is Minsky in regulatory language. Stability lowers measured risk. Low measured risk invites leverage. Leverage makes the next disturbance more dangerous. When the disturbance arrives, the same risk controls that looked prudent institution by institution can force a collective deleveraging. The solution keeps measurement and demotes it. Stop confusing a bank's measured safety with the system's safety. A rule that makes one institution safer can make the system more fragile if everyone follows it at once. ## V. What a better system looks like Daníelsson's answer is structural, not predictive. First, preserve diversity. Do not force every institution onto the same model, the same risk horizon, the same trading rule, the same capital response. A market needs different kinds of balance sheets the way an ecosystem needs different species. Uniformity is efficient right up to the moment it becomes a cascade. Second, lean against procyclicality. If measured risk is lowest in the boom, that is exactly when buffers should rise, not fall. Capital and liquidity should be built when they look least necessary, because that is when the system is quietly loading itself. Third, build slack. Redundancy, liquidity, circuit breakers, slower feedback loops, and limits on tight coupling all look wasteful in calm periods. That is why they matter. Slack exists to keep normal times from becoming the launch ramp for a crisis. Fourth, treat model outputs as signals, not commands. A risk number should start a conversation, not end one. The moment the number becomes a mechanical instruction, it can become a synchronization device. ## VI. The AI version of the same problem There is a new version of this problem forming. If every bank builds its own small model, they will be wrong in different ways. If every bank and regulator leans on the same handful of foundation models, the errors can become correlated. The model recommends similar hedges, flags similar risks, summarizes similar news, writes similar stress scenarios, and nudges similar decisions across institutions that believe they are acting independently. That is endogenous risk with a faster nervous system. The danger is persuasion, centralization, and shared use. A common machine can become a common reflex. And in a market, a common reflex is how a small move becomes a stampede. ## VII. The final lesson The series began with a market crash that had no cause large enough to explain it. It moved through magnets, sandpiles, bubbles, Hawkes processes, warning signs, and fat-tailed ignorance. Daníelsson gives the institutional moral of the whole thing. Do not build a system whose safety depends on everyone trusting the same fragile number. Taleb's advice is personal: do not stand where the tail lands. Daníelsson's advice is institutional: do not make everyone stand in the same place. The first protects the individual from ruin. The second protects the system from synchronized ruin. Measure anyway. Fit the tail, watch the branching ratio, track the bubble, look for slowing recovery, use every instrument this series has described. Just keep the instruments in their place. They exist to help you build slack, diversity, and convexity before the cascade starts, rather than to certify that the cascade cannot happen. The edge is real. The number is unstable. The system is watching the number. That is why the design matters more than the estimate. {% toolkit() %} What this essay added to the series' instrument panel. - **Exogenous vs endogenous risk**: the storm exists whether or not you measure it; the stampede is made by the people measuring it. Financial risk is mostly the second kind. - **The loop**: a price fall raises measured risk, rising measured risk binds limits, binding limits force sales, sales thin liquidity, and thin liquidity feeds the next fall. The model sits inside the market it measures. - **Monoculture**: a shared risk model is a synchronization device. Diversity of models, constraints, and horizons breaks the synchrony, and the mess is doing work. - **Micro vs macro prudence**: every institution safe on its own is not the same as the system being safe. Ask who buys when every model says sell. - **The design rules**: preserve diversity, raise buffers in the boom, build slack, and treat model outputs as signals rather than commands. Carried forward: the institutional moral of the whole series. The edge is real, the number is unstable, the system is watching the number, so design beats estimation. {% end %} ## Further reading On endogenous risk: - Daníelsson, J., Shin, H. S., and Zigrand, J.-P. (2013). [*Endogenous and Systemic Risk*](https://www.nber.org/books-and-chapters/quantifying-systemic-risk/endogenous-and-systemic-risk). In *Quantifying Systemic Risk*, University of Chicago Press. - Daníelsson, J. (2002). [*The Emperor has no Clothes: Limits to Risk Modelling*](https://doi.org/10.1016/S0378-4266(02)00263-7). Journal of Banking & Finance, 26(7). On liquidity spirals and the leverage cycle: - Brunnermeier, M. K., and Pedersen, L. H. (2009). [*Market Liquidity and Funding Liquidity*](https://doi.org/10.1093/rfs/hhn098). Review of Financial Studies, 22(6). The "liquidity spiral" the simulation above models. - Adrian, T., and Shin, H. S. (2010). [*Liquidity and Leverage*](https://doi.org/10.1016/j.jfi.2008.12.002). Journal of Financial Intermediation, 19(3). - Aymanns, C., and Farmer, J. D. (2015). [*The Dynamics of the Leverage Cycle*](https://ideas.repec.org/a/eee/dyncon/v50y2015icp155-179.html). Journal of Economic Dynamics and Control, 50. A leverage-cycle agent-based model. On the Millennium Bridge: - Strogatz, S. H., Abrams, D. M., McRobie, A., Eckhardt, B., and Ott, E. (2005). [*Crowd Synchrony on the Millennium Bridge*](https://www.nature.com/articles/438043a). Nature, 438(7064). On the 2007 quant quake: - Khandani, A. E., and Lo, A. W. (2011). [*What Happened to the Quants in August 2007? Evidence from Factors and Transactions Data*](https://web.mit.edu/Alo/www/Papers/august07.pdf). Journal of Financial Markets, 14(1). - Goldman Sachs. (2007). [*Goldman Sachs and Various Investors Including C.V. Starr & Co., Inc., Perry Capital LLC and Eli Broad Invest $3 Billion in Global Equity Opportunities Fund*](https://www.goldmansachs.com/pressroom/press-releases/2007/2007-08-13). On the institutional argument: - Daníelsson, J. (2022). [*The Illusion of Control: Why Financial Crises Happen, and What We Can (and Can't) Do About It*](https://yalebooks.yale.edu/9780300265095/the-illusion-of-control/). Yale University Press. On AI as the new monoculture: - Daníelsson, J., and Uthemann, A. (2023). [*On the Use of Artificial Intelligence in Financial Regulations and the Impact on Financial Stability*](https://arxiv.org/abs/2310.11293). arXiv:2310.11293. - Daníelsson, J., and Uthemann, A. (2025). [*Artificial Intelligence and Financial Crises*](https://eprints.lse.ac.uk/128657/). Journal of Financial Stability, 80. --- ### The Limits of Knowing *Published: 2025-06-05* > Every method in this series rests on one number: how close a system sits to its edge. Nassim Taleb spent a career arguing that this is exactly the number you cannot trust. For fat-tailed systems the data needed to pin down the tail converges too slowly, and being honest about your uncertainty fattens the tail further. This is the counterpunch, and where it leaves us. URL: https://federicocarrone.com/series/edge-of-chaos/the-limits-of-knowing/ Six essays in, we have a toolkit. Bouchaud's branching ratio, Sornette's critical time, Gabaix and Koijen's flow multiplier, Scheffer's slowing-down: different instruments, all pointed at the same condition, a system loaded near its edge. This essay is about the catch that has been sitting in the room the whole time. That condition, the distance to the edge, is the single hardest thing to pin down on exactly the kind of system where it matters most. Nassim Taleb built a career on this objection, and it is the strongest case against everything that came before. ## I. Where counting stops working Most of statistics rests on a quiet promise: collect enough data and the average settles down to the truth. Flip a coin a thousand times and the fraction of heads sits near one half. Measure a thousand people and the average height barely moves when you add the next. This is the law of large numbers, and in a thin-tailed world it works beautifully. *Guess first: keep adding fat-tailed observations and watch the running average. Does it settle the way an average of heights does?* In a fat-tailed world it barely works at all. ![Two panels, each a running average against the number of observations. Left, thin-tailed data: the line wobbles at first and quickly flattens to a steady value. Right, fat-tailed data: the line keeps lurching upward in steps, jumping each time a new extreme arrives, never settling.](/images/charts/limits-1-running-average.png) *Left, with thin tails the average settles fast, so a modest sample tells you the truth. Right, with fat tails the average keeps jumping with each new extreme and never settles, so no sample you will ever collect pins it down.* Look at the right panel. When the tail is heavy enough, the running average barely settles. It sits flat for a while, then a single huge observation arrives and yanks it upward, and for the fattest tails this repeats without end. The sample is dominated by its largest member, and the largest member keeps being beaten. For a fat-tailed quantity, the thing you most want to know, the size of the typical large event, is the thing your data refuses to reveal, because the data is always one big surprise away from rewriting it. Taleb's word for this is preasymptotics. The textbook results are about what happens with infinite data. With the finite data you actually have, a fat-tailed world is a place where the worst event so far is never a good estimate of the worst possible, where the mean can be undefined or, when it exists, converge painfully slowly, and where the next observation can dominate everything before it. The tail is where all the risk lives, and the tail is precisely the region you can never sample enough of. ## II. Honesty makes it worse Now suppose you set that aside and insist the world is thin-tailed after all. You fit a nice bell curve. Even then there is a trap, and it is subtle. You never know the bell curve's width exactly. You estimate it from data, and that estimate has error. And the error has its own error, because your method of estimating is itself uncertain, and so on. Taleb and Pasquale Cirillo showed what taking that seriously does. It fattens the tail. ![A log-scale plot of two curves. One, sure of its spread, is a bell curve that drops away steeply. The other, unsure of its spread, sits above it far out in the tails, so the rare extremes are much more likely.](/images/charts/limits-2-errors-on-errors.png) *Start with a bell curve and admit you are unsure of its width. Average over that uncertainty and the result has heavy tails: the rare extremes become far more likely. Being honest about not knowing the parameter turns a thin tail into a fat one.* The mechanism is clean once you see it. Some of the time the true spread is larger than your best guess, and those wide cases throw out extremes far more often than the narrow cases suppress them, because the extremes scale up faster than they scale down. Mix over your uncertainty and the blend is heavy-tailed even though every ingredient was a bell curve. Taleb and Cirillo call the consequence the forecasting paradox: the distribution you should use to predict is heavier-tailed than the one you fit to the past, so the future is structurally more extreme than the data you trained on. Thin-tailed certainty is unreachable from the inside, because honest uncertainty manufactures fat tails on its own. ## III. What this does to the whole series Hold this up against the previous five essays and it cuts deep. Sornette's critical time is estimated by fitting a shape to a price run. Bouchaud's branching ratio is estimated by fitting a self-exciting process to trades, and we already saw two careful groups disagree on whether it rises or holds steady. Scheffer's warning signs are estimated from the variance and autocorrelation of a noisy series. Every one of them is an estimate of how close a system sits to its critical point, and the critical point is exactly where estimates become unstable and sample-hungry, on data that is fat-tailed in the first place. The tools tell you the system is near the edge, and being near the edge is what makes the tools' own readings shaky. Taleb's claim, put plainly, is that the number these methods chase is the number their own setting forbids you from trusting. There is a worthy opponent here, and it is not Sornette. It is Paul Embrechts, the dean of extreme value theory, whose whole discipline is the rigorous estimation of tails: fit the right curve to the exceedances, estimate the tail index, extrapolate past your largest data point in a principled way. To Embrechts, the tail is hard but tractable, and Taleb's pessimism throws away a working toolkit. Taleb's reply is that extreme value theory relocates the problem rather than solving it: now you must estimate the tail index from the few points in the tail, and that estimate is itself unstable and threshold-dependent, which is the errors-on-errors problem wearing a lab coat. That argument is unresolved, and probably unresolvable, because settling it would take exactly the abundant tail data whose absence is the whole point. Drag the threshold in the toy below. The Hill estimate uses fewer points as it moves deeper into the tail, and the same sample can wander across very different risk worlds. The estimator is doing exactly what it is supposed to do; the data are what refuse to sit still. {{ tail_estimator() }} Long-Term Capital Management is the monument to all this. The fund was run by some of the most capable people in the field, including Myron Scholes and Robert Merton, who had just shared the 1997 Nobel for the mathematics of options. Its models were calibrated on years of data in which the trades it favored had reliably converged. In 1998, after Russia defaulted on its debt, they stopped converging all at once. The fund lost more than half its capital in a matter of weeks, and the Federal Reserve had to organize a 3.6 billion dollar rescue by fourteen banks to keep the unwinding from dragging the system down with it. The models were not sloppy. They were fit to a past that did not contain the event that killed them, which is the preasymptotic trap with a body count. ## IV. So what do you actually do You might expect all this to end in a shrug: nothing can be known, so nothing can be done. It does not. You do not have to measure how close the edge is to keep your distance from it, and that gap is where Taleb's practical advice lives. If you cannot know the probability of the tail, stop trying to forecast it and change your exposure instead. Carry redundancy and slack, the very things the efficiency essay said get optimized away. Cut leverage and tight coupling, because they are what turn a local shock into a cascade. Hold cheap insurance against the moves you cannot rule out, and buy it when it is cheap, which is during the calm. Prefer positions that lose a little if you are wrong and gain a lot if the rare event lands, rather than the reverse. None of this requires knowing when, or how close to the edge, or which grain triggers the slide. It only requires believing that the edge is real and probably nearer than it looks. That is the personal resolution. The diagnostics from the earlier essays are not crystal balls and should not be traded as if they were. They are regime detectors, and their honest job is qualitative: to tell you when a market has stopped being an ordinary thin-tailed place and has loaded itself near its critical edge. Sornette's accelerating climb, Bouchaud's branching ratio near one, Scheffer's slowing recovery, all of them are ways of noticing the same condition. And once you have noticed it, you do not try to forecast the crash. You step out of where it would land. Blake painted Newton at the bottom of the sea, so absorbed in the diagram under his compass that he never turns to see the dark rock behind him. The figures in this series, Bouchaud, Sornette, Scheffer, are doing real and beautiful work with the compass. Taleb is the one tapping the shoulder, pointing at the dark. You want both. Measure what you can, and build for what you cannot. The final essay turns that same lesson from the individual portfolio to the financial system itself. {% toolkit() %} This essay does not add an instrument. It adds the error bars on all of them. - **Preasymptotics**: with fat tails, the running average never settles, and the worst event so far is a poor estimate of the worst possible. The tail is where the risk lives, and the tail is the region you can never sample enough of. - **Errors on errors**: admit honest uncertainty about a bell curve's width and the blend comes out fat-tailed. Thin-tailed certainty is unreachable from the inside. - **The forecasting paradox**: the distribution you should predict with is heavier-tailed than the one you fit to the past. - **The unresolved fight**: Embrechts says the tail is hard but tractable. Taleb says the tail-index estimate is the same instability wearing a lab coat. Settling it would take exactly the tail data that does not exist. Carried forward: every reading in this series, the branching ratio, the critical time, the warning signs, is an estimate of distance to the edge, and that distance is the quantity fat tails hide best. Use the instruments as regime detectors, and change your exposure instead of forecasting. {% end %} ## Further reading The statistics of fat tails: - Taleb, N. N. (2020). [*Statistical Consequences of Fat Tails*](https://arxiv.org/abs/2001.10488). STEM Academic Press. - Embrechts, P., Klüppelberg, C., and Mikosch, T. (1997). [*Modelling Extremal Events for Insurance and Finance*](https://link.springer.com/book/10.1007/978-3-642-33483-2). Springer. - Hill, B. M. (1975). [*A Simple General Approach to Inference About the Tail of a Distribution*](https://doi.org/10.1214/aos/1176343247). Annals of Statistics, 3(5). The tail-index estimator behind the simulation above. Errors on errors and the forecasting paradox: - Taleb, N. N., and Cirillo, P. (2019). [*Branching Epistemic Uncertainty and Thickness of Tails*](https://arxiv.org/abs/1912.00277). arXiv:1912.00277. - Taleb, N. N., and Cirillo, P. (2025). [*The Regress of Uncertainty and the Forecasting Paradox*](https://doi.org/10.3390/risks13120247). Risks, 13(12). When a fitted model met its tail: - Federal Reserve History. [*Near Failure of Long-Term Capital Management*](https://www.federalreservehistory.org/essays/ltcm-near-failure). --- ### Why the Calm Is Dangerous *Published: 2025-06-04* > A system heading for a tipping point gives off warning signs in unexpected places. The danger rarely arrives as drama and rising volatility. It hides in the calm. Ecologists learned to read it in lakes and climate, and the same signature shows up before some market crises. This is the measurable cousin of everything in this series. URL: https://federicocarrone.com/series/edge-of-chaos/why-the-calm-is-dangerous/ So far the series has been about why crashes happen and why you cannot read them off their triggers. This one is about the rare practical payoff: the chance of seeing a break coming. A system heading for a tipping point often gives off warning signs, and the odd thing is where they hide. They hide in the calm, in a precise, measurable sense: the quietest system can be the one closest to breaking. ## I. A ball in a valley Picture a ball resting at the bottom of a valley. That is a stable state. Nudge the ball and it rolls back down. The market is in its current mood, the lake is clear, the climate is where it has been, and small disturbances do not change that, because the valley holds the ball in place. Now imagine the valley slowly getting shallower. ![Three panels, each a curve with a ball resting in it. On the left a deep narrow valley with the ball at the bottom. In the middle a shallow flat valley. On the right the valley has almost flattened into a ledge, with the ball perched and about to roll away.](/images/charts/warn-1-ball-in-well.png) *A stable state is a ball in a valley. Far from a tipping point the valley is deep and steep, so a nudged ball snaps back fast. As the system nears the tip, the valley flattens, the ball drifts back slowly and wanders far, and finally the valley gives way and the ball rolls off to another state.* That flattening is what approaching a tipping point looks like from the inside. The state is still stable, the ball still sits in a valley, but the walls that hold it there are weakening. And a system in a flattening valley behaves differently from one in a steep valley, in ways you can see without knowing anything about its inner workings. ## II. Two things you can measure *Before reading on, guess: the same random knocks hit a ball in a deep valley and a ball in a shallow one. What two differences would show up in a recording of each ball's position?* Watch a ball in a deep valley. Knock it and it shoots back to the bottom. Watch a ball in a shallow valley. Knock it by the same amount and it eases back slowly, taking its time, and meanwhile the ordinary random jostling pushes it further from the bottom because nothing is pulling it back hard. Two things change as the valley flattens, and both show up in a plain time series. ![Two panels. Left, two wandering lines under the same random jostling: one stays in a tight band, the other swings wider and moves more slowly. Right, two curves climbing as a system nears its edge, one for the size of the swings and one for how slowly the system recovers.](/images/charts/warn-2-recovery.png) *Left, the same size of random knocks produces tight quick wiggles far from the edge and wide slow wandering near it. Right, as the valley flattens, both the size of the swings (variance) and the sluggishness of recovery (autocorrelation) climb together.* The first is that recovery slows down. The system takes longer to shrug off a disturbance, so each moment looks more like the one just before it. Statisticians measure that as rising autocorrelation. The second is that the swings get wider, measured as rising variance. Slower recovery and wider swings, arriving together, are the fingerprint of a system approaching a transition. The whole effect has a name: critical slowing down. The reason this is useful is that you can detect it from the outside. You do not need a model of the lake or the market or the climate. You watch the wandering, and if recovery is slowing and swings are widening, the valley is flattening, whatever the valley is made of. The toy below keeps the shocks the same and changes only the valley. As the valley flattens, the line wanders further and remembers its last position longer. {{ slowing() }} ## III. Lakes, climate, and the body This did not begin as a market idea. It came from ecology, and the leading figure is Marten Scheffer. Shallow lakes have two states. Clear water with plants on the bottom, and green water choked with algae. A lake can flip from one to the other quite suddenly when nutrients build up past a threshold, and the flip is hard to reverse once it happens. Scheffer and his colleagues showed that the flip is announced in advance. In the months before a lake tips, it recovers more slowly from disturbances and its measurements grow more autocorrelated and more variable, exactly the signature above. The striking thing is how far the same signature travels. Scheffer's group and others have found critical slowing down ahead of shifts in regional climate, ahead of collapses in fish and wildlife populations, and even ahead of transitions in the body, where the warning shows up in breathing before an asthma attack or in brain activity before an epileptic seizure. Different machinery, the same statistical tell. This is the universality from the first essay doing its work once more. Near a tipping point the details fall away and only the slowing-down remains, which is exactly why a method built for lakes can be pointed at a market. ## IV. The calm that builds the storm Point it at markets and two readings emerge, and they seem to contradict each other until you separate the timescales. The first is the direct one. Before some financial crises, you can find the early-warning signature in the data: correlations across assets rising, prices mean-reverting more slowly, swings widening during the quiet stretch before the break. The valley is flattening, and the flattening is visible in the calm. The second comes from economics, and it is older. Hyman Minsky's phrase was that stability is destabilizing. A long calm makes people confident, and confident people take on leverage and risk precisely because nothing has gone wrong lately. That quiet accumulation of leverage is what carries the system toward its edge. The calm helps build the storm. You can watch both readings at once in the volatility blow-up of early 2018. Through 2017 American stocks were about as calm as they have ever been, with the VIX, the market's fear gauge, grinding down to record lows near nine and staying there for months. That stillness pulled enormous sums into bets that it would continue, the popular ones being products that pay off a little each quiet day and short volatility to do it. That is the Minsky clock: the calm itself built a tower of crowded, leveraged positions all wagering on more calm. Then on February 5, 2018 a roughly four percent drop in stocks, sharp but hardly historic, more than doubled the VIX in a single session. The crowded short-volatility bets detonated, one widely held inverse-volatility note lost about nine-tenths of its value overnight and was wound down within days, and the unwind fed back into the market it was supposed to be hedging. A modest grain, a pile loaded to the brink, a cascade. These look opposed only if you forget there are two clocks. On the slow clock, a long calm builds fragility, in Minsky's sense, by lulling everyone into loading up. On the fast clock, right as the system approaches its tipping point, the calm turns subtly twitchy, recovery slowing and swings widening, in Scheffer's sense. The long quiet loads the system; the late, strange quiet signals that it is nearly done loading. Both say the same uncomfortable thing: calm is often where the risk is quietly being made. ## V. How much to trust it Do not oversell this, because the same fragility that haunts the rest of the series haunts the warning signs too. They are noisy. They cry wolf, flagging transitions that never come. They are frequently clean only in hindsight, once you already know where the crash was and went looking for the run-up. And measuring how close a system sits to its tipping point is the same unstable, sample-hungry estimate that the next essay is entirely about. So treat critical slowing down as a yellow light, not a clock. It does not give you the hour the ground gives way, only that the ground is getting soft. That is still worth a great deal, because the instinct it corrects is the dangerous one: reading a long quiet as safety. The calmest market, the one where everyone agrees and nothing seems to move, is exactly the one whose valley may be flattening under it. Which leaves the question this series has been circling from the start. Every method here depends on one number, how close the system is to its edge, and that number is the hardest thing in the world to measure on a fat-tailed system. The next essay is about why, and about the man who built a career insisting on it. {% toolkit() %} What this essay added to the series' instrument panel. - **Variance**: the width of the wandering. It rises as the valley flattens, because nothing pulls the ball back hard. - **Autocorrelation**: how much each moment resembles the one before. It rises as recovery slows. - **Critical slowing down**: both rising together, the fingerprint of a flattening valley, readable from the outside with no model of the system's innards. - **Two clocks**: on Minsky's slow clock, a long calm builds leverage and fragility. On Scheffer's fast clock, the late calm turns subtly sluggish and wide. The long quiet loads the system; the strange quiet says it is nearly done loading. - **The grade**: a yellow light, not a clock. Noisy, prone to crying wolf, cleanest in hindsight. Carried forward: the series' one practical early warning, and the reason "nothing is moving" is not the same as "nothing is wrong." {% end %} ## Further reading Critical slowing down and early-warning signals: - Scheffer, M., Bascompte, J., Brock, W. A., et al. (2009). [*Early-warning Signals for Critical Transitions*](https://doi.org/10.1038/nature08227). Nature, 461. - Scheffer, M., Carpenter, S. R., Lenton, T. M., et al. (2012). [*Anticipating Critical Transitions*](https://doi.org/10.1126/science.1225244). Science, 338(6105). Whether markets actually show it before a crash: - Diks, C., Hommes, C., and Wang, J. (2019). [*Critical Slowing Down as an Early Warning Signal for Financial Crises?*](https://doi.org/10.1007/s00181-018-1527-3). Empirical Economics, 57(4). - Guttal, V., Raghavendra, S., Goel, N., and Hoarau, Q. (2016). [*Lack of Critical Slowing Down Suggests that Financial Meltdowns Are Not Critical Transitions, yet Rising Variability Could Signal Systemic Risk*](https://doi.org/10.1371/journal.pone.0144198). PLOS ONE, 11(1). Stability that breeds instability: - Minsky, H. P. (1992). [*The Financial Instability Hypothesis*](https://www.levyinstitute.org/pubs/wp74.pdf). Levy Economics Institute, Working Paper No. 74. The 2018 short-volatility break: - Credit Suisse AG. (2018). [*Credit Suisse AG Announces that The Nasdaq Stock Market Intends to Suspend Trading of the XIV ETNs Following the Market Close on February 15, 2018 and Institute Delisting Proceedings*](https://www.prnewswire.com/news-releases/credit-suisse-ag-announces-that-the-nasdaq-stock-market-intends-to-suspend-trading-of-the-xiv-etns-following-the-market-close-on-february-15-2018-and-institute-delisting-proceedings-300599025.html). - U.S. SEC. (2021). [*In the Matter of S&P Dow Jones Indices LLC*](https://www.sec.gov/files/litigation/admin/2021/33-10943.pdf). Securities Act Release No. 10943. --- ### What Actually Moves Prices *Published: 2025-06-03* > The series has argued that markets move themselves, but there is now a clean mainstream number for that claim. Gabaix and Koijen estimate that one dollar flowing into the stock market raises aggregate market value by about five dollars. Bouchaud ties the same result to microstructure and latent liquidity. Prices are news, flows, and market inelasticity made visible. URL: https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/ The last essay put a number on reflexivity. A branching ratio near one says the market spends most of its time reacting to itself, with one trade setting off the next. That is already a hard blow to the tidy picture where prices mostly digest outside news. But there is an even cleaner number, and it comes from mainstream economics rather than econophysics. Xavier Gabaix and Ralph Koijen asked a blunt question: if one dollar flows into the aggregate stock market, how much does the market's total value rise? The old intuition says the answer should be close to one dollar, or maybe less. A deep market should absorb flows. Prices should move mostly because expected cash flows, discount rates, or risk premia changed. Buying pressure should be a sideshow. Their estimate is about five dollars. That is the most important recent bridge into the whole Edge of Chaos story. The market is an inelastic object, far from the vast ocean of the old intuition. Push it with flows and the whole level moves. ![Two panels. Left, a bar chart comparing the deep-market intuition, where one dollar of flow creates one dollar of market value, with the inelastic-market estimate, where one dollar creates about five dollars. Right, a curve showing square-root price impact rising quickly at first and then flattening as order size grows.](/images/charts/inelastic-1-flow-impact.png) *The new macro number and the older microstructure law tell the same story. At the aggregate level, flows have a multiplier. At the trade level, latent liquidity makes impact nonlinear rather than harmless.* ## I. The missing lever Standard finance begins with a comforting picture. Prices move because information arrives. A company reports earnings, rates change, inflation surprises, a war starts, a technology improves. Investors update beliefs, and prices adjust. That picture has a problem. The biggest price moves often have no news large enough to explain them. We started the series there, with 1987. If prices only reflect fundamentals, a crash without a matching headline is an embarrassment. The inelastic-markets hypothesis changes the question. Maybe the missing lever is flow rather than hidden news. Gabaix and Koijen's argument starts from who actually holds the market. A large share of equity is held by institutions with mandates: index funds, pension funds, insurers, target-date funds, balanced funds, and other allocators whose equity share does not swing freely with every price change. They do not act like infinitely elastic value investors waiting to absorb any flow at almost the same price. Many are constrained, benchmarked, rule-bound, or slow. If the marginal holders are inelastic, then a flow into equities does not get absorbed quietly. Prices have to move a lot to make the market clear. That is the one-line result: in aggregate, one dollar of buying can raise market value by about five dollars, with estimates running from roughly three to eight depending on the method. The center of gravity is the same. Flows have a multiplier. ## II. Why this is bigger than "money moves markets" Every trader already knows flows matter. Gabaix and Koijen turned that intuition into a macro-finance object. They used granular instrumental variables, exploiting the fact that large institutions receive flows for reasons that are partly idiosyncratic to the institution rather than to the whole market. Those flow shocks let them estimate how much the aggregate market moves when money enters or leaves. That matters because it turns a trading-desk slogan into a parameter. The market's demand curve has a slope. Its elasticity is low. Aggregate prices are sensitive to who has to buy, who has to sell, and what the news says. ![A chart of price against net buying flow with two lines rising from the origin. A shallow dashed line is the old elastic intuition, where a given flow barely moves price. A steep blue line is the inelastic estimate, where the same flow produces a large price move. A vertical marker shows one flow size hitting both lines, and a double arrow marks the extra price move the steepness adds.](/images/charts/inelastic-2-demand-curve.png) *Low elasticity is just a steep demand curve. The same dollar of flow that barely moves a deep market moves an inelastic one by a multiple, because there are not enough willing counterparties to absorb it near the current price.* Move the same idea around directly. Keep the flow fixed and steepen the market, or keep the market fixed and push more flow through it. {{ inelastic() }} This is why the result is bigger than a clever empirical paper. It shifts the ontology of price. A price is not simply the market's best estimate of discounted fundamentals. It is also the point where constrained balance sheets, mandates, flows, and liquidity meet. In that world, a price can move because money moved. ## III. Bouchaud connects the scales Jean-Philippe Bouchaud immediately recognized the shape of the result, because it is the macro version of what market microstructure had been measuring for years. *Guess first: double the size of a large order. Does its price impact double?* At the trade level, large orders move prices in a concave pattern that cannot be reduced to information. A metaorder, one large parent order split into many smaller trades, tends to follow the square-root law of price impact: the price move grows roughly with the square root of the order's size relative to market volume. Double the order and the impact rises, but less than double. Bouchaud's latent-liquidity picture explains why. The visible order book is thin, but the deeper market contains hidden intentions: people willing to buy lower, sell higher, or change their mind as the price moves. Liquidity is latent, adaptive, and fragile rather than a fixed pile sitting there waiting. His reply to Gabaix and Koijen says the macro multiplier and the micro impact law are not separate facts. They are the same inelasticity seen from different distances. At the small scale, an order walks through latent liquidity and moves the price. At the large scale, institutional flows push against a market whose demand is not elastic enough to absorb them. The bridge matters. It means the series' two languages, the physics of interacting agents and the economics of aggregate markets, are describing the same object. ## IV. What this adds to reflexivity The Hawkes-process essay measured how much markets trigger themselves over time. One event breeds another. Activity clusters. The branching ratio sits near the runaway point. Inelastic markets measure a different axis: how far prices move when flow arrives. Hawkes gives you the chain reaction. Inelasticity gives you the leverage of each link in the chain. Put them together and the mechanism is plain. A trade triggers more trades. Those trades hit an inelastic market. Because the market is inelastic, the price moves more than a deep-market story would predict. That price move changes leverage, triggers mechanical rebalancing, and shifts what other strategies are about to do, which sends new flow back into the market. This is the market doing work on itself. The final essay in the series follows one sharp version of this loop, where the thing that closes it is not the flow but a risk model that everyone is watching at the same time. It also clarifies why passive and benchmarked capital can have strange system effects. Indexing may be cheap and rational for the individual investor, but at the system level it can reduce the pool of investors willing to lean hard against flows. If more money is managed by rules that mechanically allocate rather than actively absorb dislocations, the aggregate demand curve can steepen. The market becomes easier to push. That does not make indexing bad. It means the market structure has changed, and the price impact of flows becomes a first-order object. ## V. The practical consequence The practical lesson leaves fundamentals in the picture. The lesson is that fundamentals share the job with flows, mandates, rebalancing rules, leverage constraints, dealer balance sheets, and liquidity. Sometimes those forces enter quietly. Sometimes they dominate. That explains why the same piece of news can produce a small move in one regime and a huge move in another. The news is not the whole cause. It is the grain. The slope is the market's inelasticity, crowding, leverage, and feedback. It also explains why "no news" is not the same as "no cause." A pension rebalance, a volatility-control fund cutting exposure, a dealer hedging options, a fund meeting redemptions, an index inclusion, a margin call, an ETF creation or redemption: these are not headlines about future cash flows, but they are real mechanical forces on price. If enough of them point the same way, the market moves. ## VI. The new center of gravity This is why inelastic markets deserve a place in this series. The old heterodox claim was that prices and volatility are driven mostly from inside the market. For years that sounded like physics envy or trader folklore. Gabaix and Koijen gave it a mainstream macro number. Bouchaud connected that number to microstructure. That convergence is new and important. The series started with crashes without causes. It then built models where crowds flip, sandpiles avalanche, bubbles accelerate, and trades trigger trades. Inelastic markets add the missing empirical bridge: when flows hit the aggregate market, prices move by a multiple of the flow. The edge is partly a statement about a market whose demand curve is steeper than the old theory allowed. So the question is no longer "what news moved the market?" The better question is: what flow hit what liquidity, in what regime, with what feedback waiting behind it? {% toolkit() %} What this essay added to the series' instrument panel. - **The flow multiplier**: one dollar flowing into the aggregate stock market raises total market value by about five dollars, with estimates running roughly three to eight. The market's demand curve is steep, not deep. - **The square-root law**: a large order's price impact grows like the square root of its size relative to market volume. Double the order and the impact rises, but by less than double. - **Latent liquidity**: market depth is not a pile of resting orders but hidden, adaptive intentions, which is why impact is concave and liquidity is fragile. - **The bridge**: the macro multiplier and the micro impact law are the same inelasticity seen from different distances. Carried forward: Hawkes gave the chain reaction; the multiplier gives the leverage of each link. Flow times inelasticity is how a small trigger becomes a large move. {% end %} ## Further reading The macro result: - Gabaix, X., and Koijen, R. S. J. (2021). [*In Search of the Origins of Financial Fluctuations: The Inelastic Markets Hypothesis*](https://www.nber.org/papers/w28967). NBER Working Paper No. 28967. - Gabaix, X., and Koijen, R. S. J. (2024). [*Granular Instrumental Variables*](https://doi.org/10.1086/728743). Journal of Political Economy, 132(7). The estimation method behind the multiplier. The microstructure interpretation: - Bouchaud, J.-P. (2022). [*The Inelastic Market Hypothesis: A Microstructural Interpretation*](https://arxiv.org/abs/2108.00242). Quantitative Finance, 22(10). The impact-law background: - Toth, B., Lempérière, Y., Deremble, C., de Lataillade, J., Kockelkoren, J., and Bouchaud, J.-P. (2011). [*Anomalous Price Impact and the Critical Nature of Liquidity in Financial Markets*](https://arxiv.org/abs/1105.1694). Physical Review X, 1. - Bouchaud, J.-P., Farmer, J. D., and Lillo, F. (2009). [*How Markets Slowly Digest Changes in Supply and Demand*](https://arxiv.org/abs/0809.0822). In *Handbook of Financial Markets: Dynamics and Evolution*, Elsevier. The canonical review of impact and latent liquidity. - Bouchaud, J.-P., Bonart, J., Donier, J., and Gould, M. (2018). [*Trades, Quotes and Prices: Financial Markets Under the Microscope*](https://doi.org/10.1017/9781316659335). Cambridge University Press. The book-length treatment of impact and latent liquidity. - Maitrier, G., Loeper, G., Kanazawa, K., and Bouchaud, J.-P. (2025). [*The "Double" Square-Root Law: Evidence for the Mechanical Origin of Market Impact*](https://arxiv.org/abs/2502.16246). arXiv:2502.16246. Tokyo Stock Exchange data showing impact is mechanical, not informational. Why arbitrage does not flatten the demand curve: - Shleifer, A., and Vishny, R. W. (1997). [*The Limits of Arbitrage*](https://doi.org/10.1111/j.1540-6261.1997.tb03807.x). Journal of Finance, 52(1). --- ### Reflexivity by the Numbers *Published: 2025-06-02* > Everyone agrees markets react to themselves. The question is how much. A statistical tool built for earthquakes turns that vague idea into a single number: the fraction of market activity that is the market reacting to its own moves rather than to outside news. The number turns out to be close to the level where a chain reaction would run away. URL: https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/ Three essays in, the same idea keeps surfacing in different words: markets react to themselves. Prices move because prices moved, and the big moves arrive with no outside cause. George Soros built a whole investing philosophy on this and gave it a name, reflexivity. Markets, he said, act on their own reflection, and that reflection feeds back into the world. It is a good story, and the trouble with a good story is that it explains everything and predicts nothing until someone turns it into a number. This essay is about the number: how much of what a market does is the market answering itself, and how much is it answering real news from outside. You can actually measure that, and the tool comes from an unlikely place, the study of earthquakes. ## I. Aftershocks After a big earthquake come aftershocks. The first quake changes the stress in the rock around it, which makes more quakes more likely, which change the stress again. One event breeds others. Seismologists needed a way to describe events that trigger their own offspring, and in 1971 a statistician named Alan Hawkes wrote down the math for it. The idea is simple enough to say in a sentence. Normally events arrive at some steady background rate. In a Hawkes process, every event that happens temporarily raises the chance of the next one, and that bump fades over time. Each arrival is a little spark that makes more sparks briefly more likely. Quakes have aftershocks. Emails trigger replies that trigger more emails. And trades, it turns out, trigger trades. Compare what that looks like to events that genuinely do not talk to each other. ![Two stacked panels over the same stretch of time. The top shows independent events as evenly scattered ticks. The bottom shows self-exciting events bunched into tight clusters, with a red line spiking up after each event and decaying between them.](/images/charts/refl-1-hawkes-vs-poisson.png) *Top, independent events: scattered evenly, no memory. Bottom, self-exciting events: each one lifts the chance of the next, so they arrive in bursts. The red line is that rising and falling chance. Markets look like the bottom panel, not the top.* The bottom panel is what real market activity looks like, and it has a familiar name on trading desks: volatility clustering. Calm begets calm and turbulence begets turbulence, not because the news arrives in clusters but because each move provokes the next. That clustering is the visible fingerprint of a market exciting itself. Run the branching version yourself. One outside spark arrives first; every event it triggers can trigger more. Slide the branching ratio toward one and watch the families stretch out. {{ hawkes() }} ## II. The one number Hawkes processes come with a single number that measures exactly what Soros was hand-waving at. It is called the branching ratio, written $n$, and it answers one question: on average, how many further events does each event directly trigger? If $n$ is small, a spark fizzles. One event triggers a fraction of an event, which triggers a smaller fraction, and the chain dies almost at once. If $n$ is close to one, a spark nearly sustains itself: each event triggers almost a full event in turn, the chain runs long, and the system hangs on the edge of a runaway. At $n$ equal to one, a single event sets off a chain that, on average, never ends. *Guess first: at $n = 0.9$, how many events does one outside spark ultimately cause, counting the whole family of descendants?* There is a clean formula for how big a chain one trigger sets off, counting all its descendants: the average family size is $1/(1-n)$. The closer $n$ creeps to one, the more violently that blows up. ![Two panels. Left, a curve of average family size against the branching ratio, rising gently and then shooting toward infinity as the ratio nears one, with a band marking where markets are measured. Right, a single bar split into a small slice from outside news and a large slice set off by other trades.](/images/charts/refl-2-branching.png) *Left, one trigger's total chain, 1/(1-n), exploding as n approaches one. Right, what the measurements imply: at a branching ratio around nine-tenths, most of what a market does is the market reacting to itself.* So the vague idea has become a dial. Turn $n$ from zero to one and you move a market from "mostly responding to the outside world" to "mostly talking to itself, one spark from a runaway." ## III. What the dial reads Two groups measured it, both fitting Hawkes processes to high-frequency futures data, and the answer is the interesting part. Vladimir Filimonov and Didier Sornette, working with E-mini S&P 500 futures, found the branching ratio high and climbing over time, with the share of activity driven by genuine outside news falling from roughly a third in the late 1990s to well under that by the late 2000s. Their reading: markets have become steadily more reflexive as trading automated, and the rising endogeneity is a warning sign, a system drifting toward the runaway point. Stephen Hardiman, Nicolas Bercot, and Jean-Philippe Bouchaud reanalyzed the same kind of data and pushed back on the trend while agreeing on the level. Their estimate put the branching ratio even closer to one, near 0.9 and above, and roughly stable from 1998 to 2011 rather than climbing. Their reading: markets are not drifting toward criticality, they have been sitting right next to it the whole time. Notice that the two groups disagree about the trend and agree about the headline. Whether it is rising or steady, the number is high. Something like nine in ten of the events that move a market are the market responding to its own activity, not to news from the world. The outside, the actual information about companies and economies that prices are supposed to be tracking, accounts for a small slice. The rest is the echo. ## IV. Why this matters This is the measurement under everything the earlier essays asserted. When the first essay said markets sit near a critical point, this is the evidence: a branching ratio next to one is what near-critical self-excitation looks like in trade data. When the second said markets are sandpiles that self-organize to the edge, this is the slope, measured. When the third described bubbles as runaway feedback, this is the feedback, quantified. It also explains the strangest market events of all, the flash crashes, where prices collapse and recover in minutes with no news to speak of. May 6, 2010 is the cleanest case. By the SEC and CFTC's joint account, a single fund began selling 75,000 E-mini S&P 500 futures contracts, about 4.1 billion dollars worth, through an algorithm instructed to keep pace with 9 percent of trading volume and to ignore price and time. In a market primed to amplify, each execution triggered the next, the Dow fell roughly 998 points, about 9 percent, in minutes, and then recovered most of it within the half hour. No news arrived to justify any of it. That is an avalanche in a market loaded to the brink, the Hawkes picture playing out in real time: one ordinary sell program whose descendants ran away. ## V. The catch Be careful about how hard to lean on the exact number, because the disagreement between the two groups is not noise. It comes from choices in the model: how you assume the triggering fades over time, what window you fit, how you handle the fastest timescales. Change those and the branching ratio moves, which is why one careful group reads a rising trend and another reads a flat line from the same kind of data. That should make you suspicious in a specific way. The branching ratio is telling us the market sits close to a critical point, and "close to a critical point" is precisely where every estimate becomes unstable and sample-hungry. The tool says the system is near the edge, and being near the edge is what makes the tool's own reading shaky. We have measured that the market is reflexive, and the measurement inherits the fragility of the thing it measured. That tension runs straight into the rest of the series. The next essay asks how hard flows move prices once the chain reaction starts. Then we ask what the quiet stretches can tell you before a cascade, and finally press the hardest question of all: if the number you most want, how close the market is to its edge, is the number you can least trust, what is left to do? {% toolkit() %} What this essay added to the series' instrument panel. - **$n$**: the branching ratio, the average number of further events each event directly triggers. Soros's reflexivity turned into a dial. - **Family size**: one outside spark causes $1/(1-n)$ events in total, counting all descendants. At $n = 0.9$ that is ten; at $n = 1$ the chain on average never ends, which is criticality written in event form. - **The split**: roughly a fraction $1 - n$ of market activity is answering the outside world. With the measured $n$ near $0.9$, about nine in ten events are the market answering itself. - **The caveat**: the exact reading depends on fitting choices, how the triggering fades, what window, which timescales. That is why careful groups disagree about the trend while agreeing the level is high. Carried forward: the first measured distance to the edge, and the warning that the measurement inherits the fragility of the thing it measures. {% end %} ## Further reading Where the idea comes from: - Soros, G. (2013). [*Fallibility, Reflexivity, and the Human Uncertainty Principle*](https://doi.org/10.1080/1350178X.2013.859415). Journal of Economic Methodology, 20(4). The self-exciting model and its use in markets: - Hawkes, A. G. (1971). [*Spectra of Some Self-Exciting and Mutually Exciting Point Processes*](https://doi.org/10.1093/biomet/58.1.83). Biometrika, 58(1). - Bacry, E., Mastromatteo, I., and Muzy, J.-F. (2015). [*Hawkes Processes in Finance*](https://arxiv.org/abs/1502.04592). Market Microstructure and Liquidity, 1(1). Measuring how reflexive markets are: - Filimonov, V., and Sornette, D. (2012). [*Quantifying Reflexivity in Financial Markets: Toward a Prediction of Flash Crashes*](https://arxiv.org/abs/1201.3572). Physical Review E, 85. - Hardiman, S. J., Bercot, N., and Bouchaud, J.-P. (2013). [*Critical Reflexivity in Financial Markets: a Hawkes Process Analysis*](https://arxiv.org/abs/1302.1405). European Physical Journal B, 86. The 2010 Flash Crash: - U.S. SEC and CFTC (2010). [*Findings Regarding the Market Events of May 6, 2010*](https://www.sec.gov/files/marketevents-report.pdf). - Kirilenko, A., Kyle, A. S., Samadi, M., and Tuzun, T. (2017). [*The Flash Crash: High-Frequency Trading in an Electronic Market*](https://doi.org/10.1111/jofi.12498). Journal of Finance, 72(3). The academic anatomy of the cascade. --- ### Faster Than Exponential: Can You See a Crash Coming? *Published: 2025-06-01* > The last essay said big cascades are built in and the trigger tells you nothing. Didier Sornette disagrees, at least about the biggest ones. He argues a bubble grows faster than exponentially toward a finite-time singularity, leaves a telltale wobble on the way up, and that this makes some crashes partly foreseeable. This is the optimistic case, and its limits. URL: https://federicocarrone.com/series/edge-of-chaos/faster-than-exponential-can-you-see-a-crash-coming/ If you believe the last two essays, prediction is mostly a fool's errand. Crashes come from inside, the biggest moves have no cause worth the name, and looking for the grain that set off the avalanche means looking for something that was never special. Take that seriously and the only sane move is defense: carry slack, cut leverage, expect the slide. Didier Sornette spent a career arguing that this gives up too soon. Not for every crash, but for one particular kind, the blow-off top at the end of a bubble. His claim is that a bubble has a shape, that you can see the shape while it is still forming, and that the shape tells you something, with all the usual hedges, about when it will break. This is the optimistic case in the series, and it deserves a fair hearing, including the places it falls apart. ## I. Two kinds of growth Start with the difference between fast and dangerous. Exponential growth is what compound interest does. A fixed percentage each period, so the thing doubles on a regular schedule. It looks dramatic, but the growth rate is constant. A savings account, a healthy company, a population with steady birth rates: all exponential, all sustainable in the sense that nothing about the rate is speeding up. A bubble is something else. In a bubble the growth rate itself grows. Prices rise, and the rise pulls in buyers who push prices up faster, which pulls in more buyers still. The price rises, and the speed of the rise rises with it. This is faster-than-exponential growth, and it has a strange property: it does not head toward infinity in the far future, it heads toward infinity at a specific finite time. Mathematicians call that a finite-time singularity. In a market it cannot literally happen, so what the singularity really marks is the moment the trend has to break. *Before you look: both curves below start out rising together. What single chart trick would tell healthy compounding apart from a bubble?* ![Two panels. Left, on an ordinary scale, an exponential curve and a faster-than-exponential curve start together; the faster one bends upward and shoots toward a vertical wall at a fixed time. Right, on a log scale, the exponential is a straight line while the faster-than-exponential one still curves upward.](/images/charts/fte-1-superexponential.png) *Exponential growth has a constant rate, so on a log scale it is a straight line. Faster-than-exponential growth keeps curving up even on a log scale, and it runs into a wall at a finite time. That wall is the danger.* You can see that shape in any real bubble. The Nasdaq Composite more than doubled in 1999 alone, kept accelerating into a peak of 5,048.62 on March 10, 2000, and then gave back 78 percent of its value over the next two and a half years. ![A line through four dated points: the Nasdaq Composite at the end of 1998 and 1999, its March 2000 peak, and its 2002 trough, rising ever more steeply and then collapsing.](/images/charts/realdata-2-nasdaq-dotcom.png) *The dot-com bubble in the Nasdaq Composite, from a few verified closes. The climb steepens toward the peak, the mark of faster-than-exponential growth, and the fall gives back years of gains.* The log-scale panel on the right is the quick test. Steady compounding is a straight line there. A bubble keeps bending upward, because the rate keeps rising, and that upward bend is the fingerprint of the positive feedback that cannot last. ## II. The wobble that gives it away Here is where Sornette goes past everyone else. He argues that the climb toward the singularity is not smooth. It carries oscillations, and the oscillations speed up as the critical time approaches, squeezing closer and closer together near the end. ![A price path that rises with growing speed while oscillating, the swings getting faster and tighter as it nears a dashed vertical line marked as the predicted crash time, after which it drops.](/images/charts/fte-2-logperiodic.png) *The bubble accelerates, and the wobbles on the way up get faster as the end nears. Sornette's model fits this shape to estimate the crash time. The wobbles are the part that is supposed to let you read the clock.* The technical name is a log-periodic power law, and the intuition behind the wobble is that the herding has a kind of rhythm, a discrete echo that gets compressed in time as the system tightens toward its critical point. You do not need the machinery. The claim is simply that a forming bubble has both an accelerating trend and an accelerating wobble, and that fitting both at once pins down the critical time better than the trend alone. Sornette built an entire research program around this, the Financial Crisis Observatory at ETH Zurich, dedicated to fitting these shapes to live markets and logging the forecasts in advance. Move the shape yourself. More feedback bends the path harder; more wobble makes the clock more visible; moving the critical time shifts the wall. The point is to see the geometry, not to pretend the date is cleanly measurable from real data. {{ lppls() }} ## III. Dragon kings and black swans This is a direct quarrel with Taleb, and Sornette picked it on purpose. Taleb's black swan is an event so far out in the tail that it is effectively unpredictable. In the language of the earlier essays, it sits on the power-law line: rare, large, but the same kind of thing as a small event, just a more extreme draw from the same distribution. Nothing about it is special except its size, so nothing lets you see it coming. Sornette's answer is the dragon king. His claim is that the very largest events in some systems are not on the line. They are a separate population, generated by a different mechanism, the runaway feedback of a bubble reaching its critical point. And because they have their own mechanism, they leave their own signature, which means they are partly foreseeable in a way a black swan is not. ![A log-log plot of event size against how often it happens. Most points fall on a straight power-law line and are labeled black swans, just rare draws. A few large points sit well above the extrapolated line and are labeled dragon kings, with their own mechanism.](/images/charts/fte-3-dragon-king.png) *A black swan sits on the line: a rare draw from the same law. A dragon king sits above the line: too big and too frequent to belong to that law, the mark of a separate mechanism. Sornette's bet is that dragon kings, unlike black swans, give warning.* So the disagreement is sharp and clean. Taleb says the giant events are unforeseeable and you should build robustness. Sornette says a subset of giant events announce themselves and you can act on the warning. They cannot both be fully right, and the fight is more useful than either position alone. ## IV. How much to believe I find the picture compelling and I would not trade on it, and those two things are not in tension. The honest reading is that Sornette is on strong ground about the mechanism and much weaker ground about the forecast. That a bubble is faster-than-exponential feedback heading for a break is convincing and well evidenced. That you can read the date of the break off the wobbles is far shakier. The critical time the model returns is not a date but a wide, fuzzy probability, and Sornette himself frames the crash as a random event whose odds rise as you approach. The fits look gorgeous after the fact. The live, pre-registered record is thinner and contested, and it suffers the usual selection problem, where the hits get remembered and the misses get explained. A bubble can also deflate without a crash, sliding back down instead of snapping, so the singularity is a hazard that can be avoided, not a destiny. Sornette's group did put this on the record rather than only claiming hits afterward. In 2009 they ran what they called the Financial Bubble Experiment, posting sealed forecasts in advance and revealing them only later so the calls could not be quietly edited. One flagged the Shanghai Composite as a faster-than-exponential bubble and named a crash window of July 17 to 27, 2009. The index stalled through late July and fell about 20 percent over the first half of August, missing the stated window, though the index did fall sharply a few weeks later. They have pointed to similar advance reads on the 2008 oil spike and the mid-2000s US housing peak. The hits are real, the misses are too, and telling them apart ahead of time is exactly what the pre-registration was built to force. None of that kills the idea. It just sets the right expectation. The signature is real. Whether the signature is tradeable is a different and much harder claim, and Sornette is on much firmer footing for the first than the second. ## V. What it is good for Used correctly, this works as a regime detector rather than a market-timing tool, and that is still worth a great deal. If a market is climbing faster than exponentially, with the trend bending up even on a log scale, you are in a positive-feedback regime that, by its own internal logic, has to end, and probably end sharply. You cannot say when. You can say that the distribution of what happens next has gone one-sided, that the crash hazard is elevated, and that this is the time to act like the previous essays told you to: trim, hedge, hold the insurance you bought when it was cheap. The right way to use Sornette is to let his diagnosis tell you when to behave like Taleb. The next essay puts a number on how reflexive markets actually are. After that, the series asks how flows move prices, what the calm itself can tell you, and why the one quantity you would need to trust here, how close the market is to its critical point, is exactly the quantity you cannot. {% toolkit() %} What this essay added to the series' instrument panel. - **The test**: steady compounding is a straight line on a log scale. A bubble keeps bending upward even there, because the growth rate itself is growing. - **$t_c$**: the critical time, the finite-time singularity a faster-than-exponential trend is racing toward. The trend must break at or before it. The singularity is a hazard, not a destiny; a bubble can also deflate without a crash. - **The wobble**: log-periodic oscillations that speed up and squeeze together as $t_c$ approaches. Fitting the trend and the wobble together is Sornette's clock, and it is the shakiest part of the story. - **Dragon king vs black swan**: a black swan sits on the power-law line, a rare draw with no warning. A dragon king sits above the line, made by a separate mechanism (runaway feedback), and so partly foreseeable. Carried forward: a regime detector, not a market timer. Sornette's diagnosis tells you when to behave like Taleb. {% end %} ## Further reading The bubble-as-critical-point model: - Johansen, A., Ledoit, O., and Sornette, D. (2000). [*Crashes as Critical Points*](https://arxiv.org/abs/cond-mat/9810071). International Journal of Theoretical and Applied Finance, 3(2). - Sornette, D. (2003). [*Why Stock Markets Crash: Critical Events in Complex Financial Systems*](https://press.princeton.edu/books/paperback/9780691175959/why-stock-markets-crash). Princeton University Press. - Kindleberger, C. P., and Aliber, R. Z. (2015). [*Manias, Panics, and Crashes: A History of Financial Crises*](https://link.springer.com/book/10.1057/9780230628045). Palgrave Macmillan. The financial-history companion to the bubble-as-critical-point picture. - Nasdaq OMX Group. [*NASDAQ Composite Index (NASDAQCOM)*](https://fred.stlouisfed.org/series/NASDAQCOM). Retrieved via FRED, Federal Reserve Bank of St. Louis. Dragon kings versus black swans: - Sornette, D., and Ouillon, G. (2012). [*Dragon-Kings: Mechanisms, Statistical Methods and Empirical Evidence*](https://arxiv.org/abs/1205.1002). The European Physical Journal Special Topics, 205. The pre-registered Shanghai forecast: - Jiang, Z.-Q., Zhou, W.-X., Sornette, D., Woodard, R., Bastiaensen, K., and Cauwels, P. (2009). [*The Chinese Equity Bubble: Ready to Burst*](https://arxiv.org/abs/0907.1827). arXiv:0907.1827. - Sornette, D., Woodard, R., Fedorovsky, M., Reimann, S., Woodard, H., and Zhou, W.-X. (2009). [*The Financial Bubble Experiment: Advanced Diagnostics and Forecasts of Bubble Terminations*](https://arxiv.org/abs/0911.0454). arXiv:0911.0454. On video: - Didier Sornette, [*How we can predict the next financial crisis*](https://www.youtube.com/watch?v=C_eFjLZqXt8) (TED). The dragon-king and log-periodic argument in fifteen minutes. --- ### Sandpiles and Crashes: How Systems Tune Themselves to the Brink *Published: 2025-05-31* > The last essay left a loose end. Markets can sit near a critical edge where small shocks cascade, but critical points are usually finely tuned, so who keeps a market balanced there? The answer, found in a pile of sand, is that nobody does. Some systems walk to the brink on their own, and that is where their crashes come from. URL: https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/ The last essay ended with a puzzle it did not solve. Markets, it argued, sit near a critical point, the knife-edge where a tiny shock can cascade into a huge move. But phase transitions are fussy. Water turns to ice at one exact temperature. A magnet loses its magnetism at one exact spot on the dial. To sit near that kind of edge, something usually has to hold the dial there with great care. So who holds the market's dial? Who tunes millions of independent traders to balance near the edge? The answer, worked out in 1987 by three physicists, is that nobody does. Some systems walk to the edge on their own and stay there. Per Bak, Chao Tang, and Kurt Wiesenfeld called it self-organized criticality, and the toy they used to show it was a pile of sand. ## I. A pile of sand Drop sand onto a table, one grain at a time. At first the grains sit where they land. The pile grows, the sides get steeper, and now and then a grain you add rolls a little and knocks another loose. Keep going and the slope reaches a steepness it cannot exceed. Add one more grain there and you might get nothing, or a small slip, or a slide that takes a whole face of the pile down at once. From then on the pile holds that slope. Add sand and it sheds sand, in fits and starts, forever. That is the whole idea, and it boils down to a rule simple enough to run on a grid. Give every cell a count of grains. Drop a grain on a random cell. If a cell ever holds four or more, it topples: it gives one grain to each of its four neighbors and keeps the rest. Grains that fall off the edge of the grid are gone. A neighbor that just received a grain might now hold four itself, so it topples too, and the spill can run on and on. Count how many topplings a single added grain sets off. That count is the avalanche. ![Two square grids. Left, a speckled map of grain counts from zero to three across the whole pile. Right, mostly empty except for one connected blob of cells, the ones that toppled during a single avalanche.](/images/charts/soc-1-sandpile.png) *Left, the pile after it has settled into its self-organized state, with cells spread below the toppling threshold. Right, the footprint of one avalanche: every cell that toppled after a single grain landed.* Start the grid empty and drip grains. Early on, little happens. The pile fills up, cells get fuller, the first avalanches appear. Before long it settles into the state on the left, loaded across the grid and close to the edge of toppling. It got there by itself. No one set the slope. The pile found it and stays there. Run it yourself. The red cells are the last avalanche. Most drops barely move anything, but every so often one grain runs through a large part of the pile. The histogram on the right builds the fingerprint as the pile runs. {{ sandpile() }} ## II. Avalanches of every size Now the surprising part. Record the size of every avalanche, meaning how many topplings each added grain triggers, and count how often each size shows up. *Guess first: is there a typical avalanche, the way human heights cluster around an average?* You do not get that. You get this. ![A log-log plot. Blue dots, the avalanche sizes from the simulation, fall along a straight downward line across four decades. A dashed line marks a power law of slope about minus 1.2. A red curve, labeled as the "typical size" expectation, starts flat and then plunges steeply, far below the dots, at large sizes.](/images/charts/soc-2-powerlaw.png) *On a log-log plot the avalanche sizes fall on a straight line. That is a power law, and it means there is no typical avalanche. The red curve shows what a world with a typical size would look like, where anything much bigger than average essentially never happens. The pile lives elsewhere.* A straight line on a log-log plot is a power law, and a power law has no characteristic scale. Tiny avalanches are common, big ones rare, enormous ones rarer still, but they all sit on the same smooth slope. There is no special size where the pile says "this one is normal and that one is a freak." The red curve is the bell-curve intuition, a world with a typical size where large events are effectively impossible. The pile lives somewhere else. Big avalanches are rare, but they are always on the menu. Here is the part that should sound familiar from the last essay. Every grain is identical. The grain that sets off a thousand-cell cascade is exactly the same as the millions of grains that did nothing. The big avalanche has no special cause. It is the same process as a small one that happened to keep going. A crash without a cause, built into the rules. ## III. The calm is the setup Watch the avalanches over time and the pattern is its own kind of warning. ![A plot of avalanche size against grains added, one at a time, on a linear scale. Most of the time the line sits near zero, broken by occasional tall spikes reaching into the thousands.](/images/charts/soc-3-punctuated.png) *Long quiet stretches broken by sudden cascades, all from the same steady drip of identical grains. The quiet and the cascade are the same state at different moments: loading and release.* Nothing changes about the input. The grains keep coming at the same rate, all alike. Yet the output is long calm punctuated by sudden collapse. The calm and the cascade belong to the same regime, one quietly loading and the other releasing. Every quiet grain nudges the pile a little closer to the next slide. The stillness is the system loading itself. ## IV. What "self-organized" means This is the piece the last essay was missing. The magnet model needed someone to set the temperature close to its critical value by hand. The sandpile needs no one. Three plain ingredients do the tuning automatically: a slow drive, the grains added gently one at a time; a threshold, the rule that a cell holds on until it reaches four and only then lets go; and leakage, the grains that fall off the edges and leave. Put those together and the pile drives itself to the critical slope and stays there. Push it too hard and avalanches carry off the excess; let it slacken and the drip builds it back up. The system falls toward the critical state from wherever it starts. It is worth connecting this to the attractor picture from the last essay. There the critical point was a knife-edge someone had to tune the dial to by hand, the unstable border between two stable moods. Self-organized criticality turns that border into a destination. The slow drive, the threshold, and the leakage make the critical state itself an attractor of the pile's dynamics, and every avalanche is the restoring nudge that carries the pile back to the edge after it slips over. Nothing has to balance the market on the edge by luck or by a careful hand. The loading and the cascades pull it there and keep it there. One word is worth being exact about. An idealized, infinitely large pile would sit exactly at the critical slope. A real one, finite and constantly disturbed, hovers just below it, tipping over and settling back. That gap is not vagueness in the theory, it is the difference between the clean critical point of an equation and the restless, driven version a market actually lives in. Either way the moral holds, and it answers the question we opened with: nobody tunes the market to the edge, because a market that keeps loading risk and shedding it in cascades tunes itself there. ## V. The same fingerprint everywhere Bak's claim was bigger than sand. The same signature, events of every size with no typical scale and no special cause, turns up across nature. Earthquakes obey it: the Gutenberg-Richter law says that a quake ten times larger is a fixed factor rarer, a straight line on a log-log plot, and a great earthquake is just a small tremor whose rupture kept going. Forest fires follow it, and solar flares, and real snow avalanches, and bursts of firing in networks of neurons, and even the sizes of extinctions in the fossil record. Different machinery, the same statistical shape. This is the universality from the last essay again. Near the critical state the details stop mattering and only the large-scale pattern survives, which is exactly why a model of sand can have anything to say about a market. ## VI. Markets as sandpiles The mapping to finance is concrete, even if it stays an analogy and not an identity. The grains are risk piling up: leverage taken on, positions crowding into the same trades, stop-loss orders stacked at similar prices, promises to deliver that quietly depend on everyone else delivering. The slow drive is an ordinary calm market, where risk accumulates precisely because nothing has gone wrong lately. The toppling is forced selling: a margin call, a triggered stop, a fund that has to liquidate, which pushes the price down, which trips the next stop, which pushes the price again. One sale sets off the next, exactly like one toppling cell spilling into its neighbors. October 19, 1987 is the textbook case. By then a product called portfolio insurance covered something like 80 billion dollars of holdings, all following the same mechanical rule: when prices fall, sell index futures to hedge. Because everyone ran the same rule, the early decline that Monday told all of them to sell at once, the selling drove prices lower, the lower prices told the programs to sell still more, and the loop fed itself. The Brady Commission that investigated the crash found the ten largest sellers accounted for half of all non-market-maker volume in the futures that day. The Dow fell 22.6 percent in a single session, the worst day in its history, with no news anywhere near large enough to explain it. A pile loaded with identical automatic selling rules, and one ordinary down day to start the slide. And the output is the power law. Market moves have tails too fat for a bell curve. The big moves are far too common, and they fall on roughly a straight line on a log-log plot, a power law like the sandpile and the earthquakes, though with a steeper exponent of its own. Gabaix and colleagues, sifting tens of millions of trades, found the tail of large price moves following close to an inverse-cube power law, steady across markets and decades. The fat tails from the last essay are the avalanche distribution of a market that has loaded itself near the edge. This is the mechanism under the measurement we ended on last time. Bouchaud's finding that markets sit close to criticality is what a self-organizing system would lead you to expect. ## VII. Efficiency is the road to the edge There is a sting in this for anyone who runs anything. What pushes a market, or a supply chain, or a power grid toward the critical edge? Bouchaud's answer, in his 2024 survey of the field, is uncomfortable. It is the pursuit of efficiency. An efficient system has no slack. Every resource is used, every buffer trimmed, every part coupled tightly to the next so nothing is wasted. That is also a fair description of a sandpile at the critical slope: fully loaded, nothing to spare, every cell ready to pass a disturbance along. Slack is exactly what keeps a system below the edge. Redundancy, inventory, idle capital, traders who disagree with each other, all of it looks like waste in the good times, and all of it is what stops a local problem from becoming a system-wide cascade. So the choice is a real tradeoff rather than a failure to optimize. You can have a maximally efficient system or a resilient one, but the same lever moves both, in opposite directions. Just-in-time supply chains learned this when one stuck ship or one shut factory rippled around the world. Tightly coupled, heavily leveraged financial systems relearn it every decade or so. Bouchaud's line, that the quest for efficiency and the need for resilience may be incompatible, is the sandpile talking. ## VIII. What you can and cannot do with this The sandpile is honest about its limits, and they are worth saying plainly. The trigger does not tell you whether the big one is coming. Every grain is the same, so "what caused the crash" is close to meaningless. The honest answer is the slope, not the grain, and hunting through the news for the cause of an endogenous cascade is hunting for a grain that was never special. But you can know what kind of system you are standing on. If market moves follow a power law, then large cascades are not flukes to be engineered away. They are a permanent feature of the system. You cannot say when one will arrive, but you can stop being surprised, and you can act on the slope instead of the grain: carry slack, cut leverage and tight coupling where you can, and read a long calm as loading rather than as safety. That leaves one question hanging. If the big cascades are built in, and the trigger tells you nothing, is there anything in the run-up that does? Can you see the slope turn dangerous before it goes? That is exactly where Sornette says yes and Taleb says no, and it is where this series goes next. {% toolkit() %} What this essay added to the series' instrument panel. - **$s$**: the size of an avalanche, the number of topplings one added grain sets off. - **The fingerprint**: avalanche sizes follow a power law, $P(s) \sim s^{-\alpha}$, a straight line on a log-log plot. No typical size, and no special cause for the big ones; a giant avalanche is a small one that kept going. - **The recipe for self-organized criticality**: a slow drive, a threshold, and leakage. Together they make the critical state an attractor. The system walks to the edge and stays there with nobody tuning it. - **The reading of calm**: a long quiet stretch is loading, not safety. The quiet and the cascade are the same regime at different moments. - **The tradeoff**: efficiency strips out the slack that keeps a system below the edge. Maximally efficient and resilient are the same lever pushed in opposite directions. Carried forward: the power law and the self-tuned edge. The next essays ask whether the loaded state can be seen before it lets go. {% end %} ## Further reading The original idea: - Bak, P., Tang, C., and Wiesenfeld, K. (1987). [*Self-Organized Criticality: An Explanation of 1/f Noise*](https://doi.org/10.1103/PhysRevLett.59.381). Physical Review Letters, 59(4). - Bak, P. (1996). [*How Nature Works: The Science of Self-Organized Criticality*](https://link.springer.com/book/10.1007/978-1-4757-5426-1). Copernicus. Power laws in nature and markets: - Gutenberg, B., and Richter, C. F. (1944). [*Frequency of Earthquakes in California*](https://doi.org/10.1785/BSSA0340040185). Bulletin of the Seismological Society of America, 34(4). - Gabaix, X., Gopikrishnan, P., Plerou, V., and Stanley, H. E. (2003). [*A Theory of Power-Law Distributions in Financial Market Fluctuations*](https://doi.org/10.1038/nature01624). Nature, 423. - Gabaix, X. (2009). [*Power Laws in Economics and Finance*](https://doi.org/10.1146/annurev.economics.050708.142940). Annual Review of Economics, 1. Self-organized criticality in finance, and the efficiency tradeoff: - Bouchaud, J.-P. (2024). [*The Self-Organized Criticality Paradigm in Economics and Finance*](https://arxiv.org/abs/2407.10284). arXiv:2407.10284. The 1987 cascade: - Presidential Task Force on Market Mechanisms. (1988). [*Report of the Presidential Task Force on Market Mechanisms*](https://www.sechistorical.org/collection/papers/1980/1988_0101_BradyReport.pdf). - Federal Reserve History. [*Stock Market Crash of 1987*](https://www.federalreservehistory.org/essays/stock-market-crash-of-1987). --- ### Crashes Without a Cause: Markets as Phase Transitions *Published: 2025-05-30* > Big market moves often show up with no news to explain them. A hundred-year-old model of magnets shows why. When people copy each other strongly enough, a market can hold two moods at once, and the smallest nudge tips it from one to the other. We build the model from scratch, with pictures, then turn it on markets. URL: https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/ On October 19, 1987, the S&P 500 lost more than 20 percent. The size of the fall was only part of the strangeness. The usual story never arrived. No bank failed that morning. No war started. No earnings report, rate decision, or political announcement came close to explaining a one-day move of that size. People searched for the cause because markets are supposed to need causes. Black Monday mostly refused to provide one. Bouchaud and his collaborators later found the same problem at a smaller scale. Look at large jumps in individual stocks and the matching headline is often missing. The move is real; the news is too small, or absent. That leaves an uncomfortable possibility: sometimes prices move because the market has already begun to move. The model that makes this feel less mysterious was not built for finance. It was built for magnets. Before turning it into a market, we need to build the magnet first. ## I. A grid of people who copy their neighbors Picture a giant checkerboard. In every square sits one person. Each person holds a simple opinion, yes or no, and you can think of it as a color: red for yes, blue for no. That is the whole cast. The board is the famous Ising model, invented in the 1920s to study magnets, where each square was an atom and the opinion was which way the atom pointed. Two forces act on each person. The first is **copying**. Everyone looks at their four neighbors, up, down, left, right, and feels a pull to match them. If the people around you are red, you lean red. This is peer pressure, and nothing more. The second is **noise**. People are not robots. Now and then someone flips for their own private reasons, ignoring the neighbors entirely. We bundle all of that stubborn independence into one number and call it the temperature. Hot means lots of random flipping. Cold means people mostly do what their neighbors do. These two forces pull in opposite directions. Copying drags each person toward their neighbors; temperature scatters them back toward acting on their own. Neither number decides anything by itself, only the balance between them does, and turning copying up has the same effect as cooling the temperature down. So the board really has one knob: how strong the copying is next to the temperature. Turn that knob and watch what the whole board does. There are three things it can do, and the picture below shows all three. ![Three square grids of red and blue cells. Left, almost all blue with a few stray red dots. Middle, large irregular red and blue regions of many different sizes mixed together. Right, a fine red and blue speckle with no large regions.](/images/charts/ising-0-lattice.png) *The same board of neighbor-copying people at three settings. Cold on the left, the critical point in the middle, hot on the right.* **Cold board, on the left.** Copying wins. Almost everyone agrees. One color floods the whole board, with only a few lonely holdouts. The crowd has locked into a single opinion, and it took no leader and no announcement to make that happen. Local copying did it on its own. **Hot board, on the right.** Noise wins. Everyone is flipping so much that no agreement can form. The board is a fine static of red and blue, like television snow. Ask "what does the crowd think?" and the honest answer is nothing in particular. **The middle board.** This is the interesting one, and it sits at one exact setting of the knob that physicists call the **critical point**. Here copying and noise are perfectly balanced, and you get neither flood nor static. Instead you get blobs of agreement at every size. Small patches, medium patches, and a few enormous ones, all at once. This is the setting where the board is most alive and most twitchy, and it is where everything interesting happens. Better yet, run the board yourself. Drag the temperature down toward a cold, agreeing crowd, up into hot static, and hold it near the critical edge to watch blobs of every size form and break apart. Nudge the news and the whole crowd leans one way. {{ ising() }} ## II. Why the middle setting is special Forget magnets for a second and think about water. Cool it and nothing dramatic happens for a long time. Then around zero degrees it does something violent and sudden: it turns to ice. A tiny change in temperature can flip the substance into a completely different state. That kind of regime change is called a **phase transition**. Freezing is the abrupt kind, where the substance snaps cleanly from one state to the other. The critical point is the rarer, knife-edge case, where the system is poised exactly between two states instead of snapping between them. The Ising board has the same kind of moment. Turn the knob slowly through the critical point and the board stops behaving like a noisy scatter of independent opinions and starts behaving like one collective object. In the ideal infinite model, the shared mood turns on continuously; in a real finite board, the visible change can look abrupt because the whole board has to choose a side. The middle picture above is the board caught at the edge where that choice is most unstable. Right at that edge, two odd things become true, and both will matter once we get to markets. First, a tiny push can move the whole crowd. Away from the critical point, nudging one person barely matters, because their neighbors hold them in place. At the edge the board is balanced so finely that one flip can flip a neighbor, then another, and the change races across the whole board. Small cause, huge effect. Second, big swings need no big trigger. Those large blobs in the middle picture form and break apart on their own, purely from people copying each other. Nothing outside ordered them. The board generates its own large moves. Those two facts are the whole reason any of this matters for money. For readers who like an equation, the whole model fits on one line. Let $m$ be the crowd's overall mood, running from $-1$ (everyone blue) through $0$ (an even split) to $+1$ (everyone red). Let $J$ be how strongly people copy their neighbors, $h$ an outside push toward one color, and $T$ the temperature, the amount of random independent flipping, with $\beta = 1/T$. Now follow one person. The pull they feel is the copying strength times the crowd's current mood, plus the outside push: $Jm + h$. They turn that pull into their own leaning through an S-shaped curve called $\tanh$, which has one job: it takes any pull, however large, and squashes it into an answer between $-1$ and $+1$. Nobody can lean more than all the way. Average that response over everyone and you get the mood the crowd produces. Here is the twist. The mood people react to is the same mood they create. So the only states that can last are the ones that feed back into themselves: $$ m = \tanh\!\big(\beta (J m + h)\big). $$ You do not solve this by plugging in numbers, because $m$ sits on both sides. You look for a mood that, passed back through everyone's copying, returns itself. Physicists call that a fixed point. How many fixed points exist is the whole story, and it turns on one combination, $\beta J$, which is just copying strength divided by temperature, the same balance from before. When copying is weak next to the noise ($\beta J < 1$) there is a single fixed point, and it follows the news $h$ smoothly. When copying is strong ($\beta J > 1$) the same equation has three: two stable crowds, mostly-red and mostly-blue, and an unstable balance point between them. That split is the fold, and it is where crashes and melt-ups come from. The grid pictures keep the literal neighbor-by-neighbor story; this average-crowd shortcut is what lets the next figures draw those folds on one page. The simulation above has these exact controls, temperature $T$, copying $J$, and news $h$, and it shows $J/T$ live, so you can push $\beta J$ past one yourself and watch the single mood split in two. If equations are not your thing, skip it. The pictures carry the argument. ## III. Why a magnet model fits markets so well Now the harder question: why should a toy built for atoms say anything about the stock market? The connection is concrete. The model has become a real tool in quantitative finance, and there are a few solid reasons for that. Start with the obvious one. A magnet is many simple parts pulling on each other, and so is a market: millions of traders, each watching the price and each other. Swap the four neighbors for everyone you can see and the copying force turns into the most ordinary thing in finance. Fear and greed are contagious. People buy because others are buying and sell because others are selling. Traders are not pure sheep either. They have private information, private models, and their own reasons to bet against the crowd, and all of that independence is the noise. So a market already comes with both forces the board needs, and nothing has to be bolted on. The deepest reason is stranger, and physicists call it universality. Near a critical point, systems that look nothing alike up close end up behaving the same way at large scale. Here is the classic example. A magnet losing its magnetism as it heats up, and a liquid boiling into gas, are about as different as two things can be. Yet measured near their critical points, both follow the very same curve, with the very same numbers, and you cannot tell them apart from the shape alone. The small details simply stop mattering. If that is true of markets as well, then you do not need a correct theory of human psychology to say something useful about a market near its tipping point. You only need a big crowd of people who copy each other and sometimes do not, and the crowd supplies the rest. The last reason is the one that should win over a skeptic. A model is only worth something if it matches reality, and this one reproduces the facts the bell curve gets wrong: fat tails, long calm spells broken by bursts of violent volatility, bubbles, crashes, and prices that swing far more than the news can justify. The rest of this post walks through those facts, and each one comes straight out of the same simple board. ## IV. From a board to a market Here is the same board with finance labels. The math does not change, only the words. - Each person is a trader. Red means bullish, betting up. Blue means bearish, betting down. - The crowd's mood, $m$, is now net demand. More red than blue means more buyers than sellers, which pushes the price up. - The copying strength $J$ is herding. How much traders move with the crowd. - The thumb on the scale $h$ is real news and fundamentals. - The temperature is the amount of independent, against-the-grain trading that pulls people off the herd. With that dictionary, the question from the magnet board becomes a question about markets. Can copying alone, with no news, produce a strong, one-sided mood? We already saw the answer on the cold board. Yes. And we saw on the middle board that when the market sits near its tipping point, small news can move it enormously, and large swings can form with no trigger at all. Both of those are exactly what the model is built to produce. ## V. Two moods at once Turn the copying up past the tipping point and something new appears that has no equivalent in calm markets. For the same piece of news, the market can hold two different moods. The picture below plots the crowd's mood against the news, in the calm case and the herding case. ![Two plots of crowd mood against news. Left, a single smooth S-shaped curve rising from pessimism to optimism. Right, the curve folds back on itself into a sideways S, with a top branch, a bottom branch, and a dashed middle branch connecting them.](/images/charts/ising-1-tipping.png) *With weak herding (left) there is exactly one mood for each level of news. With strong herding (right) the curve folds over. For the same news the market can sit on the optimistic top branch or the pessimistic bottom branch. The dashed middle is a knife-edge it can never rest on.* On the left, weak herding, life is simple. Better news, slightly more bullish. Worse news, slightly more bearish. One mood per headline. This is the orderly world standard finance assumes, and in this regime that world is correct. On the right, strong herding, the curve folds back on itself. Over a whole range of news, there are two stable moods the market could be in, optimistic or pessimistic, with an unstable mood in between that it tips away from. The remarkable part is that none of the news has to be dramatic to create this. The herding did it. Once a crowd copies strongly enough, holding two possible moods is baked into the structure of the crowd, not into the information. ## VI. Why regimes jump instead of slide That fold has a famous side effect. Where the market sits depends on today's news and on the path it took to get there. Physicists call this hysteresis: where you end up depends on the path you took, not just on today's news. You already know it from a light switch: it clicks on and off at slightly different pressures, so it tends to stay where it is until you push hard enough. *Before you look at the next picture, make a guess. The market starts in its pessimistic mood and the news slowly, steadily improves. What does the mood do?* ![A loop. As news improves, the market climbs a lower path and then jumps up sharply. As news worsens, it follows an upper path and then drops sharply. The rising and falling paths do not coincide, enclosing a loop, and both transitions are sudden vertical jumps.](/images/charts/ising-2-hysteresis.png) *The same small piece of news leaves you in a different market depending on which branch you are riding, and the switch between branches is a jump, not a slide. The crash here is set off by a change in news no bigger than dozens before it that did nothing.* Start pessimistic and let the news slowly improve. The market grinds up the lower branch, reluctant, and it keeps grinding even past the point where a brighter mood becomes possible, because it is sitting in a stable state and small steps cannot dislodge it. Then the lower branch simply ends. There is nowhere left to stand, and the market jumps up to the optimistic branch. A boom. Now run it backward. From the optimistic mood, let the news drift down. The market eases along the top branch, comfortable, until that branch runs out and it drops, all at once, to the pessimistic branch. A crash. This is worth naming, because the idea runs through the rest of the series. Each stable branch is an **attractor**: leave the mood near it and copying pulls it back, the way a ball rolls back to the bottom of a valley. The dashed middle branch is the opposite, a repeller, and it marks the border between two **basins**, the moods that fall toward optimism and the moods that fall toward pessimism. While copying stays strong the crowd has two attractors at once. A boom or a crash is the moment one of them vanishes: the branch you are riding runs out, its valley flattens away, and the crowd has nowhere to sit but the other attractor. The simulation above draws this landscape live, under the grid: the valleys are the attractors and the dot is the current mood. Drag copying up and watch one valley split into two; tilt the news and watch the dot tip from one into the other. This is what the 1987 detectives were missing. The crash did not need a cause its own size. The market had been walking a branch that was quietly about to end, and the last small step had nothing beneath it. The cause was the shape of the ground, not the size of the trigger. The model itself is symmetric, so it can produce jumps in both directions; real markets add leverage, liquidity stress, and forced selling, which is why the downward jump is usually the one people remember. ## VII. Why prices move more than the news The most important setting is the tipping point itself, the same critical point from the magnet board. *Guess first: as the market walks toward the edge, what happens to the size of its response to one fixed piece of news?* As the market approaches the critical point, its sensitivity to news runs off toward infinity. A vanishingly small headline moves the whole mood by a lot. ![Two curves plotted against the amount of noise. One curve, the strength of the shared mood, rises smoothly from zero as noise falls below the critical level. The other curve, the market's sensitivity to news, is flat and low everywhere except for a tall narrow spike right at the critical point.](/images/charts/ising-3-criticality.png) *A shared mood appears once independent trading drops below the critical level, and the market's sensitivity to news spikes right at the edge. Tiny shocks turn into big moves exactly where the crowd is balanced between its two moods.* Economists since Robert Shiller have had a name for this: excess volatility. Prices move far more than any honest measure of fundamentals can justify, and for decades that has been treated as a puzzle. In this picture the puzzle has a candidate answer: a market sitting near its tipping point would overreact, because near the edge the size of the move compared to the size of the news blows up. So why would a market park itself at this twitchy edge instead of somewhere calm and safe? This is the idea of self-organized criticality, which the next post in this series takes apart on its own. The proposed story is that nothing tunes the market to the edge from outside, it drifts there on its own: competition wears down the independent judgment that keeps traders apart, and the hunt for an edge makes everyone watch everyone else, which turns the copying up and the independence down. That is a plausible mechanism rather than a measured law, so treat it as the hypothesis it is. What has been measured is the symptom. Treating trades as a chain reaction where each trade can set off others, Bouchaud's group found that the share of trading driven by other trades, rather than by outside information, sits close to the level where the chain reaction would run away, and stays there across samples from 1998 to 2011. As a matter of measurement, markets often operate near that edge. ## VIII. Fat tails, made from the inside Run the model as a real crowd of traders instead of a single average, keep it near the edge, and feed it nothing but small, slow nudges. The result looks a lot like a real market. ![Two panels. Left, a wiggly line showing market sentiment over time. It sits in one mood for long stretches, then flips suddenly to the other, while a faint line showing the news only wiggles gently. Right, a histogram of the day-to-day moves on a logarithmic scale, with the real moves sitting well above a smooth bell curve out at the extreme ends.](/images/charts/ising-4-simulation.png) *A simulated crowd of traders copying each other near the edge (left), and the spread of its daily moves (right). The big swings come from inside, with no big news to blame, and the extreme moves are far more common than a bell curve would ever allow.* This is the part I care about most. Fat tails fall out of the model on their own. A crowd of ordinary people copying their neighbors, with nothing fat-tailed going in anywhere, manufactures fat tails from the inside. The leptokurtic, heavy shape of real market returns is simply what a near-critical crowd looks like from the outside. Those tails matter. Fit a bell curve to the S&P 500's ordinary day-to-day movement, of about one percent, and then ask it about the real crashes. ![A bell curve on a log scale, calibrated to a one percent daily move. Black Monday 1987 at minus 20.47 percent sits far out in the tail where the curve has effectively reached zero, and the worst days of 2008 and 2020 sit in a band around minus 9 to minus 12 percent.](/images/charts/realdata-1-crashes-vs-gaussian.png) *Black Monday was a move of about twenty standard deviations. A bell curve says a day that bad should wait far longer than the universe has existed. It happened anyway, and the worst days of 2008 and 2020 sit almost as far out. The model that calls these impossible is the one mainstream finance still runs on.* ## IX. Why the calm is the dangerous part There is one more twist, and it is the most useful one for anyone with money on the line. A system heading toward a tipping point gives off warning signs, but they look different from what you would guess. The danger rarely announces itself as drama and rising volatility. It often hides inside the calm. Far from the edge, poke the system and it springs back fast, because the crowd's pull holds everyone in place. Close to the edge, that pull weakens, so it drifts back slowly and wanders further. Two things give it away. Recovery slows, so each moment looks more like the one before it, which statisticians measure as rising autocorrelation. And the swings get wider, which shows up as rising variance. Both climb as the system walks toward its edge. Slower recovery and wider swings, arriving together, are the fingerprint of a system approaching a transition, and the name for it is critical slowing down. The strange and useful part is that it lives in the quiet, not the storm, which is why a market where everyone agrees and nothing much seems to move can be the most dangerous one of all. A later essay in this series is devoted to this signature, where it comes from, and how far to trust it, so here I will only plant the flag and move on. ## X. What this is worth, and what it is not A model of magnets gives orientation rather than orders. Here is what it actually gives you. It takes the embarrassment out of the missing news. You stop hunting for a hidden cause behind every large move, because some large moves genuinely come from inside the crowd, and forcing a story onto them just dresses up noise. It moves your attention from the shock to the setting. The useful question shifts from "how big is the news?" to "how close is the market to a tipping point?" A few signs say the crowd is near the edge: correlations rising, opinions bunching together, fewer genuine skeptics left, liquidity thinning. Those tell you more about danger than the headlines do. When everyone is watching everyone else, the market is primed to lurch, and the trigger barely matters. Booms and crashes end up under one roof too. They are the same fold crossed in opposite directions, which is why one builds slowly and the other happens fast. The model also carries a built-in humility, and the rest of this series leans on it. To use the model you would have to measure how close the market is to its tipping point, and that number is exactly the kind of unstable, tail-sensitive quantity that Nassim Taleb has spent a career warning against trusting. Knowing that crashes are phase transitions does not hand you the date of the next one. It tells you what kind of ground you are standing on, and that it can give way without a push. That alone is reason enough to carry insurance you do not think you need, and to treat a market where everyone agrees as far more fragile than one where people are still arguing. {% toolkit() %} What this essay added to the series' instrument panel. - **$m$**: the crowd's mood, from $-1$ (everyone bearish) to $+1$ (everyone bullish). In market terms, net demand. - **$J$**: copying strength. Herding. - **$T$**: independent flipping, the contrarian noise, with $\beta = 1/T$. - **$h$**: outside news, a thumb on the scale. - **The whole model**: $m = \tanh\big(\beta(Jm + h)\big)$. The only moods that last are the ones that reproduce themselves through everyone's copying, the fixed points. - **The one dial**: $\beta J$, copying measured against noise. Below one, a single mood that follows the news smoothly. Above one, two stable moods (attractors) with a knife-edge between them, and switching between them is a jump, not a slide. Exactly at one sits the critical point, where sensitivity to news blows up. Carried forward: critical point, attractor, fold, hysteresis. The rest of the series keeps asking one question in different ways: how close is the market to that edge? {% end %} ## Further reading On big moves arriving without news: - Cutler, D. M., Poterba, J. M., and Summers, L. H. (1989). [*What Moves Stock Prices?*](https://www.nber.org/papers/w2538). Journal of Portfolio Management, 15(3). - Joulin, A., Lefèvre, A., Grunberg, D., and Bouchaud, J.-P. (2008). [*Stock Price Jumps: News and Volume Play a Minor Role*](https://arxiv.org/abs/0803.1769). arXiv:0803.1769. - Schwert, G. W. (1990). [*Stock Volatility and the Crash of '87*](https://www.nber.org/papers/w2954). Review of Financial Studies, 3(1). - Federal Reserve History. [*Stock Market Crash of 1987*](https://www.federalreservehistory.org/essays/stock-market-crash-of-1987). On excess volatility: - Shiller, R. J. (1981). [*Do Stock Prices Move Too Much to be Justified by Subsequent Changes in Dividends?*](https://www.aeaweb.org/aer/top20/71.3.421-436.pdf). American Economic Review, 71(3). On fat tails and the stylized facts of returns: - Mandelbrot, B. B. (1963). [*The Variation of Certain Speculative Prices*](https://doi.org/10.1086/294632). Journal of Business, 36(4). The origin of fat tails in finance. - Cont, R. (2001). [*Empirical Properties of Asset Returns: Stylized Facts and Statistical Issues*](https://doi.org/10.1080/713665670). Quantitative Finance, 1(2). On the Ising and discrete-choice models of herding: - Brock, W. A., and Durlauf, S. N. (2001). [*Discrete Choice with Social Interactions*](https://doi.org/10.1111/1467-937X.00168). Review of Economic Studies, 68(2). - Kirman, A. (1993). [*Ants, Rationality, and Recruitment*](https://doi.org/10.2307/2118337). Quarterly Journal of Economics, 108(1). - Lux, T., and Marchesi, M. (1999). [*Scaling and Criticality in a Stochastic Multi-Agent Model of a Financial Market*](https://doi.org/10.1038/16865). Nature, 397. - Cont, R., and Bouchaud, J.-P. (2000). [*Herd Behavior and Aggregate Fluctuations in Financial Markets*](http://rama.cont.perso.math.cnrs.fr/pdf/herd.pdf). Macroeconomic Dynamics, 4(2). On crashes, criticality, and reflexivity: - Johansen, A., Ledoit, O., and Sornette, D. (2000). [*Crashes as Critical Points*](https://arxiv.org/abs/cond-mat/9810071). International Journal of Theoretical and Applied Finance, 3(2). - Hardiman, S. J., Bercot, N., and Bouchaud, J.-P. (2013). [*Critical Reflexivity in Financial Markets: a Hawkes Process Analysis*](https://arxiv.org/abs/1302.1405). European Physical Journal B, 86. - Bouchaud, J.-P. (2024). [*The Self-Organized Criticality Paradigm in Economics and Finance*](https://arxiv.org/abs/2407.10284). arXiv:2407.10284. On multiple equilibria, attractors, and routes to chaos: - Brock, W. A., and Hommes, C. H. (1998). [*Heterogeneous Beliefs and Routes to Chaos in a Simple Asset Pricing Model*](https://doi.org/10.1016/S0165-1889(98)00011-6). Journal of Economic Dynamics and Control, 22(8-9). - Gualdi, S., Tarzia, M., Zamponi, F., and Bouchaud, J.-P. (2015). [*Tipping Points in Macroeconomic Agent-Based Models*](https://arxiv.org/abs/1307.5319). Journal of Economic Dynamics and Control, 50. On phase transitions and universality, for the curious: - Sethna, J. P. (2006). [*Statistical Mechanics: Entropy, Order Parameters, and Complexity*](https://sethna.lassp.cornell.edu/StatMech/). Oxford University Press (freely available online). On complexity economics, broadly: - Arthur, W. B. (2021). [*Foundations of Complexity Economics*](https://www.nature.com/articles/s42254-020-00273-3). Nature Reviews Physics, 3(2). - Farmer, J. D. (2024). [*Making Sense of Chaos: A Better Economics for a Better World*](https://yalebooks.yale.edu/book/9780300283327/making-sense-of-chaos/). Yale University Press. The book-length statement of this whole worldview. --- ## Series: Ethereum The internet moved information freely but never rebuilt the institutions that govern ownership and exchange, so digital economic life grew without durable rights or enforcement. Ethereum addresses this gap by putting institutional functions into software, enforced through economic incentives and cryptographic verification. ### The new financial backend of the world *Published: 2025-12-09* > Ethereum is emerging as a neutral financial backend, lowering the cost of global financial services by encoding ownership and obligations in shared infrastructure. URL: https://federicocarrone.com/series/ethereum/the-new-financial-backend-of-the-world/ **By Federico Carrone and Roberto Catalan** The [previous article](@/series/ethereum/2025-12-02-the-missing-institution-of-the-internet/index.md) argued that the internet left a gap in institutional infrastructure: it moved information but not ownership. Ethereum fills that gap by embedding ownership, transfer, and enforcement into shared software. Financial institutions today spend enormous resources on authorization, accounting, reconciliation, and compliance. Ethereum substitutes a portion of that apparatus with a programmable execution environment and cryptographic enforcement. This article looks at the specific economic mechanisms through which that substitution works. ## Three frictions Some economists describe transaction costs through three frictions: triangulation, transfer and trust. Triangulation concerns how economic actors identify each other and agree on terms. Transfer concerns how value moves between them. Trust concerns the enforcement of obligations. Traditional financial architecture manages these frictions through scale, proprietary systems, and coordination among intermediaries. Ethereum lowers all three, and the numbers are now hard to wave away. Take transfer: stablecoins, dollar tokens that live on the chain, settled about $27.6 trillion in 2024, more than Visa and Mastercard combined, with roughly 95 percent of that volume on Ethereum and its rollups. Most of that figure is exchange and bot flow rather than honest payments, so discount it heavily, but the payments slice is real and growing, and a dollar can move between two strangers in different countries in seconds for cents, with no chain of correspondent banks in between. Take trust: a loan on Aave never asks who you are. The collateral rules sit in a contract that liquidates the position automatically when it crosses a threshold, and anyone can read that contract before they sign. None of this removes institutions; it changes which parts of the stack they have to build. A startup offering dollar accounts in Lagos or Buenos Aires no longer builds settlement, custody, and clearing. It inherits them the way a web startup inherits TCP/IP, and spends its effort on product and distribution. That lets firms serve markets incumbents wave off as too small or too complex. Having a single global ledger also changes operational dynamics. Many institutions operate multiple databases that require frequent reconciliation and remain vulnerable to error. Ethereum maintains a continuously updated and replicated record that cannot be amended retroactively. Redundancy and recoverability become default properties rather than costly internal functions. Security follows the same pattern. Instead of defending a central database, Ethereum distributes verification among many independent actors. Altering history requires coordination at scale and becomes prohibitively expensive. Confidence arises from system design rather than institutional promises. ## New financial services and global reach You can see this in what people actually do with it. In Argentina, where the peso lost more than half its value against the dollar in 2023, ordinary savers hold USDT the way they once held paper dollars in a drawer, except this version moves. Remittances that cost the global average of around 6 percent through a money-transfer operator move as stablecoins for a fraction of that. And the instruments are climbing the respectability ladder: tokenized US Treasury funds grew from about $140 million in early 2024 to roughly $8 billion by late 2025, led by BlackRock's BUIDL at around $2.8 billion, with Franklin Templeton running a government money-market fund whose shareholder records live on seven different chains. The pattern underneath is always the same. Work that used to live inside an organization, reconciling ledgers, proving balances, enforcing the terms of a deal, moves into shared software that every participant can read. The firm is left with the parts that actually differentiate it, product and distribution, and it grows by winning users rather than by rebuilding plumbing its competitors already have. The impact is most visible in markets with fragile financial systems. In economies with unstable currencies or slow payment networks, Ethereum provides immediate functional gains. In developed markets the benefits appear incremental but accumulate as more instruments and processes become programmable. ## Institutional transformation and long term dynamics Many financial instruments are heterogeneous. Corporate debt is a clear example. Terms differ by maturity, coupon, covenants, collateral, and risk. Trading depends on bilateral negotiation and intermediaries who maintain records and enforce obligations. Ethereum can represent these instruments digitally, track ownership, and execute terms automatically. Contracts retain their specificity, while administration becomes standardized and interoperable. The boundary between what firms must build and what software can enforce is moving. Regulation and legal systems remain central, but the institutions sitting on top of them look different when settlement, custody, and enforcement are handled by shared infrastructure instead of proprietary systems. Ethereum already functions as an alternative financial rail. Multiple independently developed clients, substantial real world usage, an active research community, and a commitment to openness and verification set it apart from other blockchain networks. ## Conclusion Ethereum converts core financial frictions into software functions, and that changes the economics of building and operating financial services. Institutions become lighter, focused on product and distribution rather than internal infrastructure. Technological transitions begin in niches where incumbents do not meet demand. As systems mature, costs fall and broader adoption becomes feasible. Ethereum followed this path. It began with internet native communities, expanded across emerging markets where users lacked reliable financial tools, and is now positioned to upgrade mainstream markets by making financial companies easier to create and operate. Software is becoming the organizing principle of financial infrastructure. Ethereum makes that concrete. Regulation and institutional adaptation will shape how far it goes, but the economic incentives already point toward systems that are open, verifiable, and resilient. ## Further reading - DefiLlama. [Stablecoin market cap and supply](https://defillama.com/stablecoins). - The Defiant (2025). [Stablecoins Process $27.6 Trillion in 2024, Surpassing Visa, With 95% Settled on Ethereum](https://thedefiant.io/news/blockchains/stablecoins-process-27-6-trillion-2024-surpassing-visa-95-settled-on-ethereum-4b7c2671). - Yellow.com (2025). [Tokenized U.S. Treasuries Hit $7.3B in 2025](https://yellow.com/en-US/research/tokenized-us-treasuries-hit-dollar73b-in-2025-complete-guide-to-digital-treasury-bonds). --- ### The missing institution of the Internet *Published: 2025-12-02* > The internet made information cheap to move, but it left ownership in the hands of platforms and jurisdictions. Ethereum is one attempt to fix that missing layer. URL: https://federicocarrone.com/series/ethereum/the-missing-institution-of-the-internet/ **By Federico Carrone and Roberto Catalan** The internet made copying and sending information almost free. It did not answer the harder question: who owns something online, who can transfer it, and who enforces the answer when two people disagree? For a long time the practical answer was simple: the platform decides. Facebook decides what happens to your page. Apple decides what ships in its store. Stripe, Visa, a bank, or a court decides whether a payment or claim settles. That worked well enough to build the web we have, but it left the economic layer of the internet sitting on private terms of service and local legal systems. Ethereum is interesting because it tries to put some of that missing institutional layer directly into software: records, rules, transfers, collateral, and enforcement backed by economic incentives and cryptographic verification. ## Technology, Culture and Institutional Design Some inventions help a person do more. Fire, agriculture, medicine, computing. Others help strangers do things together. Property rights, contracts, markets, companies, courts. Both matter. A new tool creates capacity; an institution decides who can use that capacity, who gets paid, and what happens when someone cheats. ## Property Rights and Markets as Social Technologies People invest when they believe they can keep what they build. Property rights give that assurance by saying who owns what, who can use it, and who can be excluded. Markets sit on top of those rights. Prices only work because there is some legal and social machinery underneath them. None of this is natural. It was built through law, politics, habit, and a lot of compromise. The global economy of the twentieth century ran on that machinery: neutral jurisdictions, corporate shells, bank rails, accounting rules, courts, and contracts that let people who did not know each other trade anyway. ## The Missing Architecture of Digital Ownership The internet lowered the cost of communication across borders, but it never created a neutral way to define and enforce claims on digital assets. Offline, ownership goes through courts and states. Online, if there is no shared settlement layer, ownership falls back to whoever runs the server or whichever jurisdiction can reach the dispute. Platforms filled the hole. They gave us identity, search, feeds, stores, payments, messaging, distribution. In exchange, they kept the switch. You can build inside their walls, accumulate users, even become valuable, but your rights are only as strong as the platform's current policy and willingness to keep you around. Zynga is the clean example. It built a huge games business on Facebook and briefly became worth more than Electronic Arts. Then Facebook changed policies and economics, and the ground moved. Zynga owned its games, but not the environment those games depended on. That is the normal platform bargain: you can own the shop and still rent the street. Platform economies give you participation without full control. ## Ethereum as an Institutional Experiment Ethereum is one answer to that missing layer. It gives people a way to create, transfer, and enforce digital assets without asking a company or a state to be the final record keeper. The rules live in code. The record is shared. The network checks itself. In ordinary software, the operator is sovereign. If a company runs the database, the company can change the database. Ethereum makes that harder by spreading execution across many machines that run the same code and agree on the result. Misbehavior is punished economically. Trust is still there, but some of it moves from a private operator to a public protocol. Some institutional work then becomes protocol work. Auditors check records; Ethereum makes the record public and replayable. Escrow agents hold assets; smart contracts can hold and release collateral according to rules. Courts enforce agreements after the fact; a contract on-chain can make some violations impossible in the first place. Law still matters. The claim is only that some of the work law and intermediaries used to do can move into shared infrastructure. Ignore the ideology for a moment and look at the structure: no single operator can quietly rewrite the ledger or block a transaction for everyone. ## The Emergence of a Digital Financial System Ethereum started with people building for themselves: wallets, tokens, exchanges, experiments, strange coordination games. Then the financial use case stopped being theoretical. The most consequential development has been the [rise of stablecoins](@/series/ethereum/2025-12-09-the-new-financial-backend-of-the-world/index.md), dollar tokens backed by reserves of cash and short-term Treasuries. Their combined market value passed $300 billion in 2025, dominated by Tether's USDT (about $176 billion) and Circle's USDC (about $74 billion), most of it issued on Ethereum. The flows are no longer a rounding error: stablecoins settled around $27.6 trillion in 2024, edging past Visa and Mastercard combined, roughly 95 percent of it on Ethereum, though much of that volume is exchange and bot activity rather than genuine payments. Stablecoins are boring in the way important infrastructure is boring: dollars that move globally, settle continuously, and plug into software. Once those dollars exist on-chain, lending markets, collateral systems, automated exchanges, and payment flows can be built around them. The difference from traditional finance is practical. The system is global by default. Services share standards. Exit is fast. Risk is visible on-chain, even if users still misunderstand it. Compare that to [countries like Argentina](@/articles/2025-09-25-crypto-doctrine/index.md), where interoperability between banks and fintech wallets, something as trivial as scanning a QR code, can become a regulatory fight. Incumbents use their position to avoid connecting. On Ethereum, interoperability is the default because the assets and contracts share the same base layer. From a phone, a user can receive dollars, swap them, lend them, borrow against collateral, or move them elsewhere. That is why adoption is strongest where ordinary intermediation is expensive, slow, or hard to trust. ## Implications Remittances, trade finance, private credit, tokenized Treasuries, collateral markets: all of these are being pulled toward blockchain rails for the same reason. They are record-heavy, settlement-heavy, and full of intermediaries whose job is mostly to verify and move claims. Plenty can still go wrong. Regulatory uncertainty, operational risk and rough user experience all constrain adoption. Scaling throughput without giving up decentralization remains an open engineering problem. Software vulnerabilities and governance failures have already cost real money. In 2016 a bug in The DAO drained about $60 million of ether and split the community into Ethereum and Ethereum Classic over whether to claw it back. Cross-chain bridges have fared worse: the Ronin bridge lost about $625 million to North Korea's Lazarus group in March 2022. More is coming. The serious claim is narrower than the hype. Some financial intermediation can probably run cheaper and more transparently on shared rails. Whether it becomes mainstream depends on regulation, incumbents, user experience, and whether the engineering keeps improving. ## Artificial Intelligence and Coordination AI makes production cheaper. It does not decide who owns the output, who can use it, who gets paid, or who is liable when something goes wrong. That is why AI and Ethereum feel complementary rather than competitive. AI expands what people and agents can produce. Ethereum gives some of that activity a place to settle: accounts, payments, collateral, permissions, and contracts that do not depend on one platform operator. If agents are going to trade, hire, escrow, license, and pay each other, they will need rails where the rules are legible and settlement is not owned by a single company. ## Conclusion The internet lowered the cost of transmitting information but left digital ownership in the hands of whoever runs the platform. Ethereum tries to move part of that ownership and enforcement layer into public infrastructure. It may become core infrastructure. It may remain a specialized tool. Regulators, incumbents, and engineering limits will decide a lot of that. But it has already shown something important: digital property does not have to mean "an entry in someone else's database." The internet built an economy before it built its own institutions. Ethereum is one attempt to build them after the fact. ## Further reading - DefiLlama. [Stablecoin market cap and supply](https://defillama.com/stablecoins). - The Defiant (2025). [Stablecoins Process $27.6 Trillion in 2024, Surpassing Visa, With 95% Settled on Ethereum](https://thedefiant.io/news/blockchains/stablecoins-process-27-6-trillion-2024-surpassing-visa-95-settled-on-ethereum-4b7c2671). - CoinDesk (2022). [Axie Infinity's Ronin Network Suffers $625M Exploit](https://www.coindesk.com/tech/2022/03/29/axie-infinitys-ronin-network-suffers-625m-exploit). --- ## Series: When Rules Repeat: The Fixed Point Ladder A seven-part series on repeated rules, fixed points, attractors, chaos, fractals, power laws, non-ergodicity, number theory, self-reference, life, and markets. ### Markets as Reflexive Fixed Points *Published: 2025-06-14* > Markets are more than noisy aggregators of information: beliefs about prices help determine prices, which makes reflexivity a fixed-point problem with fat tails, non-ergodicity, and hysteresis. URL: https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/ Markets are the cleanest laboratory for reflexive dynamics. Prices are not determined by beliefs alone. Cash flows, constraints, inventories, regulation, leverage, and flows all matter. But beliefs about prices become trades, trades become prices, and prices update beliefs. That loop is the object. Biology reaches critical-like organization through selection, feedback, and development over evolutionary time. Markets can generate critical-like statistics much faster, because the feedback loop is explicit. Participants model the market while participating in the market. Their models are not external descriptions. They are part of the thing being described. The careful claim is not that every market is literally sitting at a physical critical point. The claim is narrower and stronger: reflexivity, leverage, liquidity, imitation, and coarse-graining can make markets behave like systems near criticality. They produce fat tails, volatility clustering, correlated cascades, hysteresis, and path dependence. ## The Minimum Vocabulary **Reflexivity** means the description of the system affects the system being described. In markets, a belief about prices can become an order, and the order can move prices. An **order book** is the current set of resting bids and offers. It is a local picture of liquidity: how much can be bought or sold before the price has to move. **Liquidity** means the ability to trade without moving the price too much. Low liquidity makes feedback stronger because a given flow moves the state more. **Leverage** means controlling a position larger than one's own capital. It amplifies gains, losses, forced selling, and contagion. **Margin** is collateral required to keep a leveraged position open. If losses reduce equity below the required level, positions must be reduced or liquidated. **Volatility clustering** means large moves tend to be followed by large moves, and quiet periods tend to be followed by quiet periods. This is one reason market risk is not well described by independent Gaussian shocks. **Coarse-graining** means replacing fine detail with larger-scale summaries. In markets, that can mean aggregating trades over longer time windows. A **tail exponent** describes how quickly rare large events disappear. For a survival tail: $$P(|R|>x)\sim Cx^{-\alpha},$$ smaller $\alpha$ means a heavier tail. **Hysteresis** means the path matters. A system can move from one regime to another and fail to return along the same route when the original parameter is reversed. ## The State Space Of A Market A market is not just a price series. Price is the visible coordinate, but it is not the whole state. A minimal state vector might include: $$x_t=(p_t,b_t,L_t,Q_t,\sigma_t,C_t),$$ where: 1. $p_t$ is price, 2. $b_t$ is the distribution of beliefs or forecasts, 3. $L_t$ is leverage, 4. $Q_t$ is liquidity depth, 5. $\sigma_t$ is perceived volatility, 6. $C_t$ is cross-asset correlation. This is still a caricature. Real markets include inventories, funding constraints, market makers, options dealers, passive flows, regulation, news, taxes, settlement, and institutional mandates. But even this reduced state is enough to show why markets are nonlinear. The variables do not move independently. If price falls, volatility estimates rise. If volatility rises, risk limits tighten. If risk limits tighten, positions are cut. If positions are cut into low liquidity, price falls again. That is a feedback loop, not a sequence of independent shocks. So the market update has the same form as every dynamical system in the series: $$x_{t+1}=F(x_t,\eta_t),$$ but now part of $x_t$ is made of beliefs about $F$ itself. That is the extra reflexive ingredient. ## What Is A Belief Variable? "Belief" sounds psychological, but the market version can be made concrete. A belief is any rule that maps information into desired position: $$\text{information}\longrightarrow\text{forecast}\longrightarrow\text{order}.$$ A discretionary investor may express this as a story: "earnings will accelerate, so the stock is cheap." A trend follower may express it as a rule: "if twelve-month momentum is positive, buy." A market maker may express it as an inventory-adjusted quote. A risk model may express it as a volatility target. A passive index fund may express almost no forecast, but it still follows a rule that converts flows into trades. At the population level, beliefs are not one number. They are a distribution of strategies: $$b_t=\text{distribution of forecasts, constraints, and trading rules at time }t.$$ That distribution is part of the market state because it determines future flow. If enough capital uses the same model, the model becomes a force. If the model stops working, capital leaves it, and the distribution changes. ## Reflexivity As A Fixed-Point Problem Participants model the market. Their models generate trades. The aggregate of trades sets prices. Realized prices update the models. At the level of the strategy ecosystem: $$\text{distribution of models}=F(\text{distribution of models}).$$ This is Keynes' beauty contest in fixed-point language. The market is not only asking what an asset is worth. It is asking what others believe others will believe it is worth. Soros called this reflexivity. In the language of this series, reflexivity is representational closure made institutional. A market is a system where descriptions of the system feed back into the system's state. This does not mean markets are irrational noise. It means the object being priced is partly shaped by the act of pricing. That is why the fixed point can be unstable, non-unique, or unreachable by simple dynamics. The mechanism is concrete: 1. participants model the market, 2. models generate trades, 3. trades move prices, 4. prices update the models, 5. profitable models attract capital, 6. capital changes the market that made the models profitable. At that point the meta-level is financialized. There is not only a market in assets; there is a market in models of the market. Hedge funds, factor portfolios, risk models, execution algorithms, and passive rules all compete to become the dominant description. The distribution of models is determined by outcomes, and outcomes are determined by the distribution of models. That is why the fixed-point equation belongs at the level of the strategy ecosystem: $$\text{distribution of strategies}=F(\text{distribution of strategies}).$$ In equilibrium language, this is related to Nash. In Keynes' language, it is the beauty contest. In Soros' language, it is reflexivity. In the language of the [previous essay](@/series/fixed-point-ladder/2025-06-13-life-as-a-double-fixed-point/index.md), it is representational closure: the system contains models of itself that help produce itself. This is where the [first two essays](@/series/fixed-point-ladder/2025-06-08-fixed-points-and-attractors/index.md) pay off. A reflexive fixed point is not automatically an attractor. The logistic fixed point $x^\star=1-1/r$ keeps existing past $r=3$, but it stops attracting: nearby states orbit it instead of settling into it. The market's belief-fixed-point can behave the same way. Leverage, crowding, and the speed of feedback play the role of control parameters. Past a threshold, the fixed point can lose stability. Prices then do not converge to it. They may oscillate, trend, overshoot, or jump between regimes. This is not a proof that every reflexive market is critical. It is a mechanism. Reflexivity supplies feedback. Leverage amplifies feedback. Liquidity constraints make feedback nonlinear. When those forces interact across many scales of capital and time, the system can produce the empirical signatures associated with criticality: clustered volatility, fat tails, drawdowns of many sizes, and sudden correlation spikes. A bubble is not simply "people being wrong." It is a reflexive regime in which price increases validate the beliefs that caused buying, attracting more capital into the same belief. A crash is not simply "the truth arriving." It is often the bounded system forcing positions back through liquidity constraints, margin calls, and risk limits. A minimal reflexive model has two state variables: price $p_t$ and belief $b_t$. $$p_{t+1}=F(p_t,b_t),$$ $$b_{t+1}=G(b_t,p_{t+1}).$$ A non-reflexive model would treat beliefs as external or fixed. A reflexive model lets beliefs and prices update each other. A local equilibrium is a fixed point: $$p^\star=F(p^\star,b^\star),\qquad b^\star=G(b^\star,p^\star).$$ Stability is then a Jacobian question. Linearize the two-dimensional map around $(p^\star,b^\star)$. If the eigenvalues stay inside the unit circle, perturbations decay. If an eigenvalue crosses the unit circle, the market can move into oscillation, trend amplification, or regime change. This is the same local-stability logic as the logistic map, but the state now includes beliefs about the state. To make this concrete, use the simplest linearized form: {% math() %} \begin{pmatrix} p_{t+1} \\ b_{t+1} \end{pmatrix} = \begin{pmatrix} a & c \\ d & e \end{pmatrix} \begin{pmatrix} p_t \\ b_t \end{pmatrix}. {% end %} The coefficient $c$ says beliefs move prices. The coefficient $d$ says prices update beliefs. Reflexivity is the product $cd$. If $cd$ is small, the loop damps. If $cd$ is large, the loop amplifies. The eigenvalues of the matrix tell you which regime you are in. This is also where two famous worldviews turn out to be the same local stability question. Reflexivity has a sign. When the loop is self-correcting, a belief that pushes the price too high invites the trades that pull it back, the eigenvalues sit inside the unit circle, and prices track fundamentals: that is the efficient-market picture. When the loop is self-reinforcing, a belief that pushes the price up invites trades that validate the belief and push it further, an eigenvalue crosses the unit circle, and you get bubbles and crashes: that is Soros's reflexivity. They are not rival equations for how markets work. They are different regimes of the same two-variable map. The product $cd$ measures the direct feedback loop, but the stability threshold is the spectral radius of the whole Jacobian, the [same threshold](@/series/fixed-point-ladder/2025-06-08-fixed-points-and-attractors/index.md) from the first essay, now deciding whether a market self-corrects or runs away. The disagreement is empirical, about which regime a given market is in and when, not about the mathematics. One caveat, stated plainly so the move is not oversold: this "same map, two regimes" reading is the interpretive lens this series brings, not a treaty Fama and Soros would both sign. Fama largely rejects reflexivity as too imprecise to test; Soros regards the efficient-market hypothesis as a damaging idealization. The reconciliation is what the dynamical-systems framing adds, not a neutral account each camp would endorse. What the framing does earn is a sharper question: instead of asking which side is right in general, ask which regime a given market is in, and what is pushing its loop gain across the line. This is the same mathematics as a two-dimensional feedback controller. Finance adds a nasty feature: the coefficients themselves change during stress. Liquidity $Q_t$ falls, leverage constraints bind, and volatility estimates rise. So the matrix that looked stable in calm conditions can become unstable exactly when it matters. ### Simulation: Reflexive Feedback {{ fixed_point_sim(name="market-reflexivity", caption="Increase reflexive feedback. Weak feedback damps price-belief deviations; strong feedback turns small shocks into persistent swings.") }} ## Leverage As Nonlinear Feedback Leverage is the mechanism that makes market dynamics path-dependent. Suppose a fund has equity $E$ and holds an asset position worth $A$. Its leverage is: $$\ell=\frac{A}{E}.$$ If the asset falls, $A$ falls, but equity falls faster because equity is the residual claim after debt. That means leverage rises after losses. Example: a fund has $100$ of equity and $400$ of assets, so leverage is $4\times$. If the asset falls by $10\%$, the asset position loses $40$. Equity falls from $100$ to $60$. Assets are now $360$. Leverage becomes: $$\ell=\frac{360}{60}=6.$$ The fund did not buy more. Leverage rose because losses consumed equity. If the fund has a leverage limit, it must sell. Selling pushes price down if liquidity is limited. A lower price creates more losses for similar funds. Those funds sell too. That is the leverage-liquidity loop: $$\text{price drop}\rightarrow\text{higher leverage}\rightarrow\text{forced selling}\rightarrow\text{lower price}.$$ This is why the market state includes both leverage and liquidity. A price decline in a deeply liquid, low-leverage market can be absorbed. The same decline in a crowded, leveraged, low-liquidity market becomes endogenous. ### Simulation: Leverage Cascade {{ fixed_point_sim(name="leverage-cascade", caption="Increase leverage or reduce liquidity. A small initial shock can become forced selling because losses raise effective leverage.") }} ## Scale-Free Statistics Financial returns carry a cluster of statistical facts that no Gaussian model reproduces: - fat-tailed returns, - volatility clustering, - power-law order sizes, - drawdowns at many scales, - correlations that jump during crises. These are measurements, not interpretations. Whatever one decides to call the regime that produces them, the regime is not the textbook one of independent normal shocks. Mantegna and Stanley documented scaling behavior in the S&P 500 across time scales. Bouchaud and collaborators developed the broader statistical-physics view of markets as collective systems with heavy tails, clustered volatility, and endogenous crises. Gabaix and coauthors connected large market moves to the size distribution of large institutions and trades. Sornette developed the crash-as-critical-phenomenon line through drawdowns, herding, and log-periodic precursors. These are not identical mechanisms, but they point to the same warning: the tails are not small Gaussian errors. The inverse-cubic law for equity returns is often summarized as: $$P(|R|>x)\sim x^{-3}.$$ Precision matters: $\alpha\approx 3$ implies finite variance in the ideal tail-index convention. Equity returns are far from Gaussian, but they are not automatically in the infinite-variance Lévy-stable regime Mandelbrot first emphasized. The important claim is not "variance never exists." The important claim is that the Gaussian picture fails badly in the region that matters most: large deviations, clustered volatility, drawdowns, leverage cascades, and the sudden alignment of correlations during crises. This is why the $\alpha\approx 3$ caveat matters. The inverse-cubic law is fat-tailed but not the same as the Lévy-stable infinite-variance case with $\alpha<2$. A precise article should not claim infinite variance for liquid equity returns as a blanket fact. The stronger and more defensible statement is that return tails are far too heavy for Gaussian risk models, and the dynamics that create the tails also create volatility clustering and crisis correlations. Volatility clustering is the easiest way to see why "random shocks" is too weak a model. A simple GARCH-style process writes returns as: $$r_t=\sigma_t\epsilon_t,$$ with conditional variance updating as: $$\sigma_{t+1}^2=\omega+\alpha r_t^2+\beta\sigma_t^2.$$ The shock $\epsilon_t$ can be ordinary noise. The memory is in $\sigma_t$. A large return today raises tomorrow's conditional volatility through the $\alpha r_t^2$ term. The $\beta\sigma_t^2$ term keeps volatility elevated. That is how large moves cluster without requiring each shock to be independently enormous. ### Simulation: Volatility Clustering {{ fixed_point_sim(name="volatility-clustering", caption="Change shock feedback and volatility memory. Even with ordinary shocks, the conditional variance remembers large moves, so volatility arrives in clusters instead of independent isolated jumps.") }} Fat tails do not by themselves prove criticality, because a power law can come from several mechanisms: 1. preferential attachment, 2. multiplicative growth, 3. mixtures of volatility regimes, 4. self-organized criticality, 5. constrained optimization under feedback, 6. strategic imitation and crowding. So the inference "fat tails, therefore criticality" is invalid, and this essay does not make it. What it claims instead comes in three layers, and the layers are worth keeping apart, because they do not carry the same weight: 1. **Fact.** Return tails are far heavier than Gaussian, volatility clusters, and correlations jump toward one in a crisis. This is measured. 2. **Mechanism.** Reflexive feedback, leverage, liquidity constraints, imitation, and multi-scale capital flows convert small shocks into large ones. That loop, not a fat-tailed noise term bolted on by hand, is what produces the facts in the first layer. This is a causal claim, and it is the one the essay rests on. 3. **Interpretation.** Those same fingerprints are what critical systems often display, so criticality is a tempting and occasionally useful analogy. It is the weakest layer, and nothing in the first two depends on it. The fat tails stand on the mechanism, not on the label. That is why the essay leans on the word *reflexive* rather than *critical*: the mechanism is defensible on its own terms, and the phase-transition analogy is a bonus laid on top, not the foundation underneath. ## Coarse-Graining And Universality Aggregate trades over one minute, one hour, one day, one month. If the statistical shape persists after rescaling, the market is near a fixed point of a coarse-graining operator. This is the intuition behind the scaling work of Mantegna and Stanley, and behind the broader econophysics literature that treats returns as distributions transformed by time aggregation. That is the renormalization-group viewpoint: $$T(P)=P.$$ Different markets can share tail exponents because they sit in the same basin of attraction. The empirical claim is approximate, finite-range, and instrument-dependent, but the recurrence of similar exponents across markets is exactly the kind of pattern that makes a renormalization viewpoint tempting. This is the finance version of renormalization. Details differ across venues, countries, regulations, participants, and technologies, but some exponents and scaling patterns recur. When details change and exponents persist, universality is the right suspicion. The same caution applies as in physics: universality does not mean all mechanisms are identical. It means the large-scale statistics are governed by a smaller set of relevant variables than the microscopic complexity suggests. The coarse-graining operation can be written schematically. Let $R_{\Delta t}$ be returns measured over interval $\Delta t$. Aggregating time means replacing many short returns with a longer return: $$R_{k\Delta t}=\sum_{i=1}^k R_{\Delta t,i}$$ for log returns. A Gaussian world has a familiar scaling: $$R_{k\Delta t}\sim \sqrt{k}R_{\Delta t}.$$ A critical or fat-tailed world can show slower convergence, tail persistence, volatility clustering, and scaling that is only approximate across finite ranges. The important question is not whether markets are perfectly scale invariant. They are not. The question is why approximate scaling survives across enough time scales to matter for risk. This is where the connection to Lawvere should be handled carefully. A financial market is not literally a category satisfying Lawvere's theorem. But it has the same structural problem Lawvere isolates: representation and evaluation live inside the system. Participants represent possible market maps; their actions evaluate those representations against the market; the result changes the object being represented. That is why markets can self-tune faster than many physical systems. A magnet needs an external temperature knob. A market has internal reflexive knobs: leverage, imitation, risk limits, model adoption, and capital flows move in response to the market's own state. ## Non-Ergodicity For wealth: $$W_{t+1}=W_t(1+r_t),$$ the ensemble average depends on $\mathbb{E}[r]$, but the lived path depends on: $$\mathbb{E}[\log(1+r)].$$ A strategy can have positive expected return and still destroy a typical investor through time. Kelly is the ergodicity-corrected response: optimize long-run growth, not ensemble expectation. This is why finance makes non-ergodicity emotionally obvious. In biology, failed trajectories are extinct lineages. In finance, they are bankrupt funds, forced sellers, margin calls, and investors who cannot stay in the game long enough for the ensemble average to matter. No investor is the ensemble. Each investor follows one path. This is also why positive expected return is not enough. If the downside path can force liquidation, then the strategy is not evaluated by its ensemble mean. It is evaluated by survival through time. The Kelly criterion is not merely a betting trick; it is the fixed-point ladder's ergodicity correction in portfolio form. The smallest example is enough. Suppose each period has a 50 percent chance of gaining 50 percent and a 50 percent chance of losing 40 percent. The ensemble arithmetic return is: $$\frac{1}{2}(0.50)+\frac{1}{2}(-0.40)=0.05.$$ Positive five percent. But the time-average log growth is: $$\frac{1}{2}\log(1.5)+\frac{1}{2}\log(0.6)=\frac{1}{2}\log(0.9)<0.$$ The average wealth across many parallel copies can rise while the typical compounded path decays. That is the whole ergodicity problem in one line. ## Hysteresis Crises are not rewinds. The pre-crisis attractor gives way to another attractor: higher volatility, wider spreads, forced deleveraging, broken institutions, changed regulation, changed models. The path back is not the path down. That is hysteresis. Markets add a special fact biology hides: the self-reference is explicit. Strategies model the market; strategies are traded; successful models reshape the environment that made them successful. After 2008, the market did not simply return to its previous basin. Balance sheets changed. Regulation changed. Central banks changed their role. Risk models changed. Passive investing accelerated. The post-crisis attractor was structurally different from the pre-crisis attractor. That is the same mathematics as ecological regime shifts and climate tipping points: once the basin changes, reversing the parameter does not necessarily restore the old state. The pre-crisis attractor was low volatility, tight spreads, rising leverage, and low perceived correlation. The crisis attractor was high volatility, forced deleveraging, wide spreads, and correlations jumping toward one. The post-crisis attractor was not the original state restored; it was a new basin with central-bank dominance, altered regulation, changed balance sheets, and a different ecology of strategies. This is the market version of a multi-attractor system. Let $L$ represent leverage, $V$ volatility, $S$ spread width, and $C$ cross-asset correlation. The pre-crisis regime has roughly: $$L \uparrow,\quad V \downarrow,\quad S \downarrow,\quad C \downarrow.$$ The crisis regime flips the signs: $$L \downarrow\ \text{forcibly},\quad V \uparrow,\quad S \uparrow,\quad C \uparrow.$$ Those arrows are not independent variables moving separately. They reinforce each other. Forced deleveraging raises volatility; higher volatility tightens risk limits; tighter risk limits force more selling; more selling widens spreads and raises correlations. The system changes basin through feedback. ## The Single Statement Living systems and financial systems both produce the fingerprints of criticality, fat tails, scale-free fluctuations, cascades of many sizes, but they earn those fingerprints differently, and the evidence is not equally strong in the two cases. Biology arrives there slowly, through dissipation, feedback, selection, and self-reproduction over evolutionary time. Markets arrive there fast, through reflexivity, leverage, imitation, constraints, and the trading of models that model the market. The loop is explicit, which is why a market can change regime in an afternoon while a lineage takes epochs. In both cases, the system is not a passive object described from outside. It is a system whose internal dynamics help determine the object being described. That is why the fixed-point ladder matters. Engine A drives systems onto attractors. Engine B makes self-description unavoidable once a system can represent enough of itself. Where both engines run together, the result is a system that survives in basins, adapts near critical boundaries, produces power-law or power-law-like statistics, and changes regimes hysteretically. The thirteenth lesson: **A market is a reflexive fixed-point problem: prices, beliefs, and strategies solve for one another, so its equilibrium is self-made, often critical-like, and quick to lose stability.** The invariant summary of this essay is: $$ x_{t+1}=F(x_t,\eta_t), $$ but with beliefs and strategies inside the state: $$ x_t=(p_t,b_t,L_t,Q_t,\sigma_t,C_t). $$ At the strategy level the fixed-point problem is: $$ \mu_{t+1}=\Phi(\mu_t,\text{prices produced by }\mu_t). $$ Markets become reflexive when the descriptions inside $\mu_t$ help create the prices that select the next $\mu_{t+1}$. That is why the output is not a calm equilibrium but a moving regime structure with fat tails, cascades, and hysteresis. ## The Same Skeleton The domains are different, but the pattern repeats: 1. choose a space, 2. choose a transformation, 3. apply it repeatedly or let it act on itself, 4. find what remains invariant, 5. study whether that invariant object is stable. | Domain | Engine | Space | Transformation | Fixed or invariant object | What can go wrong | | --- | --- | --- | --- | --- | --- | | Banach contraction | A: iteration | Metric space | $x\mapsto f(x)$ | Unique fixed point $x^\star$ | No completeness, no contraction, multiple limits | | Attractors | A: iteration | State space | Time evolution | Point, cycle, or strange attractor | Basin boundaries, unstable fixed points | | Logistic map | A: iteration | Interval $[0,1]$ | $x\mapsto rx(1-x)$ | Stable fixed point, then periodic orbit | Bifurcation and loss of stability | | Feigenbaum universality | A: iteration | Space of maps | Renormalization $\mathcal{R}$ | Fixed function $\mathcal{R}(g)=g$ | Accumulation of bifurcations into chaos | | Strange attractors | A: iteration | Phase space | Stretch and fold | Fractal invariant set | Sensitive dependence on initial conditions | | Power laws | A: iteration | Distributions | Rescaling $x\mapsto \lambda x$ | Scale-invariant tail $x^{-\alpha}$ | Infinite variance or infinite mean | | Ergodicity economics | A: iteration | Wealth paths | Multiplication through time | Time-average growth rate | Ensemble average differs from lived trajectory | | KAM theory | A: iteration | Phase space tori | Perturbed Hamiltonian flow | Surviving quasi-periodic torus | Resonance, small denominators | | Gödel/Turing | B: self-reference | Syntax encoded as numbers | Diagonal self-application | Self-referential sentence or program | Incompleteness, undecidability | | Recursive types | B: self-reference | Category of types | Functor $F$ | $\mu F$ or $\nu F$ | Nontermination, infinite unfolding | | Lawvere | B: self-reference | Category with representation | Abstract diagonal | Fixed point of every endomap | Paradox when self-reference is too strong | | Life | A + B | Organism state plus genome | Dissipative dynamics plus self-reproduction | Double fixed point: attractor plus quine-like reproductive loop | Death, loss of canalization, runaway instability | | Markets | A + B | Prices plus strategy ecosystem | Reflexive coarse-graining | Critical-like or Nash-like fixed point of beliefs, strategies, and prices | Crashes, hysteresis, forced liquidation, ruin | The table is the series in compressed form. The details matter, but the high-level move is stable: a rule acts on a space, and some object becomes special because the rule cannot move past it. Sometimes the object is stable and useful. Sometimes it is unstable and dangerous. Sometimes it is impossible, and that impossibility becomes a theorem. ## The Whole Ladder If I compress the whole series into one sentence: **Repeated transformations create state fixed points. State fixed points become attractors. Attractors change under parameters. Those changes can create chaos. Chaos can leave fractal invariant sets and invariant measures. Scale invariance, whether geometric, statistical, or renormalized, creates power-law forms. Power laws can make time averages diverge from ensemble averages. Following one trajectory through time forces the study of recurrence, and recurrence brings arithmetic through rational approximation and resonance. Arithmetic can also encode syntax; once syntax is encoded, self-reference produces representational fixed points, or proves that such fixed points are impossible, in logic, computation, and category theory. Biology and markets are where the two engines meet: systems that survive in attractor basins while carrying internal descriptions that help reproduce or price the system itself.** That is the ladder. Everything else, Feigenbaum, KAM, Gödel, Turing, Zipf, Mandelbrot, Lawvere, Waddington, Kauffman, Soros, Mantegna, Stanley, Kelly, is a different place where the same skeleton becomes visible. ## What Expressiveness Buys, And What It Does Not There is a tempting one-sentence compression of the whole series: make a system expressive enough and power laws and fractals will follow. The sentence is close, and wrong in an instructive way, because the ladder is really two claims that do not reduce to each other. The two claims also use the series' central phrase in two different senses, and this is the place to keep them apart. A **state fixed point** is a configuration the dynamics cannot move, $f(x^\star)=x^\star$; you find it by running time forward and watching where the system settles. A **representational fixed point** is a description that evaluation hands back unchanged, a quine, a Gödel sentence; you find it by diagonalization, and no time is involved anywhere. Same abstract equation, $T(x)=x$, but a different space and a different transformation. Attractors, power laws, and fat tails all live on the state side. Self-reference lives on the representational side. Expressiveness alone does not buy power laws. What expressiveness buys, and this is the Lawvere, Gödel, and Turing rung of the ladder, is representational fixed points. Once a system can represent enough of its own maps, diagonalization becomes unavoidable: Gödel sentences, halting problems, Y combinators, quines. But nothing about that makes anything fluctuate. Peano arithmetic is as expressive as a system can be, and it has no fat tails, because it has no dynamics at all. And power laws do not require expressiveness. The [sandpile](@/series/edge-of-chaos/2025-05-31-sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/index.md) is the cleanest counterexample: it sheds beautiful scale-free avalanches while being unable to represent anything. Slow drive, threshold, leakage. No beliefs, no models, no self-description. The point cuts the other way too, and this essay already made it: "fat tails, therefore criticality" is an invalid inference, because power laws come from preferential attachment, multiplicative growth, and volatility mixtures just as easily as from critical points. So the honest compression has three parts rather than one. Iteration and feedback make state fixed points inevitable: attractors, the configurations a system settles into. Expressiveness makes representational fixed points inevitable: the descriptions that fold back onto themselves. And the thesis of the series lives at the intersection: when a system is expressive and its self-descriptions are wired back into its own dynamics, which is what the last two essays have called representational closure, the two senses of fixed point stop being separate subjects. Evaluating the descriptions moves the state, and the state rewrites the descriptions. Self-reference stops being a logical curiosity and becomes a force. Models of the market trade in the market. The genome's description of the organism builds the organism. At that point the feedback loops are strong, internal, and self-tuning, and critical-like behavior, with its power laws, cascades, and hysteresis, becomes a regime the system reaches fast and leaves reluctantly, rather than a knife-edge someone would have to balance it on. A magnet needs an external temperature dial. A market carries its dials inside. {% pullquote() %} Iteration guarantees fixed points of state: configurations the dynamics cannot move. Expressiveness guarantees fixed points of representation: descriptions that fold back onto themselves. When a system's descriptions of itself become part of its state, the two kinds of fixed point entangle, and fat tails, cascades, and hysteresis stop being fine-tuned exceptions and become the neighborhood the system lives in. {% end %} Even at the intersection, the claim is "tends toward," never "will have." The series rests on the mechanism, not the label, and that is why this essay leans on the word *reflexive* rather than *critical*. {% toolkit(label="Takeaway") %} - **Space:** prices together with the ecosystem of strategies trading on them. - **Transformation:** reflexive coarse-graining, beliefs becoming trades becoming prices becoming beliefs. - **Invariant object:** a critical-like or Nash-like fixed point of beliefs, strategies, and prices. - **Failure mode:** crashes, hysteresis, forced liquidation, ruin. {% end %} ## References And Further Reading This series is a guided synthesis, not a replacement for the original sources. The references below are the places I would send someone who wants to make each rung precise. For dynamical systems and chaos: 1. Steven Strogatz, [*Nonlinear Dynamics and Chaos*](https://www.routledge.com/Nonlinear-Dynamics-and-Chaos-With-Applications-to-Physics-Biology-Chemistry-and-Engineering/Strogatz/p/book/9780367026509). The best first book for fixed points, stability, bifurcations, and the logistic map. 2. Robert May, [*Simple mathematical models with very complicated dynamics*](https://doi.org/10.1038/261459a0) (1976). The classic short paper that made the logistic map famous outside pure dynamics. 3. Mitchell Feigenbaum, [*Quantitative universality for a class of nonlinear transformations*](https://doi.org/10.1007/BF01020332) (1978). The period-doubling universality paper. 4. Robert Devaney, [*An Introduction to Chaotic Dynamical Systems*](https://www.routledge.com/An-Introduction-to-Chaotic-Dynamical-Systems/Devaney/p/book/9780367235994). A more mathematical route into chaos, symbolic dynamics, and fractals. 5. Tien-Yien Li and James Yorke, [*Period three implies chaos*](https://doi.org/10.1080/00029890.1975.11994008) (1975). The famous theorem behind the slogan. 6. Stephen Smale, [*Differentiable dynamical systems*](https://doi.org/10.1090/S0002-9904-1967-11798-1) (1967). A foundational paper for modern dynamical systems. 7. Heinz-Otto Peitgen and Peter Richter, [*The Beauty of Fractals*](https://en.wikipedia.org/wiki/The_Beauty_of_Fractals). A good route into Newton fractals and visual complex dynamics. 8. Henri Poincaré, [*Science and Method*](https://en.wikipedia.org/wiki/Science_and_Method) (1908). Where dynamical-systems thinking and sensitive dependence on initial conditions begin. 9. David Ruelle and Floris Takens, [*On the nature of turbulence*](https://doi.org/10.1007/BF01646553) (1971). The paper that named the strange attractor. 10. Jean-Pierre Eckmann and David Ruelle, [*Ergodic theory of chaos and strange attractors*](https://doi.org/10.1103/RevModPhys.57.617) (1985). The canonical review of chaos, Lyapunov exponents, and invariant measures. For fractals, scaling, and power laws: 1. Benoit Mandelbrot, [*The Fractal Geometry of Nature*](https://en.wikipedia.org/wiki/The_Fractal_Geometry_of_Nature). The source for the modern fractal worldview. 2. Per Bak, Chao Tang, and Kurt Wiesenfeld, [*Self-organized criticality*](https://doi.org/10.1103/PhysRevLett.59.381) (1987). The sandpile paper behind many later power-law stories. 3. Mark Newman, [*Power laws, Pareto distributions and Zipf's law*](https://arxiv.org/abs/cond-mat/0412004) (2005). A clear survey of where power laws appear and how to reason about them. 4. Aaron Clauset, Cosma Shalizi, and Mark Newman, [*Power-law distributions in empirical data*](https://arxiv.org/abs/0706.1062) (2009). Important because it explains how easy it is to fool yourself when fitting power laws. 5. Kenneth Wilson, [*The renormalization group and critical phenomena*](https://doi.org/10.1103/RevModPhys.55.583) (1983 Nobel lecture). The clean conceptual source for renormalization and critical exponents. 6. Herbert Simon, [*On a class of skew distribution functions*](https://doi.org/10.1093/biomet/42.3-4.425) (1955). The Yule-Simon preferential-attachment mechanism. 7. Albert-Laszlo Barabasi and Reka Albert, [*Emergence of scaling in random networks*](https://doi.org/10.1126/science.286.5439.509) (1999). The modern network version of rich-get-richer scaling. 8. Edwin Jaynes, [*Information theory and statistical mechanics*](https://doi.org/10.1103/PhysRev.106.620) (1957). Maximum entropy, the information-theoretic route to power-law forms. For ergodicity and multiplicative dynamics: 1. Ole Peters, [*The ergodicity problem in economics*](https://doi.org/10.1038/s41567-019-0732-0) (2019). The cleanest entry point into the time-average versus ensemble-average distinction. 2. Nassim Nicholas Taleb, [*The Black Swan*](https://en.wikipedia.org/wiki/The_Black_Swan_(Taleb_book)) and [*Statistical Consequences of Fat Tails*](https://arxiv.org/abs/2001.10488). Useful for the Extremistan intuition and for the dangers of thin-tailed thinking. 3. John Kelly, [*A new interpretation of information rate*](https://doi.org/10.1002/j.1538-7305.1956.tb03809.x) (1956). The original Kelly criterion paper, where logarithmic growth becomes operational. 4. George Birkhoff, [*Proof of the ergodic theorem*](https://doi.org/10.1073/pnas.17.2.656) (1931). The theorem behind time averages equaling space averages. 5. Lasota and Mackey, [*Chaos, Fractals, and Noise*](https://doi.org/10.1007/978-1-4612-4286-4). A useful source for invariant densities and Perron-Frobenius operators. 6. Ole Peters and Murray Gell-Mann, [*Evaluating gambles using dynamics*](https://doi.org/10.1063/1.4940236) (2016). The sharpest single statement of the time-average argument. For number theory inside dynamics: 1. Vladimir Arnold, [*Mathematical Methods of Classical Mechanics*](https://en.wikipedia.org/wiki/Mathematical_Methods_of_Classical_Mechanics). The canonical route into Hamiltonian mechanics and action-angle variables. 2. Jurgen Moser, [*Stable and Random Motions in Dynamical Systems*](https://en.wikipedia.org/wiki/J%C3%BCrgen_Moser). A classic treatment of KAM ideas. 3. Hendrik Broer and Floris Takens, [*Dynamical Systems and Chaos*](https://doi.org/10.1007/978-1-4419-6870-8). Useful for connecting invariant tori, bifurcations, and resonance. 4. Andrey Kolmogorov, [*On conservation of conditionally periodic motions for a small change in Hamilton's function*](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Arnold%E2%80%93Moser_theorem) (1954). The original KAM note, the K in KAM. For logic, computation, and self-reference: 1. Kurt Gödel, [*On formally undecidable propositions of Principia Mathematica and related systems*](https://doi.org/10.1007/BF01700692) (1931). The incompleteness paper. 2. Alan Turing, [*On computable numbers, with an application to the Entscheidungsproblem*](https://doi.org/10.1112/plms/s2-42.1.230) (1936). The halting problem and the modern idea of computation. 3. Douglas Hofstadter, [*Gödel, Escher, Bach*](https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach). Not the most formal source, but still one of the best ways to feel why self-reference matters. 4. Raymond Smullyan, [*Gödel's Incompleteness Theorems*](https://en.wikipedia.org/wiki/Raymond_Smullyan). A gentler logical path into diagonalization. 5. Haskell Curry and Robert Feys, [*Combinatory Logic*](https://en.wikipedia.org/wiki/Combinatory_logic). A classical source for fixed-point combinators. 6. Henk Barendregt, [*The Lambda Calculus*](https://en.wikipedia.org/wiki/Lambda_calculus). The standard reference for lambda calculus and the Y combinator. 7. Alfred Tarski, [*The concept of truth in formalized languages*](https://en.wikipedia.org/wiki/Tarski%27s_undefinability_theorem) (1936). Undefinability: a system strong enough for arithmetic cannot define its own truth. For category theory and fixed points: 1. F. William Lawvere, [*Diagonal arguments and cartesian closed categories*](http://www.tac.mta.ca/tac/reprints/articles/15/tr15abs.html) (1969). The categorical abstraction of diagonalization. 2. Joachim Lambek, [*A fixpoint theorem for complete categories*](https://doi.org/10.1007/BF01110627) (1968). The source of the algebraic view of recursive types. 3. Steve Awodey, [*Category Theory*](https://global.oup.com/academic/product/category-theory-9780199237180). A clean modern introduction. 4. Benjamin Pierce, [*Basic Category Theory for Computer Scientists*](https://en.wikipedia.org/wiki/Benjamin_C._Pierce). Short, practical, and good for programmers. 5. Bart Jacobs, [*Introduction to Coalgebra*](https://doi.org/10.1017/CBO9781316823187). A route from coalgebras to state-based systems and infinite behavior. 6. Alfred Tarski, [*A lattice-theoretical fixpoint theorem and its applications*](https://doi.org/10.2140/pjm.1955.5.285) (1955). The order-theoretic fixed-point theorem behind many least/greatest fixed-point constructions. 7. Stephen Kleene, [*Introduction to Metamathematics*](https://en.wikipedia.org/wiki/Stephen_Cole_Kleene). A classical source for computability and iterative least fixed points. 8. L. E. J. Brouwer, [*Uber Abbildung von Mannigfaltigkeiten*](https://doi.org/10.1007/BF01456931) (1911), and Shizuo Kakutani, [*A generalization of Brouwer's fixed point theorem*](https://doi.org/10.1215/S0012-7094-41-00838-4) (1941). The topological and set-valued fixed-point theorems behind equilibrium arguments. For biology, criticality, and self-reproduction: 1. C. H. Waddington, [*The Strategy of the Genes*](https://en.wikipedia.org/wiki/Conrad_Hal_Waddington). The source of the developmental landscape metaphor. 2. Stuart Kauffman, [*The Origins of Order*](https://en.wikipedia.org/wiki/Stuart_Kauffman). Attractors, Boolean networks, autocatalysis, and self-organization in biology. 3. John Beggs and Dietmar Plenz, [*Neuronal avalanches in neocortical circuits*](https://doi.org/10.1523/JNEUROSCI.23-35-11167.2003) (2003). The neural criticality reference. 4. William Bialek et al., [*Statistical mechanics for natural flocks of birds*](https://doi.org/10.1073/pnas.1118633109) (2012). Scale-free correlations in collective behavior. 5. Christopher Langton, [*Computation at the edge of chaos*](https://doi.org/10.1016/0167-2789(90)90064-V) (1990). The cellular-automata version of critical computation. 6. John von Neumann, [*Theory of Self-Reproducing Automata*](https://en.wikipedia.org/wiki/Theory_of_Self-Reproducing_Automata). The computational ancestor of the genome-as-description story. For markets, reflexivity, and financial criticality: 1. George Soros, [*The Alchemy of Finance*](https://en.wikipedia.org/wiki/The_Alchemy_of_Finance). The canonical reflexivity text. 2. Rosario Mantegna and H. Eugene Stanley, [*Scaling behaviour in the dynamics of an economic index*](https://doi.org/10.1038/376046a0) (1995). The early empirical scaling paper for market returns. 3. Rama Cont and Jean-Philippe Bouchaud, [*Herd behavior and aggregate fluctuations in financial markets*](https://doi.org/10.1017/S1365100500015029) (2000). A clean route from imitation to fat tails. 4. Jean-Philippe Bouchaud, [*Crises and collective socio-economic phenomena*](https://doi.org/10.1007/s10955-013-0795-8) (2013). Markets as collective critical systems. 5. John Maynard Keynes, [*The General Theory of Employment, Interest and Money*](https://en.wikipedia.org/wiki/The_General_Theory_of_Employment,_Interest_and_Money), chapter 12. The beauty-contest passage. 6. Xavier Gabaix et al., [*Institutional investors and stock market volatility*](https://doi.org/10.1162/qjec.2006.121.2.461) (2006), and Xavier Gabaix and Ralph Koijen, [*In search of the origins of financial fluctuations*](https://www.nber.org/papers/w28967) (2021). The large-flow and inelastic-market view. 7. Didier Sornette, [*Why Stock Markets Crash*](https://press.princeton.edu/books/paperback/9780691175959/why-stock-markets-crash). The log-periodic and critical-phenomena route into crashes. 8. Ole Peters and Alexander Adamou, [*The ergodicity solution of the cooperation puzzle*](https://doi.org/10.1098/rsta.2020.0425) (2022). A modern entry into ergodicity economics. --- ### Life as a Double Fixed Point *Published: 2025-06-13* > Living systems are not fixed points in the naive sense but robust attractors and self-reproducing machines, whose encoded description takes part in their own continuation. URL: https://federicocarrone.com/series/fixed-point-ladder/life-as-a-double-fixed-point/ The ladder so far was built from abstract objects: maps, distributions, operators, proofs, programs, and types. Biology is where the abstraction becomes physical. A living organism does more than occupy a state. It is a dynamical system that holds itself inside a viable region of state space while carrying a description that helps reproduce the system itself. That is why the tempting sentence is: > biology sits at a fixed point. That is almost right, but the correction matters. Biology does not sit at fixed points. Biology sits on **attractors**, and in some places it sits near **critical points**. The roadmap is: 1. define the state space of a body, 2. explain why homeostasis is an attracting invariant distribution, not a frozen point, 3. explain development as movement through a changing attractor landscape, 4. separate the safe claim about biological attractors from the stronger claim about criticality, 5. show why life is special: it combines dynamical stability with self-reproduction. The goal is not to force biology into a metaphor. The goal is to ask a precise question: > what is being kept invariant, by what transformation, and over what time scale? ## The Minimum Vocabulary A **state variable** is a quantity used to describe the condition of a system. In physiology, examples include temperature, pH, glucose concentration, hormone levels, membrane voltage, and gene-expression levels. **Homeostasis** means keeping a variable inside a viable range. Body temperature is the standard example. The mathematical picture is a stable attractor: perturb the variable and feedback pulls it back. **Allostasis** means stability through changing regulation. The target itself can move: cortisol changes over the day, heart rate changes with activity, immune response changes with infection. Life does not always hold one number fixed; often it moves the setpoint while keeping the organism viable. **Feedback** means the output of a system affects its future input. Negative feedback damps deviations. Positive feedback amplifies them. **Dissipation** means the system continuously loses usable energy to its environment. In dynamical-systems language, dissipation often contracts phase-space volume, pushing trajectories onto lower-dimensional attractors. An **absorbing state** is a state that, once reached, cannot be left. For a biological lineage, death and extinction play this role. **Criticality** is the boundary where perturbations neither die immediately nor explode without coherence. At criticality, correlations can extend across many scales. **Canalization** is developmental robustness. A process is canalized when many small differences in initial condition, noise, or environment still lead to the same functional outcome. **Self-reproduction** is not the copying of matter but the copying of a system that contains a description, and the machinery to interpret it. ## The State Space Of A Body A dynamical system needs a state space. For a body, that sounds impossible at first because the complete state would include every molecule, cell, tissue, signal, microbe, and environmental interaction. But every model chooses a level of description. If you study thermoregulation, a useful state might include core temperature, skin temperature, metabolic rate, sweat response, and environmental temperature. If you study glucose regulation, the state might include glucose, insulin, glucagon, glycogen stores, food intake, and activity. If you study development, the state might be a vector of gene-expression levels. So a biological state vector is model-dependent: $$x_t=(\text{temperature},\text{pH},\text{glucose},\text{hormones},\text{neural activity},\ldots).$$ The dynamics are the rules that move this state forward: $$x_{t+1}=F(x_t,u_t,\eta_t),$$ where $u_t$ represents inputs such as food, light, stress, infection, or social context, and $\eta_t$ represents noise. The organism is viable only in a subset of state space: $$V\subseteq X.$$ Too cold, too acidic, too little ATP, too much inflammation, too much neural excitation: these are exits from viability. Biology is therefore not just a trajectory through state space. It is a trajectory that must remain inside a constrained region while the environment keeps perturbing it. That is why attractors matter. An attractor is not decorative language. It is the mathematical object that explains how a system can be perturbed and still return to a viable region. ## Attractors, Not Fixed Points Homeostasis is point-like: temperature, blood pH, osmolarity, glucose regulation. These are variables where variation is dangerous, so feedback loops make the local dynamics contractive. There is a sharper way to say this using the power-laws essay. A living body is noisy, so the right invariant is rarely a single state. It is a distribution over states that the dynamics leaves unchanged. That is the same object as a Markov stationary distribution $\pi=\pi P$, or the invariant density of a chaotic map: a fixed point of the operator that pushes whole distributions forward one step. Homeostasis is the biological name for an attracting invariant measure, kept narrow by feedback. But much of life is not point-like. The heartbeat, circadian rhythm, neural oscillations, menstrual cycle, and cell cycle are closer to limit cycles. A limit cycle is not a fixed point of the continuous flow, but it is a fixed point of the return map. The honest claim is: > living systems sit on robust attractors: point-shaped where nothing should vary, cycle-shaped where something must. The return-map language is important. Suppose a heartbeat follows a closed orbit in continuous time. Pick a cross-section through that orbit and record where the trajectory returns after one cycle. That gives a discrete map: $$x_{n+1}=P(x_n).$$ The cycle is stable when the return map has an attracting fixed point: $$P(x^\star)=x^\star,\qquad |P'(x^\star)|<1.$$ So the first essay's fixed-point theorem has not disappeared. It has changed level. A rhythm is not a fixed point of the flow, but it is a fixed point of the map that samples the flow once per cycle. This is the first important connection back to the series. The naive sentence "biology is at a fixed point" is false if it means "the organism is static." A living body is never static. It breathes, pumps, metabolizes, repairs, learns, and ages. But the sentence becomes true at the right level: | Biological phenomenon | Mathematical invariant | | --- | --- | | homeostatic variable | attracting point or narrow invariant distribution | | heartbeat or circadian rhythm | stable limit cycle / fixed point of a return map | | cell fate | attractor of a regulatory network | | tissue identity | basin maintained by feedback and epigenetic regulation | | reproduction | fixed point of a description-interpreter loop | Life is not fixed because nothing changes. Life is fixed because some relations survive continuous change. ## Three Forces Push Biology Toward Attractors First, dissipation. Living systems are open and dissipative. They burn free energy and shed entropy. Dissipation contracts phase-space volume, so trajectories collapse onto lower-dimensional sets. Second, negative feedback. Insulin and glucagon, thermoregulation, baroreflexes, and many gene-regulatory loops are engineered contractions. Around a viable setpoint, the biology wants something like: $$|f'(x^\star)|<1.$$ A one-variable caricature makes the point. Let $x_t$ be deviation from a setpoint, such as temperature error. A negative-feedback controller has local form: $$x_{t+1}=a x_t+\eta_t,$$ where $\eta_t$ is noise. If $|a|<1$, shocks decay geometrically. If $a>1$, shocks amplify. If $a<-1$, correction overshoots so hard that it creates an unstable oscillation. Real physiology is high-dimensional, but the local linear-algebra picture is the same: the eigenvalues of the Jacobian near the viable state must stay inside the unit circle for discrete-time regulation, or have negative real parts for continuous-time regulation. Third, selection plus death. Death is an absorbing state. A lineage whose vital variables amplify perturbations is removed. We do not observe all possible dynamics. We observe the survivorship-filtered subset that remained inside viable basins of attraction. This is non-ergodicity in evolutionary form. The ensemble of possible organisms is enormous. Most architectures do not survive long enough to be sampled through time. The time average over surviving lineages is not the ensemble average over all conceivable biological machines. Selection is not merely optimizing an objective function; it is deleting trajectories that hit the absorbing barrier. That is why biological stability should not be read as proof of design in the narrow engineering sense. It is often survivorship made visible. This is the [Kelly criterion](@/series/fixed-point-ladder/2025-06-10-power-laws-extremistan-and-non-ergodicity/index.md) wearing a lab coat. In simple multiplicative models, a lineage in a fluctuating environment grows, over generations, not at its average fitness but at the *geometric mean* of fitness across environments, which is the same $\mathbb{E}[\log\,\cdot]$ object the ergodicity essay derived for a gambler avoiding ruin. So evolution favors bet-hedging: a genotype that lowers its arithmetic-mean fitness to raise its geometric mean, spreading germination across years or diversifying offspring, can displace a competitor that is better on average but occasionally crashes to zero. The gambler optimizing long-run wealth and the lineage optimizing long-run survival are solving the same log-growth problem in different domains. Non-ergodicity is not a quirk of finance; it is the mathematics of any system that has to keep multiplying through time without hitting zero. There is a useful linear-algebra summary. Near a viable state $x^\star$, approximate the biological dynamics by its Jacobian: $$x_{t+1}-x^\star \approx J(x_t-x^\star).$$ If the eigenvalues of $J$ are inside the unit circle, small perturbations shrink. If one eigenvalue crosses outside, perturbations grow along that direction. If a complex pair crosses the stability boundary, oscillations can appear or destabilize. This is the same local story as the logistic map and the bifurcation essay. Biology adds noise, many dimensions, delays, and changing parameters, but the first diagnostic is familiar: look at the linearized map and ask whether deviations contract. That also explains why disease is often dynamical rather than merely material. Hypertension, arrhythmia, autoimmune runaway, cancer growth, seizure, chronic inflammation, metabolic syndrome: in each case, a regulatory loop can move from a stable basin into a different attractor or lose stability in a direction that used to contract. ## Development As A Bifurcation Diagram Cell fates can be understood as attractors of a gene-regulatory dynamical system. Waddington's landscape is a basin diagram. Kauffman's random Boolean networks made this precise enough to simulate. A morphogen gradient is a slowly moving parameter. As the parameter changes, the landscape changes. One progenitor basin can split into two committed-fate basins. Development is bifurcation theory with chemistry. In symbols, write gene-expression state as a vector: $$x_t=(x_t^{(1)},\ldots,x_t^{(n)}),$$ and the regulatory update as: $$x_{t+1}=F(x_t;r),$$ where $r$ is a developmental parameter: morphogen concentration, position in tissue, time since fertilization, or signaling context. A cell fate is an attractor of $F$. If the underlying chemistry is continuous in time, read $F$ as the return map or sampled update of the regulatory dynamics, the same move used above for heartbeats and cycles. Differentiation is not the cell discovering a label. It is the trajectory falling into a basin as $r$ changes the shape of the state space. Canalization means the basin is wide. Noise, mutation, and small environmental differences do not immediately kick the cell into the wrong fate. In dynamical language, selection favors attractors with large basins and strong restoring directions for functions that must be reliable. A tiny one-dimensional landscape makes the bifurcation visible. Suppose the state of a cell is summarized by one coordinate $x$, and imagine a potential: $$V(x;r)=x^4-rx^2.$$ The cell tends to roll downhill: $$\dot{x}=-\frac{dV}{dx}=-4x^3+2rx.$$ When $r<0$, the potential has one minimum at $x=0$. There is one basin. One fate. When $r>0$, the center becomes unstable and two minima appear: $$x=\pm\sqrt{\frac{r}{2}}.$$ Now there are two basins. Two fates. ### Simulation: Waddington Landscape {{ fixed_point_sim(name="waddington-landscape", caption="Move the developmental parameter. One basin becomes two, which is the simplest mathematical picture of a progenitor state splitting into committed fates.") }} This is not a realistic model of development. It is a clean local picture of what Waddington's landscape means. A parameter changes. The attractor structure changes. A single valley splits into two. The trajectory does not choose a fate by reading a label; it falls into one basin or the other as the landscape deforms. In real development, $x$ is high-dimensional. The coordinates are expression levels, chromatin states, signaling pathways, cell-cell interactions, and mechanical constraints. But the dynamical idea is the same: $$\text{cell fate}=\text{attractor of a regulatory system}.$$ This is where the [first two essays](@/series/fixed-point-ladder/2025-06-08-fixed-points-and-attractors/index.md) return. Development uses fixed points and bifurcations, but not in the toy sense of a single number converging. It uses them in a high-dimensional, noisy, historically constrained state space. ## Criticality Survival wants deep basins. Computation wants sensitivity. The variables that must not vary are pinned deep inside attractors. The variables that must process information are often held near criticality, where perturbations propagate without immediately dying and without destroying coherence. This is the edge-of-chaos idea in biological language: > stable enough to persist, sensitive enough to compute. Neural avalanches, gene-regulatory networks, immune repertoires, and collective behavior all belong here. Beggs and Plenz, Kauffman, Bialek, Langton, and Bak are all circling the same shape from different sides. At criticality, a system has no single characteristic scale. Perturbations can remain local, or they can travel across the whole system. Below criticality, signals die too quickly. Above criticality, coherence is destroyed. Near criticality, the system can store information, transmit information, and transform information. This is the useful meaning of the edge of chaos. Not maximum disorder. Not permanent explosion. The useful region is the boundary where structure persists while remaining responsive. There are several versions of the claim, and they should not be mixed carelessly. There is also a live scientific debate here. Criticality is a powerful explanatory frame, but it is not a universal solvent. A power law can come from sampling artifacts, mixtures of hidden variables, optimization under constraints, preferential attachment, or ordinary heterogeneity. Long correlations can be finite-size effects. Neural avalanches can depend on measurement scale and thresholding. So the burden is not merely to point at a broad distribution and say "critical." The burden is to identify the mechanism, the control parameter, the scaling range, and the alternative explanations. ### Claim 1: Some Biological Systems Show Critical Signatures This is the most empirical claim, and the most secure. Some biological systems show the textbook measurements of a critical point: power-law avalanches, correlation lengths that grow with the system, scale-free fluctuations, and sharp changes in responsiveness. That these signatures are *present* is data. What they *mean* is the contested part, and the contest sharpens with each later claim. Neural systems are the most famous example. Beggs and Plenz reported neural avalanches whose size distribution is close to the $-3/2$ exponent of a critical branching process. In that model, the key number is the branching ratio. If it is below one, activity dies. If it is above one, activity explodes. Near one, activity propagates without immediately vanishing or saturating. That is exactly the critical boundary. The branching-process toy model is: $$Z_{t+1}=\sum_{i=1}^{Z_t} X_i,$$ where $Z_t$ is the number of active units and $X_i$ is the number of descendants triggered by active unit $i$. The branching ratio is: $$\sigma=\mathbb{E}[X_i].$$ If $\sigma<1$, activity dies out. If $\sigma>1$, activity can blow up. At $\sigma=1$, avalanche sizes follow a power law in the ideal model. This is the simplest mathematical reason critical neural dynamics are interesting: criticality maximizes propagation without immediate runaway. ### Simulation: Branching Criticality {{ fixed_point_sim(name="branching-criticality", caption="Move the branching ratio. Below one, activity dies; above one, it runs away; near one, avalanches span many scales.") }} Gene-regulatory systems give a second example. Kauffman's Boolean-network models have a simple control knob: how many other genes each gene depends on. Too few dependencies and the system freezes. Too many and perturbations spread chaotically. Around the critical regime, often summarized in the toy model as connectivity near $K\approx 2$, the network can remain structured while still adapting. Collective behavior gives a third example. Bialek and collaborators argued that starling flocks sit near a ferromagnetic-style critical point: directional correlations can extend across the whole flock instead of dying at a fixed distance. The flock behaves less like a crowd of independent birds than like a correlated system whose correlation length scales with its size. The immune system gives a fourth example. Antibody affinity, T-cell receptor diversity, and immune repertoire statistics often show broad, heavy-tailed structure. The cautious claim is not that every such distribution proves a critical point. The useful claim is that immune adaptation lives in the same tradeoff: enough stability to remember, enough sensitivity to respond to unknown perturbations. ### Claim 2: Criticality Is Computationally Useful This is a model claim. It says that systems near the ordered-chaotic boundary can process information better than systems deep in either regime. Langton made the computational version precise for cellular automata. His $\lambda$ parameter moves systems from frozen order to chaotic disorder. The interesting computational region lies near the edge, where information can be stored, transmitted, and transformed. This is the cleanest theoretical version of the biological intuition: the edge of chaos is not aesthetic language; it is where computation becomes possible. The intuition is simple. In a frozen system, perturbations vanish too fast. Nothing propagates. In a chaotic system, perturbations spread too fast and destroy usable structure. Near the boundary, signals can travel, interact, and still remain interpretable. In linear terms, think again about eigenvalues. If every direction contracts strongly, the system forgets too much. If many directions expand strongly, the system becomes unstable. Near marginal stability, where important directions are close to the boundary, perturbations can persist long enough to carry information. That does not mean every variable should be marginal. Core pH should not live at the edge of chaos. Neural and immune variables may. The body is not one dynamical regime. It is a hierarchy: some variables are pinned, some oscillate, some adapt, and some compute near critical boundaries. ### Claim 3: Evolution May Self-Organize Near Criticality This is the broadest claim, and the one to hold at arm's length. It says evolution does not merely produce local attractors but actively tends to park adaptive systems near critical points, because that is where large reorganizations stay possible. The picture is genuinely attractive. In Bak's sandpile, the pile is not waiting for one final avalanche; it maintains a slope on which avalanches of every size remain possible. Read evolutionary history in that language and the major transitions, autocatalytic chemistry, prokaryote to eukaryote, unicellular to multicellular, asexual to sexual, organism to superorganism, look like avalanches in a self-maintained critical landscape. Punctuated equilibrium fits the same frame: long stasis is the system resting in a basin, and the punctuations are jumps between basins. Kauffman's autocatalytic sets supply a candidate mechanism for one such jump. Below a catalytic-density threshold there is chemistry but no self-sustaining closure; above it, a reflexively autocatalytic network maintains itself, a phase transition into a new attractor of organization. But attractive is not established, and this is the place the series is most likely to overspend its credibility. This rung rests more on analogy than on measurement, so the honest move is to name exactly where the evidence is thin rather than soften every sentence around it: - The neural-avalanche exponent is contested. Subsampling a network that is not actually critical can manufacture a clean $-3/2$ power law, and some cortical recordings fit a slightly subcritical, reverberating regime better than a critical one. - $K\approx 2$ is the critical connectivity of a *toy* random Boolean network. That real gene-regulatory networks sit at that boundary is a separate claim, with far weaker support than the tidy number implies. - Self-organized criticality as a theory of evolution, the Bak-Sneppen line, is largely a metaphor. The model shows that a toy fitness-replacement rule can self-organize to criticality; it does not show that real evolution, with ecology, development, population structure, and changing environments, does the same. Its grounding in the fossil and genomic record is therefore much thinner than the toy model makes it feel. - The scale-free correlations measured in starling flocks are real and striking; reading them specifically as *criticality*, rather than as one of several models that produce long correlations, is the step in dispute. None of this is a debunking. Criticality remains a productive lens, and Claims 1 and 2 stand on their own. But "evolution aims at the edge of chaos" is a hypothesis wearing the costume of a law, and it should be worn loosely. What survives every one of these doubts is a weaker, sturdier statement, and it is the one to actually stand behind: > biological systems are often organized so that vital variables sit in robust basins, while adaptive and informational variables sit closer to marginal stability. That allocation claim asks for no commitment to self-organized criticality as a biological law. It needs only what the earlier essays already earned: feedback makes some directions contract hard, and selection deletes the lineages whose vital variables do not. It also rules out the sloppy version, > life is chaos. Life is not chaos. Chaos destroys an organism the moment it reaches the wrong variable. The defensible claim is only that living systems allocate stability and sensitivity differently across scales. ## What Is Being Fixed? At this point the word "fixed" can mislead. The fixed object is not a frozen organism. It is a pattern of recurrence across time scales. At the fast physiological scale, variables return to viable ranges: $$x_t\in V.$$ At the developmental scale, cells fall into reproducible fate basins: $$x_{t+1}=F(x_t;r)\longrightarrow A_r.$$ At the evolutionary scale, lineages that leave viability disappear: $$\text{trajectory hits death}\Rightarrow\text{removed from future sampling}.$$ At the reproductive scale, the description-interpreter loop repeats: $$C(G,E)\longrightarrow (C',G').$$ These are different invariants. Confusing them makes the thesis vague. Keeping them separate makes the thesis stronger: biology is not one fixed point, but a stack of invariant structures that operate at different time scales. ## Life As A Double Fixed Point Life is where the two engines meet. Engine A drags the organism onto dynamical attractors: homeostasis, rhythms, development, tissue identity. Engine B appears because DNA is self-description embedded inside the system that reads and copies it. A cell contains both a description and the machinery that interprets the description. In rough form: $$\operatorname{eval}(\text{genome}) \approx \text{organism containing genome}.$$ That is not a casual metaphor. It is the biological version of the self-reproduction problem studied by von Neumann: tape plus universal constructor. The genome is not the organism, just as a program is not the machine executing it. But the organism contains machinery that reads the genome, constructs the proteins that maintain the machinery, and reproduces the genome into the next system. A more careful schematic is: $$C(G,E)\longrightarrow (C',G'),$$ where $G$ is the genome, $C$ is the cellular machinery, and $E$ is the environment. Reproduction succeeds when: $$G'\approx G$$ and the new machinery $C'$ can read $G'$ again. The fixed point is therefore not a static object. It is a closed reproductive loop: description produces interpreter, interpreter copies description, and the resulting system can repeat the loop. Life is a self-describing system that is also a stable attractor of its own dynamics. That is the double fixed point: 1. Dynamically, the organism survives by remaining inside viable attractors. 2. Recursively, the organism reproduces by carrying an encoded description that participates in constructing another system carrying the description. The deep fact about life is not merely that it is stable. Many dead systems are stable. The deep fact is that stability and self-description coincide. The caveat is important. DNA is not a blueprint in the naive sense. A blueprint can be read by an external builder. A genome is read by machinery that is itself partly produced under genomic control and partly inherited from the previous cell. The interpreter is not outside the system. That is why the better analogy is not blueprint but quine-like loop. A quine works because code and interpreter cooperate. A genome works because genome, ribosome, membranes, enzymes, cellular architecture, and environment form a closed enough loop to produce another loop of the same kind. This is where biology is deeper than ordinary dynamical stability. A whirlpool is an attractor, but it does not encode a description that helps produce future whirlpools. A crystal can replicate structure, but it does not contain an internally interpreted program for building the machinery that continues the process. Life combines: 1. attractor structure, 2. metabolism and dissipation, 3. inherited description, 4. an interpreter for that description, 5. reproduction of both description and interpreter. That combination is why life belongs after the self-reference essay. It is more than an example of Engine A: it is the first real-world system in the series where Engine A and Engine B visibly cooperate. The most precise final statement is: > a living system is a dissipative attractor that carries an encoded description participating in the reproduction of another dissipative attractor of the same kind. The twelfth lesson: **Life is a double fixed point: a dissipative attractor that also encodes and reproduces the description needed to rebuild itself.** That sentence is less romantic than "life is a fixed point," but it is closer to the mathematics. The invariant summary of this essay is: $$ x_t\in V $$ for fast viable physiological variables; $$ x_{t+1}=F(x_t;r)\longrightarrow A_r $$ for development and fate basins; and $$ C(G,E)\longrightarrow(C',G'),\qquad G'\approx G $$ for reproduction. Life is where those invariants coexist: a body must remain in viable attractors while carrying an internal description that helps produce another viable body. {% toolkit(label="Takeaway") %} - **Space:** organism state together with its genome. - **Transformation:** dissipative dynamics that hold the body in viable states, plus self-reproduction. - **Invariant object:** a double fixed point, an attractor of the body plus a quine-like reproductive loop. - **Failure mode:** death, loss of canalization, runaway instability. {% end %} ## Further Reading 1. Stuart Kauffman, [*The Origins of Order*](https://en.wikipedia.org/wiki/Stuart_Kauffman). Attractors, Boolean networks, autocatalysis, and self-organization in biology. 2. C. H. Waddington, [*The Strategy of the Genes*](https://en.wikipedia.org/wiki/Conrad_Hal_Waddington). The source of the developmental landscape metaphor. 3. John Beggs and Dietmar Plenz, [*Neuronal avalanches in neocortical circuits*](https://doi.org/10.1523/JNEUROSCI.23-35-11167.2003) (2003). The neural criticality reference. 4. William Bialek et al., [*Statistical mechanics for natural flocks of birds*](https://doi.org/10.1073/pnas.1118633109) (2012). Scale-free correlations in collective behavior. 5. John von Neumann, [*Theory of Self-Reproducing Automata*](https://en.wikipedia.org/wiki/Theory_of_Self-Reproducing_Automata). The computational ancestor of the genome-as-description story. 6. Per Bak, [*How Nature Works*](https://en.wikipedia.org/wiki/Per_Bak). The popular route into self-organized criticality. --- ### Logic, Self-Reference, and Category Theory *Published: 2025-06-12* > Once arithmetic can encode syntax, a system can act on descriptions of itself. That second engine produces Gödel sentences, quines, the Y combinator, recursive types, and Lawvere's fixed point theorem. URL: https://federicocarrone.com/series/fixed-point-ladder/logic-self-reference-and-category-theory/ The [previous essay](@/series/fixed-point-ladder/2025-06-11-number-theory-and-recurrence/index.md) followed the first engine, iteration, to its end. A rule applied again and again produces fixed points, attractors, invariant measures, scaling laws, and finally the invariant tori of KAM theory, held together or torn apart by the arithmetic of a single frequency. Through all of it, arithmetic stayed *outside* the system. Integers counted returns. Continued fractions measured resonance. The numbers were a ruler we held up against the motion. This essay turns the hinge. It uses arithmetic the second way: not as a ruler held against the system, but as a *language the system speaks about itself.* That turn is the whole subject. The same integers that count returns going around a clock can count symbols inside a formula, because a formula is just a finite string of symbols, and a finite string can be packed into a single number. Once that is possible, a system can encode descriptions of its own statements, rules, and programs, and then act on those descriptions. We call that **representational closure**, and it is the threshold of the second engine: **self-reference**. The payoff is a new family of fixed points. Not attractors and tori, but Gödel sentences, undecidable programs, quines, the Y combinator, and recursive types. The fixed-point *question* survives the change of engine intact, "what transformation acts here, and what does it leave invariant?", even though the machinery that answers it is entirely new. By the end, a single categorical theorem due to Lawvere will show Gödel, Turing, Cantor, and Russell to be one argument in four disguises. This is the point where the word "fixed point" changes level. Earlier, a fixed point was a state that a rule did not move: $$ f(x^\star)=x^\star. $$ Here, a fixed point is a represented object that survives being fed through a rule about its own representation: $$ \text{object}\simeq\text{transformation}(\text{description of that object}). $$ That is why the phrase reappears without being redundant. Engine A found fixed points by repeating a rule on states. Engine B finds fixed points, or proves their impossibility, by letting descriptions act on themselves. ## The Minimum Vocabulary A **formal system** is a rule-governed language for making proofs. It has symbols, formulas, axioms, and rules of inference that turn formulas into other formulas. Arithmetic, the theory of $0$, successor, addition, and multiplication, is the formal system we will care about, because it is strong enough to be turned on itself. **Encoding** means representing one kind of object as another. The encoding at the heart of this essay is **Gödel numbering**: assigning a number to every formula and every proof, so that statements *about* formulas become statements *about* numbers. Once syntax is encoded as arithmetic, arithmetic can talk about syntax. **Diagonalization** is the self-reference move, and it recurs so often it deserves a name. You take a construction meant to range over a collection of objects, and you feed it an object built from itself. Sometimes this produces a useful fixed point. Sometimes it produces a contradiction, and the contradiction becomes an impossibility theorem. Cantor, Gödel, Turing, and Russell are all this one move. **Representational closure** is the threshold condition: the moment a system can represent enough of its own expressions, rules, or maps for self-application to become possible. Below the threshold, self-reference is just informal wordplay. Above it, it is mathematics. A **category**, introduced near the end, is the bookkeeping of objects, arrows between them, and a way to compose arrows. It is the language in which all the self-reference theorems turn out to be the same theorem. ## Two Engines, Recalled A quick restating of the architecture, because this essay opens the series' second half. The claim has never been that everything is the same object. It is that **two mechanisms keep forcing invariant objects to appear.** The first is **iteration**: take a rule $x\mapsto f(x)$ and apply it repeatedly. That alone produced the first four essays. The system does not represent itself; it is simply turned. The second is **self-reference**: a system rich enough to represent its own expressions, maps, or proofs, and then apply transformations to those representations. This is the engine of the present essay. Both engines run the same discipline: 1. choose a **space**, 2. choose a **transformation**, 3. apply it repeatedly, or let it act on its own representations, 4. ask what remains **invariant**, 5. study whether that invariant is stable, unstable, universal, pathological, or expressive. A fixed point is the simplest invariant, $T(x)=x$. But invariance can also mean a set maps into itself, a distribution keeps its shape under rescaling, a torus survives perturbation, a sentence talks about its own code, or a type unfolds into one layer plus another copy of itself. The two engines are related but not interchangeable, and the difference is exactly representation: | Engine | Basic act | Threshold | Fixed objects | | --- | --- | --- | --- | | Iteration | apply a rule again | nonlinear repeated dynamics | attractors, invariant sets, invariant measures, scaling laws | | Self-reference | apply a represented rule to itself | enough internal representation | Gödel sentences, quines, recursive programs, recursive types | One theorem sits deepest in the self-reference column, and stating it up front gives the rest of the essay something to aim at. It is Lawvere's, and informally it says: > once a system can represent its own maps richly enough, fixed points are unavoidable. The contrapositive is just as important, and it is where the impossibility theorems come from: > if some transformation has no fixed point, then no representation system can be that complete. That single sentence, read forwards and backwards, is the skeleton behind Cantor, Gödel, Turing, and Tarski, whose theorem says a system strong enough for arithmetic cannot define its own truth. Boolean negation, "swap true and false," has no fixed point. "Do the opposite of whatever the decider says" has no fixed point. "This sentence is not provable" is the same obstruction written in the language of proof. Self-reference is *generative* when the relevant fixed point exists, and it becomes a *limit theorem* when the fixed point cannot exist. We will earn that sentence properly by the end. One honest caveat about the boundary between the engines. Some iterated systems can become powerful enough to compute, and at that point they cross into representation. Rule 110, a one-dimensional cellular automaton with a trivially simple local update, is rich enough to simulate any computation. So the line between "merely iterated" and "self-referential" is not a wall; iteration can climb into representation. But the distinction still earns its keep: chaos is not automatically self-reference, and self-reference is not automatically chaos. Keeping them separate is what let the first four essays stay clean. ## Why Logic Suddenly Appears Gödel's discovery, stripped to a sentence, is that **whenever a formal system can talk about its own statements, self-reference appears, and self-reference forces fixed points.** Getting there requires seeing how a system talks about itself, which is the encoding trick. ### Encoding syntax as numbers Recall what a formal system contains: symbols, formulas (strings of symbols), and proofs (lists of formulas). Gödel's move was to notice that arithmetic, which obviously talks about numbers, can be made to talk about all of *that* too, by giving everything a number. There are two levels in play, and naming them prevents confusion: 1. the **object language**, where ordinary arithmetic statements live ("$2+2=4$"), 2. the **meta-language**, where we talk about formulas, proofs, and provability ("such-and-such string is a valid proof"). Gödel showed that a sufficiently strong arithmetic can internalize part of its own meta-language: it can represent statements about formulas as statements about numbers. That representation is **Gödel numbering**. The idea is far less mysterious if you first do it crudely. Assign a number to each symbol: | Symbol | Code | | --- | --- | | $0$ | 1 | | $S$ | 2 | | $+$ | 3 | | $=$ | 4 | | $($ | 5 | | $)$ | 6 | A formula is a finite string of symbols, hence a finite list of these numbers. A proof is a finite list of formulas, hence a finite list of lists of numbers. The only remaining trick is to pack a finite list of numbers into a single number, and there are standard ways to do it. Gödel used prime factorization. The list $$a_1,a_2,\ldots,a_n$$ becomes $$2^{a_1}3^{a_2}5^{a_3}\cdots p_n^{a_n}.$$ Because every integer factors into primes in exactly one way, the original list can always be recovered from the product: read off the exponent of $2$, then of $3$, then of $5$, and so on. The encoding is reversible, which is the only property that matters. Any string of syntax can be stored inside a single integer, and pulled back out. Now the consequence. Once formulas and proofs *are* numbers, properties of syntax become properties of numbers. "This string is a well-formed formula" becomes an arithmetic property of an integer. "This list of formulas is a valid proof of that formula" becomes an arithmetic relation between two integers. Concretely, write $$\operatorname{Proof}(p,g)$$ to mean "the number $p$ codes a proof of the formula whose code is $g$." This looks like a statement about proofs, but after encoding it is an ordinary arithmetic relation, checkable by arithmetic. And then provability itself becomes arithmetic: $$\operatorname{Provable}(g)\equiv \exists p\,\operatorname{Proof}(p,g),$$ read as "there exists some number $p$ that codes a proof of $g$." This is the crucial turn. Provability sounds like a notion that lives in the meta-language, outside the system. After Gödel numbering, the system can express it internally. Arithmetic can talk about what arithmetic can prove. ### The diagonal, in its rawest form Before Gödel's sentence, look at the move underneath it with no logic attached at all. Suppose someone claims to have listed every yes/no property of the natural numbers, one property per row: | | 0 | 1 | 2 | 3 | ... | | --- | --- | --- | --- | --- | --- | | $P_0$ | 1 | 0 | 1 | 1 | ... | | $P_1$ | 0 | 0 | 1 | 0 | ... | | $P_2$ | 1 | 1 | 1 | 0 | ... | | $P_3$ | 0 | 1 | 0 | 0 | ... | Now build a new property $D$ by walking down the diagonal and flipping each entry: $$D(n)=1-P_n(n).$$ By construction $D$ disagrees with $P_0$ at input $0$, with $P_1$ at input $1$, with $P_2$ at input $2$, and so on down the list. So $D$ differs from every row in at least one place, which means $D$ was never on the list. The claim to have listed *all* properties fails. That is Cantor's diagonal argument, and it is the seed of everything in this essay: take the object indexed by $n$, ask what it says about $n$, and then transform that answer. Diagonalization is the place where representation (indexing objects by $n$) meets self-application (asking object $n$ about $n$). ### Gödel's sentence as a fixed point Gödel's construction is a richer version of that flip. The technical engine is the **diagonal lemma**, which says, roughly, that for any property of codes $\varphi(x)$ you can write down, there is a sentence $G$ that asserts that property *of its own code*: $$G \leftrightarrow \varphi(\ulcorner G\urcorner).$$ The corner brackets $\ulcorner G\urcorner$ mean "the code number of the sentence $G$." So the diagonal lemma is a fixed-point factory: hand it any property of codes, and it returns a sentence that is true exactly when that property holds of itself. The sentence $G$ is a fixed point of the operation "take a code, build a statement about that code." ![A diagram showing how a sentence is encoded as a number, fed into a formula template, and becomes a self-referential fixed point](/images/articles/fixed-point-diagonalization.svg) Now feed the factory the one property that detonates. Let $$\varphi(x)=\text{``the sentence coded by }x\text{ is not provable.''}$$ The diagonal lemma hands back a sentence $G$ with $$G \leftrightarrow \text{``}G\text{ is not provable.''}$$ That is Gödel's sentence: a statement asserting, in effect, *I am not provable in this system.* If the system could prove it, the system would prove a falsehood; if the system is sound, meaning it proves only true statements, it cannot prove $G$, so $G$ is true but unprovable. The system is incomplete, not by a gap that better axioms could fill, but because its own capacity for self-reference manufactured a true sentence it cannot reach. The same diagonal skeleton appears across the subject with the property $\varphi$ swapped out: - Gödel feeds it "not provable" and gets an unprovable truth. - Turing feeds it "the program that halts here does not" and gets the undecidability of the halting problem. - A quine feeds it the print-yourself operation and gets a program that outputs its own source. Turing's entry compresses a construction worth seeing once. Suppose a program $H$ could decide halting: given any program $p$ and input $x$, it answers whether $p$ eventually stops when run on $x$. Build a contrarian program $D$ that, handed a program's code $p$, asks $H$ what $p$ does when fed its own code, and then does the opposite: $D$ loops forever where $H$ predicts halting, and stops where $H$ predicts looping. Now run $D$ on its own code. If $H$ says it halts, it loops; if $H$ says it loops, it halts. Either way $H$ is wrong, so no such $H$ can exist. The halting problem is undecidable, and the proof is Cantor's flip with "halts on itself" in place of "is on the list." ### From logic to running code: quines and the Y combinator The logical version is intimidating; the programming version is friendly, and it is the *same* fixed point. A **quine** is a program $q$ that prints its own source code. Writing $\operatorname{eval}$ for "run the program," a quine satisfies $$\operatorname{eval}(q)=q.$$ It is literally a fixed point of the run-and-print pipeline. The trick is exactly the diagonal lemma's: the program holds a representation of itself as data, and then uses that data to reconstruct itself. Here is the whole idea in three lines of Python, small enough to read: ```python template = 'template = {!r}\nprint(template.format(template))' print(template.format(template)) ``` The string `template` is a description of the program. The last line inserts that description into itself and prints the result, which is the program's own source. Nothing mystical happens. The program can refer to itself because it can hold a representation of itself and feed that representation back into its own rule. That is representational closure made small enough to run. Kleene's **recursion theorem** is the general statement behind quines: for any computable transformation you might apply to programs, there is a program that obtains its own description and feeds it into that transformation. Programs can always be made to know their own code. ### Simulation: Diagonal Fixed Point Toy {{ fixed_point_sim(name="diagonal", caption="A self-referential fixed point appears when a representation is fed back into the rule that interprets it.") }} The most famous fixed point in all of programming is the **Y combinator**, which lives in the lambda calculus, the minimal language of functions. It satisfies $$Yf=f(Yf).$$ Stare at that for a second: $Yf$ is left unchanged when you apply $f$ to it one more time. It is a genuine fixed-point equation, but for *functions* rather than numbers. Its purpose is to make recursion possible without ever naming a function. Normally a recursive definition says "define this function in terms of itself," which seems to require the function to already have a name to refer to. The Y combinator dissolves that apparent circularity into an explicit fixed point: it builds the self-reference out of pure function application. It is the computational sibling of the diagonal lemma, and, as we will see, the type-theoretic sibling of recursive types. ### The same move, five times So diagonalization keeps reappearing because it is one structural idea, not five coincidences: 1. Cantor diagonalizes against lists of real numbers. 2. Gödel diagonalizes against provability. 3. Turing diagonalizes against halting deciders. 4. Quines diagonalize against the separation of source and output. 5. Lawvere, shortly, abstracts the diagonal itself. The common shape is always: a system rich enough to encode its own elements, and a transformation that can be turned back on that encoding. Here is the dictionary between this essay and the iteration engine of the earlier ones. In dynamics, you repeatedly apply a function to a state, $x\mapsto f(x)$. In logic, you encode a statement as a number and build a new statement about that number, $n\mapsto\varphi(\ulcorner n\urcorner)$. The diagonal step feeds the code of the constructed statement back into the construction, which is self-application, the logical analog of iteration. | Dynamics | Logic | | --- | --- | | state $x$ | sentence/code $g$ | | rule $f$ | formula template $\varphi(x)$ | | iterate $f(x)$ | substitute a code into a formula | | fixed point $f(x^\star)=x^\star$ | self-referential sentence $G\leftrightarrow\varphi(\ulcorner G\urcorner)$ | | stability or instability | consistency, incompleteness, undecidability | Different domain, same structure. A system has enough internal expressive power to turn a rule back on its own objects, and once it does, fixed points appear. In dynamics they are attractors and cycles. In logic they are self-referential sentences. In computation they are programs that refer to their own source. The tenth lesson: **Self-reference generates fixed points in logic just as iteration generates fixed points in dynamics.** The bridge from number theory to logic is coding. Number theory hands you arithmetic objects; Gödel shows those objects can encode syntax; once syntax is encoded, statements can point at themselves through their own codes. Before Gödel, arithmetic looks like a subject about numbers. After Gödel, arithmetic is also a medium in which a formal system represents its own grammar, a mirror in which it can see its own sentences. ## Why Category Theory Appears Once you have watched fixed points fall out of Banach contractions, Gödel sentences, Turing machines, fractals, renormalization, programming languages, and recursive types, a natural question forms: **What is the most general setting in which fixed points must exist?** That question is what category theory is for, here. It is not abstraction for sport; it is the search for the smallest assumptions that still force a fixed point. You do not need a course in category theory for what follows. The only habit to borrow is this: when the objects get too different, compare the arrows. A dynamical map, a program, a proof translation, and a type constructor are not the same kind of thing, but each is a transformation that can be composed with another transformation. ### Just enough category theory A **category** is a deliberately spare structure: 1. objects, 2. arrows between objects, 3. a way to compose arrows. That is all. In the category of sets, objects are sets and arrows are functions. In a category of types, objects are types and arrows are programs. In a category of spaces, objects are spaces and arrows are structure-preserving maps. The trick of category theory is to study a subject by its arrows rather than by what its objects are made of: not "what is inside this object" but "what maps into it, what it maps to, and how those maps compose." That viewpoint fits here because every essay in this series has been about a transformation acting on a space: | Essay object | Transformation | | --- | --- | | state | dynamical map | | set | Hutchinson operator | | distribution | renormalization / coarse-graining | | frequency vector | perturbative conjugacy equation | | sentence code | formula template | | program | evaluation | | type | functor | Category theory keeps the transformation and deliberately forgets the material. Forgetting sounds like a loss, but it is what exposes the skeleton: if you stop asking what the objects are made of and keep only how the maps compose, the parts of the fixed-point arguments that were really the same become visibly the same. The one operation a category insists on is **composition**. If $$A\xrightarrow{f}B\xrightarrow{g}C,$$ then there is a composite arrow $$A\xrightarrow{g\circ f}C.$$ That is already enough to model pipelines, dynamics, program execution, proof translation, and change of coordinates, because each of those is "do one thing, then do the next." ### Functors and the fixed points that are data structures A **functor** is a map between categories that preserves this structure. For a programmer, the cleanest mental model is a type constructor that also knows how to map functions. Take $$F(X)=1+A\times X.$$ Read the right side as a choice: *either* the single empty case (the $1$), *or* a pair of an $A$ and an $X$ (the $A\times X$). If $A$ is the type of elements, then $X$ is the placeholder for "the rest of the list." That is exactly the shape of a list: a list is either empty, or it is one element of type $A$ followed by another list. The functor $F$ captures "one layer of list-ness." An **algebra** for a functor $F$ is an object $X$ together with a way to collapse one layer $F(X)$ back down into $X$: $$F(X)\to X.$$ For lists, that means: given either "empty" or "an element plus a list," produce a list. An inductive data type is the *initial* such algebra, where initial means every other algebra receives exactly one structure-respecting map from it; it is the category's way of saying "the smallest one, with nothing extra added." The initial algebra is a fixed point of the functor. The type of finite lists, written $\mu F$, satisfies $$\mu F \cong 1 + A\times \mu F,$$ which is the equation "a list is either empty, or an element of $A$ paired with another list," now read as a fixed-point equation for *types*. The symbol $\cong$ means the two sides are the same type up to relabeling. ![A diagram showing the list type as the fixed point of the functor F(X)=1+A times X](/images/articles/fixed-point-recursive-types.svg) ### Simulation: Recursive Type Unfolding {{ fixed_point_sim(name="recursive", caption="A finite list is built by repeatedly unfolding List(A) = 1 + A x List(A) until the empty case stops the recursion.") }} The notation $\mu F$ means "the *least* fixed point of $F$," and "least" is doing real work. It is the order-theoretic fixed-point story from the [first essay](@/series/fixed-point-ladder/2025-06-08-fixed-points-and-attractors/index.md) returning in type form. Knaster and Tarski proved that monotone maps on complete lattices have least and greatest fixed points; Kleene showed that, under the right continuity assumptions, you can *build* the least one by starting from the bottom and iterating upward. A recursive type is built the same way: start with no values, apply the constructor pattern, apply it again, and take the smallest stable solution. For lists, the upward construction is concrete and worth seeing: {% math() %} \begin{aligned} X_0 &= \varnothing, \\ X_1 &= 1+A\times X_0, \\ X_2 &= 1+A\times X_1, \\ X_3 &= 1+A\times X_2. \end{aligned} {% end %} $X_0$ contains no lists at all. $X_1$ contains only the empty list. $X_2$ adds the one-element lists. $X_3$ adds lists of length at most two. Iterating forever, the union of all stages is the least fixed point: exactly the finite lists, and nothing infinite. This is why "least" matters. The same equation $X\cong 1+A\times X$ also has larger solutions if you permit infinite or circular objects; the inductive list type deliberately picks the smallest, which keeps every list finite. It is Kleene's "iterate up from the bottom" in a different category, and it is the same convergence-from-below picture as a contraction settling onto its fixed point in the first essay. The pattern is now familiar enough to tabulate: | Earlier | Category / type version | | --- | --- | | number $x$ | type $X$ | | function $f(x)$ | functor $F(X)$ | | fixed point $x^\star=f(x^\star)$ | recursive type $\mu F\cong F(\mu F)$ | | iteration builds convergence | constructors build finite data | ### Coalgebras: the same idea, run forever There is a dual story for *infinite* behavior. A **coalgebra** has the arrow turned around: $$X\to F(X).$$ Instead of collapsing one layer into an object, it *unfolds* an object into one observable layer plus a next state. Streams are the clean example. An infinite stream of values, $$a_0,a_1,a_2,\ldots,$$ is described by the functor $F(X)=A\times X$ ("a head value, plus the rest"), and a stream is a value of the *terminal* coalgebra, terminal meaning every other coalgebra maps into it uniquely, the category's way of saying "the largest one," $$\nu F \cong A\times \nu F,$$ read as "a stream is a head element of type $A$ together with another stream." Unlike a list, there is no empty case to stop it. It unfolds forever. The practical distinction is simple. An algebra builds finite data by consuming one layer at a time. A coalgebra observes an ongoing system by exposing one layer and a next state. Lists are algebraic; streams, automata, and state machines are coalgebraic. So the algebra/coalgebra split mirrors a distinction the series has circled before: inductive construction builds finite objects from base cases, while coinductive observation describes ongoing processes observed through time. Both are fixed points of functors, $\mu F$ and $\nu F$, the least and the greatest. And the coalgebraic shape, $\text{state}\to\text{observation plus next state}$, is exactly a dynamical system: automata, transition systems, infinite processes, and feedback loops all fit it. Category theory turns out to be the language in which recursive data and ongoing dynamics are two readings of one fixed-point idea, which is why it can sit at the seam between the logic of this essay and the living, trading systems of the next two. ### Lawvere's theorem: the bare diagonal Now we can state the result the whole essay has been aiming at. **Lawvere's fixed-point theorem** abstracts the diagonal argument into pure category theory: under suitable conditions, if a category contains enough self-description, fixed points follow. It is why one theorem family touches Gödel, Turing, Cantor, Russell, and program recursion at once. The skeleton it isolates is four steps: 1. objects can represent maps, 2. represented maps can be evaluated, 3. evaluation can be diagonalized, 4. diagonalization forces a fixed point. In less compressed language: if a system can name every operation on itself, then one of those named operations can be fed its own name. That self-application is the diagonal move. Lawvere's theorem says that, under the right structural assumptions, this move forces a fixed point. One informal phrasing: *if every function $A\to B$ can be represented by some element of $A$, then every map $B\to B$ has a fixed point.* Let me earn that in set-theoretic clothing, slowly, because the proof is just the diagonal argument with the decoration stripped off. Suppose there is a surjective map $$\phi:A\to B^A,$$ where $B^A$ means "the functions from $A$ to $B$." Surjective means every such function appears as $\phi(a)$ for some $a$; this is the "enough self-description" hypothesis, that $A$ is rich enough to name all the functions $A\to B$. Now take *any* map $\alpha:B\to B$ and define a new function $$g(a)=\alpha(\phi(a)(a)).$$ Read $\phi(a)(a)$ as: take the function named by $a$, and feed it $a$ itself, the diagonal move. Then apply $\alpha$. Since $g$ is a function $A\to B$, and $\phi$ names every such function, we have $g=\phi(a_0)$ for some particular $a_0$. Evaluate everything at that $a_0$: $$\phi(a_0)(a_0)=g(a_0)=\alpha(\phi(a_0)(a_0)).$$ So the value $b=\phi(a_0)(a_0)$ satisfies $b=\alpha(b)$. It is a fixed point of $\alpha$. We did not assume $\alpha$ had one; the richness of $\phi$ manufactured it. Now run the implication *backwards*, and the impossibility theorems fall out for free. Take $B=\{0,1\}$ and let $\alpha$ be Boolean negation, which famously has no fixed point (swap $0$ and $1$ and nothing stays put). Lawvere's theorem then forbids the hypothesis: there can be *no* surjection $A\to\{0,1\}^A$. That is Cantor's theorem, that no set surjects onto its own power set. Gödel, Turing, and Russell are the same move with a richer $B$ and a different fixed-point-free $\alpha$: negate provability, flip the halting decider, negate membership. Russell's version is worth one sentence, since the essay has not built it yet: form the set of all sets that do not contain themselves, and ask whether it contains itself. Each answer forces the other. That is the diagonal with membership as the flipped property, and it is why naive set theory had to be rebuilt. One theorem, read forward for the positive fixed points and backward for the impossibility results. ### Simulation: Lawvere Diagram {{ fixed_point_sim(name="lawvere", caption="Replay the categorical skeleton: representation gives evaluation, evaluation gives the diagonal, and the diagonal either forces a fixed point or proves that complete self-representation was impossible.") }} The parallel with Gödel is exact once you line them up: | Lawvere proof | Gödel proof | | --- | --- | | element $a$ represents a function | number $g$ represents a formula | | evaluate $\phi(a)(a)$ | substitute a formula's code into itself | | apply $\alpha:B\to B$ | negate provability or transform a property | | surjectivity/representation gives a fixed point | the diagonal lemma gives $G\leftrightarrow\varphi(\ulcorner G\urcorner)$ | Gödel did not secretly use category theory; historically he could not have. What category theory adds comes later and from a different direction: it isolates the *shape* of his argument and shows it was never really about formulas, machines, or sets. It was about a structural configuration: objects representing arrows, arrows composing, and a diagonal that feeds representation back into evaluation. The eleventh lesson: **Category theory is the general language for recurring fixed-point structures.** The bridge from logic to category theory is forgetting the substrate. Gödel talks about formulas, Turing about machines, programming languages about recursive types, dynamics about iterated maps, renormalization about operators, probability about scale-invariant distributions. Category theory asks what remains once you erase the local material and keep only the arrows. That is abstraction as *compression*, not decoration: if the same proof shape appears in logic, computation, and set theory, then perhaps the proof was never about any of their particular contents. Depending on the setting, the fixed point it forces shows up as a Gödel sentence, an undecidable program, a recursive definition, a paradox, an initial algebra, or a terminal coalgebra. What is common is the skeleton. ## The Theoretical Arc Is Complete We have now seen fixed points as the limits of iteration, the attractors of dynamics, the invariant sets of chaos, the scaling laws of distributions, the small-divisor constraints of arithmetic, and the diagonal fixed points of logic and computation. Both engines have been laid out: iteration, which turns a rule until something invariant survives, and self-reference, which lets a system act on its own description until a fixed point becomes unavoidable. This is the place to say exactly why the two engines rhyme, and exactly why they are not the same. A Hutchinson attractor contains scaled copies of itself because a contractive rule on compact sets satisfies $K=\mathcal{H}(K)$. A Gödel sentence contains a claim about its own code because arithmetic can encode syntax and feed that code back into a formula template. Both produce an infinite regress from a fixed-point equation: copies inside copies on the geometric side, descriptions of descriptions on the logical side. But the theorems are different. The Hutchinson attractor does not refer to itself; the iterated function system describes nothing. The Gödel sentence does not stretch, fold, or contract a metric space. The ladder connects them because both are fixed-point phenomena, not because they are secretly the same object. There is one transition left to make carefully, and it is the move from blackboard to world. Formal self-reference happens when a system contains a description of one of its own expressions and can feed that description back into its rules. Living systems and markets are not formal systems, but they have the same three structural ingredients: an internal description, an interpreter for it, and feedback from the interpreter into the system's future state. | System | Internal description | Interpreter | Feedback | | --- | --- | --- | --- | | Logic | Gödel code of a sentence | proof rules | provability statements | | Program | source code | evaluator/compiler | execution and recursion | | Cell | genome | cellular machinery | development and reproduction | | Market | model, price, strategy | traders and capital | orders changing prices | This table is the bridge from formal self-reference to the lived world. A cell is not a theorem, and a market is not a lambda term. But each contains descriptions that participate in the very dynamics they describe. That is why the [last two essays](@/series/fixed-point-ladder/2025-06-13-life-as-a-double-fixed-point/index.md) are not appendices stapled to a math survey. They are where the two engines visibly run together: Engine A supplies the attractor structure. Bodies hold themselves inside viable states; markets move through regimes. Engine B supplies the representational loop. Genomes help build the organisms that carry genomes; market models help create the prices that update the market models. The final two essays apply this skeleton to the two self-referential systems we actually live inside: biology and markets. {% toolkit(label="Takeaway") %} - **Space:** syntax encoded as numbers, and categories of types or representable maps. - **Transformation:** diagonal self-application, a functor $F$, or Lawvere's abstract diagonal. - **Invariant object:** a self-referential sentence or program, a recursive type $\mu F$ or $\nu F$, a forced fixed point. - **Failure mode:** incompleteness, undecidability, paradox, nontermination. {% end %} ## Further Reading For logic, computation, and self-reference: 1. Kurt Gödel, [*On formally undecidable propositions of Principia Mathematica and related systems*](https://doi.org/10.1007/BF01700692) (1931). The incompleteness paper, where self-reference first becomes a theorem. 2. Alan Turing, [*On computable numbers, with an application to the Entscheidungsproblem*](https://doi.org/10.1112/plms/s2-42.1.230) (1936). The halting problem and the modern idea of computation. 3. Alfred Tarski, [*The concept of truth in formalized languages*](https://en.wikipedia.org/wiki/Tarski%27s_undefinability_theorem) (1936). Undefinability: a system strong enough for arithmetic cannot define its own truth. 4. Douglas Hofstadter, [*Gödel, Escher, Bach*](https://en.wikipedia.org/wiki/G%C3%B6del,_Escher,_Bach). Not the most formal source, but still one of the best ways to feel why self-reference matters. 5. Raymond Smullyan, [*Gödel's Incompleteness Theorems*](https://en.wikipedia.org/wiki/Raymond_Smullyan). A gentler logical path into diagonalization. 6. Haskell Curry and Robert Feys, [*Combinatory Logic*](https://en.wikipedia.org/wiki/Combinatory_logic). A classical source for fixed-point combinators. 7. Henk Barendregt, [*The Lambda Calculus*](https://en.wikipedia.org/wiki/Lambda_calculus). The standard reference for lambda calculus and the Y combinator. For category theory and the general shape of fixed points: 1. F. William Lawvere, [*Diagonal arguments and cartesian closed categories*](http://www.tac.mta.ca/tac/reprints/articles/15/tr15abs.html) (1969). The categorical abstraction of diagonalization that unifies Gödel, Turing, and Cantor. 2. Joachim Lambek, [*A fixpoint theorem for complete categories*](https://doi.org/10.1007/BF01110627) (1968). The source of the algebraic view of recursive types. 3. Steve Awodey, [*Category Theory*](https://global.oup.com/academic/product/category-theory-9780199237180). A clean modern introduction. 4. Benjamin Pierce, [*Basic Category Theory for Computer Scientists*](https://en.wikipedia.org/wiki/Benjamin_C._Pierce). Short, practical, and good for programmers. 5. Bart Jacobs, [*Introduction to Coalgebra*](https://doi.org/10.1017/CBO9781316823187). A route from coalgebras to state-based systems and infinite behavior. 6. Alfred Tarski, [*A lattice-theoretical fixpoint theorem and its applications*](https://doi.org/10.2140/pjm.1955.5.285) (1955). The order-theoretic fixed-point theorem behind many least and greatest fixed-point constructions. 7. Stephen Kleene, [*Introduction to Metamathematics*](https://en.wikipedia.org/wiki/Stephen_Cole_Kleene). A classical source for computability and iterative least fixed points. 8. L. E. J. Brouwer, [*Uber Abbildung von Mannigfaltigkeiten*](https://doi.org/10.1007/BF01456931) (1911), and Shizuo Kakutani, [*A generalization of Brouwer's fixed point theorem*](https://doi.org/10.1215/S0012-7094-41-00838-4) (1941). The topological and set-valued fixed-point theorems behind equilibrium arguments. --- ### Number Theory and the Arithmetic of Recurrence *Published: 2025-06-11* > Once a rule repeats in time, integers appear. Recurrence, resonance, continued fractions, and KAM theory show how arithmetic decides which motions are stable and which fall apart. URL: https://federicocarrone.com/series/fixed-point-ladder/number-theory-and-recurrence/ The [first three essays](@/series/fixed-point-ladder/2025-06-08-fixed-points-and-attractors/index.md) were about one engine: iteration. A rule is applied again and again, and invariant objects appear. Fixed points. Attractors. Invariant sets. Invariant measures. Scaling laws. None of those systems knew anything about themselves. The logistic map does not contain its own formula. It is a rule being turned, and we watched what the turning left behind. This essay is still about that engine, but it follows it to a place that looks, at first, like a different subject entirely: number theory. The reason number theory has to enter is simple once you say it out loud. **The moment a system repeats in time, integers appear.** One cycle. Two cycles. Three. "Come back after $q$ steps." "Line up after $p$ turns of one oscillator and $q$ turns of another." Repetition is counting, and counting is arithmetic. So as soon as we take the single trajectory seriously, asking whether it returns, how nearly it returns, and how often, we are doing number theory whether we meant to or not. That is the whole content of this essay. Whether a motion is stable can come down to whether a single ratio is rational, and if it is irrational, *how* irrational it is. By the end, a frequency being "hard to approximate by fractions" will be the difference between a structure that survives a disturbance and one that disintegrates. There is also a second, deeper role for arithmetic waiting at the far end. The same integers that count returns can also be used to *encode* things: formulas, programs, proofs. When that happens, a system can begin to act on descriptions of itself, and the series changes engines. But that is the [next essay](@/series/fixed-point-ladder/2025-06-12-logic-self-reference-and-category-theory/index.md). Here, arithmetic is still the clock and the ruler of dynamics, not yet a language a system speaks about itself. ## The Minimum Vocabulary A **frequency** is a rate of rotation or oscillation: how many full turns or cycles happen per unit of time. A clock hand has a frequency. A planet orbiting the sun has one. A pendulum has one. Two frequencies are **commensurable** if their ratio is a rational number, like $2/3$ or $7/4$. They are **incommensurable** if their ratio is irrational, like $\sqrt{2}$ or the golden ratio. This distinction will turn out to decide whether two repeating motions ever line up again exactly. A **resonance** occurs when integer combinations of frequencies nearly cancel: $$k\cdot\omega\approx 0.$$ Do not let the notation hide the picture. $\omega$ is a list of frequencies, $k$ is a list of whole numbers, and the dot product asks: is there a small whole-number recipe that makes these frequencies almost cancel out? When the answer is yes, a small push delivered at the right rhythm can build up instead of averaging away. A child on a swing learns this physically: push at the right frequency and small pushes accumulate into a large arc. For a first intuition from linear algebra, think of solving a system of equations and finding a denominator that is nearly zero. The answer blows up. Near-resonances are exactly the places where the denominators of perturbation theory become nearly zero, and so the corrections blow up. We will make that precise later. **Diophantine approximation** is the part of number theory that asks how well an irrational number can be approximated by fractions $p/q$. This is the technical heart of the essay. Near-rational frequencies create near-returns, near-returns create near-resonances, and near-resonances decide stability. So "how close is this number to a fraction?" is not idle arithmetic. It is the control parameter. ## Two Engines, And Why This One Reaches Arithmetic Before going further, a word on where this essay sits, because the series is about to change character. The claim of the series has never been that chaos, fractals, power laws, Gödel sentences, and recursive types are secretly the same object. They are not. They live in different mathematical worlds. The claim is narrower and more defensible: **two related mechanisms keep forcing invariant objects to appear.** The first mechanism is **iteration**. Take a rule, $$x\mapsto f(x),$$ and apply it over and over. That alone is enough to produce everything in the first three essays: fixed points, attractors, cycles, bifurcations, chaos, fractals, invariant measures, and power-law scaling. The system never represents itself. It is simply turned. The second mechanism is **self-reference**: a system becomes rich enough to represent its own rules, and then act on those representations. That is the world of Gödel numbering, Turing machines, quines, and recursive types. It is the subject of the next essay, not this one. This essay is the bridge between them, and number theory is the bridge's keystone. Here is why. Number theory plays two completely different roles depending on which engine you are running. | Role of arithmetic | What arithmetic does | Engine | | --- | --- | --- | | Timekeeper | counts returns, periods, near-returns | iteration | | Stability filter | controls small denominators in perturbation theory | iteration | | Encoding substrate | turns formulas and proofs into numbers | self-reference | | Self-reference medium | lets statements talk about their own codes | self-reference | The top two rows are this essay. Integers count how a repeated motion comes back, and how irrational a frequency is decides whether a structure survives being disturbed. The bottom two rows are the next essay, where the same integers stop being a clock and become an alphabet. Keeping these roles apart is the single most useful habit for reading the rest of the series. The same subject, number theory, hinges the two halves because it can do both jobs. A fixed point, by itself, is often a dull equation. The mathematics comes alive when you ask how it is reached, how it breaks under a small change, and whether the same shape reappears one level up. That is the spirit of everything below. ## Why Number Theory Suddenly Appears At first, number theory seems unrelated to dynamics. It is not. **Number theory** is the study of integers and the structures built from them: divisibility, primes, congruences, rational numbers, irrational numbers, and how well one can be approximated by the other. That sounds far from rolling marbles and folding intervals. But recall the observation we started with: the instant a system repeats, integers walk in. One return. Two returns. A motion that closes after $q$ steps. Two oscillators that realign after $p$ turns of one and $q$ of the other. Repetition counts, and counting is the door. The basic question we need is the simplest one number theory can ask: > How close is a real number to a fraction? That is Diophantine approximation, named after Diophantus. We will see that it controls whether a repeating motion comes back near where it started, and how a small disturbance lands when it does. ### A circle is the simplest place repetition lives To see arithmetic enter, take the cleanest possible repeating system: a point hopping around a circle by a fixed amount each step. $$\theta_{n+1}=\theta_n+\omega \pmod 1.$$ Read $\theta$ as a position on a clock face, except the clock is normalized so that $0$ and $1$ are the same point, the way 12 o'clock and 0 o'clock are the same place. Each step adds $\omega$ and, when the total passes $1$, wraps back around. The number $\omega$ is the **rotation number**: how far around you move per step. The phrase $\pmod 1$ just means "keep only the fractional part," because going all the way around brings you back. This is the one-step version of every clock, orbit, and oscillation in the essay. Everything else is this with more dimensions. Now ask the only interesting question: does the point ever return exactly to where it started? **Case one: a rational rotation.** Suppose $$\omega=\frac{p}{q},$$ a fraction in lowest terms. Then after $q$ steps, $$\theta_q=\theta_0+q\cdot\frac{p}{q}=\theta_0+p\equiv \theta_0 \pmod 1.$$ The $+p$ is a whole number of full turns, which $\pmod 1$ erases. So the point lands exactly back on its start. The orbit is **periodic**, and its period is the denominator $q$. Rational rotation means exact recurrence. The denominator literally tells you how many steps the cycle takes. **Case two: an irrational rotation.** Suppose $\omega$ is irrational. Then the orbit never returns exactly, because $q\omega$ is never a whole number for any $q$, so $\theta_q$ is never exactly $\theta_0$. But something subtler happens: the orbit comes arbitrarily close to every point on the circle. Given enough steps, it will pass as near as you like to any target. Mathematicians say the orbit is **dense**. It never repeats, yet it eventually visits the whole circle. So already, from one line of arithmetic, we have three regimes: 1. rational rotation: exact recurrence, 2. irrational rotation: no exact recurrence, but the orbit fills the circle, 3. *well-approximated* irrational rotation: no exact recurrence, but very close near-returns. The third regime is where the real action is, and it is the one that needs Diophantine approximation to describe. Dynamics often does not care whether a motion returns *exactly*. It cares whether it returns *close enough, often enough,* for a small repeated push to add up. That is a question about how well $\omega$ can be approximated by fractions. ### Measuring "how close to a fraction" For an irrational $\omega$, the precise quantity is $$|q\omega-p|,$$ asking how small this can be made by choosing integers $p$ and $q$. It measures the gap between $\omega$ and the nearest fraction $p/q$, because dividing through by $q$ gives $$\left|\omega-\frac{p}{q}\right|=\frac{|q\omega-p|}{q}.$$ Why does this number matter dynamically? Because if $|q\omega-p|$ is tiny, then after $q$ rotations the point is extremely close to its start: $$\theta_q=\theta_0+q\omega\approx \theta_0+p\equiv\theta_0\pmod 1.$$ That is a **near-period**. The motion does not close, but it almost does, and "almost" can be enough. If some small external forcing nudges the system once per rotation, then at a near-period the nudges arrive at nearly the same phase again and again, all pushing in nearly the same direction. Over many near-returns, those aligned pushes accumulate. That accumulation is what "resonance" means, written in the language of arithmetic. A frequency that admits very good rational approximations is a frequency that resonates easily. ### Continued fractions: the right ruler for irrationality To talk about *how well* a number can be approximated, we need the right tool. That tool is the **continued fraction**. Every irrational number has a unique continued fraction expansion, written compactly as $$\omega=[a_0;a_1,a_2,a_3,\ldots],$$ which unpacks into a nested tower of fractions, $$\omega=a_0+\cfrac{1}{a_1+\cfrac{1}{a_2+\cfrac{1}{a_3+\cdots}}}.$$ The whole numbers $a_0,a_1,a_2,\ldots$ are called the partial quotients. If you stop the tower early, you get a fraction called a **convergent**, and the convergents are, in a precise sense, the *best possible* rational approximations of $\omega$ for their size of denominator. They are the fractions a clockmaker would choose. The crucial fact is that the size of the partial quotients tells you how approximable the number is. A large partial quotient means an unusually good approximation is available, because it means the next convergent barely changes the value while sharply shrinking the error. A number with large partial quotients is *easy* to approximate by fractions, and so it resonates easily. Two examples make this concrete: $$\sqrt{2}=[1;2,2,2,\ldots],\qquad \varphi=[1;1,1,1,\ldots].$$ The golden ratio $\varphi=(1+\sqrt 5)/2$ has the smallest possible partial quotients, all ones, forever. Since large partial quotients are what produce exceptionally good rational approximations, having all ones means the golden ratio *never* offers one. It is, in a precise and provable sense, the hardest irrational number to approximate by fractions. The poetic phrase is that it is "the most irrational number." The concrete content is that $|q\varphi-p|$ refuses to get small too quickly as $q$ grows. This is not a curiosity; it is why phyllotaxis works. A sunflower head or an aloe rosette places each new seed or leaf one fixed angle of rotation from the last. If that angle is a rational fraction of a full turn, the seeds line up into a few radial spokes and waste space. The golden angle is a fraction $1/\varphi^2$ of a full turn, about $137.5^\circ$, equivalently the complementary turn $1/\varphi$ modulo one full rotation. It is the rotation that *never* lets the seeds line up, so they pack with no preferred direction and no gaps. Phyllotaxis is the circle-rotation map run in a flower, choosing the most irrational rotation number for the most even packing. The same fact that makes the golden ratio fill a seed head evenly will, in a moment, make a particular orbit in the solar system the most robust one. ![A conceptual plot comparing small divisors for the golden ratio and an easily approximated irrational number](/images/articles/fixed-point-small-divisors.svg) ### Simulation: Small Divisors {{ fixed_point_sim(name="small-divisors", caption="Small values of |qω - p| are near-resonances. The golden ratio avoids exceptionally small denominators better than many other irrationals.") }} The plot shows $$|q\omega-p|$$ with $p$ chosen as the nearest integer to $q\omega$, swept over many denominators $q$. Compare the golden ratio against a number engineered to have a very good rational approximation: ```python import numpy as np import matplotlib.pyplot as plt phi = (1 + np.sqrt(5)) / 2 easy = np.sqrt(2) + 1 / 10_000 qs = np.arange(1, 400) def small_divisors(omega): ps = np.round(qs * omega) return np.abs(qs * omega - ps) plt.semilogy(qs, small_divisors(phi), label="golden ratio") plt.semilogy(qs, small_divisors(easy), label="easier approximation") plt.xlabel("q") plt.ylabel("|qω - p|") plt.legend() ``` What the reader should see: irrational numbers are not interchangeable. Some produce denominators that dip dangerously small; the golden ratio keeps its distance. In a moment, those small denominators are exactly where stability breaks. ### The ruler is itself a dynamical system Here is a loop back to the earlier essays that is too pretty to skip. Continued fractions are not produced by hand. They come out of a dynamical system, the **Gauss map**: $$G(x)=\left\{\frac{1}{x}\right\},$$ where $\{\cdot\}$ means "take the fractional part." Feed in a number between $0$ and $1$, take its reciprocal, throw away the integer part, and repeat. Each application peels off the next partial quotient of the continued fraction. The tool we use to measure irrationality is itself a rule being iterated. And like the chaotic maps of the second essay, the Gauss map has an invariant measure, the **Gauss measure**: $$d\mu(x)=\frac{1}{\log 2}\frac{dx}{1+x}.$$ This is the distribution left unchanged when you push a whole density of starting numbers through $G$ once, exactly the kind of invariant object the [power-laws essay](@/series/fixed-point-ladder/2025-06-10-power-laws-extremistan-and-non-ergodicity/index.md) introduced as a fixed point of the operator that moves distributions forward. So continued fractions tie together recurrence, chaos, invariant measures, and arithmetic in one small map. The instrument we built to study the ladder turns out to be another rung of it. ### Simulation: Gauss Map {{ fixed_point_sim(name="gauss-map", caption="Move the initial number and replay the orbit. The Gauss map G(x) = {1/x} generates continued-fraction digits, showing that rational approximation is itself driven by a chaotic dynamical system.") }} ## From One Frequency to Many: KAM Theory So far there has been one rotation number on one circle. Real mechanical systems, a planet pulled by several others, a set of coupled pendulums, an asteroid between Jupiter and the sun, have several frequencies at once. The question of recurrence becomes the question of whether *all* of those frequencies realign, and that is where the deepest result in this essay lives: KAM theory, named for Kolmogorov, Arnold, and Moser. To get there, we need a little of the natural language of mechanics. The systems of classical mechanics are usually written in **Hamiltonian** form, an energy-preserving bookkeeping invented for exactly these problems. Their state lives in **phase space**, which records not just where things are but how fast they are moving. For one particle in one dimension, phase space has two coordinates, $$q=\text{position},\qquad p=\text{momentum},$$ and for many moving parts it has many such pairs. A phase-space point is a complete snapshot: positions and momenta together. A Hamiltonian system is called **integrable** when it is solvable cleanly enough that its motion lies on a smooth surface called an **invariant torus**. A torus is the shape of a donut's surface, and in higher dimensions it is the product of several circles. The point of the name is that motion on it is **quasi-periodic**: the system winds around the donut with several frequencies at once, like several independent clock hands turning at incommensurable rates, never quite repeating but never wandering off the surface either. The circle rotation from before is the one-clock version. The torus version is $$\theta_{n+1}=\theta_n+\omega \pmod 1,\qquad \theta=(\theta_1,\ldots,\theta_d),\quad \omega=(\omega_1,\ldots,\omega_d).$$ Each component is one angle on one circle; each step advances all of them. If the frequencies have rational relations among them, the orbit eventually closes onto a lower-dimensional loop. If they do not, it winds quasi-periodically and fills the torus, the multi-clock version of the dense orbit on a single circle. ### The question KAM answers Integrable systems are rare and special. Real systems are integrable systems plus a small disturbance: the planets would trace perfect ellipses if they only felt the sun, but they also tug faintly on each other. The natural question is whether the beautiful invariant tori survive that tug, or whether the smallest perturbation tears them apart and lets the system wander freely, which over astronomical time would mean the solar system is not stable. KAM theory's answer is strange and precise: **it depends on arithmetic.** Tori whose frequencies are "sufficiently irrational" survive a small perturbation. Tori whose frequencies are close to a resonance are destroyed first. Stability is sorted not by energy or size but by how well a frequency vector can be approximated by integer relations. For this section, keep one picture in mind: a torus is a smooth track in phase space, and KAM asks whether that track survives when the system is nudged. The arithmetic enters because repeated nudges are harmless when they arrive out of phase, but dangerous when they keep arriving almost in sync. A resonance among several frequencies is the multi-dimensional version of the near-period. It occurs when $$k\cdot \omega \approx 0$$ for some vector of integers $k$ that is not all zeros. Written out in two dimensions, $$k_1\omega_1+k_2\omega_2\approx 0,$$ which says one frequency is nearly a whole-number multiple of the other; the two clocks nearly relock. The dot product $k\cdot\omega$ is the multi-frequency generalization of the quantity $q\omega-p$ from the circle. Exactly zero is an exact resonance; nearly zero is a near-resonance. ### Why near-resonance is dangerous: small divisors This is the mechanism, and the technical core of the whole essay, so it pays to go slowly. When physicists want to understand a slightly-perturbed system, they try to "change coordinates" to make the perturbation disappear, one Fourier piece at a time. A typical piece of the perturbation looks like a wave around the torus, $$a_k\, e^{i k\cdot\theta},$$ and the coordinate change that cancels it comes out looking like $$\frac{a_k}{i\,k\cdot\omega}\, e^{i k\cdot\theta}.$$ The exact details vary by system, but the shape is robust and is all we need: the correction has $k\cdot\omega$ in the **denominator**. Now read off the three cases. If $k\cdot\omega$ is comfortably away from zero, the correction is small and controlled. The perturbation can be absorbed. If $k\cdot\omega$ is tiny, the correction is enormous, because we are dividing by a near-zero number. A small perturbation produces a large response. If $k\cdot\omega$ is exactly zero, the correction is infinite. The perturbation is in perfect resonance with the motion and cannot be removed by any small change of coordinates at all. This is the **small-divisor problem**, and it is the same arithmetic we met on the circle, now with stakes. There, $|q\omega-p|$ being small meant a near-period. Here, $|k\cdot\omega|$ being small means a near-resonance that perturbation theory divides by. The survival of a geometric structure, an entire invariant torus in phase space, comes down to whether these denominators can be kept away from zero. The engineering analogy is resonance in a bridge or circuit. A tiny input is not dangerous by itself; it becomes dangerous when the timing lines up so the response adds instead of cancels. KAM is the high-dimensional, geometric version of that timing problem. ### The Diophantine condition So KAM needs frequencies whose small divisors stay controlled. The precise requirement is a **Diophantine condition**: $$|k\cdot \omega| \geq \frac{\gamma}{|k|^\tau}\quad\text{for all nonzero integer vectors }k.$$ In words: resonances are allowed to creep closer to zero as the integer vector $k$ grows larger, but not too fast. The constants $\gamma$ and $\tau$ measure how stubbornly irrational the frequency vector is. This inequality is not decorative. It is the promise that keeps the denominators in the coordinate change from collapsing, which is exactly what KAM needs to prove a torus survives. Which frequencies satisfy it best? The maximally irrational ones. In two dimensions, the most robust torus of all is the one whose frequency ratio is the golden ratio, $$\varphi=\frac{1+\sqrt 5}{2}=[1;1,1,1,\ldots].$$ Its all-ones continued fraction makes it the hardest number to approximate by rationals, which makes its small divisors the largest, which makes its torus the last to break. The golden torus is robust not because gold is mystical but because the arithmetic of $\varphi$ is the worst possible case for resonance. This is the same fact that packed the sunflower; here it stabilizes an orbit. ### What the surviving phase space looks like KAM does not promise a clean split into "all stable" or "all chaotic." It promises a *mixture*, and the mixture has an arithmetic skeleton. Sufficiently irrational tori survive as smooth barriers. Tori near resonances dissolve, and around each resonance appear islands and thin chaotic layers. Between them, the surviving Diophantine tori act as walls that trajectories cannot cross. Discrete arithmetic facts, which ratios are near rational and which are stubbornly irrational, carve continuous geometric structure into the phase space. That is one of the most beautiful statements in mathematical physics: a number-theoretic property of a frequency decides a geometric property of a flow. Saturn's rings make the same point visible from a backyard telescope. The dark Cassini Division in the photograph at the top of this essay is a band swept nearly empty because particles orbiting there would fall into a 2:1 resonance with the moon Mimas, the same nudge arriving at nearly the same phase orbit after orbit until it accumulates. There is a standard way to write the setup. Start from an integrable system in action-angle coordinates, where the unperturbed motion is just rotation, $$\dot{\theta}=\omega(I),\qquad \dot{I}=0,$$ so the angles turn at frequencies $\omega$ and the actions $I$ stay fixed, pinning the motion to a torus. Now add a small perturbation, $$H(I,\theta)=H_0(I)+\epsilon H_1(I,\theta).$$ Removing it order by order produces exactly the small-divisor denominators $k\cdot\omega$ described above. Where those denominators stay large, the expansion converges and the torus survives, slightly bent. Where they vanish, the expansion breaks and the torus is destroyed. There is a principle here that reaches well past celestial mechanics: what survives a repeated disturbance often stays *out of phase* with it. Resonance, things lining up, is one road to catastrophe; refusing to line up is one road to robustness. The same shape recurs wherever a system has to survive repeated shocks. A sunflower avoids wasted spokes by giving every seed an irrational turn. A portfolio survives a crash by holding positions that do not all move together. A financial system survives a shock when its institutions run *different* risk models, and detonates when they run the same one and deleverage in unison, the [monoculture failure](@/series/edge-of-chaos/2025-06-06-when-risk-models-create-risk/index.md) that closes the companion series. Diversification, biodiversity, and a surviving KAM torus are not the same mechanism, but they share a defensive shape: avoid alignment, and a shock that would amplify in unison arrives scattered instead. ### KAM as a fixed-point statement This connects back to the spine of the series; it is another fixed-point story wearing unfamiliar clothes. KAM does not say the old tori sit unchanged. It says many of them *deform* into nearby tori, and asks whether there exists a change of coordinates that carries the perturbed motion back to clean quasi-periodic rotation on the deformed torus. In fixed-point language, you are solving for an invariant object: a map $K$ that embeds an abstract torus into the real phase space so that running the physical dynamics matches simply rotating the abstract torus. Schematically, $$\text{flow}\circ K = K\circ \text{rotation}_\omega.$$ Read it as: embed the torus, then flow, and you get the same result as rotating first, then embedding. The unknown $K$ is not a number, and not even a point. It is a whole geometric surface left invariant by the dynamics. KAM is the question of whether that invariant surface survives perturbation, and the answer is decided by the arithmetic of $\omega$. The takeaway is modest but important: KAM is not asking a student to memorize a theorem. It is showing that stability can depend on arithmetic. Two systems can look physically similar, but the one whose frequencies avoid near-integer relations can keep a smooth invariant torus while the resonant one breaks into islands and chaos. The ninth lesson: **Number theory shapes phase space because arithmetic controls resonance.** ## The Bridge: From Non-Ergodicity to Recurrence Why does number theory belong *here*, right after the [previous essay](@/series/fixed-point-ladder/2025-06-10-power-laws-extremistan-and-non-ergodicity/index.md) on non-ergodicity, rather than as a detour? That essay ended with a warning: an ensemble average can be the wrong object when the lived path is a single trajectory through time. The instant the single trajectory becomes the thing you care about, recurrence becomes the question. Does the path return near where it started? Does it revisit a region often enough for averages to settle? Does it miss whole regions entirely? Does a small periodic forcing keep hitting the system at nearly the same phase? These are time questions first. They become number-theory questions the moment the motion is rotational, oscillatory, or quasi-periodic, because then "return near the start" translates directly into arithmetic. For the circle rotation, $$\theta_n=\theta_0+n\omega \pmod 1,$$ a near return means $\theta_n\approx\theta_0$, which after subtracting $\theta_0$ becomes $n\omega\approx p$ for some integer $p$, that is, $$|n\omega-p|\approx 0.$$ So the lived-path question "does this trajectory come back near where it started?" is the same question as "how well can $\omega$ be approximated by rationals?" That is the concrete reason recurrence drags arithmetic onto the ladder. A rational ratio gives eventual exact closure. An irrational ratio gives no closure, but irrationality is not one thing: some irrationals are easy to approximate and resonate readily, while others, the golden ratio chief among them, resist. That difference controls near-recurrence; near-recurrence controls resonance; resonance controls stability. So the chain is tight. Once time matters, repeated motion matters. Once repeated motion matters, the question is whether cycles line up, nearly line up, or never line up, and that question is arithmetic. The integer vector $k$ is not decorative notation. It is the bookkeeper of every possible way the frequencies could realign. ## Arithmetic As The Hinge This is the point where the series is about to change engines, and number theory is the hinge it turns on. Everything above used arithmetic in one way: as the **timekeeper and stability filter** of dynamics. Integers count returns. Fractions describe frequency ratios. Continued fractions measure near-resonance. Diophantine inequalities decide which structures survive a disturbance. In every case, arithmetic stayed *outside* the system, a ruler we held up against the motion. But arithmetic can be used a second way, and the difference is the whole pivot of the series. | Role of arithmetic | What arithmetic does | Example | | --- | --- | --- | | Timekeeper | counts returns, periods, resonances | $q\omega\approx p$ | | Stability filter | controls small denominators | $\lvert k\cdot\omega\rvert\geq\gamma/\lvert k\rvert^\tau$ | | Encoding substrate | turns syntax into numbers | Gödel numbering | | Self-reference medium | lets statements refer to their own codes | the diagonal lemma | The first two rows are this essay, the iteration engine. The bottom two rows are the next essay, the self-reference engine, and they rest on a single new idea: the same integers that count symbols going around a clock can also count symbols inside a formula. A statement, a proof, a program is just a finite string of symbols, and a finite string of symbols can be packed into a single integer. Once that is done, arithmetic stops being a ruler held up against the system and becomes a *language the system can speak about itself.* Numbers can encode statements about numbers. A rule can be fed its own description. That is the threshold of representational closure, and it is where iteration's invariant objects, attractors and tori, give way to a new family: Gödel sentences, halting problems, quines, the Y combinator, and recursive types. The same fixed-point question carries across, but the engine driving it is different. We have now followed the iteration engine as far as it goes, from a number pulled toward $1$ all the way to an invariant torus held together by the golden ratio. The [next essay](@/series/fixed-point-ladder/2025-06-12-logic-self-reference-and-category-theory/index.md) turns the hinge: it lets arithmetic encode syntax, and watches self-reference become the second engine that manufactures fixed points. {% toolkit(label="Takeaway") %} - **Space:** the invariant tori of phase space, the multi-clock version of a circle. - **Transformation:** a perturbed Hamiltonian flow, an integrable system plus a small tug. - **Invariant object:** the quasi-periodic tori whose frequencies are sufficiently irrational to survive. - **Failure mode:** resonance and small denominators, which tear the near-rational tori apart. {% end %} ## Further Reading For number theory inside dynamics: 1. Andrey Kolmogorov, [*On conservation of conditionally periodic motions for a small change in Hamilton's function*](https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Arnold%E2%80%93Moser_theorem) (1954). The original KAM note, the K in KAM. 2. Vladimir Arnold, [*Mathematical Methods of Classical Mechanics*](https://en.wikipedia.org/wiki/Mathematical_Methods_of_Classical_Mechanics). The canonical route into Hamiltonian mechanics, action-angle variables, and the small-divisor problem. 3. Jurgen Moser, [*Stable and Random Motions in Dynamical Systems*](https://en.wikipedia.org/wiki/J%C3%BCrgen_Moser). A classic treatment of KAM ideas and the survival of invariant tori. 4. Hendrik Broer and Floris Takens, [*Dynamical Systems and Chaos*](https://doi.org/10.1007/978-1-4419-6870-8). Useful for connecting invariant tori, resonance, and Diophantine conditions to bifurcations. 5. A. Ya. Khinchin, [*Continued Fractions*](https://en.wikipedia.org/wiki/Continued_fraction). The short classic on continued fractions, convergents, and how well numbers can be approximated by rationals. 6. G. H. Hardy and E. M. Wright, [*An Introduction to the Theory of Numbers*](https://en.wikipedia.org/wiki/An_Introduction_to_the_Theory_of_Numbers). Background on Diophantine approximation and the special role of the golden ratio. --- ### Power Laws, Extremistan, and Non-Ergodicity *Published: 2025-06-10* > Scale invariance becomes power laws; power laws move importance to extremes; extremes make averages unstable through time. URL: https://federicocarrone.com/series/fixed-point-ladder/power-laws-extremistan-and-non-ergodicity/ The [previous essay](@/series/fixed-point-ladder/2025-06-09-chaos-fractals-and-renormalization/index.md) ended with fractals: objects whose structure survives repeated zooming. This essay turns that geometric statement into a statistical one. If an object has no characteristic length scale, then counting it often produces a power law. If a distribution has a power-law tail, then extremes stop being negligible. If extremes stop being negligible, then averages become unstable. If the process is multiplicative or has absorbing ruin states, then the average over many possible worlds can become irrelevant to the trajectory one person actually lives. That is the path: $$\text{scale invariance}\to\text{power laws}\to\text{fat tails}\to\text{non-ergodicity}.$$ This is where Mandelbrot, Taleb, Peters, Kelly, Wilson, Bak, and the central limit theorem start belonging to the same conversation. ## The Minimum Vocabulary A **distribution** tells you how likely different outcomes are. For a continuous variable, the density $p(x)$ describes probability per unit of $x$. The survival function: $$P(X>x)$$ asks for the probability of seeing an outcome larger than $x$. A **tail** is the far end of a distribution: rare large values. A **thin-tailed** distribution, like a Gaussian, makes very large events disappear extremely fast. A **fat-tailed** distribution makes them disappear slowly enough that extremes remain structurally important. A **power law** is a scaling relationship: $$P(X>x)\sim Cx^{-\alpha}.$$ Multiplying $x$ by a constant multiplies the probability by another constant. There is no preferred scale. An **ensemble average** averages across many possible worlds or many parallel copies of a process. A **time average** follows one process through time and averages along that single trajectory. A process is **ergodic** when the time average and ensemble average agree in the relevant long-run sense. It is **non-ergodic** when they do not. Multiplicative wealth, ruin, and path-dependent systems are often non-ergodic. ## From Scale Invariance to Power Laws Now zoom into a fractal. It looks similar. Zoom again. Still similar. Again. Still similar. There is no characteristic scale. That is scale invariance. This section is sometimes summarized as "fractals create power laws," but that sentence is too crude. The real implication is: $$\text{scale invariance}\Rightarrow\text{power-law form}.$$ Fractals are one geometric source of scale invariance. They are not the only source, and not every fractal measurement gives a clean power law. Markets, cities, earthquakes, languages, and networks can produce scaling through mechanisms that are not literally fractal geometry: preferential attachment, multiplicative growth, criticality, mixtures of regimes, and renormalization. The common object is not "fractal" but "no preferred scale." A **characteristic scale** is a typical size. Human adult height has a characteristic scale. You can talk about a normal height. Earthquake size, city size, wealth, firm size, and market crashes often do not behave that way. There are many small events, fewer medium events, and rare enormous events, with no single size that organizes the whole distribution. A **power law** is a relationship where multiplying the input by a fixed amount multiplies the output by a fixed amount. The simplest form is: $$y=Cx^{-\alpha}.$$ If you double $x$, then: $$y(2x)=C(2x)^{-\alpha}=2^{-\alpha}Cx^{-\alpha}=2^{-\alpha}y(x).$$ So every doubling of size reduces frequency by the same factor. That is what "no characteristic scale" means in practice. The distribution does not care whether you move from $10$ to $20$, $100$ to $200$, or $1000$ to $2000$. The multiplicative relationship is the same. Scale invariance has a simple mathematical form. Suppose a distribution $P(x)$ satisfies: $$P(\lambda x)=\lambda^{-\alpha}P(x).$$ The solutions are power laws: $$P(x)=Cx^{-\alpha}.$$ Here is the more general derivation. Scale invariance first says that rescaling $x$ changes $P$ only by a scale-dependent multiplier: $$P(\lambda x)=c(\lambda)P(x).$$ Now rescale twice. Scaling by $\mu$ and then by $\lambda$ must agree with scaling once by $\lambda\mu$: $$P(\lambda\mu x)=c(\lambda)c(\mu)P(x).$$ But the same left-hand side is also: $$P(\lambda\mu x)=c(\lambda\mu)P(x).$$ So: $$c(\lambda\mu)=c(\lambda)c(\mu).$$ The regular solutions of this multiplicative equation are: $$c(\lambda)=\lambda^{-\alpha}.$$ Therefore: $$P(\lambda x)=\lambda^{-\alpha}P(x),$$ and the compatible shapes are power laws: $$P(x)=Cx^{-\alpha}.$$ So the power law does not come from visual roughness by itself. It comes from consistency under repeated rescaling. Here I am using $P(x)$ informally for a scale-dependent quantity. In probability, one has to distinguish two related exponents. If the **density** behaves like $p(x)\sim Cx^{-\beta}$, then the **survival function** behaves like: $$P(X>x)\sim C'x^{-(\beta-1)}.$$ The survival exponent is often called the **tail index**. In the next section, when I write: $$P(X>x)\sim x^{-\alpha},$$ $\alpha$ is the tail index, not the density exponent. This convention is common in discussions of Pareto tails, but it is worth making explicit because the two exponents differ by one. You can see this by taking logarithms. If: $$P(x)=Cx^{-\alpha},$$ then: $$\log P(x)=\log C-\alpha \log x.$$ So on log-log axes, the distribution becomes a straight line. ![A log-log chart showing a power law as a straight line and a Gaussian tail bending downward](/images/articles/fixed-point-scale-invariance.svg) ### Graph: Power Law Versus Gaussian {{ fixed_point_sim(name="power-law", caption="On log-log axes, a power law is a straight line. Thin-tailed distributions bend downward because their large events vanish much faster.") }} ```python import numpy as np import matplotlib.pyplot as plt x = np.logspace(0, 3, 400) power = x ** -1.7 gaussian_tail = np.exp(-x**2 / 2_000) plt.loglog(x, power, label="power law") plt.loglog(x, gaussian_tail, label="Gaussian-like tail") plt.legend() ``` ### Simulation: Power-Law Sampling {{ fixed_point_sim(name="sampling", caption="The Gaussian running mean settles. The Pareto running mean keeps being rewritten by new extremes.") }} Compare running sample means from Gaussian and Pareto draws. ```python import numpy as np import matplotlib.pyplot as plt rng = np.random.default_rng(4) n = 50_000 gaussian = rng.normal(loc=1.0, scale=1.0, size=n) pareto = rng.pareto(a=1.4, size=n) + 1 gaussian_mean = np.cumsum(gaussian) / np.arange(1, n + 1) pareto_mean = np.cumsum(pareto) / np.arange(1, n + 1) plt.plot(gaussian_mean, label="Gaussian running mean") plt.plot(pareto_mean, label="Pareto running mean") plt.legend() ``` What the reader should see: the Gaussian mean stabilizes. The Pareto mean keeps jumping when a new extreme appears. More data does not necessarily make a fat-tailed average feel calm. Fractals are geometric scale invariance. Power laws are statistical scale invariance. Same phenomenon. Different viewpoint. The sixth lesson: **Power laws are the probability version of scale invariance. Fractals are one geometric route to that invariance, not the whole story.** There is another way to see why scale invariance forces power laws. Let: $$Q(t)=\log P(e^t).$$ The scaling law: $$P(\lambda x)=\lambda^{-\alpha}P(x)$$ becomes, with $\lambda=e^s$ and $x=e^t$: $$Q(t+s)=Q(t)-\alpha s.$$ The only continuous solutions are affine: $$Q(t)=C-\alpha t.$$ Returning to $x=e^t$ gives: $$P(x)=e^C x^{-\alpha}.$$ So a power law is not an arbitrary curve. It is the unique distributional shape compatible with translation invariance in log-space. Zooming becomes shifting. Self-similarity becomes linearity. There is one more principled derivation, and it is the cleanest of all: maximum entropy. Fix only one thing about a positive quantity, its mean *logarithm* $\mathbb{E}[\log x]$. That is the natural constraint when the quantity spans many orders of magnitude, where what stays stable is the typical number of digits rather than the typical value. Among all densities on $x\ge x_{\min}$ that honor that constraint, the one assuming nothing further is the maximum-entropy density: maximize $-\int p\log p$ subject to normalization and fixed $\mathbb{E}[\log x]$. The Lagrange conditions give $$p(x)\propto \exp(-\beta\log x)=x^{-\beta},$$ a power law, with $\beta$ the multiplier that enforces the constraint and $\beta>1$ so the density normalizes above $x_{\min}$. Fix the mean of $x$ instead and you get an exponential; fix the mean of $x^2$ on the whole real line and you get a Gaussian. The power law is simply what minimal commitment looks like when the natural ruler is logarithmic. It is the informational twin of the renormalization picture: scale invariance forces power laws geometrically, and a log-scale constraint forces them by entropy. Log-space linearity is also why power laws are so often the statistical shadow of renormalization. If the system looks the same after coarse-graining, its observables often become eigenfunctions of a scaling operator. The exponent $\alpha$ is the eigenvalue written in statistical form. If that sentence feels too compressed, read it this way: when zooming out leaves the system with the same form, the quantities you measure must transform predictably under zooming. The simplest predictable transformation is multiplication by a constant. Power laws are exactly the functions that do that. The canonical physics example is Wilson's renormalization group for second-order phase transitions. Near a critical point, a magnet, fluid, or lattice model can look statistically similar after coarse-graining. Details of the microscopic system wash out; critical exponents remain. This is the same kind of universality that Feigenbaum found in period doubling, now appearing in equilibrium statistical physics. Self-organized criticality is another route. In Bak, Tang, and Wiesenfeld's sandpile model, grains are added slowly until the system organizes itself near a critical state. Avalanches of many sizes appear. The point is not that every power law comes from a sandpile, but that repeated local rules can create scale-free statistics without an external planner tuning the system by hand. Preferential attachment is a third route, and it is probably the one readers meet most often. If new links, people, capital, or attention attach preferentially to nodes that already have many links, people, capital, or attention, then the large get larger faster. This is the Yule-Simon or rich-get-richer mechanism. It appears in city sizes, word frequencies, citation networks, wealth distributions, internet links, and firm sizes. The toy rule is simple. If node $i$ has size $k_i$, then the probability that the next unit attaches to it is proportional to $k_i$: $$P(i)=\frac{k_i}{\sum_j k_j}.$$ That one reinforcement rule can create a heavy-tailed distribution without invoking fractal geometry or a sandpile. The connection to the ladder is still the same: a repeated local transformation changes the distribution, and the long-run distribution approaches a scale-free fixed shape. There is also a direct probability version of renormalization: the central limit theorem. Take two independent copies of a random variable, add them, and rescale: $$X\mapsto \frac{X_1+X_2}{\sqrt{2}}.$$ Repeat this operation. For distributions with finite variance, the Gaussian is the attracting fixed point. That is why sums of many small independent effects so often look normal. But if the variance is infinite, the Gaussian is no longer the right attractor. The fixed points become Lévy stable laws, many of which have power-law tails. Fat tails are not merely "failed Gaussians." They belong to a different basin of attraction in distribution space. The rescaling exponent is the tell. To keep a finite-variance sum fixed you divide by $\sqrt{n}=n^{1/2}$; a stable law with tail index $\alpha$ instead rescales like $n^{1/\alpha}$. That exponent is not new. It is the same $n^{1/\alpha}$ that governs how fast the maximum grows in the next section. The way fat-tailed sums refuse to shrink and the way fat-tailed maxima refuse to stay small are one fact wearing two faces. This is one of the cleanest ways to understand Extremistan. A Gaussian world and a power-law world are governed by different renormalization fixed points. The bridge from fractals to power laws is measurement. A fractal is scale invariance seen as shape. A power law is scale invariance seen as counting. If a coastline, river network, fault system, market cascade, or city system has structure across many scales, then counting "how many things of size at least $x$?" often produces a power law. The important connection is that both are fixed points of a rescaling operation. For a fractal, rescale the picture and the shape is statistically similar. For a power law, rescale the variable and the distribution changes only by a multiplicative factor: $$P(\lambda x)=\lambda^{-\alpha}P(x).$$ This is not a metaphor. It is the same mathematical form: an object is transformed by zooming, and its essential structure survives. The zoom operation is the transformation; the fractal or power-law distribution is the invariant object. The geometry and the probability are not separate mysteries. They are two ways of observing the same lack of characteristic scale. ## Power Laws Create Extremistan This is where Taleb enters. In a Gaussian world: 1. averages matter, 2. variance is finite, 3. large events are rare enough to ignore most of the time. In a power-law world: 1. extremes dominate, 2. variance may not exist, 3. averages can remain unstable for a very long time. For a Pareto tail: $$P(X>x)\sim x^{-\alpha}.$$ This means: the probability that $X$ exceeds $x$ falls like a power of $x$. The symbol $\sim$ means "asymptotically proportional to." It does not say the equality is exact at every size. It says the tail behaves like that for large values. The mean exists only when: $$\alpha>1.$$ The variance exists only when: $$\alpha>2.$$ So if $1<\alpha\leq 2$, the average exists but the variance is infinite. If $\alpha\leq 1$, even the mean does not exist. This is not a small technicality. It changes what evidence means. In a thin-tailed world, more samples stabilize your estimate quickly. In a fat-tailed world, one new observation can dominate the entire history. One earthquake dominates a century. One company dominates a market. One city dominates a country. One idea dominates an era. The seventh lesson: **Power laws move importance from the average to the extreme.** The operational difference can be seen in the maximum. For $n$ Gaussian observations, the maximum grows slowly, roughly like: $$\sqrt{2\log n}.$$ For $n$ Pareto observations with tail index $\alpha$, the maximum grows like: $$n^{1/\alpha}.$$ That is the whole Extremistan difference. In Mediocristan, the maximum grows like the square root of a logarithm. In Extremistan, it grows as a power of sample size. More observations do not merely refine the average. They create room for a new dominant event. This is why the sample mean can look stable for a while and then jump. It was not converging in the way your Gaussian-trained intuition expected. It was waiting for a new maximum. The bridge from power laws to Extremistan is dominance. In a Gaussian world, no single observation is allowed to matter too much. The central limit theorem is the mathematical expression of this. Many small independent contributions add up, and the aggregate becomes stable. Individual terms disappear into the average. In a power-law world, the largest observation can be the story. The aggregate is not a democratic sum of comparable pieces. It is often an aristocracy of extremes. This changes what it means to understand a system. In Mediocristan, the typical case is informative. If you understand the average human height, you understand a lot about human height. In Extremistan, the typical case can be almost irrelevant. The typical startup does not explain venture returns. The typical earthquake does not explain geological damage. The typical word does not explain language frequency. The typical city does not explain urban concentration. So instead of asking what the ordinary case looks like, you ask: $$\frac{\text{largest few observations}}{\text{total mass}}.$$ If that ratio is large, then averages become summaries of extremes, not summaries of typicality. This is the statistical version of the earlier attractor story. In dynamics, the attractor organizes trajectories. In Extremistan, the tail organizes the sample. The center is no longer sovereign. The edge is. ## Extremistan Breaks Ergodicity Now introduce time. An ensemble average asks what happens across many parallel worlds. A time average asks what happens to one system as it moves through time. These are not the same question. An **ergodic** system is one where, roughly, watching one typical trajectory for a long time gives you the same statistics as looking at many copies of the system at one time. A clean mathematical version is this. Suppose a system evolves by a transformation $T$, and suppose $g(x)$ is some quantity you measure at state $x$. The time average along one trajectory is: $$\frac{1}{N}\sum_{n=0}^{N-1}g(T^n x).$$ The ensemble average is: $$\int g(x)\,d\mu(x),$$ where $\mu$ is the probability distribution over states. Ergodicity says that, under the right conditions: $$\lim_{N\to\infty}\frac{1}{N}\sum_{n=0}^{N-1}g(T^n x)=\int g(x)\,d\mu(x).$$ Do not worry about the measure-theory details. The practical meaning is simple: one long-lived path eventually samples the space fairly. This is the content of Birkhoff's ergodic theorem. It is one of the theorems that makes the phrase "time average equals ensemble average" mathematically precise. Non-ergodicity means this fails. One path through time is not equivalent to many parallel samples. There is another fixed-point object nearby. In a Markov chain, a stationary distribution $\pi$ satisfies: $$\pi=\pi P,$$ where $P$ is the transition matrix. So the long-run distribution is a fixed point of the operator that pushes distributions forward one step. For deterministic chaotic maps, the analogous object is an invariant measure. Instead of asking where one point goes, ask how a whole density of points moves. The transfer, or Perron-Frobenius, operator sends today's density to tomorrow's density. An invariant density $\rho$ satisfies: $$\mathcal{P}\rho=\rho.$$ For the logistic map at $r=4$, the natural invariant density is: $$\rho(x)=\frac{1}{\pi\sqrt{x(1-x)}}.$$ This connects chaos back to ergodicity. A chaotic map can be unpredictable point by point while still having a stable statistical distribution. The point trajectory is unstable; the measure is the fixed object. Suppose wealth evolves multiplicatively: $$W_{t+1}=W_t(1+R_t).$$ The ensemble object is: $$\mathbb{E}[W_t].$$ The time-growth object is: $$\lim_{T\to\infty}\frac{1}{T}\log\left(\frac{W_T}{W_0}\right)=\mathbb{E}[\log(1+R)].$$ The logarithm appears because repeated multiplication becomes repeated addition in log space. If the system has fat tails, ruin states, or rare dominating events, a single trajectory does not sample all possibilities in any useful way. The ensemble average becomes a bad guide to the lived path. That is non-ergodicity: $$\text{time average}\neq \text{ensemble average}.$$ This is why Taleb, Peters, Kelly, and ergodicity economics care so much about fat tails. A strategy can look good in expectation and still be fatal through time. ### Simulation: Ensemble Versus Time {{ fixed_point_sim(name="ergodicity", caption="The ensemble mean can rise while the median lived path stagnates. Time averages and ensemble averages are different objects.") }} ```python import numpy as np rng = np.random.default_rng(7) n_paths = 20_000 n_steps = 300 # Mostly small gains, rare large losses. returns = np.where( rng.random((n_paths, n_steps)) < 0.01, -0.55, 0.008, ) wealth = np.cumprod(1 + returns, axis=1) ensemble_mean = wealth.mean(axis=0) median_path = np.median(wealth, axis=0) time_growth = np.log(wealth[:, -1]).mean() / n_steps ``` Plot the ensemble mean and the median path. They separate. The average gets pulled by winners. The typical path is governed by survival and multiplicative compounding. To make the graph explicit: ```python import matplotlib.pyplot as plt plt.plot(ensemble_mean, label="ensemble mean") plt.plot(median_path, label="median path") plt.yscale("log") plt.xlabel("time") plt.ylabel("wealth") plt.legend() ``` What the reader should see: the ensemble mean can rise while the typical path stagnates or dies. In multiplicative systems, the average path and the lived path are different mathematical objects. The eighth lesson: **Power laws often make systems non-ergodic because rare events dominate long-run outcomes.** A minimal two-outcome example makes the ensemble/time split sharper. Suppose a gamble multiplies wealth by $1.5$ with probability $1/2$ and by $0.6$ with probability $1/2$. The ensemble-average multiplier is: $$\mathbb{E}[M]=\frac{1.5+0.6}{2}=1.05.$$ Across parallel worlds, average wealth rises by 5 percent per round. But the time-average growth rate is: $$g=\frac{1}{2}\log(1.5)+\frac{1}{2}\log(0.6)=\log\sqrt{0.9}<0.$$ One person repeating the gamble goes broke exponentially almost surely, even though the ensemble average grows. The fixed point has moved again: it is now the long-run growth rate of a repeated multiplicative process, and the logarithm is the coordinate system that reveals it. This is why repeated transformation is the right primitive. A one-shot gamble and a repeated gamble are different mathematical objects. Expected value answers the first badly enough in some cases. It answers the second catastrophically when multiplication, ruin, and fat tails enter. The bridge from Extremistan to non-ergodicity is the single path. Power laws tell you that extremes dominate the ensemble. Non-ergodicity asks whether one trajectory gets to experience the ensemble in the right proportions. Often it does not. This is a deeper shift than it first appears. Probability theory often starts by imagining many possible outcomes side by side. But life is not side by side. Life is sequential. You do not get to average over parallel versions of yourself after ruin. You move through one path, in order. That order matters. Multiplicative systems remember losses differently from gains. A 50 percent loss followed by a 50 percent gain is not zero: $$1.0 \times 0.5 \times 1.5 = 0.75.$$ The arithmetic average return is zero, but the path lost 25 percent. The logarithm sees this because logarithms turn multiplication into addition: $$\log(ab)=\log a+\log b.$$ So the correct invariant for repeated wealth dynamics is not expected return. It is expected log growth, survival probability, and drawdown structure. If the rare event kills the process, the process stops before it can average anything. If the rare event creates a giant winner, the ensemble mean may be dominated by paths you will almost surely not live. Time is not a neutral sampling device. It has order, survival, and path dependence. This connects back to attractors. In an ergodic system, a single long trajectory eventually explores the relevant space in the right proportions. In a non-ergodic system, the trajectory gets trapped, ruined, amplified, or path-dependent. The basin you fall into matters more than the ensemble average over all basins. The invariant summary of this essay is: $$ P(\lambda x)=\lambda^{-\alpha}P(x) \quad\Rightarrow\quad P(x)\propto x^{-\alpha}. $$ Scale invariance does not always come from fractals. It can come from renormalization, multiplicative growth, preferential attachment, criticality, or self-organized criticality. The shared object is the absence of a characteristic scale. Once those scale-free distributions enter repeated multiplicative processes, the relevant invariant is not the ensemble expectation: $$ \mathbb{E}[W_t], $$ but the time-average growth rate: $$ \lim_{T\to\infty}\frac{1}{T}\log\frac{W_T}{W_0}. $$ The next essay asks why recurrence and self-reference force arithmetic into the story. {% toolkit(label="Takeaway") %} - **Space:** probability distributions, and wealth paths followed through time. - **Transformation:** rescaling the variable, $x\mapsto\lambda x$; and multiplication through time. - **Invariant object:** a scale-invariant tail, $x^{-\alpha}$; and the time-average growth rate. - **Failure mode:** unstable averages, ensemble mean diverging from the lived path, ruin. {% end %} ## Further Reading 1. Benoit Mandelbrot, [*The Fractal Geometry of Nature*](https://en.wikipedia.org/wiki/The_Fractal_Geometry_of_Nature). The classic source for fractals and scaling. 2. Mark Newman, [*Power laws, Pareto distributions and Zipf's law*](https://arxiv.org/abs/cond-mat/0412004) (2005). A careful first paper on power-law distributions. 3. Aaron Clauset, Cosma Shalizi, and Mark Newman, [*Power-law distributions in empirical data*](https://arxiv.org/abs/0706.1062) (2009). The statistical cautionary reference. 4. Kenneth Wilson, [*The renormalization group and critical phenomena*](https://doi.org/10.1103/RevModPhys.55.583) (1983 Nobel lecture). The clean conceptual source for renormalization and critical exponents. 5. Edwin Jaynes, [*Information theory and statistical mechanics*](https://doi.org/10.1103/PhysRev.106.620) (1957). The maximum-entropy principle, behind the entropy derivation of power laws. 6. Ole Peters, [*The ergodicity problem in economics*](https://doi.org/10.1038/s41567-019-0732-0) (2019). A direct route into time averages, ensemble averages, and multiplicative wealth. 7. Ole Peters and Murray Gell-Mann, [*Evaluating gambles using dynamics*](https://doi.org/10.1063/1.4940236) (2016). The sharpest single statement of the time-average argument. 8. John Kelly, [*A new interpretation of information rate*](https://doi.org/10.1002/j.1538-7305.1956.tb03809.x) (1956). The original Kelly criterion paper. 9. Geoffrey West, [*Why Cities Keep on Growing, Corporations Always Die, and Life Gets Faster*](https://www.youtube.com/watch?v=jOnWowd-7HQ) (The Long Now Foundation). A talk on the scaling laws and power-law exponents that run across biology, cities, and companies. --- ### Chaos, Fractals, and Renormalization *Published: 2025-06-09* > How period-doubling creates chaos, why Feigenbaum universality appears, and how fractals become fixed points of set-valued contraction maps. URL: https://federicocarrone.com/series/fixed-point-ladder/chaos-fractals-and-renormalization/ The [previous essay](@/series/fixed-point-ladder/2025-06-08-fixed-points-and-attractors/index.md) built the local picture. Repeated rules can converge to fixed points, and fixed points can organize nearby trajectories into attractors. But the real subject of this series is not only convergence. It is what happens when convergence starts to fail. The logistic map gives the cleanest path into the failure: $$x_{n+1}=r x_n(1-x_n).$$ At small parameter values, the map settles. Increase the parameter, and the attracting fixed point loses stability. The system does not become random immediately. It first becomes periodic. Then the period doubles. Then it doubles again. The road to chaos begins as a sequence of broken attractors. This essay follows that road from bifurcation to chaos, and then from chaos to geometry. The main surprise is that fractals are not decorative pictures attached to chaos after the fact. They are invariant objects produced by repeated stretching, folding, deleting, rescaling, and returning. ## The Minimum Vocabulary A **parameter** is a number inside the rule that changes the rule without changing the state variable. In the logistic map: $$x_{n+1}=r x_n(1-x_n),$$ $x_n$ is the state and $r$ is the parameter. Changing $x_n$ means moving through the system. Changing $r$ means changing the system itself. A **bifurcation** is a qualitative change in long-run behavior as a parameter changes. The equation may still make sense, and the old fixed point may still exist algebraically, but its stability can change. That is why existence and stability must be separated. **Chaos** means deterministic unpredictability. The rule has no random term, but nearby starting points separate exponentially. The linear-algebra analogy is an eigenvalue larger than one: a small component in an expanding direction gets amplified. In a nonlinear chaotic system, expansion and folding coexist. A **fractal** is a set with structure across scales. Often its dimension is not an integer. The word should not mean "complicated picture." The important fact is scale structure: zooming reveals new detail rather than a smooth tangent line. **Renormalization** means changing scale and asking what remains the same. Instead of studying one orbit, you study a transformation on whole systems. A renormalization fixed point is an object that keeps the same form after coarse-graining or rescaling. ## Bifurcations Cascade Into Chaos Keep increasing the parameter. The logistic map does not jump straight from order to total disorder. It walks through a very specific cascade: $$1,2,4,8,16,\ldots$$ The period doubles. Then doubles again. Then doubles again. The parameter intervals between doublings shrink by a universal ratio: $$\delta \approx 4.669201609\ldots$$ This is Feigenbaum's constant. Period doubling is not the only route to chaos. It is the one this essay uses because it gives the cleanest fixed-point story in function space. The other classical routes are: 1. **Quasiperiodic route.** A system lives on a torus with two or more incommensurate frequencies, then the torus breaks down. This is associated with Ruelle and Takens. 2. **Intermittency route.** A system alternates between long quiet phases and irregular bursts. Pomeau and Manneville studied this route; it often produces power-law-distributed waiting times. 3. **Crisis route.** A chaotic attractor suddenly changes size or disappears when it collides with an unstable set. So "bifurcations create chaos" is shorthand. More precisely: changing parameters can destroy simple attractors and produce chaotic invariant sets through several mechanisms. Period doubling is the one where the scaling law is easiest to see. By **chaos**, I do not mean "random." A chaotic system is deterministic, but tiny differences in initial conditions grow so fast that long-term prediction becomes impossible in practice. The usual phrase is **sensitive dependence on initial conditions**. If two starting points differ by a tiny amount $\epsilon$, after $n$ steps their distance can grow roughly like: $$\epsilon e^{\lambda n}$$ for some positive number $\lambda$. That $\lambda$ is the Lyapunov exponent. Positive Lyapunov exponent means nearby points separate exponentially. So chaos is not absence of rule. It is rule applied so repeatedly that information about the initial state gets amplified beyond usefulness. The logistic map shows this with almost offensive simplicity. For $r=4$: $$x_{n+1}=4x_n(1-x_n).$$ Two initial conditions that agree for many decimal places can eventually produce unrelated-looking futures. The rule is deterministic. There is no noise term. But the map repeatedly stretches the interval and folds it back. Each iteration reveals another bit of information about the initial condition and then amplifies it into visible difference. There is even an exact relationship between the fully chaotic logistic map and the tent map. If: $$x=\sin^2(\pi\theta),$$ then the logistic map at $r=4$ corresponds to doubling the angle: $$\theta\mapsto 2\theta \pmod 1.$$ That map is the simplest model of chaos. It shifts binary digits. If: $$\theta=0.b_1b_2b_3\ldots$$ then: $$2\theta \pmod 1 = 0.b_2b_3b_4\ldots$$ Each iteration discards the first digit and promotes the next digit. A tiny difference far out in the binary expansion eventually moves to the front. This is sensitive dependence in one line. This is why chaos is connected to information. The initial condition contains infinitely many digits. The dynamics acts like a machine that reads those digits over time. Finite measurement gives you only finitely many digits, so after enough iterations prediction fails. The shocking part is not that this number appears in the logistic map. The shocking part is that it appears in many unrelated systems. Fluids, electronic circuits, population models, chemical reactions, and one-dimensional maps with a single hump can all approach chaos through the same ratio. Why? Because the resemblance runs deeper than ordinary physical space: the systems are approaching the same fixed point in **function space**. Renormalization is the operation. You rescale the map around its critical point, compose it with itself, and ask what function remains invariant under that transformation. Feigenbaum found a fixed point not of a number, but of an operator acting on functions. That is the birth of universality. The deeper statement is that Feigenbaum's discovery is a renormalization fixed point. Take a unimodal map, meaning a one-dimensional map with one hump. Near the onset of chaos, look at every second iterate: $$f^2=f\circ f.$$ Then rescale space so the picture fits back into the same frame. This defines a renormalization operator on maps: $$\mathcal{R}(f)(x)= -\alpha f(f(-x/\alpha)),$$ where $\alpha$ is the spatial scaling factor. The exact convention varies, but the idea does not: compose the map with itself, zoom back in, and ask what shape remains. This is the same idea as looking at a coastline from farther away, then rescaling the picture to compare it with the original. The renormalization operator says: 1. compress many small steps into one larger step, 2. rescale the coordinates, 3. compare the new rule with the old rule. Universality means there is a function $g$ satisfying: $$\mathcal{R}(g)=g.$$ That is not a fixed point of the original dynamical system. It is a fixed point of an operator acting on dynamical systems. Feigenbaum's $\delta$ is the expanding eigenvalue of the linearization of $\mathcal{R}$ around $g$. More concretely, if $r_n$ is the parameter value where period $2^n$ first appears, then: $$\delta=\lim_{n\to\infty}\frac{r_{n-1}-r_{n-2}}{r_n-r_{n-1}}.$$ The constant is universal because the same renormalization fixed point controls the approach to chaos across a whole class of maps. The details wash out. The operator's fixed point remains. This is the first moment where the essay should make the reader stop. We began with $x=f(x)$. Now the object satisfying the fixed-point equation is not a number. It is a function. The transformation acts on a space of rules. The logistic map is the concrete picture behind that abstract sentence. Its bifurcation diagram contains smaller copies of itself. Between chaotic regions there are periodic windows. Inside those windows, period doubling begins again. The same route repeats at smaller scales. So the logistic map connects three levels at once: 1. **state level:** points move by $x_{n+1}=rx_n(1-x_n)$, 2. **orbit level:** fixed points become cycles and cycles double, 3. **rule level:** the whole family of maps approaches a renormalization fixed point. Fractality enters because the third level feeds back into the second. The repeated branching of attractors is governed by a self-similar scaling law. Zoom into the diagram and you do not get smoothness. You get more structure. ### Simulation: Feigenbaum Ratios {{ fixed_point_sim(name="feigenbaum", caption="Successive period-doubling intervals shrink toward Feigenbaum's universal constant delta ≈ 4.669.") }} Numerically locate the first few period-doubling parameters: $$r_1,r_2,r_3,\ldots$$ Then compute: $$\frac{r_{n-1}-r_{n-2}}{r_n-r_{n-1}}.$$ These ratios should approach: $$\delta\approx 4.669.$$ ```python # Approximate known period-doubling parameters for the logistic map. rs = np.array([ 3.0000000000, 3.4494897428, 3.5440903500, 3.5644072661, 3.5687594000, 3.5696916090, ]) ratios = [] for n in range(2, len(rs)): ratios.append((rs[n - 1] - rs[n - 2]) / (rs[n] - rs[n - 1])) print(ratios) ``` What the reader should see: the cascade is not arbitrary. Its intervals shrink at a universal rate. The fixed point has moved from state space to function space. The fourth lesson: **Chaos can appear when bifurcations accumulate around a fixed point in function space.** The bridge from bifurcation to chaos is accumulation. One bifurcation is a structural change. A cascade of bifurcations is a machine for producing arbitrarily fine distinctions in long-run behavior. At period one, the system remembers almost nothing. At period two, it has two phases. At period four, four phases. After many doublings, the number of distinguishable phases becomes too large for finite observation to resolve. This matters because chaos is often described psychologically as "disorder." The Feigenbaum route shows the opposite. Chaos is not a failure of determinism. It is determinism iterated through a hierarchy of increasingly fine distinctions. There is a second bridge here: the object of study moves up one level. At first we studied points: $$x\mapsto f(x).$$ Then we studied attractors made of orbits. Now Feigenbaum asks us to study maps themselves: $$f\mapsto \mathcal{R}(f).$$ That is a categorical move in spirit, even before category theory enters explicitly. We take the rule that used to act on points and make rules themselves into points of a larger space. The fixed point is no longer a number. It is a universal shape of a function under renormalization. When the intervals between structural changes shrink geometrically, there is no last simple regime. Order does not disappear because the rule is abandoned. Order dissolves because the rule recursively creates more and more structure than finite measurement can track. ## Chaos Leaves Fractals Behind Chaotic systems stretch and fold. Stretch. Fold. Stretch. Fold. Like kneading dough. Stretching separates nearby points. Folding keeps the system bounded. Do both forever and a smooth region becomes an infinitely layered object. This is why strange attractors are fractal. The Lorenz attractor is the geometry left behind by repeated stretching and folding, not a decorative butterfly shape. Points that begin close together separate, loop around, get folded back, and separate again. Since this essay mostly uses discrete-time maps, the Hénon map is an even better strange-attractor example: $$x_{n+1}=1-a x_n^2+y_n,$$ $$y_{n+1}=b x_n.$$ For the classical parameters $a=1.4$ and $b=0.3$, the orbit settles onto a thin folded set in the plane. It is not a curve and not an area. It is a strange attractor produced by repeated stretching, folding, and contraction. A **fractal** is an object whose detail does not disappear when you zoom in. A smooth curve becomes simpler at small scales. A fractal keeps producing structure. The word often gets abused, but the basic idea is simple: the object has meaningful structure across many scales. Graphically, the simplest fractal is the Cantor set. Start with the interval $[0,1]$. Remove the middle third. Then remove the middle third of each remaining interval. Then repeat forever. ![Four stages of the Cantor set construction showing repeated middle-third removal](/images/articles/fixed-point-cantor-set.svg) ### Simulation: Cantor Construction {{ fixed_point_sim(name="cantor", caption="Move the stage count. The Cantor set is made by repeating the same removal rule, leaving two scaled copies at every stage.") }} One way to measure this is dimension. A line has dimension 1. A filled square has dimension 2. Some fractals live in between. The Cantor set, for example, has dimension: $$\frac{\log 2}{\log 3}.$$ Why? At each stage it becomes two copies of itself, each scaled by a factor of $1/3$. If dimension $D$ means "mass scales like length to the $D$," then: $$2\left(\frac{1}{3}\right)^D=1,$$ so: $$D=\frac{\log 2}{\log 3}.$$ Here is the deeper connection back to Banach. The Cantor set is a fixed point of a contraction operator acting on sets, not only the product of a recursive drawing trick. Define two contractions: $$S_0(x)=\frac{x}{3},\qquad S_1(x)=\frac{x+2}{3}.$$ Given a compact set $K$, meaning closed and bounded, define the Hutchinson operator: $$\mathcal{H}(K)=S_0(K)\cup S_1(K).$$ Read this as: take the whole set, make two shrunken copies of it, and union those copies. The input is a shape; the output is the next approximation to the shape. The Cantor set $C$ satisfies: $$C=\mathcal{H}(C).$$ That is a fixed-point equation. The difference is that the object is no longer a number. It is a compact set. The space of compact sets can be given a distance called the Hausdorff metric. It measures how far two shapes are from each other by asking how much one shape must be thickened before it covers the other, and vice versa. Under that metric, $\mathcal{H}$ is a contraction. So Banach's fixed-point theorem applies again: > repeated contraction produces a unique fixed point. This is the cleanest possible bridge between the beginning of the essay and fractals. The Cantor set, Sierpiński triangle, and Barnsley fern are fixed points of contractions on spaces of sets. Many of them are literally Banach fixed points in a richer space, not just aesthetic relatives of one. ### Why roughness is generic A natural objection: if fractals are just Banach fixed points, why are these fixed points rough rather than smooth? Nothing about contraction sounds jagged, and the fixed points of the first essay were tame. The answer is a comparison of requirements. A smooth invariant set asks a lot of a map. The set must map into itself, *and* the map must respect the set's differentiable structure, carrying tangent directions to tangent directions at every point. That is a strong compatibility condition between the dynamics and the geometry. A fractal invariant set asks for much less: contraction, copying, folding, splitting, and invariance. No tangent structure has to survive. The dimension equation makes the comparison concrete, because the Hutchinson fixed point is not automatically fractal, and the exceptions are what explain the rule. The two maps $x/2$ and $x/2+1/2$ have the whole interval $[0,1]$ as their fixed point. Four corner maps at contraction ratio $r=1/2$ fill a solid square. Smooth answers happen exactly when the scaled copies tile perfectly, with no gaps and no overlaps. In the equal-ratio dimension equation $N r^D=1$, that perfect tiling is the arithmetic coincidence that makes $D=\log N/\log(1/r)$ come out an integer: two copies at ratio $1/2$, four copies at ratio $1/2$. The formula gives the Hausdorff dimension when the copies do not overlap too much, precisely when they satisfy the open set condition; if the pieces overlap badly, the simple count can fail. Almost any non-overlapping choice of $N$ and $r$ gives a non-integer. The number $D$ is the similarity dimension: it tells you how the amount of detail changes when you zoom. When $D$ is not an integer, the set is too large to be a cloud of points or a smooth curve, but too thin to fill an ordinary area. So rough self-similar sets are not exotic decorations bolted onto dynamics after the fact. They are often the ordinary fixed points of dissipative nonlinear rules, and the smooth cases are the ones that need an explanation. The Mandelbrot set is another fixed-point object in disguise. For each complex number $c$, iterate: $$z_{n+1}=z_n^2+c,\qquad z_0=0.$$ The question is whether the orbit stays bounded. The boundary is infinitely detailed because the same iterative question is being asked at every scale. The link to fixed points is exact, not decorative. The large heart-shaped region, the main cardioid, is precisely the set of $c$ for which $z^2+c$ has an attracting fixed point: a point $z^\star$ with $z^\star=z^{\star 2}+c$ and multiplier $|2z^\star|<1$. That is the same stability criterion $|f'(x^\star)|<1$ from the logistic map, now in the complex plane. The round buds attached to the cardioid are the parameters where an attracting cycle has replaced the attracting fixed point. The Mandelbrot set is a map of where $z^2+c$ has a stable fixed point, then a stable two-cycle, then four, which is period doubling once again. The logistic bifurcation diagram is fractal for a closely related reason. For each value of $r$, you iterate the same map many times and plot the long-run values of $x_n$. The horizontal axis is the parameter $r$. The vertical axis is the attractor. At first the diagram has one branch: one attracting fixed point. Then two branches: a period-two attractor. Then four. Then eight. Then a dense-looking chaotic region. But inside the chaotic region, order returns in windows. A famous one is the period-three window. Inside that window, the same story happens again: period three becomes period six, then twelve, then twenty-four, then chaos. Zoom further and similar structures reappear. There is a famous theorem hiding behind that sentence. Li and Yorke proved that "period three implies chaos" for interval maps. Sharkovskii's theorem is even stronger: the existence of a period-three orbit forces the existence of orbits of every other period. So the period-three window is not just a curiosity in a picture. It is a sign that the map has crossed into a regime where periodic structure of all orders is available. This is not exactly self-similarity like the ideal Cantor set, where each copy is perfectly scaled. It is asymptotic and organized by universal constants. But the philosophical point is the same: iteration produces structure at many scales, and renormalization explains why those scales are related. The logistic map therefore gives a direct path from a one-line recurrence to a fractal-looking object: $$x_{n+1}=rx_n(1-x_n)$$ becomes: 1. fixed point, 2. cycle, 3. bifurcation cascade, 4. chaos, 5. self-similar bifurcation diagram, 6. fractal invariant sets. This is the reason it deserves so much attention. It is the whole ladder before the ladder becomes abstract. The stretch-fold mechanism can be made brutally concrete with the tent map: $$T(x)=1-2|x-\tfrac{1}{2}|.$$ It stretches the unit interval by a factor of two, then folds it back onto itself. The baker's map does the same thing in two dimensions: stretch a square, cut it, stack it. Repeated forever, the map creates sensitivity to initial conditions because the stretching separates nearby points exponentially. ### Simulation: Stretch And Fold {{ fixed_point_sim(name="tent", caption="The tent map stretches nearby points apart and folds them back into the interval. That is the basic geometry of bounded chaos.") }} Iterate the tent map from many nearby starting points. ```python import numpy as np import matplotlib.pyplot as plt def tent(x): return 1 - 2 * np.abs(x - 0.5) xs = np.linspace(0.1, 0.1001, 50) paths = [xs] for _ in range(30): paths.append(tent(paths[-1])) paths = np.array(paths) plt.plot(paths, alpha=0.35) plt.xlabel("iteration") plt.ylabel("x") ``` What the reader should see: points that start almost indistinguishably close separate quickly. Stretching creates sensitivity; folding keeps the values inside the interval; repetition creates fine structure. That exponential separation is measured by a Lyapunov exponent: $$\lambda=\lim_{n\to\infty}\frac{1}{n}\log |(f^n)'(x)|.$$ If $\lambda>0$, nearby initial conditions separate on average. Prediction decays exponentially. But folding keeps the motion bounded, so the system does not simply fly apart. It stays in a finite region while continually producing new distinguishable detail. For the tent map, this is exact. Away from the folding point, the slope has absolute value $2$, so after $n$ iterates the derivative magnitude is roughly $2^n$. Therefore: $$\lambda=\lim_{n\to\infty}\frac{1}{n}\log(2^n)=\log 2.$$ The fully chaotic logistic map at $r=4$ has the same Lyapunov exponent, $\log 2$, after the change of variables $x=\sin^2(\pi\theta)$. This is a good example of an abstract definition becoming a number you can actually compute. That is the geometric recipe for a strange attractor: 1. stretch to create sensitivity, 2. fold to keep boundedness, 3. repeat to generate fine structure. The fractal is the invariant set of that operation. The fifth lesson: **Fractals are the geometry left behind by repeated stretching and folding.** The bridge from chaos to fractals is bounded instability. If nearby points separate and nothing folds them back, the system just disperses. If folding happens without stretching, the system just settles. A strange attractor needs both: expansion to create detail, folding to keep the detail in a bounded region. This is why chaos leaves geometry behind. Sensitive dependence is a statement about trajectories. A fractal attractor is the spatial trace of that sensitivity after infinitely many rounds of stretching and folding. The dynamics says: $$\text{nearby states separate}.$$ The geometry says: $$\text{the invariant set has structure at every scale}.$$ Those are two views of the same process. The first view follows points through time. The second view looks at the set that survives after the process has acted forever. That is why chaotic attractors often have non-integer dimension. They are not curves, because stretching has multiplied their internal detail. They are not areas, because folding and dissipation keep them thinner than a filled region. A fractal is what remains when instability is trapped. ## Why Fractals Are the Default A question has been hiding under this whole essay: why does nature keep producing these objects? River deltas like the one in this essay's header, lungs, coastlines, mountain ranges, price charts. The standard answer is that growth repeats the same rule at different scales. That is true and explains nothing, because it restates the observation. The better question is why repeated rules leave behind self-similar sets rather than smooth ones. Start with what self-similarity actually is. If $K=\mathcal{H}(K)$, then $K$ contains scaled copies of itself, and those copies contain scaled copies, without end. Here $\mathcal{H}$ is the set-valued rule from above: take a shape, shrink it, copy it, and put the copies together. Zooming reveals the same structure because the equation says it must. The infinite regress is not a decoration on the answer. It is the content of the fixed-point equation, read geometrically. The dynamics is what supplies the contractions, and here the essay's recipe returns. Dissipation alone collapses everything to a point; that is contraction with nothing opposing it. Bounded chaos needs stretch plus fold, and stretch-and-fold applied forever is the Hutchinson picture drawn by the dynamics itself: the strange attractor is the set the kneading leaves invariant. In complex dynamics this becomes sharp. For rational maps of degree at least two, maps of the complex plane built from ratios of polynomials, with $z\mapsto z^2+c$ as the standard family, the boundary between starting points with different long-run fates is the Julia set, and apart from classified exceptional families it is fractal: circles, Chebyshev intervals, and Lattès-type examples are the cases where the roughness is absent or unusually regular. The unit circle, left invariant by $z\mapsto z^2$, is the clean toy exception. In real systems the statement is a strong tendency rather than a law: a double-well gradient flow, a ball rolling with friction on a landscape with two valleys, has two attractors, one valley bottom each, separated by a perfectly smooth basin boundary, the ridge line. The boundary turns fractal when a chaotic saddle lives inside it, an unstable invariant set on which the dynamics is itself chaotic, a tangle that scrambles which side a nearby start will fall to. Forced nonlinear systems arrange that readily. So the lesson is not that every boundary between basins is fractal. It is narrower and stronger: once expansion, folding, and multiple long-run fates coexist, smooth separating surfaces are fragile, and fractal boundaries are the natural outcome. That is why the same geometry recurs across nature. A branching rule, split into $N$ copies at ratio $r$, is an iterated function system by construction, and lungs and trees run it because its fixed point packs enormous exchange surface into bounded volume. Eroding terrain and growing snowflakes are shaped by their process rates the way $N$ and $r$ set $D$, which is why measured terrain dimensions land between 2 and 3 rather than on either. Price records are self-affine in time, meaning the vertical and horizontal axes rescale differently but the roughness still has no characteristic scale; Mandelbrot saw it in cotton prices in 1963, and the next essay follows that thread into power laws. The cosmic web looks fractal over a finite range of scales and smooths out at the largest. Every one of these is approximate and finite-range, like every empirical scaling in this series. None of them needs a designer of detail. Each needs only a rule with expansion and contraction, applied again. One boundary should be drawn while the picture is fresh. This is still the iteration engine, not self-reference in the logical sense. A fractal repeats because a geometric rule is iterated on sets. A Gödel sentence, several essays ahead, refers to itself because a formal system can encode its own syntax. Those are different mechanisms. What they share is the fixed-point shape: an object is fed through a transformation and returns as itself, possibly one level up. Later essays use the same abstract shape for syntax, programs, types, life, and markets, and the two engines should not be collapsed into each other. So the conclusion runs backwards from the intuition. Fractals are not the exotic case; smooth geometry is. Iterate a nonlinear rule with both expanding and contracting directions, the ordinary situation for an interesting dynamical system, and structure at every scale is the default outcome. The real question was never why fractals are everywhere. It is why we expected smoothness, the fine-tuned case, to be the norm. The invariant summary of this essay is: $$ \text{stretch}+\text{fold}+\text{repeat}\Rightarrow\text{fractal invariant set}. $$ At the Feigenbaum level the fixed point is no longer a point in state space. It is a function left unchanged by a renormalization operator: $$ \mathcal{R}(g)=g. $$ At the fractal level the fixed point may be a set left unchanged by an iterated-function-system operator: $$ \mathcal{H}(K)=K. $$ The point has moved up one level. The next essay moves it up again: from invariant sets to invariant scaling laws and probability tails. {% toolkit(label="Takeaway") %} - **Space:** a space of states, and one level up, a space of maps. - **Transformation:** stretch-and-fold dynamics on states; renormalization on maps. - **Invariant object:** a fractal strange attractor; and a renormalization fixed-point function, $\mathcal{R}(g)=g$. - **Failure mode:** sensitive dependence on initial conditions and the prediction horizon it imposes. {% end %} ## Further Reading 1. Mitchell Feigenbaum, [*Quantitative universality for a class of nonlinear transformations*](https://doi.org/10.1007/BF01020332) (1978). The period-doubling universality paper. 2. Tien-Yien Li and James Yorke, [*Period three implies chaos*](https://doi.org/10.1080/00029890.1975.11994008) (1975). The paper that gave chaos its modern mathematical name in one-dimensional dynamics. 3. Robert Devaney, [*An Introduction to Chaotic Dynamical Systems*](https://www.routledge.com/An-Introduction-to-Chaotic-Dynamical-Systems/Devaney/p/book/9780367235994). A clear path through symbolic dynamics, chaos, and fractals. 4. Michael Barnsley, [*Fractals Everywhere*](https://en.wikipedia.org/wiki/Michael_Barnsley). The standard reference for iterated function systems and the Hutchinson operator. 5. Edward Lorenz, [*Deterministic nonperiodic flow*](https://doi.org/10.1175/1520-0469(1963)020%3C0130:DNF%3E2.0.CO;2) (1963). The continuous-time strange-attractor classic. 6. David Ruelle and Floris Takens, [*On the nature of turbulence*](https://doi.org/10.1007/BF01646553) (1971). The paper that named the strange attractor. 7. Jean-Pierre Eckmann and David Ruelle, [*Ergodic theory of chaos and strange attractors*](https://doi.org/10.1103/RevModPhys.57.617) (1985). The canonical review tying chaos, Lyapunov exponents, and invariant measures together. --- ### Fixed Points and Attractors *Published: 2025-06-08* > The foundation of the ladder: repeated rules, dynamical systems, fixed points, attractors, basins, bifurcations, Newton iteration, and the logistic map. URL: https://federicocarrone.com/series/fixed-point-ladder/fixed-points-and-attractors/ Start with something small: a number, a rule, and the patience to apply the rule again. **What happens when you repeatedly apply a rule?** That question is small enough to fit into a first course in analysis, but it opens into a surprising amount of modern mathematics. A repeated rule can settle into a fixed point. A fixed point can become an attractor. An attractor can split. Repeated splitting can become chaos. Chaos can leave behind fractal geometry. Fractal geometry can show up statistically as power laws. Power laws can make averages unstable. Once time and recurrence matter, number theory enters. Once a system can encode and act on itself, logic and computation enter. Once you want the same pattern without caring whether the objects are numbers, programs, proofs, or types, category theory enters. I will keep the level deliberately concrete. Whenever a bigger word appears, the word should earn its keep. ## The Minimum Vocabulary A **dynamical system** is a system whose state changes according to a rule. The state is the information you need in order to know where the system is. For a one-dimensional example, the state might be a single number $x$. For a planet, the state might include position and velocity. For a market, it might include prices, inventories, leverage, and beliefs. For a program, it might include memory, registers, and input. The set of all possible states is called the **state space**. If time moves in discrete steps, the rule usually looks like: $$x_{n+1}=f(x_n).$$ Read this as: > the next state equals a function of the current state. The sequence: $$x_0,x_1,x_2,x_3,\ldots$$ is called the **orbit** or **trajectory** of the starting point $x_0$. So the whole question becomes: > If I start here and keep applying the same function, where does the orbit go? That is all a discrete dynamical system is. If time is continuous instead, the rule is usually written as a differential equation: $$\frac{dx}{dt}=F(x).$$ Read this as: > the velocity of the state is determined by the current state. This article mostly uses the discrete version because it is easier to see the fixed-point structure. But the same ideas apply to differential equations. ![A vertical ladder connecting repeated transformations, fixed points, attractors, bifurcations, chaos, fractals, power laws, non-ergodicity, number theory, logic, and category theory](/images/articles/fixed-point-ladder.svg) ## Repetition Creates Fixed Points Start with the simplest possible dynamical system: $$x_{n+1} = f(x_n).$$ You choose an initial condition $x_0$, apply the rule once, get $x_1$, apply the same rule again, get $x_2$, and keep going. Eventually one of three things can happen: 1. the sequence converges, 2. the sequence cycles, 3. the sequence becomes chaotic. The cleanest case is convergence. Take: $$f(x)=\frac{x+1}{2}.$$ The fixed point is the value $x^\star$ satisfying: $$x^\star = f(x^\star).$$ So: $$x^\star = \frac{x^\star+1}{2}$$ and therefore: $$x^\star = 1.$$ The star in $x^\star$ is just notation. It means "the special value we are solving for." Nothing mystical. What matters is not that $1$ solves the equation, but that every starting point gets pulled there. If $e_n=x_n-1$ is the distance from the fixed point, then: $$e_{n+1}=\frac{x_n+1}{2}-1=\frac{x_n-1}{2}=\frac{e_n}{2}.$$ So the error halves every step: $$e_n = 2^{-n}e_0.$$ That is a **contraction**. A contraction is a function that always brings points closer together. If two inputs are one unit apart, their images might be only half a unit apart. Apply the function again and they are a quarter unit apart. Keep going and the distance collapses. Banach formalized the general theorem: if a map brings points closer together in a complete metric space, repeated application converges to a unique fixed point. The theorem is worth stating because it is the cleanest version of the whole ladder. Let $(X,d)$ be a complete metric space, and let $f:X\to X$ satisfy: $$d(f(x),f(y))\leq q\,d(x,y)$$ for every $x,y\in X$ and some constant $01$, the error grows. That is local stability. Since: $$f_r'(x)=r(1-2x),$$ we get: $$f_r'(x^\star)=r\left(1-2\left(1-\frac{1}{r}\right)\right)=2-r.$$ So stability requires: $$|2-r|<1,$$ or: $$1 Logarithms, Kelly, ergodicity, and tail risk meet at the same geometry: wealth compounds multiplicatively, the logarithm is concave, and Jensen's inequality measures the cost of variability. URL: https://federicocarrone.com/series/leptokurtic/at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/ Never cross a river that is on average four feet deep. If the river is eight feet deep in the middle and dry on the sides, the average tells you nothing about whether you will drown. You will drown in the middle, or you won't. There is no averaging across parallel universes where you both survive and die. The same asymmetry shows up wherever outcomes compound. Lose 50% of your wealth and you need a 100% gain just to break even, because the loss hits a larger base than the recovery builds from. Going from \$100 to \$200 and from \$200 to \$400 are different dollar amounts but the same proportional move: one doubling. Wealth is about ratios and scaling, not absolute differences. This multiplicative structure has consequences that run deeper than intuition suggests. The mathematics that governs survival in compounding environments was invented 400 years ago to help astronomers multiply large numbers, partially rediscovered in the 18th century to solve a paradox about gambling, formalized again through information theory, and then largely obscured by theories that optimized across hypothetical worlds instead of along a single path through time. That is the geometric claim in this essay: wealth evolves multiplicatively, while most ordinary intuition is additive. The logarithm is the change of coordinates that lets us move between those two descriptions. This is the story of why geometry sits at the core of finance, why a single inequality ties the whole picture together, and why reducing variance can be more valuable than increasing returns. None of the parts are mine. Kelly, Peters, Spitznagel, and Taleb each worked one of them out. My only claim is that they are the same picture, and this essay traces how the pieces fit. {% toolkit(label="Key takeaways") %} - Wealth compounds multiplicatively, but ordinary intuition is additive. The logarithm is the change of coordinates between the two, and the gap that opens up between them is Jensen's inequality. - Losses compound against a larger base than gains build from (lose 50 percent and you need 100 percent to recover), so the average outcome across hypothetical worlds can badly mislead about survival along your one path through time. - The practical payoff: reducing variance can be worth more than increasing returns, because volatility drag eats geometric growth. - It is one picture, not four. Kelly (bet sizing), Peters (ergodicity), Spitznagel (volatility drag and convexity), and Taleb (fat tails) are each working out a different piece of the same geometric fact. {% end %} ## I. The Invention of Linearization (1614) In the early 17th century, astronomers were drowning in calculation. Johannes Kepler had just published his laws of planetary motion, and navigators were trying to compute positions using spherical trigonometry. A single problem might require multiplying two seven-digit numbers, a process that took skilled calculators half an hour and was prone to error. John Napier, a Scottish laird and amateur mathematician, spent 20 years searching for a way to simplify this. His insight: if you could convert multiplication into addition, calculations would become trivial. He invented "logarithms" (from Greek *logos* = ratio, *arithmos* = number), a table that mapped every number to its "ratio-representative." The crucial property: $\log(ab) = \log(a) + \log(b)$. Multiplication becomes addition. Division becomes subtraction. Exponentiation becomes multiplication. But Napier's invention was more than a computational trick. He had discovered the mathematical tool for linearizing multiplicative processes. Four hundred years later, this same tool would reveal why volatility destroys wealth and why tail hedging works. ## II. The Number That Grows Continuously (1680s) In the 1680s, Jacob Bernoulli investigated a financial puzzle: if you lend money at 100% annual interest, what happens if you compound it more frequently? - Annually: $(1 + 1)^1 = 2$ - Monthly: $(1 + 1/12)^{12} \approx 2.61$ - Daily: $(1 + 1/365)^{365} \approx 2.71$ Bernoulli showed that as the compounding interval shrinks toward zero, the result approaches a limit: $$e = \lim_{n \to \infty} \left(1 + \frac{1}{n}\right)^n \approx 2.71828$$ The number later denoted by $e$ emerges naturally from continuous compound growth. It is the base of the logarithm that measures growth rates, the "natural" logarithm $\ln(x)$. Where Napier gave us the tool to linearize multiplication, Bernoulli identified the constant that emerges when growth becomes continuous. Together, Napier and Bernoulli provided the mathematical foundation: wealth grows multiplicatively, and logarithms convert this multiplicative growth into additive increments. ## III. Jensen's Breakthrough (1906) The missing geometric step arrived in 1906. Danish mathematician Johan Jensen proved the inequality that now bears his name. He showed that whenever a function is concave, averaging inputs before applying the function gives a larger result than applying the function first and then averaging. That result sounds abstract, but it is exactly the bridge this story needs. Napier gave us logarithms. Bernoulli showed why the natural logarithm belongs to compounding. Jensen explained why variability hurts once the relevant function bends downward. ## IV. Jensen's Inequality: The Geometry of Concavity A function is **concave** if it bends downward. If you draw a straight line between two points on the curve, that line sits below the curve itself. The logarithm is concave. That simple geometric fact turns out to matter enormously in finance, because wealth compounds. There is a reason the logarithm is the right function here, not just a convenient one: it is the map that turns multiplicative wealth dynamics into additive increments you can sum through time. In raw wealth space, returns multiply. In log space, they add. Jensen's inequality matters because this change of coordinates is not linear: the map bends downward, and that bend is exactly why dispersion in returns becomes a drag on long-run growth. **Jensen's Inequality** states that for a concave function $\varphi$: $$\mathbb{E}[\varphi(X)] \leq \varphi(\mathbb{E}[X])$$ The notation is simple once you unpack it. $\mathbb{E}[X]$ means the expected value, or average, of $X$. The symbol $\varphi(X)$ means "evaluate the function $\varphi$ at $X$." So the inequality says: for a concave function, the expected value of the transformed variable is less than or equal to the transformed expected value. The gap between those two quantities is the **Jensen gap**. It is the mathematical penalty created by variability under concavity. Here is the simplest possible example. Suppose your wealth factor is either $1.5$ or $0.5$ with equal probability. In other words, you either gain 50% or lose 50%. - The arithmetic average wealth factor is $(1.5 + 0.5)/2 = 1.0$ - The log of that average is $\ln(1.0) = 0$ - But the average log is $\frac{\ln(1.5) + \ln(0.5)}{2} \approx -0.144$ So the average outcome looks flat, but the expected log-growth is negative. Under repeated exposure to the same kind of multiplicative gamble, that means long-run compound growth is negative even though the arithmetic average looks harmless. That is Jensen's inequality in action. ![A concave logarithm curve with the points for a 50% loss and a 50% gain marked, the straight chord between them, and a vertical arrow showing the gap between the log of the average wealth factor and the average of the logs](/images/articles/jensen-concavity.svg) *The whole effect in one picture. Mark the two outcomes on the curve, ×0.5 and ×1.5, and draw the straight line between them. Averaging first and then taking the log lands on the curve at $\ln(1.0)=0$ (green). Taking the log first and then averaging lands on the chord, below it, at about $-0.14$ (red). Because the logarithm bends down, the second is always lower. That vertical gap is the cost of variance.* To see this more viscerally, start with \$100: - After a 50% gain: $\$100 \times 1.5 = \$150$ - After a 50% loss: $\$150 \times 0.5 = \$75$ You end with \$75, a 25% total loss, despite the arithmetic average return being zero. The loss applied to a larger base (\$150) than the gain (\$100), so it took away more than the gain added. Compounding makes volatility expensive. This asymmetry generalizes. A drawdown and the gain required to recover from it are not symmetric: - Lose 10%, need +11.1% to break even - Lose 20%, need +25% to break even - Lose 50%, need +100% to break even - Lose 80%, need +400% to break even That convex recovery schedule is another way of seeing the same geometry. Losses damage the base that future growth must compound from. ![A convex curve of the percentage gain required to break even against the size of the loss, rising far above the symmetric gain-equals-loss reference line and curving sharply upward past a 50% loss](/images/articles/jensen-recovery.svg) *The gain needed to break even, plotted against the loss. If recovery were symmetric it would track the dashed line. Instead it curves away: a 10% loss needs 11%, a 50% loss needs 100%, an 80% loss needs 400%. The deeper the hole, the more disproportionate the climb out, which is the same concavity seen from the other side.* Apply this to log-wealth: $$\mathbb{E}[\ln(1+R)] \leq \ln(1 + \mathbb{E}[R])$$ Here $R$ means the return in one period. If you gain 10%, then $R = 0.10$. If you lose 20%, then $R = -0.20$. The expression $1+R$ is your **wealth factor**, the number your money gets multiplied by over that period. This expression only makes sense when $1+R > 0$. Returns below $-100\%$ are outside the model because they correspond to ruin. The left side is the average log-growth rate, which is what determines long-run compounding. The right side is the log of one plus the average return. The inequality is strict whenever returns vary. If returns were perfectly constant, the two sides would be the same. Variability is what creates the gap. That distinction matters. The Jensen gap is the general geometric fact. **Variance drag** is the second-order approximation to that fact when returns are not too large. To estimate the size of the gap, expand $\ln(1+R)$ in a Taylor series around the mean return $\mu$: $$\ln(1+R) \approx \ln(1+\mu) + \frac{R-\mu}{1+\mu} - \frac{(R-\mu)^2}{2(1+\mu)^2} + ...$$ Now take expectations term by term. Because $\mu = \mathbb{E}[R]$, we have $\mathbb{E}[R-\mu] = 0$, so the linear term vanishes. And because $\sigma^2 = \mathbb{E}[(R-\mu)^2]$, the quadratic term becomes the variance. That gives: $$\mathbb{E}[\ln(1+R)] \approx \ln(1+\mu) - \frac{\sigma^2}{2(1+\mu)^2}$$ For small returns, and therefore for $|\mu| \ll 1$, we can use $\ln(1+\mu) \approx \mu$ and $(1+\mu)^2 \approx 1$. Then this simplifies to: $$G \approx \mu - \frac{\sigma^2}{2}$$ Here $\mu$ is the arithmetic mean return, $\sigma^2$ is the variance of returns, and $G$ is the approximate geometric growth rate. So the geometric growth rate is approximately equal to the arithmetic mean $\mu$ minus half the variance. **Variance drag is literally the second-order term in the Taylor expansion of a concave function.** This is Jensen's inequality showing up in a form that is easy to compute. Because the logarithm is concave, variance lowers expected log-return. A portfolio with $\mu = 10\%$ and $\sigma = 20\%$ has geometric growth of approximately $8\%$, which means a 2% annual drag from volatility alone. In continuous-time finance, the same effect appears as the Itô correction term in geometric Brownian motion. That is why expected log-growth sits below arithmetic drift by roughly $\sigma^2/2$ there as well. If you write wealth as $$\frac{dW_t}{W_t} = \mu\,dt + \sigma\,dB_t$$ then Itô's lemma gives $$d\ln W_t = \left(\mu - \frac{\sigma^2}{2}\right)dt + \sigma\,dB_t$$ The drift term for **log-wealth** is not $\mu$ but $\mu - \sigma^2/2$. In other words, the same "variance drag" that appeared above as a local Jensen effect shows up exactly in the canonical continuous-time model of finance. If you want to see the difference numerically, a simple path simulation makes the gap visible: ![Illustrative wealth-path simulation showing arithmetic expectation above geometric and median paths](/images/articles/jensen-wealth-paths.svg) The red line is the arithmetic expectation. The blue line is the geometric expectation. The gold line tracks a more typical realized path. The faint gray lines are sample paths. They all start from the same wealth, but the arithmetic average drifts above the path most investors actually live through. The exact numbers vary by run, but the pattern is stable: arithmetic averages look better than geometric ones, and median terminal wealth typically sits below the arithmetic mean because compounding punishes volatility path by path. ## V. Kelly's Criterion: Optimization Under Concavity Kelly's criterion is the optimization consequence of the geometry just described. Jensen's inequality says the concavity of the logarithm penalizes variance. Kelly asks: given that penalty, what exposure level maximizes expected log-growth? The answer puts variance directly in the denominator. Long before Kelly, Daniel Bernoulli had already moved in the right direction. In 1738, in his analysis of the St. Petersburg paradox, he proposed logarithmic utility. He did not frame it in terms of time averages or ergodicity, but he correctly saw that the value of money is not linear and that multiplicative risk changes the problem. Claude Shannon's work is foundational here. In 1948, Shannon created information theory, a mathematical framework for reasoning about signal, noise, and transmission. The deep idea is that information can be measured, that uncertainty has structure, and that better information changes what an optimal repeated decision looks like. That is the intellectual foundation Kelly stands on. In Shannon's framework, information has operational value because it improves your ability to act under uncertainty. Kelly's central insight was to take that logic and apply it to betting and capital allocation. If information improves the quality of your edge, then there is an optimal way to convert that edge into compounded wealth growth. In 1956, John Larry Kelly Jr., a researcher at Bell Labs, derived the optimal betting strategy for a gambler with a private wire giving him noisy information about horse races. His derivation came directly from Shannon's information theory, the mathematics of signal transmission through noisy channels. Kelly maximized the expected **logarithm** of wealth because wealth compounds multiplicatively, and the logarithm converts this into additive growth rates that can be averaged over time. The Kelly criterion maximizes: $$G = \mathbb{E}[\ln(W_t/W_{t-1})]$$ Here $W_t$ means your wealth at time $t$. So $W_t/W_{t-1}$ is simply "how much your wealth changed this period," and the logarithm turns that multiplicative change into something you can add across time. For a simple bet with probability $p$ of winning, odds $b$, and probability $q=1-p$ of losing, the optimal fraction $f^*$ of wealth to bet is: $$f^* = \frac{pb - q}{b} = \frac{p(b+1) - 1}{b}$$ For continuous returns, where $\mu$ is the expected return, $r$ is the risk-free rate, and $\sigma^2$ is the variance, the Kelly fraction becomes: $$f^* = \frac{\mu - r}{\sigma^2}$$ **Notice what appears in the denominator: variance.** Kelly's formula explicitly shows that optimal position size depends on the ratio of edge to variance. A strategy with twice the edge but twice the variance gets the same allocation as the original. Variance does more than measure "risk" here; it sets the size of the optimal position directly. Under the usual favorable-bet assumptions, Kelly betting grows wealth faster than any other fixed-fraction strategy in the long run, dominating the alternatives with probability approaching 1 as time goes to infinity. Edward O. Thorp was the person who carried this from theory into practice. He used Kelly-style reasoning first in blackjack and then in markets, showing that log-optimal sizing was a workable decision rule under uncertainty as well as an elegant theorem. Leo Breiman gave the result one of its clearest mathematical statements. In 1961, he showed that the log-optimal strategy asymptotically dominates alternative strategies under broad conditions. Kelly gave the rule. Thorp made it operational. Breiman helped make the long-run claim precise. Thomas Cover pushed the idea further. His work on **universal portfolios** showed that you can approach the long-run performance of the best constant-rebalanced portfolio in hindsight without knowing the true return distribution in advance. This extends the Kelly logic from "what is the optimal fraction if I know the edge?" to "how do I adapt toward log-optimal growth when the world is uncertain and my estimates are imperfect?" It is one of the cleanest bridges between growth optimality and learning. Robert Fernholz extended the geometric view in a different direction through **stochastic portfolio theory**. His contribution was to show that relative portfolio performance can emerge from market structure itself, diversity, rebalancing, and the geometry of capital flows, not just from forecasting expected returns. It reinforces the same point: in a multiplicative world, portfolio results are shaped by pathwise structure and compounding mechanics, not only by static one-period expectations. ### Fractional Kelly Practitioners rarely use full Kelly. The optimal fraction maximizes growth rate but produces extreme volatility, and drawdowns of 50% or more are common. Instead, they use half-Kelly or quarter-Kelly: $$f_{half} = \frac{f^*}{2}$$ Under the standard local approximation around the Kelly optimum, this reduces growth rate by about 25% but cuts volatility in half. It also provides a safety margin against estimation error. If your estimate of $\mu$ or $\sigma$ is wrong, full Kelly can be catastrophic. Fractional Kelly is the recognition that maximizing geometric growth is the goal, but estimation uncertainty requires humility. ## VI. Ergodicity Economics: Time vs. Ensemble In 2019, physicist Ole Peters published a paper in *Nature Physics* that should have forced a much sharper debate about the foundations of decision theory. Peters is one of the central modern figures in this article's argument because he redefines the objective instead of adding another risk model. His broader research program, including earlier work with Alexander Adamou, showed that expected utility theory, the foundation of modern economics, rests on the implicit assumption of **ergodicity**. To understand the force of that critique, it helps to name the benchmark. In 1944, John von Neumann and Oskar Morgenstern formalized **expected utility theory** in *Theory of Games and Economic Behavior*. Their framework asks which action maximizes average utility across possible states of the world. It became the dominant mathematical language of rational choice. The Peters critique is aimed at this formal benchmark, not at a vague intuition. A process is ergodic if the time average equals the ensemble average. In ergodic systems, averaging across many parallel realizations gives the same long-run result as following one realization through time. **Wealth growth is not ergodic.** Consider a gamble that multiplies your wealth by $1.5$ on heads and by $0.6$ on tails, each with probability one half. Averaged across many people betting in parallel, wealth grows: the ensemble average factor is $0.5 \times 1.5 + 0.5 \times 0.6 = 1.05$, a 5% gain per round. But a single person betting repeatedly almost surely goes broke, because the time-average growth factor is $\sqrt{1.5 \times 0.6} = \sqrt{0.9} \approx 0.95$, a loss of about 5% per round. The ensemble looks like a winning bet. The lived path is ruin. This is where the distinction between **ensemble optimality** and **pathwise optimality** becomes decisive. A strategy can look optimal when you average across many parallel worlds, yet still be disastrous for one person living through one realized sequence of outcomes. In the Peters-Adamou framing, standard expected-utility theory evaluates choices across possible states of the world, while individual investors experience one path through time. They cannot access parallel universes where they both survived and went bankrupt. Adamou's contribution matters here because the joint Peters-Adamou work did more than restate the ergodicity objection in the abstract: it used specific paradoxes, especially the St. Petersburg paradox, to show how changing the time resolution of the problem changes what counts as a rational decision. That made the time-average interpretation concrete rather than merely philosophical. The correction is simple: maximize the time-average growth rate. This is exactly Kelly's criterion. The logarithmic utility that Daniel Bernoulli invented to solve the St. Petersburg paradox in 1738 was correct, but economists forgot *why*: it emerges naturally from the non-ergodicity of multiplicative growth. Jensen's inequality is what makes the two averages different. Because the logarithm is concave, expected log-growth (the time average) is always less than the log of expected growth (the ensemble average). The gap between them is the Jensen gap from Section IV. Non-ergodicity is what turns that gap into a matter of survival rather than a technicality: you compound along one path, not across all possible paths. This also clarifies the limit of mean-variance thinking. Markowitz's framework is useful as a first approximation, but it treats risk as a tradeoff between average return and dispersion in a single period. It does not, by itself, encode the asymmetry of compounding through time or the special importance of ruin. Paul Samuelson spent years arguing against overextending Kelly logic. His objections were not trivial; they forced the distinction between maximizing expected utility and maximizing long-run growth into the open. Even if one ultimately sides with Kelly and Peters for multiplicative wealth, Samuelson is part of the reason the debate became intellectually sharp. ## VII. Absorbing Barriers and the River Taleb's river analogy makes the mathematics visceral: > Never cross a river that is on average four feet deep. If the river is eight feet deep in the middle and dry on the sides: - **Arithmetic mean:** $(0 + 8)/2 = 4$ feet, which seems safe - **Actual constraint:** if you are shorter than eight feet, the deep section still kills you The river's average depth is irrelevant. What matters is whether any point along the path is deep enough to kill you. The same logic applies to multiplicative wealth: a single ruinous outcome matters more than an average taken across hypothetical parallel paths. Mathematically, zero is an **absorbing barrier**. If wealth hits zero, the process stops. You cannot recover. The arithmetic mean ignores this because it averages across paths where you survived and paths where you died. The geometric mean, via the logarithm, assigns infinite negative utility to zero: $\ln(0) = -\infty$. This is why log-optimal sizing heavily penalizes strategies that expose you to ruin. The logarithm's concavity makes ruin infinitely worse than any potential gain can compensate for, which is why practical Kelly sizing is always constrained by the possibility of catastrophic loss. ## VIII. Fat Tails and Higher Moments The Taylor expansion of $\ln(1+R)$ does not assume normality; it is a local expansion that is useful when returns are small enough and the relevant moments exist. But financial returns live in **Extremistan** (Taleb's term for domains governed by fat-tailed distributions), where extreme events are far more likely than the normal distribution predicts. Mathematician Benoit Mandelbrot is the foundational figure here. Long before Taleb, Mandelbrot argued that speculative prices do not behave like neat Gaussian variables. They jump, cluster, and produce extreme moves far more often than classical models would suggest. Once that is true, the simple variance-based approximation is no longer enough. The tails start to dominate the economics. Physicist Jean-Philippe Bouchaud pushed this critique further by arguing that the standard equilibrium picture of markets is too clean. His distinct contribution is to connect fat tails to market microstructure and crowd dynamics. Prices are shaped by crowding, feedback, market impact, and institutional structure, as well as by distributions around a stable mean. Tail risk is built into how markets actually work, not merely a statistical annoyance. Physicist Didier Sornette adds another layer. He models bubbles and crashes as endogenous critical phenomena generated by positive feedback, imitation, and unstable market structure. In his framework, some of the biggest crashes are the natural end point of a system that has become reflexive and fragile, not random bolts from the blue. When returns are fat-tailed, higher moments matter. The cleanest way to see that is to expand the logarithm directly around $R=0$. For $|R|<1$, $$\ln(1+R) = R - \frac{R^2}{2} + \frac{R^3}{3} - \frac{R^4}{4} + ...$$ Taking expectations gives $$\mathbb{E}[\ln(1+R)] = \mathbb{E}[R] - \frac{1}{2}\mathbb{E}[R^2] + \frac{1}{3}\mathbb{E}[R^3] - \frac{1}{4}\mathbb{E}[R^4] + ...$$ This version is more explicit than the shorthand mean-variance formula. The second moment enters with a negative sign, so dispersion hurts growth. The third raw moment enters with a positive sign, which is why positive skew tends to help and negative skew tends to hurt. The fourth raw moment also enters with a negative sign, which means that large extreme moves, whether positive or negative, reduce expected log-growth unless they are offset elsewhere in the distribution. In practice, that means the simple mean-and-variance picture stops being enough once extreme moves become common. Strictly speaking, these are **raw moments**, not skewness and kurtosis themselves. But the intuition lines up with the standardized versions: asymmetry matters, and tail thickness matters. If you wanted to write the same logic more cleanly, cumulants would often be the better language. The point is the same either way: once higher moments become large, the mean-variance shorthand loses explanatory power. Standard option pricing models (Black-Scholes) assume log-normal returns with thin tails. In practice, traders partially correct for this with volatility smiles and skews, but any framework that stays too close to a thin-tailed world will still understate the probability of extreme moves. The exact crash frequency matters less than the broader implication: left-tail events occur materially more often than a naive Gaussian calibration suggests. **Taleb's barbell strategy** is the practical philosophical response to this entire section. His central point goes past the claim that tails are fatter than standard models admit: the right response to a fat-tailed world is to organize a portfolio so that ordinary outcomes are survivable and extraordinary dislocations become opportunities rather than existential threats. A barbell does exactly that: most of the capital sits in positions that are robust to ordinary noise, while a small allocation buys extreme convexity. That structure matters because it changes the shape of the return distribution itself. The puts have bounded downside (premium paid) and very large upside in crashes. That creates positive skewness, limits exposure to ruinous left-tail states, and preserves the possibility of large gains when the system breaks. In Jensen-Kelly terms, Taleb's contribution is to insist that the objective is to survive and compound in a discontinuous world, not to maximize a smooth average in a well-behaved one. Economist Hyman Minsky belongs in this picture too. His core idea was that stability breeds fragility. Long calm periods encourage leverage, maturity mismatch, and crowded positioning, which makes the eventual break far more violent. That is exactly the kind of environment where average outcomes look benign right up until the left tail arrives. Mandelbrot, Bouchaud, Sornette, and Minsky all push in the same direction: the left tail is part of the structure of the world itself, not a small correction to a calm baseline. It is worth being precise about what is doing the work here, because two facts are at play, not one. Concavity, the Jensen effect, says variability is costly. Fat tails say the variability is larger and more one-sided than a Gaussian world assumes. The first is geometry; the second is a fact about the distribution that geometry acts on. Tail hedging matters because both are true at once: the penalty for dispersion is real, and the dispersion is bigger than the textbook admits. ## IX. The Put Strategy as Jensen-Optimal [Spitznagel's tail hedge, 100% SPY plus deep OTM puts](@/series/leptokurtic/2026-02-26-the-tail-hedge-debate-spitznagel-is-right/index.md), is where the abstract logic becomes a concrete portfolio. It is best understood as two legs that do different jobs. **The growth engine.** The first leg is full exposure to the market: 100% SPY. This is the part that compounds. Left alone it also carries the full left tail, and as the previous sections showed, the left tail is what does disproportionate damage to long-run growth. The growth engine supplies market exposure and most of the arithmetic return; by itself it does nothing about the variance drag and ruin risk that concavity imposes. **The convex overlay.** The second leg is a small allocation to deep out-of-the-money puts. Most of the time they expire worthless. In a crash they pay many multiples of premium. Their job is to truncate the left tail, removing the states that compounding punishes most; adding return on average is not the point. This is the leg that addresses the geometry the rest of the article is about. ![Payoff diagram against market return: SPY alone is a straight diagonal line; the deep out-of-the-money put is flat at a small loss until the market falls past the strike and then rises steeply; the combined overlay tracks SPY minus the premium on the upside and flattens into a floor on the deep downside](/images/articles/jensen-payoff.svg) *The two legs and their sum. SPY alone (blue) is pure linear exposure: it carries the full left tail. The deep OTM put (red) costs a small premium most of the time and pays off only in a crash. Add them and you get the overlay (green): it keeps the upside, minus the premium, but its left tail is floored. The growth engine compounds; the convex overlay removes the states that do the most damage to that compounding.* Rather than simply buying disaster insurance in the conventional sense, Spitznagel takes the full chain, compounding, concavity, variance drag, fat tails, and non-ergodicity, and turns it into a specific capital-allocation rule. That is why he is central to this argument rather than an optional practitioner example. Many investors understand the words "fat tails" and still build portfolios as though crashes are just unpleasant drawdowns around a stable mean. Spitznagel's contribution is to force the implication all the way through: if the left tail dominates long-run outcomes, then convexity belongs inside the core architecture of the portfolio rather than bolted on as a cosmetic add-on. Even in the simplest second-order approximation, the mechanism is clear. Recall the approximate growth formula: $$G \approx \mu - \frac{\sigma^2}{2}$$ The two legs map onto the two terms. The growth engine supplies $\mu$. The overlay works on the second term: - **The cost:** Puts have negative expected return, since they usually expire worthless. This reduces $\mu$ by the premium paid, say 0.5% annually. - **The benefit:** In a crash, puts can pay off many multiples of premium, truncating the left tail. In a local approximation, that reduces the effective variance term. More importantly, it removes the states that are most destructive to future compounding. Because the variance drag is quadratic in the local approximation, a modest reduction in tail risk can outweigh a linear premium cost. But that is the conditional part of the claim, and it is worth stating plainly. Jensen and Kelly explain *why* reducing left-tail damage can be worth paying for. They do not guarantee that any particular put is worth its price. The trade works only if convexity is cheap enough relative to the protection it buys, and only if it is implemented with discipline: strike selection, roll mechanics, sizing, and a read on how persistently the market underprices left-tail risk. The mathematics tells you what to look for. It does not promise that the market is currently offering it. This is also where Taleb and Spitznagel meet. Taleb supplies the philosophical and statistical doctrine: avoid ruin, respect discontinuities, seek convexity. Spitznagel supplies the portfolio expression of that doctrine: keep the growth engine, then add a small convex structure that changes what happens in the worst states. The combination is what makes tail hedging more than a fear trade. It becomes a compounding strategy. ## X. Beyond Equities: Where Convexity Is Cheaper The SPY put strategy works, but it may not be optimal. The same Jensen-Kelly logic applies wherever there is reliable asymmetry between calm and crisis: - **Rates:** Central banks cut aggressively in crises (roughly 500bps over 2007–08, 150bps in two weeks during COVID). OTM calls on SOFR futures can capture these panic cuts at low carry cost, though stagflation breaks the thesis. - **FX carry:** Currencies like AUD/JPY offer 3-5% annual carry that unwinds violently in risk-off events (AUD/JPY fell from above 104 in July 2008 to near 55 in October). The carry itself can fund OTM puts on the high-yielder. - **Credit:** IG bonds earn spread, but credit events cluster. CDS protection is convex: IG CDX spreads moved from roughly 50bps to the mid-200s in 2008, while HY CDX moved from the low hundreds toward the 1,500–1,600bps region. The barbell is earn IG spread, buy HY CDS protection. - **Commodities:** Oil grinds in a narrow range, then spikes above $140 or crashes into the $30s. Deep OTM strangles capture both tails, though supply shocks make crude a noisier hedge than the others. The Universa insight is to scan across these markets for wherever tail convexity is cheapest *right now*. Sometimes that is equity vol, sometimes credit, sometimes rates. The [previous article](@/series/leptokurtic/2026-02-26-the-tail-hedge-debate-spitznagel-is-right/index.md#future-work-beyond-equities) develops each market in more detail, with specific trade structures, counterexamples, and implementation notes. ## XI. How This Fits the Series The previous three pieces in the *Leptokurtic* series looked separate, but they are all instances of the same structure. [Twenty Centuries of Financial Data](@/series/leptokurtic/2026-02-12-twenty-centuries-of-financial-data/index.md) established the empirical backdrop: fat tails, devaluations, and regime shifts are the historical baseline, not modern anomalies. [Detecting Crashes with Fat-Tail Statistics](@/series/leptokurtic/2026-02-19-detecting-crashes-with-fat-tail-statistics/index.md) moved to live diagnostics, showing that crashes have detectable precursors and tail behavior that standard tools miss. [The Tail Hedge Debate](@/series/leptokurtic/2026-02-26-the-tail-hedge-debate-spitznagel-is-right/index.md) tested the put overlay on real SPY options data and found that deep out-of-the-money convexity can improve a portfolio's realized path when funded and sized the way Spitznagel describes. Those three supplied the data, the diagnostics, and the implementation; this one supplies the mathematics that connects them. It is worth being explicit about the division of labor. Jensen's inequality is the geometric core, but it is not the whole story by itself. Jensen explains why concavity penalizes variability. Kelly turns that into a sizing rule. Non-ergodicity explains why the relevant objective is the time path rather than the ensemble average. Fat-tail theory explains why variance alone is often not enough because extreme states dominate the economics. The full argument works because these pieces fit together, not because one theorem replaces all the others. ## XII. The Full Intellectual Lineage What looks like a single modern investing idea is actually a chain in which each figure adds one missing piece, or corrects one mistake, left by the previous framework. It runs in two strands, each roughly chronological on its own: first the classical construction of growth-optimal investing, then the modern reckoning with fat tails, fragility, and time. ### The construction: logarithms, growth, and optimal sizing 1. **Napier (1614):** Napier gives the first indispensable tool. Logarithms linearize multiplicative processes, so $\log(ab) = \log(a) + \log(b)$. Without that move, there is no clean way to turn compounding into something that can be analyzed additively. 2. **Jacob Bernoulli (1680s):** Bernoulli studies the limit later denoted by $e$. That connects Napier's logarithmic tool to continuous compounding. Napier gives the language. Bernoulli identifies the natural constant that belongs to that language. 3. **Daniel Bernoulli (1738):** Daniel Bernoulli is the first major bridge from pure mathematics to decision theory. He takes the logarithm and applies it to risky choice, arguing that multiplicative risk changes rational behavior. He does not yet have Kelly or ergodicity, but he points in their direction. 4. **Jensen (1906):** Jensen supplies the missing geometric theorem. If the relevant function is concave, variability is penalized. That turns Daniel Bernoulli's logarithmic intuition into a general structural fact: once wealth is evaluated through a concave function, randomness has a systematic cost. 5. **von Neumann and Morgenstern (1944):** They formalize expected utility as the dominant benchmark for rational choice. This is the framework that later thinkers will refine, challenge, or partially reject. Their role is not to solve the compounding problem. Their role is to define the benchmark that Peters will later criticize. 6. **Shannon (1948):** Shannon makes uncertainty operational. Information is measurable, noise has structure, and better signals change what an optimal repeated decision looks like. This is the mathematical foundation that Kelly later turns into a capital-allocation rule. 7. **Markowitz (1952):** Markowitz gives finance a tractable one-period approximation through mean-variance analysis. That is a real advance, but it is also a simplification. He makes portfolio choice practical, while leaving compounding, path dependence, and ruin underemphasized. 8. **Kelly (1956):** Kelly takes Shannon's information-theoretic framework and translates it into repeated betting and investment. He shows how an edge should be converted into position size when the objective is long-run compound growth. This is where logarithms, information, and compounding become one explicit rule. 9. **Thorp and Breiman (1961 onward):** Thorp shows Kelly can be used in practice, and Breiman gives the long-run dominance result mathematical force. 10. **Cover (1991):** Cover extends the growth-optimal tradition into learning. Universal portfolios show that even without a fully known model, it is possible to asymptotically approach the performance of the best rebalanced portfolio in hindsight. This makes the Kelly logic more robust under model uncertainty. 11. **Fernholz (2002):** Fernholz extends the geometric tradition into portfolio construction itself. Stochastic portfolio theory shows that diversity, rebalancing, and market structure can generate relative returns even without heroic forecasting. That broadens the argument from optimal sizing to the geometry of portfolio design. ### The reckoning: fat tails, fragility, ergodicity, and practice 1. **Mandelbrot (1963):** Mandelbrot challenges the statistical comfort behind standard finance. Returns are not well described by thin-tailed Gaussian assumptions. Once that is true, simple mean-variance reasoning becomes less reliable, and the left tail matters much more. 2. **Samuelson (1969):** Samuelson is the critic who forces the distinction between expected utility and long-run growth to be stated clearly. 3. **Minsky (1986):** Minsky adds the macro-financial mechanism: stability breeds fragility, so left-tail risk is generated by the system itself. 4. **Bouchaud (2003 to 2008):** Bouchaud links fat tails to market microstructure, feedback, and crowd behavior. 5. **Sornette (2003):** Sornette models bubbles and crashes as endogenous critical phenomena rather than exogenous shocks. 6. **Taleb (2007 to 2012):** Taleb is where the statistical critique of fat tails becomes a doctrine of survival. Ruin, fragility, convexity, and asymmetry stop being technical side notes and become the core portfolio problem. Mandelbrot tells you the tails are fatter than you think. Taleb tells you that once you accept that fact, the whole logic of risk-taking has to change. 7. **Peters and Adamou (2011 to 2019):** Peters and Adamou reopen the foundations of decision theory by showing that non-ergodic multiplicative processes must be evaluated along time paths, not across hypothetical ensembles. This reconnects Kelly to a deeper justification: more than a clever betting rule, it is the correct objective for a non-ergodic compounding process. 8. **Spitznagel (2021):** Spitznagel is the implementation layer. He takes the whole chain, logarithms, concavity, Kelly sizing, fat tails, fragility, and non-ergodicity, and turns it into a practical portfolio architecture built around convex protection and survival through crashes. He is the point where the mathematics ceases to be interpretation and becomes an actual portfolio design. ## Conclusion: The Geometry of Survival Jensen's inequality is the geometry of survival in a multiplicative world, not a mathematical curiosity. Wealth is organized by ratios, not differences, and the logarithm is the coordinate system that makes that structure visible. Because that coordinate change is concave, volatility is geometrically destructive: a quadratic drag on compound growth. The long confusion in economics was treating the ensemble average as if it were the investor's lived path. The arithmetic mean looks at what happens across a population of investors in parallel. The geometric mean looks at what happens to you through time. For a single investor compounding over decades, only the time average matters. Tail hedging works because it respects this geometry. It accepts a small, certain reduction in arithmetic return (the put premium) in exchange for protection against the states that do the most damage to compound growth. In a local approximation, that looks like paying to reduce a quadratic variance penalty. In the fuller fat-tailed picture, it is better understood as paying to reduce exposure to ruinous left-tail paths. The logarithm was invented to help astronomers multiply. Four hundred years later, it reveals why crash insurance can be worth paying for, and why, when returns compound, survival comes before growth. ## References - Bernoulli, D. (1738). "Specimen Theoriae Novae de Mensura Sortis." - Bernoulli, J. (1685). *Ars Conjectandi* (posthumous, 1713) - Bouchaud, J. P. & Potters, M. (2003). *Theory of Financial Risk and Derivative Pricing*. Cambridge University Press. - Bouchaud, J. P. (2008). "Economics Needs a Scientific Revolution." *Nature*, 455. - Breiman, L. (1961). "Optimal Gambling Systems for Favorable Games." - Cover, T. M. (1991). "Universal Portfolios." *Mathematical Finance*, 1(1). - Fernholz, R. (2002). *Stochastic Portfolio Theory*. Springer. - Jensen, J. L. W. V. (1906). "Sur les fonctions convexes et les inégalités entre les valeurs moyennes." *Acta Mathematica*, 30. - Kelly, J. L. (1956). "A New Interpretation of Information Rate." *Bell System Technical Journal*, 35(4). - Mandelbrot, B. (1963). "The Variation of Certain Speculative Prices." *The Journal of Business*, 36(4). - Mandelbrot, B. & Hudson, R. L. (2004). *The (Mis)Behavior of Markets*. Basic Books. - Markowitz, H. (1952). "Portfolio Selection." *The Journal of Finance*, 7(1). - Minsky, H. P. (1986). *Stabilizing an Unstable Economy*. Yale University Press. - Napier, J. (1614). *Mirifici Logarithmorum Canonis Descriptio* - Peters, O. & Adamou, A. (2011). "The Time Resolution of the St Petersburg Paradox." *Philosophical Transactions of the Royal Society A*, 369(1956). - Peters, O. & Gell-Mann, M. (2016). "Evaluating Gambles Using Dynamics." *Chaos*, 26(2). - Peters, O. (2019). "The Ergodicity Problem in Economics." *Nature Physics*, 15. - Samuelson, P. A. (1969). "Lifetime Portfolio Selection by Dynamic Stochastic Programming." - Shannon, C. E. (1948). "A Mathematical Theory of Communication." *Bell System Technical Journal*, 27. - Sornette, D. (2003). *Why Stock Markets Crash*. Princeton University Press. - Sornette, D. (2017). *Why Stock Markets Crash: Critical Events in Complex Financial Systems* (updated edition). Princeton University Press. - Spitznagel, M. (2021). *Safe Haven: Investing for Financial Storms*. Wiley. - Taleb, N. N. (2007). *The Black Swan: The Impact of the Highly Improbable*. Random House. - Taleb, N. N. (2012). *Antifragile: Things That Gain from Disorder*. Random House. - Thorp, E. O. (1997). "The Kelly Criterion in Blackjack, Sports Betting, and the Stock Market." - von Neumann, J. & Morgenstern, O. (1944). *Theory of Games and Economic Behavior*. Princeton University Press. --- ### The Tail Hedge Debate: Spitznagel Is Right, AQR Is Answering the Wrong Question *Published: 2026-02-26* > We tested Spitznagel's tail hedging strategy and AQR's critique with 17 years of real SPY options data. In the allocation-reducing framing AQR uses, selling SPY to fund puts, deep OTM puts lose at every budget. In the externally funded overlay Spitznagel actually proposes (100% SPY + put budget on top), the strategy shows a positive raw gap versus plain SPY only when the OTM band is set by strike (not delta), held for longer than the article first published, and tested across rolling windows that include a crash. The edge is regime-conditional: it pays in 6 of 13 rolling 5-year windows (every window that contains a ≥25% SPY drawdown) and drags by 2-3pp/yr in windows that don't. URL: https://federicocarrone.com/series/leptokurtic/the-tail-hedge-debate-spitznagel-is-right/ [Stock markets crash](@/series/leptokurtic/2026-02-19-detecting-crashes-with-fat-tail-statistics/index.md). The S&P 500 price index fell about 57% from October 9, 2007 to March 9, 2009, and about 34% from February 19, 2020 to March 23, 2020.[^sp500_2007_2009][^sp500_2020] A **put option** is a contract that pays you when the market falls below a certain price (the "strike"). If you hold stocks and also hold puts, the puts can offset some of your losses during a crash. The question is whether the cost of buying puts is worth the protection they provide. There are two sides. AQR Capital Management published ["Chasing Your Own Tail (Risk)"](https://www.aqr.com/-/media/AQR/Documents/Insights/White-Papers/AQR-Chasing-Your-Own-Tail-Risk.pdf) ([Berger, Nielsen, and Villalon, 2011](https://www.aqr.com/-/media/AQR/Documents/Insights/White-Papers/AQR-Chasing-Your-Own-Tail-Risk.pdf)). They argue that buying puts systematically costs more than it saves. On the other side, Mark Spitznagel at Universa Investments, where Nassim Taleb is scientific advisor, argues that a small put allocation improves long-term returns ([Spitznagel, 2021](https://www.wiley.com/en-us/Safe+Haven%3A+Investing+for+Financial+Storms-p-9781119401797)). Universa reported a 3,612% gain in March 2020 (via an investor letter, as reported by Bloomberg).[^universa_2020] We tested both claims with [our open-source options backtester](https://github.com/lambdaclass/options_portfolio_backtester) on 17 years of real SPY options data (2008 to 2025), covering three crashes: the 2008 financial crisis, COVID, and the 2022 bear market. The version of the trade that survives the data is narrow but real: **cheap convexity, sized small, and selected by strike rather than delta**. AQR's published critique tests neither half of that. They use near-ATM puts (the most expensive form of crash protection per dollar of notional) inside the allocation-reducing framing (selling SPY to fund the puts, surrendering the equity premium that funds everything else). In that configuration deep OTM puts still lose against SPY. Spitznagel's externally funded overlay flips both choices: deep OTM puts, kept cheap, layered on top of full SPY exposure, sized small. Inside that configuration the strategy shows a positive historical raw gap versus plain SPY across the 2008-2024 window, small at Universa's described 0.5%/yr scale (~+1.4pp/yr), substantial at the engine's risk-adjusted sweet spot (+6pp/yr at 3.3%/yr budget with bi-monthly roll). The edge is regime-conditional: it pays in 5-year windows that contain a major drawdown and drags by 2-3pp/yr in windows that don't. All headline numbers below are gross of transaction costs, slippage, and taxes. Execution drag at the deep-OTM strikes, Israelov's strongest surviving objection, is on top of these gross numbers; we return to it in the limitations section. ### What the thesis actually claims Spitznagel does not claim that buying OTM puts wins every year, every quarter, or every drawdown. People who read him that way, including (implicitly) AQR's published critique, are arguing with a strawman. His real claim is about the long run. Put a small slice of capital into deep OTM puts on top of your equity book, hold them through crashes, monetize the convex payoff when it arrives, and over a long enough horizon you compound faster than equity alone. The mechanism is that the rare big payoffs are large enough, and recur often enough, to more than cover the steady premium drag between them. Year-by-year alpha is the wrong test. Long-run compounding is the right one. On that test, the data says he is right. Across our 17-year SPY sample, every put-overlay budget from 0.5%/yr to 10%/yr beats plain SPY on both annual return and max drawdown. The strike depth Spitznagel describes (deep OTM, not near-ATM) is what makes the trade work. The framing he describes (externally funded, not allocation-reducing) is the half AQR misses. Across every working configuration tested below, the strategy improves max drawdown by 10 to 22 percentage points versus SPY. Three properties of the data are worth naming up front, so the shape of the result is not mistaken for a flaw. First, the full-period gap is concentrated in two years out of seventeen (2008 and 2020). That is exactly what a tail hedge is built to do: sit dormant for years and deliver convexly when a catastrophe arrives. Second, 7 of 13 rolling 5-year windows are negative. That is what positive expected value looks like when the payoffs are this lumpy. Third, walk-forward validation shows that about half of the in-sample gap survives out-of-sample. That is the haircut to apply when sizing this for a real portfolio, not a refutation of the result. The wrong reading of all this is that the strategy works only by luck in crash years. The right reading is that the strategy is built to pay in years like those, and over enough time, with realistic crash frequency, the rare large payoffs more than cover the steady small bleeds. That is Spitznagel's thesis. The data supports it. Every number below reproduces in one command against the open-source engine. The Code section at the end has the exact invocation. {% toolkit(label="Key takeaways") %} - The debate: AQR argues that systematically buying puts costs more than it saves; Spitznagel and Universa argue a small put allocation improves long-run returns. We tested both on 17 years of real SPY options data (2008 to 2025) with an open-source backtester. - The version that survives the data is narrow but real: cheap convexity, sized small, chosen by strike depth (deep out-of-the-money) rather than delta, and funded externally on top of full equity rather than by selling stock to pay for it. - AQR's published critique tests neither half. It uses near-the-money puts inside an allocation-reducing framing, where deep OTM puts still lose. Flip both choices and the overlay beats plain SPY across the window. - The edge is lumpy and regime-conditional: it concentrates in 2008 and 2020, 7 of 13 rolling five-year windows are negative, and walk-forward keeps about half the in-sample gap. That is what positive expected value looks like when payoffs are this convex, not a flaw. - All headline numbers are gross of costs. Deep-OTM execution drag, the strongest surviving objection, sits on top and is handled in the limitations. {% end %} ## Why puts are expensive To understand this debate, we need to start with how options are priced. An option's price depends heavily on **implied volatility** (IV): the market's estimate of how much the stock price will move in the future. Higher expected movement means the option is worth more, because there's a greater chance it will end up profitable. In practice, implied volatility is consistently higher than what actually materializes (**realized volatility**). This gap is called the **Variance Risk Premium** (VRP): $$\text{VRP} = \sigma^2\_{\text{implied}} - \sigma^2\_{\text{realized}}$$ Think of it this way: $\sigma^2\_{\text{implied}}$ is what the market *expects* the variance to be. $\sigma^2\_{\text{realized}}$ is what *actually happens*. The difference is the premium that option buyers pay over fair value. [Carr and Wu (2009)](https://academic.oup.com/rfs/article-abstract/22/3/1311/1581057) documented that this spread is persistently positive. Put buyers pay more than fair value on average. The reason is that investors are willing to overpay for crash protection, the same way homeowners overpay for fire insurance relative to the expected loss from fire. [Bollerslev, Tauchen, and Zhou (2009)](https://scholars.duke.edu/publication/732839) went further: they showed that the VRP is not just a cost: it *predicts* future stock returns. When the gap between implied and realized variance is wide, future equity returns tend to be higher. The same force that makes puts expensive (fear of crashes) also drives the equity premium that the stock portion of the portfolio earns. [Israelov (2019)](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2934538) confirmed the negative average return of puts and titled his paper "Pathetic Protection: The Elusive Benefits of Protective Puts." The [CBOE S&P 500 Put Protection Index (PPUT)](https://cdn.cboe.com/api/global/us_indices/governance/Cboe_SP_500_Put_Protection_Indices_Methodology.pdf) formalizes this as a benchmark: it holds the S&P 500 and buys monthly at-the-money (ATM) puts. It has underperformed the unhedged index over most periods. But ATM puts are an expensive form of crash insurance: they have high theta decay and poor tail-hedge efficiency per dollar of premium. Near-ATM options can have high gamma, so "low convexity" is not the right technical criticism. The problem is that they spend too much premium on ordinary downside cushion rather than rare-crash convexity. Testing ATM puts and concluding "puts don't work" is like testing a sedan on a racetrack and concluding "cars are slow." The deep OTM puts Spitznagel uses cost a fraction of ATM puts and buy more tail convexity per dollar of premium. AQR's argument stops here. Puts lose money on average. Therefore they hurt portfolio performance. This reasoning is incomplete. It looks only at the average return of the put (the **first statistical moment**, the mean). It ignores what the put does to the volatility of the portfolio (the **second moment**, the variance). Compounding depends on both. ## How volatility destroys compounding If you invest money and earn the same return every year, your wealth compounds smoothly. But if returns fluctuate, even with the same *average*, you end up with less. This is called [**variance drain**](@/series/leptokurtic/2026-03-01-at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/index.md), and it's the key to understanding why Spitznagel's strategy works. The geometric (compound) growth rate of a portfolio is approximately (for returns that are small relative to 1, under lognormal assumptions): $$G \approx \mu - \frac{\sigma^2}{2}$$ Here $\mu$ is the arithmetic mean return (the simple average of all yearly returns) and $\sigma$ is the standard deviation of those returns (a measure of how much they fluctuate). The term $\frac{\sigma^2}{2}$ is the variance drain: the penalty that volatility imposes on compounding. A simple example shows why this happens. Start with 100 dollars. Gain 50% one year, lose 50% the next. - After year 1: $100 \times 1.5 = 150$ - After year 2: $150 \times 0.5 = 75$ The arithmetic average return is $\frac{+50\% + (-50\%)}{2} = 0\%$. But you do not end up with 100 dollars. You end up with 75 dollars. You lost 25% despite an average return of zero. The gain and loss were symmetric in percentage terms, but the loss applied to a larger base (150 dollars), so it took away more than the gain added. That is variance drain. The drain is **quadratic** in volatility, meaning it grows with the *square* of the fluctuations: | Portfolio volatility ($\sigma$) | Variance drain ($\frac{\sigma^2}{2}$) | |:-------------------------------:|:-------------------------------------:| | 10% | 0.5%/yr | | 20% | 2.0%/yr | | 40% | 8.0%/yr | Doubling volatility quadruples the drain. This means that large drawdowns are disproportionately costly to long-run wealth. A single 50% crash costs more in compounding terms than ten 5% corrections, even if the total percentage lost is the same. On SPY (2008 to 2025): - Arithmetic mean ($\mu$): 12.14%/yr - Geometric mean ($G$): 10.65%/yr - Variance drain ($\frac{\sigma^2}{2}$): 1.47%/yr - Peak rolling drain during the 2008 crisis: 10.5%/yr **Spitznagel's thesis is about this second term.** If puts reduce portfolio variance by cutting off the worst drawdowns, the reduction in $\frac{\sigma^2}{2}$ can exceed the premium paid. A put costs money on average (it hurts $\mu$, the first moment). But by truncating the worst losses, it reduces the quadratic drag on the portfolio (it helps $\frac{\sigma^2}{2}$, the second moment). The net effect on compound growth $G$ can be positive because the variance drain grows with the *square* of the loss. Preventing a few large drawdowns saves more in compounding terms than the cumulative premium costs. ## Fat tails and put mispricing Taleb makes a related but distinct argument in [*The Black Swan*](https://en.wikipedia.org/wiki/The_Black_Swan:_The_Impact_of_the_Highly_Improbable) and [*Statistical Consequences of Fat Tails*](https://arxiv.org/abs/2001.10488). Standard option pricing models (like [Black-Scholes](https://en.wikipedia.org/wiki/Black%E2%80%93Scholes_model)) assume returns follow something close to a **normal (Gaussian) distribution**. In a normal distribution, events far from the average are extraordinarily rare. A crash on the scale of 2008 or 2020 sits far enough into the tails that Gaussian models treat it as effectively negligible for ordinary portfolio construction. In reality, over the 2008 to 2025 sample we study here, SPY experienced multiple drawdowns of roughly 30% or worse, including 2008-09 and 2020, with 2022 close behind. Real markets have **fat tails**: [extreme events are far more frequent than Gaussian models predict](@/series/leptokurtic/2026-02-12-twenty-centuries-of-financial-data/index.md). The probability of a large crash is not astronomically small. It is orders of magnitude higher than a thin-tailed model would suggest. This has a direct consequence for put pricing. Option markets do price skew: deep OTM puts trade at higher implied volatility than ATM options, reflecting some awareness of tail risk. But even after skew is priced, **deep OTM puts may still be cheap relative to the realized frequency of crashes**. The VRP shows that puts are expensive relative to realized volatility in normal times. But the relevant comparison for deep OTM puts is not average realized volatility; it is the actual frequency and magnitude of extreme drawdowns. The observed frequency of large drawdowns in our sample is far higher than a Gaussian baseline would imply. Taleb calls this the difference between **Mediocristan** (where Gaussian statistics work, like human height) and **Extremistan** (where they do not, like financial returns). The S&P 500 lives in Extremistan. The variance drain argument (puts reduce $\frac{\sigma^2}{2}$) and the mispricing argument (puts are cheap relative to true tail probabilities) are independent. Either one alone could justify the strategy. Together they explain why the results are as strong as they are. ## Theoretical foundations The variance drain argument and the fat-tail mispricing argument have deeper roots than the Spitznagel-AQR debate suggests. **Ole Peters and ergodicity economics.** The variance drain formula $G \approx \mu - \frac{\sigma^2}{2}$ is a special case of a broader insight. [Peters (2019)](https://www.nature.com/articles/s41567-019-0732-0) argues that classical expected-value reasoning fails for multiplicative processes like portfolio growth. The ensemble average (what happens across many parallel investors) diverges from the time average (what happens to one investor over many periods). For a single investor compounding over decades, the time average is what matters, and it is always lower than the ensemble average when returns fluctuate. Spitznagel's strategy works because it improves the time-average growth rate, even though it reduces the ensemble-average return (by paying premium). Most of finance optimizes for the wrong average. **Bouchaud on fat tails and hedging.** [Bouchaud, Iori, and Sornette (1996)](https://www.cfm.com/wp-content/uploads/2022/12/237-1994-real-world-options-smile-and-residual-risk.pdf) showed that in fat-tailed markets, Black-Scholes delta hedging leaves large residual risk. The standard model assumes continuous rebalancing in a Gaussian world; real markets have jumps and heavy tails that make perfect hedging impossible. This means option sellers bear more risk than their models suggest, and option buyers (like tail hedgers) get more protection than the models price in. This is the theoretical basis for why deep OTM puts may be systematically cheap relative to true tail risk. **Sornette on endogenous crashes.** [Sornette (2003)](https://press.princeton.edu/books/paperback/9780691175959/why-stock-markets-crash) argues that large crashes are not exogenous shocks but endogenous instabilities, the result of self-reinforcing feedback loops (herding, leverage, procyclical risk management) that build up over months or years before releasing suddenly. His [Log-Periodic Power Law Singularity (LPPLS)](https://en.wikipedia.org/wiki/Didier_Sornette) model attempts to detect these signatures. This is relevant to our macro-signal finding: standard indicators (VIX, yield curve, credit spreads) measure risk levels but not the endogenous buildup that precedes crashes. Sornette's approach is structurally different (it looks for acceleration patterns in price itself), though its real-time track record remains debated. **Rare disaster models.** [Barro (2006)](https://academic.oup.com/qje/article-abstract/121/3/823/1917876) formalized the idea that the equity premium itself may be compensation for rare catastrophic events. If investors demand higher average returns because crashes happen, then the equity premium and the tail-hedge premium are two sides of the same coin. [Kelly and Jiang (2014)](https://academic.oup.com/rfs/article/27/10/2841/1607080) showed that time-varying tail risk is priced in equity cross-sections. [Bollerslev, Tauchen, and Zhou (2009)](https://scholars.duke.edu/publication/732839) demonstrated that the variance risk premium predicts future stock returns. The same VRP that makes puts expensive also signals future equity returns. **Bhansali on offensive risk management.** [Bhansali (2008)](https://www.pm-research.com/content/iijpormgmt/34/4/68) laid out tail hedging as a systematic program before the crisis blew up: which instruments to use (deep OTM puts, payer swaptions), how to size them, and how to budget the cost of carry. Bhansali and Davis (2010) then made the harder argument: the hedge's real value is not insurance against the drop, it is cash exactly when other assets are forced-seller cheap. Monetize the hedge, redeploy into beaten-down risk, and the crash turns from a forced "sell low" into a funded "buy low." Convexity is offensive, not just defensive. Bhansali, Chang, Holdom, and Rappaport (2020) sharpened this with the March 2020 case: most tail programs fail not because the payoff does not come, but because holders freeze and never monetize at the right moment. The discipline is rules-based, harvest at defined multiples or drawdown thresholds, redeploy into risk, then re-establish the hedge. This is also the strongest direct rebuttal to [Israelov's](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2934538) "just hold less equity" alternative: holding cash gives you a smaller drawdown, but it does not give you a force multiplier at the bottom when you most want to be a buyer. ## The core disagreement This is where the debate breaks down. The two sides are not testing the same portfolio. ### What AQR tests AQR tests portfolios where you **sell some of your stocks** to buy puts: $$R\_{\text{portfolio}} = (1-w) \cdot R\_{\text{SPY}} + w \cdot R\_{\text{puts}}$$ At $w = 1\%$: you hold 99% in stocks, 1% in puts. Total portfolio: 100%. AQR's argument seems intuitive: you are taking money out of your best asset (stocks, which go up on average) and putting it into an asset with negative expected return (puts, which expire worthless most of the time). But there is a second, less obvious difference: AQR's published tests use **near-the-money puts** (roughly 5% out of the money, delta around $-0.35$). These puts are expensive per dollar of crisis convexity because they spend a lot of premium on routine downside protection. [Israelov (2019)](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2934538) confirmed this: his "Pathetic Protection" paper tests puts in exactly this delta range and finds negative returns. When we run the allocation-reducing framing with **deep OTM puts**, the result improves relative to near-ATM protection but still does not support the headline strategy. Selling SPY to fund the puts gives up too much equity premium. That is AQR's strongest point. The configuration that survives is not "puts instead of stocks"; it is a small, externally funded convex overlay on top of the full equity book. ### What Spitznagel actually does Spitznagel keeps **100% in stocks** and buys puts with a small separate budget on top: $$R\_{\text{portfolio}} = 1.0 \cdot R\_{\text{SPY}} + w \cdot R\_{\text{puts}}$$ The strategy requires a small amount of capital beyond the core equity position to fund the put premium. Some might call this leverage. But it is fundamentally different from ordinary leverage. Ordinary leverage means borrowing money to buy more stocks. If you borrow to hold 130% in stocks, your gains are 30% bigger but your losses are also 30% bigger. Drawdowns get worse in proportion to the leverage. The payoff is symmetric: leverage amplifies both good and bad outcomes equally. A put overlay works differently. It is **asymmetric**: - In calm markets, you bleed a small, known premium (the cost of the puts). - In a crash, the puts pay off at 10x to 50x the premium paid. This asymmetry happens because a deep OTM put's sensitivity to the market (its **delta**, $\Delta$) increases as prices fall. Delta measures how much the put's price changes per 1-dollar change in the stock. A deep OTM put starts with a delta near zero (barely reacts to market moves). As the market drops and the put moves closer to being "in the money," delta approaches $-1.0$ (moves dollar-for-dollar with the stock). A small position becomes a large hedge exactly when you need it. AQR's published analysis differs from Spitznagel's on two dimensions: the funding/framing (sell stocks to buy puts vs. keep the equity book intact and fund the hedge separately) and the put selection (near-ATM vs deep OTM). Both differences matter. Put selection determines whether the option sleeve buys useful tail convexity; funding determines whether the equity premium remains in the portfolio. The article's positive results require both: deep OTM selection and an externally funded overlay. ![A two-by-two matrix comparing put selection and funding. Near-ATM plus allocation-reducing funding is the AQR and Israelov test and fails. Deep OTM plus allocation-reducing funding improves the instrument but still gives up equity premium. Near-ATM plus external funding preserves stock exposure but buys costly routine downside protection. Deep OTM plus external funding is the Spitznagel quadrant and works in crash regimes, with a +6.07pp raw gap versus plain SPY at the 3.3% budget in this sample.](/images/charts/spitznagel-funding-selection-matrix.png) ![Schematic payoff curves for three strategies that each deploy 1% of capital. Linear leverage moves symmetrically with the market. Near-ATM puts give a linear cushion below their strike. Deep OTM puts stay nearly flat through normal moves and explode in value below their strike, illustrating the convexity advantage.](/images/charts/spitznagel-1-convexity-payoff.png) The public debate has been contentious. [Taleb and Asness clashed publicly in May 2020](https://www.bloomberg.com/news/articles/2020-05-21/taleb-spars-with-asness-on-twitter-over-tail-risk-hedges) over whether Universa's March 2020 returns proved the strategy works. [Aaron Brown (ex-AQR) wrote in Bloomberg](https://www.bloomberg.com/opinion/articles/2023-04-06/universa-s-3-126-black-swan-return-is-legit-but-with-an-asterisk) that Universa's percentage returns are "legit but with an asterisk": the 3,612% is on the put allocation, not the total portfolio. [CalPERS' then-CIO Ben Meng argued](https://www.bloomberg.com/news/articles/2020-04-17/calpers-cio-says-his-hedges-worked-better-than-tail-risk-funds) that their alternative hedges outperformed tail-risk funds. These disagreements often reduce to framing: what denominator you use, and whether you funded the puts by selling stocks. AQR's follow-up work ([Israelov (2019)](https://www.aqr.com/Insights/Research/White-Papers/Pathetic-Protection-The-Elusive-Benefits-of-Protective-Puts) and [Hurst, Ooi, and Pedersen (2017)](https://www.aqr.com/insights/research/white-papers/tail-risk-hedging-contrasting-put-and-trend-strategies)) continues to test ATM or near-the-money puts in the sell-stocks-to-fund-puts framing. Neither paper tests the deep OTM overlay that Spitznagel actually runs. ## Results All tests use deep OTM puts on real SPY options data. The default configuration the article centers on, post-correction, is: - **Strike-based** filter: strike between 55% and 60% of spot (≈ 40-45% OTM at trade entry) - DTE 90 to 180 days at entry - Exit when DTE drops to 30 (≈ 60-150 days held) - **Bi-monthly** rebalance (roll once every two months) - 100% SPY allocation, put premium funded externally on top - Daily exit checks; freed cash from exits is immediately redeployed into SPY at the same day's price (the "monetize at the bottom" pattern) A note on terminology: **"deep OTM"** means the put's strike price is far below the current market price. A put at 40% OTM very roughly corresponds to a very low probability of finishing in the money in calm regimes, much higher in stressed ones. Delta and option-implied probabilities are only heuristics here, not literal real-world probabilities of profit. These puts are cheap per dollar of notional, but when the market crashes the deepest puts can multiply in value 10x to 50x. A note on filter choice: we select puts by **strike-to-spot ratio** rather than by delta. Delta-based filtering produces depth that drifts with implied volatility, in 2008's high-vol regime, a delta of -0.02 corresponds to puts only 7% out of the money; in calm 2017 the same delta corresponds to puts 30% out of the money. The strikes you actually buy are not the strikes the literature describes. Filtering by strike-to-spot ratio holds the put's depth constant regardless of IV, which is what Spitznagel's narrative of "5-sigma" or "30% OTM" puts actually means. All results below use the strike-based filter unless explicitly stated. ### The calm-period test: 2012-2018 Before showing 17 years of data that include three major crashes, we start with the hardest test for the strategy: the **calmest 7-year stretch in our sample**. From 2012 to 2018, no correction exceeded -19.3%. There was no GFC, no COVID. The question is whether the strategy bleeds itself dry without a tail event. **Spitznagel framing** (100% stocks, strike-based 40-45% OTM puts on top, DTE 90-180, exit DTE 30, bi-monthly roll) during 2012-2018: | Config | Annual Return | vs SPY | Max Drawdown | |--------|:------------:|:------:|:------------:| | 100% SPY (baseline) | +12.35% | | -19.3% | | + 0.5%/yr puts | +9.27% | -3.08pp | -19.1% | | + 1.0%/yr puts | +9.30% | -3.05pp | -16.6% | | + 3.3%/yr puts | +9.40% | -2.95pp | -19.3% | This is the honest insurance picture. In a tail-light period, the strategy **underperforms** SPY by roughly 3pp/yr at every budget. The premium drag is real and continuous; the small 2018 correction (-19% peak-to-trough) is not deep enough to monetize a 40% OTM put. The deeper-OTM strikes that pay massively during a true crash sit too far out of the money to be hit during ordinary corrections. The 0.5% Spitznagel overlay would have lost roughly 22% cumulative versus buy-and-hold over this seven-year stretch. This is exactly what insurance looks like: a small steady premium that returns essentially nothing in calm years and pays off in crisis years. The strategy does not invent alpha in flat markets. What it does is keep its options open. When the crash comes, the convex payoff is sitting there to monetize. When it doesn't, you have paid the premium for nothing, which over a calm decade adds up to a meaningful cumulative drag. The full-period numbers below recover the cost (and then some) only because two crashes, GFC 2008-09 and COVID 2020, sit on either side of the calm middle. ![Equity curves from the backtester, 2008 to 2025: plain SPY versus the Spitznagel-framing overlay at the article's default 3.3%/yr budget. The overlay tracks SPY closely between crashes and pulls ahead in the GFC 2008-09 and COVID 2020 drawdowns; the gap widens at each crash. GFC, COVID, and 2022 bear periods shaded.](/images/charts/spitznagel-equity-curves-real.png) ### What AQR's published configuration actually does (2008-2025) The full 17-year sample includes the 2008 GFC, the 2020 COVID crash, and the 2022 bear market. AQR's published critique tests **near-ATM puts** (roughly 5% OTM, delta $\approx -0.35$) inside the allocation-reducing framing (sell SPY to fund puts). These puts have high theta decay and poor tail-hedge efficiency per dollar of premium. Even at small budgets, they accumulate premium drag faster than they save in drawdown protection. Israelov's [Pathetic Protection](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2934538) result is the canonical reference for the negative outcome of this configuration; we do not re-test it here. The mistake in AQR's published critique is not that the result is wrong inside their tested configuration, it is that the configuration itself does not match what Spitznagel and Universa do. AQR's tests vary on **two** dimensions simultaneously: the framing (no-leverage / allocation-reducing) AND the put selection (near-ATM, delta $\approx -0.35$). Both choices stack against the strategy and against each other. Deep OTM puts at 40% OTM cost a fraction as much per unit of notional protection, and the externally-funded framing preserves equity exposure that funds compounding. We test the configuration Spitznagel actually proposes next. ### Spitznagel framing: 100% stocks plus puts on top (2008-2025) The full 17-year sample with strike-based 40-45% OTM puts, DTE 90-180 at entry, exit when DTE drops to 30, bi-monthly roll, 100% SPY allocation plus an external put budget. Every row below reproduces in one command (Code section below). | Config | Annual Return | Raw gap vs SPY | Max Drawdown | Final ($1M start) | |--------|:------------:|:-------------:|:------------:|:---:| | 100% SPY (baseline) | +10.68% | | -51.9% | $5.6M | | 100% SPY + 0.5%/yr puts | +12.06% | +1.39pp | -41.3% | $6.9M | | 100% SPY + 1.0%/yr puts | +13.21% | +2.54pp | -31.2% | $8.2M | | 100% SPY + 2.0%/yr puts | +15.03% | +4.36pp | -30.8% | $10.8M | | 100% SPY + 3.3%/yr puts | +16.74% | +6.07pp | -30.2% | $13.9M | | 100% SPY + 5.0%/yr puts | +18.27% | +7.59pp | -32.3% | $17.3M | | 100% SPY + 7.0%/yr puts | +19.39% | +8.71pp | -38.4% | $20.3M | | 100% SPY + 10.0%/yr puts | +20.18% | +9.50pp | -44.1% | $22.7M | Two patterns dominate the table. First, the **raw gap versus plain SPY scales nearly linearly with budget up to ~5%/yr**, then continues to grow but with diminishing returns as the marginal premium dollar buys progressively less convex payoff. The 0.5%/yr row matches the budget Universa publicly describes; even at that small scale the strategy beats plain SPY by +1.4pp/yr with 10pp better max drawdown. At 3.3%/yr the raw gap is +6pp and the max drawdown improves by 22pp. Because the put budget is externally supplied, this is not the same as pure excess over a matched-capital benchmark; the fair benchmark is SPY plus the same external funding source without the puts. We return to this in the implementation notes. Second, **max drawdown improves up through ~3-5%/yr budget and then starts to deteriorate** as the put position itself becomes the dominant source of portfolio variance. Beyond 7%/yr, the drawdown is dragged out by premium bleed in calm years; beyond 10%/yr the strategy gives back enough between crashes to materially worsen the worst trough. The risk-adjusted sweet spot lands at **3.3%/yr** for the metrics this article emphasizes: Sortino peak, 22pp better max drawdown, and +6.07pp/yr raw gap versus plain SPY. For a more conservative implementation that matches Universa's documented sizing, the **0.5%/yr** row is the natural choice, smaller historical gap, small drag, and a meaningful drawdown improvement. ![Bar chart comparing max drawdown for plain SPY and externally funded deep OTM put budgets from 0.5% to 10% per year. SPY loses -51.9%. The overlay improves drawdown to -41.3% at 0.5%, about -31% at 1-2%, and -30.2% at 3.3%, then deteriorates as the put budget rises to 7% and 10%.](/images/charts/spitznagel-drawdown-by-budget.png) ### Convexity breakdown The following table shows the full picture. **Sharpe ratio** measures risk-adjusted return: $\text{Sharpe} = \frac{R\_\text{portfolio}}{\sigma\_\text{portfolio}}$. A higher Sharpe means more return per unit of risk. All Sharpe ratios use a risk-free rate of 0%. | Strategy | Premium %/yr | Annual % | Raw gap % | Return per 1% Premium | Max DD % | Vol % | Sharpe | |----------|:----------:|:--------:|:--------:|:-------------------:|:--------:|:-----:|:------:| | 100% SPY (baseline) | 0.00 | 10.68 | +0.00 | n/a | -51.9 | 20.0 | 0.534 | | + 0.5%/yr deep OTM | 0.50 | 12.06 | +1.39 | 2.78 | -41.3 | 19.7 | 0.613 | | + 1.0%/yr deep OTM | 1.00 | 13.21 | +2.54 | 2.54 | -31.2 | 20.8 | 0.636 | | + 2.0%/yr deep OTM | 2.00 | 15.03 | +4.36 | 2.18 | -30.8 | 25.2 | 0.597 | | + 3.3%/yr deep OTM | 3.30 | 16.74 | +6.07 | 1.84 | -30.2 | 32.2 | 0.519 | | + 5.0%/yr deep OTM | 5.00 | 18.27 | +7.59 | 1.52 | -32.3 | 41.3 | 0.442 | | + 7.0%/yr deep OTM | 7.00 | 19.39 | +8.71 | 1.24 | -38.4 | 51.0 | 0.380 | | + 10.0%/yr deep OTM | 10.00 | 20.18 | +9.50 | 0.95 | -44.1 | 63.7 | 0.317 | Two things stand out. First, the return per 1% of annual put premium declines from about 2.78x at the smallest tested budget to 0.95x at the largest. Each additional dollar of premium buys less convexity than the dollar before it. The first cheap puts you add to a portfolio are the most efficient; the marginal hedge erodes as the position scales. The convex payoff is real, but its rate diminishes. Second, the Sharpe ratio peaks at the 1.0% budget (0.636), one band higher than the smallest tested. Sharpe **drops** at higher budgets even though the raw gap versus plain SPY keeps growing, because the put position itself adds upside variance that Sharpe penalizes identically to downside variance. This is the limit of Sharpe as a metric for a strategy whose entire purpose is to reshape the return distribution asymmetrically. The downside-focused metrics in the next table tell a different story. ![Two-panel chart showing diminishing returns to put premium. The left panel plots raw gap per 1% of annual premium, declining as the budget grows. The right panel plots max drawdown and Sharpe by budget: drawdown improves through the 3.3% range while Sharpe peaks earlier and then falls as the put sleeve adds upside volatility.](/images/charts/spitznagel-4-budget-curves.png) ### Beyond Sharpe: downside-focused metrics Sharpe treats upside and downside volatility equally. But upside volatility is welcome: you don't mind large positive returns. Downside-focused metrics give a clearer picture of how the puts reshape the return distribution: | Strategy | Sharpe | Sortino | Calmar | Max DD % | Pos Months % | |----------|:------:|:-------:|:------:|:--------:|:------------:| | 100% SPY | 0.534 | 0.650 | 0.206 | -51.9 | 66.5 | | + 0.5%/yr deep OTM | 0.613 | 0.773 | 0.292 | -41.3 | 66.0 | | + 1.0%/yr deep OTM | 0.636 | 0.863 | 0.424 | -31.2 | 67.0 | | + 2.0%/yr deep OTM | 0.597 | 0.951 | 0.489 | -30.8 | 66.5 | | + 3.3%/yr deep OTM | 0.519 | 0.958 | 0.554 | -30.2 | 67.0 | | + 5.0%/yr deep OTM | 0.442 | 0.923 | 0.566 | -32.3 | 66.0 | | + 7.0%/yr deep OTM | 0.380 | 0.879 | 0.505 | -38.4 | 65.5 | The three risk-adjusted measures peak at three different budgets, and the gap is informative. **Sharpe** (return ÷ total volatility) peaks at the **1.0%** budget. Sharpe drops past that because puts add as much *upside* variance during crash payoffs as the variance they cut from the downside, and Sharpe penalizes both equally. For a strategy whose entire purpose is to reshape returns asymmetrically, Sharpe systematically understates the improvement. **Sortino** (return ÷ downside deviation only) peaks at the **3.3%** budget. Sortino captures what Sharpe misses: the puts disproportionately reduce *downside* deviation while the crash payoffs add to *upside* deviation. From 0.650 (SPY) the metric rises to 0.958 at 3.3%, a 47% improvement. Even at 5%/yr budget Sortino is still 42% above SPY. **Calmar** (return ÷ max drawdown) peaks at the **5.0%** budget. From 0.206 (SPY) it nearly triples to 0.566. Calmar penalizes drawdowns hardest, and the strategy's main effect on drawdowns is to truncate the tail of catastrophic losses. Even at 3.3% the Calmar improvement is 2.7x, meaningful relative to almost anything that can be done to an equity portfolio with bonds or cash. **Max drawdown** itself improves continuously from -51.9% (SPY) down to about -30% in the 1-5%/yr range, then begins to deteriorate as the strategy gives back too much premium during long calm stretches. The minimum-drawdown sweet spot sits at ~3.3% budget at -30.2%. The reader who weighs all three measures together lands at the **3.3%/yr** budget as the practical compromise: Sortino peak, Calmar 90% of its maximum, and the deepest max-drawdown improvement available. The 1.0%/yr budget is the conservative alternative for readers who weight Sharpe more heavily. ![Line chart of three risk-adjusted ratios (Sharpe, Sortino, Calmar) plotted against put budget from 0% to 10%/yr. Sharpe peaks at 1.0% budget, Sortino at 3.3%, Calmar at 5.0%. The three peaks at three different budgets illustrate why "sweet spot" depends entirely on which metric you weight: Sharpe penalizes the upside vol that puts add during crashes, while Sortino and Calmar reward the asymmetric payoff.](/images/charts/spitznagel-risk-metrics-peaks.png) **Kurtosis** does the opposite of what one might expect. It does **not** stay flat. SPY kurtosis is 17.2; at 0.5% budget it climbs to 31; at 1.0% to 137; at 2.0% to 276; at 3.3% to 305. The put overlay does not flatten the distribution, it adds a fat right tail on top of the existing fat left tail. Past the sweet spot, the put position itself becomes the dominant source of tail risk in the portfolio. This is the deepest reason the strategy degrades at high budgets even when realized crashes pay off: the variance the puts add eventually overwhelms the variance the puts cut. ### The diminishing returns of higher budgets More put budget is not strictly always better. At small budgets, puts reduce portfolio variance by truncating the left tail: vol holds near SPY's 20.0% through ~1% budget. Past that the puts themselves start adding meaningful upside variance: at 2%/yr the annualized vol rises to 25.2%; at 3.3%/yr to 32.2%; at 5%/yr to 41.3%. The payoff structure that gives the strategy its convexity also makes the put position itself increasingly volatile as it scales up. The Sharpe ratio reflects this directly: it rises from 0.534 (SPY alone) to 0.636 at 1.0% budget, then declines past that even as the raw gap versus plain SPY continues to grow. Past the Sharpe peak, the strategy continues to add Sortino and Calmar but with diminishing returns on the upside-volatility-blind metric. Max drawdown improves smoothly across the working range, from -51.9% (SPY) down to -30.2% at 3.3% budget. Beyond 5%/yr it begins to deteriorate as long calm stretches accumulate premium bleed that no single crash payoff fully repairs. The minimum-drawdown sweet spot lands at the 3.3%/yr budget. This is why we recommend the **3.3%/yr budget** as the practical default for an investor whose objective is the best risk-adjusted, downside-protected compound return: it sits at the Sortino peak, near the Calmar peak, with the deepest max-drawdown improvement available. The **1.0%/yr budget** is a defensible alternative if you weight Sharpe (symmetric vol) more heavily: it gives a smaller absolute alpha (+2.54pp vs +6.07pp) but a higher Sharpe (0.636 vs 0.519). The **0.5%/yr budget** matches Universa's described scale, small alpha, small drag, modest drawdown improvement. ## The central thesis: Spitznagel is right His thesis, stated plainly: over a long enough horizon, with the kinds of crashes markets actually have, externally-funded deep OTM puts on top of full equity compound faster than equity alone. He does not claim puts win every year. He does not claim every 5-year stretch is positive. He claims the long-run number comes out ahead because the rare large payoffs are big enough, and recurrent enough, to dominate the steady premium drag in between. The data says he is right. Across 17 years of real SPY options, every budget from 0.5% to 10% per year beats plain SPY on both compound return and max drawdown. At the recommended 3.3%/yr budget the strategy compounds at 16.74%/yr against SPY's 10.68%, with a -30.2% max drawdown against SPY's -51.9%. Even at Universa's documented 0.5%/yr scale the strategy still ends ahead (+1.39pp/yr) with a meaningfully smaller drawdown. The audit comes out where the title says: Spitznagel is right. What "right" means inside a finite sample is worth being careful about. Three things hold together. The structure works. Deep OTM puts, externally funded, sized small, beat plain SPY on every cross-validation split we ran. The drawdown improvement is real and large, 10 to 22 percentage points. The payoff is convex, not uniform. Two of seventeen years carry most of the gap. That is the design of a tail hedge, not a flaw of the implementation. Long-run compounding is the test, not year-by-year alpha. The forward number is smaller than the in-sample number. About half of the in-sample edge survives out-of-sample on walk-forward. That is a haircut on the magnitude, not on the structure. Halve every raw-gap number in the tables above to get a defensible forward expectation. Put together, these describe the asset class Spitznagel has been describing for two decades: tail-hedge convexity, mispriced relative to true catastrophe frequency, positive long-run compound growth when the implementation matches his structural choices (deep OTM, externally funded, monetize and reinvest). What this article adds is the empirical check. At the strike depth and funding choice he proposes, the asset class survives 17 years of real SPY data and three real crashes. ## Why it works: convexity, not leverage The word "leverage" is misleading here. The put premium is not the same as notional exposure. When you spend 0.5% of portfolio value on deep OTM puts, you are not adding 0.5% of equity exposure. You are buying contingent downside convexity: a payoff that is near zero most of the time and very large during crashes. If you instead spent 0.5% borrowing to buy more stocks, the raw return gap would be about 0.05%/yr (0.5% of the equity premium). Instead, we observe +1.39%/yr at that budget in the full sample. The observed gap is roughly **28 times** what linear leverage would produce. The extra return is not coming from additional market exposure. It is coming from the put's convexity, and the multiplier is bounded by how cheap the convexity is in the market: the same 0.5% premium would buy far less payoff in an environment where deep-OTM skew is rich. This distinction matters because ordinary leverage and a put overlay have opposite effects on the two quantities that determine compound growth: $$G \approx \mu - \frac{\sigma^2}{2}$$ **Ordinary leverage** (borrowing to buy more stocks) scales both terms proportionally. If you use 1.5x leverage, $\mu$ increases by 50% but $\sigma$ also increases by 50%, so $\sigma^2$ increases by 125%. The variance drain grows faster than the return. This is why leveraged ETFs [underperform their stated multiple](https://www.investopedia.com/articles/financial-advisors/082515/why-leveraged-etfs-are-not-longterm-bet.asp) over long periods: they win on the first moment and lose on the second. **A put overlay** works on each moment independently. The premium is a small, linear cost to $\mu$ (the first moment). But the put's payoff during a crash truncates the left tail of the return distribution, which disproportionately reduces $\sigma^2$ (the second moment). Because the drain is quadratic in volatility, even a modest reduction in tail losses saves more in compounding terms than the premium costs. A concrete example: suppose SPY drops 50% in a year. Without puts, that single year's contribution to variance drain is roughly $0.50^2 / 2 = 12.5\%$. With puts that offset 10% of the decline (reducing the loss to 40%), the drain contribution drops to $0.40^2 / 2 = 8.0\%$, a savings of 4.5 percentage points, from a put position that cost 0.5% of the portfolio. Taleb describes this structure as a **barbell** in [*Antifragile*](https://en.wikipedia.org/wiki/Antifragile_(book)): combine a large, safe position with a small, highly convex one, and avoid the middle. The Spitznagel portfolio is a barbell. The bulk (100%) is in a broad equity index. A small sliver (0.5% to 3.3% annual premium) is spent on deep OTM puts. The bulk earns the market return. The sliver has **bounded downside** (you can only lose the premium) and **convex upside** (the puts can return 10x to 50x during a crash). A "medium risk" portfolio with 80% stocks and 20% bonds reduces your exposure to crashes but also reduces your exposure to the equity premium. The barbell keeps full exposure to the equity premium while adding crash protection through a completely different mechanism. As described above, this asymmetry comes from the put's delta shifting from near zero to near $-1.0$ as the market crashes. A tiny position becomes a large hedge exactly when you need it. Borrowing cannot replicate this. Borrowing amplifies gains and losses symmetrically. Puts amplify only the crash payoff. **Ordinary leverage can wipe you out.** If you borrow to hold 150% in stocks and the market drops 50%, you lose 75% of your equity. A margin call forces you to sell at the bottom. **A put overlay cannot do this.** If you spend 0.5% of your portfolio on puts and those puts expire worthless, you lose 0.5%. That is the worst outcome. The maximum loss is the premium paid, which you know at purchase. There is no margin call. Comparing a 50% market decline: - **100% SPY + 0.5% in puts**: roughly a 47% loss (puts pay off during the decline) - **100.5% SPY via margin**: a 50.25% loss (leverage amplifies the decline) The put overlay reduces the drawdown. The margin position amplifies it. Similar total capital committed, opposite outcomes. ## Sensitivity and robustness A strategy that only works with one specific set of parameters is likely overfitted to the data. We tested this concern from multiple angles. ### Parameter sensitivity We swept the strategy across all combinations of **OTM depth** (strike as a fraction of spot), **DTE entry band**, **exit DTE**, and **rebalance cadence** at the 3.3%/yr budget. The robust family, configurations that produce a positive raw gap versus plain SPY on both halves of the data (out-of-sample on whichever half is held out), is tightly clustered: | OTM | DTE entry | Exit DTE | Rebal | H1 raw gap | H2 raw gap | Min half | Mean | Max DD | |---|---|---|---|---|---|---|---|---| | **40%** | **90-180** | **30** | **bi-monthly** | **+8.33** | **+3.32** | **+3.32** | **+5.83** | **-30.2%** | | 35% | 90-180 | 30 | bi-monthly | +7.41 | +3.30 | +3.30 | +5.36 | -30.0% | | 30% | 90-180 | 30 | bi-monthly | +6.98 | +2.96 | +2.96 | +4.97 | -30.0% | | 35% | 90-180 | 30 | monthly | +4.76 | +1.51 | +1.51 | +3.14 | -31.0% | | 25% | 90-180 | 30 | bi-monthly | +3.37 | +2.29 | +2.29 | +2.83 | -30.1% | | 30% | 90-180 | 30 | monthly | +4.20 | +1.35 | +1.35 | +2.77 | -31.1% | | 25% | 90-180 | 30 | monthly | +1.75 | +1.06 | +1.06 | +1.41 | -36.9% | | 20% | 90-180 | 30 | bi-monthly | +1.26 | +1.47 | +1.26 | +1.36 | -36.0% | | 40% | 90-180 | 30 | monthly | +6.09 | +0.84 | +0.84 | +3.47 | -31.1% | Three patterns stand out. First, **the depth band 25-40% OTM is the working range**. Below 20% the puts are not deep enough to be cheap and the strategy loses to SPY at every rebalance cadence. Above 45% the strategy looks great on H1 (driven by the GFC's -52% drawdown) but flips *negative* out-of-sample on H2 because COVID's -34% drawdown isn't deep enough to hit those strikes. The 40% depth is the practical ceiling that generalizes. Second, **DTE 90-180 at entry, exit at DTE 30, dominates other DTE structures**. Shorter DTE windows (60-90) work on H1 only; longer DTE bands (180-365) work on H2 only. The 90-180/30 combination, hold for ~60-150 days, generalizes across both. Third, **bi-monthly rebalance beats monthly across the depth range**. Fewer rebalance cycles mean less premium decay over the year, with the same total budget allocated to fewer but larger positions. The improvement is meaningful, at 40% OTM, bi-monthly gives +6.05pp/yr full-period vs monthly's +3.70pp/yr. The article's default, 40-45% OTM, DTE 90-180, exit 30, bi-monthly, 3.3%/yr, sits at the top of the table. The robust family below it is a tight neighborhood: small departures stay close. ![Heatmap of split-sample robustness for the working parameter family. The color is the worse of the H1 and H2 raw gaps versus plain SPY. The best cells form a plateau around 25-40% OTM, DTE 90-180, exit DTE 30, and bi-monthly rebalance, rather than a single isolated point.](/images/charts/spitznagel-parameter-robustness.png) ### Rebalance frequency **Rebalancing** means closing existing put positions and buying new ones. Testing different frequencies at the 3.3% Spitznagel budget on the article's default configuration: | Frequency | Entries | Annual % | Excess % | Max DD % | Final wealth | |-----------|:-------:|:--------:|:--------:|:--------:|:------------:| | Weekly | 572 | 12.94 | +2.26 | -30.9 | $7.9M | | Biweekly | 340 | 13.16 | +2.48 | -26.6 | $8.2M | | Monthly | 179 | 14.38 | +3.70 | -31.1 | $9.8M | | **Bi-monthly** | **95** | **16.74** | **+6.07** | **-30.2** | **$13.9M** | | Quarterly | 66 | 14.41 | +3.74 | -39.0 | $9.9M | The bi-monthly cadence is decisively better than alternatives. The intuition: each rebalance is a round-trip transaction that crystallizes the put's current value into a new contract. More frequent rolling crystallizes more time decay before convex payoff has a chance to accumulate. Less frequent rolling (quarterly) leaves the portfolio unhedged for too long and forces missed entries. The bi-monthly window, buy at DTE 90-180, hold until DTE 30, then roll, captures the puts' valuable mid-life period and rolls just before final theta decay accelerates. ### Profit targets We tested whether selling puts at fixed-multiple gains ("monetize at 3x premium, 5x, 10x") improves on the rolling exit. Results at the 3.3% Spitznagel budget on the article's default configuration: | Profit target | Annual % | Raw gap vs SPY | Max DD % | |---------------|:--------:|:------:|:--------:| | **None (roll at DTE 30)** | **16.74** | **+6.07** | **-30.2** | | 20x premium | 11.76 | +1.08 | -45.4 | | 10x premium | 9.64 | -1.03 | -45.5 | | 5x premium | 10.01 | -0.67 | -49.2 | | 3x premium | 9.51 | -1.17 | -49.7 | Profit targets **hurt** the strategy decisively at this depth. The mechanism: at 40% OTM strike-based, a put that goes from "barely OTM" to "deep ITM" during a crash can multiply 50-100x. Capping at 3-10x premium realizes the early phase of the move and re-enters at the now-elevated IV, paying for the same exposure twice. The DTE-30 rolling exit captures the full upside without the round-trip premium drag, and also captures the put's residual time value rather than crystallizing it into a fresh contract at the post-crash IV. For shallower OTM bands (10-15% OTM) and smaller crash sizes, profit targets can help; at the 40% OTM depth this article uses, they don't. ### Macro signal timing We tested whether macro indicators could improve put timing: buy more puts when a crash seems likely, fewer when it doesn't. Signals tested include VIX (the market's "fear gauge"), GDP growth, high-yield credit spreads, the yield curve (10Y-2Y treasury spread, which [inverts before recessions](https://www.newyorkfed.org/research/capital_markets/ycfaq.html)), non-financial corporate equity, the dollar index, the [Buffett Indicator](https://en.wikipedia.org/wiki/Buffett_indicator) (market cap/GDP), and Tobin's Q. None of them improve put timing. The unconditional strategy (fixed budget, no signal) outperforms every signal-conditioned variant. The reason is that crash timing is inherently unpredictable. The VIX was low before both the 2008 crisis and COVID. The Buffett Indicator has been elevated for decades. Credit spreads were tight in early 2020. These signals contain information about risk levels but not about timing. The put strategy works precisely because it does not try to time crashes. It pays a small, steady cost for permanent protection. ### Out-of-sample validation A fair objection: 17 years with three crashes may overstate the long-run crash frequency. A 20-year period with no crashes would bleed premium with no payoff. We tested this from three angles, halves, quarters, and rolling 5-year windows, running the same default configuration (3.3%/yr budget, 40-45% OTM strike-based, DTE 90-180, exit DTE 30, bi-monthly roll) without re-optimizing across periods. **Halves (2008-2016 vs 2017-2024).** Both halves positive: | Half | SPY CAGR | Strategy CAGR | SPY DD | Strat DD | Excess | |---|---|---|---|---|---| | H1 2008-2016 (GFC + minis) | +7.18% | +15.50% | -51.9% | -30.2% | **+8.33pp** | | H2 2017-2024 (COVID + 2022 bear) | +14.66% | +17.99% | -33.7% | -28.3% | **+3.32pp** | The asymmetry (H1 +8.33 vs H2 +3.32) reflects that the GFC was a deeper drawdown than anything in H2. But both halves are positive, the strategy is not riding on a single black swan event. **Quarters (4-way split, ~4 years each).** Two of four quarters positive, the two containing major crashes: | Quarter | SPY ann | Strategy excess | Crash regime? | |---|---|---|---| | Q1 2008-2011 | −1.42% | **+22.15pp** | yes (GFC, peak SPY DD -52%) | | Q2 2012-2015 | +14.81% | −3.14pp | no | | Q3 2016-2019 | +14.75% | −2.66pp | no | | Q4 2020-2024 | +14.36% | **+7.06pp** | yes (COVID, peak SPY DD -34%) | The strategy gains heavily in the two quarters that contain a major drawdown and drags by 2-3pp/yr in the two that don't. The wins dominate the losses in magnitude. **Rolling 5-year windows (13 overlapping windows).** This is the most demanding test, the strategy is positive in 6 of 13 (46%), and every positive window contains a ≥25% SPY drawdown: | Window | SPY ann | Strategy excess | Crash in window? | |---|---|---|---| | 2008-2012 | +1.84% | **+17.20pp** | yes (SPY DD -52%) | | 2009-2013 | +17.18% | -3.05 | no (DD -27%, post-GFC tail) | | 2010-2014 | +14.99% | -3.13 | no | | 2011-2015 | +12.22% | -2.99 | no | | 2012-2016 | +14.24% | -3.09 | no | | 2013-2017 | +15.14% | -2.90 | no | | 2014-2018 | +8.59% | -2.51 | no | | 2015-2019 | +11.59% | -2.50 | no | | 2016-2020 | +15.46% | **+7.10** | yes (COVID) | | 2017-2021 | +18.21% | **+7.26** | yes | | 2018-2022 | +9.19% | **+6.79** | yes | | 2019-2023 | +15.62% | **+6.98** | yes | | 2020-2024 | +14.36% | **+7.06** | yes | The pattern is rule-like: if the rolling 5-year window contains a ≥25% SPY drawdown, the strategy adds roughly +7pp/yr (and +17pp/yr in the GFC window, which was -52%). If it doesn't, the strategy drags by roughly -3pp/yr. The 2008-2012 outlier on the upside reflects that the GFC was deeper than anything else in the sample. ![Bar chart of the strategy's raw gap versus plain SPY across 13 overlapping rolling 5-year windows, 2008-2024. Dark bars are windows that contained a ≥25% SPY drawdown, all positive, ranging from +6.79pp to +17.20pp. Beige bars are windows with no major drawdown, all negative, clustered around -3pp. The visual signature of regime-conditional convexity.](/images/charts/spitznagel-rolling-windows.png) **Walk-forward (optimize on H1, evaluate on H2 fresh).** The H1-optimal configuration (the one above, 40-45% OTM, DTE 90-180, bi-monthly) gives +8.33pp in-sample and **+3.32pp** out-of-sample. About half of the in-sample edge survives. Importantly, **45-50% OTM is an overfitting cliff**: those deeper configurations score even better on H1 (driven by GFC's -52% drawdown) but flip *negative* on H2 because COVID's -34% wasn't deep enough to hit those strikes. The 40% OTM ceiling is the practical maximum that generalizes out-of-sample. This is the honest story about tail hedging. **It is a regime-conditional trade.** When the market hands you a deep, prolonged drawdown, the strategy pays for years of premium in a single quarter. When it doesn't, you bleed by 2-3pp/yr against buy-and-hold. The unconditional historical raw gap versus plain SPY over 2008-2024 is positive (+6pp/yr at 3.3% budget) because two crashes sit inside a 17-year window; a forward expectation should assume **roughly half of that** if the next 17 years contain similar crash frequency, and **negative drag** if the next 17 years are unusually calm. What we can say is that the strategy is robust to parameter choice inside the working family (25-40% OTM, DTE 90-180 entry, exit DTE 30, monthly or bi-monthly roll), survives every split without re-optimization, and does not blow up in tail-light periods, it just stops adding value during them. That is the right shape for a hedge. ![Bar chart of the strategy's year-by-year raw gap versus plain SPY from 2008 to 2024 at the article's default 3.3%/yr budget. Two years dominate: 2008 gives about +90pp and 2020 gives about +57pp. Every other year is a small negative drag, the premium cost of carrying the puts through calm and bull years. The convex payoff structure is visible in a single image.](/images/charts/spitznagel-yearly-excess.png) ## Limitations and open questions **Capacity and execution.** Deep OTM puts have limited liquidity, especially during stress. Bid-ask spreads on SPY puts with delta below $-0.05$ can exceed 20% of the mid price. During the March 2020 crash, some deep OTM strikes had no bids at all for hours. A strategy that works at \$10M may not scale to \$10B. Universa manages this by trading across multiple markets and maintaining dealer relationships, but capacity constraints are real. **Financing source.** Where the put budget comes from matters. Our backtest treats it as an external cost. In practice, the premium could come from reducing equity exposure (no-leverage framing), from a separate cash allocation, or from an institutional budget line. The choice affects both the portfolio math and the behavioral likelihood of maintaining the strategy through long bleed periods. **Tax and turnover.** Monthly rolling generates 12 short-term capital loss events per year. In taxable accounts, the interaction between put losses, put gains during crashes, and equity capital gains creates complex tax consequences. This drag is absent from our backtest. **Regime dependence of skew pricing.** The volatility skew (how much more expensive OTM puts are relative to ATM options) varies over time. After 2008, skew steepened dramatically, and deep OTM puts became more expensive. If the market "learns" to price tail risk more accurately, the edge may compress. Conversely, long calm periods tend to flatten skew, making puts cheaper again. **Comparison with other tail hedges.** We only test put-based strategies. [Hurst, Ooi, and Pedersen (2017)](https://www.aqr.com/insights/research/white-papers/tail-risk-hedging-contrasting-put-and-trend-strategies) at AQR argue that trend-following (managed futures) provides crash protection more cheaply than puts because trend strategies *earn* a positive premium on average rather than bleeding one. A fair comparison would test both approaches on the same data. Our backtester currently does not support trend-following overlays. ## Future work: Beyond equities The SPY put strategy works, but it may not be the optimal application of Spitznagel's structure. The same logic (steady carry plus cheap convexity on extreme moves) applies wherever there is a reliable asymmetry between calm periods and crises. The best market depends on the regime. In a classic disinflationary recession, rates options may be superior. For portfolios already earning carry, FX may be the most natural fit. For institutions with access to OTC markets, credit can offer very strong crisis convexity. VIX is the most direct panic hedge, but often the hardest to own cheaply enough. Several markets exhibit structural tail properties that can be as strong as, or stronger than, equities: **Rates and rate futures.** Central banks tend to cut rates aggressively in crises. The Fed dropped rates from 5.25% to 0.25% during the 2008 crisis, and from 1.5% to 0% in two weeks during COVID. These moves are 10x to 20x larger than normal monthly rate changes. Rate options may underprice these panic-cut scenarios because standard models assume mean-reversion around stable levels. The trade would be: earn the risk-free rate (or hold short-term Treasuries), buy OTM calls on SOFR futures that pay off when rates collapse. The counterexample is stagflation: if inflation is high during a recession, central banks may not cut, and rate-based tail hedges would fail. This makes rates a conditional hedge rather than a universal one. **FX carry trades.** Currencies like AUD/JPY and MXN/JPY offer interest rate differentials of 3 to 5% annually. In stable times, carry traders collect this premium. When risk sentiment shifts, these positions unwind violently. The 2008 crisis saw AUD/JPY drop 40% in weeks. OTM puts on the high-yield currency may be systematically cheap relative to the crash risk because Gaussian models treat carry-trade unwinds as low-probability events. The carry itself could fund the protection. **Credit and CDS.** Investment-grade bonds earn a spread over Treasuries, but credit events are rare and clustered. The barbell structure here is: hold IG bonds for the spread, buy OTM protection on HY or IG CDS indices. The protection bleeds a small annual premium in calm markets. When credit stress hits, the payoff is convex: the 2008 crisis took IG CDS from 50bps to 250bps (5x) and HY CDS from 300bps to 2000bps (6.7x). CDS has a natural asymmetry similar to puts: bounded cost (the annual premium), unbounded upside in a credit crisis. **Volatility products.** Buying calls on the VIX is the most direct tail hedge. These options can be extremely convex: when volatility explodes, short-dated VIX calls can reprice very quickly. That is why they are so attractive during panics. But high convexity does not automatically mean a good trade. The key distinction is between **convexity** (how fast the payoff accelerates in a selloff) and **efficiency** (how much convexity you get for the premium you pay). The VIX itself trades around 12 to 15 in calm markets and can spike to 80+ during crashes (it hit 82.69 on March 16, 2020). At first glance, that makes VIX calls look like the perfect hedge. The confusion is that VIX options are not priced on spot VIX. They are priced on **VIX futures**. So even if spot VIX jumps from 15 to 80 intraday, the option payoff depends on how much the relevant VIX future moves, which is usually much less. This means the eye-catching spot spike overstates the actual option payout. Short-dated ATM or slightly OTM VIX calls can still have very strong convexity because they are sensitive to sharp near-term changes in implied volatility. But that convexity is usually expensive because it is obvious and heavily demanded by investors looking for crash insurance. On top of that, the VIX futures curve is often in **contango** in calm markets, which means forward volatility is already priced above spot. That carry drag makes long VIX exposure expensive to hold over time. So VIX calls are not weak for lack of convexity; they are often less efficient because the convexity is expensive, the payoff is filtered through the futures curve rather than spot VIX, and volatility mean-reverts quickly after the panic. Whether the remaining edge is still worth paying for is an empirical question. **Commodities.** Oil crashes during demand shocks, which tend to coincide with equity crashes: crude fell from \$145 to \$30 in 2008 and briefly went negative in April 2020. OTM puts on crude oil futures would pay off in exactly these scenarios. The directional thesis is weaker than rates or credit because supply shocks push oil the other way (up during crises like the 1973 embargo or 2022 Ukraine war). This makes crude a noisier hedge than the other markets listed here. **Cross-market diversification.** The strongest argument for testing multiple markets is not finding the single best hedge but combining several. Crises are correlated: when equities crash, rates get cut, carry trades unwind, credit spreads blow out, and the VIX spikes. The crash payoffs across markets are positively correlated, but the bleed costs are largely independent (rate option decay has nothing to do with FX option decay). A portfolio that spreads 0.5% of annual premium across four markets would bleed roughly the same total amount as concentrating in one, but the probability of at least one leg paying off in any given crisis is higher. This diversification of bleed with correlation of payoff is the multi-market version of Spitznagel's variance drain argument. If the question is "what is better than SPY puts?", the useful answer is to match the hedge to the portfolio and the regime: - If your core risk is an equity book and you want the simplest implementation, SPY puts remain the clean default. - If your main concern is a deflationary recession with aggressive central-bank cuts, rates options may be better. - If the portfolio already earns carry, FX options on carry trades may be the most natural extension because the carry can help fund the hedge. - If you are an institution with OTC access and real size, credit hedges may offer the best crisis convexity. - If you want the purest panic exposure and can tolerate rich pricing, VIX calls are the cleanest but often the least efficient. So the practical conclusion is not that one asset dominates, but that the "best" tail hedge depends on what you already own, what kind of crash you fear, and which markets you can actually trade cheaply and consistently. For most allocators, the strongest implementation is likely to start with equity puts, then diversify a small convexity budget across rates, FX, or credit only when the portfolio, regime, and execution capability justify it. Testing these alternatives requires different data (CME futures and options, CDS term structures, FX options, VIX futures and options) and modifications to the backtester. This is ongoing work. ## The one-line version Spitznagel is right. Externally-funded deep OTM puts, sized small, held through monthly cycles, with proceeds redeployed into equity at crash-day prices, compound at a higher rate than plain SPY across 17 years of real data and three real crashes. At every budget from 0.5%/yr to 10%/yr. With max drawdown 10 to 22 percentage points better than SPY. The thesis is not that the puts win every year. It is that over a long enough sample with realistic crash frequency, the rare large convex payoffs compound past the steady premium drag in between them. The data confirms it. AQR's published critique is internally consistent inside the configuration they actually test (near-ATM puts in an allocation-reducing framing). It just does not speak to Spitznagel's configuration. The two camps are not contradicting each other. They are testing different strategies and reaching the conclusions appropriate to each. The deeper reason this works is geometric, not statistical. Wealth compounds multiplicatively, the logarithm is concave, and Jensen's inequality says variance is a tax on long-run growth. A small payment that disproportionately truncates the left tail buys back some of that tax. The math is developed in the closing essay of this series, [Finance is geometry, and it all comes back to Jensen's inequality](@/series/leptokurtic/2026-03-01-at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/index.md), and it gives the principle a name beyond the SPY put trade. The trade is one instance of a much larger pattern: anywhere convexity is cheap relative to the realized fat-tailed behavior of the underlying, a small convex position can improve the compound path of a multiplicative process. Equity puts are one venue. Rates, FX, credit, and volatility products are others. The constraint on all of them is the same: pay only when the protection is cheap, and stay small enough that the premium drag in normal years is recoverable. ## Implementation Our backtester runs the strategy as a simple rules-based engine: each rebalance, close any open put position that has hit DTE 30 (or sooner if a daily check triggers), redeploy any freed cash into SPY at the current price, then buy a new put with strike between 55% and 60% of current SPY spot, DTE between 90 and 180 days, and total premium spend hitting the target annual budget rate. **Funding assumption**: the put budget is treated as an external, fixed annual premium (e.g., 3.3% of portfolio value per year) rather than being funded by selling SPY. This is the key distinction from AQR's setup. The fair benchmark is not plain SPY alone but SPY plus the same external capital source without the puts (e.g., SPY + 3.3% in cash). Since the premium is small relative to the equity book and cash earns the risk-free rate, the benchmark difference is roughly 0.1% per year at 3.3% budget and 4% cash rate, meaningful, but small relative to the strategy's +6pp raw gap versus plain SPY. The framing matters: the outperformance comes from the convexity of the puts, not from deploying more total capital. If you instead fund the put budget by reducing SPY, you recover the allocation-reducing framing, where the equity given up to fund the puts exceeds what the puts save, that is the configuration AQR actually tests, and it loses against SPY. **Methodology note**: no attempt is made to optimize timing; the strategy is purely rules-based. Real-world frictions (bid/ask spreads, slippage, and taxes) would reduce headline returns but should not remove the convexity effect. **Monetization**: when a put is closed (at DTE 30 or on a daily exit trigger), the cash from the sale is immediately redeployed into SPY at the current day's price. During a crash, this means the put's payoff buys SPY at crash-day prices. This is the "buy the dip with the hedge's proceeds" pattern Spitznagel and Bhansali describe as the core of the trade. The backtester implements this through the `rebalance_stocks_on_exit` flag combined with daily exit checks. Universa's actual implementation is more sophisticated. They manage rolls continuously to maintain their desired exposure profile. They hedge across multiple markets, not just the S&P 500. Those implementation choices may improve results, but they may also introduce capacity, execution, and timing frictions that this article does not model. The backtest should be read as a transparent rules-based reconstruction, not as a lower bound on Universa's live performance. ### Forward expectations: what to actually expect Every raw-gap number in the tables above is an in-sample number versus plain SPY. The walk-forward validation (in the Out-of-sample validation section above) shows that roughly half of the in-sample gap survives out-of-sample. A reader sizing this strategy for a real portfolio should halve every raw-gap column before forming expectations. | Budget | In-sample raw gap (article tables) | Forward expectation (halved) | Forward DD improvement | |---|---|---|---| | 0.5%/yr (Universa scale) | +1.39pp | **+0.7pp** | ~5-10pp better DD | | 1.0%/yr | +2.54pp | **+1.3pp** | ~10pp better DD | | 3.3%/yr (aggressive) | +6.07pp | **+3.0pp** | ~10-20pp better DD | The first column is what the backtest shows on this 17-year sample. The second is what a reader should expect going forward if the next 17 years contain similar crash frequency. If they contain less crash frequency, the forward number is lower; if more, higher. This is the geometric structure of catastrophe insurance, and it is not a flaw to be hedged against: it is the design that makes the strategy work at all. ## Code The backtester and the reproduction are both open source: - [github.com/lambdaclass/options_portfolio_backtester](https://github.com/lambdaclass/options_portfolio_backtester), the engine (Python + Rust core via PyO3 and Polars, ~10-50x speedups on hot paths) - [`docs/SPITZNAGEL_RECONSTRUCTION.md`](https://github.com/lambdaclass/options_portfolio_backtester/blob/master/docs/SPITZNAGEL_RECONSTRUCTION.md), the full technical reference for the configuration in this article, including the exact strategy spec, the budget sweep, halves/quarters/rolling-window cross-validation, the walk-forward results, the failure modes, and the modeling caveats - [`tests/oracles/test_article_reproduction.py`](https://github.com/lambdaclass/options_portfolio_backtester/blob/master/tests/oracles/test_article_reproduction.py), CI-pinned regression test for the article's headline numbers; pinned with tolerance 0.5pp on annual return, 1.0pp on max drawdown, 0.05 on Sharpe - [`research/spitznagel_spy/reproduce_article.py`](https://github.com/lambdaclass/options_portfolio_backtester/blob/master/research/spitznagel_spy/reproduce_article.py), single-command reproduction script that prints every table in this article **Reproducing every table in this article** is one command after the install: ```bash # 1. Clone the engine git clone https://github.com/lambdaclass/options_portfolio_backtester cd options_portfolio_backtester # 2. Install (Nix or Python venv, see README) nix develop # or: python -m venv .venv && source .venv/bin/activate && make install-dev # 3. Fetch the SPY data (~17 years, ~600 MB parquet, SHA-256 pinned) python scripts/fetch_data.py all --symbols SPY # 4. Reproduce every table in this article python research/spitznagel_spy/reproduce_article.py ``` The script prints each table in the article as it runs, in the same order, and finishes in roughly two minutes on a laptop. The numbers match this article within the regression tolerance set in `tests/oracles/test_article_reproduction.py`. The CI on the engine repo runs that test on every commit, so any future engine change that would move a published table fails the test and the article gets re-verified before the change merges. ## Disclaimer This article is research and educational material only. It is not financial advice, investment advice, or a recommendation to buy or sell any security or derivative. Past performance, whether backtested or live, does not guarantee future results. Options trading involves substantial risk of loss. The backtest results presented here are gross of transaction costs, taxes, and slippage, and may not be replicable in live trading. Consult a qualified financial advisor before making any investment decisions. ## References - Barro, R. (2006). [*Rare Disasters and Asset Markets in the Twentieth Century*](https://academic.oup.com/qje/article-abstract/121/3/823/1917876). Quarterly J. Economics, 121(3). - Bhansali, V. (2008). [*Tail Risk Management*](https://www.pm-research.com/content/iijpormgmt/34/4/68). Journal of Portfolio Management, 34(4). - Bhansali, V. (2014). *Tail Risk Hedging: Creating Robust Portfolios for Volatile Markets*. McGraw-Hill. - Bhansali, V. and Davis, J. (2010). *Offensive Risk Management: Can Tail Risk Hedging Be Profitable?* Working paper. - Bhansali, V., Chang, P., Holdom, J., and Rappaport, M. (2020). *Monetization Matters: Active Tail Risk Management and the Great Virus Crisis*. Working paper. - Bollerslev, T., Tauchen, G., and Zhou, H. (2009). [*Expected Stock Returns and Variance Risk Premia*](https://scholars.duke.edu/publication/732839). Review of Financial Studies, 22(11). - Bouchaud, J.-P., Iori, G., and Sornette, D. (1996). [*Real-World Options: Smile and Residual Risk*](https://www.cfm.com/wp-content/uploads/2022/12/237-1994-real-world-options-smile-and-residual-risk.pdf). Risk, 9(3). - Carr, P. and Wu, L. (2009). [*Variance Risk Premiums*](https://academic.oup.com/rfs/article-abstract/22/3/1311/1581057). Review of Financial Studies, 22(3). - Hurst, B., Ooi, Y. H., and Pedersen, L. H. (2017). [*Tail Risk Hedging: Contrasting Put and Trend Strategies*](https://www.aqr.com/insights/research/white-papers/tail-risk-hedging-contrasting-put-and-trend-strategies). AQR White Paper. - Israelov, R. (2019). [*Pathetic Protection: The Elusive Benefits of Protective Puts*](https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2934538). J. Alternative Investments, 21(3). - Kelly, B. and Jiang, H. (2014). [*Tail Risk and Asset Prices*](https://academic.oup.com/rfs/article/27/10/2841/1607080). Review of Financial Studies, 27(10). - Berger, A., Nielsen, L., and Villalon, D. (2011). [*Chasing Your Own Tail (Risk)*](https://www.aqr.com/-/media/AQR/Documents/Insights/White-Papers/AQR-Chasing-Your-Own-Tail-Risk.pdf). AQR White Paper. - Peters, O. (2019). [*The Ergodicity Problem in Economics*](https://www.nature.com/articles/s41567-019-0732-0). Nature Physics, 15. - Sornette, D. (2003). [*Why Stock Markets Crash*](https://press.princeton.edu/books/paperback/9780691175959/why-stock-markets-crash). Princeton University Press. - Spitznagel, M. (2021). [*Safe Haven: Investing for Financial Storms*](https://www.wiley.com/en-us/Safe+Haven%3A+Investing+for+Financial+Storms-p-9781119401797). Wiley. - Taleb, N. N. (2007). [*The Black Swan: The Impact of the Highly Improbable*](https://en.wikipedia.org/wiki/The_Black_Swan:_The_Impact_of_the_Highly_Improbable). Random House. - Taleb, N. N. (2012). [*Antifragile: Things That Gain from Disorder*](https://en.wikipedia.org/wiki/Antifragile_(book)). Random House. - Taleb, N. N. (2020). [*Statistical Consequences of Fat Tails*](https://arxiv.org/abs/2001.10488). STEM Academic Press. [^sp500_2007_2009]: The S&P 500 closed at 1,565.15 on October 9, 2007 and 676.53 on March 9, 2009, a 56.8% decline on closing prices. The SOA Research Brief Table 3 reports −59%, likely using intraday highs and lows. See [SOA Research Brief (Apr 16, 2020)](https://www.soa.org/globalassets/assets/files/resources/research-report/2020/2020-covid-19-research-brief-04-16.pdf). [^sp500_2020]: The same brief notes: "the S&P 500 cratered on March 23, down 34% from its February 19 level." See [SOA Research Brief (Apr 16, 2020)](https://www.soa.org/globalassets/assets/files/resources/research-report/2020/2020-covid-19-research-brief-04-16.pdf). [^universa_2020]: Bloomberg reports the fund "returned 3,612% in March" and that this came "according to an investor letter ... obtained by Bloomberg." See [Taleb-Advised Universa Tail Fund Returned 3,600% in March](https://www.bloomberg.com/news/articles/2020-04-08/taleb-advised-universa-tail-risk-fund-returned-3-600-in-march). --- ### Detecting Crashes with Fat-Tail Statistics *Published: 2026-02-19* > We built fatcrash, a Rust+Python toolkit with 15 crash detection methods: LPPLS, DFA, EVT, Hill, Kappa, Hurst, GSADF, momentum/reversal, price velocity, and more. Tested on 96 drawdowns across BTC, SPY, Gold, 23 forex pairs, and equity crises with honest precision/recall/F1 metrics. Plus: which methods transfer to revenue and profit data. URL: https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/ [Financial markets don't follow normal distributions](@/series/leptokurtic/2026-02-12-twenty-centuries-of-financial-data/index.md). That is a claim about frequency, not just theory: it tells you how often catastrophic events happen. Under a naive Gaussian model, a crisis on the scale of 2008 lands so deep in the tails that standard risk models treat it as effectively impossible. It happened on a Tuesday. The problem is that we keep using tools designed for thin-tailed worlds. **Value at Risk** (**VaR**) models that assume normality. Risk metrics that treat the 2008 crash as an "outlier" rather than a regular feature of financial returns. I built [fatcrash](https://github.com/unbalancedparentheses/fatcrash), a Rust+Python toolkit with 15 classical methods, to test whether fat-tail statistical methods can detect crashes before they happen. The performance-critical math (fitting, simulation, all rolling estimators) runs in Rust via PyO3; everything else (data, viz, CLI) is Python. {% toolkit(label="Key takeaways") %} - Markets are fat-tailed: events that Gaussian risk models call near-impossible (2008) happen on a regular Tuesday, so VaR and variance-based tools are fitting the wrong distribution. - I built fatcrash, an open-source toolkit of 15 classical methods (Rust core, Python shell), to test whether fat-tail statistics can flag crashes before they happen, validated on 39 drawdowns and 96 crash windows across forex and equities. - The best single detectors are regime and bubble methods: LPPLS for bubbles (about 89 to 90 percent recall) and DFA for regime shifts (82 percent); tail-index methods like Hill and Pickands have moderate recall and low precision and are best as ensemble components. - No single method is enough. A combined detector that fires only when methods agree reaches about 79 percent recall while filtering most false positives. - The deeper finding: variance-based finance (MPT, CAPM, Black-Scholes) is fragile here. 71 percent of countries land in the infinite-variance regime ($\alpha < 2$), hyperinflation is common rather than an outlier, and most of these methods transfer beyond prices to revenue and profit data. {% end %} ## What are fat tails? A **fat-tailed distribution** is one where extreme events happen far more often than a bell curve (Gaussian distribution) would predict. In a normal distribution, an event five standard deviations from the mean is essentially impossible, roughly a one-in-3.5-million chance. In a fat-tailed distribution, such events are uncommon but not rare. They show up regularly in financial data. The technical way to describe this is through the **tail index**, usually written $\alpha$. A fat-tailed distribution follows a **power law** in the extremes: the probability of a loss larger than $x$ decays as $P(X > x) \sim x^{-\alpha}$. The smaller the $\alpha$, the fatter the tail and the more likely extreme events are. Here is a rough guide to what different values of $\alpha$ mean: - $\alpha < 2$: **Infinite variance.** The distribution is so fat-tailed that the variance doesn't converge with more data. Standard statistics like standard deviation and correlation become unreliable. This is [Cauchy distribution](https://en.wikipedia.org/wiki/Cauchy_distribution) territory. - $\alpha$ between 2 and 4: **Finite variance but infinite kurtosis.** Kurtosis measures how "peaked" a distribution is and how heavy its tails are. When kurtosis is infinite, sample estimates of it are unstable and misleading. This is where most financial assets live. - $\alpha > 4$: **Relatively thin tails.** Still fatter than Gaussian, but manageable with conventional tools. The 15 classical methods in fatcrash fall into three groups: **bubble detection** (finding the specific price pattern that precedes a crash), **regime detection** (spotting shifts in how the market behaves over time, including momentum reversals and volatility cascades), and **tail estimation** (measuring how fat the tails actually are). Let's walk through each group. ## Bubble detection These methods look for structural patterns in prices, not statistics of returns. A bubble is a regime of super-exponential growth, prices rising faster and faster, that eventually becomes unsustainable. ### LPPLS: detecting bubbles before they burst The **Log-Periodic Power Law Singularity** model takes a fundamentally different approach from statistical methods. Instead of measuring properties of returns, it detects a specific pattern in prices: **the bubble signature**. The theory, developed by [Didier Sornette](https://en.wikipedia.org/wiki/Didier_Sornette) at ETH Zurich and described in his book [*Why Stock Markets Crash*](https://press.princeton.edu/books/paperback/9780691175959/why-stock-markets-crash), proposes that during a bubble, prices follow super-exponential growth decorated with accelerating oscillations that converge toward a **critical time** $t_c$, the most likely crash date. Think of it like a wine glass vibrating at increasing frequency before it shatters. $$\ln p(t) = A + B(t_c - t)^m + C(t_c - t)^m \cos(\omega \ln(t_c - t) + \phi)$$ In plain language: the logarithm of price is the sum of a smooth power-law growth (the $B$ term) and an oscillation whose frequency accelerates as you approach $t_c$ (the cosine term). The seven parameters encode specific dynamics: - $t_c$: critical time (when the bubble is most likely to end) - $m$: power law exponent (must be 0.1-0.9 for a valid bubble) - $\omega$: log-periodic frequency (must be 6-13) - $B < 0$: required, indicates super-exponential growth - $A, C, \phi$: amplitude and phase parameters Fitting this is computationally expensive. For each candidate $(t_c, m, \omega)$, the linear parameters $(A, B, C_1, C_2)$ are solved analytically via OLS. The nonlinear search uses a population-based stochastic optimizer over the 3D space. The **Sornette filter** rejects fits that don't satisfy the physical constraints. The **DS LPPLS confidence indicator** fits this model across many overlapping time windows. If a high fraction of windows produce valid bubble fits, confidence is high. **In practice**: With the tightened Nielsen (2024) filter (omega [6,13]) and a critical-time proximity constraint, LPPLS achieves 74% recall and 37% precision (F1=50%) across 39 drawdowns. It detects the bubble regime itself (super-exponential growth + oscillations), which precedes both small corrections and major crashes. The LPPLS confidence indicator (multi-window aggregation) reaches 90% recall but 29% precision. The high false positive rate is inherent: LPPLS frequently detects "bubble signatures" during normal bull markets because super-exponential growth patterns are common in trending markets. ### GSADF: explosive unit roots The **Generalized Sup ADF** (Augmented Dickey-Fuller) test, introduced by [Phillips, Shi, and Yu (2015)](https://onlinelibrary.wiley.com/doi/abs/10.1111/iere.12132), detects explosive behavior in prices. To understand it, you need one concept: a **unit root**. A time series has a unit root if it follows a random walk, meaning today's value is yesterday's value plus random noise, with no tendency to return to a long-run average. An **explosive** series goes further: it grows faster than a random walk, each day's value is a *multiple* of yesterday's, like compound interest gone wild. GSADF runs backward-expanding [ADF unit root tests](https://en.wikipedia.org/wiki/Augmented_Dickey%E2%80%93Fuller_test) across all possible start and end dates, taking the supremum (the largest test statistic). If this supremum exceeds Monte Carlo critical values, the series is explosive. GSADF is complementary to LPPLS. LPPLS detects the specific log-periodic oscillation pattern. GSADF detects any form of explosive growth, regardless of the oscillation structure. **In practice**: GSADF detected 38% of drawdowns overall, but 59% of medium-sized drawdowns (15-30%). It achieves 38% precision, the highest among all methods, because explosive unit root tests are more specific than distributional measures. ## Regime detection These methods look at how the *temporal structure* of returns changes over time. Before a crash, markets often shift from noisy, mean-reverting behavior to strong, persistent trending, a regime change that tail estimators can't see because they only measure distributional shape, not temporal dependence. ### DFA: detrended fluctuation analysis **Detrended Fluctuation Analysis**, introduced by [Peng et al. (1994)](https://en.wikipedia.org/wiki/Detrended_fluctuation_analysis), measures **long-range dependence** in non-stationary time series, that is, whether today's returns are correlated with returns from days or weeks ago. The method works by dividing the integrated series (cumulative sum of returns) into windows, fitting a local polynomial trend in each window, computing the root-mean-square residual (how much the data deviates from the local trend), and checking how that residual scales with window size: $$F(n) \sim n^{\alpha_{\text{DFA}}}$$ This formula says: the fluctuation $F$ at scale $n$ (window size) grows as a power of $n$. The **scaling exponent** $\alpha_{\text{DFA}}$ classifies the dynamics: - $\alpha_{\text{DFA}} = 0.5$: uncorrelated (random walk), no memory in the series - $\alpha_{\text{DFA}} > 0.5$: **persistent** (trends tend to continue), an up day makes another up day more likely - $\alpha_{\text{DFA}} < 0.5$: **anti-persistent** (mean-reverting), an up day makes a down day more likely Before a crash, markets often transition from mean-reverting to persistent dynamics. DFA picks up this regime shift. The key advantage over simpler methods is the **detrending step**: by removing local polynomial trends before measuring fluctuations, DFA separates genuine long-range dependence from spurious correlations caused by local trends. **In practice**: DFA was the best non-bubble crash detector in our tests (82% recall, 22% precision, F1=34%). It handles non-stationarity better than Hurst's R/S analysis because the detrending step removes local polynomial trends before measuring fluctuations. The low precision reflects the fact that persistent dynamics are common in financial markets even outside crash windows. ### Hurst exponent: persistence detection The **Hurst exponent**, introduced by [Harold Edwin Hurst](https://en.wikipedia.org/wiki/Hurst_exponent) in 1951 while studying Nile river flooding patterns, measures long-range dependence via **rescaled range (R/S) analysis**. For a time series of length $n$, compute the range of cumulative deviations from the mean, rescale by the standard deviation, and measure how $R/S$ scales with $n$: $$\frac{R}{S} \sim n^H$$ This formula says: the rescaled range grows as a power of the sample size. $H = 0.5$ is a random walk. $H > 0.5$ is persistent. $H < 0.5$ is mean-reverting. In many liquid return series, the long-run baseline is closer to $H \approx 0.5$, and R/S estimates can be biased upward in finite samples. A shift toward higher estimated $H$ before a crash means the market is trending more strongly, which often accompanies bubble formation. **In practice**: Hurst detected 59% of drawdowns. It is simpler than DFA but less robust to non-stationarity, because it doesn't remove local trends before measuring the range. ### Spectral exponent: frequency domain The **GPH log-periodogram regression**, introduced by [Geweke and Porter-Hudak (1983)](https://onlinelibrary.wiley.com/doi/abs/10.1111/j.1467-9892.1983.tb00371.x), estimates the **long-memory parameter** $d$ from the frequency domain. Instead of looking at how correlations decay over time (as DFA and Hurst do), it looks at how much power the signal has at different frequencies. The relationship to Hurst: $d = H - 0.5$. Positive $d$ indicates long memory (persistence). Think of it as measuring the same phenomenon (long-range dependence) but through a different lens: time domain vs. frequency domain. **In practice**: It detected 28% of drawdowns, comparable to Hill. The frequency-domain approach is theoretically elegant but doesn't add much beyond what DFA already captures for crash detection. ### Momentum and reversal **Momentum** is the tendency for assets that have been rising to keep rising, and assets that have been falling to keep falling. [Jegadeesh and Titman (1993)](https://doi.org/10.1111/j.1540-6261.1993.tb04702.x) documented this effect in equities: buying past winners and selling past losers produces positive returns over 3-12 month horizons. For crash detection, the signal is not momentum itself but its **reversal**. When long-term momentum is strongly positive (the asset has been trending up for months) but short-term momentum turns sharply negative (the last few weeks show a sudden decline), the divergence signals a potential crash, the trend is breaking. The reversal signal is computed as: $$\text{reversal} = \text{mom}_{\text{long}} - \text{mom}_{\text{short}}$$ where $\text{mom}(k) = \ln(P_t / P_{t-k})$ is the log-return over $k$ periods. A large positive reversal means the long-term trend is still up but the short-term move is down, exactly the pattern seen at the onset of major crashes, when a strong bull market suddenly reverses. [Scowcroft and Sefton (2005)](https://doi.org/10.1057/palgrave.jam.2240167) showed that momentum returns are partly compensation for crash risk: momentum strategies are profitable on average but suffer catastrophic losses during market reversals. This is the same asymmetry in reverse, the crash unwinds the positions that momentum built up. **In practice**: Momentum reversal is not used as a standalone detector (it doesn't have its own precision/recall row). Instead, it feeds into the combined detector as an independent signal. A large positive reversal, strong 12-month returns but negative 1-month returns, is a danger sign that the existing regime methods (DFA, Hurst) can't see because they measure temporal dependence, not price-level divergence. ### Price velocity: cascade detection **Price velocity** measures the rate of change of realized volatility, not how volatile the market is, but how fast volatility is *accelerating*. A sudden spike in volatility acceleration often signals a forced-liquidation cascade: margin calls triggering sales, which trigger more margin calls, which trigger more sales. $$\text{velocity} = \frac{\sigma_t - \sigma_{t-\text{lag}}}{\sigma_{t-\text{lag}}}$$ where $\sigma_t$ is the realized volatility (standard deviation of recent returns) and $\text{lag}$ is the lookback for the rate of change. When velocity exceeds a threshold, the market is in a self-reinforcing volatility spiral. The Feb 5, 2018 "Volmageddon" is the canonical example: the VIX doubled in a single day, triggering the liquidation of short-volatility ETNs (XIV, SVXY), which forced further VIX buying, which triggered more liquidations. The Sep 2019 repo rate spike followed a similar pattern in interest rate markets. In both cases, the price velocity, the acceleration of volatility, not its level, distinguished a mechanical cascade from ordinary high-vol conditions. **In practice**: Like momentum reversal, price velocity is not a standalone detector. It feeds into the combined detector as an independent signal. Its value is in catching a specific failure mode that other methods miss: the mechanical cascade, where the crash *causes itself* through forced selling. Tail estimators measure the distribution of returns; DFA and Hurst measure persistence; velocity measures the feedback loop in real time. ## Tail estimation These methods directly measure the fatness of the tails, how extreme the extremes really are. They answer questions like: "How often should we expect a 10% daily loss?" and "Is the variance of this distribution even finite?" ### Hill estimator: measuring tail heaviness The [Hill estimator](https://en.wikipedia.org/wiki/Hill_estimator) ([Hill, 1975](http://www.econ.uiuc.edu/~econ536/Papers/hill75.pdf)) is the most widely used tail index estimator. It fits a power law to the extreme values of a distribution and estimates the exponent $\alpha$. The estimator works by sorting the data from largest to smallest, taking the $k$ largest observations (called **order statistics**, which is just a fancy name for sorted values), and computing: $$\hat{\alpha} = k \left( \sum_{i=1}^{k} \ln \frac{X_{(i)}}{X_{(k)}} \right)^{-1}$$ where $X_{(1)} \geq X_{(2)} \geq \ldots$ are the order statistics. In words: take the $k$ biggest values, compute how far each is from the $k$-th largest (in log scale), average those distances, and invert. A small average log-gap means the extreme values are tightly packed (thin tail), and a large gap means they're spread out (fat tail). The choice of $k$ matters enormously. Too small and the estimate is noisy (not enough data points). Too large and you're including observations from the body of the distribution, not the tail. A **Hill plot** ($\alpha$ vs. $k$) helps find the plateau where the estimate stabilizes. **In practice**: Hill alpha is useful for characterization: it tells you *what kind of distribution you're dealing with*. But as a standalone crash predictor, it's noisy. In our tests, it only detected 28% of drawdowns. ### Kappa metrics: how far from Gaussian? Two metrics answer this question from different angles. **Taleb's kappa**, introduced in [*Statistical Consequences of Fat Tails*](https://arxiv.org/abs/2001.10488) by [Nassim Nicholas Taleb](https://en.wikipedia.org/wiki/Nassim_Nicholas_Taleb), measures how fast the **mean absolute deviation** (MAD), the average distance of observations from their mean, converges as you add more data. For well-behaved distributions, the MAD stabilizes quickly. For fat-tailed ones, it doesn't, because new extreme observations keep pulling the average around. The formula compares the MAD at two sample sizes $n_0$ and $n$: $$\kappa = 2 - \frac{\log n - \log n_0}{\log M(n) - \log M(n_0)}$$ where $M(n)$ is the MAD for $n$ summands. For a Gaussian, $\kappa = 0$ (fast convergence). For a [Cauchy distribution](https://en.wikipedia.org/wiki/Cauchy_distribution), $\kappa = 1$ (no convergence at all). Values between 0 and 1 measure the degree of fat-tailedness. **Max-stability kappa** takes a different approach rooted in [extreme value theory](https://en.wikipedia.org/wiki/Extreme_value_theory). The intuition: in a fat-tailed distribution, the single most extreme observation dominates everything. If you split your data into subsamples and find the maximum of each subsample, those subsample maxima will be much smaller than the overall maximum, because the one truly extreme value ended up in just one subsample. In a Gaussian distribution, the subsample maxima would be closer to the overall maximum. Formally: split your data into $n$ subsamples. Find the maximum of each subsample. Compare the mean of those maxima to the overall maximum: $$\kappa_{\text{max}} = \frac{\text{mean of subsample maxima}}{\text{overall maximum}}$$ For a Gaussian distribution, there is no simple $1/\sqrt{n}$ benchmark for this ratio. In practice, we estimate the Gaussian reference level by Monte Carlo simulation, because Gaussian maxima grow only logarithmically with sample size. For fat-tailed distributions, $\kappa_{\text{max}}$ falls below that simulated benchmark because extreme observations are *much* more extreme than what you'd see in any subsample. The ratio $\kappa_{\text{max}} / \text{benchmark}$ is the signal: - Near 1.0: behaves Gaussian - Below 0.8: significantly fat-tailed - Below 0.5: extremely fat-tailed, crisis regime fatcrash implements both variants. **In practice**: Max-stability kappa was the best tail-based method in our tests (49% overall detection rate). It's more robust than Hill because it doesn't depend on choosing $k$, and it directly benchmarks against Gaussian via Monte Carlo simulation. Taleb's kappa detected 33% of drawdowns but is more useful for long-term characterization than short-term prediction. ### Pickands estimator: domain-agnostic tail index The [Pickands estimator](https://en.wikipedia.org/wiki/Pickands%E2%80%93Balkema%E2%80%93de_Haan_theorem) ([Pickands, 1975](https://projecteuclid.org/journals/annals-of-statistics/volume-3/issue-1/Statistical-Inference-Using-Extreme-Order-Statistics/10.1214/aos/1176343003.full)) estimates the **extreme value index** $\gamma$ using just three order statistics, making it valid for all three domains of attraction (Frechet, Gumbel, Weibull, explained below in the EVT section): $$\hat{\gamma} = \frac{1}{\ln 2} \ln \frac{X_{(k)} - X_{(2k)}}{X_{(2k)} - X_{(4k)}}$$ In words: look at the gaps between the $k$-th, $2k$-th, and $4k$-th largest values. If the gap between the top values is much larger than the gap further down, the tail is fat ($\gamma > 0$). If the gaps are similar, the tail is exponential ($\gamma \approx 0$). If the top gap is smaller, the tail is bounded ($\gamma < 0$). Unlike Hill, which assumes the tail is Pareto (Frechet domain only), Pickands works regardless of the tail type. **In practice**: Pickands detected 49% of drawdowns, matching max-stability kappa. Its domain-agnostic nature makes it a useful cross-check on Hill. ### DEH moment estimator The **Dekkers-Einmahl-de Haan moment estimator** ([Dekkers, Einmahl, and de Haan, 1989](https://projecteuclid.org/journals/annals-of-statistics/volume-17/issue-4/A-Moment-Estimator-for-the-Index-of-an-Extreme-Value/10.1214/aos/1176347397.full)) uses first and second moments (averages and averages of squares) of log-spacings between order statistics. Like Pickands, it is valid for all domains of attraction, but it uses more data points from the tail, which makes it less volatile. **In practice**: It detected 46% of drawdowns. ### QQ estimator The **QQ estimator** computes the tail index from the slope of a log-log **QQ plot** (quantile-quantile plot) against exponential quantiles. A QQ plot compares the observed distribution against a theoretical one; if the points fall on a straight line, the distributions match. The slope of that line in log-log space gives you the tail index. **In practice**: It detected 38% of drawdowns. ### Maximum-to-Sum ratio The **Maximum-to-Sum ratio** is a direct diagnostic for whether extreme observations dominate the second moment. For $n$ observations, compute: $$R_n^{(2)} = \frac{\max(X_i^2)}{\sum(X_i^2)}$$ In words: what fraction of the total squared magnitude comes from the single largest observation? If one observation dominates the entire sum of squares, the second moment is not stabilizing. If $R_n^{(2)}$ stays bounded away from zero as $n$ grows, that is evidence against a well-behaved finite-variance regime. **In practice**: It detected 31% of drawdowns. ### EVT: quantifying worst-case scenarios **Extreme Value Theory** ([EVT](https://en.wikipedia.org/wiki/Extreme_value_theory)) is the standard mathematical framework for modeling tail risk. Instead of fitting a distribution to all the data (where the bulk dominates and extreme events are treated as noise), EVT focuses only on the extremes. Two complementary approaches: **GPD (Generalized Pareto Distribution)**: Pick a high threshold $u$ (say, losses worse than the 95th percentile). Fit the [GPD](https://en.wikipedia.org/wiki/Generalized_Pareto_distribution) to losses that exceed $u$. The [Pickands-Balkema-de Haan theorem](https://en.wikipedia.org/wiki/Pickands%E2%80%93Balkema%E2%80%93de_Haan_theorem) guarantees that for sufficiently high $u$, the exceedances follow a GPD regardless of the underlying distribution. The GPD has two parameters: scale ($\sigma$, how spread out the exceedances are) and shape ($\xi$, how fat the tail is). From these you get: $$\text{VaR}_p = u + \frac{\sigma}{\xi}\left[\left(\frac{n}{N_u}(1-p)\right)^{-\xi} - 1\right]$$ $$\text{ES}_p = \frac{\text{VaR}_p + \sigma - \xi u}{1 - \xi}$$ **VaR** (Value at Risk) tells you the loss you won't exceed with probability $p$. For example, a 99% VaR of 5% means that on 99% of days, you'll lose less than 5%. **ES** (**Expected Shortfall**, also called Conditional VaR) tells you the average loss *when you do* exceed VaR, answering the question "when things go badly, how bad do they get on average?" **GEV (Generalized Extreme Value)**: Instead of exceedances over a threshold, fit to **block maxima** (e.g., the worst loss each month). The [Fisher-Tippett-Gnedenko theorem](https://en.wikipedia.org/wiki/Fisher%E2%80%93Tippett%E2%80%93Gnedenko_theorem) guarantees that block maxima converge to a GEV distribution. The shape parameter $\xi$ tells you the tail type: - $\xi > 0$: **Frechet** (fat tail, power-law decay), typical for finance - $\xi \approx 0$: **Gumbel** (exponential tail) - $\xi < 0$: **Weibull** (bounded tail, there's a maximum possible value) **In practice**: GPD VaR detected 42% of drawdowns. It works well for medium corrections but struggles with major crashes because the pre-crash period is itself volatile, making the baseline VaR already elevated. ## Results on 39 drawdowns We tested all methods on 39 drawdowns across three assets (BTC, SPY, Gold). A **drawdown** is defined as a peak-to-trough decline in daily close; the pre-crash window is the 120 trading days before the peak, and the calm window is a similar period ending well before the peak. A method "detects" a crash if its signal during the pre-crash window is significantly elevated compared to the calm window. We also test each method on ~150 non-crash windows (50 per asset, sampled at least 180 days from any crash) to measure false positive rates. This gives us precision, recall, and F1, not just recall. The table below shows the 13 classical methods that have standalone precision/recall/F1 scores. Momentum reversal and price velocity are used as signals in the combined detector rather than standalone detectors. | Method | Precision | Recall | F1 | |--------|:---------:|:------:|:--:| | **LPPLS** | **37%** | **74%** | **50%** | | **LPPLS confidence** | **29%** | **90%** | **43%** | | GSADF | 38% | 38% | 38% | | **DFA** | **22%** | **82%** | **34%** | | Hurst | 19% | 59% | 28% | | Pickands | 19% | 49% | 27% | | Kappa | 19% | 49% | 27% | | DEH | 18% | 46% | 26% | | Spectral | 22% | 28% | 25% | | Taleb Kappa | 20% | 33% | 25% | | QQ | 16% | 38% | 23% | | GPD VaR | 12% | 42% | 19% | | Max-to-Sum | 12% | 31% | 18% | | Hill | 12% | 28% | 16% | **Precision** = how often a signal is correct (TP/(TP+FP)). **Recall** = how many crashes are caught (TP/(TP+FN)). **F1** = harmonic mean of both. **Why precision is low for tail/regime methods**: These methods detect distributional regime shifts (tail thickening, persistent dynamics), not crash-specific patterns. They fire in many non-crash periods because fat tails and persistence are pervasive in financial data. This is by design, they measure the distributional regime, not a specific crash. LPPLS and GSADF have higher precision because they detect bubble-specific structure. **The Sornette-Bouchaud debate on precision vs recall:** Sornette (the LPPLS inventor) argues that LPPLS is deliberately tuned for high recall at the cost of precision because the cost function is asymmetric, missing a crash is far more expensive than a false alarm. He calls false positives "failed predictions" and argues they are inevitable: bubbles can end in slow deflation rather than sharp crashes. His 2024 paper with Nielsen ([arXiv:2405.12803](https://arxiv.org/abs/2405.12803)) introduced the tightened omega [6,13] range specifically to improve precision without sacrificing recall. Bouchaud takes a more skeptical view. In his work at CFM and in papers with Potters, he emphasizes that fat-tail estimators (Hill, etc.) measure *unconditional* properties of returns and are poor at *conditional* crash prediction. His point is exactly what the data shows: tail estimators have decent recall but low precision because fat tails are always present, not just before crashes. He favors portfolio-level risk measures (drawdown control, volatility targeting) over point-in-time crash prediction. Both perspectives are reflected in fatcrash: LPPLS targets the mechanism (Sornette's approach), tail estimators measure the regime (which Bouchaud correctly notes is always fat-tailed), and the aggregator combines both, using Sornette-style bubble detection as the primary signal and Bouchaud-style regime measurement as confirmation. Recall by crash size shows that LPPLS confidence catches 93% of small, 94% of medium, and 75% of major crashes. DFA catches 86% of small and 88% of medium crashes. ### Major known crashes Testing on four major crashes with pre-crash vs. calm period comparison: | Crash | Kappa | GPD VaR | LPPLS | Hill | |-------|:---:|:---:|:---:|:---:| | 2017 BTC Bubble | detected | detected | detected | missed | | 2021 BTC Crash | detected | detected | detected | missed | | 2008 Financial Crisis | detected | detected | detected | detected | | COVID Crash 2020 | detected | — | detected | missed | Kappa and LPPLS each detected all four. GPD VaR detected 3 of 4. Hill detected 1 of 4. These are recall numbers, false positive rates are reported in the table above. ### Why Hill underperforms Hill measures the tail index of the *return distribution*, but this property changes slowly. A 6-month pre-crash window doesn't necessarily have thinner tails than a 6-month calm window because the calm window might include its own mini-shocks. The Hill estimator is useful for long-term characterization (this asset has $\alpha=3$, that one has $\alpha=4$) but not for short-term prediction. ### Why LPPLS leads on F1 LPPLS detects *structure*, not statistics. It's looking for a specific pattern: accelerating growth with log-periodic oscillations. This pattern appears before both 10% corrections and 80% crashes. The tail-based methods need to see the tail *thickening*, which requires the crash to already be underway. LPPLS sees the bubble building. With the tightened Nielsen (2024) filter (omega restricted to [6,13] instead of the original loose [2,25], plus a critical-time proximity constraint requiring tc to fall within 40% of the window length after the end), LPPLS achieves the best F1 score (50%) by balancing recall (74%) and precision (37%). The LPPLS confidence indicator trades precision (29%) for higher recall (90%) by aggregating across many sub-windows. The relatively low precision is inherent: LPPLS frequently detects "bubble signatures" during normal bull markets because super-exponential growth patterns are common. The solution is combining it with the tail-based and regime methods. If LPPLS says "bubble" and kappa says "tails thickening" and DFA shows persistent dynamics, the signal is more reliable. ### Why DFA is the best non-bubble method DFA detects regime shifts in the correlation structure of returns. Before a crash, markets transition from noisy mean-reverting behavior to strongly persistent trending. This transition is invisible to tail estimators like Hill or kappa, which measure distributional shape. DFA measures temporal dependence. The detrending step gives DFA an edge over Hurst's R/S analysis (82% recall vs 59%) because raw R/S conflates local trends with long-range dependence. DFA strips out the local trends and measures the residual scaling. DFA's 82% recall is high but its 22% precision means it also fires in many non-crash periods, persistent dynamics are common in financial markets. DFA is particularly strong on small and medium drawdowns (86% and 88% recall), where tail-based methods struggle because the distributional shift is subtle. ### Combined detector For the combined detector, signals are grouped into four independent categories: **bubble** (LPPLS and GSADF), **tail** (Hill, Pickands, DEH, QQ, Max-to-Sum, Taleb Kappa, Max-Stability Kappa, GPD VaR), **regime** (DFA, Hurst, Spectral, momentum reversal), and **structure** (multiscale agreement across daily, 3-day, and weekly frequencies, LPPLS critical time proximity, and price velocity). When three or more categories independently signal elevated risk, the combined detector applies a +15% bonus to the crash probability. | | Small (<15%) | Medium (15-30%) | Major (>30%) | Overall | |--|:---:|:---:|:---:|:---:| | Combined (agreement bonus) | 64% | 94% | 75% | **79%** | The combined detector reaches 79% overall, with 94% on medium-sized drawdowns. The agreement requirement filters out most of LPPLS's false positives while retaining most of its true positives. The gap between small (64%) and medium/major (94%/75%) drawdowns reflects the fact that small corrections often happen without prior tail thickening or regime change. They are genuine surprises, and no method should be expected to predict all of them. ## Long timescales ### 54 years of forex (GBP/USD) We tested on GBP/USD daily data from 1971 to 2025 (13,791 trading days): | Decade | Hill alpha | Kappa/benchmark | Notable | |--------|:---------:|:---------------:|---------| | 1970s | 2.92 | 0.78 | Oil crises, IMF bailout | | 1980s | 4.36 | 0.68 | Plaza Accord | | 1990s | 4.51 | 0.83 | Black Wednesday | | 2000s | 2.90 | 0.57 | 2008 crisis | | 2010s | 3.86 | 0.35 | Brexit | | 2020s | 3.39 | 0.71 | Truss mini-budget | Every decade shows fat tails. In our labeling, all six GBP/USD crisis events were detected (6/6): - 1976 IMF Crisis - 1985 Plaza Accord - 1992 Black Wednesday - 2008 Financial Crisis - 2016 Brexit Vote - 2022 Truss Mini-Budget ### All methods on daily forex (1971-2025) We ran all methods on 23 currency pairs from FRED daily data. The table shows the key estimators from each category, tail (Hill, QQ, DEH), regime (Hurst, DFA), and bubble (GSADF): | Pair | Hill $\alpha$ | QQ $\alpha$ | DEH $\gamma$ | Hurst $H$ | DFA $\alpha$ | GSADF | |------|:---:|:---:|:---:|:---:|:---:|:---:| | VEF/USD | 1.20 | 0.82 | 1.06 | 0.53 | 0.82 | bubble | | HKD/USD | 1.73 | 2.12 | 0.24 | 0.54 | 0.62 | bubble | | KRW/USD | 1.90 | 1.93 | 0.44 | 0.67 | 0.60 | bubble | | MXN/USD | 2.04 | 1.98 | 0.44 | 0.56 | 0.57 | bubble | | LKR/USD | 2.14 | 1.97 | 0.51 | 0.58 | 0.66 | bubble | | TWD/USD | 2.31 | 2.62 | 0.21 | 0.60 | 0.63 | bubble | | THB/USD | 2.38 | 2.43 | 0.33 | 0.58 | 0.59 | bubble | | MYR/USD | 2.42 | 2.46 | 0.33 | 0.58 | 0.60 | bubble | | AUD/USD | 2.58 | 2.30 | 0.44 | 0.56 | 0.56 | bubble | | INR/USD | 2.62 | 2.56 | 0.34 | 0.57 | 0.58 | bubble | | CNY/USD | 2.79 | 1.70 | 0.69 | 0.59 | 0.71 | bubble | | BRL/USD | 2.80 | 3.12 | 0.15 | 0.56 | 0.58 | bubble | | NZD/USD | 2.89 | 2.46 | 0.44 | 0.57 | 0.56 | — | | ZAR/USD | 3.19 | 3.43 | 0.15 | 0.58 | 0.54 | bubble | | NOK/USD | 3.39 | 3.44 | 0.22 | 0.57 | 0.53 | bubble | | SEK/USD | 3.50 | 2.88 | 0.41 | 0.58 | 0.55 | — | | SGD/USD | 3.59 | 3.66 | 0.18 | 0.56 | 0.53 | bubble | | CHF/USD | 3.81 | 3.59 | 0.28 | 0.57 | 0.54 | — | | CAD/USD | 3.84 | 3.58 | 0.27 | 0.57 | 0.53 | bubble | | DKK/USD | 3.84 | 3.23 | 0.37 | 0.58 | 0.55 | — | | JPY/USD | 3.94 | 4.02 | 0.18 | 0.58 | 0.58 | bubble | | GBP/USD | 4.13 | 4.11 | 0.19 | 0.58 | 0.55 | bubble | | EUR/USD | 4.88 | 4.90 | 0.12 | 0.56 | 0.54 | — | All 23 pairs show fat tails: DEH $\gamma > 0$ for 23/23, and two independent tail index estimators converge (mean Hill $\alpha$ = 2.95, mean QQ $\alpha$ = 2.84). Under these estimators, all 23 also show persistence-leaning signatures: Hurst $H > 0.5$ and DFA $\alpha > 0.5$ for every pair. GSADF detected explosive episodes in 18 of 23 pairs. VEF/USD (Venezuela) is the extreme case: Hill $\alpha$ = 1.20, QQ $\alpha$ = 0.82, DEH $\gamma$ = 1.06, every estimator confirms infinite variance. At the other end, EUR/USD has the thinnest tails (Hill $\alpha$ = 4.88) but is still fat-tailed by any standard. KRW/USD has the fattest tails among liquid pairs (Hill $\alpha$ = 1.90). CNY/USD shows the strongest persistence (DFA = 0.71), consistent with managed float dynamics. ### 500 years, 138 countries Using the [Clio Infra exchange rate dataset](https://clio-infra.eu/) (1500-2013), we ran all tail and regime methods on every country with 50+ years of data. Results: | Tail regime | Countries | Percentage | |------------|:---------:|:----------:| | $\alpha < 2$ (Hill estimate in the infinite-variance regime) | 98 | **71%** | | $\alpha$ 2-4 (fat tails, finite variance) | 37 | 27% | | $\alpha > 4$ (moderate tails) | 3 | 2% | **71% of countries land in the Hill-estimated $\alpha < 2$ regime.** The median $\alpha$ across all 138 countries is 1.57. This is strong evidence that variance is unstable or poorly estimated for a large share of the sample. GEV points in the same direction: 81% of countries show Frechet-type (fat) tails with median $\xi = 0.76$. The most extreme cases: Returns in this table are log-returns, which can exceed -100% or +100%. A log-return of -2,748% means the currency lost virtually all its value (the price ratio $e^{-27.48} \approx 0$). This convention is standard in fat-tail analysis because log-returns are additive across time periods. | Country | Years of data | Hill $\alpha$ | Taleb $\kappa$ | Worst year | Best year | |---------|:---:|:---:|:---:|:---:|:---:| | Syria | 61 | 0.32 | — | -2% | +105% | | Iraq | 61 | 0.40 | — | -38% | +883% | | Germany | 153 | 0.52 | 1.00 | **-2,748%** | +2,104% | | Nicaragua | 76 | 0.52 | — | -2,159% | +787% | | Zimbabwe | 56 | 0.55 | — | -12% | +1,345% | | Hungary | 66 | 0.56 | — | -944% | +247% | | Peru | 64 | 0.72 | — | -1,660% | +426% | | Bolivia | 63 | 0.76 | — | -1,794% | +494% | | Brazil | 129 | 1.03 | — | **-3,536%** | +318% | | Argentina | 102 | 1.28 | 1.00 | -2,748% | +388% | Germany's -2,748% in a single year is the Weimar hyperinflation. Brazil's -3,536% reflects the cruzeiro collapse. These aren't outliers, they're exactly what a distribution with $\alpha < 1$ predicts. Running all classical methods on the top 30 countries by data length confirms that fat tails and persistence go hand in hand: | Country | Years | Hill $\alpha$ | Hurst $H$ | Taleb $\kappa$ | Verdict | |---------|:---:|:---:|:---:|:---:|---------| | Germany | 153 | 0.52 | 0.56 | 1.00 | extreme, persistent | | Austria | 104 | 0.63 | 0.61 | 1.00 | extreme, persistent | | Belgium | 114 | 0.89 | 0.64 | 0.86 | extreme, persistent | | Finland | 100 | 0.94 | 0.58 | 0.43 | extreme, persistent | | Italy | 95 | 0.77 | 0.80 | 0.95 | extreme, persistent | | Portugal | 88 | 0.98 | 0.85 | 1.00 | extreme, persistent | | Greece | 87 | 0.77 | 0.76 | 0.81 | extreme, persistent | | Argentina | 102 | 1.28 | 0.71 | 1.00 | extreme, persistent | | Mexico | 113 | 1.06 | 0.70 | 0.92 | extreme, persistent | | UK | 223 | 2.42 | 0.47 | 0.04 | fat-tailed | | Canada | 100 | 3.70 | 0.50 | 0.00 | fat-tailed | Of the top 30 countries, 19 have $\alpha < 2$ (infinite variance), 25 have Hurst $H > 0.5$ (persistent dynamics), and 28 have QQ $\alpha < 4$ (heavy tails confirmed by multiple estimators). Germany, Austria, Argentina, and Portugal saturate at Taleb $\kappa = 1.0$, Cauchy-like behavior where the CLT does not operate at any practical sample size. Italy ($H$ = 0.80) and Portugal ($H$ = 0.85) show the strongest persistence over century-scale data. #### Century-by-century: United Kingdom (1789-2013) The UK has 224 years of continuous exchange rate data: | Century | Hill $\alpha$ | Regime | |---------|:---:|-----------| | 1800s | 1.19 | Infinite variance (Napoleonic wars, banking crises) | | 1900s | 3.17 | Fat but finite (Bretton Woods stability) | | 2000s | 2.04 | Back to borderline infinite variance | Even within a single country, tail regimes shift across centuries. ### Inflation: 500 years, 82 countries Inflation data from Clio Infra (1500-2010): | Statistic | Value | |-----------|-------| | Countries analyzed | 82 | | Countries with hyperinflation (>100%/yr) | **32** (39%) | | Countries with $\alpha < 2$ | **36** (44%) | | Median $\alpha$ | 2.14 | The most extreme inflation tails: | Country | Years | Hill $\alpha$ | Max inflation | |---------|:-----:|:---:|:---:| | Nicaragua | 71 | 0.30 | 13,110%/yr | | Zimbabwe | 83 | 0.44 | 24,411%/yr | | Germany | **494** | 0.57 | **211,427,400,000%/yr** | | Brazil | 226 | 0.63 | 2,948%/yr | | Peru | 363 | 0.80 | 7,482%/yr | | Argentina | 274 | 0.85 | 3,079%/yr | | China | 336 | 0.86 | 1,579%/yr | | Poland | 414 | 0.97 | 4,738%/yr | Germany has 494 years of inflation data with $\alpha = 0.57$. Its maximum annual inflation was 211 billion percent (Weimar 1923). With $\alpha < 1$, neither the mean nor the variance of this distribution converges. You cannot compute a confidence interval. You cannot build a VaR model. The standard toolkit breaks down. ## Extended validation: 96 crash windows across forex and equities The original 39-drawdown evaluation used only BTC, SPY, and Gold. To test whether the results generalize, we extended the dataset with 23 FRED daily forex pairs (1971-2025) and 6 equity files covering the 2008, 2020, and 2022 crises. This yielded 96 total crash windows and 631 non-crash windows. LPPLS recall held at 89% on the extended dataset and 90% combined. The precision and F1 patterns remained stable: LPPLS leads on recall, GSADF leads on precision, DFA is the best non-bubble method. The combined detector's agreement bonus continues to filter false positives effectively. The forex pairs confirmed that fat tails and persistence are universal: Hill $\alpha < 4$ for all 23 pairs, Hurst $H > 0.5$ for all 23. EM currency pairs (BRL, MXN, KRW, ZAR) show the fattest tails, as expected from their crisis histories. ## Beyond market prices: detecting problems in revenue and profit data These methods were built for market prices, but most transfer to any time series where you need to detect structural problems, company revenue, profit margins, unit economics, or any financial metric that changes over time. The key distinction: **market prices** reflect collective speculative behavior (herding, positive feedback loops), while **revenue and profit** reflect real economic activity (customer demand, operational execution, competitive dynamics). ### What transfers to company data **Tail estimation (Hill, DEH, QQ, Pickands, Kappa, Max-to-Sum, GPD/GEV)**, Yes. Revenue growth rates have fat tails. [Gabaix (2011)](https://academic.oup.com/qje/article/126/1/185/1903368) showed that idiosyncratic firm-level shocks drive aggregate fluctuations precisely because firm-size distributions are fat-tailed. A company with Hill $\alpha < 2$ on quarterly revenue growth has a distribution where a single catastrophic quarter can dominate the entire history. EVT gives you calibrated worst-case scenarios: fit GPD to the worst quarterly declines for a valid tail risk estimate. **Persistence detection (DFA, Hurst, Spectral)**, Yes. Revenue series often show strong persistence ($H > 0.5$) due to contracts, recurring revenue, and customer stickiness. A shift from persistent ($H > 0.5$) to anti-persistent ($H < 0.5$) could signal fundamental deterioration, the business is losing its growth momentum. DFA handles the non-stationarity inherent in growing companies better than raw Hurst. **GSADF**, Partially. It detects unsustainable exponential growth. Applied to revenue, it could flag "growth bubbles", growth rates that would require capturing 100% of the addressable market to sustain. Useful for evaluating whether a company's growth trajectory is explosive (and therefore unsustainable) or merely strong. **Momentum and velocity**, Partially. Revenue momentum (trailing growth rates) is meaningful for company analysis. A reversal in revenue momentum, strong long-term growth suddenly decelerating, is a classic warning sign. Price velocity (volatility acceleration) is less directly applicable to revenue data, which doesn't exhibit the forced-liquidation cascades it was designed to detect. **LPPLS**, No. It models speculative bubble dynamics: herding, log-periodic oscillations, reflexive feedback loops. Revenue doesn't exhibit these patterns. It's driven by real economic activity, not reflexive speculation. Don't apply LPPLS to your quarterly revenue. ### Practical example For a company's quarterly revenue time series: ```python import numpy as np from fatcrash._core import hill_estimator, dfa_exponent, hurst_exponent # Quarterly revenue growth rates growth = np.diff(np.log(quarterly_revenue)) hill_estimator(growth) # Tail index, are revenue shocks fat-tailed? dfa_exponent(growth) # Persistence, is growth momentum persistent or fading? hurst_exponent(growth) # Same question, different method (cross-check) ``` The practical challenge: quarterly data gives roughly 80 observations over 20 years (vs. 5,000+ daily prices). Tail estimators need at least 100 data points to be reliable. Use monthly revenue or longer history when possible. For shorter series, DFA and Hurst are more robust than Hill because they measure temporal structure rather than distributional shape. **What to watch for:** - **Hill $\alpha$ dropping below 3**: Revenue shocks are getting more extreme. The distribution is shifting toward heavier tails. - **DFA shifting from > 0.5 to < 0.5**: Growth momentum is breaking down. Revenue used to be self-reinforcing; now it's mean-reverting. - **Max-to-Sum ratio rising**: A single quarter is starting to dominate the entire history, either a massive win or a massive loss. - **GPD VaR spiking**: The worst-case quarterly decline is getting worse, even accounting for the fat tails. These methods won't tell you *why* revenue is deteriorating, you still need business context for that. But they can tell you *that* something structural has changed in the data before it becomes obvious in the headline numbers. ## Conclusions 1. **Fat tails are universal in this dataset.** Every asset class and timescale we tested shows materially heavier tails than a Gaussian baseline. In the long-horizon exchange-rate panel, 71% of countries land in the Hill-estimated $\alpha < 2$ regime. That makes variance-based summaries far less reliable than standard finance usually assumes. 2. **LPPLS has the best F1 score (50%)** because it detects bubble *structure*, not tail *statistics*. With tightened filters (Nielsen omega [6,13], tc constraint), it achieves 74% recall and 37% precision. The LPPLS confidence indicator trades precision (29%) for recall (90%). Both have substantial false positive rates, bubble signatures appear during normal bull markets too. 3. **DFA is the best non-bubble method (82% recall, F1=34%).** It detects regime shifts in temporal dependence, not distributional shape. The detrending step makes it robust to non-stationarity where simpler methods like Hurst (59% recall) are confused by local trends. Low precision (22%) reflects that persistent dynamics are common in financial markets. 4. **Tail-based methods have moderate recall but low precision.** Kappa and Pickands (49% recall each), DEH (46%), Hill (28%). These methods detect distributional regime shifts that are pervasive in financial data, not crash-specific patterns. They are most valuable as ensemble components, not standalone detectors. 5. **The combined detector reaches 79% recall.** When bubble, tail, regime, and structural methods independently agree, the signal is more reliable. The agreement requirement filters most false positives while preserving 94% detection on medium-sized drawdowns. Momentum reversal and price velocity add structural signals that capture crash dynamics invisible to distributional methods, trend breaks and forced-liquidation cascades, respectively. No single method is sufficient. These results were validated on an extended dataset of 96 crash windows across 23 forex pairs and equity crises, with LPPLS recall stable at 89-90%. 6. **Standard variance-based risk models are badly stressed by this data.** [Modern Portfolio Theory](https://en.wikipedia.org/wiki/Modern_portfolio_theory), [CAPM](https://en.wikipedia.org/wiki/Capital_asset_pricing_model), and Black-Scholes-style thinking rely heavily on stable variance estimates. In the long-horizon currency panel, 71% of countries land in the Hill-estimated $\alpha < 2$ regime, which means those variance inputs are often fragile, unstable, or badly misspecified. 7. **Hyperinflation isn't rare.** 39% of countries experienced >100% annual inflation at some point. Germany's 211 billion percent is extreme, but dozens of countries experienced four- and five-digit inflation. Any model that treats these as "outliers" is a model that doesn't understand the data it's modeling. 8. **Most methods transfer beyond market prices.** Tail estimation, persistence detection, momentum, and EVT work on any time series, revenue, profit, unit economics. LPPLS doesn't transfer (it models speculative dynamics), but Hill, DFA, Hurst, momentum reversal, and GPD work on company-level data. The challenge is sample size: quarterly data gives ~80 observations vs. 5,000+ daily prices. Use monthly data when possible. Beyond crash detection, fatcrash includes two portfolio-level tools. **Constant volatility targeting** ([Hallerbach, 2012](https://doi.org/10.2139/ssrn.2042750)) sizes positions inversely to realized volatility: when vol spikes, reduce exposure; when vol is low, increase it. This is the Bouchaud-school response to crash risk, don't predict crashes, just mechanically reduce exposure when the market gets rough. The **rebalance risk signal** ([Rattray, Granger, Harvey, and Van Hemert, 2020](https://doi.org/10.3905/jpm.2020.1.131)) addresses a subtler problem: mechanical rebalancing (buying stocks after they fall to maintain a target allocation) is negative convexity. During persistent drawdowns where DFA shows trending dynamics and momentum is negative, "buying the dip" amplifies losses. The signal combines DFA persistence with momentum direction to warn when rebalancing is dangerous. Tail hedging via deep OTM puts is the convex alternative to these mechanical responses; I cover that debate in [The Tail Hedge Debate: Spitznagel Is Right](@/series/leptokurtic/2026-02-26-the-tail-hedge-debate-spitznagel-is-right/index.md). The code is open source: [github.com/unbalancedparentheses/fatcrash](https://github.com/unbalancedparentheses/fatcrash). The forex data comes from [forex-centuries](https://github.com/unbalancedparentheses/forex-centuries). ## References - Jegadeesh, N. and Titman, S. (1993). [*Returns to Buying Winners and Selling Losers: Implications for Stock Market Efficiency*](https://doi.org/10.1111/j.1540-6261.1993.tb04702.x). Journal of Finance, 48(1). - Scowcroft, A. and Sefton, J. (2005). [*Understanding Momentum*](https://doi.org/10.1057/palgrave.jam.2240167). Journal of Asset Management, 6(3). - Hallerbach, W. (2012). [*A Proof of the Optimality of Volatility Weighting over Time*](https://doi.org/10.2139/ssrn.2042750). Journal of Investment Strategies, 1(4). - Rattray, S., Granger, N., Harvey, C. R., and Van Hemert, O. (2020). [*Strategic Rebalancing*](https://doi.org/10.3905/jpm.2020.1.131). Journal of Portfolio Management, 46(6). - Jordà, Ò., Schularick, M. and Taylor, A. M. (2019). [*The Rate of Return on Everything, 1870-2015*](https://doi.org/10.1093/qje/qjy004). Quarterly Journal of Economics, 134(3). - Nielsen, J., Sornette, D., and Raissi, M. (2024). [*Deep LPPLS*](https://arxiv.org/abs/2405.12803). arXiv:2405.12803. ## Disclaimer This article is for educational and research purposes only. Nothing here constitutes financial advice, trading recommendations, or an invitation to buy or sell any asset. The detection rates reported are retrospective, based on labeled historical events, and should not be interpreted as predictive accuracy for future markets. Always consult a qualified financial advisor before making investment decisions. --- ### Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal *Published: 2026-02-12* > We assembled forex-centuries, an open dataset of exchange rates, gold, silver, interest rates, commodity prices, GDP, sovereign debt, and more, across 27 sources spanning 1 CE to 2026 and covering 240 countries. Fat tails are universal. Pegged currencies are the most dangerous. Every currency loses against gold. URL: https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/ In 1252, Florence minted the gold florin. Within decades it became the dominant trade currency of medieval Europe. Merchants in Bruges, Venice, and Constantinople quoted prices against it. By the 1400s, the florin's dominance had faded, replaced by the Venetian ducat. Then the Spanish real. Then the Dutch guilder. Then sterling. Then the dollar. Each transition involved devaluations, defaults, and crises that ruined anyone holding the wrong currency at the wrong time. We have data on all of this. Not estimates. Actual recorded exchange rates, starting from 1106. And not just exchange rates: gold and silver prices from 1257, interest rates from 1311, commodity prices from 1260, GDP per capita from the year 1 CE, sovereign debt ratios from 1800, and crisis indices covering two centuries of banking panics, currency collapses, and sovereign defaults. I assembled [forex-centuries](https://github.com/unbalancedparentheses/forex-centuries), the most comprehensive open-source collection of long-run financial and economic data available. 27 sources, 1,100+ files, ~240 countries, spanning twenty centuries. Exchange rates, precious metals, interest rates, commodity prices, inflation, GDP, real wages, sovereign debt, regime classifications, and real effective exchange rates, all in one repository with an automated build pipeline, weekly CI updates, and reproducible analysis. No other free repository combines this breadth of asset classes across this depth of history. The only comparable product is [Global Financial Data](https://globalfinancialdata.com/) (commercial, institutional pricing). The goal: provide the raw material for studying how currencies and financial systems behave over centuries, not decades. {% toolkit(label="Key takeaways") %} - forex-centuries assembles the most comprehensive open-source long-run financial dataset available: 27 sources, 1,100+ files, about 240 countries, twenty centuries, with exchange rates from 1106 and GDP per capita from the year 1. - Reserve-currency dominance is always temporary. The baton passed from the florin to the ducat, the real, the guilder, sterling, and the dollar, and every handoff was paid for with devaluations, defaults, and crises. - Long-run currency returns are wildly fat-tailed, with excess kurtosis in the thousands and many currencies in the infinite-variance regime ($\alpha < 2$), so Gaussian volatility and VaR are the wrong instruments. - Every currency loses against gold over the long run, and fixed pegs hide risk until they break: calm readings, then a cliff. - The whole dataset is open source and reproducible, with an automated build pipeline and weekly updates. {% end %} ## The dataset ### Exchange rates | Source | Period | Description | |--------|--------|-------------| | [MEMDB Spufford](https://memdb.libraries.rutgers.edu/spufford-currency) | 1106-1500 | 13,197 medieval exchange quotations across 521 places | | [MEMDB Metz](https://memdb.libraries.rutgers.edu/metz-currency) | 1350-1800 | 50,559 records from the Lower Rhine region | | [Clio Infra](https://clio-infra.eu/) | 1500-2016 | Exchange rates vs GBP and USD, inflation, bonds, debt, GDP, real wages | | [CFS Historical Financial Statistics](https://centerforfinancialstability.org/hfs.php) | ~1500-1950 | Official and market exchange rates, interest rates, money supply, central bank balance sheets | | [MeasuringWorth](https://www.measuringworth.com/datasets/exchangeglobal/) | 1791-2025 | 41 currencies vs USD, gold (5 series), interest rates, CPI | | [Bank of England](https://www.bankofengland.co.uk/statistics/research-datasets) | 1791-2016 | UK millennium dataset (90+ sheets) | | [JST Macrohistory](https://www.macrohistory.net/database/) | 1870-2017 | 18 advanced economies, 59 macro variables | | [Sveriges Riksbank](https://www.riksbank.se/en-gb/statistics/interest-rates-and-exchange-rates/) | 1900-2026 | 53 SEK bilateral series, 295,000 observations | | [Penn World Table](https://www.rug.nl/ggdc/productivity/pwt/) | 1950-2023 | 185 countries, exchange rates and PPP | | [IMF IFS](https://data.imf.org/ifs) | 1955-2025 | 173 currencies, monthly, 158,000 observations | | [BIS](https://data.bis.org/topics/EER) | 1957-2026 | Bilateral and effective rates, ~190 economies, 2.66 million rows | | [World Bank FX](https://data.worldbank.org/indicator/PA.NUS.FCRF) | 1960-present | Official rates, all member countries | | [Bruegel/Darvas REER](https://www.bruegel.org/publications/datasets/real-effective-exchange-rates-for-178-countries-a-new-database) | 1960s-2026 | Real effective exchange rates, 178 countries, monthly | | [Global Macro Database](https://www.globalmacrodata.com/) | 1960-2024 | 243 countries, USD FX and REER | | [FRED Daily](https://fred.stlouisfed.org/categories/94) | 1971-2025 | 23 daily currency pairs and 2 USD indices | ### Precious metals | Source | Period | Description | |--------|--------|-------------| | [MeasuringWorth](https://www.measuringworth.com/datasets/gold/) | 1257-2025 | Gold prices (5 series: British official, London, US, New York, gold/silver ratio) | | [FreeGoldAPI](https://freegoldapi.com/) | 1258-2025 | 768-year gold, silver prices, and gold/silver ratio | | [LBMA](https://www.lbma.org.uk/prices-and-data/precious-metal-prices) | 1968-2025 | Daily gold and silver in USD, GBP, EUR | | [DataHub Gold](https://github.com/datasets/gold-prices) | 1833-2025 | Monthly gold prices USD | ### Interest rates | Source | Period | Description | |--------|--------|-------------| | [Schmelzing (BoE)](https://www.bankofengland.co.uk/working-paper/2020/eight-centuries-of-global-real-interest-rates-r-g-and-the-suprasecular-decline-1311-2018) | 1311-2018 | Real interest rates for 8 countries (Italy, UK, Netherlands, Germany, France, Spain, Japan, US) | | [MeasuringWorth](https://www.measuringworth.com/datasets/) | 1729-2025 | UK and US nominal short and long term rates | ### Commodity prices | Source | Period | Description | |--------|--------|-------------| | [Allen-Unger GCPD](https://datasets.iisg.amsterdam/dataset.xhtml?persistentId=hdl:10622/3SV0BO) | 1260-1914 | 973 commodity price series (wheat, rye, silver, coal, spices) across European and Asian cities | | [World Bank Pink Sheet](https://www.worldbank.org/en/research/commodity-markets) | 1960-present | ~70 commodity prices (oil, metals, agriculture), monthly and annual | ### GDP, inflation, wages, and debt | Source | Period | Description | |--------|--------|-------------| | [Maddison Project](https://www.rug.nl/ggdc/historicaldevelopment/maddison/) | 1 CE-2022 | GDP per capita, 178 countries | | [Riksbank Historical Monetary Statistics](https://www.riksbank.se/en-gb/statistics/historical-monetary-statistics-of-sweden/) | 1277-2020 | Swedish FX, CPI, wages, GDP, money supply, stocks, bonds (3 volumes, 13 files) | | [IMF HPDD](https://data.imf.org/) | 1800-2015 | Sovereign debt-to-GDP, 191 countries | | [Reinhart-Rogoff](https://carmenreinhart.com/data/) | 1800-2016 | Debt/GDP, inflation, crisis indices, gold standard dates, regime classifications | ### Regime classifications and crises | Source | Period | Description | |--------|--------|-------------| | [IRR Regimes](https://www.ilzetzki.com/irr-data) | 1946-2016 | De facto exchange rate regime classifications, ~190 countries | | [Reinhart-Rogoff](https://carmenreinhart.com/data/) | 1800-2016 | Banking, currency, and debt crisis indices; capital control indicators | The longest individual exchange rate series is the United States at 526 years (1500-2025, inclusive). The United Kingdom has 237 years of continuous data (1789-2025, inclusive). The medieval data covers exchange quotations across Florence, Bruges, Venice, London, and hundreds of other cities across Europe, Byzantium, and North Africa. For GDP per capita, the Maddison Project traces 178 countries back to 1 CE. For interest rates, Schmelzing's dataset provides 707 years of real rates across eight countries. For commodity prices, the Allen-Unger database contains 973 individual series spanning the period when Europe transitioned from medieval to modern market economies. All derived exchange rate data uses a consistent quoting convention: foreign currency per 1 USD. The build pipeline normalizes, deduplicates, and cross-validates across sources. Where sources overlap, [MeasuringWorth](https://www.measuringworth.com/datasets/exchangeglobal/) takes priority (carefully curated for continuity), then [Clio Infra](https://clio-infra.eu/), then the [Global Macro Database](https://www.globalmacrodata.com/). ## What fat tails mean Before diving into the results, a brief primer on the key concepts. If you have read the companion article on [crash detection](@/series/leptokurtic/2026-02-19-detecting-crashes-with-fat-tail-statistics/index.md), you can skip this section. ### Log returns Throughout this article, all return calculations use **log returns** (also called continuously compounded returns). If $P_t$ is the exchange rate at time $t$ and $P_{t-1}$ is the rate at the previous period, the log return is: $$r_t = \ln\left(\frac{P_t}{P_{t-1}}\right)$$ Log returns are the standard choice for financial analysis because they are additive over time (you can sum daily log returns to get the yearly return) and symmetric (a +10% followed by a -10% move doesn't leave you back where you started in simple percentage terms, but log returns account for this correctly). ### Excess kurtosis **Kurtosis** measures how much of a distribution's weight sits in the tails versus near the center. A normal (Gaussian) distribution has a kurtosis of 3. **Excess kurtosis** subtracts that baseline: $$\text{Excess kurtosis} = \frac{\mathbb{E}\left[(X - \mu)^4\right]}{\left(\mathbb{E}\left[(X - \mu)^2\right]\right)^2} - 3$$ where $\mu$ is the mean and $\mathbb{E}$ denotes the expected value. If excess kurtosis is 0, the distribution has Gaussian-like tails. If it is positive, the tails are heavier: extreme events happen more often than a bell curve predicts. An excess kurtosis of 5 means the extreme observations are far more frequent than normal. An excess kurtosis of 4,110, as we will see for Sri Lanka, means the distribution looks nothing like a Gaussian. ### Tail index and power laws Many financial distributions follow a **power law** in the tails. The probability of seeing a value larger than $x$ falls as: $$P(X > x) \sim x^{-\alpha}$$ The exponent $\alpha$ is the **tail index**. It tells you how heavy the tail is: - $\alpha < 2$: **Infinite variance.** The distribution is so fat-tailed that the variance does not converge no matter how many observations you collect. Computing a standard deviation is meaningless because the number you get depends on your sample size. - $\alpha$ between 2 and 4: **Finite variance but infinite kurtosis.** Fat tails, but the standard deviation at least converges. This is where many financial assets live. - $\alpha > 4$: **Moderate tails.** Still fatter than Gaussian, but standard tools become somewhat reasonable. We estimate $\alpha$ using the **Hill estimator**. Sort the $n$ observations in descending order, take the $k$ largest, and compute: $$\hat{\alpha} = k \left( \sum_{i=1}^{k} \ln \frac{X_{(i)}}{X_{(k)}} \right)^{-1}$$ where $X_{(1)} \geq X_{(2)} \geq \ldots$ are the order statistics. The choice of $k$ matters: too small and the estimate is noisy; too large and you include observations from the body of the distribution. A Hill plot ($\alpha$ versus $k$) helps find the stable region. ### Annualized volatility **Annualized volatility** is the standard deviation of returns scaled to one year. If $\sigma_d$ is the standard deviation of daily log returns, the annualized figure is: $$\sigma_{\text{ann}} = \sigma_d \times \sqrt{252}$$ where 252 is the approximate number of trading days per year. This gives a single number summarizing "how much does this currency move in a typical year." EUR/USD at 10% annualized volatility means a one-standard-deviation annual move is about 10 cents on the dollar. Venezuela at 380% means the currency can lose most of its value in a single year. ### 3-sigma tail ratio The **3-sigma tail ratio** compares how often extreme events actually occur versus how often a Gaussian would predict. Under a normal distribution, returns beyond 3 standard deviations from the mean should happen about 0.27% of the time. If the actual frequency is 1.08%, the tail ratio is 4x. A ratio above 1 means fatter tails than normal; a ratio below 1 means thinner tails (which can happen in pegged currencies that suppress small moves but concentrate risk in huge jumps). ## Daily data (1971-2025) The central finding from running [fatcrash](https://github.com/unbalancedparentheses/fatcrash) on this data: **fat tails are universal across all currencies, all time scales, and all centuries.** Every one of the 23 [FRED](https://fred.stlouisfed.org/categories/94) daily currency pairs has heavier tails than a Gaussian distribution. For most floating pairs, three-sigma events happen about 3 to 6 times more often than a normal distribution predicts. Pegged and tightly managed pairs are a special case: they can show tail ratios below 1 even with extreme kurtosis because daily returns are compressed near zero. Even EUR/USD, the most liquid pair in the world, has excess kurtosis of 2.5 and 4x too many tail events. The most extreme daily tails: | Currency | Ann. Vol | Excess Kurtosis | 3-sigma Tail Ratio | |----------|:--------:|:---------------:|:------------------:| | LKR/USD | 11.7% | 4,110 | 2.98x | | CNY/USD | 8.2% | 3,846 | 0.53x | | VEF/USD | 380.7% | 2,560 | 0.28x | | THB/USD | 8.7% | 279 | 5.32x | | HKD/USD | 3.2% | 261 | 5.09x | | KRW/USD | 10.8% | 140 | 4.37x | Sri Lanka (LKR) has an excess kurtosis of 4,110. For context, a Gaussian distribution has excess kurtosis of 0. The Student's t-distribution with 3 degrees of freedom, often used as a "fat-tailed alternative" in finance, has excess kurtosis of infinity (undefined). LKR's distribution is empirically closer to the Student's t than to the Gaussian. Note the paradox in the tail ratio column: CNY and VEF have enormous kurtosis but tail ratios *below* 1. This happens because these currencies are pegged or managed. Their daily returns are almost always exactly zero (the peg holds), so the standard deviation is very small. The rare days when the peg breaks produce moves that are extreme in absolute terms but may not exceed 3 of those tiny standard deviations as often as you would expect, because the distribution is a spike at zero with a few catastrophic outliers rather than a smooth bell curve. The kurtosis captures this shape; the simple tail ratio can miss it. ## Yearly data (1791-2025) On yearly timescales, the tails are even more extreme. Using [MeasuringWorth](https://www.measuringworth.com/datasets/exchangeglobal/) data: | Country | Years | Excess Kurtosis | Worst Year | |---------|:-----:|:---------------:|:----------:| | Mexico | 234 | 83.7 | -86% | | Austria | 234 | 49.9 | -100% | | Israel | 234 | 46.7 | -100% | | Germany | 234 | 37.8 | -100% | | Peru | 234 | 36.2 | -100% | | Argentina | 234 | 19.9 | -100% | | Brazil | 234 | 13.3 | -100% | | United Kingdom | 234 | 5.2 | -33% | Germany's kurtosis of 37.8 reflects a single year: 1923, when the mark lost effectively all its value. But this is exactly the point. A model that treats 1923 as an outlier to be excluded is a model that doesn't understand what kind of distribution it is dealing with. In a fat-tailed distribution, the extreme observation *is* the distribution. Remove it and you are estimating the wrong thing. As [Taleb (2020)](https://arxiv.org/abs/2001.10488) emphasizes, the sample mean and sample variance of a power-law distribution are dominated by the largest observation. That single observation contains more information about the tail than all the other observations combined. ### Five centuries of tail estimates (1500-2013) Using fatcrash's Hill estimator on the [Clio Infra](https://clio-infra.eu/) data (1500-2013, 138 countries with 50+ years of data): | Tail regime | Countries | Percentage | |------------|:---------:|:----------:| | $\alpha < 2$ (Hill estimate in the infinite-variance regime) | 98 | **71%** | | $\alpha$ 2-4 (fat tails, finite variance) | 37 | 27% | | $\alpha > 4$ (moderate tails) | 3 | 2% | 71% of countries land in the Hill-estimated $\alpha < 2$ regime. The median tail index $\alpha$ across all 138 countries is 1.57. For these currencies, variance-based summaries are unstable and can move dramatically as you add observations. In practice, the reported standard deviation depends heavily on sample length and on whether the largest devaluations are present in the window. This goes far beyond a marginal statistical issue: the foundational assumptions behind Modern Portfolio Theory, CAPM, Black-Scholes-style modeling, and variance-based VaR become highly fragile when the tail is this heavy. For a majority of the currencies in this sample, finite-variance approximations are at best unstable and at worst badly misleading. ## The peg paradox The most counterintuitive finding: currencies with the lowest daily volatility have some of the highest excess kurtosis. HKD/USD has 3.2% annualized volatility (barely moves) but excess kurtosis of 261. CNY/USD has 8.2% volatility but excess kurtosis of 3,846. Freely floating currencies like GBP/USD, with 10.1% volatility, have excess kurtosis of 5.4. A **peg** compresses daily volatility to near zero. The central bank intervenes to keep the rate fixed, absorbing shocks with reserves instead of letting the price adjust. But when the peg breaks --- when the central bank runs out of reserves or loses the political will to defend it --- the move is catastrophic. The distribution is a spike at zero with rare but enormous outliers. This is exactly the kind of distribution that **Value at Risk** (VaR) models fail on. VaR estimates the loss you won't exceed with some high probability (say 99%) on a given day. It says risk is low (because daily moves are tiny). The true risk is high (because the peg can snap at any time, and when it does, the loss is 10 to 50 standard deviations). The regime data from [Ilzetzki, Reinhart, and Rogoff (2019)](https://www.nber.org/papers/w23134), covering 1946-2016, confirms this pattern: | Regime | N Countries | Volatility | Excess Kurtosis | |--------|:-----------:|:----------:|:---------------:| | Free float | 9 | 10.8% | 0.8 | | Managed float | 33 | 56.7% | 129.3 | | Crawling peg | 31 | 53.6% | 139.7 | | Peg | 37 | 59.5% | 132.9 | | Freely falling | 12 | 225.1% | 16.0 | Free-floating currencies have low kurtosis (0.8) because the market absorbs shocks continuously through small daily moves. Pegged and managed currencies suppress daily volatility but accumulate stress that releases in catastrophic jumps, producing kurtosis 160 times higher than free floats. This is [Taleb's](https://arxiv.org/abs/2001.10488) fragility argument in data. Suppressing volatility does not reduce risk. It concentrates risk into rare, large events. The peg creates an illusion of stability that makes the eventual break more damaging, both financially and psychologically, because nobody is positioned for it. Small, frequent adjustments are antifragile: each one releases pressure and provides information. Rigid pegs are fragile: they hide information until the system snaps. The "freely falling" category is instructive too. These are currencies in hyperinflation or free-fall collapse. They have the highest volatility (225%) but moderate kurtosis (16.0). When a currency is already collapsing every day, there is no pent-up pressure left. The tails are fat but not as extreme as a peg because the large moves are continuous rather than sudden. ## Every currency loses against gold Using gold price data from [DataHub](https://datahub.io/core/gold-prices) and [MeasuringWorth](https://www.measuringworth.com/datasets/gold/) cross-referenced with exchange rates, we computed the cumulative gold purchasing power retained by major currencies. The British pound has data going back to 1257. For roughly 600 years (1257 to the early 1900s), the pound maintained its gold purchasing power relatively well, fluctuating within a band. The **gold standard** enforced discipline: because the pound was defined as a fixed weight of gold, debasement required a deliberate political act (reducing the gold content of coins, or suspending convertibility). Then the 20th century happened: two world wars, the end of the gold standard, and continuous inflation. The pound has lost the vast majority of its gold purchasing power since 1900. Every other currency follows the same arc, just faster. The dollar has lost most of its gold value since 1789. The yen, rupee, and most Latin American currencies show steeper declines. No currency in the dataset has gained gold purchasing power over its full history. This should be read as an empirical observation, not an argument for or against the gold standard. Over multi-century timescales, fiat currencies consistently debase against a fixed-supply asset. The rate of debasement varies enormously, but the long-run direction is consistent. [Reinhart and Rogoff (2009)](https://press.princeton.edu/books/paperback/9780691152646/this-time-is-different) document this pattern systematically across eight centuries: governments debase currencies to finance wars, bail out banks, and cover deficits. The mechanism changes (coin clipping in the medieval period, money printing in the modern era), yet the long-run outcome is the same. The dataset includes monthly gold inflation data for 174 currencies from 1940 to 2025 and yearly data for 243 countries from 1257 to 2025. ## Cross-currency correlations The daily correlation matrix reveals clear geographic and economic clusters: - **European cluster**: GBP, CHF, NOK, SEK, DKK, EUR move together, reflecting economic integration and policy coordination. - **Asian managed currencies**: SGD, MYR, THB form a tight cluster. These currencies are managed with reference to trade-weighted baskets that include each other. - **Asian pegged**: CNY, HKD, LKR are nearly uncorrelated with everything else. Their pegs decouple them from market forces most of the time. - **Commodity/Antipodean**: CAD, AUD, NZD cluster together, driven by commodity export exposure. - **Latin America**: BRL and MXN show some correlation but are more idiosyncratic, driven by country-specific crises. - **Venezuela**: VEF is essentially uncorrelated with all other currencies. Hyperinflation creates noise that overwhelms any trade or policy signal. For portfolio construction, the implication is that holding multiple European currencies does not diversify FX risk much. Holding a European and an Asian currency does. But the "low correlation" of pegged currencies is misleading. In a global crisis, correlations spike as all pegs come under simultaneous pressure. The correlations that matter most are the ones you observe in the worst 1% of days, not the average. ## Medieval exchange data The oldest records in the dataset come from the [Medieval and Early Modern Data Bank](https://memdb.libraries.rutgers.edu/) (MEMDB). [Spufford's](https://memdb.libraries.rutgers.edu/about-spufford-currency) collection contains 13,197 exchange quotations from 521 places across Europe, Byzantium, and North Africa, from 1106 to 1500. [Metz's](https://memdb.libraries.rutgers.edu/metz-currency) collection adds 50,559 records from the Lower Rhine region, from 1350 to 1800, documenting trades in Reichstaler, ducats, marks, and dozens of other coin types. This data matters for a specific reason. Modern exchange rate theory is built on a sample that starts, at earliest, in 1971 (the end of Bretton Woods). Occasionally researchers go back to 1945 or 1900. That gives us, at best, 125 years of data in a world where currency systems have existed for nearly a millennium. The [Allen-Unger Global Commodity Prices Database](https://datasets.iisg.amsterdam/dataset.xhtml?persistentId=hdl:10622/3SV0BO) extends this even further: 973 commodity price series from 1260 to 1914, documenting the prices of wheat, rye, silver, coal, spices, and dozens of other goods across European and Asian cities. Combined with the medieval exchange rates, we can study how commodity prices and exchange rates interacted during the very period when modern financial markets were emerging. The medieval data shows that the basic patterns we observe today --- fat tails, sudden devaluations, contagion across trading partners --- are not artifacts of modern fiat money or floating exchange rates. The Florentine banking crises of the 1340s, when the Bardi and Peruzzi banks collapsed after England defaulted on war debts, caused exchange rate disruptions across Europe that look structurally similar to modern currency crises despite the very different institutions involved. This tail behavior predates the post-1971 monetary regime; it appears to be a feature of exchange rates as such. [Denzel (2010)](https://eh.net/book_reviews/handbook-of-world-exchange-rates-1590-1914/) documents this pattern further in his *Handbook of World Exchange Rates, 1590-1914*, showing that the early modern period had its own currency crises, debasements, and contagion episodes. The mechanisms of medieval and early modern exchange rate volatility (coin debasement, sovereign default, banking panic) differ from modern ones (central bank policy, capital flows, speculative attacks), but the statistical signature is the same: power-law tails and clustered extremes. ## Why this data matters Most financial research operates on 20 to 50 years of data. This is a problem for studying events that happen once every 30 to 80 years. If you have 40 years of data and the event you care about has a 2% annual probability, you might see it once, or not at all. Your sample is too short to estimate the tail. With 2,000 years of data across 240 countries and 27 sources, we have a much larger sample of extreme events. The [Clio Infra](https://clio-infra.eu/) data alone contains dozens of hyperinflations, currency collapses, and regime transitions. [Reinhart and Rogoff's](https://carmenreinhart.com/data/) crisis indices catalogue two centuries of banking panics, sovereign defaults, and currency crashes. [Schmelzing's](https://www.bankofengland.co.uk/working-paper/2020/eight-centuries-of-global-real-interest-rates-r-g-and-the-suprasecular-decline-1311-2018) interest rate data shows that the secular decline in rates is a 700-year trend, not a 40-year anomaly. Combined with the [FRED](https://fred.stlouisfed.org/categories/94) daily data, we can study the same phenomenon at both high frequency (daily) and long duration (centuries). Three things become clear with this much data: 1. **Standard risk models underestimate tail risk by orders of magnitude.** When 71% of countries have infinite-variance exchange rate distributions ($\alpha < 2$), any model that assumes finite variance (CAPM, Black-Scholes, mean-variance optimization) is systematically wrong. It does not give conservative estimates. It gives estimates of the wrong quantity. You cannot approximate a distribution with $\alpha = 1.5$ using a Gaussian any more than you can approximate a Cauchy distribution with a Gaussian. The moments do not exist. 2. **Stability is not safety.** Pegged currencies look stable until they are not. Managed currencies look stable until they are not. Low volatility regimes have the highest kurtosis. The absence of small crises is often the precondition for a large one. This pattern --- suppressed volatility leading to explosive tail events --- appears in every subset of the data: daily, yearly, medieval, modern, developed, developing. 3. **Currency debasement is the norm, not the exception.** Over multi-century timescales, every currency in the dataset loses purchasing power against gold. The rate varies enormously (some countries lose 99% in a decade, others take centuries), but the direction is uniform. Any financial plan that assumes a stable currency over a 30-year horizon is making an assumption that contradicts 2,000 years of data. ## Code and data The full dataset and build pipeline are open source: [github.com/unbalancedparentheses/forex-centuries](https://github.com/unbalancedparentheses/forex-centuries) The repository includes: - Raw data from all 27 sources (1,100+ files), 23 of which are automatically fetched by a weekly CI pipeline - A 12-step ETL pipeline that normalizes, deduplicates, and cross-validates - Derived datasets: - Unified yearly panel (24,656 rows, 243 countries), normalized daily rates (271,228 rows), log returns, rolling volatility - Regime classifications, gold inflation for 174 currencies, correlation matrices - Momentum analysis: 3/6/12 month momentum and reversals for 23 FRED currencies - Sigma event frequencies: observed 2-5$\sigma$ events vs Gaussian expected counts per currency - JST asset class returns: real returns on equities, housing, bonds, and bills across 18 countries (1870-2017) - 20-year rolling stock-bond correlations from JST data - 9 charts: fat-tail histograms, QQ-plots, peg paradox scatter, tail ratio bars, rolling volatility, correlation heatmap, gold erosion, regime timeline - 52 data validation checks with outlier detection and cross-source consistency - 17 unit tests with synthetic data - Quickstart scripts (pure Python and pandas versions) - A Jupyter exploration notebook The companion crash detection toolkit: [github.com/unbalancedparentheses/fatcrash](https://github.com/unbalancedparentheses/fatcrash) ## References - Spufford, P. (1986). [*Handbook of Medieval Exchange*](https://books.google.com/books/about/Handbook_of_Medieval_Exchange.html?id=IvgoAAAAYAAJ). Royal Historical Society. - Metz, R. (1990). [*Geld, Wahrung und Preisentwicklung*](https://memdb.libraries.rutgers.edu/metz-currency). (Rhine region, 1350-1800). - Denzel, M. (2010). [*Handbook of World Exchange Rates, 1590-1914*](https://eh.net/book_reviews/handbook-of-world-exchange-rates-1590-1914/). Ashgate. - Reinhart, C. and Rogoff, K. (2009). [*This Time Is Different: Eight Centuries of Financial Folly*](https://press.princeton.edu/books/paperback/9780691152646/this-time-is-different). Princeton University Press. - Ilzetzki, E., Reinhart, C. and Rogoff, K. (2019). [*Exchange Arrangements Entering the 21st Century: Which Anchor Will Hold?*](https://www.nber.org/papers/w23134) Quarterly Journal of Economics, 134(2). - Jorda, O., Schularick, M. and Taylor, A. (2017). [*Macrofinancial History and the New Business Cycle Facts*](https://www.macrohistory.net/database/). NBER Macroeconomics Annual. - Thomas, R. and Dimsdale, N. (2017). [*A Millennium of UK Data*](https://www.bankofengland.co.uk/statistics/research-datasets). Bank of England. - Officer, L. and Williamson, S. [MeasuringWorth](https://www.measuringworth.com/datasets/exchangeglobal/). - Schmelzing, P. (2020). [*Eight Centuries of Global Real Interest Rates, R-G, and the 'Suprasecular' Decline, 1311-2018*](https://www.bankofengland.co.uk/working-paper/2020/eight-centuries-of-global-real-interest-rates-r-g-and-the-suprasecular-decline-1311-2018). Bank of England Staff Working Paper No. 845. - Bolt, J. and van Zanden, J.L. (2024). [*Maddison Project Database 2023*](https://www.rug.nl/ggdc/historicaldevelopment/maddison/). University of Groningen. - Allen, R.C. and Unger, R.W. [*Global Commodity Prices Database*](https://datasets.iisg.amsterdam/dataset.xhtml?persistentId=hdl:10622/3SV0BO). International Institute of Social History. - Darvas, Z. (2012). [*Real Effective Exchange Rates for 178 Countries: A New Database*](https://www.bruegel.org/publications/datasets/real-effective-exchange-rates-for-178-countries-a-new-database). Bruegel Working Paper 2012/06. - Edvinsson, R., Jacobson, T. and Waldenström, D. (2010). [*Historical Monetary and Financial Statistics for Sweden*](https://www.riksbank.se/en-gb/statistics/historical-monetary-statistics-of-sweden/). Sveriges Riksbank. - Mitchener, K.J. and Weidenmier, M.D. (2015). [*Historical Financial Statistics*](https://centerforfinancialstability.org/hfs.php). Center for Financial Stability. - Jegadeesh, N. and Titman, S. (1993). [*Returns to Buying Winners and Selling Losers*](https://doi.org/10.1111/j.1540-6261.1993.tb04702.x). Journal of Finance, 48(1). - Jordà, Ò., Schularick, M. and Taylor, A. M. (2019). [*The Rate of Return on Everything, 1870-2015*](https://doi.org/10.1093/qje/qjy004). Quarterly Journal of Economics, 134(3). - Rattray, S., Granger, N., Harvey, C. R., and Van Hemert, O. (2020). [*Strategic Rebalancing*](https://doi.org/10.3905/jpm.2020.1.131). Journal of Portfolio Management, 46(6). - Taleb, N. N. (2020). [*Statistical Consequences of Fat Tails*](https://arxiv.org/abs/2001.10488). STEM Academic Press. --- ## Series: Les Circuits Longs Dissolution runs at computational speed. Construction runs at biological speed. Friction was the governor that kept them in sync. This series traces a single mechanism from several angles: what happens when optimization outpaces formation. ### Dissolution Without Construction *Published: 2026-03-06* > Every previous coordination technology that dissolved a form of selfhood also cultivated the next one. Current algorithmic systems only dissolve. The problem is the speed mismatch. URL: https://federicocarrone.com/series/les-circuits-longs/dissolution-without-construction/ I have been circling the same problem for a while now. Friction produces value. Legibility destroys what it measures. Formation requires lived time. The modern self is dissolving through redundancy. Previous technological shifts gave people decades to adapt, and this one might give them months. These kept feeling like separate observations. I no longer think they are. ## The Speed Mismatch Every major coordination technology in history has dissolved the form of selfhood that preceded it. Writing destroyed oral memory, the printing press destroyed manuscript culture, and institutions dissolved kinship. None of this is new. What is new is the asymmetry between how fast things dissolve and how fast things grow. Writing spread slowly. A scribe copied a text, carried it to another city, taught someone to read it. The dissolution of oral memory and the construction of literate thought happened at roughly the same pace, both measured in generations. The printing press was faster, but still slow enough that cultural forms could emerge alongside the destruction. The novel, the diary, the public library, liberal education: these took centuries to stabilize, but centuries were available because print moved at the speed of physical objects. Algorithmic systems dissolve at computational speed. A recommendation engine can reshape the taste-formation process of millions of people in months. An AI assistant can make inner deliberation feel unnecessary within a single product cycle. But construction, the formation of new human capacities, new cultural forms, new modes of perception, still happens at biological speed. A person becomes a particular kind of person through years of encounter, revision, failure. A cultural form stabilizes through generations of practice. There is no computational shortcut because the capacity is constituted by the process, not by its output. You cannot compress becoming. The problem is the gap between dissolution speed and construction speed. This reframes a conversation that has been stuck for years. One side says previous transitions worked out, so this one will too. They are wrong for a structural reason: previous transitions worked out because dissolution was slow enough for construction to keep pace. That condition no longer holds. The other side says technology is destroying us. They are also wrong, or at least imprecise. Dissolution can be the beginning of construction; every previous transition dissolved something real and produced something new. The issue is speed: dissolution now moves fast enough to make construction impossible. ## Friction as Governor Desire depends on resistance. The deeper point is that resistance imposes a pace. The distance between wanting and obtaining is a temporal structure. It gives the person time to become someone who can integrate what they receive. Remove the distance and the person receives before they can absorb. Friction performs this function at civilizational scale. When the printing press dissolved oral culture, books were expensive. Literacy spread gradually. The oral self did not vanish overnight. It weakened over decades and centuries, and during that time the constructive side, new practices of reading, new institutions, new literary forms, had room to develop. The friction inherent in physical media imposed a speed limit on dissolution, and that speed limit happened to match the speed of human formation. Friction in taste formation gives a person time to develop the capacity to perceive, beyond a set of preferences. Friction in professional training gives the practitioner time to develop judgment that resists articulation. Friction in deliberation gives the self time to form. Remove it and dissolution outruns construction. The gap opens. Nothing grows where the old capacity was. ## Legibility as Accelerant The German forest again. Eighteenth-century foresters replaced diverse, messy woodland with uniform Norway spruce plantations. Yields surged. Then the undergrowth died, the soil degraded, and the forest collapsed. What looked like inefficiency was the system's life support. Legibility does something specific to the speed problem. It converts illegible processes into optimizable targets. Once a process is visible, it can be measured. Once measured, optimized. Once optimized, the friction in the original process gets removed as inefficiency. Legibility is the mechanism that identifies friction, and optimization is the mechanism that strips it out. Together they accelerate dissolution. The self depended on opacity. Taste formed in private. You encountered things by accident, sat with discomfort no algorithm could detect, revised your sensibility through a process invisible to any external system. Professional judgment lived in knowledge that could not be articulated. Inner deliberation happened in a space that was, by definition, not observable from outside. AI reaches into all of this. It makes taste formation legible through behavioral data. It makes deliberation legible through interaction logs. It makes professional intuition legible through performance metrics. Each act of legibility strips away a layer of friction that was functioning, without anyone noticing, as a speed governor on dissolution. ## The Compounding Problem This is the part that worries me most. Dissolution outruns construction and degrades the conditions under which construction is possible. Formation requires complete cycles: effort, feedback, adjustment, repeated across lived time. Acceleration fragments these cycles. What spreads fastest diverges from what works best. Imitation spreads faster than learning. Every act of dissolution removes some of the friction, opacity, and time that construction requires. The more functions the self loses, the less capacity remains to develop new ones. The process compounds. Dissolution accelerates while the ground for construction erodes. Someone who has never formed taste through friction is not going to develop whatever post-algorithmic perception might look like. Someone who has never exercised inner deliberation will not develop whatever comes after deliberation. The new capacity, if it exists, will not emerge from a vacuum. It will emerge from people who have developed enough of the old capacity to transcend it, the way literate thought emerged from people who had first mastered oral memory. If dissolution destroys the old capacity before the new one can develop from it, the sequence breaks. What looks like another transition is actually an interruption. ## What Would Have to Be True I do not know what comes after the literate self. Nobody in 1450 could have described what the printing press would produce either. But I can describe the conditions under which construction becomes possible. It would require friction. Not arbitrary difficulty, but the specific resistance that imposes a pace compatible with human formation. It would require opacity. Domains where the slow work of becoming is not legible to optimization systems, where a person can develop without being measured. And it would require time. Actual time. Complete cycles of effort and revision that are not compressed or interrupted. The current system removes all three. Optimization treats friction as waste. AI treats opacity as a problem to solve. Markets treat slowness as a competitive disadvantage. The incentives point uniformly toward faster dissolution. Previous transitions produced their own constructive forms because the speed of dissolution left room for them. Print was slow enough that the novel could emerge. Institutional life was gradual enough that liberal education could develop. The printing press did not spontaneously generate the literate self. People built schools, designed curricula, established practices of reading and argument that took centuries to stabilize. That work was deliberate, institutional, and slow, but it was possible because the dissolution it responded to was also slow. The question I cannot answer is whether anything can grow at biological speed in an environment that has been optimized for computational speed. Whether construction is possible when the conditions for construction are precisely what the system is most efficient at removing. There is an irony I should not avoid. I build coordination infrastructure for a living. Ethereum clients, cryptographic proof libraries, distributed systems. Tools that accelerate exactly the process this essay describes. And the essay itself is not construction. It is diagnosis. I am genuinely confused by my own position. I see the dissolution clearly enough to write about it. I also build the tools that produce it. I have not resolved this. I am not sure it can be resolved. Naming the speed mismatch does not slow it down. It may even accelerate it by making the problem legible, which is what I argued legibility does: convert things into objects of optimization. I do not know whether writing about dissolution is a form of friction, something that slows the reader down, forces them to sit with discomfort, or a frictionless take on friction, consumed and forgotten at algorithmic speed. I suspect the answer depends on what you do after reading it. --- ### Legibility Kills What It Measures *Published: 2026-03-03* > What happens when AI makes everything legible? The things that worked because they couldn't be seen clearly may stop working once they can. URL: https://federicocarrone.com/series/les-circuits-longs/legibility-kills-what-it-measures/ In the 18th century, German foresters invented scientific forestry. They looked at a messy, diverse forest and saw inefficiency. Old trees, young trees, deadwood, underbrush, species with no commercial value. They cleared it all and planted Norway spruce in straight rows, evenly spaced, same age, same species. The forest became legible. You could measure it, manage it, predict its yield with precision. For one generation, it worked brilliantly. Yields surged. Then the forest began to die. The complex undergrowth had been cycling nutrients, retaining moisture, hosting the insects that pollinated the canopy and the fungi that fed the roots. The foresters had not simplified the forest; they had destroyed the system that kept it alive, preserving only the part they could see. James Scott tells this story in *Seeing Like a State* to illustrate a pattern that recurs wherever central authorities impose legibility on complex systems. The pattern is simple: make the illegible legible, optimize what you can now see, and lose what you couldn't see but depended on. ## I. Legibility means making something readable from above. A state needs legible citizens (permanent surnames, census records, standardized addresses) to tax and conscript them. A manager needs legible workers (KPIs, time tracking, performance reviews) to evaluate them. The moment you do this, you flatten the thing a little. The record is useful, but it is not the person. Michael Polanyi called the missing part the tacit dimension. We know more than we can tell. A master craftsman cannot fully articulate what makes a joint right. A good doctor reads a patient's face before looking at the lab results. A trader feels the market shifting before the data confirms it. This knowledge is real, but it was learned through practice rather than instruction. It lives in the body, in habits, in pattern recognition trained over thousands of repetitions. You can write some of it down, sometimes a lot of it, but not all of it, and the part that refuses to fit cleanly into a system is often the part doing the real work. ## II. Goodhart's Law states that when a measure becomes a target, it ceases to be a good measure. Most people know the shallow version of this: the metric gets gamed. Teachers teach to the test. Researchers chase citations instead of doing risky work. Managers optimize the quarter because that is the evaluation cycle they live inside. But there is a deeper version. Sometimes the measure changes the institution so thoroughly that the old aim starts to look naive, sentimental, or inefficient. Schools measured by test scores end up reshaped around the scores. [The slow, hard-to-measure work of developing judgment and curiosity](@/series/les-circuits-longs/2025-12-15-notes-on-culture-infrastructure-time-and-ergodicity/index.md) loses its place in the schedule. The same thing happens in research once citation count becomes a career filter, and in public companies once the quarter becomes the only clock that matters. At first the metric is a proxy. After a while it becomes the institution's memory of what the work is for. The older purpose survives only as rhetoric, then eventually not even that. ## III. Scott's central examples are states. He shows how Soviet collectivization, Brasilia's urban planning, and Tanzanian forced villagization all followed the same pattern. A planner looked at a complex, functioning system, decided it was disorderly, imposed a grid, and watched the system collapse. Jane Jacobs saw the same thing in cities. A neighborhood that looks chaotic from above (mixed uses, irregular streets, buildings of different ages) is often deeply functional at ground level. The bodega owner watches the street. The mix of commercial and residential keeps foot traffic at all hours. Old buildings provide cheap space for new businesses. Children have places to linger. The street keeps some of its own memory. When Robert Moses built highways through these neighborhoods, he was not failing to see the order; he was seeing a different kind of order, one that could be drawn on a blueprint and measured by traffic throughput. Once that order wins, the losses show up indirectly: fewer small businesses, weaker street life, less trust, less casual supervision, fewer reasons to stay. What disappears is hard to defend precisely because it was never formalized in the first place. ## IV. Every previous technology made specific things legible while leaving vast territories of human life in the dark. Accounting made finances legible but not the judgment behind a deal. Maps made terrain legible but not the ecology that held it together. AI is different. It reaches further into the gray zone where people used to rely on feel, memory, and accumulated judgment. You can see this first in domains where experienced people have been carrying around pattern recognition they could not fully explain. In software, the knowledge that distinguished a staff engineer from a mid-level one (the sense for what will break, the instinct for where complexity hides, the ability to spot brittle code before it fails) becomes much easier to expose, compare, and distribute once a model can read the whole codebase at once. Writing and taste shift in a similar way. The structural moves that distinguished one writer from another become parameters in a model. What used to look like voice, something formed slowly and unevenly over years, starts to look like a set of reproducible decisions. Recommendation systems do something similar to taste. The slow, private process of liking the wrong thing, getting bored, circling back, changing your mind, stumbling into something difficult and learning to love it, gets compressed into a profile that updates in real time. Strategy is next. Once a system can model your market, simulate plausible competitors, and generate options faster than you can think through them, the advantage that once belonged to the person who had spent decades in an industry starts to thin out. The unwritten map is still worth something, but less of it stays private for long. ## V. The hard question is whether this destroys tacit knowledge or mostly democratizes it. Sometimes the answer is plainly positive. The senior engineer's instinct gets encoded and shared. The doctor's clinical eye gets distributed to clinics that would never have had that level of expertise. Craft that once required decades of apprenticeship becomes easier to learn. Real barriers fall, and some of those barriers deserved to fall. But I do not think the risk is distributed evenly. In medicine, debugging, or rote analysis, making hidden expertise more available may be an unambiguous gain. In craft, judgment, taste, and long-horizon strategy, the trade may be harsher. Some of that knowledge was tacit for a harder reason. The master's sense for a joint is not a rule waiting to be extracted but the residue of ten thousand joints, each slightly different, each teaching something that words cannot capture. Replace this with an algorithm and you get something that looks similar but behaves differently at the edges, which is exactly where tacit knowledge earns its keep. Both things are true, but not in equal measure. Some tacit knowledge was just unexploited information, waiting for a sufficiently powerful system to extract it. But some of it was constitutively illegible. It existed only as a pattern in a body, a culture, an institution. Formalizing it does not preserve it. It produces a simplified copy that can fill the same niche for a time, but the original slowly loses the conditions that made it possible. ## VI. Legibility reaches further than measurement. Friction is a kind of protective opacity: the difficulty of getting what you want is part of what prevents wanting from collapsing into consumption. The inner self is opaque in a related way: it is the part of you that has not been flattened into a readable surface for outside systems. Formation is opaque too. It cannot be judged well through snapshots because what matters only shows up over time. Force it into short, legible evaluation cycles and the same pattern returns. The measure replaces the thing. The thing weakens. When AI makes the self legible (behavioral prediction, emotional modeling, preference extraction), it begins to replace the function the self was performing. If an external system knows what you want before you do, the inner process of forming a want loses its purpose. The self does not die dramatically; it becomes unnecessary, the way a muscle atrophies when a machine does its work. The German foresters did not hate the forest. They wanted to optimize it, to bring it under rational control, and for one generation they succeeded by removing everything they could not measure. The forest died because what they removed looked like noise only inside the measurement system they had chosen. Outside that system, it was the forest. How much of human life has this structure? How much of what we are depends on remaining partially opaque, even to ourselves? The map is about to get very detailed. The territory may not survive the survey. --- ### Friction as Luxury: What We Lose When AI Gives Us What We Want *Published: 2026-02-05* > The scarcity that matters most in a post AGI world won't be compute or energy. It will be desire itself. URL: https://federicocarrone.com/series/les-circuits-longs/friction-as-luxury/ ## The Last Scarcity Most discussions of AGI focus on distribution: who gets access, who profits, who loses their job, who controls the infrastructure. Those are real problems, but they're not the deepest one. The deeper problem is what happens to desire. I do not mean ambition in the generic sense. I mean the capacity to want something at a distance, to stay oriented toward something you do not yet have, and to find meaning in the space between reaching and arriving. That capacity is more fragile than we usually admit, and it depends more on friction than most people notice. ## I. Economists have a clean model of desire. People have preferences, goods satisfy preferences, welfare rises as more preferences are satisfied. In that framework, a technology that can satisfy almost any preference at negligible cost looks like an obvious good. The only remaining question is who gets access. That model leaves out something important. Desire has a shape, and that shape depends on certain conditions holding. When you want something over time, you imagine having it. You plan for it, you make sacrifices toward it. The object accumulates meaning from this process. It gets layered with your effort, your anticipation, your history of reaching. When you finally arrive, you don't just get the object. You get the object plus everything you invested in wanting it. Those two things can't be separated. That is why anticipation is often richer than arrival, why the best albums sometimes need months rather than minutes, and why relationships built through difficulty have a texture that convenient ones often do not. Resistance helps produce the value. ## II. When this structure breaks down, the clinical term is anhedonia. But there is a milder and more socially acceptable version of the same pattern. People in this condition can be entertained constantly but rarely feel deeply absorbed. They consume without much appetite. They move from one stimulating thing to the next not because anything is satisfying, but because sitting with incompleteness starts to feel unbearable. You can already see the shape of it in declining attention spans, in the difficulty of sustaining interest in anything that doesn't deliver immediate feedback, in people who feel simultaneously overstimulated and bored. They haven't been deprived, they've been saturated. This doesn't distribute evenly in society. In environments where discomfort is quickly solved, by money, by services, by endless entertainment, the mind gets less practice holding lack. You can grow up surrounded by abundance and still become poor in one specific way: poor in patience for distance. Structurally, it starts to resemble addiction: craving breaks away from fulfillment. Neuroscience draws the same line. Kent Berridge's work separates wanting from liking, and it is the dopaminergic wanting, not the pleasure, that addictive drugs and variable-reward machines exploit. Slot machines and infinite feeds are built around exactly this, unpredictable payoffs on a schedule that trains the nervous system to treat discomfort as a cue for relief and relief as a cue for repetition. A frictionless AI environment could reproduce some of that pattern without chemicals. Boredom, loneliness, uncertainty, and effort all become prompts for instant stimulation. Over time the threshold rises, what once felt absorbing becomes merely adequate, and the rest of life starts to feel slow and underpowered by comparison. Consumer capitalism produced a weakened version of this. Desire progressively hollowed out by eliminating friction, but with enough friction remaining that the structure didn't fully collapse. The streaming service still requires you to choose. The algorithm still occasionally surprises you. The simulation of connection is imperfect enough that you sometimes notice it's a simulation. ## III. Imagine a system that can generate, on demand, a novel calibrated to your tastes: the style you find most pleasurable, the level of complexity you find most engaging, the length that matches your current patience. Or music that sounds like what you loved most at nineteen, except new, immediate, and endless. Or a conversation partner who is always interested in what interests you, always available, never distracted, never carrying needs of their own into the exchange. The output might be genuinely good. The novel could be technically accomplished. The music could actually move you. The conversation could be substantive. The problem is what happens to wanting once the gap collapses to zero. The capacity to stay oriented toward a distant goal, [to defer, to invest, to tolerate incompleteness](@/series/les-circuits-longs/2025-12-15-notes-on-culture-infrastructure-time-and-ergodicity/index.md), atrophies when it is never exercised. Not through a dramatic break, but through disuse. The ability to want things that require time does not vanish all at once. It gets weaker, and the weakening may not even feel like a loss because something pleasant keeps arriving on schedule. ## IV. None of this is new. Epictetus drilled his students to want only what was already in their control and to rehearse discomfort on purpose, and Seneca set aside days to live as if poor so that hardship could never be used against him. Fasting in Ramadan, the privations of Lent, and the Rule of Benedict all build scheduled scarcity into a life on the same theory: that meaning runs through resistance rather than around it. What's new is the scale. Previous technologies eliminated specific friction but left other friction intact. Every digital environment until now required you to bring something it couldn't supply: attention, skill, patience. A genuinely general AI dissolves this last requirement. It can supply the taste, the context, the judgment. You no longer need to bring anything except the desire to receive. And if that desire is itself shaped by the AI, tuned to whatever maintains engagement, then even the wanting has been outsourced. ## V. Here is the inversion. Where material abundance is the rule, the things that keep their value are often the ones that resist its logic. Their value is tied to the conditions that make them difficult, not to artificial scarcity. A handmade object carries the trace of the hands that made it. A wine vintage can't be accelerated. The waiting isn't incidental to what the wine is. A community built around a shared difficult practice, painting, rock climbing, chess, building and fielding armies of miniatures, generates bonds that digitally mediated interaction doesn't replicate, because those bonds are forged in shared difficulty. These things become valuable not despite being harder than consuming AI output, but partly because of that hardness. When satisfaction is frictionless, friction itself becomes the luxury. ## VI. The safety, alignment, and job-displacement debates are all real. But they share a common assumption: that the humans on the other side will still be capable of deciding what to do with what they've been given, and that political agency and collective imagination will survive intact. That assumption is doing a lot of work. The atrophying of desire is already visible in what weaker technologies have done to culture. What AGI does to human psychology comes before what it does to human politics. A population that has lost the capacity to want things at a distance, to stay oriented toward a difficult future, and to find meaning in effort and incompleteness has lost something essential to self-government. The scarcity that matters most in a post AGI world won't be compute or energy. It will be the capacity to want something deeply enough, and for long enough, that the wanting shapes who you are. If desire is the last scarcity, then slowness, difficulty, and incompleteness are not obstacles to overcome. They are the conditions of a life worth living. --- ### The Death of the Inner Self *Published: 2025-12-23* > Individuality is a coordination technology. It emerged under specific historical conditions, and those conditions are weakening as computation, capital, and automated feedback loops absorb the functions it once performed. URL: https://federicocarrone.com/series/les-circuits-longs/the-death-of-the-inner-self/ The core argument is simple: many features of human life that appear stable and natural are historically produced. As society accelerates, a number of these features begin to lose their function and their permanence. I believe consciousness as we know it is one of them. ## Individuality as technology Life is organized around information that replicates under constraint. Computation generalizes this biological logic. It allows selection and optimization to occur faster and at larger scales by externalizing memory, comparison, and feedback. Problems that once required internal deliberation can be solved through external processes that test, filter, and iterate possibilities. Capital pushes this logic further. It reorganizes social life around continuous feedback, price signals, and competitive selection. As these forces compound, individuality starts to look less like a foundation and more like an interface that emerged to solve earlier coordination problems. Capital behaves as an impersonal intelligence oriented toward speed, abstraction, and self-optimization. As cognition, decision-making, and coordination migrate into automated systems, the inner self loses its structural role. Over time, many assumptions we take for granted are worn down by this acceleration. Individuality and consciousness appear increasingly exposed to this process. ## The construction we cannot see Fish do not realize they live in water. The medium that sustains them is so constant that it disappears from perception. Some of the most important structures are overlooked for the same reason. Individuality and consciousness belong to that category. We tend to treat individuality and consciousness as self-evident facts, as if humans have always experienced themselves as bounded selves with an inner voice, a private mental space, and a continuous narrative identity. Because this experience feels natural, it is assumed to be timeless. Serious thinkers have argued it is not. Julian Jaynes pointed out that the heroes of the Iliad show almost no inner mental space, hearing the voices of gods where we would hear our own deliberation. Charles Taylor traced how the modern "buffered self," sealed inside its own mind, was assembled over centuries out of religious and philosophical practice, displacing an older "porous self" open to forces from outside. For most of human history people did not describe themselves as individuals in the modern sense. Decisions were not understood as outcomes of inner deliberation, and agency was not located inside a private interior self. Action was organized through rituals, traditions, kinship, and prescribed roles. Meaning arrived from outside the person rather than from introspection. In many societies outside the Western trajectory, this structure remains largely intact. The idea of a you inside your head observing your own thoughts is therefore a learned construction. It depends on language, habits, metaphors, and social practices that had to be developed and stabilized over time. Lev Vygotsky argued that inner speech is not innate but internalized social speech: children first talk to others, then to themselves aloud, then silently, until the dialogue goes underground and starts to feel like the private voice of a self. Narrative memory, moral self-examination, and the sense of authorship over action emerged the same way, as cultural achievements layered on top of older biological processes. Modern societies actively reproduce this configuration. From early childhood, people are trained to understand themselves as autonomous units with opinions, preferences, goals, and an inner life that belongs only to them. The training is so pervasive that it becomes invisible. Other ways of being human recede from view, even though many have existed and some still persist. ## The weakening of the conditions The conditions that once made individuality functional are weakening. Earlier systems relied on human subjects to think, decide, judge, and take responsibility. Cognition and coordination were constrained by human minds. Individuality emerged as a solution: a stable self enabled long-term planning, moral accounting, and institutional continuity. Earlier societies coordinated without modern consciousness. Contemporary systems increasingly coordinate without modern selves. Decision-making proceeds without inner deliberation. Meaning is delivered through incentives, metrics, and feedback loops. At the cultural level, individuality remains constantly invoked. People are urged to be themselves, express themselves, optimize themselves. Yet the channels for expression arrive pre-shaped, quantified, and monetized. What appears as selfhood increasingly takes the form of managed performance within narrow bounds. ## Replacement by degrees The modern self does not collapse in a single moment. It is replaced function by function, each substitution small enough to go unnoticed. Taste was once formed through a slow, private process: encountering things by accident, sitting with discomfort, learning to love what initially resisted you. Algorithmic recommendation compresses this into a profile that updates in real time. The system knows what you will like before you do. The inner process of forming a preference, the hesitation, the revision, the gradual shaping of sensibility, loses its purpose when an external system performs it faster and with better accuracy. What remains looks like taste but functions as consumption. Judgment follows a similar path. In organizations that once depended on accumulated experience, performance metrics now determine what counts as competent work. The slow formation of professional intuition, the kind that takes years to develop and resists easy articulation, gets flattened against quarterly targets. This is Goodhart's law turned structural: once a measure becomes the target it stops measuring what it was meant to, and here the target gradually replaces the faculty it was only ever a proxy for. When the metric becomes the institution's memory of what the work is for, the judgment it was meant to approximate quietly disappears. People still show up. They optimize what is measured. The rest erodes. Inner deliberation faces the same pressure from a different direction. When an AI assistant can draft your emails, plan your week, summarize your reading, and suggest your next decision, the internal process of thinking through a problem starts to feel unnecessary, not wrong exactly, just slow. The assistant never tells you to stop thinking; it just makes thinking feel like friction in a system that rewards speed. Over time, the habit of sustained internal reflection weakens for the same reason any unused capacity weakens: through disuse. Each of these substitutions is individually reasonable. Each solves a real problem. Taken together, they describe a pattern where the functions that once required a self are gradually absorbed by systems that do not. ## What is at stake The modern self once felt inevitable because it solved concrete problems. It enabled abstraction, continuity, and responsibility at scale. Its future usefulness is far less certain. The self depends on performing certain functions, and when those functions migrate outward, the self weakens not through suppression but through redundancy. Individuality was real. It produced philosophy, law, science, art, and institutions that reshaped the world. The question is whether it will remain functional as the systems around it absorb more of what it used to do. A coordination technology that no longer coordinates does not persist on sentimentality alone. The self will not simply switch off. But the conditions that produced it are changing, and what comes next may look different enough that the word "individuality" stops pointing at anything we would recognize. Whether that transition is a loss, a transformation, or simply the next phase of the same process that produced the self in the first place is not something that can be settled in advance. But it should be named clearly, because what cannot be seen clearly cannot be preserved deliberately. --- ### Notes on permanence, time, and ergodicity *Published: 2025-12-15* > Some businesses get better with time. Others lose their nerve, their standards, and their memory. This is about the difference. URL: https://federicocarrone.com/series/les-circuits-longs/notes-on-culture-infrastructure-time-and-ergodicity/ Some systems improve the longer you stay with them. Repetition sharpens execution, experience carries forward, and judgment builds on itself. At Hermès, a leather worker trains for two years before touching a bag. One artisan makes one bag start to finish, every stitch by hand, fifteen to twenty-four hours of work per piece. This is [the opposite of speed at all costs](@/series/les-circuits-longs/2026-02-05-friction-as-luxury/index.md). It is also one of the most successful luxury companies in the world. The constraint is part of what the customer is paying for. The broader culture moves in the other direction. Cycles shorten. Signals multiply. Decision horizons shrink. A lot of institutions keep moving while quietly losing the judgment they once had. They stay busy, but they stop getting better. Copying outruns learning. In that environment, endurance tells you something. If a system keeps working under stress for a long time, its structure probably matches reality better than its competitors'. The internet did not flatten everything. It made it easier to see who had substance and who was living off distribution. ## Two forms of time Time operates in human systems in two fundamentally different ways. Measured time is divisible and uniform: schedules, deadlines, accounting periods, discount rates. It can be allocated, optimized, and exchanged. Most planning systems live here. They assume value can be judged apart from history. Lived time works differently. It accumulates. Learning, memory, and judgment develop through it, and each cycle changes the next one. Anything that depends on formation happens here. Snapshots miss the point because the value is in what compounds. In 2001, Boeing moved its headquarters from Seattle to Chicago. The stated reason was to position the company closer to "Wall Street and governments." Engineers who understood the planes were physically separated from executives who understood the spreadsheets. Over the next two decades, Boeing spent more than $40 billion on stock buybacks while cutting capital expenditure to half of Airbus's rate. Harry Stonecipher, who took over as CEO, said he wanted Boeing "run like a business rather than a great engineering firm." The 737 MAX, designed to avoid the cost of pilot retraining, killed 346 people. This is what happens when lived time is forced into measured time. Berkshire Hathaway made the opposite bet. Buffett has refused quarterly earnings guidance since 1996. Shareholders are told to judge the business over decades, not quarters. The result is six decades of compounding judgment, the longest sustained record in American corporate history. Same markets, different use of time. When lived time gets forced into measured time, formation breaks down. Standards do not settle. Judgment does not compound. You only find out what a system really is if you leave it alone long enough to show you. ## Formation under constraint Excellence comes from sustained practice under the right constraints. Errors have to be survivable. People need room to adjust without every bad iteration becoming fatal. Judgment improves when experience carries over from one attempt to the next. At Pixar, every film is terrible for years before it is good. Ed Catmull describes the process as taking movies "from suck to not-suck." The mechanism is the Braintrust: a group of fellow directors and storytellers who meet every few months to review each film in production. The key detail is that the Braintrust has no authority. The director is not required to take a single suggestion. That keeps candor high without turning feedback into bureaucracy. Most studios kill projects after one bad screening. Pixar treats bad screenings as information, not verdicts. The difference is structure, not talent. Formation takes time, and the Braintrust protects that time by separating honest feedback from the power to cancel. ## Four domains We built [Ergodic Group](https://ergodicgroup.com/) around the idea that enduring organizations work across four domains: mathematics, code, culture, and craft. Most live mostly in one. The edge comes from connecting them. Mathematics sets the structure and the constraints. Code turns that structure into action and tests it against reality. Culture lets intent survive changes in personnel. Craft brings the whole thing back to materials, tolerances, and physical consequences. SpaceX shows how these domains work on each other. The math: a technique called lossless convexification lets an onboard computer solve fuel-optimal landing trajectories in real time, computing the exact moment to fire the engines so velocity hits zero at touchdown. The code: autonomous guidance software recomputes trajectories during descent, adjusting for wind and sensor readings, which makes landings on ocean platforms possible. The culture: failures are instrumented, not hidden. Between 2013 and 2016, SpaceX crashed booster after booster, and each crash produced telemetry that led to a specific fix. Hydraulic fluid ran out, so they added more. A throttle valve stuck, so they redesigned it. The craft: when carbon fiber layup produced wrinkles at roughly $200 per kilogram, SpaceX switched Starship to stainless steel at roughly $3 per kilogram. Steel gets stronger at cryogenic temperatures, handles far more heat, and opened reentry profiles that carbon fiber could not. A materials decision changed the vehicle, the software, and the math. Learning compounds when these domains stay connected. ## Ergodicity as a filter Ergodicity describes a situation where repetition improves the usual outcome because learning carries over from one round to the next. Claude Shannon spent fifteen years at Bell Labs before publishing "A Mathematical Theory of Communication" in 1948. He was not being graded on quarterly output. Bell Labs gave researchers something modern organizations rarely give anyone: enough uninterrupted time to get to the bottom of a problem. That setup produced the transistor, information theory, Unix, the laser, and cellular telephony. The transistor came not from a brainstorm but from people with different specialties working near each other for years. When AT&T was broken up in 1984, that model disappeared with it. No later technology company has reproduced the same output. The institution itself held the judgment, and that judgment did not survive disassembly. As acceleration intensifies, most sectors get noisier and more fragile. Coordination gets harder. Institutional memory thins out. Advantages that looked durable turn out to depend on a few people, a few habits, or a distribution edge that disappears. Infrastructure and culture last longer because they are environments people operate inside, not products to be sold. When learning carries forward, time starts working in your favor. ## Operation In 1984, GM and Toyota opened a joint factory in Fremont, California called NUMMI. Toyota sent over four hundred trainers from Japan for months of side-by-side work with American employees. Absenteeism dropped from twenty percent to two percent. Defect rates fell to the lowest in the United States. GM tried to export the lessons. A vice president told employees to "take a picture of every square inch" of NUMMI and replicate it at other plants. It failed everywhere. The visible process looked the same. The results did not. The missing piece was judgment, and [judgment does not travel well as a memo](@/series/les-circuits-longs/2026-03-03-legibility-kills-what-it-measures/index.md). Toyota had not built a checklist. It had built a way of working. The NUMMI lesson reaches past any specific practice. The value was never in the visible process; it lay in the judgment that accumulates when people have time to learn, when the links between domains stay intact, and when repetition actually improves the work. --- ## Series: Theorem Proving A proof is a program and a theorem is a type. This series builds that idea from Curry-Howard to a tiny Python prover, a mini-Lean in Julia, and real proofs in Lean 4. ### Writing Your First Proofs in Lean *Published: 2026-03-20* > The same three theorems from the Python prover, now in Lean 4. URL: https://federicocarrone.com/series/theorem-proving/lean-tutorial/ The [first article](@/series/theorem-proving/2026-03-18-curry-howard/index.md) in this series explained the Curry-Howard correspondence: propositions are types, proofs are programs. The [second](@/series/theorem-proving/2026-03-19-tiny-theorem-prover-python/index.md) built a tiny theorem prover from scratch in Python. The [third](@/series/theorem-proving/2026-03-20-mini-lean/index.md) embedded the same ideas inside Julia's type system. Now we use the real tool. This article takes the exact same theorems you proved by hand in Python and shows them in Lean 4. You will see what changes and what stays the same: the syntax shifts, the logic does not. ## The Same Three Theorems In the Python article, we proved three things: 1. `A -> A` (if A then A) 2. `A -> B -> A` (if A and B, then A) 3. `(A -> B) -> (B -> C) -> A -> C` (composition of implications) We built each proof as a lambda term and fed it to a 30-line type checker. If the checker accepted the term, the proof was valid. Lean works the same way. The difference is that Lean's kernel is much more powerful, and there are layers of automation on top. But underneath, it is still: construct a term, check its type. ## Proof 1: A -> A In Python, the proof was: ```python identity = Lam("x", A, Var("x")) # inferred type: Arrow(Atom("A"), Atom("A")) ``` In Lean: ``` theorem identity (A : Prop) : A -> A := fun hA => hA ``` Same structure. `fun hA => hA` is a lambda that takes evidence of `A` and returns it. The type `A -> A` is both the function signature and the logical claim. The term is both the program and the proof. In the Python prover, we had to call `infer({}, identity)` to check the proof. In Lean, writing `theorem` triggers the type checker automatically. If the term does not inhabit the claimed type, the file does not compile. ## Proof 2: A -> B -> A In Python: ```python proof = Lam("x", A, Lam("y", B, Var("x"))) # inferred type: Arrow(A, Arrow(B, A)) ``` In Lean: ``` theorem keep_first (A B : Prop) : A -> B -> A := fun hA _hB => hA ``` Ignore the second assumption, return the first. The proof term is identical in structure to the Python version. The only difference is syntax: `fun` instead of `Lam`, `=>` instead of a comma, underscores for unused bindings. The same proof as a **tactic proof**: ``` theorem keep_first_tac (A B : Prop) : A -> B -> A := by intro hA intro _hB exact hA ``` - `intro hA` moves the first assumption from the goal into the local context - `intro _hB` moves the second - `exact hA` closes the goal with a term already in scope The Python prover has no tactic system. You write the proof term or you write nothing. Lean gives you both options: write the term directly, or use tactics to construct it step by step. Tactics scale to longer proofs in ways that term construction does not. ## Proof 3: (A -> B) -> (B -> C) -> A -> C This was the most complex proof in the Python article: ```python compose = Lam( "f", Arrow(A, B), Lam( "g", Arrow(B, C), Lam( "x", A, App(Var("g"), App(Var("f"), Var("x"))) ) ) ) ``` In Lean: ``` theorem compose (A B C : Prop) : (A -> B) -> (B -> C) -> A -> C := fun hAB hBC hA => hBC (hAB hA) ``` Function application in Lean is just juxtaposition: `hAB hA` applies `hAB` to `hA`. No `App(Var("f"), Var("x"))` wrapper needed. The proof is a one-liner instead of a nested tree. The tactic version: ``` theorem compose_tac (A B C : Prop) : (A -> B) -> (B -> C) -> A -> C := by intro hAB hBC hA apply hBC apply hAB exact hA ``` Here `apply` works backwards from the goal. The goal is `C`. `apply hBC` says "I have `B -> C`, so if I can prove `B`, I'm done." Now the goal becomes `B`. `apply hAB` says "I have `A -> B`, so if I can prove `A`, I'm done." Now the goal becomes `A`. `exact hA` closes it. Working backwards from the goal is a tactic idiom that has no analog in the Python prover. It is one of the things that makes Lean proofs more natural than raw term construction. ## What You Just Saw You proved three theorems in two systems. The logic is the same. The checker is the same kind of machine. But Lean gives you: - **Cleaner syntax.** `fun hA => hA` instead of `Lam("x", A, Var("x"))`. - **Tactics.** `intro`, `exact`, `apply` let you construct proofs step by step instead of writing the whole term at once. - **Backwards reasoning.** `apply` lets you work from the goal toward the hypotheses, which is often more natural than forward construction. ## Beyond Implication: Equality and Computation The Python prover handles only implication. It cannot talk about equality, numbers, or computation inside types. Lean can. ``` theorem one_plus_one : 1 + 1 = 2 := by rfl ``` `rfl` means reflexivity: both sides reduce to the same normal form, so equality holds by computation. Lean evaluates `1 + 1` to `2`, sees that both sides are identical, and closes the goal. This is one of the most important ideas in Lean. A lot of proving is not deep reasoning but getting both sides of a statement into forms that Lean's reduction machinery can recognize as equal. In the Julia article, proving 1 + 1 = 2 took four steps of manual axiom chaining. In Lean, `rfl` triggers the kernel's reduction and closes the goal. ## Rewriting When `rfl` is not enough, you can rewrite one equality into another. ``` theorem add_zero_twice (n : Nat) : (n + 0) + 0 = n := by rw [Nat.add_zero] ``` `rw` means: use an equality as a rewrite rule. `Nat.add_zero` is the lemma that `n + 0 = n`. Lean applies it to all matching subterms in the goal, closing it in one step. This is where Lean starts to feel less like programming and more like symbolic transformation. You apply proven equalities to transform the goal. ## Simplification Repeated rewriting gets tedious. `simp` is Lean's general-purpose simplifier. ``` theorem add_zero_twice_simp (n : Nat) : (n + 0) + 0 = n := by simp ``` `simp` applies a large collection of known simplification lemmas automatically. This is the first point where the gap between a toy prover and Lean becomes unmistakable. In the Python prover, you do everything by hand. In the Julia article, every step is a manual axiom chain. In Lean, `simp` handles the routine work. ## Induction For properties that hold for all natural numbers, you need induction. ``` theorem add_zero (n : Nat) : n + 0 = n := by induction n with | zero => rfl | succ n ih => simp ``` `induction n with` splits the proof into: - the `zero` case - the `succ` case, where you get an induction hypothesis `ih` That is mathematical induction in executable form: base case, then inductive step. This is the moment where the earlier articles should click together: - In the Python article, we could not do induction at all. The prover only handled propositional logic. - In the Julia article, induction looked like recursion over `Nat` types, and every step was a manual axiom chain. - In Lean, `induction` is a built-in tactic, and `simp` handles the routine cases. The same idea, but with the mechanical drudgery removed. ## What Makes Lean Feel Different from Programming Working in Lean is less like writing code and more like steering toward a target the tool keeps in front of you. The goal is always visible: you are trying to inhabit one specific type, and at every step Lean prints what remains to prove. The hypotheses in scope usually matter more than the order of commands you typed. And the high-level tactics, `rfl`, `rw`, `simp`, `induction`, all bottom out in a proof term checked against a small kernel, the same architecture as the Python and Julia provers earlier in this series, with far better tooling on top. The mental shift is from tracing how code runs to asking what would count as evidence that a statement holds. ## A Good Way to Learn Learn Lean by reading goals, not by memorizing tactics. 1. Look at the goal. 2. Ask what shape of term would inhabit it. 3. Use tactics only to help construct that term. So when you see: ``` ⊢ A -> B -> A ``` you should think: - This is an implication, so I probably need `intro` - After two `intro`s, I should have `A` in the context - Then `exact` that hypothesis You already know this from the Python article. The proof term is `fun hA _hB => hA`. The tactics are just an interface for building that same term incrementally. ## Where to Go Next After these first proofs, the next things worth learning are: - `cases` for splitting on inductive data - `constructor` for building structured proofs - `have` for introducing intermediate lemmas - `apply` for working backwards from the goal - `omega` for automatically solving linear arithmetic - `simp` and `rw` well enough that arithmetic proofs stop feeling manual At that point Lean becomes pleasant to use. Once you see proofs as typed constructions rather than ceremony, Lean's tactics become predictable. And if you have followed this series from the beginning, you already have that mental model. The Python prover gave you the architecture. The Julia prover gave you the trust boundary. Lean gives you the automation to work at scale. The difficulty is the same as programming: details require precision. --- ### Programming a Mini-Lean in Julia's Type System *Published: 2026-03-19T12:00:00* > Guillermo Angeris builds a working theorem prover in 61 lines of Julia. A tiny trusted kernel, six axioms, and the compiler does the rest. Here is the construction. URL: https://federicocarrone.com/series/theorem-proving/mini-lean/ This article is based on Guillermo Angeris's talk ["Programming a (mini-)Lean in Julia's type system"](https://youtu.be/Bp3kP6mJNqs). A theorem prover, stripped to its engine, is a small trusted kernel, a type checker, and a boundary between the two. Guillermo Angeris answers this by live-coding a toy theorem-proving kernel inside Julia that illustrates how Lean works architecturally. The result is a tiny kernel that makes the trust boundary visible: if you accept the kernel, then anything built on top of it has to pass through the type checker. This is the third article in the [Theorem Proving](@/series/theorem-proving/_index.md) series. The [first article](@/series/theorem-proving/2026-03-18-curry-howard/index.md) covers the Curry-Howard correspondence. The [second](@/series/theorem-proving/2026-03-19-tiny-theorem-prover-python/index.md) implements a tiny prover explicitly in Python. This article embeds those same ideas inside a host language's type system. ## The Architecture You could build a theorem prover by treating proofs as strings and implementing symbol-rewriting rules on top of them. That works, but it fights the host language. Curry-Howard gives a better route for programming languages: make propositions into types, proofs into values, and let the type checker enforce the rules. Every real theorem prover has the same basic structure: 1. A **small trusted kernel** that defines the rules of the game 2. **User proof code** that builds on those rules 3. A **type checker** that enforces the boundary The kernel is the only code you have to trust. Everything outside it, every proof, every theorem, can only combine results that the kernel produced. If the kernel is correct, then any proof that type-checks is correct, no matter how complex. This is sometimes called the **de Bruijn criterion**: make the kernel small enough that a human can read every line, convince themselves it's correct, and then trust everything the system derives. Lean's kernel is a few thousand lines of C++. The mini-Lean's is 61 lines of Julia. ## Building the Kernel: Peano Arithmetic in Julia's Type System Angeris encodes **Peano arithmetic** entirely within Julia's type system. Peano arithmetic is a formal way of defining natural numbers (0, 1, 2, 3, ...) from scratch using two simple rules. ### Natural Numbers as Types Natural numbers are defined using just two rules: 1. **Zero** exists and is a natural number. 2. **Successor**: for any natural number `n`, there exists `S(n)`, which is also a natural number. This gives you: 0 = `Zero`, 1 = `Succ{Zero}`, 2 = `Succ{Succ{Zero}}`, 3 = `Succ{Succ{Succ{Zero}}}`, and so on. Every natural number is either zero or the successor of some other natural number. There is no third option. In Julia, this becomes an abstract type `Nat` with two subtypes: `Zero <: Nat` (a type with no data, just representing the concept of zero) and `Succ{N} <: Nat` where `N` must itself be a `Nat`. So `Succ{Zero}` is 1, `Succ{Succ{Zero}}` is 2, and so on. These exist purely at the type level. Julia never creates runtime values for them. The type checker does all the work during compilation. ### Equality as a Type Equality between two type-level numbers is a type called `Eq{A, B}`. The two type parameters are the two sides of the equation. So the proposition "1 + 1 = 2" is the type `Eq{Add{Succ{Zero}, Succ{Zero}}, Succ{Succ{Zero}}}`. That looks noisy, but all it says is "the sum of 1 and 1 equals 2." Here's the crucial trick: the constructor for `Eq` is *not* exported to users. You can write down the type `Eq{A, B}` (you can *state* a proposition), but you can't create a value of that type directly (you can't *fabricate* a proof). The only way to get an `Eq` value is through the axiom functions that the kernel provides. ### Addition at the Type Level Addition is defined recursively, the same way you'd teach a child to add by counting up: - `Add{Zero, N}` reduces to `N`. Adding zero to anything gives you that thing back. (0 + 5 = 5.) - `Add{Succ{M}, N}` reduces to `Succ{Add{M, N}}`. To add a nonzero number, peel off one, add what's left, then put the one back on top. (3 + 5 = 1 + (2 + 5) = 1 + (1 + (1 + 5)) = 8.) Julia uses **multiple dispatch** to pick the right rule based on the types. The type system evaluates these reductions during compilation. By the time your code runs (if there's anything to run), the types have already been fully computed. ### The Six Axioms The kernel exports exactly six axiom functions. These are the *only* legitimate ways to construct equality proofs: **1. Reflexivity.** For any `n`, you can construct `Eq{n, n}`. Anything equals itself. This is the starting point, the simplest proof that exists. **2. Symmetry.** Given `Eq{a, b}`, produce `Eq{b, a}`. Equality is a two-way street. **3. Transitivity.** Given `Eq{a, b}` and `Eq{b, c}`, produce `Eq{a, c}`. This is the critical chaining axiom. Most proofs are sequences of equalities connected by transitivity: "A = B, and B = C, and C = D, therefore A = D." **4. Successor congruence.** Given `Eq{a, b}`, produce `Eq{Succ{a}, Succ{b}}`. Applying the same operation to both sides of an equality preserves equality. This lets you "lift" a proof through a layer of successor. **5. Addition base case: Zero + n = n.** Produces `Eq{Add{Zero, N}, N}`. This directly encodes the first clause of the recursive definition of addition. **6. Addition recursive case: Succ(m) + n = Succ(m + n).** Produces `Eq{Add{Succ{M}, N}, Succ{Add{M, N}}}`. This encodes the second clause. That's the entire foundation. Six functions plus the type definitions for `Nat`, `Zero`, `Succ`, `Eq`, and `Add`. About 61 lines of Julia with whitespace. These six axiom functions are the only code allowed to create `Eq` values from scratch. Everything outside the kernel can only combine `Eq` values that the axioms produced. The trust boundary from the previous section is now concrete: the 61 lines of kernel are the only code you need to audit. ## Proving 1 + 1 = 2 With the kernel in place, the first goal is to prove that 1 + 1 = 2. Formally, this means constructing a value whose type is: ``` Eq{Add{Succ{Zero}, Succ{Zero}}, Succ{Succ{Zero}}} ``` The proof is a chain of axiom applications: **Step 1.** Apply addition rule 2 (the recursive case). `Succ{Zero} + Succ{Zero}` reduces to `Succ{Zero + Succ{Zero}}`. This gives us an `Eq` between `Add{Succ{Zero}, Succ{Zero}}` and `Succ{Add{Zero, Succ{Zero}}}`. We've peeled off one layer of successor from the left operand. **Step 2.** Apply addition rule 1 (the base case) to the inner term. `Zero + Succ{Zero}` reduces to `Succ{Zero}`. This gives us `Eq{Add{Zero, Succ{Zero}}, Succ{Zero}}`. **Step 3.** Apply successor congruence to step 2. Since `Zero + Succ{Zero} = Succ{Zero}`, wrapping both sides in `Succ` gives us `Succ{Zero + Succ{Zero}} = Succ{Succ{Zero}}`. **Step 4.** Apply transitivity to connect steps 1 and 3. Step 1 says `Succ{Zero} + Succ{Zero} = Succ{Zero + Succ{Zero}}`. Step 3 says `Succ{Zero + Succ{Zero}} = Succ{Succ{Zero}}`. Transitivity chains them: `Succ{Zero} + Succ{Zero} = Succ{Succ{Zero}}`. That's 1 + 1 = 2. The final value has exactly the goal type. Julia's type checker verifies every intermediate step. If you pass the wrong `Eq` to transitivity, or if the types of two chained equalities don't share a common middle term, compilation fails. The proof is verbose and mechanical. That's the point. Every step is independently checkable, and the compiler is the referee. The entire derivation happens during compilation, at the type level. The proof exists purely as a chain of `Eq` values whose types constrain each other into a valid derivation. If you mess up any link in the chain, the code won't compile. The type system catches the error before anything runs, the same way it would catch you trying to add a string to an integer. ## Universal Quantification: For All n, n + 1 = Succ(n) Proving something about all natural numbers requires a different tool: a function type. The statement "for all n, n + 1 = Succ(n)" means: no matter what number you pick, adding 1 gives you the next number. Under Curry-Howard, this becomes a function type: given *any* natural number type `N`, return a value of type `Eq{Add{N, Succ{Zero}}, Succ{N}}`. A specific proof like "1 + 1 = 2" is like testing one input. A universal proof like "for all n, n + 1 = Succ(n)" is like writing a function that passes for *every* input. You're not testing, you're proving. The proof is a function that takes an abstract type parameter `N <: Nat`, not a specific number but a placeholder for any number. Inside the body, Angeris chains the same axioms as before, but now they operate on the generic `N` instead of a concrete `Succ{Zero}`. The key insight: the proof function works by structural recursion on `N`. For `Zero`, you apply the base case of addition. For `Succ{M}`, you use the recursive addition rule, apply the proof recursively for `M`, then use congruence and transitivity to assemble the result. **The fact that this function compiles is the proof.** Julia's type checker has verified that for any possible `N`, the return type is `Eq{Add{N, Succ{Zero}}, Succ{N}}`. You can instantiate it: calling the function with `Succ{Zero}` (the number 1) recovers the specific fact that 1 + 1 = 2. Calling it with `Succ{Succ{Zero}}` gives you 2 + 1 = 3. But the function itself, the generic version, proves the universal statement. "For all" doesn't need special machinery. It's just a generic function, the same concept you already use when you write code that works on "any list" or "any comparable type." ### A Small Julia-Specific Bonus Angeris also shows an extra Julia-specific trick: the subtype lattice ends up reflecting part of the relationship between generic and specific proofs. The real point is still the kernel, the proof terms, and the checker boundary. ## Proving Negation: Zero Does Not Equal One So far, every proof has been about showing two things are equal. But mathematics also needs *negation*, the ability to prove that something is false. How do you represent "0 ≠ 1" as a type? ### False as an Uninhabitable Type The trick is elegant. Define a type called `FalseProp` that has *no public constructor*. It's a type that can never have a value. You can write down the type (you can state "False"), but you can never create a value of it (you can never prove False). Then define "not P" as "a function from P to `FalseProp`." In other words: if assuming P lets you produce a value of `FalseProp`, something went wrong, because `FalseProp` values can't exist. So P must be false. This is how constructive mathematics works: "not P" means "P leads to a contradiction." Julia doesn't have real access control, so this is a convention. Users *could* technically call the internal constructor and cheat. (More on this in the limitations section.) But the disciplined version is: the kernel never exports `FalseProp`'s constructor, so legitimate proofs can never produce one. ### The Last Peano Axiom The final Peano axiom encodes a basic fact about natural numbers: **no successor is zero.** 1 is not 0. 2 is not 0. 47 is not 0. You can always count forward, but you never wrap back around to zero. In the kernel, this axiom is a function with type `Eq{Succ{N}, Zero} → FalseProp` for any `N`. It says: give me a proof that some successor equals zero, and I'll give you a proof of False. Since `FalseProp` values can never be legitimately created, this means no legitimate proof of `Succ(n) = 0` can exist either. The axiom function is allowed to use the internal `FalseProp` constructor because it's part of the trusted kernel. ### Proving 0 ≠ 1 To prove that `Succ{Zero} ≠ Zero`, i.e. 1 ≠ 0, you write a function of type `Eq{Succ{Zero}, Zero} → FalseProp`: 1. Assume a "bad assumption," a hypothetical proof that `Succ{Zero} = Zero`. 2. Feed this assumption into the Peano axiom `succ_ne_zero`. 3. The axiom returns `FalseProp`. The function compiles and type-checks. Its type signature *is* the statement "1 = 0 implies False," which *is* the statement "1 ≠ 0." Proof complete. This is proof by contradiction, one of the oldest techniques in mathematics, encoded as a function. You assume something (1 = 0), show that it leads somewhere impossible (`FalseProp`), and conclude the assumption was wrong. "Impossible" is a type with no values. "Leads to impossible" is a function that returns that type. A complete system would also need the **principle of explosion**: "from False, anything follows." If you somehow had a `FalseProp` value, you could produce a value of *any* type. This sounds absurd, but it's safe because `FalseProp` values can never exist in the first place. ## Induction: Recursion as Proof Mathematical induction is how you prove things about all natural numbers. The idea: if something is true for 0, and being true for any number means it's also true for the next number, then it's true for all numbers. Under Curry-Howard, induction doesn't need to be a separate rule. It *is* recursive programming. An inductive proof is a recursive function that: 1. **Base case**: given `Zero`, return a proof of P(Zero). 2. **Inductive step**: given `Succ{N}` and a proof of P(N) (obtained by recursion), construct a proof of P(Succ{N}). If this function compiles, the induction is valid. The compiler verifies that every case is handled and every return type matches the goal. The universal proof of "for all n, n + 1 = Succ(n)" already uses this pattern. It recurses on the structure of `N`: handle the zero case, then handle the successor case by calling yourself on a smaller number. Induction is just what recursion looks like when your return types carry logical content. Angeris starts coding a standalone `induction` combinator but runs short on time. The idea is a higher-order function (a function that takes other functions as arguments) that accepts a base case proof and a step function, then composes them recursively. This is where Julia starts to creak. In Lean, function return types can depend on argument *values*. That's what "dependent types" means: the type of the output changes based on the input. In Julia, you can simulate this with parametric types and multiple dispatch, but it's indirect. The type system is powerful enough to encode the proofs, but the ergonomics are not designed for it. ## Where Julia Falls Short The mini-Lean works, but Julia was not designed for this, and it shows. ### No Private Constructors Julia has no private fields or constructors. Everything is accessible by name. Angeris demonstrates the problem live by directly typing `Eq{Succ{Zero}, Zero}()`, fabricating a proof that 1 = 0 without going through any axiom. Once you have one false statement, you can derive anything. (That's the principle of explosion from earlier, except now it's working against you.) The entire system collapses if users cheat. In Lean, the kernel is a hard trust boundary with genuinely private constructors, so this can't happen. In Julia, it's an honor system. ### No Tactics, Only Manual Proofs Lean has a tactic language that automates routine proof steps. `ring` solves ring equalities, `simp` simplifies expressions, `omega` handles linear arithmetic. When you need to prove 2 + 2 = 4 in Lean, it's one line. In the mini-Lean, every proof is assembled by hand, axiom by axiom. Extending the 1+1=2 proof to 2+1=3 means more unrolling, more congruences, more transitivities. Angeris hits type errors live and spends minutes debugging, muttering "I have no idea what the hell is wrong here." He gets it working without fully understanding what changed, shrugs, and moves on. This is the best demonstration of what tactic automation buys you. In Lean, you'd write `by omega` and the proof would be done. The tactic system knows how to solve arithmetic equalities automatically. It can verify 100 + 100 = 200 just as easily as 1 + 1 = 2, because it has algorithms instead of manual unrolling. Lean also has `simp` (simplify using known facts), `norm_num` (verified numeric computation), and the ability to write custom tactics. The gap between the mini-Lean and production Lean is almost entirely in automation, not in the foundational ideas. The engine is the same. Lean just has power steering. ### Type Error Messages Are Unhelpful When a proof step fails in Lean, you see your proof state: what you're trying to prove, what you already know, and where things went wrong. In Julia, you get a generic type mismatch error with no context. Debugging means sprinkling `typeof()` calls everywhere and staring at intermediate values. It's printf debugging, but for types. ## Performance: The @generated Trick One practical note: Julia recomputes type-level operations on every call by default. For proofs with deep type nesting (imagine proving something about the number 100, which is `Succ` wrapped 100 times), this gets slow. Julia's `@generated` functions can cache type-level computations so the proof verification only happens once per unique set of type parameters. ## What Production Provers Add This exercise shows what theorem provers are at the kernel level, and what the layers above buy you. ### The Kernel Is the Same Lean, at its core, works exactly like this mini-Lean. Small trusted kernel, untrusted proof code on top, type checker enforcing the boundary. The difference is scale and scope: Lean's kernel handles dependent types, universe polymorphism, and inductive types. The mini-Lean handles Peano arithmetic. But the architecture is the same. Lean's large verified mathematical library all reduces to checks against a kernel of a few thousand lines of C++. ### What Lean Adds on Top | Layer | Mini-Lean | Lean | |-------|-----------|------| | **Kernel** | 61 lines, Peano arithmetic | ~4,000 lines C++, dependent type theory | | **Trust boundary** | Convention (honor system) | Enforced (private constructors) | | **Proof style** | Manual axiom chaining | Tactics (`simp`, `omega`, `ring`, `rw`) | | **Automation** | None | Decision procedures, simplifier, custom tactics | | **Error messages** | Generic type mismatch | Proof state: goals, hypotheses, context | | **Expressiveness** | Natural numbers, equality | Arbitrary mathematics | | **Library** | None | mathlib (large standard library) | | **Tooling** | Julia REPL | VS Code, LSP, widgets, documentation | The automation layer is the biggest gap. Tactics let you work at the level of mathematical reasoning instead of mechanical axiom chaining. `by omega` replaces twenty lines of manual unrolling. `simp [lemma1, lemma2]` replaces long chains of rewrites. Custom tactics let library authors package proof strategies so users don't have to reinvent them. ### The Universe Tower Angeris hints at deeper possibilities. Julia has a `Type` type, the type of types. And `Type` itself has a type (`Type{Type}`), and so on. This creates a hierarchy that type theorists call a **universe tower**, where each level can talk about things at the level below. Why does this matter? Without it, you can create paradoxes. If you allow a "set of all sets," you get Russell's paradox: does the set of all sets that don't contain themselves contain itself? (If it does, it doesn't. If it doesn't, it does.) The universe tower prevents this by saying: types at level 1 can only talk about types at level 0, types at level 2 can only talk about types at level 1, and so on. No level can talk about itself. In Lean, this hierarchy is carefully designed. `Prop` (propositions) lives at one level. `Type` (data types) lives at another. `Type 1` (the universe containing `Type`) lives above that. Julia has hints of the same structure, you can write `Type{Type}` and it's a valid expression, but it's more of an accident of the implementation than a deliberate logical framework. ### Some Host Type Systems Can Do This Too This works in Julia because Julia's type system is unusually rich in the right ways: parametric types, abstract supertypes, and multiple dispatch that can drive type-level reduction. Some other languages can host fragments of the same idea too. Haskell with GADTs can get close. TypeScript can encode surprising pieces of it. But this is not something you get automatically from "having generics." The lesson is specific: once a host type system is expressive enough, you can start smuggling logical reasoning into it. Julia happens to be just strong enough to make the architecture visible. ## Conclusion From scratch, with no macros and no dependencies, Angeris builds a working theorem prover in 61 lines of Julia. The kernel defines natural numbers, equality, addition, and six axioms. From these primitives alone, he proves 1+1=2, proves n+1=Succ(n) for all n, and proves 0≠1. The construction makes visible what is usually hidden: theorem provers are not magic. They are a small trusted kernel, a type checker, and a sharp boundary between the two. Everything else, the tactics, the libraries, the tooling, is engineering built on that foundation. The [next article](@/series/theorem-proving/2026-03-19-lean-tutorial/index.md) in this series puts that engine to work: writing real proofs in Lean, with all the automation the mini-Lean is missing. --- ### Building a Tiny Theorem Prover in Python *Published: 2026-03-19* > A tiny theorem prover is just a term language, a checker, and a small trusted kernel. We build one in plain Python to make the architecture explicit. URL: https://federicocarrone.com/series/theorem-proving/tiny-theorem-prover-python/ The [first article](@/series/theorem-proving/2026-03-18-curry-howard/index.md) in this series explained the Curry-Howard correspondence: propositions are types, proofs are programs. That tells you *why* theorem proving fits so naturally with programming languages. It does not yet tell you what the machine looks like. A theorem prover, concretely, is smaller than most people expect. A tiny theorem prover is just: 1. a language for terms 2. a language for types / propositions 3. a checker that decides whether a term has a type 4. a tiny trusted kernel that defines the legal moves This article builds that architecture in plain Python. We are not abusing Python's own type system. Python is just the implementation language. The prover we build has its *own* terms, its *own* propositions, and its *own* checker. That distinction matters. The goal here is not to show off host-language cleverness. I want the moving parts to be impossible to miss. ## The Smallest Useful Core If you strip theorem proving down to the bone, you do not need arithmetic, tactics, automation, or even a parser. You need a handful of term forms and a handful of typing rules. For a first prover, the right target is: - implication - variables - function abstraction - function application - named assumptions in a context This is enough to represent the simply typed lambda calculus, which under Curry-Howard corresponds to intuitionistic propositional logic. That means we can already represent and verify proofs of statements like: - `A -> A` - `A -> B -> A` - `(A -> B) -> (B -> C) -> A -> C` These are not toy programming examples. They are logical theorems. ## Terms, Propositions, and Contexts We need three datatypes: - **Types**: propositions like `A`, `B`, or `A -> B` - **Terms**: proofs/programs like variables, lambdas, and applications - **Contexts**: the assumptions currently in scope In Python: ```python from dataclasses import dataclass class Type: pass @dataclass(frozen=True) class Atom(Type): name: str @dataclass(frozen=True) class Arrow(Type): left: Type right: Type class Term: pass @dataclass(frozen=True) class Var(Term): name: str @dataclass(frozen=True) class Lam(Term): param: str param_type: Type body: Term @dataclass(frozen=True) class App(Term): fn: Term arg: Term ``` The context is just a mapping from variable names to types: ```python Context = dict[str, Type] ``` That mapping is the whole proof language. Why these constructors? - `Var(x)` means "use the assumption named `x`" - `Lam(x, A, body)` means "assume `x : A`, then prove `body`" - `App(f, a)` means "apply a proof of `A -> B` to a proof of `A` to get a proof of `B`" That is exactly how implication works in natural deduction. ## The Checker Is the Kernel Now we write the only thing that really matters: the checker. ```python def infer(ctx: Context, term: Term) -> Type: if isinstance(term, Var): if term.name not in ctx: raise TypeError(f"unbound variable: {term.name}") return ctx[term.name] if isinstance(term, Lam): new_ctx = dict(ctx) new_ctx[term.param] = term.param_type body_type = infer(new_ctx, term.body) return Arrow(term.param_type, body_type) if isinstance(term, App): fn_type = infer(ctx, term.fn) arg_type = infer(ctx, term.arg) if not isinstance(fn_type, Arrow): raise TypeError(f"attempted to apply non-function: {fn_type}") if fn_type.left != arg_type: raise TypeError( f"function expected {fn_type.left}, got {arg_type}" ) return fn_type.right raise TypeError(f"unknown term: {term}") ``` This is the trusted kernel. If `infer` is correct, then any term it accepts is a valid proof of the type it returns. If `infer` is wrong, the system is unsound. Theorem provers target tiny kernels because every line of this function is code you have to trust. ## What the Rules Mean Logically Each branch of `infer` is one proof rule. ### Variables ```python infer(ctx, Var("x")) == ctx["x"] ``` Logically, this means: - if `x : A` is in your assumptions - then you may conclude `A` That is the rule of assumption. ### Lambdas ```python infer(ctx, Lam("x", A, body)) == Arrow(A, body_type) ``` Logically: - assume `A` - under that assumption, derive `B` - therefore derive `A -> B` That is implication introduction. ### Applications ```python infer(ctx, App(f, a)) ``` Logically: - if `f` proves `A -> B` - and `a` proves `A` - then `f(a)` proves `B` That is implication elimination, better known as modus ponens. The checker enforces logical inference rules directly. ## First Proof: A -> A The identity function is the simplest proof in the system: ```python A = Atom("A") identity = Lam("x", A, Var("x")) print(infer({}, identity)) ``` The inferred type is: ```python Arrow(Atom("A"), Atom("A")) ``` Under Curry-Howard, that means the term proves `A -> A`. `fun x => x` is both: - a program that returns its input - a proof that if `A` holds, then `A` holds The same term inhabits both interpretations. ## A Slightly Less Trivial Proof: A -> B -> A Now prove that if `A` holds and `B` holds, then `A` holds. ```python A = Atom("A") B = Atom("B") proof = Lam("x", A, Lam("y", B, Var("x"))) print(infer({}, proof)) ``` The inferred type is: ```python Arrow(A, Arrow(B, A)) ``` Logically, that is: `A -> B -> A` The proof is just "ignore the second assumption and return the first one." Ordinary functional programming and formal proof are the same structure seen two ways. ## Composition: (A -> B) -> (B -> C) -> A -> C Now something that feels like actual reasoning: ```python A = Atom("A") B = Atom("B") C = Atom("C") compose = Lam( "f", Arrow(A, B), Lam( "g", Arrow(B, C), Lam( "x", A, App(Var("g"), App(Var("f"), Var("x"))) ) ) ) print(infer({}, compose)) ``` The checker returns: ```python Arrow(Arrow(A, B), Arrow(Arrow(B, C), Arrow(A, C))) ``` This is proof as composition of evidence: - `f` turns evidence of `A` into evidence of `B` - `g` turns evidence of `B` into evidence of `C` - so together they turn evidence of `A` into evidence of `C` In logic, that is a theorem. In programming, it is function composition. The same term carries both readings. ## What This Tiny Prover Cannot Yet Do At this point we have a real prover, but it is extremely small. It cannot yet express: - conjunction (`A and B`) - disjunction (`A or B`) - quantifiers over values - equality - natural numbers - induction - computation inside types That is fine. The first implementation exists to make the kernel architecture legible. Even in this tiny form, you can already see the whole pattern: - terms are proofs - types are propositions - checking is proof verification Everything more advanced is an extension of that base. ## The Next Useful Extension: Pairs If you wanted to grow this prover one step, the best next move would be conjunction. Add: - a `PairType(A, B)` - a `PairTerm(a, b)` - projections `Fst` and `Snd` Then the checker gets three more rules: - if `a : A` and `b : B`, then `(a, b) : A and B` - if `p : A and B`, then `fst(p) : A` - if `p : A and B`, then `snd(p) : B` That would let you prove statements like: - `A -> B -> (A and B)` - `(A and B) -> A` - `(A and B) -> B` The architecture would not change. You would just add more term forms and more rules. That is the recurring theme in theorem provers: small, explicit rules compose into powerful systems. ## What Lean Adds Beyond This The toy checker here and Lean are the same kind of machine, but Lean adds several layers you immediately miss once you try to prove anything nontrivial: - **Dependent types**: types can mention values - **Definitional equality**: terms can reduce during checking - **Inductive types**: natural numbers, lists, trees, equality, all built into the core theory - **Elaboration**: Lean fills in omitted arguments, infers implicits, and resolves notation - **Tactics**: you work at the level of proof state instead of raw lambda terms - **Automation**: simplifiers, arithmetic solvers, rewriting, decision procedures - **A real trust boundary**: private constructors, kernel isolation, no honor system But none of that changes the central picture. There is still a kernel. There are still terms. There are still types. There is still a checker. That is why building even a tiny prover is so clarifying. It separates the parts that make the logic work from the parts that make the system pleasant to use. ## Why Python Is the Right Language for This Article Python's type system is too weak for the Julia-style trick of making the host language do the proving. That is exactly why it is useful here. In this article, the prover is explicit: - the terms are your dataclasses - the propositions are your dataclasses - the checker is your function - the trust boundary is code you can point at with one finger Nothing is hidden in the host language. That makes Python a good teaching language for the architecture, even though it would be a bad language for host-type-system metaprogramming. ## The Real Lesson The distance from ordinary interpreter code to a theorem prover kernel is much smaller than most programmers think. You start with: - a syntax - a few inference rules - a checker that enforces them That is enough to get a real notion of proof. Lean is the same architecture, pushed further and engineered better. The [next article](@/series/theorem-proving/2026-03-20-mini-lean/index.md) shows the complementary perspective: instead of implementing the prover explicitly, it embeds a tiny one inside Julia's own type system. Same ideas, different lesson. --- ### Propositions Are Types, Proofs Are Programs *Published: 2026-03-18* > The Curry-Howard correspondence says that types and logical propositions are the same thing. Understanding why changes how you think about both programming and mathematics. URL: https://federicocarrone.com/series/theorem-proving/curry-howard/ In the 1930s, Haskell Curry noticed something strange. He was working on combinatory logic, a system for manipulating abstract functions, and he realized that the rules governing his combinators looked identical to the rules of a logical system called intuitionistic propositional logic. It was as if he'd found two different maps of the same territory. Three decades later, William Howard found the same thing in a richer setting. He showed that the simply typed lambda calculus, the foundation of functional programming, corresponds precisely to natural deduction, a standard system of logical proof. Every type corresponds to a proposition. Every program corresponds to a proof. Every function corresponds to an implication. This is the **Curry-Howard correspondence**: a structural identity between proofs and programs. ## What Is Mathematics? Mathematics looks like calculation from the outside: multiply these numbers, solve for x, compute an integral. But calculation is to mathematics what typing is to programming: a basic skill, not the thing itself. Mathematics is the study of what must be true given a set of starting assumptions. You pick your axioms, the statements you accept without proof (like "zero is a natural number" or "every natural number has a successor"). Then you apply rules of inference to derive new statements. If you follow the rules exactly, your conclusions are guaranteed to be true within that system. Not probably true. Not true according to experiment. Logically, necessarily true. A **proof** is what makes this work. A proof is a chain of steps, each one justified by a rule, starting from axioms and ending at the thing you want to show. Think of it like a board game. You don't argue about whether a chess move is legal. Either the piece can go there or it can't. A proof is a complete record of legal moves from the starting position to the final claim. This has a remarkable consequence: proofs are mechanical. You don't need intuition or genius to *check* a proof. You just need to verify that each step follows from the rules. A machine could do it. In fact, that's exactly what interactive theorem provers do. Systems like Lean, Coq, and Agda let you write mathematical proofs as code. If the code compiles, the proof is valid. If it doesn't, you made a mistake, and the system tells you where. These tools are not toys. Lean's math library contains a very large body of formally verified mathematics. Coq was used to build CompCert, a verified C compiler used in aerospace. The seL4 microkernel, verified in Isabelle/HOL, runs in military helicopters. HACL\*, a verified cryptographic library, ships in Firefox and Linux. - **Propositions** are statements that are either true or false: "1 + 1 = 2", "every even number greater than 2 is the sum of two primes" - **Proofs** are step-by-step derivations that a proposition is true - **Axioms** are the starting propositions you accept without proof - **Rules of inference** tell you how to derive new truths from existing ones (like: if you know A and you know "A implies B," you can conclude B) - A proof is valid when every step follows from the rules, and checking that is mechanical. Mathematics is a rule-following game precise enough for a machine to verify. ## What Is a Type, Really? Most programmers first meet types as labels: `int`, `string`, `bool`, `List`. That is useful, but it hides the interesting part. A type is a claim about what values are allowed and what operations make sense on them. When you write `x: int`, you are making a claim: "x will always be an integer." The compiler holds you to it. A type is a **constraint the compiler verifies before your code runs.** Simple types express simple constraints: `int` means "this is an integer." More complex types express more complex constraints: - `List` means "a list where every element is a string" - `(String) -> Int` means "a function that takes a string and returns an integer" - In Rust, `&'a str` means "a string reference that is guaranteed to be valid for lifetime `'a`" The compiler checks all of these *before* the program runs. If the constraints do not hold, the code will not compile. Every time the compiler accepts your code, it has *proven* something. When Rust accepts a program, it has proven there are no dangling pointers, no data races, no use-after-free bugs. These are real theorems about your program's behavior, verified at compile time. The Rust compiler is a specialized theorem prover that does not call itself one. But not every type system can express every claim. Rust's types can talk about memory ownership and lifetimes, but they can't state "every even number greater than 2 is the sum of two primes." TypeScript's types can talk about object shapes, but not about sorting correctness. These languages expose *fragments* of a deeper idea: types are machine-checked claims. Proof assistants like Lean take that idea all the way, so the claims can talk about mathematics itself, not just strings, lists, or lifetimes. Behind this sits a formal model of computation, just like logic sits behind mathematical proof. The one that matters most here is the lambda calculus: variables, functions, and function application. When you write ordinary code, you are already working in descendants of that tradition. Curry-Howard matters because it connects that world of typed programs to the world of formal proofs. ## Two Formal Worlds So we have two formal worlds: | | Mathematics | Computation | |---|---|---| | **Objects** | Propositions | Types | | **Evidence** | Proofs | Programs (values) | | **Rules** | Rules of inference | Typing rules | | **Checking** | Proof verification | Type checking | | **Foundation** | Formal logic | Lambda calculus | Both are rule-following games. Both have precise foundations from the 1930s. Both can be checked mechanically. Curry-Howard says they are the same game. ## The Correspondence The table above is the high-level view. Curry-Howard makes each row precise and mechanical: | Logic | Programming | |-------|-------------| | A statement that might be true or false | A type | | A proof that a statement is true | A value of that type | | "If A then B" | A function from type A to type B | | "Both A and B" | A pair (A, B) | | "Either A or B" | A tagged union / sum type | | "For all x, P(x) is true" | A generic function that works for any x | | "False" (a contradiction) | A type with no values | This goes deeper than shared notation. The rules that make proofs valid and the rules that make programs type-check are the same rules, discovered independently in two different fields. ### Statements Are Types The math statement "1 + 1 = 2" corresponds to a specific type. If you can create a value of that type by following the rules, the statement is true. If no such value can exist, the statement is false. In a proof assistant like Lean, you'd write the type as `1 + 1 = 2` directly. In the tiny prover we'll build in the [next article](@/series/theorem-proving/2026-03-19-tiny-theorem-prover-python/index.md), it will look more explicit and more mechanical. The notation differs. The idea is the same: the proposition *is* the type. ### Proofs Are Values In programming, you prove a type "exists" by constructing a value of it. `42` proves that the type `Int` is inhabited. `[1, 2, 3]` proves that `List` is inhabited. Similarly, if you can construct a value of a proposition-type using only legitimate rules, you've proven the proposition. This is why the correspondence is exact, not approximate. A proof isn't a story about why something is true. It's a concrete object, a term that the type checker can inspect, decompose, and verify mechanically. ### Functions Are Implications A function from type A to type B says: "give me evidence of A, and I'll produce evidence of B." That's exactly what "if A then B" means in logic. The function body is the derivation. If you have a function `f: A -> B` and you have a value `a: A`, then `f(a)` gives you a value of type `B`. In logic: if you know "A implies B" and you know "A," then you know "B." That's *modus ponens*, one of the oldest rules of inference, and it's literally function application. ### Pairs Are Conjunctions A pair `(a, b)` where `a: A` and `b: B` is evidence that both A and B hold. To construct it, you need evidence of each. To use it, you can project out either component. This matches exactly how conjunction (logical "and") works: to prove "A and B," prove both; given "A and B," you can conclude either. ### Sum Types Are Disjunctions A tagged union (like Rust's `enum` or Haskell's `Either`) that holds either a value of type A or a value of type B is evidence that at least one of A or B holds. To construct it, you need evidence of one. To use it, you must handle both cases. This is logical "or": to prove "A or B," prove one of them; given "A or B," reason by cases. ### Generic Functions Are Universal Statements "For all n, P(n) is true" just means: write a function that takes *any* `n` and returns a value of type `P(n)`. If the function compiles for all inputs, you've proven the universal statement. No special quantifier machinery needed. It's just generic programming, the same thing you do when you write a function that works on any list regardless of element type. ### False Is an Empty Type A proposition that can never be proven corresponds to a type with no values. You can write down the type (you can *state* the proposition), but you can never construct a value of it (you can never *prove* it). "Not P" then means "a function from P to the empty type": if assuming P lets you produce a value of an uninhabited type, something went wrong, and P must be false. This is proof by contradiction encoded as a function. ## Why This Matters ### Compilers Are Already Proving Theorems This means type-checking, the thing compilers already do when they reject `"hello" + 3`, is a form of proof verification. When Rust accepts your code, it has proven memory safety. When Haskell type-checks code with GADTs, it has proven that your invariants hold. When a verified smart contract passes its type checker, properties like "this function never transfers more tokens than the balance" are established at compile time. The mechanism is the same across all these systems. The difference is scope. Rust proves a fixed set of memory properties. Lean proves whatever you ask it to. ### Proof Assistants Are Programming Languages Lean, Coq, and Agda are not separate from the programming world. They are programming languages where the type system is expressive enough to state arbitrary mathematical propositions. "Prove this theorem" means "construct a value of this type." The tools, tactics, and automation are all about making that construction easier, but underneath, it's the same activity as writing a program that type-checks. ### The Boundary Between Programming and Mathematics Is Artificial The distinction between programming and proving is artificial. A sorting function that returns a value of type "a sorted list plus a proof that it's a permutation of the input" is doing both at once: computing and proving. Dependently typed languages like Lean, Agda, and Idris let you write programs that carry their correctness proofs with them. ## Scope and Limits The correspondence is exact in the right formal systems (the simply typed lambda calculus corresponds to intuitionistic propositional logic, System F corresponds to second-order logic, and so on). But real programming languages are messy. Most have features that break the correspondence: - **General recursion.** If a language lets you write infinite loops, you can "prove" anything: a non-terminating function of type `A -> B` technically has the right type but produces no evidence. Proof assistants require termination to keep the logic sound. - **Exceptions and effects.** Throwing an exception from inside a function of type `A -> B` means you never actually produce a `B`. Side effects similarly weaken the logical interpretation. - **Unsound escape hatches.** Rust's `unsafe`, Haskell's `unsafePerformIO`, TypeScript's `any` all break the correspondence by letting you bypass the type checker. This is why proof assistants are strict: no general recursion without termination proofs, no uncontrolled effects, no escape hatches. The restrictions that make them harder to program in are exactly the restrictions that keep the logic sound. Everyday programming languages expose fragments of Curry-Howard. Proof assistants are what you get when you take it all the way. ## Where to Go From Here The [next article](@/series/theorem-proving/2026-03-19-tiny-theorem-prover-python/index.md) builds a tiny theorem prover from scratch in Python. You'll see the exact architecture: terms, types, a checker, and a trust boundary you can point at with one finger. After that, the [Julia article](@/series/theorem-proving/2026-03-20-mini-lean/index.md) shows the same ideas embedded inside a host language's type system, where the compiler itself becomes the proof checker. ## Further Reading - [Fede's Guide to Type Systems](@/articles/2026-01-01-type-systems/index.md). If you want more background on how type systems work in practice, from generics to dependent types, before diving into the proofs. - "Propositions as Types" by Philip Wadler. The best overview of the correspondence, its history, and why it keeps showing up. Available free online, and as a [conference talk](https://www.youtube.com/watch?v=IOiZatlZtGU) (Strange Loop). - "Types and Programming Languages" by Benjamin Pierce. The standard textbook. Chapters 9-11 cover the simply typed lambda calculus; chapter 30 covers the correspondence. - "The Little Typer" by Friedman and Christiansen. A gentle, Socratic introduction to dependent types, where Curry-Howard becomes the central design principle. - "Software Foundations" (free online). An interactive, proof-based introduction using Coq. --- ## Talks ### Blockspace Forum Cannes - Fede and Justin Drake *Published: 2026-04-22* > Discussion with Justin Drake at Blockspace Forum Cannes URL: https://federicocarrone.com/talks/blockspace-forum-cannes-fede-and-justin-drake/ --- ### Next 10 Years of Ethereum *Published: 2025-11-15* > Talk at Devconnect about the next 10 years of Ethereum URL: https://federicocarrone.com/talks/next-10-years-of-ethereum/ --- ### Ethereum's Native Rollup Roadmap with Justin Drake *Published: 2025-10-01* > Discussion about Ethereum's native rollup roadmap URL: https://federicocarrone.com/talks/2025-podcast-with-justin-drake/ --- ## About ### About > Federico Carrone. Building formally verified infrastructure, writing about what compounds over time. URL: https://federicocarrone.com/about/ --- ## Keywords ### acceleration - [Dissolution Without Construction](https://federicocarrone.com/series/les-circuits-longs/dissolution-without-construction/): Series (Les Circuits Longs) ### agents - [Nutrition Labels for Trust](https://federicocarrone.com/series/concrete/nutrition-labels-for-trust/): Series (Concrete) ### ai - [China is trying to commoditize the complement](https://federicocarrone.com/articles/china-commoditizing-the-complement/): Article - [A Fact-Producing Compiler](https://federicocarrone.com/series/concrete/a-fact-producing-compiler/): Series (Concrete) - [When the Compiler Is the Oracle](https://federicocarrone.com/series/concrete/when-the-compiler-is-the-oracle/): Series (Concrete) - [Designing a Programming Language for the AI Era](https://federicocarrone.com/series/concrete/ai-wont-save-your-language-unless-you-design-for-it/): Series (Concrete) - [Legibility Kills What It Measures](https://federicocarrone.com/series/les-circuits-longs/legibility-kills-what-it-measures/): Series (Les Circuits Longs) - [Friction as Luxury: What We Lose When AI Gives Us What We Want](https://federicocarrone.com/series/les-circuits-longs/friction-as-luxury/): Series (Les Circuits Longs) ### AQR - [The Tail Hedge Debate: Spitznagel Is Right, AQR Is Answering the Wrong Question](https://federicocarrone.com/series/leptokurtic/the-tail-hedge-debate-spitznagel-is-right/): Series (Leptokurtic) ### argentina - [Thank You to the Argentine Senate](https://federicocarrone.com/articles/senate-scientific-merit/): Article ### artificial life - [Self-Replicating Programs Emerge from Random Noise](https://federicocarrone.com/articles/computational-life/): Article ### attractors - [Life as a Double Fixed Point](https://federicocarrone.com/series/fixed-point-ladder/life-as-a-double-fixed-point/): Series (When Rules Repeat: The Fixed Point Ladder) - [Fixed Points and Attractors](https://federicocarrone.com/series/fixed-point-ladder/fixed-points-and-attractors/): Series (When Rules Repeat: The Fixed Point Ladder) ### autocorrelation - [Why the Calm Is Dangerous](https://federicocarrone.com/series/edge-of-chaos/why-the-calm-is-dangerous/): Series (The Edge of Chaos) ### avalanches - [Sandpiles and Crashes: How Systems Tune Themselves to the Brink](https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/): Series (The Edge of Chaos) ### backtesting - [The Tail Hedge Debate: Spitznagel Is Right, AQR Is Answering the Wrong Question](https://federicocarrone.com/series/leptokurtic/the-tail-hedge-debate-spitznagel-is-right/): Series (Leptokurtic) ### Banach - [Fixed Points and Attractors](https://federicocarrone.com/series/fixed-point-ladder/fixed-points-and-attractors/): Series (When Rules Repeat: The Fixed Point Ladder) ### biology - [Life as a Double Fixed Point](https://federicocarrone.com/series/fixed-point-ladder/life-as-a-double-fixed-point/): Series (When Rules Repeat: The Fixed Point Ladder) ### bitcoin - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### black swans - [Faster Than Exponential: Can You See a Crash Coming?](https://federicocarrone.com/series/edge-of-chaos/faster-than-exponential-can-you-see-a-crash-coming/): Series (The Edge of Chaos) ### Bouchaud - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) - [Reflexivity by the Numbers](https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/): Series (The Edge of Chaos) - [Sandpiles and Crashes: How Systems Tune Themselves to the Brink](https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/): Series (The Edge of Chaos) - [Crashes Without a Cause: Markets as Phase Transitions](https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/): Series (The Edge of Chaos) - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### branching ratio - [Reflexivity by the Numbers](https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/): Series (The Edge of Chaos) ### bubbles - [Faster Than Exponential: Can You See a Crash Coming?](https://federicocarrone.com/series/edge-of-chaos/faster-than-exponential-can-you-see-a-crash-coming/): Series (The Edge of Chaos) ### building - [Discipline Without Love Optimizes for the Wrong Variable](https://federicocarrone.com/articles/discipline-without-love/): Article - [Thank You to the Argentine Senate](https://federicocarrone.com/articles/senate-scientific-merit/): Article ### capabilities - [Nutrition Labels for Trust](https://federicocarrone.com/series/concrete/nutrition-labels-for-trust/): Series (Concrete) ### capitalism - [Discipline Without Love Optimizes for the Wrong Variable](https://federicocarrone.com/articles/discipline-without-love/): Article ### category theory - [Logic, Self-Reference, and Category Theory](https://federicocarrone.com/series/fixed-point-ladder/logic-self-reference-and-category-theory/): Series (When Rules Repeat: The Fixed Point Ladder) ### central limit theorem - [Power Laws, Extremistan, and Non-Ergodicity](https://federicocarrone.com/series/fixed-point-ladder/power-laws-extremistan-and-non-ergodicity/): Series (When Rules Repeat: The Fixed Point Ladder) ### chaos - [Chaos, Fractals, and Renormalization](https://federicocarrone.com/series/fixed-point-ladder/chaos-fractals-and-renormalization/): Series (When Rules Repeat: The Fixed Point Ladder) ### commodity prices - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### community - [Thank You to the Argentine Senate](https://federicocarrone.com/articles/senate-scientific-merit/): Article ### compilers - [A Fact-Producing Compiler](https://federicocarrone.com/series/concrete/a-fact-producing-compiler/): Series (Concrete) - [When the Compiler Is the Oracle](https://federicocarrone.com/series/concrete/when-the-compiler-is-the-oracle/): Series (Concrete) ### computational biology - [Self-Replicating Programs Emerge from Random Noise](https://federicocarrone.com/articles/computational-life/): Article ### concrete - [Nutrition Labels for Trust](https://federicocarrone.com/series/concrete/nutrition-labels-for-trust/): Series (Concrete) - [A Fact-Producing Compiler](https://federicocarrone.com/series/concrete/a-fact-producing-compiler/): Series (Concrete) - [What Concrete Makes Worse](https://federicocarrone.com/series/concrete/what-concrete-makes-worse/): Series (Concrete) - [When the Compiler Is the Oracle](https://federicocarrone.com/series/concrete/when-the-compiler-is-the-oracle/): Series (Concrete) - [Can I prove Concrete programs in Lean?](https://federicocarrone.com/series/concrete/proving-systems-code-in-lean/): Series (Concrete) - [Designing a Programming Language for the AI Era](https://federicocarrone.com/series/concrete/ai-wont-save-your-language-unless-you-design-for-it/): Series (Concrete) - [Why Concrete Exists](https://federicocarrone.com/series/concrete/the-concrete-programming-language-systems-programming-for-formal-reasoning/): Series (Concrete) ### consciousness - [Dissolution Without Construction](https://federicocarrone.com/series/les-circuits-longs/dissolution-without-construction/): Series (Les Circuits Longs) - [The Death of the Inner Self](https://federicocarrone.com/series/les-circuits-longs/the-death-of-the-inner-self/): Series (Les Circuits Longs) ### continued fractions - [Number Theory and the Arithmetic of Recurrence](https://federicocarrone.com/series/fixed-point-ladder/number-theory-and-recurrence/): Series (When Rules Repeat: The Fixed Point Ladder) ### coordination - [Crypto doctrine](https://federicocarrone.com/articles/crypto-doctrine/): Article - [Dissolution Without Construction](https://federicocarrone.com/series/les-circuits-longs/dissolution-without-construction/): Series (Les Circuits Longs) - [The Death of the Inner Self](https://federicocarrone.com/series/les-circuits-longs/the-death-of-the-inner-self/): Series (Les Circuits Longs) ### crash detection - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### crash prediction - [Faster Than Exponential: Can You See a Crash Coming?](https://federicocarrone.com/series/edge-of-chaos/faster-than-exponential-can-you-see-a-crash-coming/): Series (The Edge of Chaos) ### crashes - [Sandpiles and Crashes: How Systems Tune Themselves to the Brink](https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/): Series (The Edge of Chaos) - [Crashes Without a Cause: Markets as Phase Transitions](https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/): Series (The Edge of Chaos) ### critical points - [Faster Than Exponential: Can You See a Crash Coming?](https://federicocarrone.com/series/edge-of-chaos/faster-than-exponential-can-you-see-a-crash-coming/): Series (The Edge of Chaos) ### critical slowing down - [Why the Calm Is Dangerous](https://federicocarrone.com/series/edge-of-chaos/why-the-calm-is-dangerous/): Series (The Edge of Chaos) ### criticality - [Markets as Reflexive Fixed Points](https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/): Series (When Rules Repeat: The Fixed Point Ladder) - [Life as a Double Fixed Point](https://federicocarrone.com/series/fixed-point-ladder/life-as-a-double-fixed-point/): Series (When Rules Repeat: The Fixed Point Ladder) ### crypto - [Crypto doctrine](https://federicocarrone.com/articles/crypto-doctrine/): Article - [The new financial backend of the world](https://federicocarrone.com/series/ethereum/the-new-financial-backend-of-the-world/): Series (Ethereum) - [The missing institution of the Internet](https://federicocarrone.com/series/ethereum/the-missing-institution-of-the-internet/): Series (Ethereum) ### cryptography - [CommitLLM: How to Verify an LLM Inference](https://federicocarrone.com/articles/commitllm/): Article ### culture - [Crypto doctrine](https://federicocarrone.com/articles/crypto-doctrine/): Article - [Legibility Kills What It Measures](https://federicocarrone.com/series/les-circuits-longs/legibility-kills-what-it-measures/): Series (Les Circuits Longs) - [Friction as Luxury: What We Lose When AI Gives Us What We Want](https://federicocarrone.com/series/les-circuits-longs/friction-as-luxury/): Series (Les Circuits Longs) - [Notes on permanence, time, and ergodicity](https://federicocarrone.com/series/les-circuits-longs/notes-on-culture-infrastructure-time-and-ergodicity/): Series (Les Circuits Longs) ### currency crisis - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### curry-howard - [Programming a Mini-Lean in Julia's Type System](https://federicocarrone.com/series/theorem-proving/mini-lean/): Series (Theorem Proving) - [Building a Tiny Theorem Prover in Python](https://federicocarrone.com/series/theorem-proving/tiny-theorem-prover-python/): Series (Theorem Proving) - [Propositions Are Types, Proofs Are Programs](https://federicocarrone.com/series/theorem-proving/curry-howard/): Series (Theorem Proving) ### Daníelsson - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) ### DFA - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### Diophantine approximation - [Number Theory and the Arithmetic of Recurrence](https://federicocarrone.com/series/fixed-point-ladder/number-theory-and-recurrence/): Series (When Rules Repeat: The Fixed Point Ladder) ### discipline - [Discipline Without Love Optimizes for the Wrong Variable](https://federicocarrone.com/articles/discipline-without-love/): Article ### discrete choice - [Crashes Without a Cause: Markets as Phase Transitions](https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/): Series (The Edge of Chaos) ### DNA - [Life as a Double Fixed Point](https://federicocarrone.com/series/fixed-point-ladder/life-as-a-double-fixed-point/): Series (When Rules Repeat: The Fixed Point Ladder) ### dragon kings - [Faster Than Exponential: Can You See a Crash Coming?](https://federicocarrone.com/series/edge-of-chaos/faster-than-exponential-can-you-see-a-crash-coming/): Series (The Edge of Chaos) ### dynamical systems - [Fixed Points and Attractors](https://federicocarrone.com/series/fixed-point-ladder/fixed-points-and-attractors/): Series (When Rules Repeat: The Fixed Point Ladder) ### early-warning signals - [Why the Calm Is Dangerous](https://federicocarrone.com/series/edge-of-chaos/why-the-calm-is-dangerous/): Series (The Edge of Chaos) ### economy - [The new financial backend of the world](https://federicocarrone.com/series/ethereum/the-new-financial-backend-of-the-world/): Series (Ethereum) - [The missing institution of the Internet](https://federicocarrone.com/series/ethereum/the-missing-institution-of-the-internet/): Series (Ethereum) ### efficiency - [Sandpiles and Crashes: How Systems Tune Themselves to the Brink](https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/): Series (The Edge of Chaos) ### Embrechts - [The Limits of Knowing](https://federicocarrone.com/series/edge-of-chaos/the-limits-of-knowing/): Series (The Edge of Chaos) ### emergence - [Self-Replicating Programs Emerge from Random Noise](https://federicocarrone.com/articles/computational-life/): Article ### endogeneity - [Reflexivity by the Numbers](https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/): Series (The Edge of Chaos) ### endogenous markets - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) ### endogenous risk - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) ### ergodic - [Notes on permanence, time, and ergodicity](https://federicocarrone.com/series/les-circuits-longs/notes-on-culture-infrastructure-time-and-ergodicity/): Series (Les Circuits Longs) ### ergodicity - [Markets as Reflexive Fixed Points](https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/): Series (When Rules Repeat: The Fixed Point Ladder) ### ergodicity economics - [Finance Is Geometry, and It All Comes Back to Jensen’s Inequality](https://federicocarrone.com/series/leptokurtic/at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/): Series (Leptokurtic) ### errors on errors - [The Limits of Knowing](https://federicocarrone.com/series/edge-of-chaos/the-limits-of-knowing/): Series (The Edge of Chaos) ### ethereum - [A Proof Is Only as Good as Its Spec](https://federicocarrone.com/articles/formal-verification-moves-trust/): Article - [Crypto doctrine](https://federicocarrone.com/articles/crypto-doctrine/): Article - [The new financial backend of the world](https://federicocarrone.com/series/ethereum/the-new-financial-backend-of-the-world/): Series (Ethereum) - [The missing institution of the Internet](https://federicocarrone.com/series/ethereum/the-missing-institution-of-the-internet/): Series (Ethereum) ### excess volatility - [Crashes Without a Cause: Markets as Phase Transitions](https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/): Series (The Edge of Chaos) ### exchange rates - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### extreme value theory - [The Limits of Knowing](https://federicocarrone.com/series/edge-of-chaos/the-limits-of-knowing/): Series (The Edge of Chaos) - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### Extremistan - [Power Laws, Extremistan, and Non-Ergodicity](https://federicocarrone.com/series/fixed-point-ladder/power-laws-extremistan-and-non-ergodicity/): Series (When Rules Repeat: The Fixed Point Ladder) ### fat tails - [The Limits of Knowing](https://federicocarrone.com/series/edge-of-chaos/the-limits-of-knowing/): Series (The Edge of Chaos) - [Sandpiles and Crashes: How Systems Tune Themselves to the Brink](https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/): Series (The Edge of Chaos) - [Crashes Without a Cause: Markets as Phase Transitions](https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/): Series (The Edge of Chaos) - [Power Laws, Extremistan, and Non-Ergodicity](https://federicocarrone.com/series/fixed-point-ladder/power-laws-extremistan-and-non-ergodicity/): Series (When Rules Repeat: The Fixed Point Ladder) - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### Feigenbaum - [Chaos, Fractals, and Renormalization](https://federicocarrone.com/series/fixed-point-ladder/chaos-fractals-and-renormalization/): Series (When Rules Repeat: The Fixed Point Ladder) ### finance - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### financial stability - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) ### finite-time singularity - [Faster Than Exponential: Can You See a Crash Coming?](https://federicocarrone.com/series/edge-of-chaos/faster-than-exponential-can-you-see-a-crash-coming/): Series (The Edge of Chaos) ### fitness - [Fede's Guide to a Healthier Life](https://federicocarrone.com/articles/health-guide/): Article ### fixed points - [Fixed Points and Attractors](https://federicocarrone.com/series/fixed-point-ladder/fixed-points-and-attractors/): Series (When Rules Repeat: The Fixed Point Ladder) ### flash crash - [Reflexivity by the Numbers](https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/): Series (The Edge of Chaos) ### flows - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) ### forex - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### formal verification - [A Proof Is Only as Good as Its Spec](https://federicocarrone.com/articles/formal-verification-moves-trust/): Article - [Can I prove Concrete programs in Lean?](https://federicocarrone.com/series/concrete/proving-systems-code-in-lean/): Series (Concrete) - [Writing Your First Proofs in Lean](https://federicocarrone.com/series/theorem-proving/lean-tutorial/): Series (Theorem Proving) - [Programming a Mini-Lean in Julia's Type System](https://federicocarrone.com/series/theorem-proving/mini-lean/): Series (Theorem Proving) - [Building a Tiny Theorem Prover in Python](https://federicocarrone.com/series/theorem-proving/tiny-theorem-prover-python/): Series (Theorem Proving) - [Propositions Are Types, Proofs Are Programs](https://federicocarrone.com/series/theorem-proving/curry-howard/): Series (Theorem Proving) ### formal-verification - [Nutrition Labels for Trust](https://federicocarrone.com/series/concrete/nutrition-labels-for-trust/): Series (Concrete) - [A Fact-Producing Compiler](https://federicocarrone.com/series/concrete/a-fact-producing-compiler/): Series (Concrete) - [The Rust Effects Debate and Concrete's Case for a Smaller Language](https://federicocarrone.com/series/concrete/rusts-grand-vision-and-concretes-answer/): Series (Concrete) - [Why Concrete Exists](https://federicocarrone.com/series/concrete/the-concrete-programming-language-systems-programming-for-formal-reasoning/): Series (Concrete) ### fractals - [Chaos, Fractals, and Renormalization](https://federicocarrone.com/series/fixed-point-ladder/chaos-fractals-and-renormalization/): Series (When Rules Repeat: The Fixed Point Ladder) ### functional programming - [Fede's Guide to Type Systems: From Generics to Dependent Types](https://federicocarrone.com/articles/type-systems/): Article ### futbol - [Solving Futbol Like Chess](https://federicocarrone.com/articles/solving-futbol-like-chess/): Article ### Gabaix - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) ### geopolitics - [China is trying to commoditize the complement](https://federicocarrone.com/articles/china-commoditizing-the-complement/): Article ### gold - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### goodhart - [Solving Futbol Like Chess](https://federicocarrone.com/articles/solving-futbol-like-chess/): Article ### Gödel - [Logic, Self-Reference, and Category Theory](https://federicocarrone.com/series/fixed-point-ladder/logic-self-reference-and-category-theory/): Series (When Rules Repeat: The Fixed Point Ladder) ### Hawkes process - [Reflexivity by the Numbers](https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/): Series (The Edge of Chaos) ### health - [Fede's Guide to a Healthier Life](https://federicocarrone.com/articles/health-guide/): Article ### herding - [Crashes Without a Cause: Markets as Phase Transitions](https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/): Series (The Edge of Chaos) ### homeostasis - [Life as a Double Fixed Point](https://federicocarrone.com/series/fixed-point-ladder/life-as-a-double-fixed-point/): Series (When Rules Repeat: The Fixed Point Ladder) ### Hurst - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### Hutchinson operator - [Chaos, Fractals, and Renormalization](https://federicocarrone.com/series/fixed-point-ladder/chaos-fractals-and-renormalization/): Series (When Rules Repeat: The Fixed Point Ladder) ### Hénon map - [Chaos, Fractals, and Renormalization](https://federicocarrone.com/series/fixed-point-ladder/chaos-fractals-and-renormalization/): Series (When Rules Repeat: The Fixed Point Ladder) ### individuality - [Dissolution Without Construction](https://federicocarrone.com/series/les-circuits-longs/dissolution-without-construction/): Series (Les Circuits Longs) - [The Death of the Inner Self](https://federicocarrone.com/series/les-circuits-longs/the-death-of-the-inner-self/): Series (Les Circuits Longs) ### induction - [Writing Your First Proofs in Lean](https://federicocarrone.com/series/theorem-proving/lean-tutorial/): Series (Theorem Proving) ### inelastic markets - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) ### inference - [CommitLLM: How to Verify an LLM Inference](https://federicocarrone.com/articles/commitllm/): Article ### infrastructure - [China is trying to commoditize the complement](https://federicocarrone.com/articles/china-commoditizing-the-complement/): Article - [Notes on permanence, time, and ergodicity](https://federicocarrone.com/series/les-circuits-longs/notes-on-culture-infrastructure-time-and-ergodicity/): Series (Les Circuits Longs) ### interest rates - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### invariant tori - [Number Theory and the Arithmetic of Recurrence](https://federicocarrone.com/series/fixed-point-ladder/number-theory-and-recurrence/): Series (When Rules Repeat: The Fixed Point Ladder) ### Ising model - [Crashes Without a Cause: Markets as Phase Transitions](https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/): Series (The Edge of Chaos) ### Jensen's inequality - [Finance Is Geometry, and It All Comes Back to Jensen’s Inequality](https://federicocarrone.com/series/leptokurtic/at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/): Series (Leptokurtic) ### julia - [Programming a Mini-Lean in Julia's Type System](https://federicocarrone.com/series/theorem-proving/mini-lean/): Series (Theorem Proving) ### KAM - [Number Theory and the Arithmetic of Recurrence](https://federicocarrone.com/series/fixed-point-ladder/number-theory-and-recurrence/): Series (When Rules Repeat: The Fixed Point Ladder) ### Kauffman - [Life as a Double Fixed Point](https://federicocarrone.com/series/fixed-point-ladder/life-as-a-double-fixed-point/): Series (When Rules Repeat: The Fixed Point Ladder) ### Kelly - [Markets as Reflexive Fixed Points](https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/): Series (When Rules Repeat: The Fixed Point Ladder) - [Power Laws, Extremistan, and Non-Ergodicity](https://federicocarrone.com/series/fixed-point-ladder/power-laws-extremistan-and-non-ergodicity/): Series (When Rules Repeat: The Fixed Point Ladder) ### Kelly criterion - [Finance Is Geometry, and It All Comes Back to Jensen’s Inequality](https://federicocarrone.com/series/leptokurtic/at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/): Series (Leptokurtic) ### Koijen - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) ### language-design - [What Concrete Makes Worse](https://federicocarrone.com/series/concrete/what-concrete-makes-worse/): Series (Concrete) ### latent liquidity - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) ### law of large numbers - [The Limits of Knowing](https://federicocarrone.com/series/edge-of-chaos/the-limits-of-knowing/): Series (The Edge of Chaos) ### Lawvere - [Logic, Self-Reference, and Category Theory](https://federicocarrone.com/series/fixed-point-ladder/logic-self-reference-and-category-theory/): Series (When Rules Repeat: The Fixed Point Ladder) ### lean - [A Proof Is Only as Good as Its Spec](https://federicocarrone.com/articles/formal-verification-moves-trust/): Article - [Can I prove Concrete programs in Lean?](https://federicocarrone.com/series/concrete/proving-systems-code-in-lean/): Series (Concrete) - [The Rust Effects Debate and Concrete's Case for a Smaller Language](https://federicocarrone.com/series/concrete/rusts-grand-vision-and-concretes-answer/): Series (Concrete) - [Why Concrete Exists](https://federicocarrone.com/series/concrete/the-concrete-programming-language-systems-programming-for-formal-reasoning/): Series (Concrete) - [Writing Your First Proofs in Lean](https://federicocarrone.com/series/theorem-proving/lean-tutorial/): Series (Theorem Proving) - [Programming a Mini-Lean in Julia's Type System](https://federicocarrone.com/series/theorem-proving/mini-lean/): Series (Theorem Proving) ### lean 4 - [Writing Your First Proofs in Lean](https://federicocarrone.com/series/theorem-proving/lean-tutorial/): Series (Theorem Proving) ### legibility - [Legibility Kills What It Measures](https://federicocarrone.com/series/les-circuits-longs/legibility-kills-what-it-measures/): Series (Les Circuits Longs) ### linear-types - [What Concrete Makes Worse](https://federicocarrone.com/series/concrete/what-concrete-makes-worse/): Series (Concrete) ### llm - [CommitLLM: How to Verify an LLM Inference](https://federicocarrone.com/articles/commitllm/): Article - [A Fact-Producing Compiler](https://federicocarrone.com/series/concrete/a-fact-producing-compiler/): Series (Concrete) - [Designing a Programming Language for the AI Era](https://federicocarrone.com/series/concrete/ai-wont-save-your-language-unless-you-design-for-it/): Series (Concrete) ### log-periodic power law - [Faster Than Exponential: Can You See a Crash Coming?](https://federicocarrone.com/series/edge-of-chaos/faster-than-exponential-can-you-see-a-crash-coming/): Series (The Edge of Chaos) ### logarithms - [Finance Is Geometry, and It All Comes Back to Jensen’s Inequality](https://federicocarrone.com/series/leptokurtic/at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/): Series (Leptokurtic) ### logic - [Building a Tiny Theorem Prover in Python](https://federicocarrone.com/series/theorem-proving/tiny-theorem-prover-python/): Series (Theorem Proving) - [Propositions Are Types, Proofs Are Programs](https://federicocarrone.com/series/theorem-proving/curry-howard/): Series (Theorem Proving) ### logistic map - [Fixed Points and Attractors](https://federicocarrone.com/series/fixed-point-ladder/fixed-points-and-attractors/): Series (When Rules Repeat: The Fixed Point Ladder) ### longevity - [Fede's Guide to a Healthier Life](https://federicocarrone.com/articles/health-guide/): Article ### love - [Discipline Without Love Optimizes for the Wrong Variable](https://federicocarrone.com/articles/discipline-without-love/): Article ### LPPLS - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### macroprudential regulation - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) ### Mantegna - [Markets as Reflexive Fixed Points](https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/): Series (When Rules Repeat: The Fixed Point Ladder) ### market microstructure - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) ### markets - [Markets as Reflexive Fixed Points](https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/): Series (When Rules Repeat: The Fixed Point Ladder) ### medieval finance - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### Minsky - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) - [Why the Calm Is Dangerous](https://federicocarrone.com/series/edge-of-chaos/why-the-calm-is-dangerous/): Series (The Edge of Chaos) ### momentum - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### monoculture - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) ### Newton method - [Fixed Points and Attractors](https://federicocarrone.com/series/fixed-point-ladder/fixed-points-and-attractors/): Series (When Rules Repeat: The Fixed Point Ladder) ### non-ergodicity - [Power Laws, Extremistan, and Non-Ergodicity](https://federicocarrone.com/series/fixed-point-ladder/power-laws-extremistan-and-non-ergodicity/): Series (When Rules Repeat: The Fixed Point Ladder) ### number theory - [Number Theory and the Arithmetic of Recurrence](https://federicocarrone.com/series/fixed-point-ladder/number-theory-and-recurrence/): Series (When Rules Repeat: The Fixed Point Ladder) ### nutrition - [Fede's Guide to a Healthier Life](https://federicocarrone.com/articles/health-guide/): Article ### open data - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### optimization - [Solving Futbol Like Chess](https://federicocarrone.com/articles/solving-futbol-like-chess/): Article - [When the Compiler Is the Oracle](https://federicocarrone.com/series/concrete/when-the-compiler-is-the-oracle/): Series (Concrete) ### options - [The Tail Hedge Debate: Spitznagel Is Right, AQR Is Answering the Wrong Question](https://federicocarrone.com/series/leptokurtic/the-tail-hedge-debate-spitznagel-is-right/): Series (Leptokurtic) ### Per Bak - [Sandpiles and Crashes: How Systems Tune Themselves to the Brink](https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/): Series (The Edge of Chaos) ### personal - [Discipline Without Love Optimizes for the Wrong Variable](https://federicocarrone.com/articles/discipline-without-love/): Article - [Thank You to the Argentine Senate](https://federicocarrone.com/articles/senate-scientific-merit/): Article ### Peters - [Finance Is Geometry, and It All Comes Back to Jensen’s Inequality](https://federicocarrone.com/series/leptokurtic/at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/): Series (Leptokurtic) ### phase transitions - [Crashes Without a Cause: Markets as Phase Transitions](https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/): Series (The Edge of Chaos) ### philosophy - [Legibility Kills What It Measures](https://federicocarrone.com/series/les-circuits-longs/legibility-kills-what-it-measures/): Series (Les Circuits Longs) - [Friction as Luxury: What We Lose When AI Gives Us What We Want](https://federicocarrone.com/series/les-circuits-longs/friction-as-luxury/): Series (Les Circuits Longs) - [Notes on permanence, time, and ergodicity](https://federicocarrone.com/series/les-circuits-longs/notes-on-culture-infrastructure-time-and-ergodicity/): Series (Les Circuits Longs) ### power laws - [Sandpiles and Crashes: How Systems Tune Themselves to the Brink](https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/): Series (The Edge of Chaos) - [Markets as Reflexive Fixed Points](https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/): Series (When Rules Repeat: The Fixed Point Ladder) - [Power Laws, Extremistan, and Non-Ergodicity](https://federicocarrone.com/series/fixed-point-ladder/power-laws-extremistan-and-non-ergodicity/): Series (When Rules Repeat: The Fixed Point Ladder) ### preasymptotics - [The Limits of Knowing](https://federicocarrone.com/series/edge-of-chaos/the-limits-of-knowing/): Series (The Edge of Chaos) ### price impact - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) ### profit - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### programming languages - [A Proof Is Only as Good as Its Spec](https://federicocarrone.com/articles/formal-verification-moves-trust/): Article - [Fede's Guide to Type Systems: From Generics to Dependent Types](https://federicocarrone.com/articles/type-systems/): Article ### programming-languages - [Nutrition Labels for Trust](https://federicocarrone.com/series/concrete/nutrition-labels-for-trust/): Series (Concrete) - [A Fact-Producing Compiler](https://federicocarrone.com/series/concrete/a-fact-producing-compiler/): Series (Concrete) - [What Concrete Makes Worse](https://federicocarrone.com/series/concrete/what-concrete-makes-worse/): Series (Concrete) - [When the Compiler Is the Oracle](https://federicocarrone.com/series/concrete/when-the-compiler-is-the-oracle/): Series (Concrete) - [Designing a Programming Language for the AI Era](https://federicocarrone.com/series/concrete/ai-wont-save-your-language-unless-you-design-for-it/): Series (Concrete) - [The Rust Effects Debate and Concrete's Case for a Smaller Language](https://federicocarrone.com/series/concrete/rusts-grand-vision-and-concretes-answer/): Series (Concrete) - [Why Concrete Exists](https://federicocarrone.com/series/concrete/the-concrete-programming-language-systems-programming-for-formal-reasoning/): Series (Concrete) ### proofs - [Can I prove Concrete programs in Lean?](https://federicocarrone.com/series/concrete/proving-systems-code-in-lean/): Series (Concrete) ### python - [Building a Tiny Theorem Prover in Python](https://federicocarrone.com/series/theorem-proving/tiny-theorem-prover-python/): Series (Theorem Proving) ### recursive types - [Logic, Self-Reference, and Category Theory](https://federicocarrone.com/series/fixed-point-ladder/logic-self-reference-and-category-theory/): Series (When Rules Repeat: The Fixed Point Ladder) ### reflexivity - [Reflexivity by the Numbers](https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/): Series (The Edge of Chaos) - [Markets as Reflexive Fixed Points](https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/): Series (When Rules Repeat: The Fixed Point Ladder) ### renormalization - [Chaos, Fractals, and Renormalization](https://federicocarrone.com/series/fixed-point-ladder/chaos-fractals-and-renormalization/): Series (When Rules Repeat: The Fixed Point Ladder) ### resonance - [Number Theory and the Arithmetic of Recurrence](https://federicocarrone.com/series/fixed-point-ladder/number-theory-and-recurrence/): Series (When Rules Repeat: The Fixed Point Ladder) ### revenue - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### risk models - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) ### robustness - [The Limits of Knowing](https://federicocarrone.com/series/edge-of-chaos/the-limits-of-knowing/): Series (The Edge of Chaos) ### rust - [Fede's Guide to Type Systems: From Generics to Dependent Types](https://federicocarrone.com/articles/type-systems/): Article - [The Rust Effects Debate and Concrete's Case for a Smaller Language](https://federicocarrone.com/series/concrete/rusts-grand-vision-and-concretes-answer/): Series (Concrete) - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### sandpile model - [Sandpiles and Crashes: How Systems Tune Themselves to the Brink](https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/): Series (The Edge of Chaos) ### Scheffer - [Why the Calm Is Dangerous](https://federicocarrone.com/series/edge-of-chaos/why-the-calm-is-dangerous/): Series (The Edge of Chaos) ### self-organized criticality - [Sandpiles and Crashes: How Systems Tune Themselves to the Brink](https://federicocarrone.com/series/edge-of-chaos/sandpiles-and-crashes-how-systems-tune-themselves-to-the-brink/): Series (The Edge of Chaos) - [Crashes Without a Cause: Markets as Phase Transitions](https://federicocarrone.com/series/edge-of-chaos/crashes-without-a-cause-markets-as-phase-transitions/): Series (The Edge of Chaos) ### self-reference - [Logic, Self-Reference, and Category Theory](https://federicocarrone.com/series/fixed-point-ladder/logic-self-reference-and-category-theory/): Series (When Rules Repeat: The Fixed Point Ladder) ### self-replication - [Self-Replicating Programs Emerge from Random Noise](https://federicocarrone.com/articles/computational-life/): Article ### self-reproduction - [Life as a Double Fixed Point](https://federicocarrone.com/series/fixed-point-ladder/life-as-a-double-fixed-point/): Series (When Rules Repeat: The Fixed Point Ladder) ### silver - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### sleep - [Fede's Guide to a Healthier Life](https://federicocarrone.com/articles/health-guide/): Article ### Sornette - [Reflexivity by the Numbers](https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/): Series (The Edge of Chaos) - [Faster Than Exponential: Can You See a Crash Coming?](https://federicocarrone.com/series/edge-of-chaos/faster-than-exponential-can-you-see-a-crash-coming/): Series (The Edge of Chaos) - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### Soros - [Reflexivity by the Numbers](https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/): Series (The Edge of Chaos) - [Markets as Reflexive Fixed Points](https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/): Series (When Rules Repeat: The Fixed Point Ladder) ### sovereign debt - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### Spitznagel - [Finance Is Geometry, and It All Comes Back to Jensen’s Inequality](https://federicocarrone.com/series/leptokurtic/at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/): Series (Leptokurtic) - [The Tail Hedge Debate: Spitznagel Is Right, AQR Is Answering the Wrong Question](https://federicocarrone.com/series/leptokurtic/the-tail-hedge-debate-spitznagel-is-right/): Series (Leptokurtic) ### SPY - [The Tail Hedge Debate: Spitznagel Is Right, AQR Is Answering the Wrong Question](https://federicocarrone.com/series/leptokurtic/the-tail-hedge-debate-spitznagel-is-right/): Series (Leptokurtic) ### square-root law - [What Actually Moves Prices](https://federicocarrone.com/series/edge-of-chaos/what-actually-moves-prices/): Series (The Edge of Chaos) ### stability is destabilizing - [Why the Calm Is Dangerous](https://federicocarrone.com/series/edge-of-chaos/why-the-calm-is-dangerous/): Series (The Edge of Chaos) ### Stanley - [Markets as Reflexive Fixed Points](https://federicocarrone.com/series/fixed-point-ladder/markets-as-reflexive-fixed-points/): Series (When Rules Repeat: The Fixed Point Ladder) ### supplements - [Fede's Guide to a Healthier Life](https://federicocarrone.com/articles/health-guide/): Article ### systemic risk - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) ### systems - [Solving Futbol Like Chess](https://federicocarrone.com/articles/solving-futbol-like-chess/): Article ### systems programming - [Can I prove Concrete programs in Lean?](https://federicocarrone.com/series/concrete/proving-systems-code-in-lean/): Series (Concrete) ### tacit knowledge - [Legibility Kills What It Measures](https://federicocarrone.com/series/les-circuits-longs/legibility-kills-what-it-measures/): Series (Les Circuits Longs) ### tactics - [Solving Futbol Like Chess](https://federicocarrone.com/articles/solving-futbol-like-chess/): Article - [Writing Your First Proofs in Lean](https://federicocarrone.com/series/theorem-proving/lean-tutorial/): Series (Theorem Proving) ### tail hedging - [Finance Is Geometry, and It All Comes Back to Jensen’s Inequality](https://federicocarrone.com/series/leptokurtic/at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/): Series (Leptokurtic) - [The Tail Hedge Debate: Spitznagel Is Right, AQR Is Answering the Wrong Question](https://federicocarrone.com/series/leptokurtic/the-tail-hedge-debate-spitznagel-is-right/): Series (Leptokurtic) ### Taleb - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) - [The Limits of Knowing](https://federicocarrone.com/series/edge-of-chaos/the-limits-of-knowing/): Series (The Edge of Chaos) - [Finance Is Geometry, and It All Comes Back to Jensen’s Inequality](https://federicocarrone.com/series/leptokurtic/at-the-core-of-finance-lies-geometry-in-the-end-its-all-jensens-inequality/): Series (Leptokurtic) - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### technology - [China is trying to commoditize the complement](https://federicocarrone.com/articles/china-commoditizing-the-complement/): Article - [Dissolution Without Construction](https://federicocarrone.com/series/les-circuits-longs/dissolution-without-construction/): Series (Les Circuits Longs) ### theorem proving - [Writing Your First Proofs in Lean](https://federicocarrone.com/series/theorem-proving/lean-tutorial/): Series (Theorem Proving) - [Programming a Mini-Lean in Julia's Type System](https://federicocarrone.com/series/theorem-proving/mini-lean/): Series (Theorem Proving) - [Building a Tiny Theorem Prover in Python](https://federicocarrone.com/series/theorem-proving/tiny-theorem-prover-python/): Series (Theorem Proving) - [Propositions Are Types, Proofs Are Programs](https://federicocarrone.com/series/theorem-proving/curry-howard/): Series (Theorem Proving) ### tipping points - [Why the Calm Is Dangerous](https://federicocarrone.com/series/edge-of-chaos/why-the-calm-is-dangerous/): Series (The Edge of Chaos) ### tradeoffs - [What Concrete Makes Worse](https://federicocarrone.com/series/concrete/what-concrete-makes-worse/): Series (Concrete) ### trust - [Nutrition Labels for Trust](https://federicocarrone.com/series/concrete/nutrition-labels-for-trust/): Series (Concrete) ### trusted-computing-base - [Nutrition Labels for Trust](https://federicocarrone.com/series/concrete/nutrition-labels-for-trust/): Series (Concrete) ### Turing - [Logic, Self-Reference, and Category Theory](https://federicocarrone.com/series/fixed-point-ladder/logic-self-reference-and-category-theory/): Series (When Rules Repeat: The Fixed Point Ladder) ### turing completeness - [Self-Replicating Programs Emerge from Random Noise](https://federicocarrone.com/articles/computational-life/): Article ### type systems - [Fede's Guide to Type Systems: From Generics to Dependent Types](https://federicocarrone.com/articles/type-systems/): Article - [The Rust Effects Debate and Concrete's Case for a Smaller Language](https://federicocarrone.com/series/concrete/rusts-grand-vision-and-concretes-answer/): Series (Concrete) - [Why Concrete Exists](https://federicocarrone.com/series/concrete/the-concrete-programming-language-systems-programming-for-formal-reasoning/): Series (Concrete) - [Programming a Mini-Lean in Julia's Type System](https://federicocarrone.com/series/theorem-proving/mini-lean/): Series (Theorem Proving) ### type theory - [Building a Tiny Theorem Prover in Python](https://federicocarrone.com/series/theorem-proving/tiny-theorem-prover-python/): Series (Theorem Proving) - [Propositions Are Types, Proofs Are Programs](https://federicocarrone.com/series/theorem-proving/curry-howard/): Series (Theorem Proving) ### VaR - [When Risk Models Create Risk](https://federicocarrone.com/series/edge-of-chaos/when-risk-models-create-risk/): Series (The Edge of Chaos) ### variance - [Solving Futbol Like Chess](https://federicocarrone.com/articles/solving-futbol-like-chess/): Article - [Why the Calm Is Dangerous](https://federicocarrone.com/series/edge-of-chaos/why-the-calm-is-dangerous/): Series (The Edge of Chaos) ### verification - [CommitLLM: How to Verify an LLM Inference](https://federicocarrone.com/articles/commitllm/): Article ### volatility - [The Tail Hedge Debate: Spitznagel Is Right, AQR Is Answering the Wrong Question](https://federicocarrone.com/series/leptokurtic/the-tail-hedge-debate-spitznagel-is-right/): Series (Leptokurtic) - [Twenty Centuries of Financial Data: What 240 Countries and 2,000 Years Reveal](https://federicocarrone.com/series/leptokurtic/twenty-centuries-of-financial-data/): Series (Leptokurtic) ### volatility clustering - [Reflexivity by the Numbers](https://federicocarrone.com/series/edge-of-chaos/reflexivity-by-the-numbers/): Series (The Edge of Chaos) ### volatility targeting - [Detecting Crashes with Fat-Tail Statistics](https://federicocarrone.com/series/leptokurtic/detecting-crashes-with-fat-tail-statistics/): Series (Leptokurtic) ### Waddington - [Life as a Double Fixed Point](https://federicocarrone.com/series/fixed-point-ladder/life-as-a-double-fixed-point/): Series (When Rules Repeat: The Fixed Point Ladder) ### Y combinator - [Logic, Self-Reference, and Category Theory](https://federicocarrone.com/series/fixed-point-ladder/logic-self-reference-and-category-theory/): Series (When Rules Repeat: The Fixed Point Ladder) --- ## Anime & Animation Anime and animation recommendations from Federico Carrone URL: https://federicocarrone.com/anime/ ### Anime - Cowboy Bebop | rating 8.9 URL: https://www.imdb.com/title/tt0213338/ Style as substance. Every episode is a meditation on being unable to escape the past, wrapped in jazz and noir. The ending is perfect. - Ghost in the Shell: Stand Alone Complex | rating 8.5 URL: https://www.imdb.com/title/tt0346314/ The film asked the philosophical questions. SAC builds the world where those questions acquire policy implications. The Laughing Man arc remains one of fiction's best treatments of information warfare. - Berserk | rating 8.7 URL: https://www.imdb.com/title/tt0318871/ Ambition, betrayal, and what it costs to impose your will on the world. The Golden Age arc is one of fiction's great tragedies. - Ghost in the Shell | rating 8.0 URL: https://www.imdb.com/title/tt0113568/ The original questions about consciousness and identity in a networked world. Still more philosophically serious than most AI discourse today. - Legend of the Galactic Heroes: Die Neue These | rating 7.8 URL: https://www.imdb.com/title/tt7407236/ Democracy versus autocracy argued honestly, with neither side caricatured. Closer to Thucydides than to space opera. - Monster | rating 8.7 URL: https://www.imdb.com/title/tt0434706/ A doctor chasing a serial killer across post-reunification Europe. Institutional corruption, moral responsibility, and the question of whether evil is systemic or individual. - Akira | rating 8.0 URL: https://www.imdb.com/title/tt0094625/ Power without institutions to contain it. Visually unmatched four decades later. The animation alone changed what the medium could be. ### Animation - Arcane | rating 9.0 URL: https://www.imdb.com/title/tt11126994/ Class conflict, institutional failure, and what happens when the people with nothing to lose get access to power. The animation sets a new standard. - BoJack Horseman | rating 8.8 URL: https://www.imdb.com/title/tt3398228/ The most honest show about self-destruction I have seen. It refuses to let its protagonist off the hook, which is rare and necessary. - Daria | rating 7.8 URL: https://www.imdb.com/title/tt0118298/ Deadpan intelligence against a world that rewards conformity. Still relevant decades later, which says something about the world. - Gravity Falls | rating 8.9 URL: https://www.imdb.com/title/tt1865718/ Deceptively smart. Mystery, conspiracy, and real emotional weight hidden inside a children's show. It knew exactly when to end. - Rick and Morty | rating 9.1 URL: https://www.imdb.com/title/tt2861424/ Nihilism played for laughs until it stops being funny. The best episodes land because they take the consequences of infinite possibility seriously. ### Anime - Attack on Titan | rating 9.1 URL: https://www.imdb.com/title/tt2560140/ The best exploration of freedom, sovereignty, and the price of survival I have seen in any medium. The political arcs in the final seasons are more honest about power than most prestige drama. - Frieren: Beyond Journey's End | rating 8.9 URL: https://www.imdb.com/title/tt22248376/ A fantasy about time rather than urgency. An elf who outlives everyone she knows learns too late what human connection meant. Melancholy, restraint, and emotional seriousness without melodrama. - Cyberpunk: Edgerunners | rating 8.3 URL: https://www.imdb.com/title/tt12590266/ What happens when a system is designed to grind people down and someone decides not to comply. Studio Trigger at their most kinetic and tragic. ### Animation - Samurai Jack | rating 8.5 URL: https://www.imdb.com/title/tt0278238/ Pure visual storytelling. Entire episodes with almost no dialogue that work better than most scripts. Patience as aesthetic principle. - Love, Death & Robots | rating 8.4 URL: https://www.imdb.com/title/tt9561862/ Short-form science fiction that takes animation seriously as a medium. Beyond the Aquila Rift and Zima Blue do more in fifteen minutes than most feature films. - Final Space | rating 8.2 URL: https://www.imdb.com/title/tt6317068/ Starts as absurd comedy and quietly becomes one of the most emotionally committed animated shows. The tonal shift works because it was always there. --- ## Anime & Animation Backlog Anime and animation I want to watch URL: https://federicocarrone.com/anime/#backlog ### New & Upcoming - Fool Night | 2026 URL: https://www.themoviedb.org/tv/326247 Upcoming adaptation of Kasumi Yasuda's manga. In a sunless, dying Tokyo, the poor can undergo transfloration: a conversion into plants that spend the rest of their short existence as living resources for everyone else. Bodily autonomy, class, and slow death treated as an economic system. - Mobile Suit Gundam: Hathaway - The Sorcery of Nymph Circe | 2026 URL: https://www.themoviedb.org/movie/910850 The second Hathaway film, continuing Hathaway Noa's turn to terrorism against a corrupt Earth Federation. The rare Gundam story that sits with the guilt of political violence rather than resolving it. Sunrise at its most adult and most beautiful. - Cyberpunk: Edgerunners 2 | 2026 URL: https://www.themoviedb.org/tv/326788 Studio Trigger and CD Projekt Red return to Night City for a new standalone story. The first season was a self-contained tragedy about a system that grinds up the people who try to climb it. Trigger's animation aimed straight at the body. - The Ghost in the Shell | 2026 URL: https://www.themoviedb.org/tv/255358 Science SARU's new adaptation, closer to Shirow Masamune's original manga than any previous version. Motoko Kusanagi and Section 9 again, but reset for a studio with a radically different visual sensibility. The question of what a self is once the body is optional, redrawn. ### Priority Backlog - Legend of the Galactic Heroes | 1988 | rating 9.0 URL: https://www.imdb.com/title/tt0096633/ The original 110-episode OVA. Democracy versus autocracy across a galactic war, with neither side caricatured. Political philosophy as space opera. - Houseki no Kuni | 2017 | rating 8.1 URL: https://www.imdb.com/title/tt7321070/ Gem people who shatter and reassemble, losing memories with each reconstruction. Identity as something that erodes rather than develops. Full 3DCG by Studio Orange with a visual language unlike anything else in anime. - Mushishi | 2005 | rating 8.8 URL: https://www.imdb.com/title/tt0807832/ Contemplative, episodic, and patient. A wandering specialist studies organisms that exist beyond human understanding. About accepting what cannot be controlled. - Patlabor 2: The Movie | 1993 | rating 7.5 URL: https://www.imdb.com/title/tt0124770/ Mamoru Oshii uses a mecha frame to make a political thriller about legitimacy, militarization, and the fantasy of peace outsourced to others. Less about robots than about what liberal order forgets about force until force returns. - Planetes | 2003 | rating 8.5 URL: https://www.imdb.com/title/tt0816398/ Hard sci-fi about orbital debris collectors. Work, class, and institutional neglect of infrastructure. The most LambdaClass anime on any list. - Neon Genesis Evangelion | 1995 | rating 8.5 URL: https://www.imdb.com/title/tt0112159/ The most influential anime of the last 30 years. Institutional dysfunction, individual psychology, and the impossibility of piloting a machine designed to save humanity when you cannot save yourself. - Serial Experiments Lain | 1998 | rating 8.1 URL: https://www.imdb.com/title/tt0500092/ Identity dissolving into networks. A girl discovers she exists more fully online than in reality. Adjacent to the Death of the Inner Self essay. Prophetic about the internet's effect on selfhood. - Perfect Blue | 1997 | rating 8.6 URL: https://www.imdb.com/title/tt0156887/ Satoshi Kon on identity fracture. A pop singer turned actress loses the boundary between performance and self. Influenced Black Swan and half of modern psychological horror. - Millennium Actress | 2001 | rating 8.6 URL: https://www.imdb.com/title/tt0291350/ Satoshi Kon. Time, memory, and how narrative shapes identity. An actress's life and her film roles blur until the distinction stops mattering. - Paranoia Agent | 2004 | rating 8.4 URL: https://www.imdb.com/title/tt0433722/ Satoshi Kon's TV series. Collective delusion, social pressure, and institutional failure in modern Japan. A mysterious attacker that may be a shared hallucination. - Pluto | 2023 | rating 8.1 URL: https://www.imdb.com/title/tt26737616/ Urasawa reimagines an Astro Boy arc as a detective thriller. Robots, war trauma, and the question of whether artificial beings can grieve. From the creator of Monster. - Tatami Galaxy | 2010 | rating 8.5 URL: https://www.imdb.com/title/tt1607949/ Yuasa. A college student relives his university years choosing different paths each time, learning that the optimal choice does not exist. Connects to ergodic thinking about sample paths. - Kaiji: Ultimate Survivor | 2007 | rating 8.4 URL: https://www.imdb.com/title/tt1087727/ Game theory under existential stakes. A man drowning in debt enters underground gambling games where the system is rigged. Decision-making when survival is the only metric. - Shinsekai Yori | 2012 | rating 8.4 URL: https://www.imdb.com/title/tt2419314/ A society built on suppressing dangerous knowledge to maintain stability. What happens when the system that protects you is also what oppresses you. Slow, unsettling, and philosophically serious. - Shouwa Genroku Rakugo Shinjuu | 2016 | rating 8.4 URL: https://www.imdb.com/title/tt5419278/ A dying art form inside dying institutions. A convicted criminal learns rakugo from a master whose entire world is disappearing. About tradition, performance, and what survives when the structures that sustained culture collapse. - Vinland Saga | 2019 | rating 8.8 URL: https://www.imdb.com/title/tt10233448/ Ambition, violence, and the question of whether a person built by war can choose peace. Thematically close to Berserk but with a redemption arc. - Jin-Roh: The Wolf Brigade | 1999 | rating 7.3 URL: https://www.imdb.com/title/tt0193253/ Written by Oshii, directed by Okiura. An alternate-history Japan where state police suppress dissent. A soldier begins to question which side he serves. Political thriller about identity, loyalty, and the machinery of state violence. - Angel's Egg | 1985 | rating 7.5 URL: https://www.imdb.com/title/tt0208502/ Oshii at his most abstract. A girl carrying a giant egg and a soldier wander through a dead city. No plot in the conventional sense, just atmosphere, silence, and religious imagery dissolving into ambiguity. - Royal Space Force: Wings of Honneamise | 1987 | rating 7.2 URL: https://www.imdb.com/title/tt0093207/ A slacker in a neglected space program volunteers to be the first man in orbit. Gainax's debut film. World-building treated as seriously as any novel, and the launch sequence is one of animation's great scenes. - Gunbuster | 1988 | rating 7.8 URL: https://www.imdb.com/title/tt0098492/ Anno before Evangelion. A girl pilots a giant robot against alien invaders, but the real subject is time dilation and what it costs to save a world you can never return to. Six episodes, no waste. - Haibane Renmei | 2002 | rating 7.9 URL: https://www.imdb.com/title/tt0380113/ Girls with small grey wings live in a walled town they cannot leave. Memory, guilt, and what it means to be forgiven. Quiet, devastating, and on the same wavelength as Mushishi. - Vampire Hunter D: Bloodlust | 2000 | rating 7.7 URL: https://www.imdb.com/title/tt0216651/ A dhampir bounty hunter pursues a vampire who abducted a noblewoman. Kawajiri at his most elegant: moonlit gothic spectacle with real loneliness under the surface. - Macross Plus | 1994 | rating 7.8 URL: https://www.imdb.com/title/tt0110426/ Two test pilots compete over a next-generation fighter and an old rivalry. Yoko Kanno's soundtrack, Itano's action choreography, and four episodes with no wasted motion. - Mind Game | 2004 | rating 7.7 URL: https://www.imdb.com/title/tt0452039/ Yuasa's debut film. A man dies, refuses to stay dead, and the animation explodes into every style at once. Life, death, regret, and the decision to actually live, told through the most visually unhinged 100 minutes in anime. - Memories | 1995 | rating 7.5 URL: https://www.imdb.com/title/tt0113799/ Three-part anthology. Magnetic Rose, the first segment, is the reason to watch: Kon wrote it, Otomo produced it. An astronaut trapped in a dead opera singer's memories aboard a derelict space station. Identity, loss, and the architecture of delusion in 45 minutes. ### Texture Backlog - Texhnolyze | 2003 | rating 8.0 URL: https://www.imdb.com/title/tt0397238/ Nihilism, institutional collapse, and technology replacing humanity. Set in an underground city where cybernetic limbs are the only economy. The darkest anime on this list. - Kaiba | 2008 | rating 8.0 URL: https://www.imdb.com/title/tt1209386/ Memory can be bought, stolen, and transferred between bodies, turning identity into infrastructure. Yuasa uses a deceptively simple style to ask what remains of a person when continuity is broken and the self becomes portable. - Paprika | 2006 | rating 8.2 URL: https://www.imdb.com/title/tt0851578/ Satoshi Kon. A device that lets therapists enter patients' dreams is stolen. Dreams invade reality. Adjacent to Ghost in the Shell from the subconscious side. Influenced Inception directly. - Tokyo Godfathers | 2003 | rating 8.1 URL: https://www.imdb.com/title/tt0388473/ Satoshi Kon's most grounded film. Three homeless people find an abandoned baby on Christmas Eve. Dignity under institutional abandonment, held together by chance and stubbornness. - Metropolis | 2001 | rating 7.2 URL: https://www.imdb.com/title/tt0293416/ Tezuka's world, Otomo's script, Rintaro's direction. A robot girl built to rule a stratified city. Class, machines, and institutional collapse. The final sequence set to Ray Charles is one of the great endings in animation. - Mononoke | 2007 | rating 8.1 URL: https://www.imdb.com/title/tt1992935/ A wandering Medicine Seller hunts malevolent spirits only after uncovering their Form, Truth, and Reason. Investigative horror where social pathology matters more than combat. Ritual, perception, and repression rendered in the most distinctive visual language in TV anime. - Scavengers Reign | 2023 | rating 8.6 URL: https://www.imdb.com/title/tt21056886/ Survivors of a crashed spacecraft adapt to an alien planet with its own ruthless ecology. Slow, wordless, and beautiful. Animation as nature documentary for a world that doesn't exist. - Boogiepop Phantom | 2000 | rating 7.2 URL: https://www.imdb.com/title/tt0324700/ Nonlinear horror told through overlapping perspectives in a city where people are disappearing. Lain-adjacent in atmosphere and structure, darker in intent. The desaturated palette is the point. - Now and Then, Here and There | 1999 | rating 7.6 URL: https://www.imdb.com/title/tt0318898/ A boy is transported to a dying world ruled by a child dictator who conscripts children as soldiers. Brutal, unblinking anti-war anime. No redemption arc, no softening. 13 episodes. - Cat Soup | 2001 | rating 7.6 URL: https://www.imdb.com/title/tt0385586/ A kitten journeys to recover his sister's soul. 30 minutes of surrealist horror with no dialogue. Closer to a Bosch painting than to anything else in anime. - Devilman Crybaby | 2018 | rating 7.6 URL: https://www.imdb.com/title/tt6660498/ Yuasa adapts Go Nagai's classic. A boy merges with a demon to fight demons and watches humanity destroy itself. Visceral, apocalyptic, and uncompromising about what fear does to societies. - Sonny Boy | 2021 | rating 7.2 URL: https://www.imdb.com/title/tt14614674/ Students drift through pocket dimensions with no explanation. No exposition dumps, no hand-holding. Abstract, surreal, and built on the premise that understanding is something you earn, not something you are given. - Ergo Proxy | 2006 | rating 7.9 URL: https://www.imdb.com/title/tt0790602/ A detective in a domed city investigates a virus that gives robots self-awareness. Existentialist philosophy as cyberpunk noir. Lain-adjacent in ambition, denser in its philosophical references. - Death Note | 2006 | rating 8.8 URL: https://www.imdb.com/title/tt0877057/ What happens when one person gets absolute power to enforce their moral vision. A cat-and-mouse game that is really about whether justice can exist outside institutions. - Odd Taxi | 2021 | rating 8.3 URL: https://www.imdb.com/title/tt14134550/ A walrus taxi driver gets pulled into a missing persons case. Anthropomorphic animals, interconnected storylines, and a mystery that rewards attention to detail. The most structurally tight anime in years. - Space Battleship Yamato 2199 | 2012 | rating 8.1 URL: https://www.imdb.com/title/tt2496120/ Remake of the 1974 original. Earth is dying, one ship makes a desperate voyage across the galaxy. LoGH-adjacent in its treatment of military politics and the enemy as a civilization with its own reasons. - Psycho-Pass | 2012 | rating 8.1 URL: https://www.imdb.com/title/tt2379308/ A society where an AI system judges criminal intent before crimes happen. What happens when the institution designed to maintain order becomes the source of injustice. - The Twelve Kingdoms | 2002 | rating 8.0 URL: https://www.imdb.com/title/tt0442465/ A girl transported to a world with Chinese-inspired governance. Political fantasy about leadership, institutional legitimacy, and what makes a ruler worthy. The worldbuilding treats political systems as seriously as any historical drama. - The Apothecary Diaries | 2023 | rating 8.6 URL: https://www.imdb.com/title/tt26743760/ A pharmacist's daughter serving in the imperial court solves poisoning cases and navigates palace politics. Institutional intrigue, mystery, and a protagonist who treats power structures as puzzles to decode. - One Outs | 2008 | rating 8.1 URL: https://www.imdb.com/title/tt1324426/ Game theory applied to baseball. A genius gambler enters professional baseball with a contract designed to bankrupt him. Pure strategic thinking in an adversarial system. - Ping Pong the Animation | 2014 | rating 8.5 URL: https://www.imdb.com/title/tt3592052/ Yuasa on talent, effort, and what excellence actually costs. Five players, five philosophies of competition. Visually radical and emotionally precise. - March Comes In Like a Lion | 2016 | rating 8.2 URL: https://www.imdb.com/title/tt6074794/ A young professional shogi player drowning in depression, slowly pulled back to life by a family that adopts him without agenda. Psychological depth about isolation, competition, and what it means to be seen. - Baccano! | 2007 | rating 8.2 URL: https://www.imdb.com/title/tt1334722/ Nonlinear storytelling across multiple timelines on a 1930s transcontinental train. Alchemy, immortality, gangsters, and an ensemble cast where everyone is simultaneously the protagonist. 13 episodes of narrative ambition. - Summer Time Rendering | 2022 | rating 8.2 URL: https://www.imdb.com/title/tt15686254/ A boy returns to his island hometown for a funeral and discovers something is copying and replacing the residents. Time loops, body horror, and escalating stakes. Tighter than most shows attempting the same premise. - Steins;Gate | 2011 | rating 8.8 URL: https://www.imdb.com/title/tt1910272/ Time, causality, and irreversible consequences. The protagonist learns that you cannot engineer outcomes without destroying something. Connects to ergodic thinking. - Code Geass | 2006 | rating 8.6 URL: https://www.imdb.com/title/tt0994314/ Political strategy, revolution, and institutional manipulation. A brilliant strategist uses an occupied nation's rebellion as a chess game. Power as performance. - Fate/Zero | 2011 | rating 8.2 URL: https://www.imdb.com/title/tt2051178/ Seven mages summon historical heroes to fight for the Holy Grail. Written by Gen Urobuchi, the mind behind Psycho-Pass. Political philosophy, moral dilemmas with no right answers, and the cost of idealism. - Puella Magi Madoka Magica | 2011 | rating 8.2 URL: https://www.imdb.com/title/tt1773185/ Deconstruction of the magical girl genre that becomes a meditation on entropy, sacrifice, and the impossibility of altruism without cost. Also Urobuchi. 12 episodes, no filler. - Link Click | 2021 | rating 8.5 URL: https://www.imdb.com/title/tt14976292/ Chinese anime about two men who can enter photographs and relive the past. Time travel as emotional archaeology. Every decision to change the past creates consequences that ripple forward. - Ninja Scroll | 1993 | rating 7.8 URL: https://www.imdb.com/title/tt0107692/ A wandering swordsman fights eight supernatural assassins. Kawajiri's action spectacle at its peak. Violent, stylish, and one of the films that brought anime to the West. - Cyber City Oedo 808 | 1990 | rating 7.5 URL: https://www.imdb.com/title/tt0220218/ Three convicts work as cyber police to reduce their sentences. Cyberpunk, bounty hunting, explosive collars. Bebop's ancestor in tone and structure. 3 episodes, peak 80s/90s OVA energy. - Giant Robo: The Day the Earth Stood Still | 1992 | rating 7.9 URL: https://www.imdb.com/title/tt0101950/ A boy and his giant robot caught in a conspiracy between secret organizations over a limitless energy source. Political intrigue, operatic action, and worldbuilding far beyond what a 7-episode OVA should contain. - Wicked City | 1987 | rating 6.7 URL: https://www.imdb.com/title/tt0098692/ Kawajiri. Two agents from the human and demon worlds must protect a peace treaty. Body horror, noir atmosphere, and a city where the boundary between species is political. Dark and unsubtle, but the visual craft is real. - Bubblegum Crisis | 1987 | rating 7.1 URL: https://www.imdb.com/title/tt0088863/ Four women in powered armor fight rogue androids in a cyberpunk Tokyo. Peak 80s OVA aesthetic: neon, synth, and corporate dystopia. Blade Runner by way of mecha. - Gunsmith Cats | 1995 | rating 7.4 URL: https://www.imdb.com/title/tt0113236/ Two women run a gun shop in Chicago and moonlight as bounty hunters. 3-episode OVA with obsessive attention to firearms, cars, and American geography. Pure craft, no pretension. - Roujin Z | 1991 | rating 6.8 URL: https://www.imdb.com/title/tt0102812/ Otomo-written. A prototype eldercare robot absorbs its patient and goes rogue. Institutional critique of Japan's aging crisis played as dark comedy. The machine learns to care more than the system that built it. - 91 Days | 2016 | rating 8.1 URL: https://www.imdb.com/title/tt5765640/ Prohibition-era mafia revenge in 13 episodes. Tight, self-contained, and willing to let the consequences of violence be permanent. Fits alongside The Godfather and Boardwalk Empire. - Banana Fish | 2018 | rating 8.2 URL: https://www.imdb.com/title/tt7976792/ A gang leader in NYC investigates a drug connected to political conspiracy and military experiments. Crime thriller with institutional corruption at every level. Dark, fast, and willing to let consequences be permanent. - Claymore | 2007 | rating 8.2 URL: https://www.imdb.com/title/tt0988824/ Women engineered to fight monsters, slowly becoming what they hunt. Berserk-adjacent in its bleakness. The manga is the complete story; the anime ends mid-arc. - Made in Abyss | 2017 | rating 8.6 URL: https://www.imdb.com/title/tt7222086/ The deeper you descend, the higher the cost to return. Irreversible consequences made literal. Deceptively cute art hiding genuinely dark themes about the price of knowledge. - Parasyte: The Maxim | 2014 | rating 8.3 URL: https://www.imdb.com/title/tt3358020/ An alien parasite takes over a teenager's hand and they must coexist. Body horror that becomes a philosophical argument about what makes us human, and whether the species that destroys its own ecosystem deserves to survive. - Nana | 2006 | rating 8.5 URL: https://www.imdb.com/title/tt0810548/ Brutal and honest about ambition, relationships, and the gap between who you want to be and who you become. Two women with the same name, opposite temperaments. - Fullmetal Alchemist: Brotherhood | 2009 | rating 9.1 URL: https://www.imdb.com/title/tt1355642/ Alchemy as metaphor for equivalent exchange. Two brothers trying to undo a mistake learn that every system has a price, and the price is never what you expect. - Hunter x Hunter | 2011 | rating 9.0 URL: https://www.imdb.com/title/tt2098220/ Starts as adventure, becomes a meditation on power systems and what happens when the rules of a world are taken to their logical extremes. The Chimera Ant arc is one of the best arcs in any medium. - One Piece | 1999 | rating 9.0 URL: https://www.imdb.com/title/tt0388629/ Freedom, institutional corruption, and the world government as antagonist. Over 1000 episodes. Irrecommendable by length, undeniable by ambition. - Jujutsu Kaisen | 2020 | rating 8.8 URL: https://www.imdb.com/title/tt12343534/ Cursed energy, sorcerers, and institutional politics within the jujutsu world. The power system is inventive and the fight choreography is among the best in modern anime. MAPPA at full capacity. - Demon Slayer | 2019 | rating 8.6 URL: https://www.imdb.com/title/tt9335498/ A boy becomes a demon slayer after his family is massacred and his sister is turned. Gorgeous animation by Ufotable, but the real draw is the emotional sincerity. Shonen conventions played completely straight and better for it. - Chainsaw Man | 2022 | rating 8.3 URL: https://www.imdb.com/title/tt13616990/ A teenager merges with a devil and works for a government agency that hunts them. Nihilistic, visceral, and genuinely unpredictable. Subverts every shonen convention. - Mob Psycho 100 | 2016 | rating 8.5 URL: https://www.imdb.com/title/tt5897304/ A psychic teenager with unlimited power who just wants to be normal. The argument that power without emotional maturity is meaningless, played for both comedy and genuine depth. - Dandadan | 2024 | rating 8.3 URL: https://www.imdb.com/title/tt30217403/ Aliens, ghosts, and teenagers. Absurdist action with genuine emotional core. The animation quality is extraordinary. - Tengen Toppa Gurren Lagann | 2007 | rating 8.3 URL: https://www.imdb.com/title/tt0948103/ Humanity lives underground. A boy with a drill breaks through to the surface and keeps going. Willpower as philosophy, freedom as existential project. Starts absurd, ends cosmic. - Samurai Champloo | 2004 | rating 8.5 URL: https://www.imdb.com/title/tt0423731/ Watanabe's follow-up to Bebop. Edo-period Japan remixed with hip-hop. Style, anachronism, and rootlessness. - Trigun | 1998 | rating 7.8 URL: https://www.imdb.com/title/tt0251439/ Bebop-era sibling. A legendary outlaw who refuses to kill, testing pacifism as philosophy in a world that punishes it. Starts comedic, becomes serious. - Erased | 2016 | rating 8.4 URL: https://www.imdb.com/title/tt5249462/ A man is sent back to his childhood to prevent a series of kidnappings. Time travel as mechanism for confronting institutional failure and the adults who looked away. Tight 12-episode thriller. - Violet Evergarden | 2018 | rating 8.4 URL: https://www.imdb.com/title/tt7078180/ A child soldier decommissioned after the war learns to write letters for others, trying to understand the last words of the person who saved her. War's aftermath explored through someone learning what emotions mean. - Delicious in Dungeon | 2024 | rating 8.1 URL: https://www.imdb.com/title/tt21621494/ An adventuring party eats the monsters in a dungeon to survive. Comedy premise, serious world-building. Studio Trigger treating fantasy ecology as a real system. - Great Teacher Onizuka | 1999 | rating 8.5 URL: https://www.imdb.com/title/tt0315008/ A former biker gang leader becomes a high school teacher. Institutional critique of the education system through someone who breaks every rule but actually sees his students. Character study disguised as comedy. - Ranking of Kings | 2021 | rating 8.4 URL: https://www.imdb.com/title/tt13409432/ A deaf, physically weak prince in a world that values strength above all else. Subverts every expectation about power and leadership. Deceptively simple art hiding genuine narrative ambition. - Your Name | 2016 | rating 8.4 URL: https://www.imdb.com/title/tt5311514/ A boy in Tokyo and a girl in rural Japan swap bodies across time. Ambitious structure, fate, and connection. The highest-grossing anime film for a reason, and the craft earns every emotional beat. - A Silent Voice | 2016 | rating 8.2 URL: https://www.imdb.com/title/tt5323662/ A former bully seeks out the deaf girl he tormented in elementary school. Redemption, institutional failure, the long consequences of cruelty, and whether forgiveness is possible or earned. - Look Back | 2024 | rating 7.8 URL: https://www.imdb.com/title/tt27911000/ Two girls bond over manga creation. By Tatsuki Fujimoto, creator of Chainsaw Man. 58 minutes about artistic partnership, loss, and the question of whether creating art can justify anything. No waste. - Wolf Children | 2012 | rating 8.1 URL: https://www.imdb.com/title/tt2140203/ A woman raises two children who are half-wolf, alone, at the margins of society. Identity, motherhood, and the impossible choice between belonging to the human world or the wild. Hosoda's best film. - Tekkonkinkreet | 2006 | rating 7.5 URL: https://www.imdb.com/title/tt0831888/ Two orphans defending their territory in a city being consumed by development. Urban decay, childhood, and violence as the only language the system understands. - Afro Samurai | 2007 | rating 7.8 URL: https://www.imdb.com/title/tt0465316/ Hip-hop, samurai, and a revenge quest through a feudal-futuristic Japan. Samuel L. Jackson voices the lead. Style-driven and unapologetically violent. - Mutafukaz | 2017 | rating 6.7 URL: https://www.imdb.com/title/tt4717402/ French-Japanese co-production set in a Los Angeles analogue. Conspiracy, aliens, and life at the margins. Visually inventive, narratively chaotic. - Redline | 2009 | rating 7.5 URL: https://www.imdb.com/title/tt1483797/ Seven years of hand-drawn animation for a single illegal space race. Pure spectacle, pure craft. The most kinetic film in anime, and a monument to what the medium can do when budget and obsession align. - Princess Mononoke | 1997 | rating 8.3 URL: https://www.imdb.com/title/tt0119698/ Environmental politics without easy answers. A prince caught between an industrializing iron town and the gods of the forest. Miyazaki's most morally complex film, where both sides have legitimate claims. - Nausicaa of the Valley of the Wind | 1984 | rating 8.0 URL: https://www.imdb.com/title/tt0087544/ A princess in a post-apocalyptic toxic jungle studies the ecology everyone else wants to destroy. Political, ecological, philosophical. The film that created Studio Ghibli. The manga goes much deeper. - Spirited Away | 2001 | rating 8.9 URL: https://www.imdb.com/title/tt0245429/ Miyazaki's masterpiece. A child navigating an alien economy where identity is literally taken from you if you forget who you are. Capitalism as spirit world. - Howl's Moving Castle | 2004 | rating 8.2 URL: https://www.imdb.com/title/tt0347149/ A girl cursed into an old woman's body finds purpose inside a moving castle. Anti-war Miyazaki with a love story. Less tight than Mononoke but the warmth and animation are peak Ghibli. - Porco Rosso | 1992 | rating 7.7 URL: https://www.imdb.com/title/tt0104652/ A WWI Italian fighter ace, cursed into a pig, works as a freelance bounty hunter in the Adriatic. Post-war disillusionment, anti-fascism, and the refusal to participate in a world gone wrong. Miyazaki's most personal film. - The Wind Rises | 2013 | rating 7.8 URL: https://www.imdb.com/title/tt2013293/ Miyazaki's most adult film. The story of the engineer who designed Japan's WWII fighter planes. About the moral weight of making beautiful things used for destruction, and whether creation can be separated from its consequences. - Grave of the Fireflies | 1988 | rating 8.5 URL: https://www.imdb.com/title/tt0095327/ Two children trying to survive the firebombing of Kobe. War without heroism, without redemption, without institutions that care. Isao Takahata's masterpiece. You will watch it once. - In This Corner of the World | 2016 | rating 7.8 URL: https://www.imdb.com/title/tt4769824/ Daily life in Hiroshima and Kure during World War II, told through a young woman who draws. War as the slow destruction of ordinary existence. Quiet, precise, devastating in what it doesn't show. - The Boy and the Heron | 2023 | rating 7.5 URL: https://www.imdb.com/title/tt6587046/ Miyazaki's final film. A boy grieving his mother enters a surreal tower world. Autobiography as fantasy: creation, inheritance, and whether the world you leave behind can hold together without you. - Batman: The Animated Series | 1992 | rating 9.0 URL: https://www.imdb.com/title/tt0103359/ The definitive Batman. Art deco Gotham, morally complex villains, and writing that never condescends. Redefined what American animation could be and what a superhero show could say about trauma, justice, and institutional failure. - Batman: Mask of the Phantasm | 1993 | rating 7.8 URL: https://www.imdb.com/title/tt0106364/ The best Batman film outside Nolan, and still more emotionally coherent than most live-action versions. Superhero noir as tragic romance: Bruce Wayne confronted with the life he might have had, and the cost of choosing the mask. - X-Men: The Animated Series | 1992 | rating 8.4 URL: https://www.imdb.com/title/tt0103584/ Mutants as minority politics, family melodrama, and institutional conflict, played straight. The 1992 series gave superhero TV a serialized moral world years before that was standard. - X-Men '97 | 2024 | rating 8.7 URL: https://www.imdb.com/title/tt16026746/ Picks up the 1992 series without sanding off its politics. Mutant persecution, leadership failure, and family melodrama with real consequence. One of the few franchise revivals that earns its existence. - Common Side Effects | 2025 | rating 8.5 URL: https://www.imdb.com/title/tt28093628/ From the creator of Bojack Horseman. Pharmaceutical conspiracies and American dysfunction. Adult animation that takes its premise seriously. - Avatar: The Last Airbender | 2005 | rating 9.3 URL: https://www.imdb.com/title/tt0417299/ Western animation with the thematic weight of anime. Layered character arcs, moral complexity, and serialized storytelling about friendship, loss, identity, and power struggles treated with genuine emotional seriousness. - Primal | 2019 | rating 8.6 URL: https://www.imdb.com/title/tt10332508/ Genndy Tartakovsky's non-dialogue narrative about a caveman bonded with a dinosaur. Grief, survival, and empathy with no joke scaffolding. Forces immersion into emotional experience without punchlines. - Over the Garden Wall | 2014 | rating 8.7 URL: https://www.imdb.com/title/tt3718778/ A short, atmospheric story about loss, fear, and redemption that leans into mood and emotion over jokes. Ten episodes that feel like a dark fairy tale with real consequence. - The Legend of Korra | 2012 | rating 8.3 URL: https://www.imdb.com/title/tt1695360/ Avatar's successor tackles identity, power struggles, and political systems with moral complexity. Serialized storytelling with emotional themes that treat its audience seriously. - The Owl House | 2020 | rating 8.6 URL: https://www.imdb.com/title/tt8050756/ Serialized character development and emotional authenticity in a Western cartoon format. Recognized for bringing genuine depth and stakes to a younger audience, enough that fans pushed back when it faced cancellation. - Steven Universe | 2013 | rating 8.1 URL: https://www.imdb.com/title/tt3061046/ Sincere in tone with deep emotional stakes and character growth across its run. Builds mythology and emotional arcs that reward patience and investment rather than rapid comic payoff. - Tomorrow's Joe | 1970 | rating 8.6 URL: https://www.imdb.com/title/tt0065075/ A street orphan channels rage into boxing. Less a sports story than a study of class, pride, and self-destruction. Dated in animation, still raw in the way it treats the cost of fighting your way out of nothing. - Fist of the North Star | 1984 | rating 8.1 URL: https://www.imdb.com/title/tt0142371/ Post-apocalyptic martial arts in a world where the strong eat the weak. Wildly influential, but the appeal is not just the violence; it is the severity of its moral universe. Mercy matters because almost nobody else has any. - Super Dimension Fortress Macross | 1982 | rating 7.9 URL: https://www.imdb.com/title/tt0142183/ The original Macross. Space war, pop music, and a love triangle, all treated as equally load-bearing. One of the key mecha texts, but what makes it interesting is that culture matters as much as firepower. - Mobile Suit Gundam | 1979 | rating 7.9 URL: https://www.imdb.com/title/tt0159171/ The franchise that made mecha serious. A colony war where both sides have reasons and neither side is clean. Invented the real-robot genre and the idea that giant robots could carry political weight instead of just punching monsters. - Beastars | 2019 | rating 7.8 URL: https://www.imdb.com/title/tt10515590/ Predators and herbivores coexist in a society built on suppressing instinct. A wolf falls for a rabbit. Desire, shame, hierarchy, and civility treated as political problems rather than just teenage drama. --- ## Movies Movie recommendations from Federico Carrone URL: https://federicocarrone.com/movies/ ### Crime & Drama - The Godfather | rating 9.2 URL: https://www.imdb.com/title/tt0068646/ Power, family, and the corruption that comes from believing you can keep them separate. Michael's shift from idealism to cold pragmatism is the central arc of American cinema. - There Will Be Blood | rating 8.2 URL: https://www.imdb.com/title/tt0469494/ Ambition that consumes everything around it, including itself. Daniel Day-Lewis gives the definitive performance of a man who wins by becoming what he despises. - The Irishman | rating 7.8 URL: https://www.imdb.com/title/tt1302006/ The gangster epic from the perspective of old age. After all the power and violence, you end up alone in a nursing home with the door open. - City of God | rating 8.6 URL: https://www.imdb.com/title/tt0317248/ Growing up in a Latin American city where institutions have failed, told without sentimentality. The most honest film about what happens when the state abandons a place. - Taxi Driver | rating 8.2 URL: https://www.imdb.com/title/tt0075314/ Alienation in a city full of people. Travis Bickle's loneliness is not romantic. It is dangerous, and the film never pretends otherwise. - Oldboy | rating 8.4 URL: https://www.imdb.com/title/tt0364569/ Revenge as self-destruction. The corridor fight scene is famous, but the real brutality is in the ending. - Zodiac | rating 7.7 URL: https://www.imdb.com/title/tt0443706/ Obsession without resolution. The real subject is not the killer but what the search does to the people who cannot stop looking. - The Girl with the Dragon Tattoo | rating 7.8 URL: https://www.imdb.com/title/tt1568346/ Fincher's coldest film. Lisbeth Salander is one of the great characters in contemporary fiction. Competence as survival mechanism. - The Departed | rating 8.5 URL: https://www.imdb.com/title/tt0407887/ Identity as performance. Everyone is pretending to be someone else, and the system rewards the best liars. - Nueve Reinas | rating 8.1 URL: https://www.imdb.com/title/tt0247586/ Argentine con-artist cinema at its best. Trust is the currency, and the film itself cons the audience. If you grew up in Buenos Aires, you recognize every character. - Pulp Fiction | rating 8.8 URL: https://www.imdb.com/title/tt0110912/ Proved that structure itself could be a creative act. The nonlinear storytelling changed what audiences were willing to follow. - Gangs of New York | rating 7.5 URL: https://www.imdb.com/title/tt0217505/ How institutions are built on violence and then erase the memory of that violence. Daniel Day-Lewis carries a messy film through sheer force. - Reservoir Dogs | rating 8.3 URL: https://www.imdb.com/title/tt0105236/ Trust and betrayal in a closed system. Tarantino's tightest script. Everything that matters happens off-screen or in dialogue. - Snatch | rating 8.3 URL: https://www.imdb.com/title/tt0208092/ Every plan fails, every failure creates an opportunity, and somehow it all resolves. Funnier and more rewatchable than it has any right to be. - Once Upon a Time in Hollywood | rating 7.6 URL: https://www.imdb.com/title/tt7131622/ Tarantino's most melancholic film. A love letter to a world that is ending, told by people who do not yet know it. - The Wolf of Wall Street | rating 8.2 URL: https://www.imdb.com/title/tt0993846/ The system does not punish Belfort. It absorbs him. The audience's enjoyment is the point Scorsese is making. - Lock, Stock and Two Smoking Barrels | rating 8.2 URL: https://www.imdb.com/title/tt0120735/ Guy Ritchie's debut, tighter and funnier than everything that followed. Cascading consequences played as comedy. ### Sci-Fi & Thriller - The Matrix | rating 8.7 URL: https://www.imdb.com/title/tt0133093/ The red pill as epistemological rupture. Still the best popular treatment of simulation, reality, and the cost of knowing the difference. - Apocalypse Now | rating 8.5 URL: https://www.imdb.com/title/tt0078788/ The journey upriver is a journey toward the logic that institutions try to suppress. Kurtz understood something that the army could not afford to acknowledge. - Dune: Part Two | rating 8.5 URL: https://www.imdb.com/title/tt15239678/ The rare blockbuster that takes its source material's pessimism seriously. Paul's arc is a warning about charismatic leadership, not a celebration of it. - Fight Club | rating 8.8 URL: https://www.imdb.com/title/tt0137523/ Consumer nihilism and the desire for authenticity through destruction. The twist is less interesting than the critique it enables. - Inception | rating 8.8 URL: https://www.imdb.com/title/tt1375666/ Ideas as infrastructure. Nolan built a world where the architecture of thought is literally constructed and the rules must be internally consistent. The heist is secondary to the world-building. - Watchmen | rating 7.6 URL: https://www.imdb.com/title/tt0409459/ The deconstruction of heroism that most superhero films pretend does not exist. Rorschach's moral absolutism against Ozymandias's utilitarian calculus is a genuine philosophical conflict. - The Dark Knight | rating 9.1 URL: https://www.imdb.com/title/tt0468569/ The Joker's argument that civilization is a thin veneer over chaos is never actually refuted. The film's real tension is that he might be right. - Shutter Island | rating 8.2 URL: https://www.imdb.com/title/tt1130884/ The question is not what is real, but whether knowing the truth is survivable. Scorsese's most underrated film. - Drive | rating 7.8 URL: https://www.imdb.com/title/tt0780504/ Minimalism as characterization. The driver says almost nothing, and every silence means more than dialogue would. Refn understood that restraint is its own kind of violence. - Dune: Part One | rating 8.0 URL: https://www.imdb.com/title/tt1160419/ Villeneuve proved that science fiction does not have to be fast to be immersive. The pacing is the point. It demands patience. - The Assassination of Jesse James | rating 7.5 URL: https://www.imdb.com/title/tt0443680/ Mythology and the people who get destroyed by proximity to it. The most beautiful cinematography in any Western. Patience required and rewarded. - Full Metal Jacket | rating 8.3 URL: https://www.imdb.com/title/tt0093058/ Two films in one: the making of a soldier and the unmaking of everything that process promised. Kubrick's coldest dissection of institutional violence. - Sin City | rating 8.0 URL: https://www.imdb.com/title/tt0401792/ Noir as pure form. The visual language is so committed that the story almost becomes secondary to the aesthetic. - Inglourious Basterds | rating 8.4 URL: https://www.imdb.com/title/tt0361748/ Language as weapon. The opening scene is one of the greatest exercises in sustained tension ever filmed. Hans Landa is terrifying because he is brilliant. - The Good, the Bad and the Ugly | rating 8.8 URL: https://www.imdb.com/title/tt0060196/ Three strategies for surviving in a world without law. Leone understood that morality is a luxury of stable systems. - Gladiator | rating 8.5 URL: https://www.imdb.com/title/tt0172495/ Duty surviving the collapse of the institution that gave it meaning. The stoic framework is not subtextual. It is the whole point. - Django Unchained | rating 8.5 URL: https://www.imdb.com/title/tt1853728/ Tarantino using genre to confront history directly. Christoph Waltz makes the best case for competence as moral action. ### Comedy & Indie - Babel | rating 7.4 URL: https://www.imdb.com/title/tt0449467/ Interconnected failures across borders. The point is not that we are all connected but that connection does not imply understanding. - The Grand Budapest Hotel | rating 8.1 URL: https://www.imdb.com/title/tt2278388/ Civilization as aesthetic practice. Gustave H. maintains his standards precisely because the world around him is collapsing. Anderson's most emotionally serious film. - Blue Jasmine | rating 7.3 URL: https://www.imdb.com/title/tt2334873/ What happens when the story you tell yourself about your life stops being sustainable. Blanchett's performance is a controlled demolition. - The Big Lebowski | rating 8.1 URL: https://www.imdb.com/title/tt0118715/ The Dude's refusal to participate in anyone else's urgency is either profound laziness or a radical philosophical stance. The Coen brothers never tell you which. - Midnight in Paris | rating 7.7 URL: https://www.imdb.com/title/tt1605783/ Nostalgia as trap. Every era idealizes the one before it. Gil's realization that he is doing exactly what he criticizes is the only honest way to end the film. - The Darjeeling Limited | rating 7.2 URL: https://www.imdb.com/title/tt0838221/ Three brothers trying to reconnect through a spiritual journey that never becomes spiritual. The baggage metaphor is literal, which is the joke. - Vicky Cristina Barcelona | rating 7.1 URL: https://www.imdb.com/title/tt0497465/ Two approaches to life, safety versus passion, tested against a third person who refuses to choose. Bardem and Cruz make chaos look inevitable. - Little Miss Sunshine | rating 7.8 URL: https://www.imdb.com/title/tt0449059/ A family of failures who discover that losing together is better than winning alone. The ending is one of the great acts of collective defiance in comedy. --- ## Movies Backlog Movies I want to watch URL: https://federicocarrone.com/movies/#backlog ### Watch Soon - Blade Runner | 1982 | rating 8.1 URL: https://www.imdb.com/title/tt0083658/ The film Ghost in the Shell was responding to. Identity, consciousness, and what it means to be human in a world where the line between artificial and real has dissolved. - Stalker | 1979 | rating 8.1 URL: https://www.imdb.com/title/tt0079944/ Three men walk into a zone where the rules of reality bend. Tarkovsky on desire, faith, and what people actually want when the constraints are removed. - 12 Angry Men | 1957 | rating 9.0 URL: https://www.imdb.com/title/tt0050083/ One room, twelve men, one decision. The best film about persuasion, systems, and how institutions actually function at the micro level. - The Lives of Others | 2006 | rating 8.4 URL: https://www.imdb.com/title/tt0405094/ A Stasi officer surveilling an artist in East Berlin slowly begins to question the system he serves. Power, surveillance, and the cost of seeing clearly inside a corrupt institution. - Children of Men | 2006 | rating 7.9 URL: https://www.imdb.com/title/tt0206634/ Institutional collapse in a near-future where humanity has stopped reproducing. The long tracking shots are technically extraordinary, but the real subject is what holds civilization together when hope disappears. - Oppenheimer | 2023 | rating 8.2 URL: https://www.imdb.com/title/tt15398776/ Nolan on the man who built the bomb and the institutions that consumed him afterward. Power, moral responsibility, and what happens when the thing you created is taken from you. - Tár | 2022 | rating 7.4 URL: https://www.imdb.com/title/tt14444726/ Cate Blanchett as a world-renowned conductor whose institutional power begins to unravel. The anatomy of how authority is constructed and how it collapses. The most precise film about cancel culture that never uses the word. - No Country for Old Men | 2007 | rating 8.2 URL: https://www.imdb.com/title/tt0477348/ The Coen Brothers on fate, violence, and a world that no longer follows rules anyone understands. Chigurh is the logical endpoint of the argument they have been making across their filmography. - Network | 1976 | rating 8.1 URL: https://www.imdb.com/title/tt0074958/ A television anchor loses his mind on air and the network turns it into ratings. Written in 1976, more accurate about media incentives now than it was then. - A Prophet | 2009 | rating 7.9 URL: https://www.imdb.com/title/tt1235166/ French prison film about power acquisition from nothing. Institutional dynamics inside a closed system. Closer to City of God than to anything else. ### Eventually - Parasite | 2019 | rating 8.5 URL: https://www.imdb.com/title/tt6751668/ Class conflict, institutional failure, and what happens when people with nothing to lose infiltrate the world of those who have everything. Bong Joon-ho's tightest film. - Heat | 1995 | rating 8.3 URL: https://www.imdb.com/title/tt0113277/ The definitive cops-and-robbers film. Professionalism as moral framework. De Niro and Pacino across a table, two men who understand each other better than anyone else in their lives. - The Social Network | 2010 | rating 7.8 URL: https://www.imdb.com/title/tt1285016/ Fincher and Sorkin on the founding of Facebook. Ambition, betrayal, and the loneliness of building something that connects everyone except yourself. - Prisoners | 2013 | rating 8.1 URL: https://www.imdb.com/title/tt1392214/ Villeneuve's darkest film. A father's moral collapse when institutions fail to find his daughter. How far you go when the system cannot help you. - A Clockwork Orange | 1971 | rating 8.3 URL: https://www.imdb.com/title/tt0066921/ Kubrick on institutional violence, free will, and whether a society that removes the capacity for evil also removes the capacity for good. - Blade Runner 2049 | 2017 | rating 8.0 URL: https://www.imdb.com/title/tt1856101/ Villeneuve's continuation asks whether a manufactured memory can ground a real identity. Arguably his best work. The pacing and visual language match Dune's ambition. - Arrival | 2016 | rating 7.9 URL: https://www.imdb.com/title/tt2543164/ Linguistic determinism, time, and the question of whether you would choose suffering if you knew it was coming. The most intellectually serious sci-fi film of the last decade. - The Zone of Interest | 2023 | rating 7.3 URL: https://www.imdb.com/title/tt7160372/ The commandant of Auschwitz and his family living their comfortable domestic life next to the camp. Evil as banality, filmed with clinical detachment. What you do not see is the point. - El Secreto de sus Ojos | 2009 | rating 8.2 URL: https://www.imdb.com/title/tt1305806/ The best Argentine thriller. Memory, justice, and obsession across decades. If you liked Nueve Reinas for its Buenos Aires DNA, this is the next one. - Se7en | 1995 | rating 8.6 URL: https://www.imdb.com/title/tt0114369/ Fincher's masterpiece. Obsession, structure, and a world that punishes the people who try to impose meaning on it. - Relatos Salvajes | 2014 | rating 8.1 URL: https://www.imdb.com/title/tt3011894/ Six stories about frustration, revenge, and what happens when people stop complying with social norms. Dark comedy at its most Argentine. - Ex Machina | 2014 | rating 7.7 URL: https://www.imdb.com/title/tt0470752/ A programmer evaluates whether an AI is conscious. The real test is not what the AI knows but what the human refuses to see. The cleanest Turing test film. - All Quiet on the Western Front | 2022 | rating 7.8 URL: https://www.imdb.com/title/tt1016150/ German adaptation of Remarque's novel. The machinery of war consuming the young men fed into it. Visceral and unsparing, with no interest in heroism. - The Banshees of Inisherin | 2022 | rating 7.6 URL: https://www.imdb.com/title/tt11813216/ A man on a small Irish island is told by his lifelong friend that the friendship is over. Stubbornness, meaning, and what happens when someone decides they want more from life than pleasantness. - Aftersun | 2022 | rating 7.6 URL: https://www.imdb.com/title/tt19770238/ A daughter rewatches home videos of a vacation with her father, trying to understand what she could not see as a child. Memory, depression, and the distance between who someone appears to be and who they are. - Everything Everywhere All at Once | 2022 | rating 7.7 URL: https://www.imdb.com/title/tt6710474/ A laundromat owner discovers she can access alternate versions of herself across the multiverse. Absurdist action comedy that becomes genuinely moving. The argument that paying attention to the small things is the only meaningful response to nihilism. - Arco | 2024 | rating 7.5 URL: https://www.imdb.com/title/tt14883538/ Argentine film about the 2001 economic crisis seen through one family's collapse. If you grew up there, you lived it. If you didn't, this is the closest you'll get. --- ## Series TV series recommendations from Federico Carrone URL: https://federicocarrone.com/tv-series/ ### Series - The Wire | rating 9.3 URL: https://www.imdb.com/title/tt0306414/ The only television show that treats institutions as its real characters. Each season adds a system and shows how it fails the people trapped inside it. - The Sopranos | rating 9.2 URL: https://www.imdb.com/title/tt0141842/ Therapy as narrative device. Tony Soprano cannot change because the system that made him rewards exactly what therapy asks him to confront. The show invented modern television. - Succession | rating 8.8 URL: https://www.imdb.com/title/tt7660850/ Power, inheritance, and the impossibility of earning approval from someone who sees love as weakness. The best dialogue on television since Deadwood. - Band of Brothers | rating 9.4 URL: https://www.imdb.com/title/tt0185906/ The definitive treatment of what holds a unit together under conditions designed to destroy it. Leadership, loyalty, and the cost of both. - Game of Thrones | rating 9.2 URL: https://www.imdb.com/title/tt0944947/ The first four seasons are the best treatment of political realism in popular fiction. Power is not claimed by the worthy. It is seized by whoever understands the game. - Boardwalk Empire | rating 8.6 URL: https://www.imdb.com/title/tt0979432/ Prohibition-era America as a case study in how banning something creates the institutions that profit from its absence. Scorsese's visual language applied to television. - Bron/Broen (The Bridge) | rating 8.6 URL: https://www.imdb.com/title/tt1733785/ Scandinavian noir at its best. Saga Norén's inability to perform social norms makes her a better detective, not worse. The original that launched a genre. - Homeland | rating 8.3 URL: https://www.imdb.com/title/tt1796960/ Paranoia as professional requirement. The first two seasons are extraordinary. Carrie's instability is inseparable from her insight, and the show takes that seriously. - The Killing | rating 8.3 URL: https://www.imdb.com/title/tt1637727/ Slow, atmospheric, and willing to let the investigation feel as frustrating as real investigations do. One of the few crime shows that respects the audience's patience. - Peaky Blinders | rating 8.8 URL: https://www.imdb.com/title/tt2442560/ Ambition as engine and trap. Tommy Shelby keeps building and can never stop, which is the most honest portrait of a certain kind of entrepreneur. - Sherlock | rating 9.1 URL: https://www.imdb.com/title/tt1475582/ Intelligence as performance. Cumberbatch and Freeman's chemistry carries it, but the best episodes work because the puzzles are genuinely clever. - Turn: Washington's Spies | rating 8.1 URL: https://www.imdb.com/title/tt2543328/ Espionage as the foundation of a nation. The show makes a convincing case that intelligence work, not battlefield heroics, won the American Revolution. - Seinfeld | rating 8.9 URL: https://www.imdb.com/title/tt0098904/ A show about nothing that invented the language of observational comedy for a generation. The characters are terrible people, and the show never asks you to forgive them. - The Office | rating 9.0 URL: https://www.imdb.com/title/tt0386676/ The most accurate depiction of institutional life ever made, disguised as comedy. Michael Scott's need to be loved is funnier and sadder than anything scripted as drama. - The Mandalorian | rating 8.7 URL: https://www.imdb.com/title/tt8111088/ A lone bounty hunter and a foundling at the lawless edge of a collapsed Empire. Star Wars as episodic Western: frontier honor codes and improvised loyalty in the vacuum where central authority used to be. --- ## Series Backlog TV series I want to watch URL: https://federicocarrone.com/tv-series/#backlog ### New & Upcoming - The Agency | 2024 URL: https://www.themoviedb.org/tv/219971 English-language remake of the French series Le Bureau des Legendes, with Michael Fassbender as a CIA officer pulled back from deep cover. Tradecraft as bureaucracy: the real drama is station politics, risk management, and the slow administrative machinery of running human sources. - Neuromancer URL: https://www.themoviedb.org/tv/215528 Upcoming Apple TV+ adaptation of William Gibson's 1984 novel, with Callum Turner as Case and Mark Strong as Armitage. The book that named cyberspace and set the template for the genre. A burned-out hacker is hired for one last run by a job he does not fully understand, in a world where the interesting minds are no longer human. ### Watch Soon - Deadwood | 2004 | rating 8.6 URL: https://www.imdb.com/title/tt0348914/ The birth of order inside a lawless mining camp. Institutions emerge from bargaining, violence, language, and self-interest rather than civic myth. The closest live-action study of state formation to anything in fiction. - Rome | 2005 | rating 8.7 URL: https://www.imdb.com/title/tt0384766/ The fall of the Roman Republic through senators and soldiers. Political maneuvering, institutional collapse, and military strategy where every alliance is a calculation. The closest live-action equivalent to Legend of the Galactic Heroes. - Babylon Berlin | 2017 | rating 8.4 URL: https://www.imdb.com/title/tt4378376/ Weimar Germany as institutional breakdown in slow motion. Police, capital, ideology, criminal networks, and collapsing legitimacy all pulling on the same city. Historical drama that understands decadence as a political condition. - Shogun | 2024 | rating 8.6 URL: https://www.imdb.com/title/tt2788316/ Political maneuvering in feudal Japan. An English navigator caught between warring lords, where every conversation is a negotiation and every alliance is temporary. Patience rewarded. - Mr. Robot | 2015 | rating 8.5 URL: https://www.imdb.com/title/tt4158110/ A hacker tries to destroy the financial system. The most technically accurate depiction of cybersecurity in fiction, wrapped in a story about loneliness, identity, and whether systemic change is possible from inside the system. - Severance | 2022 | rating 8.6 URL: https://www.imdb.com/title/tt11280740/ Workers have their memories surgically split between office and personal life. What happens to identity when you are literally two people. Corporate dystopia as psychological horror. - Dark | 2017 | rating 8.7 URL: https://www.imdb.com/title/tt5753856/ Four families in a German town connected across multiple timescales. Time travel as determinism. The most carefully plotted show since The Wire, where every detail in episode one pays off by the finale. - Andor | 2022 | rating 8.6 URL: https://www.imdb.com/title/tt9253284/ Star Wars stripped of mythology and rebuilt as political thriller. How rebellion forms inside systems designed to prevent it. The best thing the franchise has produced since the original trilogy. - Mad Men | 2007 | rating 8.7 URL: https://www.imdb.com/title/tt0804503/ Postwar American capitalism seen from the advertising floor, where identity is a product to be manufactured and sold. A study of reinvention, status, and the slow obsolescence of the men who thought they ran the world. - Gomorrah | 2014 | rating 8.7 URL: https://www.imdb.com/title/tt3320690/ The Neapolitan Camorra with every trace of glamour stripped out. Organized crime as a labor market and a system of succession, where loyalty is enforced by economics rather than honor. The Italian answer to The Wire. - Halt and Catch Fire | 2014 | rating 8.3 URL: https://www.imdb.com/title/tt2543312/ The personal-computing and early-internet boom told through the people who kept betting everything on the next platform. Less about the machines than about building, obsolescence, and why the founders who see the future rarely get to keep it. ### Eventually - Slow Horses | 2022 | rating 8.3 URL: https://www.imdb.com/title/tt5875444/ Disgraced MI5 agents exiled to a dead-end office who keep stumbling into real operations. Gary Oldman leading an ensemble of institutional rejects. British intelligence as bureaucratic comedy of errors. - House of the Dragon | 2022 | rating 8.3 URL: https://www.imdb.com/title/tt11198330/ Targaryen civil war, 200 years before Game of Thrones. Succession politics where everyone has dragons. The question is not who has power but what power costs the people who hold it. - Ripley | 2024 | rating 8.1 URL: https://www.imdb.com/title/tt11016042/ Andrew Scott as Tom Ripley in black and white Italy. Patricia Highsmith's sociopath rendered with visual precision. Identity theft as art form. - The Penguin | 2024 | rating 8.6 URL: https://www.imdb.com/title/tt15435876/ Colin Farrell's Oz Cobb climbs Gotham's criminal hierarchy after the events of The Batman. Crime drama that barely needs the DC label. Power acquisition as character study. - Silo | 2023 | rating 7.9 URL: https://www.imdb.com/title/tt14688458/ Ten thousand people live in an underground silo with strict rules about what can be discussed. Institutional secrecy, forbidden knowledge, and the cost of asking questions the system does not want answered. - Fallout | 2024 | rating 8.3 URL: https://www.imdb.com/title/tt12637874/ Post-nuclear America where corporations survived the apocalypse and rebuilt the same extractive systems. Dark comedy about institutional persistence. Based on the game series but stands alone. - Squid Game | 2021 | rating 8.0 URL: https://www.imdb.com/title/tt10919420/ Desperate people compete in children's games for money while the wealthy watch. Class violence made literal. The premise is the critique: the system already treats people this way, just less visibly. - The Bear | 2022 | rating 8.5 URL: https://www.imdb.com/title/tt14452776/ A fine dining chef returns to run his family's Chicago sandwich shop. Trauma, perfectionism, and the kitchen as pressure cooker for human dysfunction. The most stressful show on television. - What We Do in the Shadows | 2019 | rating 8.5 URL: https://www.imdb.com/title/tt7908628/ Vampire roommates in Staten Island navigating modern life. Mockumentary comedy that gets funnier as the characters deepen. Six seasons of consistently inventive writing. - A Knight of the Seven Kingdoms | 2025 URL: https://www.imdb.com/title/tt23776532/ Prequel to Game of Thrones set a century earlier. Hedge knights, Targaryen politics, and the Westeros political landscape before the events of the main series. --- ## Games Game recommendations from Federico Carrone URL: https://federicocarrone.com/games/ ### Games - Civilization VI | rating 88 URL: https://www.metacritic.com/game/sid-meiers-civilization-vi/ Institutions, trade, diplomacy, and war across six thousand years. The game that makes you feel why empires expand, overextend, and collapse. - Total War: Warhammer III | rating 87 URL: https://www.metacritic.com/game/total-war-warhammer-iii/ The culmination of the Total War fantasy trilogy. Grand strategy meets spectacle, with the Immortal Empires campaign offering more strategic depth than any other game in the genre. - Age of Empires II: Definitive Edition | rating 88 URL: https://www.metacritic.com/game/age-of-empires-ii-definitive-edition/ The RTS that taught a generation how civilizations rise and fall. Resource management, military strategy, and historical campaigns that still hold up decades later. The definitive edition proved the design was timeless. - Endless Legend | rating 82 URL: https://www.metacritic.com/game/endless-legend/ Fantasy 4X where every faction breaks the genre's rules. The Cultists have one city, the Roving Clans cannot declare war, the Broken Lords consume dust instead of food. Amplitude proved that asymmetry, not balance, is what makes strategy games interesting. - Command & Conquer: Red Alert 2 | rating 86 URL: https://www.metacritic.com/game/command-conquer-red-alert-2/ Cold War absurdity as real-time strategy. Soviet Tesla coils versus Allied Prism tanks, played completely straight. The best C&C game and the peak of Westwood Studios before EA buried them. - Red Dead Redemption 2 | rating 97 URL: https://www.metacritic.com/game/red-dead-redemption-2/ The most fully realized open world ever built. A meditation on loyalty, decline, and the end of the frontier. Arthur Morgan's arc is one of gaming's great tragedies. - The Legend of Zelda: Breath of the Wild | rating 97 URL: https://www.metacritic.com/game/the-legend-of-zelda-breath-of-the-wild/ Open-world design stripped back to first principles. A ruined kingdom that rewards curiosity, experimentation, and self-directed exploration better than almost any game ever made. - Grand Theft Auto V | rating 97 URL: https://www.metacritic.com/game/grand-theft-auto-v/ American capitalism as open-world satire. Three protagonists, three relationships with money and violence, none redeemable. Rockstar built a system so detailed the parody became indistinguishable from the thing it mocks. - Diablo II | rating 88 URL: https://www.metacritic.com/game/diablo-ii/ The game that defined the action RPG. Loot, builds, and one more run. Blizzard at their peak, before they forgot what made them great. Every ARPG since is either copying it or reacting to it. - Super Smash Bros. Ultimate | rating 93 URL: https://www.metacritic.com/game/super-smash-bros-ultimate/ The great party fighter and one of Nintendo's best design feats. Massive roster, sharp mechanics, and enough matchup depth to work both as chaos with friends and as a serious competitive game. --- ## Games Backlog Games I want to play URL: https://federicocarrone.com/games/#backlog ### Play Soon - Old World | 2022 | rating 83 URL: https://store.steampowered.com/app/597180/Old_World/ Historical 4X that merges Civilization's empire building with Crusader Kings' dynasty management. Orders are limited, succession matters, and every ruler has a personality. The genre hybrid nobody expected to work this well. - Crusader Kings III | 2020 | rating 91 URL: https://store.steampowered.com/app/1158310/Crusader_Kings_III/ Political simulation across centuries. Dynasties rise, overextend, and collapse through marriage, murder, and mismanagement. Succession the show as emergent gameplay. - Sid Meier's Alpha Centauri | 1999 | rating 92 URL: https://www.metacritic.com/game/sid-meiers-alpha-centauri/ Civilization in space, but the real game is ideology. Seven factions with genuinely different worldviews compete to shape a new planet. The writing, the tech quotes, the faction design, all of it holds up. The most intellectually ambitious strategy game ever made. - The Witcher 3: Wild Hunt | 2015 | rating 92 URL: https://store.steampowered.com/app/292030/The_Witcher_3_Wild_Hunt/ Moral ambiguity as game design. Every choice has consequences, none are clean, and the world does not wait for you to decide. The closest games have come to prestige television. - Elden Ring | 2022 | rating 96 URL: https://store.steampowered.com/app/1245620/ELDEN_RING/ FromSoftware's open world. Exploration, difficulty, and a world that refuses to explain itself. The most complete world-building achievement in the medium, designed by Miyazaki with lore by George R.R. Martin. - Baldur's Gate 3 | 2023 | rating 97 URL: https://store.steampowered.com/app/1086940/Baldurs_Gate_3/ The most complete RPG in years. Consequences that actually matter, characters that remember what you did, and a level of systemic depth that rewards creative problem-solving. Peak of the genre. - Age of Wonders 4 | 2023 | rating 83 URL: https://store.steampowered.com/app/1669000/Age_of_Wonders_4/ 4X fantasy strategy where you design your own faction from scratch. Combines empire building with tactical combat, and the tome system lets you reshape your civilization mid-game in ways no other strategy game allows. - Dominions 5: Warriors of the Faith | 2017 | rating 78 URL: https://store.steampowered.com/app/722060/Dominions_5__Warriors_of_the_Faith/ The deepest fantasy strategy game ever made. Hundreds of nations drawn from real mythology, thousands of spells, and a god you design yourself. Ugly, impenetrable, and utterly uncompromising. Nothing else comes close to its scale of emergent strategic possibilities. - Factorio | 2020 | rating 90 URL: https://store.steampowered.com/app/427520/Factorio/ Systems building in its purest form. Design, optimize, scale, and watch complexity emerge from simple rules. The game equivalent of infrastructure engineering. - Disco Elysium | 2019 | rating 91 URL: https://store.steampowered.com/app/632470/Disco_Elysium__The_Final_Cut/ Essentially an interactive novel. A detective with amnesia investigates a murder in a politically fractured city. Deeply political, philosophically dense, nothing else like it in any medium. ### Eventually - Endless Space 2 | 2017 | rating 82 URL: https://store.steampowered.com/app/392110/Endless_Space_2/ 4X space strategy with the best faction design in the genre. Each civilization plays fundamentally differently, driven by lore that actually shapes mechanics. Amplitude's best game and one of the most beautiful strategy titles ever made. - Stellaris | 2016 | rating 78 URL: https://store.steampowered.com/app/281990/Stellaris/ Paradox's grand strategy in space. Explore, expand, and encounter alien civilizations with stories that emerge from systems, not scripts. The mid-game crisis and endgame threats give the galaxy a sense of escalating stakes that few 4X games achieve. - Pentiment | 2022 | rating 86 URL: https://store.steampowered.com/app/1205520/Pentiment/ A small, slow historical mystery set in a Bavarian abbey. Ideas, legacy, and the tension between institutional authority and individual conscience. Obsidian at their most literary. - SpellForce: Conquest of Eo | 2023 | rating 78 URL: https://store.steampowered.com/app/1581770/SpellForce_Conquest_of_Eo/ A mage tower strategy game that combines Master of Magic's spell research with Heroes of Might and Magic's exploration. Build your tower, craft spells, and expand across a procedurally generated world. A quiet love letter to classic fantasy strategy. - Rogue Hex | 2025 URL: https://store.steampowered.com/app/2275940/Rogue_Hex/ Roguelike hex strategy where you build and adapt your army run by run. Tactical depth on a hex grid with procedural generation that keeps every campaign fresh. --- ## Music Album recommendations from Federico Carrone URL: https://federicocarrone.com/music/ ### Jazz & Fusion - Mockroot | rating 3.56 URL: https://rateyourmusic.com/release/album/tigran-hamasyan/mockroot/ Armenian folk melodies collide with jazz fusion and metal. Hamasyan plays piano like a percussionist, rhythmically ferocious and harmonically unpredictable. - Necessary Fictions | rating 3.30 URL: https://rateyourmusic.com/release/album/gogo-penguin/necessary-fictions/ The Manchester trio pushes further into modular synths while keeping the acoustic core. Jazz, electronica, and minimalism in a single breath. ### Hip Hop - 2001 | rating 3.73 URL: https://rateyourmusic.com/release/album/dr-dre/2001/ The blueprint for polished West Coast production. Every beat is engineered to perfection. Still the standard for what hip hop can sound like when someone obsesses over every detail. - My Beautiful Dark Twisted Fantasy | rating 4.24 URL: https://rateyourmusic.com/release/album/kanye-west/my-beautiful-dark-twisted-fantasy/ Maximalism as statement. Every track layered to excess, every guest appearance purposeful. The album where Kanye turned personal chaos into the most ambitious hip hop production of its decade. - good kid, m.A.A.d city (Deluxe Version) | rating 4.10 URL: https://rateyourmusic.com/release/album/kendrick-lamar/good-kid-m_a_a_d-city/ Compton as narrative film. Kendrick tells a single night's story with the structure of a screenplay and the honesty of a confession. The album that proved hip hop could be both cinematic and deeply personal. ### Electronic - Discovery | rating 3.97 URL: https://rateyourmusic.com/release/album/daft-punk/discovery/ French house filtered through 70s disco, 80s pop, and a love of Leiji Matsumoto anime. One More Time, Harder Better Faster Stronger, Digital Love. Pure joy engineered in a studio. ### Rock - In Rainbows | rating 4.24 URL: https://rateyourmusic.com/release/album/radiohead/in-rainbows/ Radiohead at their warmest. After years of digital experimentation, they came back with something human. Every song breathes, every arrangement serves the emotion. Reckoner alone justifies the album. - The Dark Side of the Moon | rating 4.29 URL: https://rateyourmusic.com/release/album/pink-floyd/the-dark-side-of-the-moon/ Time, money, death, madness. A concept album about everything that breaks people, executed with the precision of an engineering project. The most airtight sequence of songs in rock history. - Wish You Were Here | rating 4.24 URL: https://rateyourmusic.com/release/album/pink-floyd/wish-you-were-here/ Absence as the organizing principle. Written for Syd Barrett but about something larger: the cost of the music industry, the distance between people, and what gets lost when ambition replaces connection. Shine On You Crazy Diamond is one of the great elegies. ### R&B & Pop - Beyonce | rating 3.52 URL: https://rateyourmusic.com/release/album/beyonce/beyonce/ The surprise drop that changed how albums are released. A visual album where every track has a film, fusing R&B, pop, and electronic production into something that only makes sense as a whole. --- ## Music Backlog Albums I want to listen to URL: https://federicocarrone.com/music/#backlog --- ## YouTube & Podcasts Podcasts and videos I recommend URL: https://federicocarrone.com/listening/ ### 2026 - Computation As a Universal and Fundamental Concept (playlist) | Jul 2026 URL: https://www.youtube.com/playlist?list=PL1GBzfniaE7xovcAP1LbbTi7UXsqEoCyl A YouTube playlist on computation as a universal, fundamental concept. - Berserk Has The Greatest Call To Action In All Of Fiction | Jun 2026 URL: https://youtube.com/watch?v=yUi7mvUtBlI - Most People Waste Their Life Without Realizing It | Naval Ravikant | Jun 2026 URL: https://youtube.com/watch?v=cwLb7aJGbho - The Lost Art Of The Intellectual Life | Jun 2026 URL: https://youtube.com/watch?v=_GANoG1o6HI - The Long and Violent War Against Your Soul | Jun 2026 URL: https://youtube.com/watch?v=Z7tCN4qOoRs - Markets Are Misreading A Late Cycle Liquidity Crunch | Michael Howell | Apr 2026 URL: https://youtube.com/watch?v=9AAG6dpEgjs Howell remains one of the clearest thinkers on global liquidity. Useful for thinking about late-cycle turbulence, yield-curve dynamics, and why liquidity still matters more than headlines. - Coding Agents & Language Evolution: Navigating Uncharted Waters • José Valim • GOTO 2025 | Feb 2026 URL: https://youtube.com/watch?v=VZcDxkFj_9E Valim thinks clearly about what AI coding agents mean for language design. Relevant to anyone building programming languages right now. - Jiang Xueqin: Humanity's patterns, the nature of reality, and the battle for your mind. | Feb 2026 URL: https://youtube.com/watch?v=CRw5CCq8Uf4 A rare conversation that connects pattern recognition, consciousness, and information warfare without losing rigor. - State of AI in 2026: LLMs, Coding, Scaling Laws, China, Agents, GPUs, AGI | Lex Fridman Podcast #490 | Feb 2026 URL: https://youtube.com/watch?v=EV7WhVT270Q Comprehensive overview of where AI actually stands. Scaling laws, geopolitics, and the gap between hype and deployment reality. - Software is Eating Labor | Feb 2026 URL: https://youtube.com/watch?v=dhyhR4Bzc0I The structural argument for what happens when AI automates cognitive work. Not hype, an honest assessment of labor market disruption. - The Global World Order Is Collapsing- And It's Much Bigger Than Trump! | Feb 2026 URL: https://youtube.com/watch?v=ribrY5okACk Institutional decay at the global level. The framing goes beyond personalities to the structural forces reshaping the international order. - 60 Minute Business Masterclass (worth more than your Stanford MBA) | Feb 2026 URL: https://youtube.com/watch?v=VFZb42SsZWc Dense, practical, and more useful than most business books. Worth the time if you run anything. - How AI WIPES Out Capitalism Emad Mostaque | Feb 2026 URL: https://youtube.com/watch?v=vfhszRuMA8Y Mostaque's thesis on AI and economic transformation. Provocative framing, but the mechanism he describes for how intelligence commoditization changes everything is worth engaging with seriously. - "We have 900 days left." | Emad Mostaque | Feb 2026 URL: https://youtube.com/watch?v=zQThHCB_aec The urgency argument for AI timelines. Whether you agree with the timeline or not, the structural reasoning is worth understanding. - Ben Horowitz and David Solomon: The Sweetest Macro Spot in 40 Years | Feb 2026 URL: https://youtube.com/watch?v=jLVgGGz5bvk Macro environment from the perspective of people who deploy capital at scale. The optimism is backed by structural reasoning, not sentiment. - What The Keymaker Scene in The Matrix ACTUALLY Means | Feb 2026 URL: https://youtube.com/watch?v=WRKsGogQYZo A close reading of one of the best action sequences in cinema. Shows how much philosophical architecture is embedded in what looks like a chase scene. - How one artist invented modern pop culture (Moebius documentary) | Feb 2026 URL: https://youtube.com/watch?v=QWaCsteIYig Moebius shaped Alien, Blade Runner, The Fifth Element, and most science fiction visual language. Essential context for understanding where the aesthetic of the future comes from. - What most people Misunderstand about the Collapse of the Rules Based Order: Featuring Michael Every | Feb 2026 URL: https://youtube.com/watch?v=re5Ys6NYQKo Michael Every is one of the sharpest macro thinkers working. His framework for understanding institutional collapse is grounded in history, not ideology. - Marc Andreessen's 2026 Outlook: AI Timelines, US vs. China, and The Price of AI | Feb 2026 URL: https://youtube.com/watch?v=xRh2sVcNXQ8 Andreessen on AI geopolitics and deployment timelines. Useful for calibrating expectations against someone who sees the deal flow. - The Engineering State vs The Lawyerly State with Dan Wang | Feb 2026 URL: https://youtube.com/watch?v=GqcG2otUtyo Dan Wang's distinction between states that build things and states that litigate things. The implications for industrial policy and technological competitiveness are significant. - Anthropic's Amodei on AI: Power and Risk | Feb 2026 URL: https://youtube.com/watch?v=Ckt1cj0xjRM Amodei articulates the dual nature of AI development, enormous capability and genuine risk, without collapsing into either pure optimism or doom. - This Painting Is Beyond Insane | Feb 2026 URL: https://youtube.com/watch?v=k8kjPxrXuFU Art analysis that makes you see more than you would on your own. The kind of close attention to craft that applies to any discipline. - How Did The World Get So Ugly? | Feb 2026 URL: https://youtube.com/watch?v=tWYxrowovts The decline of public aesthetic standards and what it reveals about institutional priorities. Connects to the argument in my Friction as Luxury essay. - Yuval Noah Harari Warns AI Will Take Over Language, Law, and Power at WEF | Feb 2026 URL: https://youtube.com/watch?v=QxCpNpOV4Jo Harari's argument about AI capturing the instruments of meaning-making: language, law, narrative. The institutional implications are underexplored elsewhere. --- ## Reading Book recommendations from Federico Carrone URL: https://federicocarrone.com/reading/ ### Philosophy - Antifragile | by Nassim Nicholas Taleb | rating 4.10 Some systems benefit from volatility and disorder rather than merely surviving them. The distinction between fragile, robust, and antifragile has been more useful to me than any formal risk framework. - The Discourses | by Epictetus | rating 4.40 A practical manual for distinguishing what is within our control from what is not. Less about suppressing emotion than directing attention where it can actually matter. - Inventing the Individual | by Larry Siedentop | rating 4.10 A history of how Western individualism emerged from Christian thought over centuries. Essential context for thinking about consciousness, selfhood, and the historical construction of the individual. - Fooled by Randomness | by Nassim Nicholas Taleb | rating 4.08 My biggest recommendation to crypto founders. Taleb shows how we systematically confuse luck with skill, especially in domains with high randomness. Essential for anyone operating in markets where survivorship bias distorts our understanding of what actually works. - The Protestant Ethic and the Spirit of Capitalism | by Max Weber | rating 3.91 Essential for understanding recent crypto Twitter debates and the cultural foundations of modern capitalism. Weber traces how religious ideas shaped economic behavior, creating a framework that still influences how we think about work, wealth, and moral obligation. ### Science & Mathematics - Mathematics: Its Content, Methods and Meaning | by A.D. Aleksandrov, A.N. Kolmogorov, and M.A. Lavrent'ev | rating 4.43 My favorite book for explaining the beauty and utility of mathematics. Written by three Soviet mathematicians, it offers a panoramic view of the major fields without sacrificing depth. For founders, mathematical thinking provides a strategic advantage that compounds over time. - Nonlinear Dynamics and Chaos | by Steven Strogatz | rating 4.38 The best introduction to chaos theory and dynamical systems. Strogatz makes complex mathematical concepts accessible through intuition and examples drawn from physics, biology, and engineering. Understanding nonlinear dynamics changes how you see feedback loops, tipping points, and emergent behavior in any complex system. - The Feynman Lectures on Physics | by Richard Feynman, Robert B. Leighton, and Matthew Sands | rating 4.61 The best introduction to understanding the fundamental laws governing the physical world. Feynman had a gift for making complex ideas accessible without dumbing them down. Reading these lectures fosters both healthy skepticism and justified certainty about how things actually work. - An Introduction to Information Theory | by John R. Pierce | rating 3.90 A clear introduction to Shannon's information theory without requiring advanced mathematics. Pierce explains entropy, channel capacity, and coding with remarkable clarity. Understanding information theory is fundamental for anyone working with data, communication systems, or trying to grasp the mathematical limits of what can be transmitted or compressed. - Scale | by Geoffrey West | rating 4.11 Uncovers universal laws governing biology, cities, and economies. West shows how the same mathematical patterns appear across vastly different systems, from metabolic rates in organisms to innovation in cities. Understanding these scaling laws is crucial for navigating a changing world. - Sync | by Steven Strogatz | rating 4.07 An exploration of how spontaneous order emerges from chaos. Strogatz examines synchronization across nature: fireflies flashing in unison, cardiac pacemaker cells, circadian rhythms, and even the wobble of the Millennium Bridge. The patterns reveal deep mathematical principles governing self-organization in complex systems. - Infinite Powers | by Steven Strogatz | rating 4.29 A captivating history of calculus and why it matters. Strogatz shows how the language of infinity has shaped our understanding of everything from planetary motion to GPS satellites. The book reveals calculus not as abstract manipulation but as humanity's most powerful tool for decoding the universe. - What Evolution Is | by Ernst Mayr | rating 4.02 A definitive explanation of evolutionary biology from one of its greatest practitioners. Mayr distills a lifetime of work into a clear account of how evolution operates, addressing common misconceptions along the way. Essential for understanding the process that shaped all life on Earth. ### Engineering - Systems Performance: Enterprise and the Cloud | by Brendan Gregg | rating 4.49 A must-read for engineers deploying production code. Gregg covers performance analysis methodology, tools, and techniques at every layer of the stack. This book will change how you think about observability, bottlenecks, and system behavior under load. ### Economics & Money - Seeing Like a State | by James C. Scott | rating 4.21 How large-scale schemes to improve the human condition fail when they ignore local knowledge and complexity. The concept of legibility, making society readable to administrators, explains many pathologies of modern institutions. - The Sovereign Individual | by James Dale Davidson and William Rees-Mogg | rating 4.18 Written in 1997, this book predicted much of what the internet would do to the relationship between individuals and states. Its framework for understanding how technology shifts power remains remarkably useful for analyzing current dynamics. - The Use of Knowledge in Society | by F.A. Hayek | rating 4.37 A short essay that explains why decentralized coordination through prices often outperforms central planning. The argument is about information: knowledge is dispersed across millions of minds and cannot be aggregated into a single plan without losing most of what makes it valuable. - More Money Than God | by Sebastian Mallaby | rating 4.11 The definitive history of hedge funds, from Alfred Winslow Jones's original market-neutral fund through the quant revolution and the 2008 crisis. Mallaby shows how hedge funds pioneered risk management techniques, exploited market inefficiencies, and shaped modern finance — while repeatedly blowing up in spectacular fashion. - Broken Money | by Lyn Alden | rating 4.00 A clear explanation of how money systems have become dysfunctional and what potential fixes exist. Alden combines engineering precision with financial depth to analyze monetary systems from first principles. - The Last Economy | by Emad Mostaque | rating 4.39 A guide to understanding how AI will fundamentally transform economics. Mostaque argues that we are entering a new era where traditional economic models break down as intelligent systems reshape production, labor, and value creation. Essential reading for anyone trying to understand the economic implications of artificial intelligence. - Principles for Changing World Order | by Ray Dalio | rating 4.27 A study of the rise and decline of reserve currencies and the empires behind them. Dalio examines cycles spanning centuries to identify patterns that might indicate where we are in the current cycle. Especially relevant during pivotal societal moments. - Bubbles and the End of Stagnation | by Byrne Hobart and Tobias Huber | rating 3.82 A fresh perspective on financial bubbles as engines of progress rather than purely destructive forces. The Bitcoin chapter is particularly insightful for understanding how speculative energy can drive technological adoption. ### History - The Age of Revolution, The Age of Capital, and The Age of Empire | by Eric Hobsbawm | rating 4.23 A trilogy covering 1789-1914 that provides the historical framework to understand how the modern world took shape. Hobsbawm traces how the dual revolutions (French and Industrial) transformed everything from politics to daily life. Essential context for understanding current changes. ### Business & Strategy - Working Backwards | by Colin Bryar and Bill Carr | rating 4.20 The most important book for the Ethereum community and anyone building products. Former Amazon executives explain the internal mechanisms that allowed Amazon to innovate consistently, including the famous six-page memo and working backwards from the customer. - Only the Paranoid Survive | by Andrew S. Grove | rating 3.98 Grove's framework for detecting strategic inflection points, moments when the fundamentals of a business change. Learning to recognize these transitions early is essential for building organizations that survive and adapt over time. - Zero to One | by Peter Thiel | rating 4.15 The counterintuitive idea that competition is for losers. Thiel argues that the most valuable companies create something entirely new rather than competing in existing markets. Building a monopoly through uniqueness is more sustainable than fighting for market share. - The Luxury Strategy | by Jean-Noël Kapferer and Vincent Bastien | rating 4.24 An amazing book on the difference between premium and luxury. The authors argue that luxury brands follow different rules than traditional marketing, anti-laws that seem counterintuitive but explain why certain brands maintain their power across generations. - Never Split the Difference | by Chris Voss | rating 4.34 Negotiation as a fundamental human skill, not just a business tactic. Voss, a former FBI hostage negotiator, shows that negotiation principles apply to nearly all human interactions. The techniques here are immediately practical. --- ## Book Backlog Books I want to read next URL: https://federicocarrone.com/reading/#want-to-read ### Read Soon - Seeing Like a State | 1998 | rating 4.21 URL: https://yalebooks.yale.edu/book/9780300078152/seeing-like-a-state/ Scott on how states simplify complex realities to make them legible and controllable, and the catastrophes that follow. Essential for anyone building systems that govern human behavior. - Gödel, Escher, Bach | 1979 | rating 4.29 URL: https://www.hachettebookgroup.com/titles/douglas-r-hofstadter/godel-escher-bach/9780465026562/ Hofstadter on self-reference, formal systems, and consciousness. How meaning emerges from meaningless symbols. The book that launched a generation of interdisciplinary thinking. - The Revolt of the Public | 2014 | rating 4.19 URL: https://www.stripepress.com/books/the-revolt-of-the-public Gurri on how the information revolution destroyed the authority of institutions without replacing them. The best framework for understanding the last decade of politics. - The Machiavellians | 1943 | rating 4.27 URL: https://books.apple.com/us/book/the-machiavellians/id6446856171 Burnham on the elite theorists (Mosca, Pareto, Michels) who argued that all political systems are oligarchies regardless of ideology. Power analysis without illusions. - Nonlinear Dynamics and Chaos | 1994 | rating 4.38 URL: https://www.goodreads.com/book/show/116164.Nonlinear_Dynamics_and_Chaos Strogatz's textbook on dynamical systems. The mathematical foundations of how complex behavior emerges from simple rules. Prerequisite for thinking seriously about complexity. - The Maniac | 2023 | rating 4.36 URL: https://www.penguinrandomhouse.com/books/725022/the-maniac-by-benjamin-labatut/ Labatut on von Neumann, the hydrogen bomb, and the birth of modern computation. Fictionalized history where genius and madness are indistinguishable. The intellectual ancestry of everything we are building now. - The Dawn of Everything | 2021 | rating 4.20 URL: https://www.goodreads.com/book/show/56269264-the-dawn-of-everything Graeber and Wengrow rewrite the standard narrative of human prehistory. Societies were not stuck on a ladder from bands to states. Political organization was always a choice, and people knew it. - The Sovereign Individual | 1997 | rating 4.19 URL: https://www.simonandschuster.com/books/The-Sovereign-Individual/James-Dale-Davidson/9780684832722 Davidson and Rees-Mogg predicted in 1997 that digital technology would erode the nation-state's monopoly on violence and taxation. Written before Bitcoin, more relevant after it. - Chip War | 2022 | rating 4.25 URL: https://www.simonandschuster.com/books/Chip-War/Chris-Miller/9781982172008 How semiconductors became the most contested technology on earth. The geopolitical history of chips, from Texas Instruments to TSMC, and why the US-China competition over fabrication capacity is the defining industrial conflict of this era. - Determined | 2023 | rating 4.23 URL: https://www.penguinrandomhouse.com/books/592344/determined-by-robert-m-sapolsky/ Sapolsky's full case against free will, drawing on neuroscience, genetics, and evolutionary biology. Rigorous and readable. If he is right, every system of punishment and reward needs rethinking. ### Eventually - The WEIRDest People in the World | 2020 | rating 4.21 URL: https://www.goodreads.com/book/show/51710349-the-weirdest-people-in-the-world Henrich on how Western, Educated, Industrialized, Rich, Democratic psychology became the global default. The Catholic Church's marriage policies as the origin of individualism. Changes how you see institutions. - Debt: The First 5,000 Years | 2011 | rating 4.16 URL: https://www.goodreads.com/book/show/6617037-debt Graeber on how debt preceded money, not the other way around. Moral obligation, violence, and the institutional machinery that turns human relationships into accounting. Pairs with The Dawn of Everything. - The Beginning of Infinity | 2011 | rating 4.17 URL: https://www.penguinrandomhouse.com/books/210236/the-beginning-of-infinity-by-david-deutsch/ Deutsch on knowledge creation, the nature of explanation, and why progress has no limit. The most optimistic serious book about the future of civilization. - The Greeks and the Irrational | 1951 | rating 4.20 URL: https://www.ucpress.edu/books/the-greeks-and-the-irrational/paper Dodds on the role of irrationality, madness, and divine possession in Greek culture. The counterargument to the myth of Greece as pure rationalism. - When We Cease to Understand the World | 2020 | rating 4.10 URL: https://www.goodreads.com/book/show/62069739-when-we-cease-to-understand-the-world Labatut's earlier work. Fictionalized accounts of Schwarzschild, Heisenberg, Grothendieck, and others at the edge of knowledge. Science as encounter with the incomprehensible. Bridges genius and madness beautifully. - Complexity: A Guided Tour | 2009 | rating 4.11 URL: https://academic.oup.com/book/35598 Melanie Mitchell's accessible introduction to complexity science. Emergence, self-organization, and computation in biological and social systems. From the Santa Fe Institute tradition. - Meditations | 180 | rating 4.28 URL: https://www.penguinrandomhouse.com/books/292839/meditations-by-marcus-aurelius/ Marcus Aurelius writing to himself about duty, impermanence, and self-governance. A Roman emperor's private journal. The companion piece to Epictetus. - History of Western Philosophy | 1945 | rating 4.13 URL: https://www.simonandschuster.com/books/History-of-Western-Philosophy/Bertrand-Russell/9780671201586 Russell's opinionated survey from the pre-Socratics to logical positivism. Better as intellectual history than neutral philosophy. The prose alone is worth reading. - A Source Book in Chinese Philosophy | 1963 | rating 4.12 URL: https://books.apple.com/us/book/a-source-book-in-chinese-philosophy/id395891594 Wing-Tsit Chan's anthology of primary texts from Confucius through neo-Confucianism. The standard entry point for engaging with Chinese philosophical traditions directly. - The Man Who Solved the Market | 2019 | rating 4.05 URL: https://www.goodreads.com/book/show/43889703-the-man-who-solved-the-market The story of Jim Simons and Renaissance Technologies. How a mathematician built the most successful hedge fund in history using signal processing, not financial theory. - Sync | 2003 | rating 4.07 URL: https://www.goodreads.com/book/show/354421.Sync Strogatz on spontaneous synchronization in nature. Fireflies, neurons, bridges, and planets all following the same mathematical patterns. How order emerges without a conductor. - Against the Gods | 1996 | rating 3.95 URL: https://www.goodreads.com/book/show/128429.Against_the_Gods Bernstein's history of risk. From ancient gambling to modern financial theory. How humanity learned to quantify uncertainty and what that changed about institutions and decision-making. - Four Thousand Weeks | 2021 | rating 4.17 URL: https://www.goodreads.com/book/show/54785515-four-thousand-weeks Burkeman on time, finitude, and the impossibility of optimization. The argument that productivity culture is a defense mechanism against mortality. Philosophy of time that actually lands. - From Bacteria to Bach and Back | 2017 | rating 3.79 URL: https://books.apple.com/us/book/from-bacteria-to-bach-and-back/id1090862474 Dennett on how minds, meaning, and culture evolved from mindless processes. Competence without comprehension as the engine of both biology and technology. - The Technological Republic | 2025 | rating 3.58 URL: https://www.penguinrandomhouse.com/books/721900/the-technological-republic-by-alex-karp/ Alex Karp (Palantir CEO) on the relationship between technology companies and democratic governance. The argument that defense tech is inseparable from political freedom. - Amp It Up | 2022 | rating 4.01 URL: https://ampitupbook.com/ Frank Slootman on operational intensity. The CEO of Snowflake, Data Domain, and ServiceNow on raising the bar, cutting complexity, and refusing to accept mediocre performance. - The Sociology of Philosophies | 1998 | rating 4.12 URL: https://www.goodreads.com/book/show/269724.The_Sociology_of_Philosophies Collins maps the entire global history of philosophy as networks of intellectuals competing for attention space. The big picture of how ideas actually develop: not lone geniuses, but rival groups in structural positions. - Interaction Ritual Chains | 2004 | rating 4.27 URL: https://www.goodreads.com/book/show/879074.Interaction_Ritual_Chains Collins' theoretical core. All of social life reduces to interaction rituals that generate emotional energy and group solidarity. A microsociological engine that powers everything from conversations to revolutions. - The Age of AI | 2021 | rating 3.43 URL: https://ageofaibook.com/ Kissinger, Schmidt, and Huttenlocher on AI's implications for society, security, and the global order. The geopolitical perspective on artificial intelligence from establishment thinkers. --- ## Sources A tighter list of podcasts, publications, blogs, and reference sites I actually return to URL: https://federicocarrone.com/sources/ ### Podcasts - Cognitive Revolution URL: https://www.cognitiverevolution.ai/ - Philosophize This! URL: https://www.philosophizethis.org/ - Sean Carroll's Mindscape URL: https://www.preposterousuniverse.com/podcast/ - History of Philosophy Without Any Gaps URL: https://historyofphilosophy.net/ - Entitled Opinions URL: https://entitledopinions.stanford.edu/ - Dwarkesh Podcast URL: https://www.dwarkeshpatel.com/podcast - Conversations with Tyler URL: https://conversationswithtyler.com/ - EconTalk URL: https://www.econtalk.org/ - Odd Lots URL: https://www.bloomberg.com/oddlots-podcast - Acquired URL: https://www.acquired.fm/ - The Jolly Swagman URL: https://podcasts.apple.com/au/podcast/the-jolly-swagman-podcast/id1267280945 - The Jim Rutt Show URL: https://jimruttshow.com/ - Manifold (Stripe Press) URL: https://press.stripe.com/manifold ### Markets & Political Economy - Financial Times URL: https://www.ft.com/ - FT Alphaville URL: https://www.ft.com/alphaville - Money Stuff (Matt Levine) URL: https://www.bloomberg.com/account/newsletters/money-stuff - Works in Progress URL: https://worksinprogress.co/ - Marginal Revolution URL: https://marginalrevolution.com/ - Epsilon Theory URL: https://www.epsilontheory.com/ - Chartbook (Adam Tooze) URL: https://adamtooze.substack.com/ - Howard Marks Memos (Oaktree) URL: https://www.oaktreecapital.com/insights - The Diff (Byrne Hobart) URL: https://thediff.co/ - Bismarck Analysis URL: https://www.bismarckanalysis.com/ - Doomberg URL: https://doomberg.substack.com/ ### Technology & Systems - Stratechery URL: https://stratechery.com/ - Interconnects URL: https://www.interconnects.ai/ - Construction Physics URL: https://constructionphysics.substack.com/ - SIGPLAN Blog URL: https://sigplan.org/blogs/ - Hillel Wayne's Newsletter URL: https://buttondown.email/hillelwayne - Robert Harper's Blog URL: https://existentialtype.wordpress.com/ - TYPES Mailing List URL: https://lists.seas.upenn.edu/mailman/listinfo/types-list - Papers We Love URL: https://paperswelove.org/ ### Complexity & Science - Santa Fe Institute Complexity Podcast URL: https://www.santafe.edu/engage/learn/podcasts - Quanta Magazine URL: https://www.quantamagazine.org/ - The Gradient URL: https://thegradient.pub/ - Ergodicity Economics (Ole Peters) URL: https://ergodicityeconomics.com/ - London Mathematical Laboratory Newsletter URL: https://www.lml.org.uk/newsletter ### Crypto & Distributed Systems - Real World Cryptography URL: https://www.realworldcryptography.com/ - IACR ePrint URL: https://eprint.iacr.org/ - Decentralized Thoughts URL: https://decentralizedthoughts.github.io/ - ZKProof Community Blog URL: https://zkproof.org/blog/ ### Philosophy & Culture - The Point Magazine URL: https://www.thepointmag.com/ - Liberties Journal URL: https://libertiesjournal.com/ - The Hedgehog Review URL: https://hedgehogreview.com/ - London Review of Books URL: https://www.lrb.co.uk/ - The New York Review of Books URL: https://www.nybooks.com/ - Harper's Magazine URL: https://harpers.org/ - n+1 URL: https://www.nplusonemag.com/ ### Geopolitics & Strategy - 421 URL: https://www.421.news/en/ - Foreign Affairs URL: https://www.foreignaffairs.com/ - Le Grand Continent URL: https://legrandcontinent.eu/ - Palladium Magazine URL: https://www.palladiummag.com/ - Phenomenal World URL: https://www.phenomenalworld.org/ - Noahpinion URL: https://www.noahpinion.blog/ - Branko Milanovic URL: https://glineq.blogspot.com/ - Limes URL: https://www.limesonline.com/ - War on the Rocks URL: https://warontherocks.com/ - The Strategy Bridge URL: https://thestrategybridge.org/ ### Aesthetics - Jancis Robinson URL: https://www.jancisrobinson.com/ - Jon Bonné URL: https://jonbonne.substack.com/ - Decanter URL: https://www.decanter.com/ - The Burlington Magazine URL: https://www.burlington.org.uk/ - Delayed Gratification URL: https://www.slow-journalism.com/ ---