The Entropy Gate
An arcane codex of probability & statistics —
the mathematics behind luck itself.
ENTROPY GATE — scatter, then settle
Open the codexChapter I
Foundations — The Grammar of Chance
Probability begins with a deceptively simple act: listing everything that can happen. The sample space Ω is the set of all possible outcomes of an experiment; an event is a subset of Ω. Flip three coins and the sample space is the eight strings HHH, HHT, …, TTT; the event "exactly one head" is the subset {HTT, THT, TTH}. Probability assigns each event a number in [0, 1], and the entire theory — from blackjack to quantum mechanics — is disciplined by three axioms.
A probability measure P on Ω assigns every event A a number P(A) such that:
- P(A) ≥ 0 for all events A (non-negativity);
- P(Ω) = 1 (normalization);
- For disjoint events A₁, A₂, … (countably many), P(∪ᵢ Aᵢ) = Σᵢ P(Aᵢ) (countable additivity).
Everything else — conditional probability, expectation, the CLT — is a corollary of these three sentences.
Counting is the engine of classical probability. When every outcome of a finite sample space is equally likely, P(A) = |A| ⁄ |Ω| — so computing a probability is a problem of counting sets. The two fundamental counters are permutations (ordered selections) and combinations (unordered selections):
P(n, r) = n!(n − r)! C(n, r) = n!r! (n − r)!
With repetition allowed, ordered counting gives nr, and unordered counting gives the stars-and-bars count C(n + r − 1, r). These four quantities generate every poker hand, every lottery edge, and most of discrete probability.
For P(B) > 0, the conditional probability of A given B is
P(A | B) = P(A ∩ B)P(B)
Events A and B are independent when P(A ∩ B) = P(A) P(B) — equivalently P(A | B) = P(A), meaning knowledge of B carries no information about A. Most real-world events are dependent; the Law of Total Probability organizes that dependence: if B₁, …, Bₖ partition Ω, then P(A) = Σᵢ P(A | Bᵢ) P(Bᵢ).
Interactive — Sample-Space Explorer
Enumerate Ω completely, then measure an event by direct counting: P(A) = |A| ⁄ |Ω|.
Interactive — Combinatorics Calculator
Exact BigInt arithmetic — no floating-point rounding, even for 60-choose-30.
Values beyond memory are exact big integers — formatted with digit grouping. Cap: n ≤ 60 keeps nine-digit displays sane.
Interactive — Conditional Probability Table
Edit the 2×2 contingency counts. Watch P(A | B) ≠ P(B | A) — and test independence, P(A ∩ B) vs. P(A)P(B).
| B: carries umbrella | ¬B: no umbrella | |
|---|---|---|
| A: rains | ||
| ¬A: no rain |
Chapter II
Distributions — The Shapes of Randomness
A random variable X is a function from outcomes to numbers — it turns "the coin came up heads" into X = 1. Its behavior is captured by a probability mass function (PMF) for discrete variables, p(x) = P(X = x), or a probability density function (PDF) for continuous ones, with the cumulative distribution function F(x) = P(X ≤ x) unifying both.
Two numbers summarize any distribution. The expected value E[X] = Σ x p(x) is the long-run average; the variance Var(X) = E[(X − μ)²] measures spread. Higher moments refin the picture: skewness γ₁ = E[(X−μ)³] ⁄ σ³ detects asymmetry, and kurtosis (excess γ₂ = E[(X−μ)⁴] ⁄ σ⁴ − 3) measures tail weight.
- Binomial(n, p): the number of successes in n independent trials, each succeeding with probability p. P(X = k) = C(n, k) pᵏ (1−p)n−k; μ = np, σ² = np(1−p). Models: streaks, win counts.
- Poisson(λ): the limit of Binomial as n → ∞, p → 0 with np = λ fixed — counting rare events. P(X = k) = e−λλᵏ ⁄ k!; μ = σ² = λ.
- Geometric(p): the waiting time until the first success. P(X = k) = (1−p)k−1p; μ = 1/p. It is memoryless: P(X > m+n | X > m) = P(X > n).
- Normal(μ, σ²): the continuous limit of aggregated noise. f(x) = 1σ√(2π) exp(−(x−μ)² ⁄ 2σ²). Symmetric; skew 0, excess kurtosis 0. Its CDF Φ is computed here via an erf rational approximation, and its quantile Φ⁻¹ by bisection on Φ.
Interactive — Distribution Playground
PMF/PDF bars (violet) with the CDF overlaid on the right axis (gold). Then draw real samples and watch the empirical histogram converge.
Chapter III
Core Theorems — Why Order Emerges from Chaos
Three theorems elevate probability from bookkeeping to law. The Law of Large Numbers says averages converge to expectations; the Central Limit Theorem says the shape of that convergence is always the same bell; and Bayes' Theorem tells you exactly how evidence should move belief.
Let X₁, X₂, … be i.i.d. with E[Xᵢ] = μ, and let p̂n = (X₁ + … + Xₙ) ⁄ n. The weak law: for any ε > 0, P(|p̂n − μ| > ε) → 0 as n → ∞. The strong law (Kolmogorov): P(p̂n → μ) = 1 — convergence happens on almost every infinite sequence. Chebyshev's inequality makes the rate explicit: P(|p̂n − μ| ≥ ε) ≤ σ² ⁄ (n ε²) — the ±3σₜ band drawn in the simulator below shrinks like 1 ⁄ √n.
For i.i.d. Xᵢ with mean μ and finite variance σ², the standardized sample mean satisfies
p̂n − μσ ⁄ √n ⟶ N(0, 1) in distribution as n → ∞.
The miracle is universality: dice, coins, incomes, lifetimes — any population with finite variance yields bell-shaped sample means. The standard error σ ⁄ √n quantifies exactly how sampling size purchases precision: quadrupling n halves the spread.
P(A | B) = P(B | A) P(A)P(B) = P(B | A) P(A)P(B | A)P(A) + P(B | ¬A)P(¬A)
Posterior belief = prior belief × likelihood of the evidence, renormalized. The classic trap — the base-rate fallacy — is reading P(disease | positive test) as if it were the sensitivity P(positive | disease). When disease prevalence is 1%, even a 99%-sensitive, 95%-specific test yields a posterior of only ≈ 17%. The denominator carries the base rate; intuition never does.
Interactive — Law of Large Numbers
A biased coin flipped at speed. The running proportion (gold path) is reined into the true bias (white line); the ±3σₜ convergence band shrinks like 1 ⁄ √n.
Interactive — Central Limit Laboratory
Sample means from skewed populations. At n = 1 the histogram copies the population; raise n and the bell asserts itself. The gold curve is the fitted N(μ, σ² ⁄ n).
Interactive — Bayes and the Base-Rate Trap
1,000 people: true positives, false positives, false negatives, true negatives. Prevalence quietly dominates the posterior.
Intuition says "the test is 99% accurate." Arithmetic disagrees — the denominator is crowded with healthy false positives.
Chapter IV
The Gambler's Path — A Cautionary Calculus
Casinos do not gamble; they engineer expected value. The expected value of a bet that pays b units on a win (probability p) and loses the 1-unit stake otherwise is EV = p·b − (1 − p). European roulette's single number pays 35:1 against success probability 1 ⁄ 37, giving EV = −1 ⁄ 37 ≈ −2.70% per unit staked; the American double-zero wheel costs −2 ⁄ 38 ≈ −5.26%. The house edge is the negative of your EV — and it is assessed on every unit wagered, every spin.
A gambler starts with i units, wins each round with probability p (q = 1 − p), gains or loses 1 unit per round, and stops on reaching 0 (ruin) or N (target). Solving the recurrence u(i) = p·u(i+1) + q·u(i−1) with boundary conditions u(0) = 0, u(N) = 1:
P(reach N before ruin) = 1 − (q ⁄ p)i1 − (q ⁄ p)N for p ≠ ½, and i ⁄ N for p = ½.
When p < ½ the formula is brutal: raise the target N and survival probability collapses geometrically. Persistence is not a strategy; it is a denominator.
Double your bet after every loss; on the first win you recoup everything plus one base unit. The scheme converts many small wins into rare, ruinous losses — the probability of busting after k consecutive losses is qk, never zero. Crucially, expectation is linear and indifferent to staking schemes: every spin still carries EV = (p − q)·bet < 0, so no sequence of negative-EV bets can sum to a positive-EV system. A finite bankroll merely decides when the arithmetic collects.
Independent trials have no memory: after five heads, P(heads) is still p — "tails is due" is the gambler's fallacy. Its mirror, the hot-hand fallacy, projects streaks onto independent processes and expects them to continue. Both are refuted by the same line of algebra: P(A | B) = P(A) when A and B are independent. Streaks are not information; they are realizations.
Interactive — Expected Value & House Edge
Set win probability p and net payout b per 1 unit staked. EV = p·b − (1 − p). Negative EV means the house collects the difference as its edge.
| Bet | p | Payout | EV per unit | House edge |
|---|
Interactive — Gambler's Ruin Random Walks
Each path is one bankroll's walk between 0 and N. Simulation counts drift toward the exact ruin formula — theory and tally converge as runs accumulate.
Interactive — Martingale Under a Finite Bankroll
Double after losses until one win (profit of one base bet) or the bankroll can no longer cover the stake (bust). Hundreds of tiny profits; occasional total ruin. The expectation never moves.
Interactive — The Coin Has No Memory
Flip a fair coin. heads, tails. However long the streak, P(next heads) = 0.5 — independence is the whole story.
You just watched 5 heads in a row. Honest odds: what is P(tails on the next flip)?
The codex awaits your answer.
Chapter V
Advanced Methods — Measuring the Unseen
Mature statistics is a technology for extracting signal from randomness. Monte Carlo methods replace hard integrals with cheap samples; hypothesis testing formalizes "is this real?"; Bayesian updating turns evidence into movement of belief; and Shannon entropy prices uncertainty itself in bits.
To estimate π, throw uniform darts at the unit square: the fraction landing inside the quarter circle is π ⁄ 4, so π̂ = 4·(# inside) ⁄ N. By the CLT the error is O(1 ⁄ √N) — quadrupling your darts halves the error. The same σ ⁄ √N law governs every simulation on this page: scatter, then settle.
Testing H₀: p = p₀ for a coin with K heads in n flips uses the z-statistic
z = K − n p₀√(n p₀ (1 − p₀))
with two-sided p-value 2(1 − Φ(|z|)). Rejecting a true H₀ is a Type I error (controlled at α, here 0.05); failing to reject a false H₀ is a Type II error. Power — the chance of catching a real deviation — grows with sample size; significance is a property of the evidence, not a switch.
Competing hypotheses (fair coin θ₁ = 0.5 vs. biased θ₂ = 0.7) update by multiplying the prior odds by the likelihood ratio: heads multiplies P(H | θ₁) ⁄ P(H | θ₂), tails the converse. In log-odds form each flip adds a fixed increment — evidence accumulates linearly in the log, so posterior probability sigmoids toward the truth. Bayes factors make belief revision arithmetic rather than opinion.
H(X) = −Σx p(x) log₂ p(x) — the average surprise, in bits, of sampling X. For k outcomes, H is maximized by the uniform distribution at H = log₂ k bits, and collapses to 0 when a single outcome is certain. Entropy and variance are twin measures of scatter: variance weighs squared distance, entropy counts unexpected symbols. A fair die carries log₂ 6 ≈ 2.585 bits; a loaded die squanders them. The gate opens at maximum entropy; knowledge closes it.
Interactive — Monte Carlo π
Darts at the unit square (gold inside the quarter circle, violet outside). π̂ = 4 · (inside ⁄ total); the error chart tracks |π̂ − π| against N with a ∝ 1 ⁄ √N reference band.
Interactive — Hypothesis Testing a Suspicious Coin
H₀: the coin is fair (p₀ = 0.5), α = 0.05. Flip it, and the z-statistic converts the head-count into a two-sided p-value. Mystery mode hides the truth — reveal it when your verdict is in.
Interactive — Bayesian Updating, Flip by Flip
Two hypotheses: θ₁ fair (p = 0.50) or θ₂ heads-biased (p = 0.70). Each flip multiplies the odds by a Bayes factor; the posterior bar renormalizes. Choose the true coin — or let fate choose.
Interactive — Entropy Laboratory
H = −Σ p log₂ p. The uniform distribution maximizes entropy at log₂(k); every skew leaks bits. Watch a die's 2.585 bits drain away as you load it.