Blog

Why cosine is standard, how recos follows from the rearrangement inequality, and how both metrics can be understood as orbit-normalized alignments under different symmetry groups. Includes random-vector asymptotics, rotation tests, STS results, kernel diagnostics, and interactive laboratories.

A careful overview of IMLE: the core formulas, nearest-sample intuition, algorithm pseudocode, animated synthetic fits, and interactive laboratories for Gaussian, Gaussian-sum, Cauchy, and failure cases.

A method-by-method review of value, Perlin, simplex, OpenSimplex, cellular, Gabor, phasor, wave, and wavelet noise, followed by octave combinations and domain warping. Five interactive laboratories connect the constructions to their visual and spectral behavior.

One matrix wearing three hats: the precision of the best estimate in statistics, the metric of information geometry, and the curvature behind second-order optimizers. Built up from the Hessian of a log-likelihood — the score's two moments, the osculating circle at the MLE, the Cramér–Rao bound, the KL-divergence Hessian, the reparameterization tensor law, and the Gauss–Newton decomposition.

Suh's completion capacity recasts matrix completion as a Shannon problem: how many entries does one observation resolve? A walk through the theory, an implementation that verifies every closed form, a GF(2) decoder that exhibits the threshold, and several verified extensions.

One method, five derivations: best linear approximation, minimal reconstruction error, metric MDS, maximum variance with decorrelation, and the SVD. All reduce to the eigenvectors of the covariance matrix.

The two standard methods for estimating model parameters: definitions, worked examples, how they relate as the sample grows, and the zero-count problem that motivates smoothing.

From the likelihood of a Bernoulli model, cross-entropy follows as the negative log-likelihood, and the same quantity reappears as KL divergence, code length, a proper scoring rule, and mode-covering KL. Concludes with the comparison to squared error for classification.