Sequences of Bernoulli Outcomes with the Same Probability
A first-principles probability note on why conditioning on the total
count makes all Bernoulli sequences with exactly k
successes equally likely.
Diffusion from First Principles
Notes, derivations, and small experiments from my ongoing study of diffusion and generative models.
I'm working through a 52-week curriculum at my own pace, starting with probability and linear algebra and moving toward diffusion models, SDEs, and flow matching. These posts are where I share the questions I'm working through, the things I try, and what I think I learned.
Read from the beginning or pick a theme. The notes follow publication order, so you can follow the questions as they developed.
Probability and score functions
From conditioning and expectation to Gaussian scores and the limits of denoising.
-
-
Expected Value of a Simple Forward Process
A diffusion note on what it really means for noisy images to stay centered around the original image on average.
-
Score of a Gaussian
A derivation-driven note on why the score of a one-dimensional Gaussian points toward higher log-density, which means back toward the mean.
-
Denoising Is Not Inversion
A conditional-expectation note on why denoising under squared error is estimation, not exact inversion.
-
When the Score Does Not Point Straight to the Mean
An interactive look at how covariance reshapes the score of a multivariate Gaussian, and why the direction of higher density does not always point straight to the mean.
Transforming densities and learning to denoise
Change of variables, posterior averages, a toy score-matching experiment, and reverse steps.
-
Changing Variables Without Changing Probability
A short derivation of the density of a transformed random variable, with intuition for how the Jacobian compensates for stretched lengths and areas.
-
Averaging Paths to the Peaks of Our Mountains
A derivation of how the noisy marginal score connects to the posterior conditional mean through an average over plausible clean samples.
-
Two Ways to Learn the Same Score Field
A small experiment comparing two ways to train a toy neural network to learn the score of a noisy three-component Gaussian mixture.
-
KL Divergence Is Not a Distance
A daily-life example using car and train commute-time distributions to understand why KL divergence is asymmetric and how it can be used to compare distributions.
-
Sampling Our Way Down, One Gaussian Step at a Time
A small-step Bayes derivation showing why the reverse conditional can be approximated by a Gaussian even when the previous-state marginal is not Gaussian.
Gaussian geometry and linear representations
Connecting the forward process to affine transformations, then asking what PCA leaves out.
-
Forward Process, Transformations, and Gregor Samsa
A derivation and geometric playground connecting the affine Gaussian transformation in a diffusion forward process to covariance and the Jacobian determinant.
-
In Search of PCA’s Lost Variance
A geometric derivation of why PCA reconstruction mean-squared error equals the sum of the discarded eigenvalues, through orthogonal projections and a 2D example.
Latent variables and Monte Carlo
From sampled latent vectors to expected reconstruction losses.
-
When the VAE Meets Monte Carlo
Why averaging sampled reconstruction losses estimates an expectation in a VAE, through the law of large numbers and an indicator-variable approach to integration.
Browse all writing or subscribe via RSS for new notes.