Research Log
When the VAE Meets Monte Carlo
Why averaging sampled reconstruction losses estimates an expectation in a VAE, through the law of large numbers and an indicator-variable approach to integration.
Read noteWriting
Published notes, tutorials, and research logs from personal machine learning projects.
Research Log
Short, question-first notes on papers, derivations, experiments, and ideas I want to understand better.
Research Log
In Search of PCA’s Lost Variance
A geometric derivation of why PCA reconstruction mean-squared error equals the sum of the discarded eigenvalues, through orthogonal projections and a 2D example.
Read noteResearch Log
Forward Process, Transformations, and Gregor Samsa
A derivation and geometric playground connecting the affine Gaussian transformation in a diffusion forward process to covariance and the Jacobian determinant.
Read noteResearch Log
Sampling Our Way Down, One Gaussian Step at a Time
A small-step Bayes derivation showing why the reverse conditional can be approximated by a Gaussian even when the previous-state marginal is not Gaussian.
Read noteResearch Log
KL Divergence Is Not a Distance
A daily-life example using car and train commute-time distributions to understand why KL divergence is asymmetric and how it can be used to compare distributions.
Read noteResearch Log
Two Ways to Learn the Same Score Field
A small experiment comparing two ways to train a toy neural network to learn the score of a noisy three-component Gaussian mixture.
Read noteResearch Log
Averaging Paths to the Peaks of Our Mountains
A derivation of how the noisy marginal score connects to the posterior conditional mean through an average over plausible clean samples.
Read noteResearch Log
Changing Variables Without Changing Probability
A short derivation of the density of a transformed random variable, with intuition for how the Jacobian compensates for stretched lengths and areas.
Read noteResearch Log
When the Score Does Not Point Straight to the Mean
An interactive look at how covariance reshapes the score of a multivariate Gaussian, and why the direction of higher density does not always point straight to the mean.
Read noteResearch Log
Denoising Is Not Inversion
A conditional-expectation note on why denoising under squared error is estimation, not exact inversion.
Read noteResearch Log
Score of a Gaussian
A derivation-driven note on why the score of a one-dimensional Gaussian points toward higher log-density, which means back toward the mean.
Read noteResearch Log
Expected Value of a Simple Forward Process
A diffusion note on what it really means for noisy images to stay centered around the original image on average.
Read noteResearch Log
Sequences of Bernoulli Outcomes with the Same Probability
A first-principles probability note on why conditioning on the total
count makes all Bernoulli sequences with exactly k
successes equally likely.
Articles and Tutorials
Tutorial
Using Hugging Face to Train a GPT-2 Model for Music Generation
A practical walkthrough for preparing symbolic-music data, training a tokenizer and GPT-2-style model, and deploying an interactive music demo.
Read article