How much do we lose?

Rotate the line through the mean. The data stay fixed; their projections change.

0°

At 0°, we keep the first principal direction.

The data form a tilted cloud. Rotating the projection line away from its first principal direction increases the reconstruction error.

Blue points are the original data. Orange circles are their reconstructions on the blue projection line. Gray segments show every twentieth residual. The dashed gray line marks the fixed first principal direction. The small cross marks the mean. Both axes use the same scale.

Retained variance
8.194
Reconstruction MSE
0.314
Total variance
8.509
Minimum MSE · λ₂
0.314
Retained varianceDiscarded variance

Retained variance + MSE = total variance.

Here, MSE = λ₂ ≈ 0.314. Rotating the line increases the error.

The same 400 points as in the notebook. MSE averages squared distances over the points, using 1/n as in the post.