Part of the Pristone Academy AI Technical Track

Principal Component Analysis — the art of the right angle.

How a machine throws away most of your data and keeps almost all of the meaning.

200,000 FT

The Big PictureWhy it exists

Most of what you measure is the same thing wearing different costumes.

A couple is planning a wedding. They survey all 200 guests on 30 possible dishes — rate each one, one to five. That's 30 numbers per guest, 6,000 in total. It feels like 30 independent opinions. It isn't.

Because tastes travel together. The guest who loves the braised short rib almost certainly loves the pork belly too. The one who picks the quinoa bowl also wants the roasted-vegetable tart. Those 30 dishes aren't 30 separate questions — they're a handful of underlying tastes in disguise: maybe “how much meat do you want,” “how adventurous are you,” “how rich versus light.” Three dials, dressed up as thirty.

PCA is the tool that finds those hidden dials. It looks at all 30 columns and discovers that, really, people only differ along a few directions — and it hands you those directions, ranked by how much they matter.

That's the whole promise from orbit: find the few axes along which things actually differ, and ignore the rest. It doesn't sort people into groups — it re-describes your data using fewer, smarter measurements, so 30 dish-ratings collapse into “this guest is a 2 on meat-love, a 4 on adventure,” and you've lost almost nothing.

The catch, stated once

PCA's new axes are blends of your original ones — “0.4 × short-rib + 0.3 × pork-belly − 0.5 × quinoa…”. Mathematically perfect, but nobody labeled them “meat-love.” The compression is guaranteed; the interpretation is on you.

30,000 FT

The MechanicsHow it runs

Find the direction of greatest spread. Then the next. Then stop early.

Strip away the jargon and PCA is a search for the most informative direction. “Informative” has a precise meaning here: the direction along which the data is most spread out. Spread is information — if everyone scores the same on something, that something tells you nothing.

Rotate the line below. As you turn it, watch each point drop onto it. The more spread those projections, the more of the data's variation that single line preserves. The angle that maximizes that spread is the first principal component — the single best one-dimensional summary of a two-dimensional cloud.

Live · variance-finder

drag to explore

Rotate the line. The purple drop-lines show each point projecting onto it — the more spread the projections, the more information that angle keeps.

Rotate the axis

20°

Variance captured

THIS ANGLE—

BEST POSSIBLE (PC1)—

PC2 HOLDS—

your axisPC1 (max spread)PC2 (perpendicular)

Once you've found that first direction, the second principal component is the most-spread direction at a right angle to it — the best of what's left after the first explanation is removed. Keep going and you get a full set of perpendicular axes, each capturing less than the last. Keep only the top few and you've compressed the data.

UNDER THE HOOD

The Special DirectionsEigenvectors

The principal components are the eigenvectors of the data's spread.

Here's the idea that ties it to the math you may have heard of. Every matrix transforms space — it stretches, squishes, and rotates the points you feed it. For almost every input direction, the output points somewhere new. But a few special directions come out pointing the same way, only longer or shorter. Those are the eigenvectors, and how much they stretch is the eigenvalue.

Drag the input arrow below and hunt for the angles where the output lines up exactly with it. Those are the natural axes of this transformation — and for the matrix that describes how your data is spread (its covariance matrix), those same special directions are the principal components.

Live · eigenvector hunt

drag the arrow

Drag the teal input arrow. The amber output is what the matrix does to it. Hunt for the two angles where they line up exactly.

Input direction

15°

Alignment (input vs output)

IT TURNED BY—

IT STRETCHED ×—

input arrow (you)output = matrix × inputthe eigen-directions

Eigenvectors and the covariance matrix are the heart of linear algebra; “spread” made precise as variance is statistics. This is the kind of bridge between two courses that's much faster to cross with someone drawing it for you.

How many components do you keep?

Each component captures a slice of the total variance. Plot those slices in order and they fall off a cliff: the first few explain most of everything, the rest are rounding error. You keep components up to the bend — enough to capture, say, 95% of the variance — and drop the tail. It's a judgment call, and learning to make it well is exactly the kind of thing a 1:1 session sharpens.

0 FT

What to Carry OffThe summary

Three things to keep about PCA.

Spread is information

PCA ranks directions by how much the data varies along them. The most spread-out direction is the most informative summary you can keep.

The axes are perpendicular and ranked

Each principal component is at a right angle to the others and captures less than the one before — so you can safely keep the top few.

It compresses, it doesn't label

The new axes are blends of your originals. PCA guarantees you lose little; what they mean is your interpretation, not the math's.

Eigenvectors are the engine

The principal components are the eigenvectors of the covariance matrix — the directions the data's spread only stretches, never turns.

The one sentence to remember

PCA finds the few perpendicular directions of greatest spread and re-describes your data using only those — keeping almost all of the meaning in a fraction of the numbers.

Frequently asked questions

PCA, or Principal Component Analysis, finds the few directions along which your data actually varies and re-describes everything using just those. It's a way to take many correlated measurements and compress them into a handful of meaningful ones, losing as little information as possible.

A principal component is a direction through your data that captures as much of the spread (variance) as possible. The first principal component is the single line along which the data is most spread out; the second is the most-spread direction at a right angle to the first, and so on.

It reduces dimensionality. High-dimensional data is hard to visualize, slow to compute on, and full of redundancy because measurements travel together. PCA strips the redundancy, lets you plot complex data in two or three dimensions, and often makes downstream models faster and more robust.

They answer different questions. Clustering sorts items into groups. PCA doesn't group anything — it re-describes each item using fewer, smarter axes. You often use PCA first to compress noisy data, then cluster in that cleaner space.

The principal components are the eigenvectors of the data's covariance matrix, and each one's eigenvalue tells you how much variance that direction captures. Intuitively, eigenvectors are the special directions a transformation only stretches, never turns — which is exactly the 'natural axis' PCA is hunting for.

Intro to Artificial Intelligence

What AI actually is, and how learning from data differs from following rules.

K-Means Clustering

How a machine sorts a pile of points into clean groups with no labels.

Attention Is All You Need

The self-attention idea behind every modern transformer and LLM.

The AI Layer Shift Framework

How AI redistributes work across the layers of a job — and where humans stay scarce.

Curious who's behind these lessons? See the proof of work.