🎵 What Is an LLM?

The page in one sentence

Prediction Ridge (预测山脊) is an interactive 3D animation that treats a melody as a sequence which is constantly predicting its next step — and uses that picture to explain what a large language model does.

There is no long essay behind the animation. The page gives you a small stage: the current note, the next candidate note (for example 1 / Do), a timeline running from 00 to 32, and a prediction-certainty percentage. Press play and the melody advances note by note; the percentage shows how confident the model is about what comes next at each step.

The animation uses a simplified probability model to simulate sequence prediction. It is not a direct visualisation of the neuron activations inside a real LLM — the source page says so explicitly.

It is a small piece of interaction rather than a lecture: Play advances the melody, Reset returns to the first note; you drag to rotate the 3D view, scroll to zoom, and double-click to restore the default camera. Watching the certainty figure move as the tune progresses is the whole lesson.

Why a melody is a good stand-in for text

Music and language share a structure that matters here: both are sequences where each step is only meaningful in the context of what came before. After a certain run of notes, some continuations feel natural and others feel wrong. A melody that jumps at random stops sounding like a melody.

The metaphor is useful because it removes the mystique. A model is not retrieving a finished paragraph from a drawer; it is choosing one small step at a time, with the choice shaped by everything already played.

How next-token prediction works

1. Split the text into tokens. The input is broken into pieces — roughly words or word fragments. Each token has an ID, and each ID maps to a vector.
2. Read the whole context. The model processes the tokens so far and builds an internal representation of the sequence. In a transformer this is done with attention, which lets every position look at the positions before it.
3. Score every candidate. The final layer produces a score for every token in the vocabulary. After a softmax, those scores become a probability distribution over "what comes next".
4. Pick one. The model selects a token — usually by sampling from that distribution rather than always taking the single most likely one. That is why the same prompt can produce different answers.
5. Append and repeat. The chosen token is added to the sequence, and the whole loop runs again. One generated word is one complete pass of this cycle.

In the melody version, steps 1–4 are the animation: at each of the 32 positions the page holds a distribution over the next note, displays the leading candidate together with a certainty figure, then commits to a note and moves the cursor forward.

Reading the "ridge"

The page is called Prediction Ridge for a reason. Imagine the space of all possible next moves as a landscape. Most continuations are flat, unlikely ground. The plausible ones form a narrow ridge running through it. Generation is a walk along that ridge — at every step the model picks a point on the ridge, then re-draws the landscape from the new position.

• A sharp ridge means high certainty: after "the capital of France is", almost all the probability sits on one token.
• A flat, wide ridge means low certainty: after "once upon a", many continuations are legitimate, so the percentage drops.
• A wrong but confident step is how a model can produce a fluent sentence that is factually false. Fluency measures the shape of the ridge, not the truth of the destination.

What the metaphor explains — and what it leaves out

It explains why LLMs are fluent, why they are sensitive to the exact wording of a prompt, why they can continue almost any pattern (prose, code, a melody), and why they sometimes drift: a small early choice changes the landscape for every later choice.

It does not explain where the probabilities come from. The ridge is the product of training on a very large amount of text, where the model repeatedly adjusted its parameters to make the observed next token more likely. The song analogy shows the shape of the process, not its cause — and, as the page warns, the animation is a simplified model, not a look inside a network.