One Hop Through Attention

Field notes

The explainer uses a deliberately small transformer—two layers and 48 dimensions rather than the paper’s six layers and 512 dimensions—to make the forward pass inspectable. Its example task converts dates written in messy formats into yyyy-mm-dd, giving the attention mechanism a concrete input and output. Visitors can choose a character, follow query-key scoring through softmax, and see value vectors blended; other modules expose scaling, multi-head behavior, positional encoding, and masking.