This is a population-wide view of the three fresh ImageNet runs. Every native SwiGLU product in every block is included at every saved checkpoint. Nothing is filtered out because its response is weak or inconvenient.
Following one neuron
Dot positions and indices stay fixed within a model. Moving the training slider follows the literal same neuron, even when its preferred images change. Independent seeds do not have matched neuron identities. Stage and block labels are one-based; neuron numbers and the raw coordinate names retain their zero-based indices.
Positive and negative responses
A SwiGLU neuron can output either sign. “Mean · negative” ranks images by the negative of the spatial mean. “Spatial trough · negative” uses the negative minimum. Changing the convention changes what counts as a strong response; it does not create another neuron.
Color and curves
AUROC measures how often a positive image ranks above a negative one. 0.5 is chance ordering; 1 is perfectly selective in the positive direction; 0 is perfectly inverse. Each target is a different comparison. Ambulance and cab have only 25 evaluation positives each; the road comparison has 44 other-road negatives.
The dashed lines
For the selected neuron, we compare its response with every neuron in strictly earlier blocks using the same response convention, then show the eight strongest absolute Pearson correlations. The sign is preserved. This is response similarity, not a weight, anatomical edge, ablation, or evidence of causal inheritance. Common inputs, class membership, color and background can all cause correlation.
How the images were sampled
All scores, examples and associations use the same frozen 4,210-image evaluation half: 250 car-family images plus four per other ImageNet class. The exact 224-pixel center crops used by the model appear here. Cars are deliberately overrepresented; this affects response correlations. The original discovery half selected only the three labeled starting points.
What remains unresolved
Browsing thousands of evaluation scores is exploratory and can overfit the evaluation set. An image ranking is not a complete neuron interpretation. Temporal precedence and correlated activation do not establish consolidation or inheritance. The sparse ablation study remains in the linked report; it does not define this map’s edges.
Sixteen measured snapshots cover 25,020 updates, or ten epochs over 1,281,167 training images. The slider spaces saved checkpoints equally; no intermediate measurements are invented.