# Early ImageNet consolidation: three fresh runs

The measured trajectories show broad car-family selectivity before stronger ambulance selectivity in the final selected coordinates. They do not yet demonstrate a causal generalist-to-specialist consolidation mechanism.

Two 91,828-parameter models (seeds 0 and 1) and one 345,552-parameter model (seed 0) trained from scratch for 25,020 updates, or ten ImageNet epochs. All 1,281,167 training images were available. Sixteen checkpoints were analyzed, with dense sampling in the first epoch. The recipe applies the same mixup to images and labels, correcting a label-only mixup issue found in the historical trainer. These runs therefore should not be treated as exact replications of the historical pilot.

Each score uses 8,420 validation images split into disjoint discovery and evaluation halves. The broad family contains ambulance, beach wagon, cab, convertible, jeep, limousine, minivan, Model T, racer, and sports car. There are 250 evaluation positives, including 25 ambulances and 25 cabs. Native-coordinate candidates include positive and negative means and spatial extremes of every SwiGLU product. The illustrated coordinates were selected on discovery data at epoch 10, then followed literally backward through training; the curves use evaluation data.

| Run | Fixed broad coordinate first sustained ≥0.85 | Fixed ambulance coordinate first sustained ≥0.80 | Car AUROC at epoch 10 | Car vs other road vehicles | Ambulance vs other cars |
|---|---:|---:|---:|---:|---:|
| Tiny, seed 0 | Between epochs 4 and 5 | Between epochs 5 and 7 | 0.861 | 0.507 | 0.864 |
| Tiny, seed 1 | Between epochs 2 and 3 | Between epochs 4 and 5 | 0.926 | 0.537 | 0.849 |
| Wider, seed 0 | Between epochs 1 and 2 | Between epochs 4 and 5 | 0.970 | 0.615 | 0.934 |

“Sustained” means two consecutive measured checkpoints above the declared threshold. It is an operational description, not a biological phase transition, a confidence interval on an onset time, or a population estimate. None of the fixed final cab coordinates meets the sustained 0.80 criterion within ten epochs. Other coordinates can be transiently more cab-selective.

The multi-neuron readout is already strong at random initialization (AUROC 0.855–0.874). Dataset structure and generic visual cues therefore account for part of the broad score. The hard comparison against other road vehicles is much weaker, with only 44 held-out hard negatives. “Car-selective coordinate” is a measurement label, not a demonstrated complete neuron interpretation.

## Testing inheritance

The strongest final ambulance coordinate is upstream of the strongest final broad car coordinate in all three models. It therefore cannot inherit through an ordinary forward connection from that particular late generalist. This matters for the proposed explanatory sequence.

For each run, a separate candidate was selected at approximately epoch 1: the best discovery-set broad car coordinate strictly upstream of the eventual ambulance coordinate. At epochs 0, 1, 2, 5, and 10, we zeroed the candidate’s SwiGLU product and compared it with 16 same-block controls matched on RMS residual contribution scale. All selection and scale matching used discovery images. Evaluation used 250 held-out car images plus 250 sampled held-out non-car images.

| Run | Early upstream candidate | Final ambulance AUROC before → after ablation | Final response decrease (SD) |
|---|---|---:|---:|
| Tiny, seed 0 | S1.B1.N46, negative mean | 0.8636 → 0.8644 | +0.0140 |
| Tiny, seed 1 | S2.B1.N31, negative mean | 0.8492 → 0.8492 | −0.0052 |
| Wider, seed 0 | S1.B1.N34, positive mean | 0.9342 → 0.9303 | +0.0496 |

Indices in the coordinate names are zero-based. Positive response decrease means the selected oriented signal fell under ablation. Effects lie within the range of matched controls. These interventions do not identify a privileged precursor. They test one selection rule and one candidate per run; they do not exclude distributed ancestry, redundancy, other routes, or later development.

## What follows from this

The experiment supports a developmental difference in measured selectivity, with substantial seed and capacity dependence. It does not establish semantic dust consolidating into a causally reused generalist, nor does it prove that the emerging ambulance score corresponds to an ambulance concept rather than a correlated visual feature. The most useful next mechanistic step is spatially localized evidence for the early ambulance-selective units, followed by input-path interventions and matched feature controls. A cohort of multiple candidate units, further seeds, and more hard negatives would be needed for a broader consolidation claim.

The earlier 90-epoch pilot remains exploratory because of its training-recipe issue. It is retained separately in `figures/`, along with the raw metadata, rather than merged with the fresh-run results.

## Artifacts

- `fresh-figures/early-training.png`, `.svg`, `.pdf`: fixed-coordinate trajectories.
- `fresh-figures/early-training-controls.png`, `.svg`, `.pdf`: hard negatives and causal controls.
- `fresh-figures/summary.json`: compact numerical results.
- `fresh-*/metrics.json`, `lineage.json`, `causal.json`: complete exported measurements.
- `PROTOCOL.md`: initial operational definitions and controls.

Training checkpoints and full probe arrays remain in the devbox experiment directory. No image in these scientific figures is generated by an image model.