Preprint finds measurable patterns in how AI skills emerge and fade

How skills appear and fade

The researchers introduced occupancy, or the fraction of a model in use for learned behavior. They measured it by resetting a small part of a model to its untrained state and checking whether anything the model had learned became worse. Models with the same loss—a common measure of prediction error—could have occupancy values from 0.006 to 1.00, suggesting occupancy captures something different from loss.

For a capability that requires K separate parts and appears in training data at frequency p, the proposed emergence formula is t* ≈ 1/(νS·p^K·X(F)^K). In plain language, capabilities that need more parts, or that are based on rarer examples, should take longer to appear. The formula includes an occupancy-related factor that is about 1 below an occupancy of roughly 0.55 and falls sharply above 0.7. The researchers say fixed constants forecast emergence on six held-out models with a 5% median error.

Forgetting followed different measures. It was not determined by occupancy, but by how widely a capability was distributed across the model and by the strength of each copy, measured through its logit margin. In the reported tests, a capability concentrated in one head disappeared after 43 steps of training on data that did not support it; spread across backups, it lasted 3,283 steps; after being re-concentrated late in training, it lasted 96 steps. Across independent trainings, matching the spread but having a deeper margin was associated with a tenfold increase in persistence. The researchers say their forgetting forecast was made before measurement and matched the results.

Why the measures matter

The work offers a way to study capability changes using quantities other than the usual loss curve. In particular, the proposed measures could indicate when a capability is becoming established and how vulnerable it is to being forgotten, while distinguishing a capability stored in one location from one distributed across backups. The abstract presents these patterns as measurable laws for the tested language models, rather than as a general guarantee for every AI system.

Evidence and limits

This is a preprint on arXiv, so its findings have not yet undergone peer review and may change. The study reports results from open model suites ranging from 14 million to 1 billion parameters, with emergence forecasts tested on six held-out models. The abstract does not establish how well the proposed laws apply beyond these models, training setups or capabilities.