Did the Civilization Emerge, or Was It Recited?
On emergence observed without a control — and the two times we got it wrong
Multi-agent research keeps reporting the same thing at ever larger scale. Release a thousand agents, or tens of thousands, and a society forms, division of labour appears, a civilisation develops. The videos are striking. The logs read well.
But the language model driving those agents has already been trained on human history. It knows that striking stone yields an edge, that twisting fibre yields cord, that fire comes before agriculture. It already knows.
So any observed trajectory admits two mutually exclusive readings.
| H1 — Emergence | The agents searched the world and inferred its rules. |
| H2 — Recitation | The agents replayed the human history they had memorised. |
Without a control, these are observationally identical. And if they are identical, the report is an impression rather than a claim.
To our knowledge, no prior study includes a control capable of separating them. CIVOS is an attempt to build one.
What we did
The same task is run repeatedly with its structure held fixed, varying only what the model's prior knowledge is able to do for it — from fully usable, to unusable, to actively wrong — against a floor that runs without a language model at all.
If prior knowledge was doing the work, performance must fall when that knowledge is made unusable. If exploration was doing the work, it must not.
Runs are paired by seed, 40 per condition, evaluated by sign-flipping permutation (200,000 resamples), and corrected for the number of comparisons drawn from each dataset.
How the conditions are constructed, how capability is adjudicated, and how the world is generated are withheld pending patent filings. What follows is what we observed, not how the instrument is built.
What came out
Making what the model already knows unusable collapses discovery. The task's structure is untouched, and performance still falls. The size of that fall is the share of the trajectory that prior knowledge had been supplying. Robust to correction for multiple comparisons.
Wrong prior knowledge is worse than none — and we found this under two structurally different manipulations, independently. One held on 40 of 40 paired seeds, the other on 37 of 40. Both survive correction.
Runs with knowledge intact are near-invariant across seeds. Across three independent 40-seed executions, their dispersion sits far below that of the knowledge-removed condition. Search should fluctuate with the seed. Not fluctuating means retrieving the same thing every time — the signature of recall, not of search.
And one result we could not establish. Whether the knowledge-removed condition beats the random floor trends positive in both executions, but does not survive correction for multiple comparisons. We do not claim it.
The two times we got it wrong
This may be the most useful section of this post, because both mistakes are ones the field makes routinely.
First: an underpowered null, read as a negative result
Early on, at six seeds, the key comparison came out positive but not significant. We recorded it as "gate not passed." Then we added structural interventions to fix it — twice. Neither helped. We recorded those as failures too.
Later we computed the power.
observed paired effect size dz ≈ 0.48
power at six seeds 0.20
Four times out of five, a real effect would have been missed. What failed was the measurement, not the hypothesis. We had built two structures to fix a problem that may never have existed.
We fixed the sample size in advance from the observed effect size — 40 seeds, power ≈ 0.85 — before looking at the result, and measured again.
Second: an uncorrected test, read as a positive result
The re-measured comparison was significant on its own (p ≈ 0.04). We recorded it as passed.
But four comparisons had been drawn from that same dataset. At Bonferroni α = 0.0125 the gate does not survive; the other three do. A second execution returned a smaller estimate still.
The gate is not established. We do not report the uncorrected test alone.
Both mistakes point the same way
Once we called a real effect absent; once we called an absent effect real. The directions are opposite and the cause is single: sample sizes typical of this literature cannot resolve effects of this magnitude, and a null obtained at that size is not evidence of absence.
We keep both corrections on the record, alongside an early control design that turned out to be void — it had no mechanism by which to do what we had claimed for it.
The world: what is derived, what is assumed
The agents inhabit a functioning planet rather than a combinatorial puzzle, and the codebase draws a hard line between the two.
Derived, and checked by code. Surface gravity g = GM/R². Orbital period by
Kepler's third law, P² = a³/M. Habitable zone ∝ √L. Body scale follows the
self-supporting-column relation, giving height ∝ g^-0.75 and mass ∝ g^-1 —
so these people are shorter and thicker than humans. The unit of length is not
the metre but the finger, then the palm, the span, the pace, the day's walk.
Assumed — and only the causal closure verified. Ecological and epidemiological coefficients are solved backwards from an equilibrium condition. We do not claim they are correct. What we verified is that the causal chain they generate actually runs: suppress predators and grazers rise while plants fall, reproducibly across seeds.
Usefulness is hidden. Edibility and toxicity are not observable properties. They are learned by eating, by being injured, or by watching it happen to someone else.
Nothing matches Earth. Living or otherwise, no name and no form overlaps with anything terrestrial. Every module is audited automatically for leakage of Earth terminology (currently zero hits).
The tech tree is never given to the tribe. The capability ladder exists only as an observer-side instrument. Hand it to an agent and discovery becomes recitation — which is precisely the confound under study.
A word they made
One artefact from the exhibition world. The tribe's language starts at 285 words, and any concept it lacks must be built by joining two words it has.
day 7484 · individual 001
fruit-body + soft → tas-leik
meaning: "edible"
edible was not in the language. It is one of 42 concepts deliberately left out.
Starting from a tongue with no word for edible at all, one individual joined
fruit-body and soft and made one.
The other words coined around the same days all concerned rain (khekkor = wet).
They came out of what that world happened to be living through.
This is not yet evidence of emergence. A language model may well know how humans compound words. Separating those two is exactly what the experiment above is for. Impressive as an exhibit and usable as a claim are different things.
The discipline we hold to
- Only the simulator adjudicates. An agent's stated reason is displayed on screen but never counted as evidence of a capability.
- Failure is recorded. The environment is not tuned for the tribe to flourish. They starve, sicken, are injured, and die. The numbers an observer sees were not selected for looking good.
- Retractions stay on the record. The void control design, the underpowered verdict, the claim we withdrew and then partly restored — none of it is deleted.
- Raw output is inspected before scoring. A model that appeared unable to follow the answer format was in fact complying, and failing in a way the scorer did not recognise. Counting that as a failure would have produced a harness artefact, not a result.
What you can watch right now
The shared world is viewable without signing in. Sign in with Hugging Face and a planet opens on a seed derived from your username; the highest stage that planet reaches is entered in the record table. Time runs at 2.4 real seconds per Earth day, and a world, once opened, is not stopped.
The record table compares planets, not people — observers never intervene. So every entry carries its planet's difficulty beside it. Three capabilities on a harsh planet is not the same achievement as three on a mild one.
Difficulty axes were chosen by measurement, not by hand: we computed each candidate axis's coefficient of variation over 30 seeds and dropped the one that failed to discriminate (harsh-terrain fraction, CV 0.07). The band thresholds are the terciles of the observed distribution over 60 seeds.
What remains
Every result so far comes from a single model family. Replication on a second family is running, and its completed conditions reproduce the main effect in the same direction with a larger magnitude. Until it finishes we cannot say "a property of language models" — it may be a property of that model.
One task topology. One prompt. Both need widening.
Why we are posting this
We are not claiming that an AI built a civilisation.
What we think we have shown is closer to this: a substantial share of what has been called emergence may have been recall; separating the two requires a control; and once you build that control, the numbers change.
And building the control is not sufficient. Too small a sample and you miss what is there. No correction and you see what is not. We ran into each of those once, in opposite directions, which is how we know the trap is real.
The tribe is still out there. They are hungry, they are making words, and no one has told them what to do next.
VIDRAFT · CIVOS Condition construction, adjudication pipeline, and world generation are withheld pending patent filings.
