How the universe stores information

Two dashed hopes that seem to rhyme:

• The hope that, in human DNA, one gene would cor­re­spond to one devel­op­mental feature: “size of nose”, “propensity for prostate cancer”: a vast bank of bio­log­ical switches.

• The hope that, in neural networks, one weight would cor­re­spond to one infor­ma­tional feature: “Python function”, “malevolent plan”: a per­fect intel­lec­tual X-ray.

The map­ping of the human genome had less of an impact on the world than a person might have pre­dicted back around 2000, because tracing the con­nec­tion between base pairs and big noses turned out to be — turns out to be — much more dif­fi­cult than anyone expected. “Yes, that gene codes for a cute lil schnoz, but only in the pres­ence of this other gene, plus these two, and only when acti­vated by a pro­tein pro­duced by THAT one … ”

Likewise, modern AI seems to grant its prac­tioners such power: they DO have that X-ray, a full readout of every value in every crevice of their creations … but the image is, if not unreadable, then very very confusing.

What rhymes is the way both infor­ma­tional sys­tems pile meaning onto their symbols: in DNA, one gene can do many dif­ferent things, in many dif­ferent contexts; in an AI model, one weight can do many dif­ferent things, in many dif­ferent contexts. This is superposition, a huge sub­ject in AI inter­pretability research.

Both sys­tems, even though they are plenty com­plex — think of the thou­sands of dimen­sions of the imag­i­nary space in which an AI model operates — seem to be coarse “shadows” of even MORE com­plex constructions. Imagine the shadow of a cube, the shape it makes on a tabletop — the way a 3D object can become two-dimensional. I’ve also heard AI researchers talk about models as “simulators” of more com­plex models — and I wonder if DNA couldn’t also be said to “simulate” some other, richer record, way off in Pla­tonic space.

Inter­esting to see a glimpse of this architecture — this strategy — in two dif­ferent sys­tems, both oper­ating under harsh con­di­tions of selection: one natural, the other artificial. One won­ders if this is simply how the uni­verse prefers to store infor­ma­tion; and if, in any con­strained system, given the choice between “playing it straight” and sim­u­lating a better system, the sim­u­la­tion ekes out a victory.

P.S. Philip Ball’s How Life Works is your ultimate, up-to-the-minute guide to the mad­dening indi­rec­tion of DNA.