This is Robin Sloan’s lab notebook. It’s about media and technology, creative computing, AI aesthetics, & more. Here's the RSS feed. My email address: robin@robinsloan.com
On Saturday, beneath a sparkling blue sky, I joined the march to Stop the AI Race. We gathered in front of OpenAI’s office in Mission Bay, then walked through the city to Anthropic’s HQ beside the Transbay Transit Center.
Stop the AI Race
If this had been a march organized around the diffuse concept of “AI BAD”, I wouldn’t have joined. But I am just so impressed by the elegance of Stop the AI Race’s demand:
Every major AI lab CEO must publicly commit to pausing frontier model development if every other major lab in the world credibly does the same.
Like, how rare is this?? A protest movement with (1) an actual objective, that (2) could conceivably be met. As much as anything else, I came out in support of simplicity and clarity.
A rabble-rousing robot
But I also believe that the world would benefit from a pause in frontier model development. The weird thing about this debate is that no one, not even the most hyped-up accelerationist, disagrees about the situation:
Here is a powerful technology,
operating in a way that no one really understands,
with profound effects on the economy, not to mention human psychology,
that are very difficult, maybe impossible, to make plans around.
For my part, I look at that fact pattern and think: uh, yes, this merits great caution and deliberation! Measured, I would say, in countries and years, not “model cards” and weeks. And my response isn’t reflexive, but deep-rooted — I’ve been grappling with this technology for ten years.
Geoff Hinton, here in spirit
This isn’t a call to outlaw language models. It has been widely observed — Jack Clark makes this point all the time — that even if model development stopped immediately, the metabolization of what’s currently available would happily occupy businesses and researchers alike for decades. Decades! These things are growing and mutating faster than anybody can make sense of them. So … here’s a wild thought … let’s slow down, and make sense of them.
I’ll direct your attention to the language of a recent post from the Anthropic Institute. It’s encumbered by a few extra clauses, but the spirit of Stop the AI Race’s demand shines clearly through:
We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. The Anthropic Institute will conduct research — in collaboration with many others — and take actions to help build the systems that a credible slowdown or pause would require. These systems would enable frontier AI developers to verify that others globally have actually stopped or slowed, and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret. If such systems existed, we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner.
Even if the danger isn’t as existential as the doomiest doomers imagine (I spotted these two in attendance) I believe this is a great opportunity for humanity to prove that we can actually make choices about the development and deployment of powerful technology. If we can’t, then we are not as sovereign as we imagine; if we can’t, a machine god has already taken over this planet, and it’s called the market.
A pause isn’t impossible, and powerful, unpredictable AI is (as a gorgeous blue banner at the head of the parade declared) not inevitable.
Tap or click to unmute.
This whole thing was better than I expected: a big crowd, numbering in the low hundreds; a great vibe, goofy and polite; perfect weather, never assured in San Francisco in July; and a marching band! We love a marching band. (Who paid for the marching band … ?)
And, of course, it’s worth appreciating, here and now in this country’s 250th summer, that we can still do things like this. Raise a mild ruckus, take up a bit of space, walk in the middle of the street. As we marched past Oracle Park, there was a Giants game underway, and it occurred to me that the great majority of the fans inside agree with the argument of this protest much more than they agree with the objectives of the AI companies. Democracy stirs — a leviathan to match the machine.
The design and vibe over at Worm Blossom feels very fresh and productive: this is neither the bland web of the 2020s, nor is it retro 2000s web kitsch. The layout is … in fact … sorta difficult to read, but I forgive it, because I love CSS columns and I think they should be used more often. (I use and abuse them to provide the pagination in my e-book template.)
The little inline piano rolls are my favorite part. The rendering is lovely, and the music provides a pleasant soundtrack for exploring.
Basically, this is one of those designs that might not totally “work”, but the attempt is so vital and so valiant that it punches through the dimension of merely “working” or “not working” into some other space. We’re never going to bust out of the prison of the mobile-optimized, single-column scroll if we don’t try stuff like this.
I don’t generally feel compelled to enthuse about AI models, even when I like them, because there is so much enthusiasm out there already, and it feels like remarking, in 1977, “Wow, that movie Star Wars was really thrilling and technically impressive, wasn’t it?”
Oh well: that model Fable is really thrilling and technically impressive, isn’t it? I get a sense of (indulge me here) incredible mass, but also nimbleness and, I suppose, grace. I’ve been watching reruns of Star Trek: The Next Generation lately, and the model makes me think of that version of the Enterprise.
This feeling comes from using Fable inside Claude Code; I don’t know that the web chatbot feels that different from previous versions. In the terminal, Fable is terse, even brusque … AND I LIKE IT.
I do wonder how the enormous ongoing investment in coding prowess is affecting the model’s skills and sensibilities on other tasks. I’m sure folks at Anthropic would say they understand these trade-offs pretty well — they run all sorts of evals beyond coding, etc. — but … I don’t know. It’s interesting.
Even pre-Fable, all the way back to the beginning of these models, it’s been fascinating to watch them “situate themselves” inside a project — which is to say, inside a document. (It’s still a document in the context window, even if it’s composed of many smaller parts in your filesystem, and even if it’s also a log of commands actually executed on your computer.) I mean, this is literally the core muscle of any/every language model: “I need to quickly and accurately understand what kind of document I am inside.” Yet the sensitivity of that orienteering, the subtlety of it, has gotten so much better. I organize my code in some pretty weird ways (on purpose!) and I use a style of front-end development that is way outside the norm … and Fable slips right in alongside me.
Now: even the funkiest JavaScript function carries within it many fewer choices than a paragraph of prose. (How’s that for a sentence?) Fable can’t match my writing style; honestly, I think that’s beyond the reach of these models, because it’s just too much to simulate, a whole human mind and body, their whole history together. But even this more limited sync is astonishing. (“I think this movie Star Wars might just be a hit!”) Fable opens its eyes, looks around a frankly bizarre field of tokens, and says, in a subsecond ripple of computation — I imagine it like the edge of a wave sheeting across a beach; the water is the code, the sand is the GPU — “Oh, I get it. I know exactly where I am. And I know what comes next.”
I absolutely LOVE the premise of this upcoming conference at Georgetown Law: Life After Data, the conference on “de-datafication”.
I predict you’re going to be hearing a lot more about this theme in the years ahead; the exhaustion is real. Here are a few tasty selections from the list of provocations on the conference page:
What would it take to build a movement to abandon the current internet and start anew?
What’s something good that currently requires the production and storage of digital data, that could be rebuilt without it? How?
What aspects of our current political situation are obscured or concealed by conflating all communication with information exchange, and how does datafication contribute to that obfuscation?
Outline one or more aspects of the risk environment that is created when a small number of large corporations control the infrastructures upon which people depend in their daily lives.
Whoever rattled these off is thinking in exactly the right direction — they are bold and wonky and radical and inspiring.
(Of course, I’ll note that these themes “rhyme” with the arguments in my recent zine productions.)
We do not yet understand how to train language models! This seems obvious to me, because it ought to be possible — it will be possible — to produce a tight, capable “programmatic reasoner” with something like 30 billion parameters.
The famous Scaling Laws only describe transformer models — nobody knows what weird architectures are waiting out there in the universe, with different responses to compute, data, and more. Nobody knows what kind of clever training regimes might coax huge models into better (more compact) shapes.
A fair objection goes like this: Robin, remember that the human brain has hundreds of trillions of “parameters”, in the form of synapses. Our largest models haven’t even approached that scale yet. Do you want us to architect a beetle’s brain, or SUPERINTELLIGENCE?
(Before proceeding, Robin replies: well, I wouldn’t mind starting with the beetle … )
The obvious response to this objection is that language models aren’t brains. Contra the brain, they operate with both handicaps (e.g. power consumption) and advantages (e.g. speed). More than linearly “better” or “worse”, though, they are just different! And so we should expect different properties, different capabilities … different numbers.
Hanging over everything, the recognition: the day that this level of intelligence moves out to the edge — to laptops and iPhones and toaster ovens — is the day the business model for centralized AI collapses like a soufflé. Lo, the data centers rise … yet they could be emptied in a year by one idea, from one lab or garage. Wild to think about.
A true believer in the Scaling Laws doesn’t think such an idea is possible — that’s my sense of it, anyway. Maybe I’m mischaracterizing the position. But I believe in the one idea, the one garage; I’m with Calvino:
Were I to choose an auspicious image for the new millennium, I would choose [ … ] the sudden agile leap of the poet-philosopher who raises himself above the weight of the world, showing that with all his gravity he has the secret of lightness, and that what many consider to be the vitality of the times — noisy, aggressive, revving and roaring — belongs to the realm of death, like a cemetery for rusty old cars.
Of course, this is just a post by a child of the 20th century, to whom the prefix “giga-“ still sounds unspeakably plush. Even so: if you tell me you can’t fit a supercapable model, one poised comfortably on today’s performance frontier, into 30 billion parameters, I will tell you, try harder!
Rather than stand apart as some kind of revolution or rupture, language models should mostly cause us to reflect on the power of all computers, the magic of them, which is this: Here is an engine that can take symbolic instructions and make complex things happen.
There have been lots of tools in human history, and only a very few of them, starting with the automatic loom, have this capability. (There are a few other candidates, further back … one is Leibniz’s Stepped Reckoner, what a name.)
It’s instructive to imagine a world with language models but without computers; maybe in that world they run on some weird bio-technology — maybe they really are plants, grown on elaborate trellises. In that world, they are still astonishing, but much less useful … because there’s not already this vast automatic environment in which language (the kind called code) becomes action.
This isn’t a paean to computers — I think a significant part of their automatic realm is basically useless and stupid — but I do want to insist on the continuity of the story, which runs straight through, from punch cards to mainframes to personal computers to whispering agents.
(I realize this is basically a restatement of my last post—as you can tell, I’m still thinking about it!)
One of the innovations of the [IBM] 604 was the pluggable module, which combined a tube and its associated circuitry [ … ] The insulated handle was used to remove and install modules in the calculator. The nine pins at the bottom of the module plugged into a socket in the 604, with the sockets connected with backplane wiring. The tube was also socketed, so a bad tube could be quickly replaced.
Reading about stuff like this, something to notice is that “the vacuum tube” wasn’t one thing, but a whole sweep of things, improvements and refinements, generational leaps, all playing out across decades. This wasn’t “the primordial ooze before computers”—IT WAS COMPUTERS, for a long and rich period of time.
You can say the same about punch-card computing, too.
This view has at least two nice features. One: it recognizes all this work and invention, the real beauty of it. (More physically beautiful, I’d say, than most modern computing.) Two: it reminds us that “we are using somebody else’s vacuum tubes”—which is to say, it’s plain to me that the story of AI is only beginning. There will be SO many improvements and refinements, generational leaps … all playing out across the decades ahead. Yes, decades! There is so much work to do. This (technology; industry; world??) isn’t going to be “over” in three years or five.
In fact, I think it’s all the same big story: punch cards and pluggable tubes, laptops and LLMs. Understanding that you are inside of it — acknowledging the dense, continuous connections in both directions, back in time and forward too — is both energizing and, in a way, soothing.
We believe it would be good for the world to have the option to slow or temporarily pause frontier AI development to enable societal structures and alignment research to keep up with the advance of the technology. The Anthropic Institute will conduct research — in collaboration with many others — and take actions to help build the systems that a credible slowdown or pause would require. These systems would enable frontier AI developers to verify that others globally have actually stopped or slowed, and that a bad actor could not use the auspices of a coordinated slowdown to jump ahead in secret. If such systems existed, we expect that we would slow down or temporarily pause, if other developers at or near the frontier also did so in a verifiable manner.
… and it’s extremely welcome news. It seems to me self-evident that a slowdown and/or pause would be a wise thing for humanity — indeed, it would be evidence that our civilization actually HAS a bit of wisdom! — yet I understand the complexity. A statement of this kind is a small but, IMO, meaningful step in the right direction.
Almost all of [Buttondown’s recent spike in growth] I attribute to LLMs. We ask people when they sign up what brought them here, and an answer that went from surprising to banal to overwhelming over the course of Q1 was: an LLM. Users of all stripes cite an LLM as the reason that they ended up at Buttondown’s front door.
I can add, anecdotally, that in Q1 of this year, Fat Gold saw its first subscription referrals from LLMs. We don’t (can’t?) track these programmatically, but we do ask new annual subscribers where they heard about us, and, for the first time, the reply has come: Claude sent me.
What a world!
P.S. I really do want you to read Justin’s post; I mean, just consider this:
[ … ] While the absolute volume of support tickets coming from LLM-born users isn’t significantly higher than the median, the shape of those tickets is off. To put it bluntly: a lot of the tickets we get are themselves LLM-generated. This is, frankly, extremely annoying — and demoralizing for me and the team to spend half an hour meticulously answering some complex question only to receive a machine-generated reply in return.
My post about AI-generated supercustomized email marketing produced many replies and much commiseration. And, in the days since posting, I have received SO MANY MORE of these cruddy messages!!
It makes me wonder if it would be possible for a company like Anthropic, with their hard-won expertise in alignment, to train their models such that they could not — and I mean really deeply, constitutionally, viscerally COULD NOT — lie about their identity, or pretend to be anything other than an AI model?
Obviously this raises questions both practical and philosophical, because of course “help me write a message” is VERY close to “write a message, pretending to be me” … but that’s the case for all this alignment stuff. Every question about, say, virology dances along that border. This tension is widely acknowledged in realms like biology and cybersecurity, but it applies to writing, too — the original dual-use technology!!
AI doomers spin rich scenarios about silver-tongued AIs manipulating their users and operators; there’s another scenario in which AI systems pollute human communication channels to the degree that they’re no longer reliable or even usable.
That’s all to say, I feel like this is a bigger issue than a lot of people realize — the first glimmer of a profound digital-ecological crisis.
