The cybernetic CEO

If, right now, at this very moment, the newly Claude-empowered tech CEO can sud­denly jump into the code­base and imple­ment a new fea­ture (to the product team’s horror?), then what’s coming is surely stranger:

With time, the struc­ture of firms them­selves will evolve to max­i­mize the utility of [AI] agents. But even before that evolution, agents will enable the people who lead firms to exer­cise far greater cyber­netic con­trol over the teams they lead. Today, when the CEO of a com­pany wants to make some change to a busi­ness process, they relay that com­mand through chains of leadership, and each time it loses some fidelity. Maybe it will be misinterpreted. Maybe someone in some layer of the com­pany does not want to do it, and so ignores it, imple­ments the order half-heartedly, or engages in mali­cious compliance. For a wide variety of busi­ness processes, this problem will dis­ap­pear entirely, and for many others, it will be sig­nif­i­cantly lessened. CEOs and man­agers will be able to say “jump,” and in unison, tens, hundreds, thousands, or millions, of agents will say “how high?”

I like Dean Ball’s writing on this because he empha­sizes the deep strangeness—the ways in which this sce­nario doesn’t simply “solve” some long­standing problem, making the world neater, but rather opens up a whole new field of possibilities, making the world weirder:

This will make firms (and other organizations) strange. I am not sure that it will straight­for­wardly make them better, but it will almost cer­tainly make more effi­cient and profitable. They will prob­ably be heavier at the top than they are today, and so con­ceiv­ably far more variable. We describe firms as per­sons for legal purposes, but they really might start to feel alive, almost bio­log­ical in their ability to adapt quickly to changing circumstances. Over the next decade, a new kind of life form will emerge on the world stage: the AI-enabled firm.

This, like the near-psychedelic microfic­tions Jack Clark pub­lishes in his newsletter, is the kind of writing, and thinking, that AI deserves. If Dean is a believer — one who will state plainly, “Most of the thinking and doing in America will soon be done by machines, not people”—then at least he is a very inter­esting believer, writing with a den­sity and quality that’s provoca­tive and truly useful.

For my part, (1) I’m not sure any of this is really going to func­tion out­side of the magic circle of SAAS com­pa­nies building soft­ware for SAAS com­pa­nies, and (2) the prospect of working in this kind of firm, one of the out­num­bered humans toiling along­side the CEO’s servile AI swarm, sounds pretty awful.