Robin Sloan
the lab
February 2022

Bad hosts, or:
how I learned to stop worrying and love the overlay network

A warm scene of a party in a cozy lounge, everyone in suits and gowns.
Viggo Johansen, An Evening Party in the Artist's Home, 1889

Concluding my notes on Web3, I wrote that

Ethereum should inspire any­one inter­ested in the future(s) of the inter­net, because it proves, pow­er­fully, that new pro­to­cols are still pos­si­ble.

and, accept­ing my own admonition, I have been tin­ker­ing on a new pro­to­col — simple and, shall we say, “hypermedia-ish”—that I hope will be inter­est­ing to others. More on that later this year.

Today, I want to doc­u­ment a real­iza­tion that struck along the way.

The mod­ern internet has ren­dered the fig­ure of “an indi­vid­ual host­ing a small inter­net ser­vice, all on their own” as antique as a blacksmith; this, I sorta knew. But where before I might have called it a mat­ter of ergonom­ics or motivation, I now under­stand that it’s deeper in the net­work.

Really, I'm talking about just two things: IPv4 addresses and NAT.

IPv4 addresses are the ones you know, with four blocks of digits; Google’s is 142.250.189.206. There are 4.3 bil­lion pos­si­ble addresses of this kind … which isn’t nearly enough for all the com­put­ers that want to con­nect to the inter­net in 2022.

The remedy, or hack, has been Net­work Address Translation, famil­iar to any home internet user: your modem and router are assigned a pub­lic IPv4 address, reach­able from any­where on the inter­net, while your lap­top and your phone (and your Apple TV, and your wash­ing machine, maybe) get a local address, a clas­sic like 192.168.0.3 or 10.0.0.7.

In this way, your router can use its sin­gle IPv4 address to “cover” all the com­put­ers in your home. When you open a web­site, issu­ing an HTTP request to that web server, the router uses NAT to relay its response to your lap­top, rather than the Apple TV.

Ah, but what if I sent an HTTP request to your pub­lic IPv4 address? It would arrive at your router, which wouldn’t have any idea where to send it — your lap­top? the Apple TV? the wash­ing machine? Con­fused and/or suspicious, the router would sim­ply drop or refuse my request.

NAT as one-way mirror.

Critics of cen­tral­iza­tion will eagerly describe the cen­tripetal forces suck­ing every­one and every­thing into the big plat­forms. Those forces are real! However, there ARE cen­trifu­gal forces — political, moral, creative, artistic — pushing back out toward the edges … it’s just that those coun­ter­vail­ing forces are can­celled almost totally by NAT.

It’s not impos­si­ble to host a small inter­net ser­vice from a com­puter in your home, but it takes some fairly intense tin­ker­ing and maintenance, so it remains the pas­time of … the fairly intense. Brit­tle router admin is no match for the sleek, there-it-is seduc­tion of big plat­forms and/or cloud data centers.

Honestly, though, I am start­ing to think 50% of the ease and power of cen­tral­iza­tion is just a sta­ble, pub­lic IPv4 address.

This is all com­ing out of my own experience, my own thought experiments, as I sketch out pro­to­cols and apps, “ways of relating” across a net­work; and every time I muse about some­thing decen­tral­ized, I bump up against this barrier: a per­son con­nected to the inter­net from home can­not host a small ser­vice of their own.

There are workarounds for NAT, ubiq­ui­tous hacks, but they all require cen­tral­ized intermediaries. Think of video chat. While we’re chatting, the video is flow­ing straight from my com­puter to yours — in a sense, we are each host­ing a small inter­net ser­vice for each other! But we can’t ini­ti­ate that con­nection ourselves. It requires a third host, one with a pub­lic IPv4 address. That host “punches a hole” through our one-way NAT mir­rors and ties us together.

The con­nection is ephemeral. Our video chat ends, and my Wi-Fi router’s heart flutters, and you are lost to me again.

The workarounds are fine as far as they go, but NAT tricks can’t get us the one thing we really want, the foun­da­tional internet thing: the abil­ity to sim­ply lis­ten for con­nections. Therefore, whole classes of pos­si­ble ser­vices and rela­tion­ships don’t exist; a whole alter­nate inter­net history.

As home inter­net users, we can only speak and request, not lis­ten and serve.

Computers with the abil­ity to lis­ten on the internet are called “hosts”, and there’s an inter­est­ing ety­mo­log­i­cal impli­ca­tion there. Today, as home inter­net users, we are not hosts; and per­haps we are miss­ing out, therefore, on a degree of etiquette, and conviviality, and satisfaction.

I find it melan­choly: all these pow­er­ful com­put­ers, my lap­top and yours, not to men­tion the servers in my office, your supercom­puter smartphone — they could be doing inter­esting things together, shut­tling data around in inter­est­ing ways. Back when most (of the rel­a­tively few) internet users could host freely, none of them had a giga­bit con­nection at home; now, many inter­net users have band­width and proces­sor cycles to spare, but we can’t host. Technological irony.

What of the dream of IPv6?

This is the newer scheme in which com­put­ers have longer addresses writ­ten with hex digits; Google’s IPv6 address is 2607:f8b0:4005:812::200e.

One of the great upsides of IPv6 is that there are more than enough addresses for every computer con­nected to the inter­net, so every com­puter could be reach­able from any­where — no NAT required.

But, the roll­out of IPv6 across the internet has been stub­bornly slow. Com­put­ers support it — odds are very good your phone and lap­top speak IPv6 with per­fect fluency. It’s the home internet providers that have been con­spic­u­ous laggards; I get inter­net from both AT&T and Sonic, and nei­ther of them sup­port IPv6.

I truly did not have an opin­ion on IPv6 before Decem­ber 2021. Suddenly, I am a wild-eyed evangelist, because this scheme, which works and is widely sup­ported but not widely deployed, sup­ports an inter­net in which every com­puter can be a host, if it wants to be.

For my part, I think an IPv6 inter­net would be almost unimag­in­ably fer­tile and productive. New kinds of apps and ser­vices would bloom like flow­ers in a dry meadow after the rain.

Ohhh welllll


The rise of “over­lay net­works” is a nat­ural response to the lim­i­ta­tions and frus­tra­tions of the pub­lic IPv4 inter­net, and I think their accel­er­a­tion in the last few years deserves more attention.

These are illu­sory net­works estab­lished on top of the pub­lic inter­net. Often, you install a small pro­gram on your com­puter, and it allows the rest of your appli­ca­tions to “see” a group of other com­put­ers as if they were close at hand on your local net­work, even if they’re actu­ally far away, deep in NAT hell, unreach­able by normal means.

Here, I’ll make it more concrete:

For years, I was vexed by the task of reli­ably reach­ing the two servers in my office from any­where outside, e.g. from home. The office doesn’t have any fancy net­work hardware, just a Wi-Fi router, and the best I could man­age was a brit­tle port-forwarding scheme that never worked for more than a cou­ple of months at a time.

Then I discovered ZeroTier, which allows you to cre­ate and man­age these over­lay net­works. I installed it on my lap­top, my old iMac, and my two servers, as well as an EC2 instance, and ever since, I have hopped between them with ease, no mat­ter where I am. Each com­puter has a sta­ble IP address in the 10.0.0.0/8 range, reserved for pri­vate net­works. It’s fabulous. They are all hosts again.

There’s a sim­i­lar ser­vice called Tailscale, along with Slack’s some­what more robotic Nebula, and I’m sure a ton more.

I am the only inhab­i­tant of my ZeroTier network, and I get the sense a lot of peo­ple use it this way, but both ZeroTier and Tailscale allow you to cre­ate over­lay net­works with many users — hundreds or more. In those cases, the net­works become lit­tle mini-inter­nets for your coworkers, your group of friends, your pirate armada, whatever.

ZeroTier’s founder Adam Iery­menko some­times calls the ser­vice a “planetary data center”, insist­ing that it ought to feel like every com­puter IN THE WORLD is in the same room, regard­less of its actual loca­tion or net­work disposition. What a vision.

Along the same lines, one of Tailscale’s founders, David Crawshaw, has a blog post, actu­ally quite moving, titled Remembering the LAN. He writes:

The LAN was a mag­i­cal place to learn about com­put­ers. Besides the phys­i­cal aspect of assembling and disassem­bling machines, I could safely do things unthink­able on the mod­ern inter­net: permission-less file sharing, exper­i­men­tal servers with no security, shared soft­ware where any one machine could eas­ily bring down the network by typ­ing in an innocu­ous command. Even when I did bring down the net­work the impact never left the building. I knew who I had to apologize to.

These new (old) “net­works within net­works” are, in 2022, both use­ful and evocative, and I think they open some VERY inter­est­ing space for work on peer-to-peer pro­to­cols and “ways of relating”—I keep writ­ing that, I know it’s vague — appropriate for mini-inter­nets of only 50 or 100 peo­ple.

I mean, you know I love com­put­ing at this scale.

But, as cool and promis­ing as the over­lay net­works are, I am not will­ing to sac­ri­fice “pub­lic” entirely, because what is the inter­net, if not an open invitation? And a suggestion, recurring, that you might not already know all the peo­ple you want to know. (If you’ve found your way to this newsletter, this web page, then you under­stand what I mean.)

Again, I feel the res­o­nance of the word “host”, and again, I think about etiquette, and conviviality, and satisfaction.


There’s also the peer-to-peer browser called Beaker, a pow­er­fully cen­trifu­gal project; the idea is, or was, that your browser might host a web­site as eas­ily as it nav­i­gates to one. Very 1993! (Thanks to David for prompt­ing this addition.)

Its successor/inheritor/whatever, a protocol called Hypercore, nudges in the same direction. That melan­choly feel­ing again: when you look at the code, you find it writhing with NAT contortions. So much time and energy and creativity, just to “get back to zero”, the abil­ity to lis­ten on the inter­net.

I mocked some­thing up using Hypercore’s great imple­men­ta­tion of a peer-to-peer DHT swarm, the same technique (PDF) used to coordinate, among other things, the nodes of the Ethereum net­work. (There are rel­a­tively few Ethereum nodes — on the order of thousands.) It’s a clever model, but/and, I found it really “heavy”, and, call me greedy or unreasonable, but I just want the simplicity of

socket.lis­ten

which, when it works on the pub­lic inter­net, has decen­tral­iza­tion built in.


My interest in all of this is a bit odd-angled, because I don’t actu­ally care about decen­tral­iza­tion that much; or, I think it’s inter­esting, but I think a lot of things are inter­est­ing, and I’m will­ing to weigh them against each other, make compromises. TONS of compromises.

What I’m really inter­ested in — what I dream about — is the oppor­tu­nity to play with new pro­to­cols with­out tak­ing on, perforce, the burden of infrastruc­ture. If we cast our gaze back to the early days of the World Wide Web, we find all these peo­ple com­ing up with cool new ideas for HTTP and HTML, writ­ing new server soft­ware and new browsers with new features … and very con­spic­u­ously not “operating the web”. They didn’t have that power or that bur­den. The web was … out there.

Pretty good deal if you can get it.

Defining a pro­to­col is challenging enough; pro­gramming a client appli­ca­tion more chal­leng­ing still; but backing up the database? I am just ter­mi­nally uninter­ested in that kind of work, that level of stress.

This, too, is part of Web3’s appeal: you can invent new things with­out run­ning the infrastruc­ture that sup­ports them. Of course, it trades that stress for a new kind, the mad­ness of immutable code, and charges you for every pulse of the vir­tual machine … so, for me, it’s a pass.


I have no great con­clu­sion to offer, and I’m sure most of this is old news to those of you who have tan­gled with peer-to-peer pro­to­cols before. I guess the surprise, for me — the thing I felt an urge to share — was the real­iza­tion that at least some of this ten­dency towards cen­tral­iza­tion is inher­ent in the archi­tec­ture of the inter­net itself. The trap­door of IPv4 exhaus­tion deliv­ered us to this place, stuck behind NAT, pound­ing on the walls. It’s a huge bummer!

I should add, as a coda, that ZeroTier has an inter­est­ing offering, a set of public networks that any­one can join. Each only allows access to a par­tic­u­lar range of ports — you can choose them — via IPv6, which works for every­one, because it’s vir­tualized through ZeroTier’s soft­ware; it’s “fake IPv6”, I guess. These net­works are a lovely affordance, and there’s a world in which the instruc­tion man­ual for my new pro­to­col begins with: “Step 1. Down­load ZeroTier and join pub­lic net­work X.” Voila, the NAT prob­lem goes away; it’s 1993 again; we can all see each other.

The thing that stops me from doing this, for now, is the lin­ger­ing sense that, come on, there must be a way to offer some­thing inter­est­ing that is not woven into the infrastruc­ture and pro­to­col of one par­tic­u­lar company.

But who knows! I might get over it!

February 2022, Oakland