Bad hosts, or:
how I learned to stop worrying and love the overlay network
Concluding my notes on Web3, I wrote that
Ethereum should inspire anyone interested in the future(s) of the internet, because it proves, powerfully, that new protocols are still possible.
and, accepting my own admonition, I have been tinkering on a new protocol —
Today, I want to document a realization that struck along the way.
The modern internet has rendered the figure of “an individual hosting a small internet service, all on their own” as antique as a blacksmith; this, I sorta knew. But where before I might have called it a matter of ergonomics or motivation, I now understand that it’s deeper in the network.
Really, I'm talking about just two things: IPv4 addresses and NAT.
IPv4 addresses are the ones you know, with four blocks of digits; Google’s is
18.104.22.168. There are 4.3 billion possible addresses of this kind … which isn’t nearly enough for all the computers that want to connect to the internet in 2022.
The remedy, or hack, has been Network Address Translation, familiar to any home internet user: your modem and router are assigned a public IPv4 address, reachable from anywhere on the internet, while your laptop and your phone (and your Apple TV, and your washing machine, maybe) get a local address, a classic like
In this way, your router can use its single IPv4 address to “cover” all the computers in your home. When you open a website, issuing an HTTP request to that web server, the router uses NAT to relay its response to your laptop, rather than the Apple TV.
Ah, but what if I sent an HTTP request to your public IPv4 address? It would arrive at your router, which wouldn’t have any idea where to send it —
NAT as one-way mirror.
Critics of centralization will eagerly describe the centripetal forces sucking everyone and everything into the big platforms. Those forces are real! However, there ARE centrifugal forces —
It’s not impossible to host a small internet service from a computer in your home, but it takes some fairly intense tinkering and maintenance, so it remains the pastime of … the fairly intense. Brittle router admin is no match for the sleek, it-just-works seduction of big platforms and/or cloud data centers.
Honestly, though, I am starting to think 50% of the ease and power of centralization is just a stable, public IPv4 address.
This is all coming out of my own experience, my own thought experiments, as I sketch out protocols and apps, “ways of relating” across a network. Every time I muse about something decentralized, I bump up against this barrier: a person connected to the internet from home cannot host a small service of their own.
There are workarounds for NAT, ubiquitous hacks, but they all require centralized intermediaries. Think of video chat. While we’re chatting, the video is flowing straight from my computer to yours —
The connection is ephemeral. Our video chat ends, and my Wi-Fi router’s heart flutters, and you are lost to me again.
The workarounds are fine as far as they go, but NAT tricks can’t get us the one thing we really want, the foundational internet thing: the ability to simply listen for connections. Therefore, whole classes of possible services and relationships don’t exist; a whole alternate internet history.
As home internet users, we can only speak and request, not listen and serve.
Computers with the ability to listen on the internet are called “hosts”, and there’s an interesting etymological implication there. Today, as home internet users, we are not hosts; and perhaps we are missing out, therefore, on a degree of etiquette, and conviviality, and satisfaction.
I find it melancholy: all these powerful computers, my laptop and yours, not to mention the servers in my office, your supercomputer smartphone —
What about the dream of IPv6?
This is the newer scheme in which computers have longer addresses written with hex digits; Google’s IPv6 address is 2607:f8b0:4005:812::200e.
One of the great upsides of IPv6 is that there are more than enough addresses for every computer connected to the internet, so every computer could be reachable from anywhere —
But, the rollout of IPv6 across the internet has been stubbornly slow. Computers support it —
I truly did not have an opinion on IPv6 before December 2021. Suddenly, I am a wild-eyed evangelist, because this scheme, which works and is widely supported but not widely deployed, supports an internet in which every computer can be a host, if it wants to be.
For my part, I think an IPv6 internet would be almost unimaginably fertile and productive. New kinds of apps and services would bloom like flowers in a dry meadow after the rain.
The rise of “overlay networks” is a natural response to the limitations and frustrations of the public IPv4 internet, and I think their acceleration in the last few years deserves more attention.
These are illusory networks established on top of the public internet. Often, you install a small program on your computer, and it allows the rest of your applications to “see” a group of other computers as if they were close at hand on your local network, even if they’re actually far away, deep in NAT hell, unreachable by normal means.
Here, I’ll make it more concrete:
For years, I was vexed by the task of reliably reaching the two servers in my office from anywhere outside, e.g. from home. The office doesn’t have any fancy network hardware, just a Wi-Fi router, and the best I could manage was a brittle port-forwarding scheme that never worked for more than a couple of months at a time.
Then I discovered ZeroTier, which allows you to create and manage these overlay networks. I installed it on my laptop, my old iMac, and my two servers, as well as an EC2 instance, and ever since, I have hopped between them with ease, no matter where I am. Each computer has a stable IP address in the
10.0.0.0/8 range, reserved for private networks. It’s fabulous. They are all hosts again.
There’s a similar service called Tailscale, along with Slack’s somewhat more robotic Nebula, and plenty more to come, I’m sure.
I am the only inhabitant of my ZeroTier network, and I get the sense a lot of people use the service this way, but both ZeroTier and Tailscale allow you to create overlay networks with many users —
ZeroTier’s founder Adam Ierymenko sometimes calls the service a “planetary data center”, insisting that it ought to feel like every computer IN THE WORLD is in the same room, regardless of its actual location or network disposition. What a vision.
Along the same lines, one of Tailscale’s founders, David Crawshaw, has a blog post, actually quite moving, titled Remembering the LAN. He writes:
The LAN was a magical place to learn about computers. Besides the physical aspect of assembling and disassembling machines, I could safely do things unthinkable on the modern internet: permission-less file sharing, experimental servers with no security, shared software where any one machine could easily bring down the network by typing in an innocuous command. Even when I did bring down the network the impact never left the building. I knew who I had to apologize to.
These new (old) “networks within networks” are, in 2022, both useful and evocative, and I think they open some VERY interesting space for work on peer-to-peer protocols and “ways of relating”—I keep writing that, I know it’s weird —
I mean, you know I love computing at this scale.
But, as cool and promising as the overlay networks are, I am not willing to sacrifice “public” entirely, because what is the internet, if not an open invitation? And a suggestion, recurring, that you might not already know all the people you want to know. (If you’ve found your way to this newsletter, this web page, then you understand what I mean.)
Again, I feel the resonance of the word “host”, and again, I think about etiquette, and conviviality, and satisfaction.
There’s also the peer-to-peer browser called Beaker, a powerfully centrifugal project; the idea is, or was, that your browser might host a website as easily as it navigates to one. Very 1993! (Thanks to David for prompting this addition.)
Its successor/inheritor/whatever, a protocol called Hypercore, nudges in the same direction. That melancholy feeling again: when you look at the code, you find it writhing with NAT contortions. So much time and energy and creativity, just to “get back to zero”, the ability to listen on the internet.
I mocked something up using Hypercore’s great implementation of a peer-to-peer DHT swarm, the same technique (PDF) used to coordinate, among other things, the nodes of the Ethereum network. (There are relatively few Ethereum nodes —
which, when it works on the public internet, has decentralization built in.
My interest in all of this is a bit odd-angled, because I don’t actually care about decentralization that much; or, I think it’s interesting, but I think a lot of things are interesting, and I’m willing to weigh them against each other, make compromises. TONS of compromises.
What I’m really interested in —
Pretty good deal if you can get it.
Defining a protocol is challenging enough; programming a client application more challenging still; but backing up the database? I am just not interested in that kind of work, that level of stress.
This, too, is part of Web3’s appeal: you can invent new things without running the infrastructure that supports them. Of course, it trades that stress for a new kind, the madness of immutable code, and charges you for every pulse of the virtual machine … so, for me, it’s a pass.
I have no great conclusion to offer, and I’m sure most of this is old news to those of you who have tangled with peer-to-peer protocols before. I guess the surprise, for me —
I should add, as a coda, that ZeroTier has an interesting offering, a set of public networks that anyone can join. Each only allows access to a particular range of ports —
The thing that stops me from doing this, for now, is the lingering sense that, come on, there must be a way to offer something interesting that is not woven into the infrastructure and protocol of one particular company.
But who knows! I might get over it!
February 2022, Oakland