Bad hosts, or: how I learned to stop worrying and love the overlay network

Con­cluding my notes on Web3, I wrote that

Ethereum should inspire anyone inter­ested in the future(s) of the internet, because it proves, pow­erfully, that new pro­to­cols are still pos­sible.

and, accepting my own admonition, I have been tin­kering on a new pro­tocol — simple and, shall we say, “hypermedia-ish”—that I hope will be inter­esting to others. More on that later this year.

Today, I want to doc­u­ment a real­iza­tion that struck along the way.

The modern internet has ren­dered the figure of “an indi­vidual hosting a small internet ser­vice, all on their own” as antique as a blacksmith; this, I sorta knew. But where before I might have called it a matter of ergonomics or motivation, I now under­stand that it’s deeper in the net­work.

Critics of cen­tral­iza­tion will eagerly describe the cen­tripetal forces sucking every­body and every­thing into the big plat­forms. Those forces are real! However, there ARE cen­trifugal forces — political, moral, creative, artistic — pushing back out toward the edges … it’s just that they are can­celled almost totally by NAT.

It’s not impos­sible to host a small internet ser­vice from a com­puter in your home, but it takes some fairly intense tin­kering and maintenance, so it remains the pas­time of … the fairly intense. Brittle router admin is no match for the sleek, it-just-works seduc­tion of big plat­forms and/or cloud data centers.

Honestly, though, I am starting to think 50% of the ease and power of cen­tral­iza­tion is just a stable, public IPv4 address.

This is all coming out of my own experience, my own thought experiments, as I sketch out pro­to­cols and apps, “ways of relating” across a net­work. Every time I muse about some­thing decen­tral­ized, I bump up against this barrier: a person con­nected to the internet from home cannot host a small ser­vice of their own.

There are workarounds for NAT, ubiq­ui­tous hacks, but they all require cen­tral­ized intermediaries. Think of video chat. While we’re chatting, the video is flowing straight from my com­puter to yours — in a sense, we are each hosting a small internet ser­vice for each other! But we can’t ini­tiate that con­nection ourselves. It requires a third host, one with a public IPv4 address. That host “punches a hole” through our one-way NAT mir­rors and ties us together.

The con­nection is ephemeral. Our video chat ends, and my Wi-Fi router’s heart flutters, and you are lost to me again.

The workarounds are fine as far as they go, but NAT tricks can’t get us the one thing we really want, the foun­da­tional internet thing: the ability to simply listen for con­nections. Therefore, whole classes of pos­sible ser­vices and rela­tion­ships don’t exist; a whole alter­nate internet history.

As home internet users, we can only speak and request, not listen and serve.

Com­puters with the ability to listen on the internet are called “hosts”, and there’s an inter­esting ety­mo­log­ical impli­ca­tion there. Today, as home internet users, we are not hosts; and per­haps we are missing out, therefore, on a degree of etiquette, and conviviality, and satisfaction.

I find it melan­choly: all these pow­erful com­puters, my laptop and yours, not to men­tion the servers in my office, your supercom­puter smartphone — they could be doing inter­esting things together, shut­tling data around in inter­esting ways. Back when most (of the rel­a­tively few) internet users could host freely, none of them had a gigabit con­nection at home; now, many internet users have band­width and processor cycles to spare, but we can’t host. Technological irony.

I truly did not have an opinion on IPv6 before December 2021. Suddenly, I am a wild-eyed evangelist, because this scheme, which works and is widely sup­ported but not widely deployed, sup­ports an internet in which every com­puter can be a host, if it wants to be.

For my part, I think an IPv6 internet would be almost unimag­in­ably fer­tile and productive. New kinds of apps and ser­vices would bloom like flowers in a dry meadow after the rain.

Ohhh welllll

The rise of “overlay net­works” is a nat­ural response to the lim­i­ta­tions and frus­tra­tions of the public IPv4 internet, and I think their accel­er­a­tion in the last few years deserves more attention.

These are illu­sory net­works estab­lished on top of the public internet. Often, you install a small pro­gram on your com­puter, and it allows the rest of your appli­ca­tions to “see” a group of other com­puters as if they were close at hand on your local net­work, even if they’re actu­ally far away, deep in NAT hell, unreach­able by normal means.

Here, I’ll make it more concrete:

For years, I was vexed by the task of reli­ably reaching the two servers in my office from any­where outside, e.g. from home. The office doesn’t have any fancy net­work hardware, just a Wi-Fi router, and the best I could manage was a brittle port-forwarding scheme that never worked for more than a couple of months at a time.

Then I discovered ZeroTier, which allows you to create and manage these overlay net­works. I installed it on my laptop, my old iMac, and my two servers, as well as an EC2 instance, and ever since, I have hopped between them with ease, no matter where I am. Each com­puter has a stable IP address in the 10.0.0.0/8 range, reserved for pri­vate net­works. It’s fabulous. They are all hosts again.

There’s a sim­ilar ser­vice called Tailscale, along with Slack’s some­what more robotic Nebula, and plenty more to come, I’m sure.

I am the only inhab­i­tant of my ZeroTier net­work, and I get the sense a lot of people use the ser­vice this way, but both ZeroTier and Tailscale allow you to create overlay net­works with many users — hundreds or more. In those cases, the net­works become little mini-internets for your coworkers, your group of friends, your pirate armada, whatever.

ZeroTier’s founder Adam Iery­menko some­times calls the ser­vice a “planetary data center”, insisting that it ought to feel like every com­puter IN THE WORLD is in the same room, regard­less of its actual loca­tion or net­work disposition. What a vision.

Along the same lines, one of Tailscale’s founders, David Crawshaw, has a blog post, actu­ally quite moving, titled Remembering the LAN. He writes:

The LAN was a mag­ical place to learn about com­puters. Besides the phys­ical aspect of assem­bling and disassem­bling machines, I could safely do things unthink­able on the modern internet: permission-less file sharing, exper­i­mental servers with no security, shared soft­ware where any one machine could easily bring down the net­work by typing in an innocuous command. Even when I did bring down the net­work the impact never left the building. I knew who I had to apologize to.

These new (old) “net­works within net­works” are, in 2022, both useful and evocative, and I think they open some VERY inter­esting space for work on peer-to-peer pro­to­cols and “ways of relating”—I keep writing that, I know it’s weird — appropriate for mini-internets of only 50 or 100 people.

I mean, you know I love com­puting at this scale.

But, as cool and promising as the overlay net­works are, I am not willing to sac­ri­fice “public” entirely, because what is the internet, if not an open invitation? And a suggestion, recurring, that you might not already know all the people you want to know. (If you’ve found your way to this newsletter, this web page, then you under­stand what I mean.)

Again, I feel the res­o­nance of the word “host”, and again, I think about etiquette, and conviviality, and satisfaction.

There’s also the peer-to-peer browser called Beaker, a pow­erfully cen­trifugal project; the idea is, or was, that your browser might host a web­site as easily as it nav­i­gates to one. Very 1993! (Thanks to David for prompting this addition.)

Its successor/inheritor/whatever, a pro­tocol called Hypercore, nudges in the same direction. That melan­choly feeling again: when you look at the code, you find it writhing with NAT contortions. So much time and energy and creativity, just to “get back to zero”, the ability to listen on the internet.

I mocked some­thing up using Hypercore’s great imple­men­ta­tion of a peer-to-peer DHT swarm, the same technique (PDF) used to coordinate, among other things, the nodes of the Ethereum net­work. (There are rel­a­tively few Ethereum nodes — on the order of thousands.) It’s a clever model, but/and, I found it really “heavy”, and, call me greedy or unreasonable, but I just want the simplicity of

socket.listen

which, when it works on the public internet, has decen­tral­iza­tion built in.

My interest in all of this is a bit odd-angled, because I don’t actu­ally care about decen­tral­iza­tion that much; or, I think it’s inter­esting, but I think a lot of things are inter­esting, and I’m willing to weigh them against each other, make compromises. TONS of compromises.

What I’m really inter­ested in — what I dream about — is the oppor­tu­nity to play with new pro­to­cols without taking on, perforce, the burden of infra­struc­ture. If we cast our gaze back to the early days of the World Wide Web, we find all these people coming up with cool new ideas for HTTP and HTML, writing new server soft­ware and new browsers with new features … and very con­spic­uously not “operating the web”. They didn’t have that power or that burden. The web was … out there.

Pretty good deal if you can get it.

Defining a pro­tocol is chal­lenging enough; pro­gramming a client appli­ca­tion more chal­lenging still; but backing up the database? I am just not inter­ested in that kind of work, that level of stress.

This, too, is part of Web3’s appeal: you can invent new things without run­ning the infra­struc­ture that sup­ports them. Of course, it trades that stress for a new kind, the mad­ness of immutable code, and charges you for every pulse of the vir­tual machine … so, for me, it’s a pass.

I have no great con­clu­sion to offer, and I’m sure most of this is old news to those of you who have tan­gled with peer-to-peer pro­to­cols before. I guess the surprise, for me — the thing I felt an urge to share — was the real­iza­tion that at least some of this ten­dency towards cen­tral­iza­tion is inherent in the archi­tec­ture of the internet itself. The trap­door of IPv4 exhaus­tion deliv­ered us to this place, stuck behind NAT, pounding on the glass. It’s a huge bummer!

I should add, as a coda, that ZeroTier has an inter­esting offering, a set of public net­works that anyone can join. Each only allows access to a par­tic­ular range of ports — you can choose them — via IPv6, which works for everyone, because it’s vir­tualized through ZeroTier’s soft­ware; it’s “fake IPv6”, I guess. These net­works are a lovely affordance, and there’s a world in which the instruc­tion manual for my new pro­tocol begins with: “Step 1. Down­load ZeroTier and join public net­work X.” Voila, the NAT problem goes away; it’s 1993 again; we can all see each other.

The thing that stops me from doing this, for now, is the lin­gering sense that, come on, there must be a way to offer some­thing inter­esting that is not woven into the infra­struc­ture and pro­tocol of one par­tic­ular company.

But who knows! I might get over it!