Bang paths, source routing, and how email trips were planned
Turn by turn directions, for email
Every email has a past, a history, a trace-route embedded one layer deep. Underneath the “Hey there!” and “Thank you for subscribing” text, in your email’s HTML code, lies a record of every hop the message took to reach your inbox.
A Buttondown email takes four stops on its way to your inbox, as do most emails from third-party services. Buttondown’s API creates the email in Postmark. Postmark’s MTA server looks up your domain’s MX records and finds (say) smtp.google.com, and so sends the message to one of Gmail’s 142.250.x.x servers. Gmail’s inbound MX node sends the message along to Gmail’s internal router, which in turn drops the email in your Gmail mailbox.
A circuitous route, akin to FedExing a box from Bangkok to Boston only to see it stop off at Guangzhou, Osaka, Anchorage, and Memphis en route. Yet today, for email and envelopes alike, the logistics are abstracted. You only need to know the destination, and the respective services handle the minutiae.
Today, that is. For email’s first fifteen years of existence, the only way to reliably send emails was to map out their path ahead of time.
“A billion years ago you used to be able to route email just via the address,” said Richard Hewitt, with source routing-formatted email addresses like @foo.com,@bar.com,@baz.com,@nasa.gov,@etc.com:case@wintermute.com, where foo, bar, baz, NASA, and etc are hops, and case@wintermute.com is the destination address. Like a digital version of travel-hacking, it at times turned sending emails into a mad game of routing roulette. “Think the limit was 30 hops? So we used to have competitions to see how far round the world we could make emails go before coming back,” said Hewitt.
Over five decades later, with DNS and SMTP reliably routing our messages, there’s still an official way to route emails manually, hidden in plain sight in the SMTP protocol, theoretically ready to route emails when swaths of the internet are down (cough AWS cough Cloudflare) even if we don’t use it.
You’ve got mail
Email, originally, wasn’t very mobile. Neither was computing, for that matter. Your average computer took up a closet and cost anywhere from $110,000 to $342,000—or as much as $3 million in today’s dollars. You didn’t own a computer. Your company or university lab did, and you merely shared it with hundreds of others.
In that environment, email was little more than shared notes on the same computer. Like writing a grocery list on a shared iPad and leaving it on the fridge for your family to see, the first email inboxes were a single text file with appended messages. A user would write a message during their shared computing time, software would append it to their mailbox file, and the recipient would see a You have mail message when they logged in.
Everything happened between different accounts on the same computer, something you can still do, today, with the mail command—descendant from the same mail app bundled with the first version of UNIX in 1971—on any modern Mac or Linux computer. Even a decade later, when people first got email, often “only people we could email were each other,” as @coldcode reminisced about their first email address in 1984.
Sending a message to your colleague was nice. That is, until the lab splurged on a second computer, and you were back to leaving paper notes on their desk. MIT solved the problem, locally, in 1965 with MAILBOX that could send messages between time-sharing computers in the same lab. And Ray Tomlinson, famously, turned mail into email by sending the first messages between two different ARPANET-connected computers in 1971, with someone’s account name, an @ symbol, then their server’s name.
Two short years later, email was the most popular ARPANET use-case, responsible for 75% of all traffic on the network.
Yet the network itself was comparatively tiny, with 4 nodes in 1969 and 15 in 1971. If you wanted to email someone on an ARPANET node from an ARPANET node, email worked much like today. If they were on another computing system and another network with different messaging software, your message needed you to play travel agent.
Beating a path to your inbox
It’s easy to take for granted the DNS system that powers the modern internet. Good luck remembering 142.250.66.110, but google.com is easy enough to remember, and DNS lets machines handle the numbers for you.
But when Tomlinson invented email, DNS was twelve years from inception, and MX records—the DNS bits with a domain’s email server—wouldn’t come until 1986, fifteen years after the first email between ARPANET computers. In the meantime, the only way to figure out which computer was which was the official hosts.txt file, hosted by Stanford, with a list of every known computer on the network, its name, and IP address.
Email was left to figure things out on its own. You didn’t just email bob@mit.edu. You had to figure out if your computer spoke the same language as theirs, if you were both on the same network, and which intermediary computers connected you both. Harvard and MIT, both on ARPANET, had things easy. Smaller universities often had to first send a message to an email gateway then to an ARPANET node and finally to the recipient. Skip one leg of the journey, or send to an incompatible computer, and your email would disappear into the ether.
That’s why the first version of Simple Mail Transfer Protocol (what we know as SMTP today) wasn’t all that simple in 1982. Along with the specifics that underpin email to this day, it also defined source routing to spell out your email’s journey in an attempt to codify ad hoc ways to route emails that had sprung up over email’s decade-long life.
“The forward-path may be a source route of the form ‘@ONE,@TWO:JOE@THREE’, where ONE, TWO, and THREE are hosts,” explained RFC 821, the format that, today, is still officially supported in the current SMTP spec even though it’s been long-since deprecated. The destination email address is JOE@THREE, or perhaps joe@three.com in a more modern rendition. @ONE and @TWO are servers the email should pass, en route to its destination.
Emailing from your smaller research lab would have required a source routed email address that looked something like @csnet-gateway,@sri-unix.arpa,@mit.edu:bob@mit.edu to reach a Bob at MIT via a CSNet gateway and an ARPANET node. “The route is information about how to get there,” emphasized the spec.
ARPANET and its fledgling SMTP weren't the only messaging plays on the early internet, though. As a way to broaden accessibility, while it wasn’t officially supported, percent signs were often used in source routes instead of commas. And UUCP, a set of UNIX commands to copy files that also powered an early stab at email, had server names separated by ! or bangs, as they were called.
“There are (among others) two major kinds of mailing address syntax used in the UUCP world,” writes RFC 976, published in 1986. Alongside the “user@domain syntax,” the spec noted, “The a!b!c!user ("bang paths") is used by older UUCP software to explicitly route mail to the destination.” It resulted in unwieldy addresses that listed every server, starting with the sender and ending with the recipient, like utzoo!watmath!clyde!burl!ulysses!allegra!mit-eddie!rms@mit-prep, Richard Stallman’s actual email address from a 1985 Usenet group. “Bang paths of eight to ten machines (or hops) were not uncommon in 1981,” notes an anonymous Wikipedia editor, often to route messages and optimize delivery paths, as otherwise “late-night dial-up UUCP links could cause week-long transmission times.”
As such, some of the earliest email lists and newsletters had subscriber rolls filled with a wide range of different email address formats.
A discussion about one of the earliest jazz mailing lists, started in 1986, listed the email addresses of their original member roll—and over half included source routing. Perfectly modern emails like ebm@ingres.berkeley.edu lived alongside davidsen@ge-crd.arpa (complete with an original .arpa domain), gidi%wisdom.bitnet@wiscvm.wisc.edu (with a percent-formatted source routed email), and the bang-filled domain-less decvax!decwrl!dec-rhea!dec-fremen!ryan.
“Most of you have probably never seen email addresses like that, involving UUCP ("!") and source routing ("%"), and domains ending in .arpa -- as the migration to today's domain naming system of .com and .net and so on was in-progress at the time,” wrote newsletter admin Rich Kulawiec. “It wasn't just as simple as user@domain back then; you often had to specify the delivery route explicitly, as many of the addresses with multiple !'s illustrate.”
What is dead may never die
By 1989, seven years after SMTP was codified, the workarounds and hacks were deemed unnecessary. “The intent is to discourage all source routing and to abolish explicit source routing for mail delivery within the Internet environment. Source-routing is unnecessary; the simple target address ‘user@domain’ should always suffice,” stated RFC 1123.
Yet the Internet—and remember, this was pre-WWW, two years before Tim Berners-Lee would publish the first web page on the internet—was still not ubiquitous. “There are non-Internet mail environments where delivery mechanisms do depend upon source routes.” Thus Stallman’s a!b!c!user@gateway-domain-style email, with the bang paths showing the local network how to route the message to the internet in the first place, with the more modern domain email section routing the message on the open internet.
“Source routing was sometimes needed to make sure that a message was routed through a gateway that was known to be connected to a remote island,” noted an RFC in 1994. It was a compatibility hack as much as anything, one that the spread of DNS and the success of the internet and the WWW increasingly rendered irrelevant.
But progress can be slow. Even a decade later, people were still using source routing to solve random email problems. A network admin for the American Center for Physics, recommended the percent-variant of source routing in 1996 as a way to resolve an issue where 3% of their emails never seemed to be delivered. Source routing was the “the Michael Jordan of e-mail, over the sun, around the unix, under the dos and nothing but gateway,” concluded the discussion, yet it was the only way at the time to resolve their issues.
As late as 2008, another team was using source routing to get email working between servers. “There is no other way to specify an SMTP server other than in the TO field,” they concluded, and had to get the source routing syntax correct to get their emails through.
Old protocols die hard
For all the standardization attempts, source routing remained a hack to link non-compatible computers, software, and networks together. It resisted all attempts at conformity, and each team and implementation seemed to have their own tiny tricks. It was the last-ditch weapon ’90’s system admins would use to deliver the most difficult-to-send emails.
“I think it's a bit premature to abolish source routing,” wrote DNS security researcher Florian Weimer in 2001, concerned as he was about “a future fragmented DNS.” And, perhaps, inertia and precaution together explain why source routing has lived on in SMTP’s specs, why it still theoretically works today if you tried hard enough, though it is largely ignored by modern email servers and software.
But, short of the end of the internet as we know it, it’s hard to imagine anyone today needing to learn source routing syntax. “Frankly, it would be really insane to reestablish source routing for email in order to make it easier to fragment DNS,” replied one Keith Moore. “If we're going to take two giant steps backward in networking, why not go all the way back to semaphores and smoke signals?”
Source routing is left to history as the most esoteric of old networking arts. When, to Zoomers, @yahoo and @hotmail emails confer seniority on the internet, source routing addresses are what truly dates one. “How about us guys who exchanged mail before DNS using bang paths?,” replied Eric J Hägerström to the discussion. “I'm ‘gave people my email address in bang path form’ years old,” wrote Ripple CTO David Schwartz.
And in a way, today, when you build Gmail filters to auto-sort messages, when you add a period to your Gmail email to create an alias, when you build an automated workflow in your newsletter to send specific messages based on criteria, you’re continuing the ad-hoc work of building email, one hack and workaround at a time.
You’re able to message anyone, anywhere on earth, regardless of their operating system or internet service provider or server, thanks to the hacks upon hacks that have given us this nebulous connection, fragilely-robust system we call email. Source routing, lying fallow in email’s specs, reminds us of the path it took to get here.