auspicacious

This is the written version of a talk I gave at Nerd Nite Tokyo on November 18, 2022. It is not a transcription.

When browsing the Web, always make sure you are using HTTPS. It ensures that the connection between you and the service you're using is end-to-end encrypted, keeping you safe. This is common advice, and good advice.

But what does "end-to-end" actually mean? And does using HTTPS really ensure safety?

The answer is complicated; it grows more complicated each year. But, by starting with older, simpler systems, we can build up a mental model. That's why I'm going to start with Abraham Lincoln, make a stop in the 1980s, and finally wind up in the present day. There are two intertwined stories here: the story of surveillance and the story of freedom of access.

Private? Who cares?

What is valuable is always vulnerable, and as more of our lives have moved online, it is harder for us to understand what could be valuable to all the worst elements of society, whether hackers, stalkers, thieves, corporations, or governments. Some people may feel entirely safe in sharing everything, while others have reason to fear exposure. Protecting all of our communications is a prerequisite for protecting everyone, regardless of their needs.

The telegram: professionals only

The telegram was the first means of instantaneous communication at a distance, though it differed critically from telephones and the Internet: operation required specialized equipment and operators, so messages were generally sent through centralized offices.

No reasonable person would expect their messages to remain shielded from strangers' eyes as they handed them over to be keyed in, and yet remarkably personal and private information was shared all the same, as seen in this archive of Civil War messages. Just as in the early days of the Internet, users placed a high degree of trust in the operators in exchange for the revolutionary abilities the telegraph afforded them.

Much like the Internet, the telegraph network also experienced explosive growth. By 1861, scarcely fifteen years after the first test messages were sent between Baltimore and Washington, telegraph lines connected most parts of America together -- just in time for the Civil War. The value of the information being sent through the wire was immediately obvious. If it could be collected and analyzed, the direction of the war might be changed.

Lincoln's telegraph spying

Lincoln did just that. In 1862, he authorized Secretary of War Edwin M. Stanton to secretly reroute the nation's telegraph lines through his office, not only providing intelligence, but enabling the censorship of messages deemed to be harmful to the war effort.

Encryption likewise was used by both the North and South to try and regain privacy; today, there are still Signal Corps reenactors who keep knowledge of those techniques alive.

The beginning of end to end

Encryption used at this time fulfilled part of the definition of end-to-end, though not all of it. A telegram sent from, for instance, President Lincoln to Ulysses S. Grant, might be obscured in such a way that telegraph operators didn't understand its content. Only the sender and the receiver, using matching codebooks, would be able to understand the message.

The way this worked was simple: each codebook contained a list of sensitive phrases, such as "signed," "Pres. of U.S." or "Ulysses S. Grant," and corresponding nonsense words. The phrase "yawl Balfour" might be decoded as "signed Pres. of U.S." Such a method is easy to use in the field and does not rely on complex mathematics, although its utility is limited to only the situations anticipated by the author.

These methods were also largely independent of the means of transmission: although this example was sent by telegraph, the same techniques could be used to obscure a message sent by mail, smoke signal, or carrier pigeon. This is an inherent quality of end to end encryption: it should remain secure regardless of how messages are sent.

However, there's more to it than this.

The telephone and truly personal communication

Looking at the average American home a century later, in the 1960s, it is likely that you would find a telephone -- possibly, if they were in a city, even a touch-tone phone using the same keypad displayed on a smartphone today.

Over the previous decades, it had become possible to directly dial people in the same city as you without the help of a human operator; by the 1960s, it was becoming possible to dial anyone in America without the assistance of a professional. Freedom of access to telecommunications was increasing, and things once only said and done in person were rapidly moving to the wire.

At the same time, the government surveillance apparatus established during the second World War had not disappeared, but was rather increasing in scope as a reaction to the Cold War. Although any American might know that someone could be listening in on their conversation, increasingly, using a phone felt private in a way that a telegraph or operator-moderated phone call could not.

This feeling of freedom and personal control would soon grow.

Ring, ring, ring, ring, ring, ring, ring, ring: it's Carterphone

(Apologies to Raffi).

AT&T, a company which would later choose the Death Star as its corporate logo, maintained absolute control over what could be connected to the public telephone network. Small companies who wished to connect their devices to a telephone had to use an acoustic coupler rather than plugging in directly, which reduced the functionality and quality that they could offer.

One such device was the Carterfone, designed to allow the connection of a two-way radio to a telephone line. The makers of the Carterfone petitioned the FCC to allow them to connect directly, rather than through an acoustic coupler; after years of deliberation, this was permitted in 1968. As a result of this decision, "any lawful device" -- any device that did not interfere with AT&T's network -- could be plugged directly into the wall.

In America, this opened up a new era in personal communication: the most obvious example of this was the many novelty telephones that would be made in the next several decades, but, more importantly, fax machines, answering machines, and modems, all in their infancy at this time, could now be developed, marketed, and freely used without having to ask permission from AT&T.

More valuable, more vulnerable

At around the same time, practical, end-to-end encrypted devices began to arrive on the market. All governments already had a clear need for private communications, and as cost declined, corporations also began to use them, with good reason: by the year 2000, it was public knowledge that American intelligence had used intercepted phone calls to gain an advantage for American industry over their European competitors.

While there is some debate about the moral qualities of that particular instance of spying -- it uncovered serious corruption and bribery around the world -- the need for encryption was increasingly clear to any entity with deep pockets.

The online Crypto Museum has an excellent collection of early cryptographic devices and articles on how they worked (or didn't) in practice. By the 1970s, it was possible to buy briefcase-sized devices that connected to normal telephone lines and allowed users to send and receive encrypted text messages.

The rest of end to end

Consider encrypted Civil War telegrams. Although an operator might not be sure what "yawl Balfour" meant, they could still modify the words that were sent, possibly changing the meaning. A simple change of a particle, such as "from" to "to," could have a devastating impact on troop movements.

On the other hand, by the 1970s, encrypted teletype machines were general-purpose: any information that could be reduced to digital bits could be encrypted without leaving any particular fingerprints behind. Each user might also have an individualized key, often contained on a special card; using math appropriately, it became possible to demonstrate that the message had not been modified by any interlopers along the way, and also provide some assurance that the person on the other end was who you thought they were (or at least had the correct key card).

These devices, like earlier systems, could be adapted to any means of communication, not just the telephone.

Already, then, at this point in history, it is possible to find devices that fulfill a basic definition of end to end security:

• Only the people (or machines) at the beginning and end should be able to understand the message. Any operators who see the message along the way cannot understand it.
• The message cannot be modified in transit.
• You are communicating with whom you think you are communicating with.

Cryptography and its discontents

Cryptography works. In any given era, there will be practical methods of encryption that are good enough to protect their users from attacks. Over time, weaknesses might be found in older systems, but new systems have always been ready to replace them.

As encryption tools began to be adopted, the growing opacity of communications was a problem for anyone who wanted to intercept or modify them; the most obvious such group in the 1970s were national intelligence agencies. Even the NSA cannot break good encryption, but they can work around it in other ways.

In 1970, the West German BND and the American CIA jointly purchased a Swiss maker of encryption devices, Crypto AG. For the next few decades, in what was originally called Operation Rubicon, they intentionally sold faulty encryption devices, allowing them to eavesdrop on communications that the owners -- including many of the world's governments -- believed to be secret.

It is easier to work around encryption than through it.

To the Internet

The modern Internet is a pretty good place: it has a very high level of freedom of access that allows virtually anyone to connect almost anything to it. As long as you pay your monthly bill and don't send too much junk, there are essentially no gatekeepers. This might seem entirely natural, but it is worth remembering that the most successful online service before the Internet was the French Minitel, which heavily restricted what terminals could do, and, since the rise of the Internet, we have seen new spaces and services also tend towards control -- consider Apple or Google's walled-garden app stores. The Carterfone decision, in some ways, was an aberration.

This freedom of access, however, also means that surveillance is now something that is done by ordinary people sitting at coffee shops or airports, not just by government agencies. Surveillance is done for many more reasons as well, from stealing credit card numbers to stalking ex-girlfriends. Encryption is more important than ever.

The true beginning of end to end

We are lucky, therefore, that in the past decade good quality encryption has started becoming more and more available to the average person, in the places that they need it most.

For instance, anything has been possible at Zombo.com since 1999, but, thanks to HTTPS becoming much easier and cheaper to set up in the past few years, as of 2022 only you and Zombo.com's operators are likely to know just what you did there. Imagine it this way:

┌──┐ HTTPS ┌──────────────────────┐
│Me├───────┤https://www.zombo.com/│
└──┘       └──────────────────────┘

You can sit in a coffee shop and log in to virtually any website without fearing that your password will be exposed to everyone in the cafe. So many sites now use HTTPS that Firefox has introduced an HTTPS-only mode which, if switched on, will warn you any time you are not using a secure connection. (You should turn on this mode).

What makes this even better is that the standards HTTPS builds on, such as TLS and AES, have been developed in an open, collaborative fashion. Some of the most commonly used implementations are also open source. It would be difficult for a single intelligence agency, such as the NSA, to subvert a large number of devices at once in the same way they did decades ago with Operation Rubicon.

The end of end to end

Freedom of access has another downside, however. The Internet is full of broken and hacked devices -- not just the PC sitting under your parents' desk since 2005, but also many machines run by highly technical companies sitting in datacenters all around the world. These systems are regularly used to launch attacks on websites, especially DDoS attacks which can send huge volumes of requests to a site, overwhelming it or costing the operator enormous sums of money.

Even very technically-capable companies have difficulty handling these attacks themselves; a very small number of CDN providers, such as Cloudflare, however, have built businesses around attack protection and prevention.

By their own estimates, Cloudflare now handles over 20% of the Internet's traffic. Even more remarkably, many of their customers pay nothing at all for protection, and Cloudflare itself is not profitable. For these reasons, the rest of this article will focus on Cloudflare rather than their competitors, which all work in the same way.

When a site is protected by Cloudflare, all requests to that site must first be received and reviewed by Cloudflare. Imagine it this way:

┌──┐ HTTPS ┌──────────┐ HTTPS ┌──────────────────────┐
│Me├───────┤Cloudflare├───────┤https://www.zombo.com/│
└──┘       └──────────┘       └──────────────────────┘

There are now two end-to-end encrypted connections: one from you to Cloudflare, and another from Cloudflare to Zombo.com. Cloudflare not only has visibility into everything sent between you and Zombo.com, they could modify messages as well or reroute them to some kind of fake Zombo.

Wild speculation

Is this end-to-end encryption?

Many people seem to think so: since Cloudflare has contracted with the website to provide a service, they are effectively part of the site's infrastructure in the same way that the site's hosting provider is. And, of course, Cloudflare has gone to significant lengths to build trust; their engineering is considered to be industry-leading, and they have been adamant about protecting free speech on their platform.

However, the reality remains that Cloudflare is a single point of vulnerability that 20% of all Internet traffic passes through. This makes them an incredibly valuable target for any government intelligence agency from America to China to Russia to New Zealand and everyone in between. In today's open world, there are likely criminal organizations with the capacity to attempt penetrating Cloudflare as well.

I am not aware of any evidence that Cloudflare is a front company in the way that Crypto AG was. That being said, Cloudflare is a money-losing American company that is a target for every well-equipped, malicious hacker on the planet. As critical infrastructure for the western Internet, it seems likely to be the case that they cooperate closely with the American law enforcement and intelligence communities in order to survive the threats against them.

Again, though, that does not necessarily imply that they are anything like Crypto AG. It is, however, also worth considering that Cloudflare's adamant protection of websites promoting hate and violence -- repeatedly protecting some of the most vile people on the Internet, even when they are not paying customers -- can be interpreted either as a sign of their committment to non-discrimination, or as a sign that they are helping gather intelligence on those hate sites for a partner.

Coalescing so much of the Internet's traffic and trust into a single company clearly creates significant risks for a free and open society. I cannot judge whether those risks are worse than that of some of the potential alternatives, given the current Internet landscape, but I must believe that we can do better.

Breaking CDNdependence

Centralized CDNs like Cloudflare have become a critical and unavoidable part of the Internet, but there are a few things that ordinary people, engineers, and governments can do to improve their lives and reduce their dependence on them.

My earlier article "Rehumanize Yourself" contains relevant advice and specifics as well.

Use uBlock Origin to block ads and trackers

uBlock Origin is a browser add-on which can be used to block any content; it is commonly used to block advertising, trackers, and annoyances.

This will reduce your exposure to embedded third-party resources served from CDNs and used to serve advertising and track you. Any one of these CDNs, and there are many, could be compromised and used to monitor your activity on sites you regularly visit. Plus, the web is faster, more pleasant, and easier to navigate without advertising. There are other, similar blockers available, but I do not have personal experience using them.

Of course, there is a risk that uBlock Origin could be compromised in a way that could leak secrets as you browse the Web. My own opinion is that this risk is less than that posed by advertising and tracking CDNs, but I also think that this is impossible to quantify.

Use Decentraleyes to avoid loading more files from CDNs

Web developers often load common JavaScript libraries, fonts, and other resources from free CDN services. The goal is to allow for faster loading of websites; however, it also exposes end-users to the risk that a single hacked CDN could compromise many websites.

The browser add-on Decentraleyes redirects many common requests away from these CDNs to local copies of the same files, avoiding potential compromise. (On the other hand, if a malicious version of Decentraleyes were to be uploaded, you could be compromised in the same way).

Support alternative services

Often, CDNs are used because they are the easiest and most expedient way to make something work. Find and support services that are taking more difficult routes to build better products. For instance, Signal, a nonprofit competitor to Facebook Messenger, WhatsApp, LINE, iMessage, and other message apps, is very open about the steps they take to ensure that they are adhering to a much higher standard of end-to-end encryption.

Make boring, plain websites

As websites become more complex, CDNs become more critical -- both because it is difficult to serve large image, script, and video files without one, and because DDoS attacks can often succeed by exploiting computationally-heavy programs runnable through websites.

Focusing on simplicity brings other benefits: the time that was being used to over-design and distract can instead be spent on making sites more accessible to people with visual impairments, and on building safer, faster, and more reliable systems.

Incentivize secure computing

This suggestion gets into the realm of policy, not individual action. An Internet of better-secured and maintained devices would be safer for everyone, and prevent many attacks before they start. More secure systems could be incentivized through well-designed government policy. However, there is also a risk that poorly-designed policy could lead to a significant drop in freedom of access to the Internet.

Better police Internet crime

Increasing the potential costs for attackers by solving a larger portion of Internet crimes is another non-technical and likely necessary solution. Unfortunately, there are many countries that have demonstrated little interest in pursuing criminals within their own borders, as long as those criminals only attack people in other countries. It is difficult to see how this situation could improve in the near future.

Figure out how to build a DDoS-resistant Internet

It is often more difficult, if not impossible, to solve a problem through purely technical means instead of through a combination of social and technical factors. Building a more resilient, decentralized Internet will require both social and technical means.

I do not know where to begin on the technical side, except to comment that nothing in the cryptocurrency or Web3 worlds is the answer. Moxie's commentary on web3 is instructive as to why.

Beyond the end

The problem with building a safe Internet for everyone is that there are many different aspects to safety. Ensuring end to end encryption is just a small part of secure system design, and yet has enough challenges on its own to occupy many lifetimes.

This article began with a simple Civil War encryption system, which was augmented and replaced as new threats emerged. It is unlikely that there will ever be a perfectly safe or secure system, for any definition of those words. Instead, we must continue to build and update our own mental models of how these systems work, constantly asking how they can be made better without losing sight of the context that they operate in.

Or maybe we should just go back to 1980s novelty phones.