Keep It Encrypted, Keep It Safe: Working with ESNI, DoH, and DoT

The latest developments in protecting privacy on the internet include encrypted TLS server name indication (ESNI) and encrypted DNS in the form of DNS over HTTPS (DoH), both of which are considered highly controversial by data collectors.

So here’s a quick look at the reasons they exist, the details about what they are, and the technology behind how they work.

DNS Needs to Function Correctly

Split-tunnel virtual private network (VPN) connections, the web proxy auto-discovery (WPAD) protocol, zero-configuration multicast DNS (mDNS), real-time blacklists (RBL), and many other widely deployed technologies break when DNS doesn’t operate correctly.

Breaking the Internet for Profit and Fame

As far back as 2003, internet users learned about the importance of DNS on a global scale when the company which then ran the .com top-level domain (TLD) chose to stop issuing NXDOMAIN responses. They instead impersonated any non-existent domain in an attempt to serve up more ads, sell more domains, and ultimately generate more revenue. The unanticipated knock-on effect resulted in a public debate and a damning report from ICANN’s Security and Stability Advisory Committee (SSAC). This project was indeed shut down but from a technical standpoint, the vulnerability persisted.

Later in 2008, another attempt at manipulating DNS for profit was called out publicly when some of the largest ISPs ended up introducing various avenues for cross-site scripting attacks against literally any website. Due to the nature of the vulnerabilities, even websites hosted on a private network and inaccessible from the internet can be exploited.

The problem found isn't with the core internet protocols but instead is a problem exacerbated by inappropriate monetization of certain DNS features.

Paul Vixie, Internet Systems Consortium (ISC)

Although it is true that the protocols themselves are not really the cause of the problem, it is also true that these protocols don’t prevent bad actors from abusing the system. So from a technical standpoint, the vulnerability persisted.

Breaking the Internet for Surveillance and Censorship

In 2016, it was observed that ISPs in Iran were manipulating DNS. This time, instead of injecting adverts, they were blocking access to the email servers of 139 out of the top 10,000 domains on the internet. It’s not clear if this is an intentional policy of denial of service against these specific domains—similar to the world-renowned censorship in China—or perhaps just an example of a bad technical implementation of whatever system is intercepting the DNS queries.

Not knowing why DNS is being intercepted is important: Even if we put aside the moral and legal questions about blanket surveillance and censorship, the technology used to do this is—by its very nature—not standards-compliant and could very well be interfering with your normal, everyday, reasonable, and lawful use of the internet.

Yet from a technical standpoint, the vulnerability persisted.

Breaking the Internet for Nefarious Purposes

Of course, it’s not just commercial entities and governments that are trying to intercept internet traffic for their own means. There are many examples of criminals attempting to hijack connections, usually for the purposes of stealing user data or tricking users into revealing important access credentials. Most prominently, DNS cache poisoning has been used to direct users to phishing sites, deploy ransomware, deploy botnets, deny service to specific websites, and much more.

TLS Leaks Who Connects to Whom

Transport Layer Security (TLS) is the technology behind HTTPS, the secure version of HTTP that we all rely on every day. Establishing a TLS connection requires a number of steps, during which time the server proves its identity by presenting a certificate, and new encryption keys are exchanged.

Having the server use a certificate to prove its identity is a very important step, as a part of the certificate is an asymmetric public encryption key.

When the client sends a message using this key, only the server which is in possession of the associated private key can read the message. As a result, anyone intercepting or listening in on the connection is locked out and unable to read the content.

However, that initial exchange is still done in the clear without encryption, which means that an observer will always know the identity of the server.

Domain Fronting

A few anti-censorship type tools worked around this leak in TLS for a period of time using a technique known as domain fronting. This exploits the fact that once an HTTPS connection is established, most large hosting providers do not check to see if the hostname presented in each HTTP request matches the one used in the TLS handshake. In terms of privacy tools, this was seen as a useful feature allowing secret communication with a hidden site. For hosting providers and data collectors, this was seen as an abuse of a way that the specification was implemented.

This is in and of itself a vulnerability, and as such has been fixed by several major hosting providers, including AWS.

Solving the Problem by Changing the Standard: Encrypted SNI (ESNI)

The idea behind ESNI is to prevent TLS from leaking any data by encrypting all of the messages, including the initial Client Hello message. This leaves any observer completely in the dark about what server certificate the server is presenting.

To do this, the client needs an encryption key before making the connection. Therefore, ESNI requires a specific set of ESNI encryption keys be placed in an SRV record in DNS.

The exact details of this are still in flux, however, as the specification is yet to be finalized. Despite this, an implementation of ESNI has already been put into production by Mozilla Firefox</a> and Cloudflare.

Securing and Encrypting DNS

For ESNI keys to be delivered without being intercepted, it’s important to safeguard against DNS tampering.

Ever since as far back as 1993, the internet community has been trying to secure DNS. The IETF notes that early problem-solving meetings discussed:

1. Protecting against the disclosure of DNS data to unauthorized parties
2. Ensuring data integrity
3. Data origin authentication

These meetings resulted in the Domain Name System Security Extensions (DNSSEC) standard in 1997. The standard chose to address two out of three of these concerns, as the design team made an explicit decision to rule all threats of data disclosure explicitly out of scope.

As such, DNSSEC means that users can trust that the answers to their DNS queries are what domain owners intend them to be. In the context of ESNI, this means that we know that the key we’re receiving hasn’t been tampered with, and thankfully, a lot of vulnerabilities mentioned above go away when DNSSEC is in use. However, it doesn’t ensure privacy and is therefore still vulnerable to problems introduced by surveillance and censorship systems.

Unfortunately, as it’s fully backward-compatible with “insecure DNS” and quite difficult to implement correctly, adoption of DNSSEC is very low. Many domain owners are giving up partway through trying to configure it, as evidenced by numerous invalid and half-set-up configurations seen in the wild.

DNS over HTTPS (DoH)

For ESNI keys to be delivered without watchers knowing which site users are trying to visit, it’s important to safeguard against DNS eavesdropping. As such, it is quite logical and understandable to say that encrypted DNS (such as with DoH) is a good thing. Yet, as it stands today, DNS is not encrypted.

There are moves by Mozilla, Google, APNIC, Cloudflare, Microsoft, and others to change this.

The Ideal Encrypted User Experience

A user wanting to leverage the technologies above might have a rather long list of requirements when it comes to the practical UX details of working with encryption. Likely, they would want to avoid the likes of:

• Being forced to use a specific DNS service provider (no matter how good it is) or for the choice to be invisible or difficult to find
• Each app on a device handling DNS differently (e.g., Firefox can find things that Safari cannot)
• All apps on a device having to create its own secure DNS implementation within itself
• Having to manually configure DNS multiple times
• Having to opt into privacy and security
• Apps falling back to insecure operation without user consent

Privacy-conscious users would instead want:

• Privacy from unwarranted surveillance by anyone
• Protection from targeted advertising they haven’t consented to
• Protection of their own published content (e.g., on a personal website) from being changed or manipulated en route to other viewers
• Assurance that viewers of their own published content won’t get into trouble simply for accessing it
• To continue to have the option of controlling DNS on their own networks (because sometimes, split-horizon DNS is good for keeping internal resources scoped to internal users)
• The ability to opt into, or at least opt out of, filtering at the DNS level (e.g., Quad9, CleanBrowsing, and OpenDNS Family Shield)
• Easy, hassle-free configuration (i.e., DHCP)
• To be asked to consent to using DNS without encryption
• Protection not just for browser traffic, but for all types of traffic, e.g., email, Slack, VoIP, SSH, VPN, etc.

Current Encryption Efforts

What options are there for users with the above goals?

Mozilla’s solution is a start, but far from ideal. They are only securing Firefox; the default is to override your OS settings in favor of their choice of provider (Cloudflare) without saying so, and it silently falls back to being insecure (in cases of encrypted DNS being blocked, etc.)

Google’s solution is a better approach. They are securing Android 9+—which applies to all apps. (I’m not sure on their plans for Chrome OS, but I suspect it’s in the works.) They are also securing Chrome on all platforms, but it only triggers security when the host platform is configured to use a provider that supports security. This is good in terms of user choice, but not ideal because it silently falls back to being insecure.

All other major browsers are also implementing support.

In the ideal situation for users, the wider community of software and OS developers would:

1. Stop implementing new DNS security features at the application level
2. Start implementing them at the OS level
3. Respect OS-level configuration, or get user consent

Implementing encrypted DNS at the OS level, we could continue to follow the same distributed model that we currently have for DNS resolvers. Running one’s own DNS server on their own network, and being able to make that resolver secure, makes sense to continue being an option, as should using a centralized provider.

Linux and BSD already have this functionality with a variety of options available. Unfortunately, no distro is enabling any of them by default, as far as I know. Check out nss-tls if you want something that will just plug into glibc.

Google’s DNS-over-TLS implementation in Android 9+ also does this already. It functions by testing the DNS server for encryption support. If it has it, then it will use it. If it doesn’t, then—as with Chrome—it continues in an insecure manner, without prompting for consent.

It’s worth noting that most networks are configured to use DNS servers that don’t support encryption, so the solution built into Android doesn’t yet change anything for most users. Maybe it would be better to offer to fall back to a centralized encrypted DNS in cases where decentralized does not support encryption.

For Apple and Microsoft flavor devices, support for encrypted DNS is yet to officially arrive as of this writing. With Microsoft having announced in November 2019 their intentions to support encrypted DNS, hopefully Apple will follow soon.

Encrypted DNS Workarounds

There are some workarounds in the form of proxies that can be run locally. With these, a user’s computer speaks non-encrypted DNS to itself, which then speaks encrypted DNS to whatever provider it’s configured to use. This is not an ideal solution, but as workarounds go, it is decent.

The inspiration for writing this article is the DNSCrypt proxy, which is very stable, has lots of bells and whistles, and is cross-platform. It supports the older DNSCrypt protocol, as well as the newer DNS over TLS (DoT) and DNS over HTTPS (DoH) protocols.

For Windows users, there’s an installer and GUI called Simple DNSCrypt, which is feature-complete and very easy to use.

I recommend it, but be aware that the world probably isn’t ready for you yet, and you may need to disable it from time to time (e.g., when you have to use a captive portal at your favorite cafe, or at a LAN party).

Other Options

Additionally, there is the Technitium DNS Server, which supports encrypted DNS, is cross-platform (Windows, MacOS, Linux on ARM/Raspberry Pi), and features a slick web interface. It’s under “other” because it’s more of an all-round tool than a solution specific to this issue. (It would likely be a good choice if you want to run your Raspberry Pi DNS server at home. I’d be interested to hear feedback from people who try it out in the comments below.)

For your Android or iOS (iPhone, iPad, etc.) devices, there is also the 1.1.1.1 app, which will force all your DNS queries over the Cloudflare Encrypted DNS service. I hear that there are more flexible apps also, such as Intra, but I’ve not spent the time to test them out yet.

Of course, you can also enable encrypted DNS in both Firefox and Chrome - just bear in mind all the caveats discussed above.

DNS Reliability: Job Number One

There’s a lot of controversy with internet privacy technology. However, when it comes to implementing security and privacy measures, the technology in question is not primarily about keeping secrets. It’s about ensuring reliability and guaranteeing that the technology continues to function correctly despite the behavior of others. Yet, we need to be mindful that, just as technology without privacy safeguards is bad, safeguards that are poorly implemented can only make the situation worse.