“If a tree falls in a forest and no one is around to hear it, does it make a sound?” Because I am fascinated by quantum mechanics, that’s one of my favorite quotes. It also helps illustrate a situation where significant changes have come to DNS that service providers need to prepare. What’s interesting is that most users and subscribers are entirely unaware of these changes, which can end up affecting their online experience.
In the early days of the Internet, we started with a situation where we had computers on the early Internet that couldn’t find each other. And people were kind of “memorizing” IP addresses. That evolved into a situation where we had the hosts.txt file that network operators distributed and used. But after a while, the number of servers grew too large, and the file just couldn’t easily be distributed amongst other folks. And so, the DNS service was created. And this was some time ago during a point in the evolution of the Internet where encryption was really not given much thought. Things like security, especially early in the 70s and 80s was not a priority. And so, DNS was created unencrypted. And it was one of the earlier protocols out there on the Internet. It predates things like HTTP by several years.
But oh, how things have changed. We’ve evolved into that situation now, where we’re dealing more and more with things like encrypted DNS. So, let’s talk about what a typical subscriber DNS transaction looks like today or what it looked like previously. It’s a straightforward transaction; it’s a two-packet conversation where you send a DNS query and then receive a response. If your DNS server already knows the answer, it will send it back. If it doesn’t know the answer, it’ll go out, ask the Internet, bring that answer in, put it in its cache and send it back down to the user. DNS transactions run on port 53 today in the clear.
The “Last Mile” Problem
The negative to that is this is done in the clear. There’s a “last mile” security problem where DNS traffic could be interfered with in a way that you don’t like. You can send that query out, and you don’t guarantee that the response received came from the place you thought it. While there are technologies like DNSSEC that can help secure part of that, DNSSEC doesn’t guarantee at the last mile that response received was the response that you’re looking for. We’ve seen malware take advantage of this to trick your users or their devices into thinking that a site is one thing when it’s another. We’ve seen malware that makes you think your banking website is not your banking website. Users may give their credentials to someone unintentionally because the DNS traffic is unencrypted. But on the flip side, we also see good things by the way unencrypted DNS functions. For example, network operators often use this unencrypted DNS traffic to see if their users receive a positive subscriber experience. Now that we can encrypt this traffic, that may affect some service provider’s ability to monitor those situations.
Encrypted DNS to the Rescue!
We have two solutions put out by the IETF–conflicting solutions (think of it as a VHS and Betamax kind of situation). We have two protocols that will allow you to encrypted DNS: DNS over TLS and DNS over HTTPS. Both of them solve the problem, they encrypt the traffic pretty adequately between the destinations, but they go about it in a very different way. DoT uses a dedicated port that’s easy for people to observe. There’s only one thing that uses Port 853: it’s dedicated to DNS over TLS. Whereas with DoH, we have Port 443, which is the same port as HTTPS. That makes it hard for providers to distinguish between DoH traffic and regular HTTPS traffic on their networks. And so, it’s tough for you to observe and determine if users still have a positive network experience using DNS because DoH is the most commonly adopted encrypted DNS solution.
So, we run into a situation where this encrypted DNS traffic can bypass centralized provider DNS. Having encrypted traffic like this on your network can be troublesome when you don’t have that visibility to what’s going on with your subscribers. But how is this happening? Are subscribers aware?
They are not.
Are Most Users Aware?
No! And that’s a problem. We’ve run into a situation where encrypted DNS is rolling out in ways that service providers aren’t easily able to interact. Still, the general public is not even aware of the change. While we’ve witnessed cases where it was not necessarily harmful to service providers, but we’ve also seen situations where it was directly adverse. For example, when provider DNS infrastructure is bypassed, DNS-based security controls protecting subscribers from common threats such as lookalike domains and malware sites, increasing exposure to data exfiltration and malware proliferation on the provider network. It’s one of the reasons why the United States National Security Agency (NSA) recently posted guidance that organizations host their own DoH resolvers and avoid sending internal DNS traffic to external third-party resolvers. Within months of being released, we saw malware that used DoH to encrypt malicious communications allowing it to hide in regular HTTPS traffic and install malware that can steal data or add a victim to a botnet.
For providers, it’s Apple and Mozilla that have the most troubling rollout methodologies. My first job was in IT, and I can easily foresee how these changes can affect customer service and the subscriber experience.
Apple
Apple has updated iOS 14 and macOS Big Sur last year and changed how DNS operated in their operating system. macOS and iOS 14 now prefer encrypted DNS services over the unencrypted DNS services that most service providers offer. We also have a situation where domain owners can sign their domain to signal to an Apple device to send traffic for that domain directly to that domain server encrypted. A similar negative you run into with them is that it works for good domain owners and bad domain owners. So a malware domain owner can effectively bypass any security functionality you may have implemented to protect your users and subscribers using DNS.
Mozilla
We also similarly situate with Mozilla, where Cloudflare is now the default trusted recursive resolver using DoH. Mozilla rolled out support for DoH here in the United States, and they are talking about rolling it out in other countries around the world. For users, Mozilla displays a pop-up in their browser that effectively announces, “we are going to encrypt your traffic securely, and send it to Cloudflare.” Their options are effectively “Disable” or “OK.” From the standpoint of most users, this is worded in such a way that most will click “OK.” Upon accepting the change, the browser will route DNS traffic to Mozilla’s service.
The Bottom Line for Service Providers
The best way to deal with this situation as a service provider is to offer encrypted DNS on your network rather than trusting it to third parties where you may not have a business relationship. If you don’t have encrypted DNS servers on your network, applications can override your network DNS settings. Apple allows application creators to include a small profile (which is effectively just a small text file) that will redirect any DNS traffic to a DNS server of the application’s choice. While subscribers use that particular application, all the DNS for that application will get routed out to an encrypted DNS server operated by that application owner. That can be problematic with applications owned by perhaps hostile countries or social networking applications that may not respect your country’s laws or your subscribers’ privacy.
