For many business IT teams today the discussion around container solutions (often in the same breath as microservices) is happening. There are some interesting challenges running containers in a production environment, many of them around operational issues and tools. But a great ecosystem is forming to help with the operational issues that containers and microservices present. Specifically, solutions like Goggle Kubernetes, Apache Mesos, Docker Swarm, and Hashicorp Atlas come to mind. These solutions help make life easier for an operations team dealing with containers and their deployment and lifecycle. However, these operations may be impacted in other ways and IPv6 could help address some of the integration issues of running containers in an existing data center environment. Let’s go over some of the shortcomings of current containers and microservices and the challenges of deploying them before looking at how IPv6 could help improve the situation.
Since Docker is the current hot container ecosystem solution let’s address what is and isn’t possible with Docker and how IPv6 impacts Docker’s implementation in an existing data center environment.
IPv4 address size issues with containers:
A Docker host has the potential of hosting hundreds to thousands of Docker guest applications depending on how it is implemented. One of the principal differences of Docker is the fact that a full networking stack is partitioned for the guest containers that are running on that host. It is not uncommon to see dozens to hundreds of IPv4 addresses being used in these sort of implementations. While RFC 1918 allows companies to operate without a large volume of public address space many network designs in existing data centers cannot accommodate the volume of IPv4 addresses a Docker host might need. This problem is a fundamental scaling problem with how we have traditionally built out our networks using and assigning IPv4 addresses. A /24 IPv4 subnet is the industry standard for many Layer 2/3 subnets today. Because of this, Docker can quickly run out of IPv4 addresses in a given subnet. This has forced Docker to implement port mapping and forwarding to help reserve IPv4 addresses, even in RFC 1918 address space. This is highly inefficient and requires that Docker networking solutions maintain too much state around ports and applications.
What does Docker provide for networking?
Docker has support for several different network implementations including macvlan, ipvlan and 802.1q trunking. Specifically these can support Layer 2 or bridge mode allowing the Docker container to appear on the same network as the Docker host. It’s also possible to support Layer 3 options with multiple IPv4 address space configurations. It can do NAT and RFC 1918 with port address translation overloading to allow Docker containers to come up and function. This is a common use case for developers when they are building a test or application environment to run on their local laptop. Alternately, it is possible to support a full layer 3 routed configuration where the Docker host routes the network address space and assigns those IPv4 addresses to containers as they are built. This means they are unique from an IPv4 address perspective and are capable of running application services with no port forwarding or mapping. This is the ideal way to run Docker as it keeps the container design clean, requires no state be maintained, and can scale to the IPv4 address range allocation.
So how would IPv6 change any of this?
One of the key value propositions of IPv6 has to do with the scale and size of the IPv6 address space. Remember that a standard prefix allocation for a single subnet is a /64 or 2^64 (e.g., 20 trillion trillion addresses). With that amount of addresses it’s possible to no longer worry about IP allocation ranges and dependencies on things like port forwarding.
Are there other ways in which IPv6 is better in this situation? Is it really that impactful? I believe it is for several structural reasons:
- Layer 3 (IPv4 or IPv6) scales and is a better network method for containers
- IPv6 /64 prefix allows for one time use of an address per container
- IPv6 allows for extremely efficient routing
- IPv6 routing to a Docker host reduces the need for Neighbor Discovery
- It is possible to use DHCPv6 Prefix Delegation so a Docker host can request additional IPv6 prefixes if it needs them
- IPv6 requires no state within the network be maintained
- IPv6 with Global Unicast Addresses means logging and rules are extremely easy to implement and only need to be tracked in a single network flow
So how would IPv6 best be leveraged in Docker?
The following diagram shows a sample data center running Docker and native Layer 3 networking. This design is what I consider a great starting point for many who wish to leverage Docker, IPv6, and scalable services (microservices).
I think what is most compelling about using IPv6 for containers is the elimination of scaling problems that are introduced with IPv4 and NAT. The challenges with scale by being restricted by the subnet size allocated for IPv4 is not an issue with the default /64 prefix used in IPv6. The logging and state information that NAT introduces that cause a huge data explosion with no operational benefit reducing your ability to scale because of unwanted costs and increased network brittleness. With IPv6 you only record 4to6 translations and the actual end to end traffic (source and destination IPv6 addresses) in the logs. Docker will maintain logging information about the Docker instance which will align to an IPv6 address (likely globally unique). This is very much unlike a configuration using IPv4 and NAT where the ADC will have to log IPv4 to IPv4, then Docker will again translate IPv4 to IPv4 not to mention port translation. The IPv4 addresses will be reused so they are not unique.
IPv6 is able to address all these scaling problems while making things operationally easier. (And if that isn’t a compelling and unique differentiator I don’t know what is!) In addition, the math works in favor of IPv6. Even with 10 million containers per second on a single Docker host it would take 58,494 years to use up the IPv6 address pool in a single /64. This means, for all practical purposes an IPv6 address is unique for the lifetime of a modern data center. So for developers, scale away and never run out of unique addresses to run your services on. And for operators, reduce storage for logs, have unique end to end session data and keep the routing simple and easy. It’s a win all the way around and I hope it’s adopted as the way forward!
You can find me on twitter as @ehorley and remember…
IPv6 is the future and the future is now!
– Ed
