Does IPv6 change the number of possible nodes on a single LAN segment?
As people start learning about and deploying IPv6 they start considering how IPv6 changes the way they design and deploy networks and systems. Based on the IPv6 Addressing Architecture (RFC 4291), the standard IPv6 prefix length for LANs is a /64 which provides an astronomically large number of possible host Interface Identifiers (IIDs). In IPv6, it is common to have the first “high-order” 64-bits represent the network number and the last “least-significant” 64-bits represent the individual node number. If we calculate that out; 2^64 equals 18,446,744,073,709,551,616 unique IIDs, but “18 quintillion” is much easier to say. That is an extremely large number of possible nodes on a single network, but a network surely would not have nearly this many nodes (much less IPv6-capable ones).
Wasteful IPv6 Addressing?
To many, it may seem wasteful to use a /64 if you only have a few nodes on that network. If a single /64 can have 18 quintillion nodes, but the network only had a hundred computers connected, that may seem wasteful. Furthermore, if you have a back-to-back direct connection (i.e., a point-to-point link) between two network devices and you allocate a /64 prefix, then you only have two active nodes in all that address space. You could use a /127 prefix (RFC 6164) for an IPv6 network with two devices, but for IPAM simplicity, you might still reserve a /64 for that network.
The reality, is that it is not wasteful to use a /64 for any-sized IPv6 network. Consider this, if you had a network with 1,000,000 nodes connected, your address efficiency is still infinitesimally small. One million divided by 18 quintillion is an extremely small number. If you have a network with only two nodes, two divided by 18 quintillion is just about as infinitesimally small as the efficiency for the million-node network. Therefore, whether you have two nodes or two million nodes, you are using only an extremely small fraction of the /64. If you want to see the math to calculate the efficiency of address allocation, consult Tom Coffeen’s IPv6 Address Planning book and look on page 3.
Even though an IPv6 prefix allows for 18 quintillion possible interface identifiers, you certainly wouldn’t have that many nodes on a LAN. Though IPv6 allows for this theoretical number, we might wonder how does IPv6 practically change the number of nodes on a LAN? In other words, how many IPv6 nodes could you possibly have on a single LAN segment and what is typical or practical? To answer this question, we will consider how many nodes are possible/typical on an IPv4-only network, a dual-protocol network, and an IPv6-only network.
IPv4-Only Network
In the first place, creating a truly IPv4-only network may be difficult because most nodes now support IPv6. IPv6 is enabled by default is most operating systems. IPv6 packets are traversing the LAN network even if there isn’t a first-hop router that is acting as an IPv6 default gateway. To create an IPv4-only network, you would need to disable the IPv6 protocol stack in all the nodes on the LAN. Not only is this difficult to achieve, but it is not recommended by OS manufacturers.
If, in fact, a network only has IPv4 nodes on it then the network may have up to a couple hundred nodes. This isn’t necessarily related to the fact that IPv4 addresses are becoming increasingly scarce. Although, this continues to be true even now that public IPv4 address exhaustion has occurred. The limit of the number of IPv4 nodes on a LAN is more closely tied to the amount of broadcast traffic.
The problem with LAN broadcasts is that every host interface on the LAN gets interrupted and must process the broadcast packet. Every node on the LAN, whether a packet is intended for it or not, must analyze the broadcast and determine if the packet requires processing. IPv4 Address Resolution Protocol (ARP) uses broadcasts, all nodes receive, and check ARP requests, regardless if they are the queried node or not. If the ARP request is for the receiving node, then ARP responses are unicast back to the node that sent the ARP request.
It is typical for IPv4 networks to use a /24 prefix length for LANs. IPv4 can be “chatty” and send many messages out on the LAN as broadcasts. If the IPv4 subnet for the LAN is 192.168.1.0/24, then the subnet broadcast address would be 192.168.1.255 and the destination MAC address of this Ethernet II frame would be ff:ff:ff:ff:ff:ff). Other systems on a LAN may send out packets destined to the IPv4 address 255.255.255.255 (RFC 919) which is a full broadcast, which also has a destination MAC address of ff:ff:ff:ff:ff:ff:ff. Printers send many of these types of packets. It is possible for broadcasts to get compounded and lead to a broadcast storm, which are debilitating for all nodes on the LAN. Other desktop user applications feel the need to send subnet broadcasts as part of their plug-and-play, peer-to-peer, or LAN-based discovery features (e.g. Dropbox LAN sync Discovery Protocol). If you are curious, open up your Wireshark protocol analyzer and set your capture filter to “eth.addr == ff:ff:ff:ff:ff:ff”.
Even if a network has a /22 subnet length assigned to the LAN, due to broadcasts, it might be possible to only have 300 nodes in that LAN. It might not be possible to have an IPv4-only network with 400 to 500 nodes on it before the broadcast traffic becomes overwhelming for all nodes. This becomes a problem for large-scale data center networks and software container environments with a large number of nodes on a single flat network.
Dual-Protocol Network
On most modern networks, there are nodes that are capable of both IPv4 and IPv6 communications. In many cases, the nodes have IPv4 addresses either statically assigned or leased with DHCP, while the IPv6 nodes have only a link-local address. If there isn’t an IPv6-enabled router active on the LAN, then the IPv6 nodes do not receive a global unicast address. However, increasingly, LANs are actively using IPv6 and the end nodes have both an IPv4 and an IPv6 address and operate in dual-protocol mode.
Because IPv4 is still required for reachability to much of the Internet, it is unlikely that you will have an IPv6-only network today. You can add IPv6 to a LAN, but you must keep IPv4 active because most services still use only IPv4. Therefore, dual-protocol operations are preferred, but the network would have all the IPv4 ARP traffic on the LAN, in addition to the IPv6 traffic.
Keep in mind though that IPv6 operates differently than IPv4 on a LAN. As we know, IPv6 does not use broadcast message delivery. IPv6 strives for increased efficiency using only unicast, multicast, or anycast. IPv6 makes extensive use of multicast and there are many well-known IPv6 multicast addresses that are used on LANs. IPv6 also uses ICMPv6 (RFC 4443) to perform the function of Neighbor Discovery Protocol (NDP) (RFC 4861). ICMPv6 has a variety of message types that are used for LAN administration and NDP and we will review several of the popular ones now.
Router Solicitation (RS) and Router Advertisement (RA)
When an IPv6-capable node boots up, it wants to discover the network it has connected to and learn from the local first-hop gateway about the IPv6 address it should use. The IPv6 node sends an ICMPv6 (Type 133) Router Solicitation (RS) message to the all-routers multicast group address FF02:0:0:0:0:0:0:2 (FF02::2). The RS packet only goes to the router(s) listening on the all-routers (FF02::2) multicast address. When the IPv6 node sends these multicast messages out, they are sent as Ethernet II frames destined to the multicast MAC address (33:33:00:00:00:02) (RFC 7042), whereby the last 32-bits of the IPv6 multicast group address are placed into the last 32-bits of the MAC address. Therefore, you are likely to see a lot of “3s” when you perform an IPv6 packet capture (“Understanding IPv6: A Sniffer Full Of 3s”).
When the local first-hop router receives the RS message, it immediately sends a Router Advertisement (RA) ICMPv6 (type 134) message to the all-nodes multicast group address FF02:0:0:0:0:0:0:1 (FF02::1). This will tell the nodes on the LAN about the parameters of the LAN, the local IPv6 prefix, and the method they should use to obtain their IID to form their IPv6 address and complete their connectivity to the LAN. Periodically, the first-hop router also sends the RA messages to the all-nodes (FF02::1) multicast group once every 200 seconds (though this, along with other ND values, is configurable) to keep the nodes updated on any changes that might have occurred or in case an RS was missed. Sending RAs is an essential step in making an IPv6 network operational.
It has already been mentioned that IPv6 does not use broadcast message forwarding model, but the all-nodes multicast group (FF02::1) is the closest thing that IPv6 has to an IPv4 broadcast. RSs are only sent on boot up of the node or on network reconnect, which occur seldom. RAs are typically only send once every 200 seconds or when a RS is received. Therefore, these messages are not an impact on the number of IPv6 nodes on a LAN.
Neighbor Solicitation (NS) and Neighbor Advertisement (NA)
IPv6 networks use the NDP multicast messages to create a neighbor cache binding table between IPv6 addresses and Ethernet MAC addresses. This is the same function as IPv4 ARP performs using broadcasts. When an IPv6 node needs to learn the MAC address for a given IPv6 address, it sends a Neighbor Solicitation (NS) ICMPv6 (type 135) multicast message to the solicited-node multicast address (RFC 2373) for that node. These solicited-node multicast addresses follow the format FF02:0:0:0:0:1:FFXX:XXXX whereby the last 24 bits are carried over from the IPv6 address to be queried. These NS messages are very efficient because they are sent to a very narrow group of nodes, as opposed to IPv4’s broadcast of ARP requests.
Denise Fishburne, Proof of Concept Engineer at Cisco, wrote some great articles titled “Understanding IPv6: What Is Solicited-Node Multicast?”, “Understanding IPv6: Prepping For Solicited-Node Multicast”, “Understanding IPv6: The Ping Before Solicited-Node Multicast” and “Understanding IPv6: Solicited-Node Multicast In Action” which covers this topic completely.
When an IPv6 node on the LAN possesses the IPv6 address being queried, it is listening for IPv6 multicasts sent to its solicited-node multicast address and receives the query. In response to the NS it receives, the node sends back a Neighbor Advertisement (NA) ICMPv6 (type 136) unicast message directly back to the IPv6 node that sent the query. That node can then cache this response in its neighbor cache and use this MAC address for future communications.
These NS messages are typically sent when the neighbor cache times out (approximately every 5 minutes). The NA are typically sent only when a NS query is sent and received. Therefore, NS and NA messages are far more efficient than IPv4 ARP broadcast requests and replies and are orders of magnitude less a limiting factor to the number of nodes possible on a LAN.
Other IPv6 LAN Multicasts
There are a few other IPv6 multicast messages that are present on a LAN but they don’t occur frequently enough to impact the scalability of the LAN. DHCPv6 (RFC 3315) messages are sent out as multicast packets (FF02::1:2, FF05::1:3). It is also possible on an IPv6 network to have a few (though seldom-sent) Unsolicited Neighbor Advertisement which are sent to the all-nodes multicast group address (FF02::1). ICMPv6 Redirects (type 137) can also be sent out when a host sends a packet to a non-optimal gateway, but these are actually unicast. There may also be multicast streaming traffic on a LAN, but the multicast destination address will not be local in scope for such traffic. Rather it will realm-local, admin-local, site-local, organization-local, or global in scope (RFC 7346).
Summary of Dual-Protocol Node Scalability
IPv6 is far more efficient with its use of multicast. None of the above-mentioned IPv6 NDP messages are nearly as “chatty” as IPv4 broadcasts. It comes down to the fact that in a dual-protocol LAN situation, you have both the IPv4 broadcasts and the IPv6 NDP multicasts occurring on the LAN at the same time resulting in the total amount of broadcast/multicast LAN administrative overhead traffic on a LAN. Dual-protocol networks have the sum of the IPv4 broadcasts and the IPv6 multicasts. IPv4 broadcasts are substantial, but IPv6 multicasts are negligible. Therefore, the additive result of the multicast and broadcast messages, mean that the real limiting factor is the number of IPv4 nodes possible on a single LAN segment.
IPv6-Only Network
Today, it is rare to have an IPv6-only network. You could disable IPv4, but it is likely not recommended and difficult to completely deactivate IPv4. Most modern computer operating systems use both IPv4 and IPv6 and both protocols are active on the network interfaces by default. If there is a network that has a first-hop gateway that only has IPv6 configured on it, it would send out IPv6 RA messages, but it wouldn’t have an IPv4 address configured on its interface. If nodes on the LAN still attempted to use IPv4, they would try to send an IPv4 DHCP Discover broadcast message, but they would not get a response. Therefore, the IPv4 enabled nodes would end up with an Automatic Private IP Addressing (APIPA) 169.254.X.X network address. Even with an APIPA address, and IPv4-capable node might still generate a small amount of IPv4 traffic on the LAN.
If the network truly is an IPv6-only network, then it might be possible to have a few thousand nodes on a LAN segment at the same time. This number may also vary based on the frequency of NS-NA messages and that can be controlled by the RA with the NS frequency/duration settings (Router Lifetime, Reachable Time, and Retransmission Timer). An IPv6-only network may be able to have up to ~5000 nodes on it due to the efficiencies of multicast.
As they say, “your mileage may vary” depending on the amount of east-west node-to-node traffic or the amount of north-south node-to-router traffic. If there is more east-west traffic then ~3000 nodes might be possible on an IPv6-only LAN. If there is more north-south traffic then ~6000 nodes might be possible on an IPv6-only LAN. Hyper-scale data centers and cloud environments are using high-density of nodes on a LAN and the use of software containers can definitely benefit from IPv6’s scalability. You might not want to design a network with this many nodes on it, but it is comforting to know that IPv6 is efficient and imposes far fewer limitations on LAN size than IPv4.
