Intro
Welcome to Part 13 of the NSX-V Lab Series. In the previous post, we created our Logical Switches.
In this post we will configure our Distributed Logical Router and connect our test App Logical Switches to it.
What is a Distributed Logical Router?
A distributed logical router (DLR) is a virtual appliance that contains the routing control plane, while distributing the data plane in kernel modules to each hypervisor host. The DLR control plane function relies on the NSX Controller cluster to push routing updates to the kernel modules.
Logical router kernel modules in the host perform routing between VXLAN networks, and between virtual and physical networks. An NSX Edge Appliance provides dynamic routing ability if needed. Logical routers can be created on both primary and secondary NSX Managers in a cross-vCenter NSX environment, but universal logical routers can be created and managed only on the primary NSX Manager.
Essentially it’s a router thats distributed 😉 across all your hosts.
A key benefit here is for example you have two VM’s on a host each in a different subnet. Without the DLR in order for the two VM’s to communicate traffic would need to flow up to the physical network be routed and then sent back down to the same host to reach the second VM, this is known as ‘Hairpinning’ With NSX and the DLR the traffic from one subnet to the other is routed locally on the host so the traffic never leaves the host and is processed in the kernel. This reduces the amount of traffic on the physical network and drastically increases the flow speed.
The Build
We deploy DLR’s and UDLR’s from the same section as we do NSX Edges, from the ‘+ ADD’ drop down menu select ‘Distributed Logical Router’ for a global i.e. site local DLR or as in our case select ‘Universal Distributed Logical Router’ to create a multi site DLR.
Give it a name and ensure that ‘Deploy Control VMs’ and ‘High Availability’ are enabled. I won’t use local egress for now I’ve only used this twice in customer deployments in the last 3 years as it adds extra complexity for ingress traffic control and 90% of the time the customer uses the secondary site as a passive site only or just has the VM traffic egress and ingress via the primary site so local egress is not a common configuration, I will probably cover it later in a blog post but it’s a rather large subject so not for a build guide.
We need to deploy a control VM as we will be setting up Dynamic routing and the control VM is required for that.
We also deploy the DLR in HA mode. For the lab this is not strictly needed however in all my customer deployments the DLR has been built in HA mode so for a lab we will stick to using it.
Set a complex password and for the lab I’ll turn on SSH access.
Because we selected to deploy a control VM we now need to configure it.
The control VM is an Edge Appliance (well sort of).
Ensure the correct data center is selected and click the + button.
We will select the Management / HA Interface after the Edge Appliance configuration.
Selected the Cluster or resource pool. Placement of the DLR control VM depends on several factors including whether you have a collapsed environment i.e. edge and compute on the same cluster, number of edge or compute nodes, number of ECMP Edges and also personal preference.
Placing the control VM on the Edge cluster is a valid option as the Edge clusters are often not fully utilized from a compute perspective, in contrast Compute clusters are normally heavily utilised.
However when deploying the DLR control VM we need to ensure that it does not reside on a host that also has the ECMP Edge which is connected to the same DLR.
The reason for this is that if a host with the active DLR control VM and an ECMP Edge VM dies the traffic to that edge will be black holed. In other words the hosts will continue to send traffic to the dead edge until the DLR control VM fails over to the passive node. ESXi host VMkernel routes are not updated until the DLR control VM functionality is restored on the passive VM as such the traffic will still be send to the dead Edge.
The default HA timer on the DLR is 15 seconds, this can be reduced but the best option is to avoid black holing the traffic by placing the control VM on different hosts.
We do this with DRS anti-affinity rules.
If the Edge cluster does not have enough hosts to keep the ECMP Edges and the DLR control VMs separate then the DLR control VM should be placed on the compute cluster.
In my lab I’m placing the Edges and DLR control VM’s on the same three hosts so I’m ignoring this rule but hey it’s a lab so it’s not an issue I could add another host to get around this.
If this was a production system then I would technically need a fourth host to run just two ECMP Edges and the active and passive DLR control VM.
The reason is that the DLR control VMs should be separated as there is no point in them both being on the same host as a host failure would kill the pair of them which is what we are trying to avoid in the first place.
Likewise we also need to ensure the active DLR control VM is not on a host with the Edge.
Since we need to make sure that the active control VM is separate we have to have the DRS rule keep all four VMs separate, if we were to reboot the active control VM the passive one will become active and so would need to be on a host away from the Edge’s.
Dependant on the number of hosts, number of Edge’s and DLR’s etc the design can become quite complex.
This is often why the DLR control VMs are just placed on the compute cluster as it simplifies the DRS rules and VM placement.
Anyway pick your desired cluster.
Select the storage.
For a production deployment we would now click on ‘ADD’
The system will reserve the CPU and memory.
For my lab I don’t want this reservation.
So I select ‘No Reservation’
We now need to select the Management / HA Interface and for this I will use my HA-VXLAN logical switch.
The next step is to configure the DLR interfaces, you can skip this and do it later but since we are here we may as well do it now. Click the ‘+ ADD’
Give the interface a name. I always set the first interface as the uplink so ensure the radial button for Uplink is selected.
For the ‘Connected To’ Network select the UDLR-ESG-Transit network.
Give it an IP and subnet prefix. and ensure that the MTU is set to your desired value, by default these will always be 1500. For consistancy it’s good practise to set all the MTU’s to the same value for me that’s 1600.
I’ll now add the rest of the interfaces.
First off the Web-LS.
Ensure that the Type is set to internal.
Note the MTU is 1500 by default.
Click the Pencil on the ‘Connected To’ section.
Pick the Web-LS Logical Switch and click ‘OK’
Set the MTU to the correct value and click ‘OK’
Repeat the process for the remaining Logical Switches.
Set the default gateway to disabled. Since we have multiple Edges connected to the DLR we don’t want to force all the traffic to use one default route.
Click on Finish
The DLR will now be deployed.
If we check the cluster where we deployed the DLR control VM we can see the edge appliance pair.
The system will automatically generate the DRS anti-affinity rule to keep the control VM’s separate. If we have ECMP Edges deployed that are connected to this DLR then we would need to add them to this rule as well.
Since I only have three hosts I’m not going to do this step.
A quick final test we can now ping the App and DB VM’s from the Web VM.
That’s it for the DLR. The next step is to deploy our ECMP Edges.
NSX-V Lab Part:14 NSX-V Edge Services Gateways