NSX-T HA VIP

Intro

I recently had a customer that I was deploying a multi site configuration for, however they were not comfortable with deploying BGP due to limited knowledge internally. Instead they opted to wait for OSPF to be added to the product however in the interim they still needed a workable solution using static routing.

In steps HA VIP.
HA VIP is a bit of an unknown since there is very limited documentation on it and in fact a colleague of mine flagged it to me as they had seen it in the Kubernetes documentation.

What is HA VIP?

HA VIP stands for “High Availability Virtual IP”
Essentially it allows us to configure an Active-Standby Tier-0 Gateway with a floating IP address to which we can route traffic. A Tier-0 Gateway is operational even if one uplink is down. The physical router interacts with the HA VIP only.

The Tier-0 is configured with an uplink interface for each Edge Node as well as the HA VIP which is bound to those interfaces.
The HA VIP is assigned to the Active Tier-0 uplink interface, the standby Tier-0 instance does not own the HA VIP until it becomes the active node.
The Active Tier-0 is responsible for replying back to any ARP request generated by Top of Rack switch or router resolving data link address (MAC) for the Next Hop IP address (Configured inside the static route to reach NSX-T downlinks configured on the T-0 or T-1).

HA VIP does not have election protocol to elect which T-0 instance becomes the Master (the one that owns the VIP) both the Active and standby instances exchange keepalive messages, in comparison to  VRRP which uses advertisement messages to know the state to action any failover.
Hence in HA VIP the Master role is always provided by the active T-0 instance and depends on NSX-T detecting a failure to failover to the standby instance.

There are some prerequisites that need to be taken into consideration before we deploy a HA VIP.

Prerequisites

• The Tier-0 Gateway must be configured in Active-Standby mode.
• Edges should have a single Uplink Interface.
• The HA VIP subnet must be the same as the uplink interfaces that it is bound to.
• HA VIP is intended to be used with static routing.

It’s a good idea to pair HA VIP with VRRP (Virtual Router Redundancy Protocol) on the physical network this way we have a single static route that we can direct our Egress traffic but with VRRP we have redundancy on the physical network as if one TOR fails the VRRP will move to the other TOR but the NSX-T route will not need to be changed.
A good blog to read that compares these two technologies is https://www.livefire.solutions/nsx-t/nsx-t-ha-vip-and-vrrp-are-same-or-different/

Design

The diagram below shows how HA VIP will look and an example static route to set on the T0 and the physical network.

From a vSphere and NSX-T connectivity perspective the below diagram illustrates how the vDS and the Edge N-VDS will look.
Starting with the vDS we are using dedicated physical NICs, there are three port groups the Management, Overlay and External Trunk.
The management port group does not have to live on the Edge vDS it can happily be connected to the Management vDS.
The Overlay trunk and the External trunk are essentially the same they both trunk all VLANs. This is standard practise for the Edge port groups as it adds simplicity and tagging is done at the NSX-T layer.
The key thing to note here is the teaming policy for those port groups.

Looking at the Overlay trunk portgroup you can see that Uplink 1 is active and Uplink 2 is standby.

Likewise the Uplink trunk is the opposite with Uplink 2 active and Uplink 1 standby.
This configuration gives us a dedicated NIC for each traffic type while allowing us to have redundancy in the physical NICs. Should a physical uplink fail the port group will simply switch over to the other physical NIC thus even though the Edge node has a single uplink in reality we have two for redundancy at the vDS level.
We don’t need the redundancy at the Edge node level as it is very unlikely that the Edge uplink will fail and even if it does the T0 is configured in Active/Standby mode so a failure would simply cause the T0 to switch to the other Edge node.

The Build

I won’t run through the step by steps of deploying Edges and T-0’s here as that’s covered in my lab build guides but I will cover the differences with this configuration.

Uplink Profile

First off the uplink profile.
As this build requires the Edges to have a single uplink we need to define this in the Uplink profile. The Default Teaming is set to Failover order with E-UP01 as the Active uplink and no Standby uplinks (the name can be anything you like)
We then have a named Teaming in my case called DCA-Edge-Uplink1 again Failover Order this time with E-UP02 as Active and no Standby.

Edge Node configuation

The edge nodes are configured with a single N-VDS.
They are part of the Overlay Transport Zone and the Edge VLAN Transport Zone. You can use the default VLAN TZ but I prefer to use an Edge VLAN TZ so the Uplinks are not available for normal VM’s to connect to.
The Uplink Profile is the one we configured in the previous step.
For the TEP IP I am using an IP Pool this pool is the same as the Hosts TEP Pool as I have dedicated NICs for the Edges.

Finally I have assigned the DCA-Overlay-Trunk Port Group to E-UP01 and the DCA-Uplink-Trunk Port Group to E-UP02

Uplink Segment

Next up is the Uplink Segment, key points here are that the Segment is not Connected to a Gateway, the Transport Zone is the Edge VLAN TZ, the uplink VLAN is tagged on the segment and the Uplink Teaming Policy is set to DCA-Edge-Uplink1 remember that was set it the Edge uplink profile.

Tier 0

Once you have those settings configured you can deploy the Tier-0 Gateway for the HA VIP configuration we only need one uplink for the Tier-0 as there are two edge nodes in the cluster that means two uplinks, one per Edge node.

with the uplinks configured we can setup the HA VIP, under ‘HA VIP Configuration’ click Set

Click ADD HA VIP CONFIGURATION

Enter the VIP in CIDR format then click the ‘Interface’ box and select the two interfaces, you must select two interfaces no more no less.

Click ADD

Finally click APPLY

The final step is to configure the static route. Expand ‘ROUTING’ and under ‘Static Routes’ click Set

Click ADD STATIC ROUTE

Enter a ‘Name’ and the route in CIDR format, I am using a default route all 0’s next click Set Next Hops

Click ADD NEXT HOP

Enter the next hop IP this would be the gateway on the physical network such as the VRRP IP for the Uplink VLAN, adjust the admin distance if you need to and don’t set the Scope then click ADD and then APPLY

Finally click SAVE and then CLOSE then Close Editing on the T0

Thats it HA VIP is now configured giving us redundancy with a static route and no need for additional detection failure mechanisms.