A while ago I wrote a couple of articles on HCX, VMware’s solution for workload mobility across VMware stacks. I also recorded a couple of lightboards on HCX together with Jeffrey Kusters. Next week I will be talking again about HCX within the context of VMware Cloud on AWS at the Virtual NLVMUG Usercon. The session is titled “Realize a hybrid cloud with VMware Cloud on AWS and HCX”.
In this article we will have a closer look at Mobility Optimized Networking, one of the enhanced new HCX features now available in VMware Cloud on AWS and also one of the things I would like to discuss in my NLVMUG session. With Mobility Optimized Networking (MON) you can optimize the routing for the VMs that are running at the cloud side after you’ve stretched a L2 network segment using the HCX Network Extension (HCX-NE) capability.
Without MON the routing for a L2 extended segment is done at the source site, this results in tromboning at the source site and adds extra latency specifically in case where network traffic for a VM at the cloud site has to be routed to another VM that is also running at the cloud site (in a different network). After MON is enabled routing of extended networks to other extended networks and/or cloud networks can be performed locally using the local NSX-T T1 router.
Mobility Optimized Networking versus Proximity Routing
At this point you might think…Mobility Optimized Networking? I remember the Proximity Routing option that was available in the past. True, but Proximity Routing is a NSX-V based routing optimization solution, while MON is based on NSX-T 3.0+. On top of that Proximity Routing had some additional limitations: it requires some manual configuration, there’s no VMC support, can only be enabled when the initial stretched network is created and requires a network switchover on reboot. All these issues are resolved with MON: optimized routing is configured in an automated way, MON is supported on VMConAWS, you can toggle on/off MON for new and existing networks and the gateway switch-over is available on-demand.
How it works
So how is Mobility Optimized Networking working? Gabe Rosas over at hcx.design published some excellent networking diagrams that clearly explains MON architecture and network flows. The original Visio and OmniGraffle files are also available for download.
The following diagram (coming from Gabe’s website hcx.design) shows a MON enabled HCX setup:
Before I dive into the details, it’s important to understand that the main use case for MON is to reduce the tromboning effect in case you have more than one extended network, but also in a scenario where you have an extended network to connects to local or to remote (cloud) networks (within the scope of the vSphere/SDDC environments).
In a regular HCX-NE scenario no local router at the cloud side (the right side of the diagram) is used. The Tier-1 NSX-T router stays disconnected when using HCX-NE without MON. With MON enabled the NSX-T Tier-1 router is connected, in the diagram through the 10.5.0.1/32 address, to perform the local routing tasks.
The way traffic flows is best explained on the basis of the forwarding preference. In case of MON enabled, networking flows are:
- L2 over HCX bridge – If VM B (10.5.0.3/24) wants to connect to VM A (10.5.0.2/24), this traffic is directly flowing through the NCX-NE appliances.
- Tier 1 connected/static routes – If VM B (10.5.0.3/24) wants to connect to VM D (10.7.0.3/24) traffic is routed locally through the Tier 1 router (because MON is enabled). With MON disabled this traffic would be routed at the source (left) side and from there following the regular routing path.
- HCX Policy route – You can also set a HCX Policy route, this means you can configure certain network to always be routed through the router at the source site. An example use-case is you want the traffic to flow through some kind of security device at the source site.
- Send traffic to Tier0 at the cloud site – In case of remaining (internet) traffic you can choose to have breakout at the cloud side. In this case a NAT configuration is required because inbound traffic at cloud site is not supported with MON.
Notice that MON has no impact on the routing between two networks that are not extended, these networks are routed through regular routing constructs/paths.
I hope this was useful and provides you some more insight on how Mobility Optimized Networking works. More information is available in the HCX documentation. I also want to thank Tom Zukowski for the excellent content that helped me to write this article.