NSX Design Notes (2/10) – Logical Switching

1) Segment Id:

  1. Logical switching is defined by segment Id (VXLAN id).
  2. Segment ID planning is required for cross-vc connectivity

2) Replication Modes

Replication modes for multi destination traffic.

  1. Multicast mode :
    1. NSX leverages L2/L3 capability of physical network
    2. Configure IGMP snooping on physical switches to deliver L2 multicast traffic
    3. Configure PIM and enable L3 multicast routing
    4. Replication in Multicast mode

    5. A multicast IP address must be assigned to each defined VXLAN segment
    6. Consider Mapping VXLAN segments and Multicast groups –
      1. VXLAN : Multicast group -> 1:1
      2. VXLAN : Multicast group -> m:1
      3. VXLAN : Multicast group -> m:n ( common strategy)
  2. Unicast mode :
    1. Does not require explicit configuration on physical network
    2. Best for small and medium environments where BUM traffic is not high and all VTEPs are in same L2 Domain (subnet)
    3. This mode scales well with L3 topologies where UTEP boundaries are clearly defined ( each L3 rack has its own subnet)
    4. VTEP segments and UTEP role
      1. ESX Groups: ESX hosts in the NSX domain are divided into separate groups (VTEP Segments) based on IP subnet of VTEP interfaces. Note: Don’t confuse VTEP segment with VXLAN segment
      2. UTEP role: An ESX host in each group (VTEP segment) is selected to play role of Unicast tunnel end point role. UTEP receives BUM traffic from ESXi hosts in other VTEP segments and replicates to all hosts in its segment.
      3. Multi destination traffic in unicast mode

  3. Hybrid mode :
    1. Leverages L2 Multicasting capability of physical network. Does not require physical network L3 multicasting.
    2. Addresses the requirement of BUM replication in large scale design regardless of underlying topology.
    3. Mix of both unicast and multicast mode

    4. Recommendation: Since PIM is not required, it is strongly recommended to define an IGMP querier per VLAN to ensure successful L2 multicast delivery and avoid non deterministic behavior.

3) Populating Controller Tables

Controller table handles information essential for L2 Unicast communication

  1. VTEP table
    Controller node sends a VNI-VTEP report message to all ESXi servers hosting VMs actively connected on the VXLAN segment. ESXi servers are aware of other VTEPs in VXLANs.

  2. MAC table
    Controller node does not send a VNI-MAC report to ESXi servers. ESXi servers are aware of only local connected MAC addresses.

  3. ARP Table
    Controller populates ARP table to perform ARP suppression

    1. Learning Dynamic IP: ESXi host snoops DHCP response from DHCP server.
    2. Learning Static IP : Learns from ARP requests originated by VM

4) Unicast traffic (Virtual to Virtual Communication)

  1. ARP resolution:
    1. ESXi intercepts the ARP request and sends ARP response.
    2. ESXi first queries control node , if mapping information (IP-MAC) is available in controller
    3. If mapping information is not available then ESXi would broadcast the ARP request frame ( depends on VXLAN replication mode)

  2. Data traffic
    1. If VM is residing in another host then ESX host encapsulates the original packet in vxlan packet and sends to destination host VTEP.

5) Unicast Traffic (Virtual to Physical Communication)

  1. Circumstances
    1. Deployment of multi-tier application – one or more tiers residing in physical network
    2. P2V Migration – During an ongoing migration project
    3. Leverage external physical devices as Default gateway for VMs connected to logical switch
    4. Deployment of physical appliances – firewalls, load balancers etc.
  2. NSX L2 bridging allows VMs connected at Layer 2 to physical network through VXLAN-VLAN ID mapping
  3. VXLAN-VLAN Bridging
    1. Supported even on ESXi servers not physically connected to the L2 Physical Network (VLAN)
    2. Part of DLR Distributed router configuration ( DLR Control VM)
    3. Bridge instance is always active on single ESXi host
    4. Layer 2 bridging data path is entirely preformed in ESXi kernel space. Control VM is to determine on which ESXi host bridge instance is active
    5. DLR control VM runs on active-standby mode. If active node fails convergence of bridging traffic is governed by heart-beat timer
    6. VXLAN:VLAN mapping is always 1:1

  4. ARP resolution
    1. Bridge instance forwards the ARP broadcast packet into the physical network.
    2. Physical server generates unicast ARP reply to VM


  5. Data traffic: Same as unicast virtual to virtual L2 communication.

Source: http://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/products/nsx/vmw-nsx-network-virtualization-design-guide.pdf

NSX Design Notes (1/10) - Functional Components
NSX Design Notes (3/10) - Logical Routing
No tags for this post.

Leave a Comment