VMWorld 2017 – NET1535BE – VMware NSX Design: Reference Design for SDDC with NSX and vSphere: Part 1
- NSX Manager: DB Schema is specific to version – Restore works only with like-to-like version
- Storage resiliency is required in terms of access, paths, disks and LUNs.
- Consider IO Oversubscription (Frequent writes and Random Reads.)
VDS & Transport Zone:
1 VDS for Compute Cluster and 1 VDS for Edge cluster -Recommended Configuration
- Flexibility of choosing NIC teaming mode
- Edge VLANs (talks to physical hardware) need not configure for compute cluster. (Restrict vlan proliferation).
- SPAN, IPFIX, Packet Capture configuration flexibility
- Transport zones span Edge cluster and compute cluster.
- Transport zone is provisioning and management boundary – not security and data boundary
- 1600 required. Recommended 9000 (future proof –for high throughput traffic, 8900 MTU on Storage VM )
VXLAN VLAN Id – Consistent across transport zone
- L2 design – vlan and subnet spans across racks
- L3 design –subnets are specific to rack, but vlan spans across racks
VDS Uplink design
- Source port id based : Recommended
LACP teaming mode: Discouraged
- LACP NSX edge gateway has supportability issues ( required from hardware switch vendors)
- LACP can be used in compute cluster
- Brownfield: if we cannot , yes proceed
# of VTEPs:
- if VXLAN throughput requirement is more than 10 Gbps, then you need more than 1 VTEP
- if deterministic traffic mapping to uplink is desired ( Explicit failover only)
- L2 Fabric: Single Subnet. Reserve IP address for future growth: /22 recommended
- L3 Fabric: Multiple subnets (one per rack) for L3 fabrics.
Edge Cluster Design
- Consider Rack availability
Active – Standby
- Heartbeat: 9 seconds. L2 connectivity required between active and standby
- Protocol timers: hello/hold (40/120). If we don’t tune timers adjacency will be lost on ToR switches and induces another failure.
- No Heartbeat.
- Protocol timers: hello/hold (1/3) 4 second convergence
- Sub second convergence not possible for real time traffic (VoIP, Video Conference, etc.)
- Bidirectional Forward Detection BFD not yet available.
- DLR Control VM and NSX controllers are not in data path
ECMP with DLR and Edge
- Don’t put ECMP NSX Edge VMs and DLR Control VM on same host.
- Consider graceful restart design options.
Graceful Restart Guidance
- Active-Standby: Enable Graceful restart.
ECMP: Disable Graceful restart
- ToR has single control plane ( no dual hypervisor) : Disable graceful restart
- ToR has dual hypervisor: Choice -> Disable @ physical router or @ ESG. Recommended -> Disable @ ESG.
NSX edge routing design with Rack Mount Server
- Edge uplink = Host uplink = VLAN = Adjacency
Routing Protocol & Topology
NSX domain act as a stub network
- Send default route to NSX edges. NSX edges will send summarized routes.
- For OSPF it is a Stub Area
- Use consistent one protocol end-to-end between ESG-Physical and ESG-DLR
- Recommended: BGP
- Multi Tenancy is difficult
- Multi Tenancy is possible.
NSX connectivity with BGP
- Advertise summarized routes from NSX domain to Physical
- Advertise default route to NSX domain.
- Recommended: Run EBGP from Physical -> Edge->Control VM
VMWorld 2017 – NET1536BE – VMware NSX Design: Reference Design for SDDC with NSX and vSphere: Part 2
DC Design consideration – Compute Cluster
- Rack based vs Multi rack (horizontal) striping.
- Use case: Single rack design, Only Micro segmentation.
- VM mobility is within a rack – no need of VXLAN, DLR
- Centralized Edge: Active – Standby Edge. ( use stateful resources)
- Separate Edge cluster and computer cluster ( Medium & Large)
- You can combine Management & Edge Cluster: When you grow Edge can be separated from Management
- Don’t combine Edge Cluster & Compute cluster: Compute cluster may grow (type of hardware may change, Operational boundary, VLAN prowl)
- Edge Cluster: Minimum 3 hosts: ECMP Edge1 (Host1), ECMP Edge 2 (Host2), DLR Control Active VM ( Host3)
For Cross-VC and SRM Deployments: Separation of Management cluster is inevitable.
- Dedicated Edge cluster
- Minimum four hosts: ECMP Edge 1-2 (Host1), ECMP Edge 3-4 (Host2), DLR Control VM Active ( Host3), DLR Control VM Standby (Host4)
- (Optional): NSX Controllers can be hosted on Edge cluster for optimizing Edge host utilization.
NIC Card performance
- Core limits
For higher throughput
- Higher MTU
- TSO, LRO & RSS enabled cards
- Disable CPU power saving mode
- Disable Hyper threading on host
Edge cluster design: Oversubscription
VMWorld 2017 – NET1775BU – Advanced VMware NSX: Demystifying the VTEP, MAC, and ARP Tables
- NSX Controller tables
- Controller Disconnected Operation (CDO) Mode : CDO Logical switch is used, when Controllers are not available, by all hosts for BUM traffic
No tags for this post.