Multi-PoD網(wǎng)絡(luò)架構(gòu)設(shè)計指南

1、Multi-PoD 網(wǎng)絡(luò)架構(gòu)設(shè)計指南（配圖）技術(shù)創(chuàng)新，變革未來1.1.1-圖1IP網(wǎng)絡(luò)集群節(jié)點1二層DCI網(wǎng)絡(luò)心跳信息私網(wǎng)IP1公網(wǎng)IP n 客戶端儲存網(wǎng)絡(luò)存儲資源存儲資源業(yè)務(wù)訪問VIP主用數(shù)據(jù)中心A主用數(shù)據(jù)中心B VIP集群節(jié)點n集群節(jié)點2集群節(jié)點3公網(wǎng)IP2公網(wǎng)IP3私網(wǎng)IP2私網(wǎng)IP3私網(wǎng)IPn公網(wǎng)IP1IP網(wǎng)絡(luò)二層DCI網(wǎng)絡(luò)客戶端存儲資源存儲資源L3網(wǎng)關(guān)主用數(shù)據(jù)中心A主用數(shù)據(jù)中心BVM2HypervisorHypervisor 存儲網(wǎng)絡(luò)VM1VMIP1VMIP2VMIP3L3網(wǎng)關(guān)數(shù)據(jù)中心網(wǎng)絡(luò)數(shù)據(jù)中心網(wǎng)絡(luò)VM1VMIP1VM3就近訪問（可選）1.1.2圖1二層網(wǎng)絡(luò)業(yè)務(wù)集群IPA n主用數(shù)

2、據(jù)中心A主用數(shù)據(jù)中心B IPA2IPA3IPA1業(yè)務(wù)主網(wǎng)關(guān)業(yè)務(wù)備網(wǎng)關(guān)FW主FW備網(wǎng)絡(luò)容災(zāi)，主備主路由VIPVIP備路由1.1.3圖1Fabric 1Fabric nSDN控制器 VXLANBGP EVPN互聯(lián)IP networkLeafLeafSDN控制器（備） 業(yè)務(wù)控制層基礎(chǔ)設(shè)施層轉(zhuǎn)發(fā)實現(xiàn)層VMMServer PoolVMMServer PoolDC 1DC n1.2.1圖1業(yè)務(wù)層基礎(chǔ)設(shè)施層VPC nRouterVPC 1RouterSubnetSubnetSubnetVPC 2RouterSubnetSubnetVPC ConnectionVPC ConnectionOrchestrato

3、rFabric nFabric 1Fabric 1Fabric nDC 1數(shù)據(jù)中心交換核心數(shù)據(jù)中心交換核心DC nService Network Hardware Network Network Infrastructure ManagerOrchestrator布放編排配置映射1.2.1圖2iMaster NCE1iMaster NCEn端到端VXLANFabric 1Fabric 2SpineLeafSpineLeafACAC(Active)(Standby)IP互聯(lián)網(wǎng)絡(luò)Border LeafBorder LeafFabric-GWFabric-GW主出口備出口業(yè)務(wù)VPC1.2.1圖3Bo

4、rder LeafPoD nVTEPVTEPServerSpineLB FWVTEPService LeafServer Leaf1.2.2圖1Border LeafPoD 1VTEPVTEPServerSpineLB FWVTEPService LeafServer LeafCore switchDC核心1.2.2圖2InternetIP networkBorder LeafDC1VTEPVTEPServerSpineLB FWVTEPService LeafServer LeafBorder LeafDC2VTEPVTEPServerSpineLB FWVTEPService LeafSe

5、rver LeafInternet主備容災(zāi)大二層DC 2DC 1AC主AC備VXLAN主備出口分布式路由大二層管理控制層數(shù)據(jù)承載層仲裁服務(wù)VPC 1243Domain 1VPC 211.2.3圖1Fabric nFabric 1Fabric 1Fabric nDC 1數(shù)據(jù)中心交換核心數(shù)據(jù)中心交換核心DC n控制器 n控制器 1編排器云平臺 N（可選）云平臺 1（可選）DCIn1.2.4圖1iMaster NCE1iMaster NCEFabric 1Fabric nPOD 1POD 2控制器跨PoD互聯(lián)層云平臺 （可選）1.2.4圖2iMaster NCEPEBorder Leaf業(yè)務(wù)分區(qū)VT

6、EPVTEP管理節(jié)點仲裁服務(wù)器PEServer Leaf業(yè)務(wù)分區(qū)VTEPVTEP管理節(jié)點管理接入交換機(jī)管理核心交換機(jī)Border LeafServer管理接入交換機(jī)Server管理分區(qū)管理分區(qū)DC 1仲裁DCFWFW仲裁核心交換機(jī)SpineSpine管理核心交換機(jī)DC 2存儲分區(qū)存儲分區(qū)存儲核心交換機(jī)存儲核心交換機(jī)存儲LB FWVTEPLB FWVTEPService LeafService LeafServer Leaf2.1圖1Internet2.1 圖2IP networkBorder LeafMain DC 1VTEPVTEPServerSpineLB FWVTEPService Le

7、afServer LeafRemote PoDBorder LeafMain DC 2VTEPVTEPServerSpineLB FWVTEPService LeafServer LeafRemote PoDVTEPServerSpineServer LeafVTEPServerSpineServer LeafActiveStandbyFabric 1Fabric 2iMaster NCEiMaster NCE2.2.1Border LeafPoDServerSpineLB FWService LeafServer LeafServerServer LeafExternal network2.

8、2.22.2.3圖12.2.3圖22.2.4Border Leaf業(yè)務(wù)分區(qū)VTEPVTEPServerSpineLB FWVTEPService LeafServer Leaf1234567Border LeafDC2VTEPVTEPServerSpineLB FWVTEPService LeafServer Leaf2.3圖1InternetBorder LeafDC1VTEPVTEPServerSpineLB FWVTEPService LeafServer LeafCore switchOption 1Option 2MPLS VPNOption 3Bare optical fiber/

9、DWDMIP networkWANInternet2.3圖2MPLS L3 VPNVXLAN VXLANIGP/BGPIGP/BGPMP-IBGPMPLS L3 VPN傳遞的Underlay路由InternetIP NetworkBorder LeafDC1VTEPVTEPServerSpineLB FWVTEPService LeafServer LeafVXLANIGP/BGPIGP/BGPIGP/BGPVRF-Lite傳遞的Underlay路由Option 1Option 2Border LeafDC2VTEPVTEPServerSpineLB FWVTEPService LeafSer

10、ver LeafInternet2.3圖3Outer Ethernet headerOuter IP headerVXLAN headerInner Ethernet headerInner IP headerPayloadUDP headerVXLAN封裝格式Dest MACSrc MACTPIDVLAN tagLength/Type66(2)(2)214(18)BytesIP Misc DataProtocolChecksumSrc IPDest IP9124420BytesSrc PortDest PortUDP LengthChecksum22228BytesVXLAN FlagsRe

11、servedVNIReserved13318Bytes50(54)Bytes overheadOriginal frame2.4圖1IP networkMain DC 1VTEPVTEPVTEPServerLB FWVTEPRemote PoDMain DC 2VTEPVTEPLB FWVTEPVTEPRemote PoD主用AC控制器納管所有POD的交換機(jī)兩套AC形成主備關(guān)系，支持主備倒換ActiveStandbyiMaster NCEiMaster NCE管理節(jié)點仲裁服務(wù)器管理節(jié)點管理接入交換機(jī)管理核心交換機(jī)管理接入交換機(jī)管理區(qū)管理區(qū)DC 1仲裁DCFWFW仲裁核心交換機(jī)管理核心交換機(jī)DC

12、 2三層網(wǎng)關(guān)三層網(wǎng)關(guān)二層接入二層接入三層網(wǎng)關(guān)2.4圖2Site: QuorumDC 1DC 2AC主AC備數(shù)據(jù)同步仲裁仲裁仲裁2.4圖3VXLANVXLANDC1DC2SpineLeafSpineLeafACAC(Active)(Standby)Server Pool云平臺2.5圖1VXLANVXLANDC1DC2SpineLeafSpineLeafACAC管理平臺(Active)(Standby)VMMVMM網(wǎng)絡(luò)虛擬化2.5圖2Internet2.6圖1IP NetworkBorder LeafMain DC 1VTEPVTEPRemote LeafVTEPServerServerSpine

13、LB FWVTEPService LeafServer LeafRemote PoDBorder LeafMain DC 2VTEPVTEPServerSpineLB FWVTEPService LeafServer LeafRemote LeafVTEPServerRemote PoDActiveStandbyOSPFOSPF發(fā)布Remote Leaf Anycast VTEP IP Management IPOSPFOSPFMPLS VPN, VRF-Lite打通Main DC與Remote POD之間的Underlay，保證各站點的Anycast VTEP IP，設(shè)備Management

14、 IP，AC controller IP可達(dá)發(fā)布Remote Leaf Anycast VTEP IP Management IP發(fā)布Main DC Leaf Anycast VTEP IP和各設(shè)備Management IP，發(fā)布AC Controller IPiMaster NCEiMaster NCEPEPEBorder Leaf業(yè)務(wù)分區(qū)VTEPVTEPServer Leaf業(yè)務(wù)分區(qū)VTEPVTEPBorder LeafServerServerSpineSpineLB FWVTEPLB FWVTEPService LeafService LeafServer LeafOSPFArea1OS

15、PFArea2OSPFArea02.6圖2Internet2.7.1圖1IP networkMain DC 1VTEPVTEPVTEPServerLB FWVTEPRemote PoDMain DC 2VTEPVTEPLB FWVTEPVTEPRemote PoDActiveStandbyeBGPeBGPStaticBGP EVPNBGP RRBGP clientBGP clientBGP clientBGP clientBGP RRBGP clientBGP clientStaticBGP clientBGP clientMP-BGP session between remote Leaf

16、nodes and RR on Main DC SpineMP-BGP session between Main DC leaf nodes and RR on Main DC SpineMP-BGP session between Main DC leaf nodes and RR on Main DC spine iMaster NCEiMaster NCE2.7.2圖1IP networkMain DC 1VTEPVTEPVTEPServerLB FWVTEPActive DC 2VTEPVTEPLB FWVTEPVTEPRemote PoD 1StaticBGP EVPNStaticV

17、M AMAC_A / IP_A:10.1.1.11VMBMAC_B / IP_B:10.1.1.14MACIPVNINext Hop: VTEPMAC_B10.1.1.14910011.4.4.9MAC_C10.1.1.12910011.2.2.9MACIPVNINext Hop: VTEPMAC_A10.1.1.11910011.1.1.9MAC_C10.1.1.12910011.2.2.9VM CMAC_C / IP_C:10.1.1.12MACIPVNINext Hop: VTEPMAC_A10.1.1.11910011.1.1.9MAC_B10.1.1.14910011.4.4.9Re

18、mote PoD 22.7.3圖1VPCFWDC 1公網(wǎng)GWVMVM主備公網(wǎng)VMVMDC 2FWGW路由直通路由直通DVRDVRInternet2.7.3圖2IP networkMain DC 1VTEPVTEPVTEPServerLB FWVTEPRemote PoDMain DC 2VTEPVTEPLB FWVTEPVTEPRemote PoDActiveStandby主用出口，高優(yōu)先級默認(rèn)路由引入BGP-EVPNBorder LeafBorder LeafVTEP1基于BGP-EVPN路由，優(yōu)選主用出口基于BGP-EVPN路由，優(yōu)選主用出口基于BGP-EVPN路由，優(yōu)選主用出口VRF IP PrefixNext HopOutIF10.0.0.0/0InternetPEBDif1VRF IP PrefixL3VNIRouter MACNext Hop10.0.0.0/010000VTEP1 MACVTEP1VRF IP PrefixL3VNIRouter MACNext Hop10.0.0.0/010000VTEP1 MACVTEP1VTEP2備用出口，低優(yōu)先級默認(rèn)路由引入BGP-EVPN，不優(yōu)選iMaster NCEiMaster NCEPEPEBorder LeafDC 1VTEPVTEPServer LeafDC