openstack使用linux bridge+vxlan模式的網(wǎng)絡變化與分析.doc

目前網(wǎng)絡上大部分講解openstack的網(wǎng)絡模塊neutron的方式都是基于ovs進行講解的，很少有涉及l(fā)inux bridge的，由于本人初次接觸openstack的版本是ocata，而官方文檔的部署模式是基于linux bridge的，網(wǎng)上的資料較少，本文按照自己的思路進行描述，可能存在不對的地方。、先說創(chuàng)建過程：1. 創(chuàng)建外部網(wǎng)絡provider。2. 創(chuàng)建外部網(wǎng)絡provider的子網(wǎng)，創(chuàng)建過程中創(chuàng)建網(wǎng)橋brq-4c,同時創(chuàng)建dhcp命名空間，并且在網(wǎng)橋上創(chuàng)建端口tap-f3與DHCP服務連接，同時因為provider網(wǎng)絡使用的網(wǎng)橋是有eth0提供，所以eth0轉(zhuǎn)為網(wǎng)橋brq-4c的接口。IP地址轉(zhuǎn)移為網(wǎng)橋的地址。3. 創(chuàng)建私網(wǎng)網(wǎng)絡selfservice4. 創(chuàng)建私網(wǎng)網(wǎng)絡selfservice的子網(wǎng)，創(chuàng)建過程中創(chuàng)建網(wǎng)橋brq-9d，同時創(chuàng)建DHCP命名空間，并且在網(wǎng)橋上創(chuàng)建端口tap-35與DCH服務連接，改網(wǎng)橋的vxlan的id為48，隧道為vxlan-485. 創(chuàng)建路由器router6. 將私網(wǎng)網(wǎng)絡selfservice與路由器router連接，這個時候創(chuàng)建路由器router命名空間，同時在私網(wǎng)網(wǎng)橋brq-9d上新增一個端口tap-8f，同時可以通過路由器router查看上面有一個端口與私網(wǎng)selfservice一致。7. 在provider網(wǎng)絡上創(chuàng)建router的網(wǎng)關，這個時候在provider的網(wǎng)橋brq-4c上創(chuàng)建一個端口tap-ba，同時在路由器上看到有對應的網(wǎng)關（注：要切換到admin權(quán)限下才能看到）。至此：控制（網(wǎng)絡）節(jié)點上的網(wǎng)絡配置全部創(chuàng)建完成，對應的網(wǎng)橋和端口如下：對應的命名空間如下：對應的網(wǎng)絡如下：對應的路由如下：8. 創(chuàng)建虛擬機的時候，會在對應的計算節(jié)點創(chuàng)建對應的網(wǎng)橋，而控制（網(wǎng)絡）節(jié)點不會有任何的變化，創(chuàng)建網(wǎng)橋的時候用什么網(wǎng)絡的ID，就創(chuàng)建于空間（網(wǎng)絡）節(jié)點名稱一樣的網(wǎng)橋，網(wǎng)橋如下：所有結(jié)構(gòu)如下圖所示再分解如下：一開始controller上網(wǎng)絡接口的配置如下：rootwcontroller # ifconfigbrq2e52256f-38: flags=4163 mtu 1450 ether 22:46:9e:24:f7:9c txqueuelen 1000 (Ethernet) RX packets 20 bytes 1736 (1.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0brq3d648406-f4: flags=4163 mtu 1500 inet 192.168.211.11 netmask 255.255.255.0 broadcast 192.168.211.255 ether 06:82:7c:00:01:b2 txqueuelen 1000 (Ethernet) RX packets 189744 bytes 60059122 (57.2 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 56953 bytes 18259759 (17.4 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0brqd73f593e-3a: flags=4163 mtu 1450 ether 1e:a7:45:09:d2:25 txqueuelen 1000 (Ethernet) RX packets 68 bytes 3478 (3.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0docker0: flags=4099 mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 ether 02:42:68:2a:17:10 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0eth0: flags=4163 mtu 1500 inet6 fe80:482:7cff:fe00:1b2 prefixlen 64 scopeid 0x20 ether 06:82:7c:00:01:b2 txqueuelen 1000 (Ethernet) RX packets 423257 bytes 136849834 (130.5 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 167688 bytes 32119313 (30.6 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0eth1: flags=4163 mtu 1500 inet 10.10.1.195 netmask 255.255.255.0 broadcast 10.10.1.255 inet6 fe80:70ff:fef4:16 prefixlen 64 scopeid 0x20 ether 02:00:70:f4:00:16 txqueuelen 1000 (Ethernet) RX packets 823534 bytes 263237259 (251.0 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 713441 bytes 219028904 (208.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 :1 prefixlen 128 scopeid 0x10 loop txqueuelen 1 (Local Loopback) RX packets 7501903 bytes 2489191984 (2.3 GiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 7501903 bytes 2489191984 (2.3 GiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tap13e6f3df-44: flags=4163 mtu 1450 ether da:53:12:6b:c6:c4 txqueuelen 1000 (Ethernet) RX packets 16 bytes 2608 (2.5 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24 bytes 2958 (2.8 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tap3993ded3-a9: flags=4163 mtu 1500 ether be:fd:f8:4f:4a:5d txqueuelen 1000 (Ethernet) RX packets 24318 bytes 2087968 (1.9 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 147064 bytes 11429892 (10.9 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tap6f9999ab-75: flags=4163 mtu 1500 ether 82:81:c7:25:f0:f9 txqueuelen 1000 (Ethernet) RX packets 21920 bytes 1814175 (1.7 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 151249 bytes 11756457 (11.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tapcfaba77e-18: flags=4163 mtu 1450 ether ba:d5:fd:76:36:39 txqueuelen 1000 (Ethernet) RX packets 31659 bytes 2370348 (2.2 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 24852 bytes 1947535 (1.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tapeb1df5ee-9e: flags=4163 mtu 1450 ether 9e:d5:ce:08:48:92 txqueuelen 1000 (Ethernet) RX packets 17 bytes 1926 (1.8 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 64 bytes 4082 (3.9 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0vxlan-46: flags=4163 mtu 1450 ether 1e:a7:45:09:d2:25 txqueuelen 1000 (Ethernet) RX packets 9 bytes 1338 (1.3 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 60 bytes 2664 (2.6 KiB) TX errors 0 dropped 11 overruns 0 carrier 0 collisions 0vxlan-68: flags=4163 mtu 1450 ether 22:46:9e:24:f7:9c txqueuelen 1000 (Ethernet) RX packets 24858 bytes 1600645 (1.5 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 31659 bytes 1928386 (1.8 MiB) TX errors 0 dropped 9 overruns 0 carrier 0 collisions 0網(wǎng)橋結(jié)構(gòu)如下：rootwcontroller # brctl showbridge namebridge idSTP enabledinterfacesbrq2e52256f-388000.22469e24f79cnotap13e6f3df-44tapcfaba77e-18vxlan-68brq3d648406-f48000.06827c0001b2noeth0tap3993ded3-a9tap6f9999ab-75brqd73f593e-3a8000.1ea74509d225notapeb1df5ee-9evxlan-46docker08000.0242682a1710nocompute1的網(wǎng)絡接口rootwcom1 # ifconfigbrq2e52256f-38: flags=4163 mtu 1450 ether 5a:48:96:7a:35:31 txqueuelen 0 (Ethernet) RX packets 11 bytes 1116 (1.0 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0brqd73f593e-3a: flags=4163 mtu 1450 ether f2:ed:f1:a2:17:e8 txqueuelen 0 (Ethernet) RX packets 57 bytes 2682 (2.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0docker0: flags=4099 mtu 1500 inet 172.17.0.1 netmask 255.255.0.0 broadcast 0.0.0.0 ether 02:42:1e:6e:81:f9 txqueuelen 0 (Ethernet) RX packets 0 bytes 0 (0.0 B) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 0 bytes 0 (0.0 B) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0eth0: flags=4163 mtu 1500 inet 192.168.211.25 netmask 255.255.255.0 broadcast 192.168.211.255 inet6 fe80:40d:f6ff:fe00:1c0 prefixlen 64 scopeid 0x20 ether 06:0d:f6:00:01:c0 txqueuelen 1000 (Ethernet) RX packets 541257 bytes 402036673 (383.4 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 272370 bytes 22098834 (21.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0eth1: flags=4163 mtu 1500 inet 10.10.1.242 netmask 255.255.255.0 broadcast 10.10.1.255 inet6 fe80:20ff:fe29:18 prefixlen 64 scopeid 0x20 ether 02:00:20:29:00:18 txqueuelen 1000 (Ethernet) RX packets 527416 bytes 154641475 (147.4 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 607894 bytes 202236972 (192.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0lo: flags=73 mtu 65536 inet 127.0.0.1 netmask 255.0.0.0 inet6 :1 prefixlen 128 scopeid 0x10 loop txqueuelen 0 (Local Loopback) RX packets 375937 bytes 19737009 (18.8 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 375937 bytes 19737009 (18.8 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0tape802bb66-5a: flags=4163 mtu 1500 inet6 fe80:fc16:3eff:fefb:ca0b prefixlen 64 scopeid 0x20 ether fe:16:3e:fb:ca:0b txqueuelen 500 (Ethernet) RX packets 18158 bytes 1495667 (1.4 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 18130 bytes 1290301 (1.2 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0vxlan-46: flags=4163 mtu 1450 ether f2:ed:f1:a2:17:e8 txqueuelen 0 (Ethernet) RX packets 60 bytes 2664 (2.6 KiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 9 bytes 1608 (1.5 KiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0vxlan-68: flags=4163 mtu 1450 ether 5a:48:96:7a:35:31 txqueuelen 0 (Ethernet) RX packets 22633 bytes 1387715 (1.3 MiB) RX errors 0 dropped 0 overruns 0 frame 0 TX packets 18158 bytes 1786195 (1.7 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0因開始部署測試了幾臺虛擬機，導致環(huán)境很亂，無法重新每一部發(fā)生的過程，先清理環(huán)境取消浮動IP綁定，命令行：openstack server delete floating ip selfservice-instance 203.0.113.104刪除浮動IPopenstack floating ip delete 192.168.211.209查看浮動IP列表 openstack floating ip list刪除虛擬機openstack server delete selfservice-instance以上操作對于controller的接口和網(wǎng)橋沒有任何變化，但是compute1的接口發(fā)生了變化rootwcom1 # brctl showbridge namebridge idSTP enabledinterfacesbrq2e52256f-388000.5a48967a3531novxlan-68brqd73f593e-3a8000.f2edf1a217e8novxlan-46docker08000.02421e6e81f9no可以發(fā)現(xiàn)tape802bb66-5a接口已經(jīng)不存在了，說明隨著虛擬機的刪除，對接該虛擬機的tape802bb66-5a接口也刪除了，但是網(wǎng)絡還存在。開始刪除網(wǎng)絡：目前網(wǎng)絡結(jié)構(gòu)如下：另外還有一個router，計劃先刪內(nèi)網(wǎng)，再刪router，再刪provider刪除serfservice11、 因為selfservice1有和router對接，先刪除接口neutron router-interface-delete router selfservice12、 查看對應的router接口3、 刪除子網(wǎng)發(fā)現(xiàn)端口tapeb1df5ee-9e刪除了，那刪除的這個端口是干嘛的呢，其實是把對應的dhcp服務也刪除了，DHCP服務的端口對應也刪除了。這個時候compute沒有任何變化查看對應服務4、 刪除網(wǎng)絡發(fā)現(xiàn)網(wǎng)橋brqd73f593e-3a也被刪除了同時compute上的對應的網(wǎng)橋也被刪除了5、 那么現(xiàn)在剩下的就是一個私網(wǎng)selfservice，一個路由router，一個公網(wǎng)provider，兩個dhcp服務，對應如下：6、 開始刪除私網(wǎng)selfservice,先刪除在router中的接口7、 刪除路由router注意發(fā)現(xiàn)兩個網(wǎng)橋上都少了一個接口brq2e52256f-38 網(wǎng)橋上的tapcfaba77e-18；brq3d648406-f4網(wǎng)橋上的tap6f9999ab-75，這說明這兩個接口都是網(wǎng)橋連接router的。8、 刪除子網(wǎng)可以看到，刪除子網(wǎng)后，網(wǎng)橋還在，但是少了一個接口，同時DHCP服務也少了一個，說明少的這個接口是連接dhcp服務的。這個時候computer是沒有變化的9、 刪除私網(wǎng)網(wǎng)絡網(wǎng)橋被刪除，網(wǎng)絡被刪除，同時computer上的網(wǎng)絡也被刪除了10、 刪除provider網(wǎng)絡子網(wǎng)可以發(fā)現(xiàn)，dhcp服務沒有，網(wǎng)橋還