辦公存儲升級改造方案

1、辦公存儲升級改造設(shè)計方案辦公存儲升級改造設(shè)計方案63/63辦公存儲升級改造設(shè)計方案文檔信息目錄1文檔歸納32方案要點43系統(tǒng)設(shè)計大綱113.1需求解析113.2設(shè)計目標113.3系統(tǒng)架構(gòu)123.4管理平臺12命令行管理方式（PuttyviaSSH）12圖形化管理方式（SystemManager）124存儲硬件安裝部署144.1機柜部署15機柜前端視圖15機柜后端視圖184.2設(shè)備上架204.3MCC設(shè)備結(jié)構(gòu)拓撲圖224.4存儲控制器FAS9000224.5Csicoswitch6505244.6ATTOFibreBridge7500N254.7電氣要求285網(wǎng)絡(luò)設(shè)計30網(wǎng)絡(luò)配置規(guī)范30網(wǎng)絡(luò)資源

2、統(tǒng)計346存儲配置376.1Aggr配置方案377存儲控制器配置397.1初始配置397.2輸入LICENSE467.3配置集群同樣網(wǎng)絡(luò)（clusterpeeringnetwork）477.4鏡像rootaggr487.5存儲空間配置步驟487.6MCC配置497.7檢查并模擬cluster切換498參數(shù)和協(xié)議層配置498.1時區(qū)和NTP時鐘同步498.2SNMP設(shè)置508.3集群管理508.4VServer創(chuàng)辦和管理508.5VServer下volume創(chuàng)辦和管理518.6其他設(shè)置519VServer規(guī)劃529.1概覽529.2LIF規(guī)劃539.3SAN卷布局建議549.4卷劃分方案5510

3、存儲系統(tǒng)監(jiān)控方案5710.1服務(wù)器資源匯總需求5710.2Tiebreaker簡稱MCTB軟件安裝配置59網(wǎng)絡(luò)架構(gòu)59安裝配置步驟59MCTB工作體系介紹6011附錄6111.1為VServer的rootvolume創(chuàng)辦LSmirror61文檔歸納本手冊為BBACFAS9000MCC項目實施定制，包含系統(tǒng)整體架構(gòu)規(guī)劃、存儲空間規(guī)劃、網(wǎng)絡(luò)連接規(guī)劃等方面的方案設(shè)計，以及詳細的存儲配置步驟等，用于指導(dǎo)項目實施時的配置操作，保證存儲部署工作取得預(yù)期的效果。如有發(fā)現(xiàn)本手冊存在錯誤或陳舊，請聯(lián)系LenovoNetApp工程師。項目背景和方案要點本項目為甲方北京奔馳汽車有限公司（以下簡稱BBAC）對現(xiàn)有辦公

4、存儲系統(tǒng)整合和升級項目，主要業(yè)務(wù)挑戰(zhàn)如下：?現(xiàn)平臺運行時間已達6年，設(shè)備相對老舊，且存儲設(shè)備原廠支持將于2018年12月31日停止服務(wù)。故當前存儲已經(jīng)不能適應(yīng)未來業(yè)務(wù)快速增長，并為業(yè)務(wù)提供可靠優(yōu)質(zhì)的基礎(chǔ)架構(gòu)?規(guī)范數(shù)據(jù)管理和存儲運維項目實施地址在北京市亦莊北京經(jīng)濟技術(shù)開發(fā)區(qū)博興路8號，BBAC廠區(qū)內(nèi)。當前BBAC有兩個數(shù)據(jù)中心（DC1，DC2），均位于BBAC廠區(qū)內(nèi)，兩個數(shù)據(jù)中心相距2公里，兩個數(shù)據(jù)中心已有裸光纖連接。本次項目希望在DC1和DC2之間成立雙活存儲平臺架構(gòu)，該存儲平臺能同時知足應(yīng)用系統(tǒng)SAN和NAS數(shù)據(jù)接見需求，主要的應(yīng)用系統(tǒng)包括：Exchange、VMware、Oracle、Sh

5、arePoint、ShareFolder，雙活存儲平臺能實現(xiàn)智能且自動的應(yīng)用及數(shù)據(jù)的雙活容災(zāi)保護，達到應(yīng)用和數(shù)據(jù)接見不中止，數(shù)據(jù)不扔掉的目的。BBAC現(xiàn)有DC1、DC2散布圖本方案在當前存儲架構(gòu)基礎(chǔ)上，分期有序，逐步完成新存儲系統(tǒng)的上線，并完成老存儲數(shù)據(jù)遷移和下線或利舊流程。BBAC當前準備升級全新的FAS9000雙活架構(gòu)存儲平臺。在存儲控制器方面，希望成立一個支持多協(xié)議統(tǒng)一存儲平臺，RPO=0的高可用性系統(tǒng)。BBACMCC項目將如下階段進行。第一階段搭建由2個FAS9000控制器組成的MCC（MetroClusterforClusterMode）第二階段將對運行在MCC上做硬件失敗應(yīng)付以及容

6、災(zāi)應(yīng)付測試演練。第三階段遷移老存儲FAS6280和3210數(shù)據(jù)第四個階段利舊老存儲控制器和磁盤柜，做數(shù)據(jù)融資第五個階段業(yè)務(wù)割接，結(jié)合調(diào)試測試，上線投產(chǎn)。本方案著重第一階段的工作，即搭建MCC基礎(chǔ)架構(gòu)平臺和空間布局建議，為后續(xù)遷移測試做好鋪墊。詳細配置情況如下：Clustername安裝區(qū)主機名控制器磁盤柜描述磁盤配置特點安裝點A站點DS224C15*(24*1.2T)SASsiteA_01FAS90003*(24*4T)NLSASMCC北京ClusterADS212C（DC1）1*(12*960G)SSDClustername安裝區(qū)主機名控制器磁盤柜描述磁盤配置特點安裝點B站點DS224C15

7、*(24*1.2T)SASsiteB_01FAS90003*(24*4T)NLSASMCC北京ClusterBDS212C（DC2）1*(12*960G)SSD本次在MCC存儲項目中一套MCC共兩個cluster節(jié)點，包括：兩套1節(jié)點集群960GBSSD磁盤共24塊1.2TSAS磁盤共720塊4TNLSAS磁盤共144塊ONTAP版本選擇ONTAP版本統(tǒng)一采用9.5P2詳細配置信息安裝地SO#SNModel點配置:每個存儲控制器,DC1FAS9000-1ADC2FAS9000-2A為了便于新老系統(tǒng)平滑升級，需要熟悉當前存儲使用情況，統(tǒng)計如下：?核心生產(chǎn)MCC存儲站點site存儲控制器8.2.4

8、P67-Mode聚合名稱磁盤柜聚合名稱磁盤柜聚合名稱磁盤柜聚合名稱磁盤柜DC1FAS6280（850000196852）s688f002擁有shelf4x8x3x7x號盤柜ownership16組450G，8組900Gaggr_450_01（raid4）Shelf40DS2246SAS450GBPool0Shelf41DS2246SAS450GBPool0Shelf42DS2246SAS450GBPool0Shelf80DS2246SAS450GBPool0Shelf81DS2246SAS450GBPool0Shelf82DS2246SAS450GBPool0Shelf83DS2246SAS4

9、50GBPool0aggr_900_01（raid-dp）Shelf43DS2246SAS900GBPool0Shelf44DS2246SAS900GBPool0Shelf45DS2246SAS900GBPool0Shelf46DS2246SAS900GBPool0Shelf80DS2246SAS450GBPool0Shelf81DS2246SAS450GBPool0Shelf82DS2246SAS450GBPool0Shelf83DS2246SAS450GBPool0aggr_450_root（raid4）Shelf40DS2246SAS450GBPool0Shelf41DS2246SAS4

10、50GBPool0Shelf20DS2246SAS450GBPool1DC2FAS6280（850000196747）s688f001擁有shelf1x5x2x6x號盤柜ownership16組450G，8組900GShelf30DS2246SAS450GBPool1Shelf31DS2246SAS450GBPool1Shelf32DS2246SAS450GBPool1Shelf70DS2246SAS450GBPool1Shelf71DS2246SAS450GBPool1Shelf72DS2246SAS450GBPool1Shelf73DS2246SAS450GBPool1Shelf33DS2

11、246SAS900GBPool1Shelf34DS2246SAS900GBPool1Shelf35DS2246SAS900GBPool1Shelf36DS2246SAS900GBPool1Shelf70DS2246SAS450GBPool1Shelf71DS2246SAS450GBPool1Shelf72DS2246SAS450GBPool1Shelf73DS2246SAS450GBPool1Shelf30DS2246SAS450GBPool1Shelf31DS2246SAS450GBPool1aggr_450_01（raid4）Shelf10DS2246SAS450GBPool0聚合名稱磁盤

12、柜聚合名稱磁盤柜ATTO對內(nèi)FCP交換機對外FCP交換機對外以太網(wǎng)交換機InitiatorTarget以太網(wǎng)口其他Shelf21DS2246SAS450GBPool1Shelf22DS2246SAS450GBPool1Shelf23DS2246SAS450GBPool1Shelf24DS2246SAS450GBPool1Shelf60DS2246SAS450GBPool1Shelf61DS2246SAS450GBPool1Shelf62DS2246SAS450GBPool1Shelf63DS2246SAS450GBPool1Shelf25DS2246SAS900GBPool1Shelf26DS

13、2246SAS900GBPool1Shelf27DS2246SAS900GBPool1Shelf28DS2246SAS900GBPool1Shelf60DS2246SAS450GBPool1Shelf61DS2246SAS450GBPool1Shelf62DS2246SAS450GBPool1Shelf63DS2246SAS450GBPool1Shelf20DS2246SAS450GBPool1Shelf21DS2246SAS450GBPool16500N*4sw3_s688f002博科300sw4_s688f002博科300BS_01博科5100BS_02博科5100DS_300B_Pri_

14、1DS_300B_Pri_2？0a,0c5a,5c6a和6b(fcvi)Slot1,slot11,slot12(12c,12dDS_300B_Pri_1，DS_300B_Pri_2)Sp,e0a,e0bslot4(e4c100M)Slot2(NVRAM8)，slot7和13(FlashCache)Shelf11DS2246SAS450GBPool0Shelf12DS2246SAS450GBPool0Shelf13DS2246SAS450GBPool0Shelf14DS2246SAS450GBPool0Shelf50DS2246SAS450GBPool0Shelf51DS2246SAS450GB

15、Pool0Shelf52DS2246SAS450GBPool0Shelf53DS2246SAS450GBPool0aggr_900_01（raid-dp）Shelf15DS2246SAS900GBPool0Shelf16DS2246SAS900GBPool0Shelf17DS2246SAS900GBPool0Shelf18DS2246SAS900GBPool0Shelf50DS2246SAS450GBPool0Shelf51DS2246SAS450GBPool0Shelf52DS2246SAS450GBPool0Shelf53DS2246SAS450GBPool0aggr_450_root（r

16、aid4）Shelf10DS2246SAS450GBPool0Shelf11DS2246SAS450GBPool06500N*4sw1_s688f001博科300sw2_s688f001博科300SBDC2N001博科6510SBDC2N002博科65100a,0c5a,5c6a和6b(fcvi)Slot1,slot11,Sp,e0a,e0bslot4Slot2(NVRAM8)，slot7和13(FlashCache)近線備份存儲（第一套）站點siteDC1存儲控制器8.2.3P27-ModeFAS3210A（850000196723）s688f003獨立擁有shelf2shelf11shel

17、f14號盤柜ownership（除了shelf13一塊磁盤）；共有shelf1,shelf4,shelf12；FAS3210A（85000019673）s688f004獨立擁有shelf3shelf13號盤柜ownership（除了shelf11一塊磁盤）；共有shelf1,shelf4,shelf12；聚合名稱磁盤柜（共8組2T）對內(nèi)FCP交換機對外FCP交換機對外以太網(wǎng)交換機aggr_2000_01（raid_dp，27顆盤）aggr_2000_01（raid_dp，32顆盤）aggr_2000_02（raid_dp，46顆盤）aggr_2000_02（raid_dp，32顆盤）aggr_

18、2000_03（raid_dp，50顆盤）Shelf2DS4243Shelf1DS4243NLSAS2TNLSAS2TShelf3DS4243NLSAS2TShelf11DS4243Shelf4DS4243NLSAS2TNLSAS2TShelf12DS4243NLSAS2TShelf14DS4246Shelf13DS4243NLSAS2TNLSAS2TDS_300B_Pri_1DS_300B_Pri_2BS_01BS_02？InitiatorSAS0a,0b1b,1d0a,0b1b,1dInitiatorFC0c2d（offline狀態(tài)）0c2dTargetFC2a,2c2a,2c以太網(wǎng)口Sp

19、,e0a,e0bSp,e0a,e0b近線備份存儲（第二套）站點siteDC2存儲控制器8.2.3P27-Mode聚合名稱FAS3210A（940000029050）s688f005獨立擁有shelf2,3,4,7,8號盤柜ownership（除了shelf1一塊磁盤）；共有shelf6;aggr_2000_01（raid_dp，32顆盤）aggr_2000_02（raid_dp，46顆盤）FAS3210A（940001053890）s688f006獨立擁有shelf0,1,5號盤柜ownership（除了shelf2和shelf3各一塊磁盤）；共有shelf6；aggr_2000_01（rai

20、d_dp，32顆盤）aggr_2000_02（raid_dp，48顆盤）aggr_2000_03（raid_dp，48顆盤）Shelf0DS4243Shelf2DS4243Shelf1DS4243NLSAS2TShelf3DS4243NLSAS2TShelf4DS4243NLSAS2TShelf5DS4243Shelf7DS4246Shelf6DS4243NLSAS2TNLSAS2TShelf8DS4246NLSAS2T對外SBDC2N001FCP交換機SBDC2N002對外？以太網(wǎng)交換機InitiatorSAS0a,0b0a,0bTargetFC1a,1c1a,1c以太網(wǎng)口Sp,e0a,e0

21、bSp,e0a,e0bNLSAS2TNLSAS2TNLSAS2T系統(tǒng)設(shè)計大綱3.1需求解析BBAC計劃使用NetApp全閃存AFFFAS9000MCC雙活架構(gòu)系統(tǒng)為其虛構(gòu)化業(yè)務(wù)平臺成立一個高性能、高可靠性、無中止可擴展的統(tǒng)一存儲平臺來知足其業(yè)務(wù)不斷發(fā)展的需求。3.2設(shè)計目標為知足客戶需求和利用NetAppClusteredOntap特點和FAS9000高端存儲優(yōu)勢。將根據(jù)以下策略進行配置：本次配置為MCC，由2個FAS9000控制器組成。在兩個站點成立兩個switchless種類的集群。Vserver（SVM）的劃分將基于簡化管理、共享運行能力和存儲能力的原則，在一個Vserver中配置多個節(jié)

22、點的計算和網(wǎng)絡(luò)資源以及存儲資源SAN應(yīng)用需求放在一個SVM，CIFS應(yīng)用需求放在一個SVM，如果日后有NFS和ISCSI應(yīng)用需要再新增SVMVserver的volume將散布在各個Aggregate上快照設(shè)置所有VOL快照設(shè)置參照老存儲存儲效率的部署默認開啟Dedup和壓縮功能ONTAP版本選擇ONTAP版本統(tǒng)一采用9.5P23.3系統(tǒng)架構(gòu)基于NetAppMCC集群存儲環(huán)境的系統(tǒng)架構(gòu)示意圖如下：3.4管理平臺Cluster-mode的管理方式主要經(jīng)過命令行CLI和圖形化GUI管理方式進行。針對本次實施，存儲經(jīng)過cluster管理地址接見到systemmanager,經(jīng)過內(nèi)嵌的管理軟件對存儲進行

23、管理和監(jiān)控。命令行管理方式（PuttyviaSSH）Cluster-mode的命令行管理方式主要經(jīng)過Putty或近似的支持SSH的客戶端經(jīng)過SSH協(xié)議對Cluster進行管理。命令行方式管理可以管理整個Cluster-mode(經(jīng)過cluster-mgmtIP)，也可以只是管理某個VServer（經(jīng)過VServer的managementIP）。如果需要，也可以經(jīng)過SPIP對某個控制器進行硬件基本的管理。因此，要求管理工作臺與Cluster-mode的管理端口和各個SP端口可以經(jīng)過SSH協(xié)議聯(lián)通。本次實施不設(shè)置直接收理Vserver的體系。圖形化管理方式（SystemManager）Cluste

24、r-mode的圖形化管理方式主要經(jīng)過NetApp管理軟件OncommandSystemmanager對Cluster進行管理。在MCC中，systemmanager軟件集成在操作系統(tǒng)版本中，經(jīng)過cluster管理IP就可以直接接見，也許安裝，可以經(jīng)過這個軟件即管理MCC，也可以管理其他7-ModeandC-Mode存儲。對安裝主機的物理要求：Pentiumx86processor1GBRAM1GBvideodisplayRAM1GBfreediskspaceIfyouareupgradingfromanearlierversion,youmightrequireadditionaldisksp

25、acefortheexistinglogfiles.WirelessorEthernetconnectiontothenetworkA32-bitor64-bitWindowsoperatingsystemAdobe?FlashPlayer11.0orlaterMicrosoft?hotfixformanagingstoragesystemsoveraVPNconnectiononWindows7andWindowsServer2008.YoucandownloadthehotfixfromtheMicrosoftSupportSitepage.32-bitor64-bitOracleJava

26、RuntimeEnvironment(JRE)7Installing32-bitor64-bitJREdependsontheoperatingsystem.Ifyouhavea32-bitWindowsoperatingsystem,32-bitJREmustbeinstalled.Similarly,ifyouhavea64-bitWindowsoperatingsystem,64-bitJREmustbeinstalled.支持如下Windows系統(tǒng)：WindowsVistaWindows7Windows8WindowsServer2003WindowsServer2008Windows

27、Server2008R2WindowsServer2012WindowsServer2012R2支持如下瀏覽器：InternetExplorer?9.0or10.0Note:InternetExplorerlaunchedinthecompatibilitymodeisnotsupported.MozillaFirefox?33GoogleChrome?39存儲硬件安裝部署本次設(shè)備到貨為原廠安裝，各節(jié)點的存儲硬件連接按照存儲內(nèi)部最正確配置。38個磁盤柜分為8組Stack，每組Stack最多包含10個磁盤柜。每個站點有1臺FAS9000控制器，4個ATTO7500N橋接器，兩臺Brocade65

28、05交換機。4.1機柜部署4.1.1機柜前端視圖Cabinet1Cabinet2ProjectDrawingTitleLegendBBACFAS9000CabinetViewDateDrawnByScalePageFilenameRevision02Apr,2019BBACFAS90001:101of41Cabinet1Cabinet2ProjectDrawingTitleLegendBBACFAS9000CabinetViewDateDrawnByScalePageFilenameRevision02Apr,2019BBACFAS90001:101of41部署設(shè)備前端視圖4.1.2機柜后端視

29、圖FAS9000(BBAC-A1)Cabinet1Cabinet2ProjectDrawingTitleLegendBBACFAS9000CabinetViewDateDrawnByScalePageFilenameRevision02Apr,2019BBACFAS90001:101of41FAS9000(BBAC-B1)Cabinet1Cabinet2ProjectDrawingTitleLegendBBACFAS9000CabinetViewDateDrawnByScalePageFilenameRevision02Apr,2019BBACFAS90001:101of41部署設(shè)備后端視圖4

30、.2設(shè)備上架根據(jù)機房管理員安排，完成機柜（若新購買）定位，設(shè)備上架（若和機柜分別散發(fā)）等，并接好電源（注意卡緊插頭，確認NetApp存儲所在的機柜的電源開關(guān)都在關(guān)閉狀態(tài)，不要加電；也可以等全部連線結(jié)束后才連接電源）。物理地址信息如下：機柜Position（U）安裝設(shè)備機柜AFC交換機機柜Position（U）安裝設(shè)備位于DC1機房機柜FC交換機DiskShelfDiskShelfDiskShelfDiskShelfATTO7500ATTO7500FAS9000控制器FC交換機FC交換機DiskShelf機柜BDiskShelf位于DC2機房機柜DiskShelfDiskShelfATTO750

31、0ATTO7500FAS9000控制器4.3MCC設(shè)備結(jié)構(gòu)拓撲圖4.4存儲控制器FAS9000下列圖是FAS9000本次項目中配置的后端口示意圖：FAS9000后端口示意圖下表本次項目中配置的FAS9000單控制器卡詳細列表：卡槽型號1X91143A2X91143A345X91929A6X91340A789X91120A10X91143A11卡名稱端口速率4口UTA210Gb/16Gb4口UTA210Gb/16Gb空空4口FCVI卡光口16GbNVRAM卡NVRAM空空4口萬兆電口網(wǎng)卡10Gb/1GbRJ454口UTA210Gb/16GbFAS9000端口比較表4.5Brocadeswitch

32、6505下列圖是Brocadeswitch6505端口連接示意圖本次實施過程，6505交換機微碼將使用V8.2.1版本單cluster磁盤籠連接表：代表站點A和BConfigurationsusingFibreBridge7500NusingbothFCports(FC1andFC2)ComponentCluster_X_N1Cluster_X_N2portSwitch1Switch2FC-VIport5a1FC-VIport5b1FC-VIport5c2FC-VIport5d2HBAport10a3HBAport10b3HBAport11a4HBAport11b4FC-VIport5a5FC

33、-VIport5b5FC-VIport5c6FC-VIport5d6HBAport10a7HBAport10b7HBAport11a8HBAport11b8bridge_X_1FC19FC29bridge_X_2FC110FC210ISL1414ISL1515ISL本次每套FC6505交換機使用2個口4.6ATTOFibreBridge7500NATTOFibreBridge7500N參數(shù)指標：本次項目中ATTOFibreBridge微碼將使用Firmwarev2.80。一對7500N最多可以連接4組堆棧，一個SSD堆棧最多有96個磁盤，所以一對7500N最多支持384塊磁盤，16個DS224

34、C磁盤柜。ATTOFibreBridge7500N與DS224C磁盤柜之間連線示意圖：ATTOFibreBridge7500N與Brocade6505連線示意圖4.7電氣要求每個站點各類設(shè)備電氣要求如下:CurrentACPowerACPowerThermalRatingPowerSystemQtyRack(Amps)(Watts)(VA)(BTU/hr)(kWh/year)ComponentsUnitsTypicalWorstTypicalWorstTypicalWorstTypicalWorstTypicalWorstBrocade6505,24-port(12licensed),Ente

35、rprise,220.60.8120160126.32168.424105481,051.921,402.5616GbShortwaveSFPsFAS9000(1xControllers,184.995.419771,0591,028.421,114.743,3353,6158,564.389,283.191xChassis)ATTOFibre441.22224384235.79404.217641,3121,963.583,366.14Bridge7500NDS224Cw/24x1.2TBSAS15301829.853,5255,8353,710.556,142.0512,04519,920

36、30,900.1551,149.5510kX342ADS212Cw/12x4TBNL-363.425.04669984704.221,035.782,2863,3605,864.468,625.75SAS7.2kX336ADS224Cw/12x960GB120.521.31100255105.26268.42342871876.62,235.33SSDX371ATotal265228.7344.415,6158,6775,910.569,133.6219,18229,62649,221.0976,062.52每個站點的機柜內(nèi)需要空間共計52U，機柜內(nèi)部需要電源線104根。每個機柜需要外接4個單

37、相電電源線。存儲硬件加電測試：確認Filer所有的機柜的電源開關(guān)都在關(guān)閉狀態(tài)用萬用表測量Filer的輸入電壓火線-地線電壓為200V240V零線-地線電壓約1V左右，小于2V設(shè)置好仿真終端（15000，8，N，1）連接Filer機頭Console口和仿真終端之間的RS232串口線打開Filer的磁盤柜的所有電源打開Filer的機頭的所有電源察看仿真終端上的Filer加電自檢的顯示確認Filer自檢信息正常確認Filer機頭和磁盤柜前面板和后邊板的指示燈正常網(wǎng)絡(luò)設(shè)計5.1.1網(wǎng)絡(luò)配置規(guī)范結(jié)合Cluster-mode的要求和過往經(jīng)驗，提出下列網(wǎng)絡(luò)配置相關(guān)規(guī)范：Cluster-Mode內(nèi)部組網(wǎng)方式：

38、本MetroCluster-CMode采用intercluster內(nèi)部直連方式組成集群。e4a,e4b是40Gb的網(wǎng)口，同一站點兩控制器直連方式做集群互聯(lián)（集群節(jié)點間傳輸數(shù)據(jù)）。Cluster-Mode外部組網(wǎng)方式：存儲網(wǎng)與現(xiàn)有架構(gòu)持平:每個控制器采用NAS接入，以數(shù)據(jù)塊方式提供空間服務(wù)。每個控制器使用4個10Gb用于NAS對外提供服務(wù)。Cluster-Mode管理網(wǎng)（ClusterMGMT）組網(wǎng)方式：SiteA_01/SiteB_01node端e3b做為集群管理網(wǎng)絡(luò)入口每個控制器有一個端口(e0M、SP)接入設(shè)備網(wǎng)交換機每個控制器的e0M作為nodemgmtCluster-Mode集群同樣網(wǎng)

39、（ClusterPeeringnetwork）組網(wǎng)方式：每個控制器采用e3a、e3c網(wǎng)口來做集群同樣網(wǎng)配置復(fù)制服務(wù)CRS（configurationReplicationservice）經(jīng)過集群同樣網(wǎng)絡(luò)執(zhí)行，CRS會將本地節(jié)點配置同步復(fù)制到配對集群中的DR配對節(jié)點，復(fù)制的信息包括集群配置和SVM配置。下表列出了站點ACluster-Mode不同節(jié)點的網(wǎng)端口的用途和需要分配的IP地址。NodeInterface網(wǎng)線數(shù)量用途IP數(shù)量NameSiteA_01e0M1節(jié)點管理1e3b1集群管理1e3a1clusterpeering1e3c1clusterpeering1e2a1NAS業(yè)務(wù)接見1e2c1

40、e9a1e9c1NodeInterface網(wǎng)線數(shù)量用途IP數(shù)量NameSiteA_02e0M1節(jié)點管理1e3b1集群管理1e3a1clusterpeering1e3c1clusterpeering1e2a1NAS業(yè)務(wù)接見1e2c1e9a1e9c1e3a,e3b,e3c,e3d為10Gb1Gb自適應(yīng)電口網(wǎng)口，e0M為1Gb100Mb自適應(yīng)電口網(wǎng)口下表列出了站點BCluster-Mode不同節(jié)點的網(wǎng)端口的用途和需要分配的IP地址。NodeInterface網(wǎng)線數(shù)量用途IP數(shù)量NameSiteB_01e0M1節(jié)點管理1集群管理1e3a1clusterpeering1e3c1clusterpeerin

41、g1e2a1NAS業(yè)務(wù)接見1e2c1e9a1e9c1SiteB_02e0M1節(jié)點管理1集群管理1e3a1clusterpeering1e3c1clusterpeering1e2a1NAS業(yè)務(wù)接見1e2c1e9a1e9c1e3a,e3b,e3c,e3d為10Gb1Gb自適應(yīng)電口網(wǎng)口，e0M為1Gb100Mb自適應(yīng)電口網(wǎng)口下表是站點ACluster-Mode需要成立的FailoverGroup來實現(xiàn)控制器間網(wǎng)絡(luò)的高可用。FailoveGroupNodePortNameNotesDataSiteA_01a0aNAS_liffailovergroupSiteA_02a0a下表是站點BCluster-M

42、ode需要成立的FailoverGroup來實現(xiàn)控制器間網(wǎng)絡(luò)的高可用。FailoveGroupNodePortNameNotesDataSiteA_01a0aNAS_liffailovergroupSiteA_02a0a5.1.2網(wǎng)絡(luò)資源統(tǒng)計站點A存儲網(wǎng)絡(luò)平面網(wǎng)線IP數(shù)IPaddressNetmaskDefaultGateway網(wǎng)絡(luò)種類數(shù)（Gb）控制器管理A_01111Gb控制器管理A_02111Gb集群管理A2110Gb控制器管理A_01_SP011Gb控制器管理A_02_SP011Gb集群同樣網(wǎng)絡(luò)CRS_A_011110Gb集群同樣網(wǎng)絡(luò)CRS_A_021110Gb集群同樣網(wǎng)絡(luò)CRS_A_0

43、31110Gb集群同樣網(wǎng)絡(luò)CRS_A_041110Gb站點B存儲網(wǎng)絡(luò)平面網(wǎng)線IP數(shù)IPaddressNetmaskDefaultGateway網(wǎng)絡(luò)種類數(shù)（Gb）控制器管理B_01111Gb控制器管理B_02111Gb集群管理B2110Gb控制器管理A_01_SP011Gb控制器管理A_02_SP011Gb集群同樣網(wǎng)絡(luò)CRS_B_011110Gb集群同樣網(wǎng)絡(luò)CRS_B_021110Gb集群同樣網(wǎng)絡(luò)CRS_B_031110Gb集群同樣網(wǎng)絡(luò)CRS_B_041110Gb站點ABrocade6505交換機交換機名字網(wǎng)線數(shù)IP數(shù)IPaddressNetmaskDefaultGateway網(wǎng)絡(luò)種類（Gb）F

44、C_switch_A_1111FC_switch_A_2111站點BBrocade6505交換機交換機名字網(wǎng)線數(shù)IP數(shù)IPaddressNetmaskDefaultGateway網(wǎng)絡(luò)種類（Gb）FC_switch_B_1111FC_switch_B_2111站點AATTO7500N交換機名字網(wǎng)線數(shù)IP數(shù)IPaddressNetmaskDefaultGateway網(wǎng)絡(luò)種類（Gb）bridge_A_1111bridge_A_2111站點BATTO7500N交換機名字網(wǎng)線數(shù)IP數(shù)IPaddressNetmaskDefaultGateway網(wǎng)絡(luò)種類（Gb）bridge_B_1111bridge_B_2

45、111兩個站點存儲總合需要16根網(wǎng)線，14個IP兩個站點Brocade6505交換機共需要4根網(wǎng)線，4個IP兩個站點ATTO7500N共需要4根網(wǎng)線，4個IP總計：總合需要24根網(wǎng)線，22個IP存儲配置6.1Aggr配置方案空間劃分方案綜合考慮以下因素：最正確實踐建議每個節(jié)點由獨立的rootaggregate（由3塊盤組成）每個pool保留三塊熱備盤，剩下的磁盤成立數(shù)據(jù)aggregate，采用RAIDDP配置64bitAggr；MCC架構(gòu)中，DataAggr和RootAggr都需要做鏡像，磁盤平均散布在兩個Pool中綜合上述因素，我們介紹設(shè)備的Aggr配置方案如下：RootAggrsNodeS

46、iteDevicesTotalDataParityRAID/SizeReserveWAFLParityRoot(TiB)(TiB)(TiB)(TiB)netapp1_rootBBAC-A1A1.2TB312RAID_DP/0.04900.10892.17880.9314SAS10k3312Totals0.04900.10892.17880.9314A站點，共配置19個磁盤籠，詳細分配方式如下：RootMirrorNodeSiteDevicesTotalDataParityRAID/SizeReserveWAFLParityMirror(TiB)(TiB)(TiB)(TiB)netapp1_ro

47、otBBAC-A1B1.2TB312RAID_DP/0.04900.10892.17880.9314SAS10k3312Totals0.04900.10892.17880.9314DataAggrsNodeSiteDevicesTotalDataParityRAID/SizeReserveWAFLParityUsable(TiB)(TiB)(TiB)(TiB)netapp1_aggr1BBAC-A1A1.2TB17516114RAID_DP/7.892817.539515.2518149.9630SAS10k25netapp1_aggr2BBAC-A1A4TBNL-17152RAID_DP/2

48、.45125.44717.262846.5727SAS7.2k17netapp1_aggr3BBAC-A1A960GB532RAID_DP/0.11790.26191.74612.2393SSD1019717918Totals10.461923.248524.2607198.7750AggrsMirrorNodeSiteDevicesTotalDataParityRAID/SizeReserveWAFLParityMirror(TiB)(TiB)(TiB)(TiB)netapp1_aggr1BBAC-A1B1.2TB17516114RAID_DP/7.892817.539515.2518149

49、.9630SAS10k25netapp1_aggr2BBAC-A1B4TBNL-17152RAID_DP/2.45125.44717.262846.5727SAS7.2k17netapp1_aggr3BBAC-A1B960GB532RAID_DP/0.11790.26191.74612.2393SSD1019717918Totals10.461923.248524.2607198.7750SparesDevicesTotalCapacityCapacityAllocation(TiB)(TiB)Data1.2TB44.3665DriveFormatting1.03580.19%Data4TB2

50、7.2774Root0.93140.17%Data960GB21.7466Parity52.87909.89%WAFL46.71498.74%Total13.3905Mirror199.706537.37%Reserve21.02183.93%Spare13.39052.51%Usable198.775037.19%Total534.4548100.0%B站點，共配置19個DS224C磁盤籠，RootAggrsNodeSiteDevicesTotalDataParityRAID/SizeReserveWAFLParityRoot(TiB)(TiB)(TiB)(TiB)netapp2_rootB

51、BAC-B1B1.2TB312RAID_DP/0.04900.10892.17880.9314SAS10k3312Totals0.04900.10892.17880.9314詳細分配方式如下：RootMirrorNodeSiteDevicesTotalDataParityRAID/SizeReserveWAFLParityMirror(TiB)(TiB)(TiB)(TiB)netapp2_rootBBAC-B1A1.2TB312RAID_DP/0.04900.10892.17880.9314SAS10k3312Totals0.04900.10892.17880.9314DataAggrsNod

52、eSiteDevicesTotalDataParityRAID/SizeReserveWAFLParityUsable(TiB)(TiB)(TiB)(TiB)netapp2_aggr1BBAC-B1B1.2TB17516114RAID_DP/7.892817.539515.2518149.9630SAS10k25netapp2_aggr2BBAC-B1B4TBNL-17152RAID_DP/2.45125.44717.262846.5727SAS7.2k17netapp2_aggr3BBAC-B1B960GB532RAID_DP/0.11790.26191.74612.2393SSD10197

53、17918Totals10.461923.248524.2607198.7750AggrsMirrorNodeSiteDevicesTotalDataParityRAID/SizeReserveWAFLParityMirror(TiB)(TiB)(TiB)(TiB)netapp2_aggr1BBAC-B1A1.2TB17516114RAID_DP/7.892817.539515.2518149.9630SAS10k25netapp2_aggr2BBAC-B1A4TBNL-17152RAID_DP/2.45125.44717.262846.5727SAS7.2k17netapp2_aggr3BB

54、AC-B1A960GB532RAID_DP/0.11790.26191.74612.2393SSD1019717918Totals10.461923.248524.2607198.7750SparesDevicesTotalCapacityCapacityAllocation(TiB)(TiB)Data1.2TB44.3665DriveFormatting1.03580.19%Data4TB27.2774Root0.93140.17%Data960GB21.7466Parity52.87909.89%WAFL46.71498.74%Total13.3905Mirror199.706537.37

55、%Reserve21.02183.93%Spare13.39052.51%7存儲控制器配置Usable198.775037.19%Total534.4548100.0%7.1初始配置存儲控制器初始配置時，系統(tǒng)會成立rootaggregate（可根據(jù)設(shè)計需要，更正aggrname）。其余aggregate劃分請參照“存儲空間劃分”章節(jié)；OnTap操作系統(tǒng)手動指定配置在每個POOL0所在的磁盤籠中前3塊盤里，之后再鏡像到POOL1的磁盤籠中。本次MCC實施，有2套FAS9000，位于兩個站點，以下是cluster成立和加入案例。第1個節(jié)點Clustersetup:clustersetupWelco

56、metotheclustersetupwizard.Youcanenterthefollowingcommandsatanytime:helpor?-ifyouwanttohaveaquestionclarified,back-ifyouwanttochangepreviouslyansweredquestions,andexitorquit-ifyouwanttoquittheclustersetupwizard.Anychangesyoumadebeforequittingwillbesaved.Youcanreturntoclustersetupatanytimebytypingclus

57、tersetup.Toacceptadefaultoromitaquestion,donotenteravalue.Doyouwanttocreateanewclusterorjoinanexistingcluster?create,join:createSystemDefaults:Privateclusternetworkportse0a,e0b.ClusterportMTUvalueswillbesetto9000.ClusterinterfaceIPaddresseswillbeautomaticallygenerated.Doyouwanttousethesedefaults?yes

58、,noyes:noStep1of5:CreateaClusterYoucantypeback,exit,orhelpatanyquestion.Listtheprivateclusternetworkportse0a,e0b:EntertheclusterportsMTUsize9000:Entertheclusternetworknetmask255.255.0.0:GeneratingadefaultIPaddress.Thiscantakeseveralminutes.EntertheclusterinterfaceIPaddressforporte0a169.254.229.244:G

59、eneratingadefaultIPaddress.Thiscantakeseveralminutes.EntertheclusterinterfaceIPaddressforporte0b169.254.134.253:Entertheclustername:bbacEntertheclusterbaselicensekey:CreatingclusterbbacStartingclustersupportservicesClusterbbachasbeencreated.Step2of5:AddFeatureLicenseKeysYoucantypeback,exit,orhelpata

60、nyquestion.Enteranadditionallicensekey:CIFSLicensewasadded.Enteranadditionallicensekey:NFSLicensewasadded.Enteranadditionallicensekey:SnapMirrorDataProtectionLicensewasadded.Enteranadditionallicensekey:Step3of5:SetUpaSVMforClusterAdministrationYoucantypeback,exit,orhelpatanyquestion.Entertheclustera