Pivotal開源大數(shù)據(jù)平臺(tái)概覽

1、A NEW PLATFORM FOR A NEW ERAO Copyright 2013 Pivotal. All rights reserved.Pivotal 介紹一家新成立的公司正式成立于 2013年4月1日繼承了 EMC 和 VMWare 兩家公司在特定產(chǎn)品和技術(shù)方向上的人員和資產(chǎn)股東: EMC, Vmware, GE公司使命: 為一個(gè)嶄新的技術(shù)時(shí)代構(gòu)建支撐平臺(tái) Copyright 2013 Pivotal. All rights reserved. 日程為什么需要一個(gè)新的技術(shù)支撐平臺(tái)?技術(shù)支撐平臺(tái)分類與演進(jìn)技術(shù)傳承，團(tuán)隊(duì)組成與未來(lái)計(jì)劃特別嘉賓主題 Copyright 2013 Piv

2、otal. All rights reserved.企業(yè) IT 的帶系發(fā)展MAINFRAME實(shí)現(xiàn)財(cái)務(wù)處理自動(dòng)化MainframesISAMCLIENT-SERVER& WEB實(shí)現(xiàn)大部分紙面處理流程的自動(dòng)化: ERP, CRM, Email, Relational DatabasesMinis & PCsCloud-Enabled DatacenterNew Data-fabricsCONSUMER GRADE新的客戶體驗(yàn)新的業(yè)務(wù)模式由服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)引領(lǐng)潮流 新的應(yīng)用構(gòu)建和支撐方式 Copyright 2013 Pivotal. All rights reserv

3、ed.CONSUMER GRADE先行者: 服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)MAINFRAMESoftware-Defined DatacenterNew Data-fabrics新的客戶體驗(yàn)新的業(yè)務(wù)模式由服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)引領(lǐng)潮流CLIENT-SERVER & WEBSoftware-Defined DatacenterNew Data-fabrics私有的數(shù)據(jù)中心與基礎(chǔ)架構(gòu) Copyright 2013 Pivotal. All rights reserved.CONSUMER GRADE先行者: 服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)MAINFRAMESoftware-Defi

4、ned DatacenterNew Data-fabrics新的客戶體驗(yàn)新的業(yè)務(wù)模式由服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)引領(lǐng)潮流CLIENT-SERVER & WEBSoftware-Defined DatacenterNew Data-fabrics私有的數(shù)據(jù)中心與基礎(chǔ)架構(gòu) Copyright 2013 Pivotal. All rights reserved.CONSUMER GRADE先行者: 服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)MAINFRAMESoftware-Defined DatacenterNew Data-fabrics新的客戶體驗(yàn)新的業(yè)務(wù)模式由服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)引

5、領(lǐng)潮流CLIENT-SERVER & WEBSoftware-Defined DatacenterNew Data-fabrics私有的數(shù)據(jù)中心與基礎(chǔ)架構(gòu) Copyright 2013 Pivotal. All rights reserved.CONSUMER GRADE先行者: 服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)MAINFRAMESoftware-Defined DatacenterNew Data-fabrics新的客戶體驗(yàn)新的業(yè)務(wù)模式由服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)引領(lǐng)潮流CLIENT-SERVER & WEBSoftware-Defined DatacenterNew Da

6、ta-fabrics私有的數(shù)據(jù)中心與基礎(chǔ)架構(gòu) Copyright 2013 Pivotal. All rights reserved.來(lái)自企業(yè)的獨(dú)有需求 MAINFRAMECLIENT-SERVER & WEBCONSUMER GRADE新的客戶體驗(yàn)新的業(yè)務(wù)模式由服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)引領(lǐng)潮流大量設(shè)備大量設(shè)備, 海量日志海量日志, 多種應(yīng)多種應(yīng)用用無(wú)處不在的數(shù)據(jù)采集無(wú)處不在的數(shù)據(jù)采集Software-Defined DatacenterNew Data-fabrics Copyright 2013 Pivotal. All rights reserved.來(lái)自企業(yè)的獨(dú)有需求 M

7、AINFRAMECLIENT-SERVER & WEBCONSUMER GRADE新的客戶體驗(yàn)新的業(yè)務(wù)模式由服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)引領(lǐng)潮流大量設(shè)備大量設(shè)備, 海量日志海量日志, 多種應(yīng)多種應(yīng)用用無(wú)處不在的數(shù)據(jù)采集無(wú)處不在的數(shù)據(jù)采集Software-Defined DatacenterNew Data-fabrics Copyright 2013 Pivotal. All rights reserved.來(lái)自企業(yè)的獨(dú)有需求 MAINFRAMECLIENT-SERVER & WEBCONSUMER GRADE新的客戶體驗(yàn)新的業(yè)務(wù)模式由服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)引領(lǐng)潮流大

8、量設(shè)備大量設(shè)備, 海量日志海量日志, 多種應(yīng)多種應(yīng)用用無(wú)處不在的數(shù)據(jù)采集無(wú)處不在的數(shù)據(jù)采集Software-Defined DatacenterNew Data-fabrics Copyright 2013 Pivotal. All rights reserved.來(lái)自企業(yè)的獨(dú)有需求 MAINFRAMECLIENT-SERVER & WEBCONSUMER GRADE新的客戶體驗(yàn)新的業(yè)務(wù)模式由服務(wù)于海量客戶的互聯(lián)網(wǎng)巨頭企業(yè)引領(lǐng)潮流大量設(shè)備大量設(shè)備, 海量日志海量日志, 多種應(yīng)多種應(yīng)用用無(wú)處不在的數(shù)據(jù)采集無(wú)處不在的數(shù)據(jù)采集Software-Defined DatacenterNew Da

9、ta-fabrics.ETC Copyright 2013 Pivotal. All rights reserved.New ExperiencesNew Biz Modelspioneered by new Consumer Internet giantsInternet-of-thingsPervasive telemetryMAINFRAMECLIENT-SERVER & WEBCONSUMER GRADESoftware-Defined DatacenterNew Data-fabrics.ETC上述因素促成了對(duì)新平臺(tái)的需求 Copyright 2013 Pivotal. Al

10、l rights reserved. 典型目標(biāo)場(chǎng)景 需要新的消費(fèi)級(jí)支撐能力 通過(guò)遙測(cè)與遙控實(shí)現(xiàn)智能工業(yè)控制與維護(hù)診斷 實(shí)時(shí)電信應(yīng)用場(chǎng)景 未來(lái)新零售場(chǎng)景 大規(guī)模智能農(nóng)業(yè)應(yīng)用 還有許多其他應(yīng)用場(chǎng)景A NEW PLATFORM FOR A NEW ERA Copyright 2013 Pivotal. All rights reserved.新一代企業(yè)級(jí) PaaS 的設(shè)計(jì)準(zhǔn)則 與當(dāng)前各種 IaaS 平臺(tái)配合工作 不論是私有的還是公有的 充分關(guān)注新一代開發(fā)人員和新一代應(yīng)用的需求 為有效進(jìn)行實(shí)時(shí)大容量信息處理而設(shè)計(jì) 同時(shí)兼顧與傳統(tǒng)應(yīng)用的互操作性 Copyright 2013 Pivotal. All r

11、ights reserved.ETC支持開放標(biāo)準(zhǔn)并與開源有效互動(dòng)一數(shù)據(jù)為中心不與特定云平臺(tái)綁定針對(duì)開發(fā)者的需求針對(duì)企業(yè)的需求Cloud FabricData FabricApplication FabricScale-out storage: HDFS/ObjectLanguages& FrameworksIngest & Query: very high-capacity & in-memoryAnalyticsServicesCloud Abstraction (portability)Automation: App Provisioning & Life-

12、cycleService RegistryPivotal One: 重新定義企業(yè)級(jí)重新定義企業(yè)級(jí) PaaS Copyright 2013 Pivotal. All rights reserved.Cloud FabricData FabricApplication FabricScale-out storage: HDFS/ObjectLanguages& FrameworksIngest & Query: very high-capacity & in-memoryAnalyticsServicesCloud Abstraction (portability)Auto

13、mation: App Provisioning & Life-cycleService RegistryPivotal One: 傳承傳承vFabricGemFirePivotal ConfidentialInternal Use OnlyROB MEE & TEAMAgile & RADHadoopPIVOTAL ONE HERITAGECloudJava/SpringOUTSTANDING TECHNICAL TEAM+700 ENGINEERSIMDGWeb/MobileData Science & AnalyticsMPP SQLCHRIS BEAMS

14、JUERGEN HOELLERNOELLE SIOKAUSHIK DASDAN HOLLEHUYLA EMIR-FARINASGAVIN SHERRYLUKE LONERGANEDWARD HIEATTALEXIS RICHARDSONJAMES WATTERSJAGS RAMNARAYANMILIND BHANDARKARAPURVA DESAIADRIAN COLYERSUDHIR MENONCHRISTOS TRYFONASPivotal ConfidentialInternal Use OnlyCLOUDFABRICELISABETH HENDRICKSONMATTHEW KOCHER

15、JAMES BAYERJEFF NICKJAMES WATTERSDAVID STEVENSONPivotal ConfidentialInternal Use OnlyDATA FABRICDAN HOLLELUKE LONERGANMILIND BHANDARKARAPURVA DESAIGAVIN SHERRYCHRISTOS TRYFONASJAGS RAMNARAYANSUDHIR MENONPivotal ConfidentialInternal Use OnlyAPPLICATIONFABRICCHRIS BEAMSJUERGEN HOELLERALEXIS RICHARDSON

16、ADRIAN COLYERMATTHIAS RADESTOCKSALVATORE SANFILIPPO Copyright 2013 Pivotal. All rights reserved.Cloud Abstraction (portability)Automation: App Provisioning & Life-cycleService Registry基于 Cloud Foundry開放并以開源方式實(shí)現(xiàn): C設(shè)計(jì)目標(biāo):-可移植性-可擴(kuò)展性-自動(dòng)化-彈性不斷成長(zhǎng)的生態(tài)系統(tǒng)Pivotal One: 云支撐架構(gòu)云支撐架構(gòu)Cloud FabricDa

17、ta Fabric.ETCApplication FabricCloud Abstraction (portability)Automation: App Provisioning & Life-cycleService Registry Copyright 2013 Pivotal. All rights reserved.Cloud FabricData & Analytics Fabric.ETCPivotal One: 數(shù)據(jù)支撐架構(gòu)Application FabricScale-out storage: HDFS/ObjectCatalogResource MgmtDa

18、taMgmtTXNProcessHAWQMPPRealTimeDBMap ReduceHbaseK/V統(tǒng)一基于 Hadoop通用的全局分布式集群文件系統(tǒng) (HDFS)與高級(jí)數(shù)據(jù)處理服務(wù)緊密集成-MPP SQL-實(shí)時(shí)交易處理-流式數(shù)據(jù)處理原生設(shè)計(jì) PaaS 設(shè)計(jì)并作為 Pivotal One 的有機(jī)組成部分: 支持彈性伸縮與多租戶環(huán)境Analytics Suite: Chorus/MADlibGPDBRDB Copyright 2013 Pivotal. All rights reserved.HAWQ: 性能測(cè)試結(jié)果對(duì)比4.21988.71612.04152.71,2852.81,81547X

19、19X208X476X648X4.2378.75962.0502.7552.8599X69X25X20X21XPivotal HDPivotal HD Copyright 2013 Pivotal. All rights reserved.ETC有效支撐快速應(yīng)用開發(fā)簡(jiǎn)化在開發(fā)，測(cè)試和生產(chǎn)環(huán)境間的部署和遷移支持多種語(yǔ)言和開發(fā)框架下的自動(dòng)部署和自動(dòng)擴(kuò)展業(yè)界領(lǐng)先的 Java/Spring 運(yùn)行時(shí)支撐松耦合的即時(shí)服務(wù)綁定缺省提供運(yùn)行時(shí)數(shù)據(jù)的分析和可視化監(jiān)控功能Pivotal One: 應(yīng)用支撐架構(gòu)Cloud FabricData FabricApplication FabricLanguages&am

20、p; FrameworksAnalyticsServices Copyright 2013 Pivotal. All rights reserved.Cloud FabricData FabricApplication FabricData & AnalyticsApp DevCloud Automation Pivotal HD GemFire GPDB Spring vFabric Cloud FoundryServices Pivotal Labs & Data Science LabsPivotal One: 切入點(diǎn)切入點(diǎn)PIVOTAL ONEPLATFORM Copy

21、right 2013 Pivotal. All rights reserved.Pivotal 專業(yè)服務(wù): 迅速實(shí)現(xiàn)價(jià)值 Pivotal Labs: 快速開發(fā)和部署新應(yīng)用 經(jīng)過(guò)廣泛證明的方法論降低風(fēng)險(xiǎn)同時(shí)加速開發(fā)過(guò)程Pivotal Data Science Labs: 經(jīng)過(guò)檢驗(yàn)的的數(shù)據(jù)科學(xué)實(shí)踐加速分析項(xiàng)目實(shí)施 用數(shù)據(jù)分析來(lái)驅(qū)動(dòng)業(yè)務(wù)價(jià)值實(shí)現(xiàn)Open Source Support: 協(xié)作式客戶驅(qū)動(dòng)開源支持，服務(wù)和共同研發(fā)Pivotal Data Science Labs Copyright 2013 Pivotal. All rights reserved.關(guān)鍵時(shí)間點(diǎn) 2012 戰(zhàn)略規(guī)劃階段:-四月: 形成正式構(gòu)想-十二月: 構(gòu)想在內(nèi)部發(fā)布 2013 正式發(fā)布與推進(jìn):-二月: Pivotal HD 正式發(fā)布-四月: 公司完成正式注冊(cè)-第二季度: Cloud Foundry on VMware Cloud & AWS-第四季度: Pivotal One GAA NEW PL