2023學(xué)年完整公開課版雅虎云計算

CLOUDCOMPUTING@YAHOO！2Whatwe’llcovertoday…WhyCloud?ScaleandAbstraction;QualityandAgilityYahoo!’suniquefootprintYahoo!’sCloudStrategyOverviewoftheYahoo!CloudvisionandportfolioDeepdiveonHorizontal&FunctionalCloudServicesTheYahoo!OpenStrategyMarryingYahoo!’s“OpenStrategy”,itsplatformsandethicwithexternalCloudservices3Agility&InnovationQuality&Stability

WhyCloud?BenefitsforYahoo!HigherAgility&StabilitywhilemaintainingScaleAbstractionEnabledeveloperstofocuson

theirapplications,notinfrastructureAcceleratinginnovationAddingnewfeaturesandproducts

ataneverfasterrateIncreasingScale&AvailabilityMorerobustly,moreglobally,

morecompletely,foragivenbudgetCloudispushinguptheOperationExcellenceCurve

4Yahoo!’sUniqueCloud:

UnprecedentedScaleMassiveuserbaseandengagement500M+uniqueuserspermonthHundredsofpetabyteofstorageHundredsofbillionsofobjectsHundredofthousandsofrequests/secGlobalTensofgloballydistributeddatacentersServingeachregionatlowlatencies5ChallengingUsersRapidlyextractingvaluefromvoluminousdataDowntimeisnotanoption(outagescost$millions)VariableusagepatternsYahoo!’sUniqueCloud:

UnprecedentedScale6Yahoo!CloudServicesROI&Innovation

Users

Applications

FunctionalCloud

Services

HorizontalCloud

Services

PhysicalLayer

Y!OS,BOSS,YQL,APT,Analytics,…Storage,Batch,EdgeServing,…7Yahoo!CloudServices:

FocusonPaaSofferingsROI&InnovationIaaSPaaSSaaS

Users

Applications

FunctionalCloud

Services

HorizontalCloud

Services

PhysicalLayer

8FromInfrastructureto

ShareholdersbenefitHorizontalCloudFocusonopensourceandcollaborativeR&Dwithindustry,academiaandgovernmentFunctionalCloudFocusondeveloping"openstrategy"frameworks,toolsandservicesfordevelopers(atYahoo!andbeyond)CombinedTogetherLeverageouruniquescale,assetsanddatatodrivedisruptiveinnovationsinthemarketandexpandYahoo!’scompetitivedifferentiation9Yahoo!CloudStrategyinAction:

TheFrontPageCaseStudyHorizontalCloud–Storage&HadoopAnalyzeextremelylargecontentdatasetsFunctionalCloud–ContentOptimizationRatecontentitemsbasedonvariousparametersApplications–Yahoo’sFrontPageDisplay“highrating”itemstotherightusersBenefitconsumersandadvertisersandgrow

Yahoo!’srevenue10Yahoo!CloudStrategyinAction:

TheInquisitorCaseStudyHorizontalCloud–HadoopAnalyzelargesearch-indexdatasetsFunctionalCloud-BOSSExposethedatainastructured,open,flexible

and“cloudlike”wayApplications-iPhoneTMInquisitorLeverageBOSStoprovideinnovativeconsumerexperienceBenefitconsumersandgrowYahoo!’srevenue11HorizontalCloudServicesROI&Innovation

Users

Applications

FunctionalCloud

Services

HorizontalCloud

Services

PhysicalLayer

12HorizontalCloudServicesOptimizedforYahoo!-scaleYahoo!-internalfocusDataprocessingandservingenvironmentsDrivefasterinnovationandagilityShorterproductdevelopmentcyclesReducelaborandcostsforinfrastructureMulti-yeareffortStrategicinvestmentacrossthecompany13HorizontalCloudServices:

ConceptualView14HorizontalCloudServices:UseCasesAdsOptimizationContentOptimizationSearchIndexImage/VideoStorage&DeliveryMachineLearning

(e.g.Spamfilters)AttachmentStorage15Yahoo!DistributionofHadoopHadoopinanutshellOpensourcedistributedfilesystem¶llelexecutionenvironmenttoprocessmassiveamountsofdataStartedin2005,becametop-levelApacheprojectin2008SimpleDesignforHorizontalScalingoncommodityHWYahoo!DistributionofHadoopSourcedistributionofYahoo!’simplementationofHadoop

(BasedentirelyoncodefoundintheApacheHadoop)TestedanddeployedatYahoo!’smassivescaleBenefitthelargerecosystem,Increasepaceofinnovation/hadoop16Yahoo!runsthelargest

HadoopClustersintheWorld25,000+nodesClustersofupto4,000nodes4TiersofclustersDevelopment&Testing,POCs,Science&Research,ProductionTerasortBenchmarks62secondstosortOneTerabyte(runon1,500nodes)16.25hourstosortOnePetabyte(runon3,700nodes)Webmapapplication~490TBshuffling~280TBoutput17CaseStudy-SearchAssist?DatabaseforSearchAssist?isbuiltusingHadoop.3yearsoflog-data,20-stepsofmap-reduceLeverageHadoop’sscalability,loadbalancingandresiliencySimplifiedaccess,flexibilityforrapidinnovation(fromC++toPython)BeforeHadoopAfterHadoopTime26days20minutesDevelopmentTime2-3weeks2-3days18FunctionalCloudServicesROI&Innovation

Users

Applications

FunctionalCloud

Services

HorizontalCloud

Services

PhysicalLayer

19FunctionalCloudServicesProvidesfunctionalcapabilitiesforapplicationsHelpdeveloperstoaccomplishintegratedwebexperiencesinafasterandeasierwayProvidescommonsetoffunctional“buildingblocks”“Poweredby”thehorizontalcloudservicesAbstractsinfrastructureservicesfromtheApplicationE.g.Storage,Compute,Serving,RobustnessandScalabilitySelf-Served,Global,Managed,ElasticandMetered20FunctionalCloudServices:

YQL&BOSS

Asingleendpointservicethatenablesdeveloperstoquery,filterandcombinedataacross

Yahoo!andbeyond

/yql/console/ProvidingYahoo!SearchinfrastructureandtechnologytodevelopersandcompaniestohelpthembuildtheirownsearchexperiencesBuildyourOwnSearchService/search/boss/Yahoo!QueryLanguage21BuildyourOwnSearchService(BOSS)Yahoo!'sopensearchwebservicesplatformServinghundredsofmillionsofusersacrosstheWeb.Goal:fosterinnovationinthesearchindustryBuildandlaunchweb-scalesearchproductsthatutilizetheentireYahoo!Searchindex.AccesstoYahoo!'sinvestmentsincrawlingandindexing,rankingandrelevancyalgorithms22Yahoo!QueryLanguage(YQL)Singleendpointservicetoquery,filterandcombinedataacrossYahoo!andbeyondThe“InternetAPI”SQL-likeSELECTsyntaxforgettingtherightda