NVIDIA 面向大語(yǔ)言模型的全棧 AI 解決方案

NVIDIA面向大語(yǔ)言模型的全棧AI解決方案大語(yǔ)言模型生態(tài)系統(tǒng)快速演進(jìn)導(dǎo)致優(yōu)化異常挑戰(zhàn)?新的算子和定制技術(shù)使得優(yōu)化成為持續(xù)變化的目標(biāo)需要全棧AI解決方案助力模型開(kāi)發(fā)和應(yīng)用部署LLaMaLLaMaGPTFalconStarcoderChatGLMMPT…ImagefromMooler0410/LLMsPracticalGuideASurveyonChatGPTandBeyond.arXiv[Cs.CL].Retrievedfrom/abs/2304.13712FP32FP16BF16FP8(E5M2)FP8(E4M3)FP8相較于FP16/BF162倍的吞吐signmultiplyaccumulateintoTCRangePrecisionexponentsignmultiplyaccumulateintoTCRangePrecisionexponentmantissaDD——s—D——DD—— bias/act/… convertFP32|FP16|BF16|FP8TransformerEngineFP8混合精度訓(xùn)練其它操作仍在高精度參數(shù)更新 TransformerEngine算子融合混合精度訓(xùn)練thehistoryandJax?Hyperparameter:GBS2048_SEQ4096_MBS2_TP2_PP1_DP128?Hyperparameter:GBS1024_SEQ4096_MBS4_TP4_PP1_DP32HellaSwagWinoGrandeHellaSwagWinoGrande數(shù)據(jù)中心級(jí)別的計(jì)算機(jī)由網(wǎng)絡(luò)定義數(shù)據(jù)中心AIEthernetFabric?TraditionalEthernetnetworkcansuffice?TraditionalEthernetnetworkforNorth-Southtraffic?NVIDIASpectrum-XEthernetforAINVIDIASpectrum-X-世界上第一款為AI而生的以太網(wǎng)平臺(tái)融合標(biāo)準(zhǔn)以太網(wǎng)連接和專業(yè)的高性能架構(gòu)PerfectEffectiveStandardEthernetSpectrum-4Spectrum-4EthernetSwitchsupportedsoftwarethatspeedsdeveloperexecutionofeverydeveloper’sjobssupportedsoftwarethatspeedsdeveloperexecutionofeverydeveloper’sjobsthousandsAccelerateend-to-performanceNVIDIAAIEnterpriseAIFrameworksNVIDIAAIWorkflowManagementandMLOpsJobScheduling&OrchestrationPROVISIONINGMONITORINGMANAGINGClusterManagementPROVISIONINGMONITORINGMANAGINGCLUSTERINGCLUSTERINGNETWORKIOSTORAGEIOIOMANAGEMENTNetwork/StorageNETWORKIOSTORAGEIOIOMANAGEMENTIN-NETWORKCOMPUTEIN-NETWORKCOMPUTEOperatingSystemNVIDIANeMo大語(yǔ)言模型應(yīng)用落地全流程Acceleratedcitedresponses數(shù)據(jù)處理提升模型性能NeMoDataCuratorsteps:?Classifier-basedfilteringInternetscaledatasetsInternetscaledatasets.△.口△口口△口!!extracttextTextre- NVIDIAMegatron-CorePerformanceatScalePerformanceatScaleMemory,Compute,andCommunicationOptimizationParallelismTechniquesMemorySavingTechniquesSelectiveActivationRecomputeDistributedOptimizersPrecisionawareoptimizers(BF16,FP8)withFusedAdam●●●●●●●●●●●architectureFormalizedProductFormalizedProductSupportLatestresearchandNVIDIANeMo對(duì)大語(yǔ)言模型定制的套件根據(jù)場(chǎng)景定制大語(yǔ)言模型的不同方式Accuracyforspecificuse-casesPROMPTENGINEERINGPROMPTLEARNINGPARAMETEREFFICIENTFINE-TUNINGFINETUNINGPROMPTENGINEERING?LeastexpertiseBenefits??SFTNVIDIATriton推理服務(wù)器快速可擴(kuò)展簡(jiǎn)單的推理服務(wù)開(kāi)源軟件(Jetson)Kubernetes,KServe,PlatformsLatencyLLM生產(chǎn)部署的極致性能TensorRT-LLMisanopen-sourcelibrarytooptimizeinferenceperformanceonthelatestLargeLanguageModelsforNVIDIAGPUs.ItisSoTAPerformancePerformanceTCOEaseExtension#defineanewactivationdefsilu(input:Tensor)→Tensor:#implementmodelslikeinDLFWsclassLlamaModel(Module)def__init__(…)self.layers=ModuleList([…])forlayerinself.layers:LLMBatchingwithTritonStaticInflightBatchingCost為RAG帶來(lái)最佳的準(zhǔn)確度和延時(shí)Acceleratedvectorsearch★ApplyforEarlyAccessStateoftheArtAccuracy?Latesttechniquesforchunkingandmetadata?Alldatasources,suchasunstructured(vectordatabases)andstructured(SQL,Tabular)LowestLatencyandHighestThroughputProductionReadywithEnterpriseSupport?ProductionreadyservicenterprisesupportNVIDIANeMoGuardrails保持LLM安全TopicalGuardrailsTopicalGuardrailsSafetyGuardrailsSecurityGuardrailsLLMs聽(tīng)取ChinaAIDay任一演講，即可獲取DL會(huì)后您將收到優(yōu)惠碼專屬郵件任選右側(cè)一門(mén)公開(kāi)課使用優(yōu)惠碼（僅限一門(mén)）開(kāi)啟LLM實(shí)戰(zhàn)之旅構(gòu)建基于大語(yǔ)言模型（LLM）的應(yīng)用利用大模型開(kāi)源生態(tài)系統(tǒng)快速開(kāi)發(fā)基于預(yù)訓(xùn)練大語(yǔ)言模型的應(yīng)用基礎(chǔ)基礎(chǔ)高效定制大語(yǔ)言模型（LLM）學(xué)習(xí)提示工程和各類高效參數(shù)微調(diào)方法對(duì)預(yù)訓(xùn)練LLM模型進(jìn)行定制以適應(yīng)特定應(yīng)用進(jìn)階進(jìn)階大語(yǔ)言模型（LLM）、生成式AI系列近期公開(kāi)