HEVC編碼技術(shù)概覽

HEVC編碼技術(shù)概覽■

ISO-IEC/MPEG和ITU-T/VCEG成立了一個(gè)研究視頻編碼的聯(lián)合協(xié)作小組JCT-VC(thejointcollaborativeteamonvideocoding)，其宗旨是建立新一代的視頻編碼標(biāo)準(zhǔn)。

■

2010年4月在德國(guó)德雷斯頓召開(kāi)了JCT-VC(JointCollaborativeTeamonVideoCoding)第一次會(huì)議，確定了新一代視頻編碼標(biāo)準(zhǔn)的名稱(chēng)：HEVC(HighEfficiencyVideoCoding)。并且建立了測(cè)試模型TMuC(TestModelunderConsideration)，分領(lǐng)域搜集和審閱技術(shù)提案。初步定于2012年7月完成標(biāo)準(zhǔn)的最終稿。

■

2010年10月在廣州召開(kāi)了JCT-VC第三次會(huì)議，會(huì)上確立了HEVC的第一個(gè)試驗(yàn)?zāi)Ｐ虷M1。

■

新一代視頻壓縮標(biāo)準(zhǔn)的核心目標(biāo)，是在H.264的基礎(chǔ)上將壓縮效率提高一倍。即在保證相同視頻圖像質(zhì)量的前提下，視頻流的碼率減少50%。在提高壓縮效率的同時(shí)，可以允許編碼端適當(dāng)提高復(fù)雜度。

HMcontainsonlyaminimumsetofwell-testedtoolsthattogetherfromacoherentdesignthatisconfirmedtoshowgoodcapability.

Twosettingshavebeendefined：1)highefficiency;2)low-complexity.HMHMConfigurationHighEfficiencyConfigurationLowcomplexityConfigurationCodingUnittreestructure(8×8upto64×64lumasamples)PredictionUnitsTransformunittreestructure(3levelmax.)Transformunittreestructure(2levelmax.)Transformblocksizeof4x4to32x32samples(alwayssquare)AngularIntraPrediction(34directionsmax.)DCT-basedinterpolationfilterforlumasamples(1/4-sample,12-tap)Directionalinterpolationfilterforlumasamples(1/4-sample,6-tap)Bi-linearinterpolationfilterforchromasamples(1/8-sample)AdvancedmotionvectorpredictionContextadaptivebinaryarithmeticentropycodingLowcomplexityentropycodingphase2Internalbit-depthincrease(4bits)XXTransformprecisionextension(4bits)DeblockingfilterAdaptiveloopfilterX1、PicturePartitioning2、IntraPrediction3、InterPrediction4、TransformandQuantization5、EntropyCoding6、LoopFiltering7、CostFunction1、PicturePartitioning1.1.Treeblock(TB)PartitioningPicturesaredividedintoslicesandslicesaredividedintoasequenceoftreeblocks.AtreeblockisanNxNblock

oflumasamplestogetherwiththetwocorrespondingblocksofchromasamplesforapicturethathasthreesamplearrays.ThetreeblockconceptisbroadlyanalogoustothatofthemacroblockinpreviousstandardssuchasH.264/AVC[6].Themaximumallowedsizeofthetreeblockis64x64lumasamples.

TheCodingUnit(CU)isthebasicunitofregionsplittingusedforinter/intraprediction.

Codingunit(CU)thebasiccodingunitlikethemacroblock.CUscanbefurthersplitintopredictionunits(PUs).transformunit(TU)isdefinedfortransformandquantization.Itisalwayssquareanditmaytakeasizefrom8x8lumasamplesuptothesizeofthetreeblock.(8x8,16x16,32x32,64x64)TheCUconceptallowsrecursivesplittingintofourequallysizedblocks,startingfromthetreeblock.Thisprocessgivesacontent-adaptivecodingtreestructurecomprisedofCUblocks.1.2.CodingUnit(CU)structureGiventhattheregionofinterestishomogeneousandthelargesizesaCUcanhave,arelativelylargeCUcanrepresentthisregioninasmallernumberofsymbols.ThewiderangeofsizesforLCUqualitiesthecodectobeefficientandcarefullyoptimizedaccordingtousageorapplication

ThePredictionUnit(PU)isthebasicunitusedforcarryingtheinformationrelatedtothepredictionprocesses.PUsaredefinedafterthelastlevelofCUsplittingIngeneral,itisnotrestrictedtobeingsquareinshape,inordertofacilitatepartitioningwhichmatchestheboundariesofrealobjectsinthepicture.EachCUmaycontainoneormorePUs.1.3.PredictionUnit(PU)structureFigure1.3.FourtypesofPredictionUnitstructurePredictiontypecanhavethefollowingvalues:SkipIntra-prediction

Inter-prediction

illustratesallpossiblePUsplittingaccordingtodifferent

prediction

types.從上圖中可以看到一種新的不對(duì)稱(chēng)運(yùn)動(dòng)分割預(yù)測(cè)（AsymmetricMotionPartition,AMP）,這也是HEVC與H.264在分塊預(yù)測(cè)技術(shù)中最為不同之處。所謂AMP即將編碼單元分割成兩個(gè)大小不一致的預(yù)測(cè)塊其中一個(gè)PU單元的寬/長(zhǎng)為CU單元的3/4,這種預(yù)測(cè)考慮了大尺寸可能的紋理分布可以有效提高大尺寸塊的預(yù)測(cè)效率

TheTransformUnit(TU)isthebasicunitusedforthetransformandquantizationprocesses.Itisalwayssquareanditmaytakeasizefrom4x4upto32x32lumasamples.EachCUmaycontainoneormoreTUs,wheremultipleTUsmaybearrangedinaquadtreestructure,asillustratedinFigure1.4below.3-levelquadtreeforhighefficiencyconfiguration.Samemaximumquadtreedepthforlumaandchroma.1.4.TransformUnit(TU)structureFigure1.4.ExampleofTransformUnitstructure配合不對(duì)稱(chēng)預(yù)測(cè)單元以及矩形預(yù)測(cè)單元，新的HM4.0模型還采納了相應(yīng)的矩形四叉數(shù)TU結(jié)構(gòu)突破了方塊變換的限制，下圖展示了3級(jí)矩形四叉樹(shù)變換水平TU結(jié)構(gòu)，同理可以有垂直分割結(jié)構(gòu)SummeryofCU,PU,TUthreeindependentblockconcepts:CU(Codingunit):thebasicunitofregionsplitting;PU(Predictionunit):thebasicunitofinter/intraprediction;TU(Transformunit):thebasicunitoftransform.CUisanalogoustotheconceptofmacroblock,butitdoesnotrestrictthemaximumsizeanditallowsrecursivesplittingintofourequalsizeCUstoimprovethecontentsadaptivity.EachCUmaycontainoneormorePUs.PUmaycontainmultiplearbitraryshapepartitionsinasinglePUtoeffectivelycodeirregularimagepatterns.TUcanbedefinedindependentlyfromthePU;however,itssizeislimitedtotheCUwhichtheTUbelongsto.Thisseparationoftheblockstructureintothreedifferentconceptsallowseachtobeoptimizedaccordingtoitsrole,whichresultsintheimprovedcodingefficiency.Codingunittreestructureislimitedfrom8×8to64×64forluma,thatistosay,nosplittingisallowedforCU3.IftheLCUsize=16andhierarchicaldepth=2,thenthisisasimilarcodingstructuretomacro-blockandsub-macro-blockinH.264/AVC.TospecifythepredictionmethodusePUtypeandPUsplitting.PUtypescanbeskip,intraandinter.ItshouldbenotedthatthesizeofTUmaybelargerthanthatofthePUbutnotexceedthatoftheCU.TUhavedifferentsplittingforlowcomplexity(LC)andhighefficiency(HE)configurations.2、IntraPredictionTheunifiedintrapredictioncodingtoolprovidesupto33directionalpredictionmodesandoneDCpredictionmodeforeachPU.Figure2.1.The33intrapredictiondirectionsFigure2.2.MappingbetweenintrapredictiondirectionandintrapredictionmodeTable2.1.NumberofsupportedintramodesaccordingtoPUsizePUsizeNumberofintramodes41783416343234645ThetotalnumberofavailablepredictionmodesisdependentonthesizeofthecorrespondingPU,asshowninTable2.1below.Thepixelisreconstructedusingthelinearinterpolationofthereferencetoporleftsamplesat1/32thpixelaccuracyforallblocksizes,insteadofdifferentaccuraciesfordifferentsizes.Twoarraysofreferencesamplesareused,correspondingtotherowofsampleslyingabovethecurrentPUtobepredicted,andthecolumnofsampleslyingtotheleftofthesamePU.Givenadominantpredictiondirection(horizontalorvertical),oneofthereferencearraysisdefinedtobethemainarrayandtheotherarraythesidearray.ThereferencerowabovethePUiscalledthemainarrayandthereferencecolumntotheleftofthesamePUiscalledthesidearray,incaseofverticalprediction.WhilethereferencecolumntotheleftofthePUiscalledthemainarrayandthereferencerowabovethePUiscalledthesidearray,incaseofhorizontalprediction.Onlythemainarraysamplesareusedforpredictionwhentheintrapredictionangleispositive,Whentheintrapredictionangleisnegative,aper-sampletestisperformedtodeterminewhethersamplesfromthemainorthesidearrayshouldbeusedforprediction,asshowninfigure.Whenthesidearrayisused,thecomputationoftheindexofthesidearrayrequiresadivisionoperation.Thedivisionoperationisremovedbyusingalook-uptable(LUT)technique.Thistechniqueisusedforthenegativeangularpredictionprocessinthecalculationofthey-interceptincaseofverticalpredictionorthex-interceptin

caseofhorizontalprediction.Theintegerandfractionalpartsoftheinterceptarecalculatedusingthefollowingequations:deltaIntSide=(256*32*(l+1)/absAng)>>8deltaFractSide=(256*32*(l+1)/absAng)%256absAngistheabsolutevalueofintrapredictionangle(=[2,5,9,13,17,21,26,32])andlisx/ypixellocationforvertical/horizontalprediction.UsingtheLUTtechniques,theaboveequationscanbereplacedwiththefollowingequations:deltaIntSide=(invAbsAngTable[absAngIndex]*(l+1))>>8deltaFractSide=(invAbsAngTable[absAngIndex]*(l+1))%256whereinvAbsAngTable=[4096,1638,910,630,482,390,315,256].ByusingLUT,thereisnodivisionduringthecomputationoftheindexofthesidearray.However,therestillexiststheper-sampletesttodeterminethemainorthesidearrayforprediction.Tosimplifythisprocess,themainarrayisextendedbyprojectingsamplesfromthesidearrayontoitaccordingtothepredictiondirection,asshowninFigure.Duringtheprojection,thefractionalpartoftheinterceptisomittedandtheinterceptisroundedtothenearestinteger:deltaInteger=(invAbsAngTable[absAngIndex]*(l+1)+128)>>8.Atlast,thepredictionprocessonlyusestheextendedmainarrayandthesamesimplelinearinterpolationformulatopredictallsamplesinthePU.

如果經(jīng)過(guò)DC預(yù)測(cè)后，當(dāng)前塊的左側(cè)和上側(cè)邊緣跟周?chē)闹貥?gòu)像素可能會(huì)不大一致，HEVC對(duì)經(jīng)過(guò)DC模式預(yù)測(cè)后的變換單元的邊緣像素進(jìn)行濾波以使變換塊的邊緣跟參考像素之間更加的平滑一些。濾波策略如圖從上圖中我們可以看到，左側(cè)和上側(cè)的灰綠色區(qū)域是已經(jīng)解碼的重構(gòu)像素，當(dāng)前塊中被紅色標(biāo)注的像素是當(dāng)前塊與已重構(gòu)像素相鄰接的邊緣像素，需要對(duì)這些像素進(jìn)行濾波處理。根據(jù)這些邊緣像素與已重構(gòu)的像素相鄰接的像素個(gè)數(shù)來(lái)決定是使用2-tap濾波器還是3-tap濾波器。其中左上角那個(gè)桃紅色像素因?yàn)楹椭車(chē)鷥蓚€(gè)像素相鄰，所以使用3-tap濾波器；別的邊緣像素只跟周?chē)粋€(gè)像素相鄰，所以只使用2-tap濾波器。HEVC對(duì)不同大小的變換單元使用不同強(qiáng)度的濾波器。對(duì)于4x4大小的變換單元使用FilterA所指的濾波器，對(duì)于8x8大小的變換單元使用FilterB所指的濾波器，對(duì)于16x16大小的變換單元使用FilterC所指的濾波器。3、InterPrediction3.1.MotionVectorPridictionForeachCUorPU,thebestmotionvectorpredictoriscomputedwiththeprocessspecifiedasfollows.Firstly,asetofmotionvectorpredictorcandidatesshall

be

normativeprocess,byreferringtomotionparametersofneighboringpartitions.Then,thebestonefromthecandidatesetisdeterminedbyacriterionthatselectsamotionvectorpredictorcandidatethatminimizesthecostJmotion.3.1.1.CodingModeIntheHMencoder,aninterCUcanbesegmentedintomultipleinterPUs.AninterCUcanbeencodedwithoneofthefollowingcodingmodes:MODE_SKIP,MODE_DIRECT,MODE_INTER,andMODE_MERGE.ForMODE_SKIP,MODE_DIRECTandMODE_MERGEcases,anysub-partitioningisnotallowed.Onthecontrary,fourtypesoffurtherpartitioningintheCUcanbeallowedforMODE_INTERcase.

TheHMencoderfirstevaluatesCUcodingcostswithskipanddirectmode.

Forthesecases,motionvectorpredictorforthecurrentCUisdirectlyusedascurrentmotionvector.ThemotionvectortobeusedforMODE_SKIPorMODE_DIRECTaredeterminedbycheckingallpossiblemotionvectorcandidatesandselectingthebestonethatminimizesthecostJmode.TheHMencodercomparesthecodingcostofMODE_SKIPandMODE_DIRECTfirst,takesthebettermode,andthenproceedtocheckingotherremainingvariations.3.1.2.Skip/DirectmodeIfthecodingcostJmodeissmallerthantheoneobtainedinSkip/Directmode,pred_modeissetto“MODE_MERGE”.Inthiscase,motionvectoroftheselectedleftorupperpartitionaredirectlyusedascurrentmotionparameters.Aftertheprocessspecifiedintheprevioussection,theHMencoderevaluateswhethermotionmergingshouldbeenabledornotforthecurrentCU.DecisionontheuseofmotionmergingisperformedonlyatCUlevelwhosePUsizeisSIZE_2Nx2N,whichisidenticaltoCUsize.Thedecisionontheuseofmotionmergingstartswithcheckingavailabilitiesofleftandupperneighboringpartitions.Ifthereisnoavailablepartitionformotionmerging,theHMencoderdoesnotchooseMODE_MERGEforthecurrentCU.Otherwise,thecodingcostJmodeiscomputedforavailablepartitions.3.1.3.MergemodeIftheprocesssuggestsfurtherevaluationofinterpredictionmodes,theHMencodercomputescodingcostsofCUpartitioningmodesSIZE_2Nx2NandSIZE_NxNofMODE_INTERcase.Then,theHMencoderfurtherevaluatesSIZE_2NxNandSIZE_Nx2Npartitions.Inthesecases,motionvectorsisderivedbymotionestimationprocessspecifiedinsection3.1.IfthecostJmodeofthebestPUpartitioningamongallevaluatedmodesissmallerthantheoneobtainedinMergemode,pred_modeissetto“MODE_INTER”.3.1.4.IntermodeLumainterpolationfilter1/4-sample,12-tapDCT-basedinterpolationfilter(DCT-IF)forhighefficiencyconfiguration(HE)1/4-sample,6-tapdirectionalinterpolationfilter(DIF)forlowcomplexityconfiguration(LC)Chromainterpolationfilter1/8-sample,bi-linearinterpolationfilterforbothHEandLC3.2.InterpolationfilterMotioncompensationisthekeyfactorforhighefficientvideocompressing,wherefractionalpelaccuracyrequiresinterpolationofthereferenceframe.InH.264/AVC,a6-tapfixedWienerfilterisfirstusedforhalfpelaccuracyinterpolationandthenabilinearcombinationofintegerandhalfpelvaluesisusedtoprovide?pelaccuracyinterpolation.Insteadofadaptiveinterpolationfilter.HMstilladoptsthefixed12-tapDCT-basedinterpolationfiltertoprovidefractionalpelaccuracyinterpolationbyreplacingthecombinationofWienerandbilinearfilterswithasetofinterpolationfiltersatthedesiredfractionalaccuracy.Morespecifically,onlyonefilteringprocedureisneededtoprovidetheinterpolationpixeltoanypixelaccuracy,insteadofacombinationof6-tapandbilinearfilteringproceduresinH.264/AVC.Thus,themotioncompensationprocesscanbesimplifiedintheimplementationpointofviewandthecomplexitycanalsobereducedforquarter-pelaccuracy.TheDCT-basedinterpolationfilteringprocessusesthehorizontalneighboringintegerpixelsI(x,0),wherex=-5…6,tointerpolatethehorizontalfractionalpixels,a,bandc,byusingthefollowingequations,a=\sum_{x=-5}^{6}I(x,y)*f(x,1/4)(1)b=\sum_{x=-5}^{6}I(x,y)*f(x,2/4)(2)c=\sum_{x=-5}^{6}I(x,y)*f(x,3/4)(3)

wherey=0.Intheaboveequantions,f(x,1/4),f(x,2/4)andf(x,3/4)denotethe12-tapDCT-basedinterpolationfiltercoefficientsinfractionalpositions1/4,2/4,and3/4,respectively,aslistedinTable

AsetofinterpolationfilterforlowcomplexityconfigurationisreferredtoastheDirectionalInterpolationFilter(DIF).Foreachofthethreehorizontalfractionalpositions,a,bandc,andthethreeverticalfractionalpositions,d,handn,whicharealignedwithfullpixelpositions,asingle6-tapfilterisused.ThefiltercoefficientsofDIFare{3,-15,111,37,-10,2}/128for?position(andmirroredfor?position),{3,-17,78,78,-17,3}/128for?position.

a=(3H-15I+111J+37K-10L+2M+64)>>7b=(3H-17I+78J+78K-17L+3M+64)>>7c=(2H-10I+37J+111K-15L+3M+64)>>7

d=(3B-15E+111J+37O-10S+2W+64)>>7h=(3B-17E+78J+78O-17S+3W+64)>>7

n=(2B-10E+37J+111O-15S+3W+64)>>7在高效預(yù)測(cè)模式下，HEVC仍然采用H.264中的等級(jí)B預(yù)測(cè)方式同時(shí)還增加了廣義B（GeneralizedPandBpicture,GPB）取代低時(shí)延場(chǎng)景下中的P的預(yù)測(cè)方式G。GPB預(yù)測(cè)結(jié)構(gòu)是指對(duì)傳統(tǒng)P幀采用類(lèi)似于B幀的雙向預(yù)測(cè)方式。在這種預(yù)測(cè)模式下，前向和后向參考列表中的參考圖像都必須為當(dāng)前圖像之前的圖像。對(duì)P幀采用B幀的預(yù)測(cè)方式增加了運(yùn)動(dòng)估計(jì)得準(zhǔn)確度，提高編碼效率，同時(shí)也有利于編碼流程統(tǒng)一。 4、TransformandQuantization4.1.TransformTransformsofsizesfrom4x4to32x32aresupported.Fortransformsofsize4x4and8x8,thesamemethodasthatspecifiedintheH.264/AVC[6]standardisused.Fortransformsof16x16and32x32,fastintegertransformsusingabutterflystructureareused.4.2.ScalingandQuantizationThesamequantizationmethodasinH.264/AVCisused..經(jīng)過(guò)不同的預(yù)測(cè)之后，殘差存在不同的方向性，經(jīng)過(guò)量化變換后，不同預(yù)測(cè)方向產(chǎn)生的差異性仍然存在，因此如果使用一致的zig-zag掃描，并不是最佳的掃描方法，為了克服這個(gè)缺點(diǎn)，HEVC采用水平，垂直和zig-zag三種掃描方式。為了獲得最佳效果，HEVC對(duì)不同大小的變換單元和幀內(nèi)預(yù)測(cè)模式采用查表的方式獲得其掃描方式，并且因?yàn)榱炼认袼睾蜕认袼氐哪Ｊ接泻艽蟮牟煌?，所以?duì)亮度像素和色度像素，采用不同的表格

TwoalternativeentropycodingschemesaresupportedintheHM:ContextAdaptiveBinaryArithmeticCoding(CABAC)andLowComplexityEntropyCoding(LCEC).Contextadaptivebinaryarithmeticcoding(CABAC)forhighefficiencyconfigurationLowcomplexityentropycoding(LCEC)phase2forlowcomplexityconfiguration5、EntropyCoding6、LoopFilteringThedeblockingmethodisbasedonthatusedinH.264/AVCRecallthatforblocksizesgreaterthan16×16,only16×16transformisused.Thus,for32×32and64×64blocks,de-blockingfilterisappliedonlyalongthe16×16blockedges.Thisreducesthecomputationalcomplexityforthede-blockingoperationifthebiggerblocksarechosenfrequently.4x4blockboundariesarenotdeblockedinordertoreducethecomplexity.6.1.Deblockingfilter自適應(yīng)樣點(diǎn)補(bǔ)償技術(shù)是一個(gè)自適應(yīng)選擇過(guò)程，在去塊濾波后進(jìn)行。目前自適應(yīng)樣點(diǎn)補(bǔ)償方式分為帶狀補(bǔ)償（BandOffset,BO）和邊緣補(bǔ)償（EdgeOffset,EO）兩大類(lèi)帶狀補(bǔ)償將像素值強(qiáng)度等級(jí)分為若干個(gè)條帶，每個(gè)條帶內(nèi)的像素?fù)碛邢嗤难a(bǔ)償值。進(jìn)行補(bǔ)償時(shí)，根據(jù)重構(gòu)像素點(diǎn)所處的帶選擇相應(yīng)的帶狀補(bǔ)償值進(jìn)行補(bǔ)償。現(xiàn)在的HM模型將像素值強(qiáng)度從0到最大值劃分為32個(gè)等級(jí)。同時(shí)這32個(gè)等級(jí)條帶還分為兩類(lèi)，第一類(lèi)是位于中間的16個(gè)等級(jí)條帶，剩余的為第二類(lèi)條帶。編碼時(shí)只將一類(lèi)條帶的補(bǔ)償信息寫(xiě)入片頭；另一類(lèi)條帶信息則不傳送。這樣方式