thu15info-chiv part211星期四信息部分

第六限失真數(shù)據(jù)壓第六限失真數(shù)據(jù)壓離散無記憶信§1.1失真度函數(shù)和失真矩§率失真函數(shù)及其性率失真函數(shù)的計(jì)§§簡單情參數(shù)方程法求1§連續(xù)無記憶信§連續(xù)無記憶信3.1率失真函數(shù)的定§§解的充要條差失真度量的計(jì)連續(xù)有記憶信§§率失真函數(shù)的定高斯連續(xù)有記憶信源的率失真函25.模擬信5.模擬信§§高斯模擬信源的率失真函限失真信源編碼定理（香農(nóng)第三定理3R(D){I(X;YR(D){I(X;Yaip(bj4 R(Dmin R(Dmin 圖RD)XX1X2...X編碼器輸出為：YXX1X2...X編碼器輸出為：Y1平均失真：d(xy)d(x,)kkNkRN(D)min{I(X；Y)，E{d(x,y)}Q平均失真：E{d(xyp(x)q(yx)d(xy)1R(D)NRNN5限失真數(shù)據(jù)限失真數(shù)據(jù)壓 R(D):給定允許失真D,表示一個(gè)信源符號(hào)可以被壓縮R(D)比特D增大,R(D)減小,可以進(jìn)一步提高壓縮率離散無記憶信源的二進(jìn)制限失真壓縮X(X1X2...XN Z(Z1Z2...Zk NE[d(xi,yi)]N比特Z1Z2Zk表示k個(gè)信源符號(hào)X1X2XN的平均失真不大于6數(shù)據(jù)處理I(X;Y)RN(D)I(X數(shù)據(jù)處理I(X;Y)RN(D)I(XYI(Z;YH(Zk壓縮后每個(gè)信源符號(hào)1RR(D)NNN例H(Y)log2KK1log(K1)R(1)log(2K)1log(2K1)227)XX1X2...XXX1X2...XY(Xi取值集合可以不同Nd(x,y)d(xk,ykk在相同s8 R(D(s))R(Dk D(s)Dkk k矢量量矢量量Rk到一包含N個(gè)輸出（重構(gòu)）C的映射，即QRk其中Cy1y2yNyRki1,2,...,N}。集合C量，為Rk中的矢量大小為N。碼書的N個(gè)元素稱為碼字或碼y yj1yj2...y矢量量91980年矢量量化器設(shè)計(jì)算法LBG量化過:11:4Q:y241980年矢量量化器設(shè)計(jì)算法LBG量化過:11:4Q:y24:kR33:5:NN胞腔：Rix:Q(x)yiRiiRk,RiRj,i 2N[R(D)]，這樣碼率RlogR(DR(D),而此時(shí)的平均失真DDN意義：存在信源編碼（編碼器），其平均失真D有限的失真測度。則對(duì)任意D0,0,0,以及任意足夠長的碼長N,一定存在一種信源編碼C其碼字的個(gè)數(shù)：MeN[R(D)],而編碼后的平均失真DD 失真聯(lián)合設(shè)XY空間的聯(lián)合概率分布為 失真聯(lián)合設(shè)XY空間的聯(lián)合概率分布為p(xy)，其函數(shù)為d(x,y)。若對(duì)任意0有N長序列對(duì)（x,滿1logp(x)H(X)N1logp(y)H(Y)N1logp(xy)H(XY)Nd(x,y)E[d(x,y)]1logp(xy)H(XY)Nd(x,y)E[d(x,y)]則稱（xy）為失真聯(lián)合典型序列（失真聯(lián)合典型序列）（d用(XY(XY){(x,y)XY（d( （d定理設(shè)序列XX1X2...XNYY1Y定理設(shè)序列XX1X2...XNYY1Y2...YN,N（dP((x,y)（d(XY),即滿足式(6.1),根據(jù)大數(shù)定理得1logp(x)依概率收斂于E[logp(xHXN于是對(duì)于任意小的正數(shù)0,0,limP[1logp(x)H(X)]NN即存在一個(gè)N1,當(dāng)NN1Pr[1logp(x)H(即存在一個(gè)N1,當(dāng)NN1Pr[1logp(x)H(X)]1N同理，存在N2N3N4,當(dāng)NN2N3N4Pr[1logp(y)H(Y)]1N1logp(xy)H(XY)]1Pr[NPr[d(x,y)E[d(x,y)]]1選擇Nmax(N1N2N3N4),(XY))P(證明：由失真聯(lián)合典型序列的定義(x,y)（d，，證明：由失真聯(lián)合典型序列的定義(x,y)（d，，XY),可以得到界p(xy)2N[H(XY)]；p(x)2N[H(X)]；p(y)2N[H(Y)2N[H(XY)2N[H(X)]2N[H(Y)q(yx)p(x)p(y)2N[I(X,Y)3則，p(y)q(yx)2NIX,Y)3對(duì)所有序列對(duì)（x,XY),成p(y)q(yx)2N[I(X,Y)3定理引理證明：設(shè)函數(shù)fy引理證明：設(shè)函數(shù)fyey1y,f(0此函數(shù)的一階導(dǎo)數(shù)為fy)－ey＋10,當(dāng)y所以，對(duì)于y0，fy)0,1yey,0y(1y)neny,0yeynyn(1xy)ng(x)(1x)g(0)xg (1x)1x(1y)n1xxeyn1xe對(duì)于0xy1,n0成立不等式(1xy)n1xeyn第三定理證明(二進(jìn)制)：設(shè)信源序列XX1X第三定理證明(二進(jìn)制)：設(shè)信源序列XX1X2...XN是獨(dú)立同分布的隨機(jī)序列，Xn的分布為p(x)d(x,y),率失真函數(shù)為設(shè)I(q(yx))需證明，對(duì)于任意RR(D)時(shí)，存在一種編碼方式，使信源的信息傳輸率為R,平均失真DDN在Y空間中，按輸出的概率分布p(y)pyn選取M2NR個(gè)隨機(jī)序列作為碼字。這M書C，用書C，用[1,2,...,2NR]作為M個(gè)碼字的標(biāo)號(hào)。將此碼書呈現(xiàn)20將X空間中的所30在譯碼器的輸出端再現(xiàn)序列y()若碼書中存在一個(gè)標(biāo)號(hào)為的碼字，[1,2,...,2NR使(x,y(G(dXY),則x編碼為對(duì)應(yīng)的碼字若碼書中存在幾個(gè)標(biāo)號(hào)為的碼字，[1,2,...,2NR]，使(x,y())G(d)(XY),則將x編碼為一個(gè)最小的標(biāo)號(hào)對(duì)應(yīng)的碼字若對(duì)于信源序列x,不存在一個(gè)，使x與y()構(gòu)成失真聯(lián)合典型序列，則將x編碼為1對(duì)應(yīng)的碼字。40：平均將失真函數(shù)40：平均將失真函數(shù)對(duì)所有可能隨機(jī)選取的碼書統(tǒng)計(jì)平均D(C)E[d(x,對(duì)于某一固定的碼書C和0,(1)所有x:在碼書C中存在，使(x,y())G(dXY則d(x,y())D所以這些失真典型序列對(duì)的貢獻(xiàn)最多為D中不))(dx:在碼書(x,(XY，中不))(dx:在碼書(x,(XY，(dXY)。設(shè)所有這些序概率為Pe,則這些信源序列對(duì)D(C)的貢獻(xiàn)最多為D(C)DPeDmaxDmax)J(C){x(x,(d(XY則Pe是由信源序列xJ(C)引起的，于Pex:xJ(CC所有不能用碼字來描述的信源序列的概率對(duì)所有不能用碼字來描述的信源序列的概率對(duì)所可能產(chǎn)生的隨機(jī)碼書進(jìn)行統(tǒng)計(jì)平Pep(x)C:xJ(Cx定義函(x,y)G(d)(XYK(x,y) (x,y)G(d)(XY0p{y:(x,y)G(d)p{y:(x,y)G(d)(XY)}p{y:K(x,y)1p(y)K(x,yp{y:(x,y)G(d)(XY)}YN中隨機(jī)選取的碼字不與信列x構(gòu)成失真聯(lián)合典型序列的概率p(C)[1p(y)K(x,C:xJ(CyPp(x)[1p(y)K(x,exyPp(x)[1p(y)K(x,ePp(x)[1p(y)K(x,exy由定理p(y)K(x,y)q(yx)2N[I(X;Y)3]K(x,yyxy2N[I(X;Y)3Pq(yx)2K(x,e[12N[I(X;Y)3]q(yx)K(x,y)]2NRy1q(yx)K(x,y)e2N[I(X;Y)3]yP1p(x)q(yx)K(x,y)e2N[RI(X;Y)3得e 選擇試R(D)選擇IX;Y)信道選擇試R(D)選擇IX;Y)信道p,充分小時(shí)，e2NRRD)3則當(dāng)而(XY)}p(xy)K(x,y)p{(x,y)(dGxy則對(duì)任意0,當(dāng)N,充分小時(shí)D(C)D所以，至少存在一種碼書C，其碼字的2N[R(D)為],而碼的平均失D(C)D證明：NRH(YH(YH(YXI(XN證明：NRH(YH(YH(YXI(XNH(Xn)H(XNNNH(Xn)H(XnYn)I(Xn;YnNNNNNR(D)R(D)Dnnn對(duì)與任何平均失真小于D的率失真碼（2NR,必有RR(D)對(duì)理論對(duì)理論的實(shí)質(zhì)的理,Q qjexp(sd)，i1,2,...,K,j1,2,..., KKipiexp(sdij)1,(i1,2,...,K),當(dāng)qj 0(類似于(類似于信道容量解的充要條件定理，應(yīng)用Kuhn-Tucker定理其中 qjexp(sdijj 此時(shí)有：Dpqjexp(sd jKR(Ds)pilogiKs{(1,2,...,K):iKs{(1,2,...,K):ipiexp(sdij)對(duì)s0,s試驗(yàn)信道QK I(P,Q)sDlogiI(P,Q)sDlog j I(P,Q)slogpiqij j j qpq j KR(D) ＝(1,2,...,K) 基本不等pq[1ij q j 基本不等pq[1ij q j J1q1piqj) jjK得，I(PQ)sDpilogKR(D)sDpilogKmax{sDlogR(D)piKR(D)max{sDplogiiKR(D)sD由解的充要條件定理和sKR(D)sD由解的充要條件定理和s0,logKmax{sDplogR(D)即iiKR(D)max{sDplogiiQ×Q11)Q2,Q×D回 p exp(sdDjs j回 p exp(sdDjs jKR(Ds)pilogiR(D){I(X;Yq(y計(jì)算對(duì)于離散無記憶信源具有對(duì)稱性，有解析參數(shù)方程迭代算KR(D) {pilogi＝(1,2,...,K K其 s{(1,2,...,K):i0，ipiexp(sdij)R(D)minI(X;YDR(D) {I(X;Yp(bjaiR[Dmin,Dmax 連續(xù)無記憶信 率失真函數(shù)的定(1)基本概連續(xù)無記憶信 率失真函數(shù)的定(1)基本概設(shè)連續(xù)無記憶信源,取值于整個(gè)實(shí)數(shù)域連續(xù)隨機(jī)變量Y,取值于整個(gè)實(shí)數(shù)域?？啥xX與Y實(shí)數(shù)dxy)為失真函數(shù)。其轉(zhuǎn)移概率密度函數(shù)為qyx)I(X;Y)h(Y)X信道的互信息為確定一允許失真D滿足DDBD{q(yx):DDE[d(x,y)]p(x)q(yx)d(x,p(x)infd(p(x)infd(x,yp(x)d(x,yR(D)[Dmin,DmaxR(D) {I(X;Yq(y d(x,y)(x d(x,y)xx d(x,y)x x d(x,y) x為D允許信道的充分必要條件是存在一個(gè)輸出密度函數(shù)pyq為D允許信道的充分必要條件是存在一個(gè)輸出密度函數(shù)pyq(yx)(x)p(y)exp(sd(x,(x)p(x)exp[sd(x,y)]dx(x)p(x)exp[sd(x,y)]dxpy0py0其中(x)p(y)exp(sd(x,對(duì)于s0,R(D)和D參數(shù)s表示為Ds(x)p(x)p(y)exp[sd(x,y)]d(x,R(Ds)sDsp(x)log(x)dx定理差失真度量定理差失真度量：dxy)x(如xy)xy)2或xy)xy由定理4，取xpxK(s)為與x無關(guān)為s的函數(shù)R(D) sup{sD p(x)log (s{(x):(x)p(x)exp[sd(x,y)1dx由于xs,即－K(s由于xs,即－K(sexp[sxK(s)－exp[s(xy)]dx令xydxy)xy)(s則得－exp[s()]d選擇K(s)exp[s)]dexp[s()]d取K(s)K由定理4，R(DsD－pxlogKp(sDp(x)sDh(X)－sDh(X)－p(x)logKsDh(X)logKsDh(X)log－exp[s(定義RL(DssDhXlog－exp[s(為R(D)的下RL(Ds)關(guān)于s的一階導(dǎo)數(shù)為RL(D,s)D－exp[s()](＋(D－gs()(() exp[s( ,顯然＋g()d() exp[s( ,顯然＋g()d其中ss－exp[s(則gs()是以s為參量的隨機(jī)變量RL(Ds)對(duì)s的二階導(dǎo)的概率密度函數(shù)()2()d()()dR(D,s)ggLssE[2()]E2[()]E[()E[()]]2則RL(Ds)是關(guān)于s的上凸函數(shù)，在滿足RL(Ds的s點(diǎn)上達(dá)到極大值由R(Ds0得，gs()()dLR(D)supRL(D,R(D)supRL(D,s－gs()()dh(X)logexp[s(h(X)－[s()logexp[s()]d]gs(()logexp[s()]]g(＋gs(sh(X)－gs()loggs()dh(X)h(gs(gs( {g(gs( {g())g(D,g()}Dexp[s(h(gs())h(g(g(（使下界更小R(D)h(X)maxh(g(g((1959年h(X)maxh(g(g((x)1p(y)exp(sd(x, K(x)1p(y)exp(sd(x, K(x)p(x)Kp(y)exp(sd(x,p(y)K(s)exp(s(1K(s)exp(s( exp(s( dyp(x)p(y)gs(xp(exp(s(即對(duì)于給定的p(x)和s0，若存在滿足上式的輸出概率p(由定理4，就有R(DRLp(xpy)gs(xp(xpy)gs(xy)dy，p(x)是p(y)和gs(z)的卷積，p(x)p(y)gs(z)p(x已知，當(dāng)()確定后，gs()也可以確定,py)設(shè)Px(Py(),Gs()分別為p(x)、py)和得Px(Py(Gs(x)的特征函數(shù)1)exp(y)dp(Py2P()Px Gs (x)p(x)p(y)exp[sd(x,y)]d(x, (x)p(x)p(y)exp[sd(x,y)]d(x,(x)p(x)exp[sd(x,y)][p(y)d(x,(p(y)(K(s)()]K(s)p(g(s則R(Ds)sDsp(x)log(x)dx求得py)和(x)后可以確定qyxqyx)py(xexp[sd(x,y)],即試驗(yàn)信道可例exp(s求得py)和(x)后可以確定qyxqyx)py(xexp[sd(x,y)],即試驗(yàn)信道可例exp(sssgs()exp(s)exp(s則2exp(s1ss1exp(s)dDs2s下面求gs()的Fourier變 (x設(shè)高斯信源X：p(x) 2 2設(shè)d(xyxysgG()()exp(j)djss2P(sgG()()exp(j)djss2P(得Py(Px(),求其Fourier反變換xG(sD2(y(y12[1 ]24則當(dāng)y足夠大時(shí)y)會(huì)變?yōu)樨?fù)值只能求R(D)的下界值(D),R(D)RL(D)h(X)h(gs(1log2e2log2s1log2e2121log2e2log2s1log2e2122s2g()exp(s2則s2 (x設(shè)高斯信源X：p(x) 2 2設(shè)d(x,y)(xy)21()2dDgss21gs() 2(1))g()是的概率密度1()2dDgss21gs() 2(1))g()是的概率密度，而是一個(gè)均值為零，方差為s的高斯隨機(jī)變量()exp(j)dexp(1D2gG()ss21P() D)P22是正態(tài)分G(yxsP()exp(1N(m,2jmx21) D)22Py1) D)22Py21(y2(2p(y)]2(2則D),y,p(y)2R(D)RL(D)h(X)h(gs( 2 log2 log2 差方失真下的率失真函數(shù)的設(shè)d(xyxy)2s2gs(差方失真下的率失真函數(shù)的設(shè)d(xyxy)2s2gs()exp(s2則exp(s211D()gs()d2s,gs()2Dexp(2D下界為hXh(g(hX1log(2eD1logPes22D1Pexp(2h(XeR(D)1log(Pe 定理5設(shè)一連續(xù)非正態(tài)分布信源X,方差為定理5設(shè)一連續(xù)非正態(tài)分布信源X,方差為2,微分熵為h(X失真函數(shù)定義為dxyxy)2,則其信息率失真函數(shù)滿足h(X)1log2eDR(D)1log 證明上界設(shè)加性噪聲高斯試驗(yàn)信道YKXN證明上界設(shè)加性噪聲高斯試驗(yàn)信道YKXN其中X,N均值為exp((y1YK(XN),q(yx))2K2n2K22n2 N(2nDE[(xy)2]E[(K(xn)x)2]E[((K1)xKn)2(K1)2E[xDE[(xy)2]E[(K(xn)x)2]E[((K1)xKn)2(K1)2E[