信息論第四章答案

p(x)

x2 又有2

px)dx1h(X p(x)logAcosxlogAdxAcosxlogcosAlogAsinx2Acosxlogcos22AlogAAcosxlogcos而 logeln1sin2xdsin1logeln(1sinx)ln(1sinx)dsin21logeln(1sinx)dsinx1logeln(1sinx)dsin (1sinx)ln(1sin

1sinln(1sinx)dsin

2ln2

2 1sin2

dsinln(1sinx)dsin

ln(1sinx)d(1sin(1sinx)ln(1sinx)22

1sinx1sin

dsin2ln2

h(X

2AlogAAloge(2ln222ln222AlogA2Aloge2Alogeln2AlogA2Aloge2而2px)dx1A2h(X)log1loge11loge1log2pxexx0,px1exx2

h(X p(x)logexlog logexlog1

h(X

loglogetlnt0logelogloglogp(x)log1exlog1ex exlog1 exlog1dxexlog log2loglogp(x)

0x 試求這隨機變量的熵。又若Y1XK(K0)Y22X，試分別求出Y1和Y2h(Y1和h(Y2。 p(x)logabx2log0logb2blogeax2ln02a3bloge2a3blogalog 由于p(x)dx1，因此a3b3h(X)2logelogalog3當

XK(K0)時 h(Y)h(X)E[log1]h(X)2logeloga當

2X

2h(Y)h(X) 1h(X)2logelogalogE[log 1p(x1x2)2 x1,x21e 求隨機變量Y1X1X2的概率密度函數(shù)，并計算變量Y1e 1e11e12p(x1x2)

11e

2p(x)p(xp(xx)

ye1h(Y)1log2e21log 是多少？又放大器的帶寬為F，問單位時間內(nèi)輸出最大信息量是多少？【4.6a1和a2之間，量X和Y的聯(lián)合概率密度函數(shù)p(xy) 試計算hXh(YhXYIX;Y。 p(x, a2b同理，p(y)b h(X)p(x)logp(x)dxlog(a2a1h(Y)p(y)logp(y)dylog(b2b1h(XY)p(x,y)logp(x,y)dxdylog(a2a1)log(b2b1I(X;Y)h(X)h(Y)h(XY)hX|YhX，當且僅X和Y統(tǒng)計獨立時（2）h(X1X2 XN)h(X1)h(X2)h(XN)，當且僅當X1X2 XN彼計 p(y)dyp(x|y)logp(x|p(y)dyp(x|y)logp(x,y)logh(X等號成立當且僅px|yp(xpxyp(xpy，因此僅X和Y統(tǒng)計h(X1X2XN)h(X1)h(X2|X1)h(X3|X1X2)h(XN|X1X2XN1h(X1X XN)h(X1)h(X2) h(XNp(x2|x1)p(x2p(x3|x1x2)p(x3p(xN|x1 xN1)p(xN即p(x1x2)p(x1)p(x2p(x1x2x3)p(x1)p(x2)p(x3p(x1 xN)p(x1)p(x2 p(xNp(x)dxxp(x)dx目標：求pxlogpx)dx的最大值。F( p(x)logp(x)dxp(x)dxp(x)logp(x)p(x)dFp(x))0dp(x)logp(xp(xxp(x)0 logp(x)logexp(x)2xlog根據(jù)p(x)dx1xp(x)dxA2xlogedx12logxp(x)dxA1logA 2xlogpxlogeA

Ah(X

【4.9】N維連續(xù)型隨機序列X1X XN，有概率密度p(X1X XN)以及EXm2。

Nlog2e(2 2)1/ h(X1X XN)h(X1)h(X2) h(XNEXm2 h(X)1log2e h(X X)1log2e21log2e2 1log2e1

12 21log2e1

NN【4.10】N維連續(xù)型隨機序列X1X2 XN，其各分量幅度分別受限為[ai,bi]。證N (biaih(X1X XN)h(X1)h(X2) h(XN為

h(Xi)logbiaih(X1X X logbNaNN biai 12，…,2，均值分別為m,m,,m。試證明YX X仍是正 隨機變量，其均值為mmi，方差22iXX是相互獨立的正態(tài)分布的隨機變量，其均值為m，方差為2 Y2X1X2，

X1Y2X1Xp(x,y p(x,

)2 2

p(x1,x2)0p(x1,x2)p(x1)p(x2p(x1)p(y2x1

p(y2 p(x1)p(y2x1 x

2 yxm2 exp

exp dx1 xm yxm22 exp 2 dx12

1 而

xm yxm 1xm yxm2 2xm22yxm2 1 1 2 1 12 1 1 2m2ym2 2m22y222ym x12x11 1 12 2 1 122 2 2 21212121211m2 x 1 12 (ymm)2 2 2 2 1 1 2m2ym2 1 12 ( 1 12 2 2 2

2p(y 2

xm yxm2exp 2 dx1

1

22

)(y2m1m2 1

11

2

2ym22 2

2

2

x 1 12

2

exp

121 12121 11 (ymm)2 因此，YXX是均值為mm，方差為22的高斯分布，同理 Y3X1X2X3，……，YX1X2XNYX1X2 N

是正態(tài)隨機變量，其均值為mmi，方差2ip(y|x)

(y1x)2/3 p(x)信源的熵hX

ex2/4 ex2/41ex1 p(x,

p(x)p(y|x) 12 y x exp 2234 23 4x24y24xy2323

x1y2 y2 exp

x1y

y2 exp 2323而

x2y

y2 p(x,y)dx

23 23 x1y

x1y 2 2 3 3 x 23 23

42e

dx

e42e

d

3 2221

42et e4因此Y0，方差為22h(Y)1log2e221log4e h(Y|X p(x)p(y|x)logp(y| 12 y2xp(x,y)

y1x p(x,y) dxdyp(x,y)logy1x

232log3log而

p(x,y)

2

xy 1 1 xy2y y 2 2 2 23p(x23

dxdy

422122

y2x4 1

y

x2

1y2x

dxdyx

y x 2

1 y x 2

421 y2x 1 2 e42dxy

2而 1 y2x

2y2x

y y x y1xe 2dy3

e d 3 3 2

3333t2et2 2 y1x 1y 2 2

1 dxy x 32p(x,

dxdy634e

2

444 6 2

42dx

2

e42y1xh(Y|X log3log

p(x,y)

2

log31log2

2 H(Y)H(Y|X 1log

1te2dt（2）t2et2dt 【4.13IX;YX的概率密度函數(shù)px)的型凸函數(shù)。 p(x)p(y|x)logp(y|x)I(X;Y

p(p(x)p(y|x) p(y| p(x)p(y|Xp1xp2x，參數(shù)01

p(y| p(x)p(y|x)

dxdy p(x)p(y|x)

1 p( 1

p(x)p(y|x)logp(y|x) p(x)p(y|x)logp(y)1 p( 1而

p1(x)p(y|x)logp(y)

p1(x,y)logp(y) p1(x,y)p(y)logp1(x|y)p(2 p(x)p(y|x)logp(2

dxdy0 p2(

p(y|x) p(yi|xip(y1 yN| xNp(y1|x1 xN)p(y2| xNy1 p(yN|x1 xNy1 yN而p(y|x xy p(x1 xNy1 yN1yN 1 N1 N p(x xy 1 N1 Np(y1 yN1yN1 N1 Np(y1 yN1| xNNNp(yi|xi p(y1 yN1yN|x1 xN p(yi|xi p(yi|xi Np(yi|xip(yN|xN

Np(yi|xip(yN1| xNy1 yN2)p(yN1|xN1p(y2| xNy1)p(y2|x2p(y1|x1 xN)p(y1|p(yN1| xNy1 yN2)p(yN1|xN1p(y2| xNy1)p(y2|x2p(y1|x1 xN)p(y1|而

p(y1 yN| xNp(y1|x1 xN)p(y2| xNy1 p(yN|x1 xNy1 yNNp(y|x) p(yi|xi【4.15X的相對熵hXpx的型凸函數(shù)。Xp1xp2x，參數(shù)01p(x)logp(x) p(x)logp(x)dx p(x)logp(x) p p(121而p1(x)logp(x)

p(x)p(x) p(x)logp(x)dx

2 p2(2【4.16】設(shè)信道輸入是連續(xù)型隨機序列X1X XN，輸出也是連續(xù)型隨機序 YN，信道傳遞概率密度為py|x)I(X1X XN YN)I(Xi;YiI(X1X XN YN)I(Xi;Yi

I(X1X XN YN p(x x,y y)log xN|y1 yN12 1 1 p(x12

xN12 p(x x,y y)logp(y1 yN|x1 xN12

1

1 p(y

yN p(x,y)logp(xi|yj)dx dy i p(x i

p(x x,y y)logp(x1|y1)p(x2|y2 p(xN|yN)dx dy 1 1 p(x)p(x p(x p(x x,y y)logp(y1|x1)p(y2|x2 p(yN|xN)dx dy 1 1 p(y)p(y p(

當信源無記憶時，即p(x1 xN)p(x1)p(x2 p(xNI(Xi;Yi)I(X1X YN p(x1|y1)p(x2|y2 p(xN|yN) xN,y1 yN) p(x

x|y

dxN y 1 1 p(x x,y y)p(x1|y1)p(x2|y2 p(xN|yN)dxx 1 1 p(x x|yy

1 1 x, y)p(x1|y1 p(xN|yN)p(y1 yN1 1 p(

x,

y logp(x1|y1 p(xN|yN)p(y1 yN dxN 等號成立當且僅當p(x1|y1)p(x2|y2) p(xN|yN)p(x1x2 xN|y1y2yN)當p(y1 yN|x1 xN)p(y1|x1) p(yN|xN)時，根據(jù)信源的無記憶性，即p( yN| xN) xN)p(y1x1 p(yN|xN)p(x1 p(xN即p(x1 xN,y1 yN