[ Pobierz całość w formacie PDF ]
.This is due to the fact that for each long sample of message x t and recovered message y t the evaluationapproaches v1 (with probability 1).It is interesting to note that, in this system, the noise in the recovered message is actually produced by akind of general quantizing at the transmitter and not produced by the noise in the channel.It is more or lessanalogous to the quantizing noise in PCM.29.THE CALCULATION OF RATESThe definition of the rate is similar in many respects to the definition of channel capacity.In the formerZZP x; yR = Min P x; y log dxdyPx y P x P yZZwith P x and v1 = P x; y x; y dxdy fixed.In the latterZZP x; yC = Max P x; y log dxdyP x P x P ywith Px y fixed and possibly one or more other constraints (e.g., an average power limitation) of the formRRK = P x; y x; y dxdy.A partial solution of the general maximizing problem for determining the rate of a source can be given.Using Lagrange s method we considerZZP x; yP x; y log + P x; y x; y + x P x; y dxdy:P x P y50The variational equation (when we take the first variation on P x; y ) leads to, x;yPy x = B x ewhere is determined to give the required fidelity and B x is chosen to satisfyZ, x;yB x e dx = 1:This shows that, with best encoding, the conditional probability of a certain cause for various receivedy, Py x will decline exponentially with the distance function x; y between the x and y in question.In the special case where the distance function x; y depends only on the (vector) difference between xand y,x; y = x, ywe haveZ, x,yB x e dx = 1:Hence B x is constant, say , and, x,yPy x = e :Unfortunately these formal solutions are difficult to evaluate in particular cases and seem to be of little value.In fact, the actual calculation of rates has been carried out in only a few very simple cases.If the distance function x; y is the mean square discrepancy between x and y and the message ensembleis white noise, the rate can be determined.In that case we haveR = Min H x , Hy x = H x , MaxHy x2with N = x, y.But the MaxHy x occurs when y, x is a white noise, and is equal to W1 log2 eN whereW1 is the bandwidth of the message ensemble.ThereforeR = W1 log2 eQ, W1 log2 eNQ= W1 logNwhere Q is the average message power.This proves the following:Theorem 22: The rate for a white noise source of power Q and band W1 relative to an R.M.S.measureof fidelity isQR = W1 logNwhere N is the allowed mean square error between original and recovered messages.More generally with any message source we can obtain inequalities bounding the rate relative to a meansquare error criterion.Theorem 23: The rate for any source of band W1 is bounded byQ1 QW1 log R W1 logN Nwhere Q is the average power of the source, Q1 its entropy power and N the allowed mean square error.2The lower bound follows from the fact that the MaxHy x for a given x, y = N occurs in the whitenoise case.The upper bound results if we place points (used in the proof of Theorem 21) not in the best waypbut at random in a sphere of radius Q, N.51ACKNOWLEDGMENTSThe writer is indebted to his colleagues at the Laboratories, particularly to Dr.H.W.Bode, Dr.J.R.Pierce,Dr.B.McMillan, and Dr.B.M.Oliver for many helpful suggestions and criticisms during the course of thiswork.Credit should also be given to Professor N.Wiener, whose elegant solution of the problems of filteringand prediction of stationary ensembles has considerably influenced the writer s thinking in this field.APPENDIX 5Let S1 be any measurable subset of the g ensemble, and S2 the subset of the f ensemble which gives S1under the operation T.ThenS1 = TS2:Let H be the operator which shifts all functions in a set by the time.ThenH S1 = H TS2 = TH S2since T is invariant and therefore commutes with H.Hence if m S is the probability measure of the set Sm H S1 = m TH S2 = m H S2= m S2 = m S1where the second equality is by definition of measure in the g space, the third since the f ensemble isstationary, and the last by definition of g measure again.To prove that the ergodic property is preserved under invariant operations, let S1 be a subset of the gensemble which is invariant under H , andlet S2 be the set of all functions f which transform into S1.ThenH S1 = H TS2 = TH S2 = S1so that H S2 is included in S2 for all.Now, sincem H S2 = m S1this impliesH S2 = S2for all with m S2 = 0; 1 [ Pobierz całość w formacie PDF ]