Normalization

We have seen that one of the requirements that a probability assignment must satisfy is normalization: the sum of the probabilities associated with each of the points in the probability space must be equal to one. This gives us an easy way to go from relative proportions, which need not satisfy normalization, to probabilities, which do. In fact, often one might think of probabilities as relative proportions that satisfy normalization. For example, suppose that the relative proportion of black to red marbles is 3 to 4. Since all the marbles are black or red,

Pr(B) + Pr(R) = 1, (1)

which just expresses the normalization requirement. Hence, there must be a number c such that

(2)

Solving for c we obtain

c=7, (3)

so that Pr(B) = 3/7 and Pr(R) = 4/7. (Note that we could also have written and obtained c=1/7; in fact, this is what we shall do from now on).

The final relative proportion between black and red marbles can be expressed both as 3 to 4 or as 3/7 to 4/7. However, while 3 and 4 are not probabilities, 3/7 and 4/7 are.

Example

Suppose that Pr(B) = p and Pr(R) = q and that the number of B’s remains unchanged, while by time t that of R’s decreases by a fraction s. Then, by time t, the relative ratio between B’s and R’s will be of p to q(1-s), where (1-s) is the fraction of R‘s remaining at time t. Now we normalize:

cp + cq(1-s) =1. (4)

Hence, solving for c,

c = 1/[p + q(1-s)]. (5)

Consequently, at time t,

Pr(B) = p/[p+q(1-s)] (6)

and

Pr(R) = q(1-s)/[p+q(1-s)]. (7)

Note that the sum of the two probabilities is equal to 1, as it should be.

So, suppose that p = 5/8, q = 3/8, and s = 1/3. Then, by plugging the figures into (6) and (7), we see that at time t,

Pr(B) = 5/7 and Pr(R) = 2/7.

Normalization becomes harder when we are dealing with sample spaces that have an infinity of points, but the treatment is analogous.