Back to the on-screen lesson ·
Renormalising one row of a joint table, the variable that the conditional expectations make up, and the tower property that averages them back.
Paper packet. Every task here also exists on screen, where it is checked automatically; answers written on paper are not assessed by Nydus. When you are back at a device, enter your answers there.
You will build the conditional distribution of one variable given a value of another by dividing a row of the joint table by its own total, take its expectation, and recognise that the collection of those expectations is itself a random variable. You will also use the tower property to recover an expectation by conditioning on whatever makes the inside easy.
Conditioning on an event restricted the sample space and renormalised what was left. A joint table lets that be done to a whole row at once, which turns a conditional probability into a conditional distribution and then into a conditional expectation.
A conditional distribution of $Y$ given $X = x$ is one row of the joint table divided by that row's own total. $E[Y \mid X = x]$ is its mean, which is a number. $E[Y \mid X]$ is the random variable whose value is that number for each $x$. The tower property says the expectation of that variable is $E[Y]$.
Given $P(X = x) > 0$, the conditional distribution of $Y$ given $X = x$ is $$p_{Y \mid X}(y \mid x) = \frac{p(x, y)}{p_X(x)},$$ which is the row of the joint table at $X = x$, divided by that row's total. It is a genuine distribution: its entries are non-negative and they add to one.
The conditional expectation is that distribution's mean: $$E[Y \mid X = x] = \sum_{y} y \, p_{Y \mid X}(y \mid x).$$ That is a number, one for each $x$. Letting $x$ range gives a function of $X$, written $E[Y \mid X]$ — and a function of a random variable is a random variable. Keeping those two objects apart is most of what makes this lesson hard: $E[Y \mid X = 3]$ is a number, $E[Y \mid X]$ is a variable.
Because it is a variable it has an expectation, and the tower property says $$E\!\left[E[Y \mid X]\right] = E[Y].$$ The proof is the law of total probability with $y$ carried along: each row total cancels against the denominator of its own conditional expectation. Read forwards it is a consistency check; read backwards it is a method — condition on whatever makes the inside easy, then average the cases.
Another way: story
A shop's daily takings depend on how many customers come in. Working out the expected takings directly means knowing the distribution of a sum of a random number of purchases. Conditioning on the number of customers makes the inside trivial — that many customers, each spending an average amount — and the tower property puts the cases back together.
Another way: steps
Twenty cases, counts shown.
| $Y=0$ | $Y=1$ | Row total | |
|---|---|---|---|
| $X=0$ | 2 | 8 | 10 |
| $X=1$ | 6 | 4 | 10 |
$E[Y \mid X = 0] = 8/10 = 0.8$ and $E[Y \mid X = 1] = 4/10 = 0.4$. So $E[Y \mid X]$ is the variable taking $0.8$ and $0.4$ with probability a half each, and its expectation is $0.6$ — which is $12/20$, the plain $E[Y]$. The tower property is that arithmetic, and it is worth doing once by hand so that it stops looking like a trick.
Confusing $E[Y \mid X = x]$ with $E[Y \mid X]$. The first is a number, the second a random variable. The tower property is unreadable until they are kept apart.
Dividing by the wrong total. A conditional distribution given $X = x$ divides by that row's total, not by the grand total and not by the column total.
Forgetting to weight. Averaging the conditional expectations without their probabilities gives the right answer only when the cases are equally likely.
Expecting $E[Y \mid X]$ to equal $E[Y]$. It does so only when the conditional expectation is the same in every case — which is a weaker condition than independence, and worth knowing is weaker.
A fair die is rolled; then that many fair coins are tossed. $Y$ is the number of heads.
The direct distribution of $Y$ is awkward.
Given $X = x$, the heads are binomial, so $E[Y \mid X = x] = x/2$.
The inside is easy.
So $E[Y] = E[X/2] = 3.5/2 = 1.75$.
The tower property, and linearity for the outside.
$E[Y \mid X = x] P(X = x) = \sum_y y \, \dfrac{p(x,y)}{p_X(x)} \, p_X(x)$.
Write the contribution out in full.
The two copies of $p_X(x)$ cancel, leaving $\sum_y y \, p(x, y)$.
That is the whole trick.
Summing over $x$ gives $\sum_{x,y} y \, p(x,y) = E[Y]$.
The tower property, proved.
The row $X = 0$ totals $0.4$, so the conditional probabilities are $0.25$ and $0.75$.
Divide the row by its total.
So $E[Y \mid X = 0] = 0.75$, and similarly $E[Y \mid X = 1] = 0.2/0.6 = 1/3$.
Weighting: $0.4(0.75) + 0.6(1/3) = 0.5 = P(Y = 1) = E[Y]$.
Two indicators have the joint probabilities $P(0,0) = 3/20$, $P(0,1) = 3/20$, $P(1,0) = 7/20$ and $P(1,1) = 7/20$. Given that $X = 0$, give the conditional distribution of $Y$ and its expectation.
| Value | |
|---|---|
| $P(Y = 0 \mid X = 0)$ | |
| $P(Y = 1 \mid X = 0)$ | |
| $E[Y \mid X = 0]$ |
With $P(0,0) = 2/5$, $P(0,1) = 1/10$, $P(1,0) = 1/10$ and $P(1,1) = 2/5$, what is $E[Y \mid X = 1]$?
Answer:
Put the steps of computing an expectation by conditioning into the order they are carried out in.
Number the steps in order (write the number in the box):
With $P(0,0) = 1/20$, $P(0,1) = 9/20$, $P(1,0) = 3/10$ and $P(1,1) = 1/5$, give $E[Y \mid X = 0]$ and $E[Y \mid X = 1]$.
$E[Y \mid X = 0]$ is p and $E[Y \mid X = 1]$ is q.
With $P(0,0) = 1/20$, $P(0,1) = 9/20$, $P(1,0) = 3/10$ and $P(1,1) = 1/5$, recover $E[Y]$ by conditioning on $X$. Give each case's weighted contribution and the total.
| Value | |
|---|---|
| Contribution of the case $X = 0$ | |
| Contribution of the case $X = 1$ | |
| $E[Y]$ |
Lesson test: one question per skill, one attempt each, no hints. Your answers are checked when you submit.
$X$ is the number showing on a fair die and $Y$ is $3$ times that number. What kind of object is $E[Y \mid X]$?
You can renormalise a row of a joint table, compute a conditional expectation, and recover the plain expectation by weighting the cases. Say in your own words how a conditional expectation given a value differs from the conditional expectation given the variable.
9. Your turn: $P(0,0) = 0.1$, $P(0,1) = 0.3$, $P(1,0) = 0.4$, $P(1,1) = 0.2$; find $E[Y \mid X = 0]$ and check the tower property, step 3