Back to the on-screen lesson ·
How two variables move together, the term it supplies in the variance of a sum, and why zero covariance is weaker than independence.
Paper packet. Every task here also exists on screen, where it is checked automatically; answers written on paper are not assessed by Nydus. When you are back at a device, enter your answers there.
You will compute a covariance from a joint distribution as the expectation of the product minus the product of the expectations, use it to find the variance of a sum, and scale it into a unit-free correlation. You will also say what a covariance cannot establish: neither a cause nor, when it is zero, independence.
Variances of independent variables add. Nothing so far says what happens when they are not independent, and the missing term is exactly what this lesson measures.
The covariance is $E[(X - \mu_X)(Y - \mu_Y)]$, a measure of how two variables move together, carried in the units of their product. The correlation $\rho$ is that divided by the two standard deviations, so it is unit-free and lies in $[-1, 1]$. Uncorrelated means the covariance is zero, which is a weaker statement than independent.
$$\operatorname{Cov}(X, Y) = E\!\left[(X - \mu_X)(Y - \mu_Y)\right] = E[XY] - E[X]E[Y].$$
Read the first form. When $X$ and $Y$ are usually above their means together, and below together, the products are mostly positive and the covariance is positive. When one is high while the other is low, the products are mostly negative. The second form is the one to compute with.
It supplies the missing term in the variance of a sum:
$$\operatorname{Var}(X + Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X, Y),$$ so variances add exactly when the covariance vanishes — which independence guarantees and is weaker than.
Covariance carries the units of both variables multiplied together, so its size means nothing on its own. Dividing it out gives the correlation $$\rho = \frac{\operatorname{Cov}(X, Y)}{\sigma_X \sigma_Y} \in [-1, 1],$$ which is unit-free, with $\pm 1$ exactly when $Y$ is an increasing or decreasing linear function of $X$.
The word to hold on to is linear. Independence implies zero covariance; zero covariance does not imply independence, and the standard counterexample is $Y = X^{2}$ for a symmetric $X$ — a variable completely determined by another, with covariance exactly zero.
Another way: picture
A cloud of points with the two means drawn as crossed lines, cutting the plane into four quadrants. Points in the upper-right and lower-left contribute positively; the other two contribute negatively. The covariance is the balance — and a cloud shaped like a parabola fills all four quadrants evenly, which is why it balances to nothing.
Another way: steps
The implications run one way, and it is worth having them in order.
| From | To | Holds |
|---|---|---|
| $X, Y$ independent | $\operatorname{Cov} = 0$ | yes |
| $\operatorname{Cov} = 0$ | $X, Y$ independent | no |
| $\operatorname{Cov} = 0$ | variances of the sum add | yes |
| $\rho = \pm 1$ | $Y$ is a linear function of $X$ | yes |
| $\operatorname{Cov} > 0$ | $X$ causes $Y$ | no |
The two that fail are the two that get published. A covariance is a statement about co-movement in a joint distribution, and it identifies neither a mechanism nor a direction.
Reading zero covariance as independence. It means no linear association. $Y = X^{2}$ with symmetric $X$ is the counterexample to keep.
Reading a covariance as a cause. It cannot distinguish $X$ causing $Y$, $Y$ causing $X$, or a third thing causing both.
Comparing covariances of different quantities. The units differ, so the sizes are not comparable. That is what correlation is for.
Forgetting the factor of two. $\operatorname{Var}(X + Y)$ carries $2\operatorname{Cov}$, because the cross term appears twice when the square is expanded.
$P(1,1) = 0.4$, with margins $P(X = 1) = 0.5$ and $P(Y = 1) = 0.6$.
$XY$ is one only in that corner.
$E[XY] = 0.4$ and $E[X]E[Y] = 0.3$.
Both off the table.
$\operatorname{Cov} = 0.1$: the two occur together more often than independence would give.
Positive association.
$X$ takes $-1, 0, 1$ equally; $Y = X^{2}$, so $Y$ takes $1, 0, 1$.
$Y$ is determined by $X$.
$E[X] = 0$ and $E[XY] = E[X^{3}] = 0$ by symmetry.
The cubes cancel.
So $\operatorname{Cov} = 0$, with the two as dependent as possible.
Linear association is all it sees.
$\operatorname{Var}(X + Y) = 4 + 9 + 2(-3) = 7$.
Twice the covariance.
$\sigma_X = 2$ and $\sigma_Y = 3$.
Square roots.
So $\rho = -3 / 6 = -0.5$.
Two indicators have $P(0,0) = 2/5$, $P(0,1) = 1/10$, $P(1,0) = 1/10$ and $P(1,1) = 2/5$. Give the two products and the covariance.
| Value | |
|---|---|
| $E[XY]$ | |
| $E[X] \, E[Y]$ | |
| $\operatorname{Cov}(X, Y)$ |
With $P(0,0) = 3/10$, $P(0,1) = 1/10$, $P(1,0) = 1/5$ and $P(1,1) = 2/5$, what is $\operatorname{Var}(X + Y)$?
Answer:
With $P(0,0) = 1/10$, $P(0,1) = 1/10$, $P(1,0) = 2/5$ and $P(1,1) = 2/5$, give $\operatorname{Var}(X)$ and $\operatorname{Var}(Y)$.
$\operatorname{Var}(X)$ is p and $\operatorname{Var}(Y)$ is q.
$X$ takes the values $-4$, $0$ and $4$ with probability one third each, and $Y = X^{2}$. What is the covariance, and what does it show?
Four sentences from a report on $354$ households. Mark the two that claim more than a covariance can establish.
This task has no paper form; do it on a device.
Lesson test: one question per skill, one attempt each, no hints. Your answers are checked when you submit.
Match each pair of quantities, measured over $110$ households, to the sign its covariance would have.
| Positive | Negative | Zero | |
|---|---|---|---|
| The height and the weight of an adult | |||
| The fare charged and the number of journeys made | |||
| Shoe size and the day of the month of birth | |||
| Heating and cooling under a thermostat set to cancel them |
You can compute a covariance, use it in the variance of a sum, and read its sign as an association. Say in your own words how two variables can have zero covariance while one of them is completely determined by the other.
9. Your turn: $\operatorname{Var}(X) = 4$, $\operatorname{Var}(Y) = 9$, $\operatorname{Cov}(X, Y) = -3$; find $\operatorname{Var}(X + Y)$ and $\rho$, step 3