Back to the on-screen lesson · Print

The Gaussian channel

C = (1/2) log2(1 + P/N) per use; power constraint; Gaussian input is optimal.

Paper packet. Every task here also exists on screen, where it is checked automatically; answers written on paper are not assessed by Nydus. When you are back at a device, enter your answers there.

1. What you will learn

By the end of this lesson you will be able to explain why the Gaussian channel needs a power constraint, derive its capacity $\tfrac{1}{2} \log_2 (1 + P/N)$ from the maximum-entropy property of the Gaussian distribution, convert signal-to-noise ratios between decibels and ratios, and picture the sphere-packing argument behind the coding theorem for it. You will apply the Shannon-Hartley law $C = W \log_2 (1 + \text{SNR})$ to size real links, solve it for the bandwidth or the SNR a target rate demands, and explain the trade-off between power and bandwidth, including the finite capacity that remains when bandwidth is unlimited.

2. The Gaussian channel

The Gaussian channel is $Y = X + Z$ with real-valued input and noise $Z \sim \mathcal{N}(0, N)$ independent of $X$. Without a limit on the input it could carry infinitely many bits per use, so a power constraint $E[X^2] \le P$ is imposed. Differential entropies replace entropies: $I(X; Y) = h(Y) - h(Y \mid X) = h(Y) - h(Z)$, with $h(Z) = \tfrac{1}{2} \log_2 (2\pi e N)$, and among all distributions with variance $P + N$ the Gaussian has the largest differential entropy, $\tfrac{1}{2} \log_2 \big(2\pi e (P + N)\big)$, which the Gaussian input $X \sim \mathcal{N}(0, P)$ attains. Hence $$C = \frac{1}{2} \log_2 \left(1 + \frac{P}{N}\right) \text{ bits per use.}$$ The ratio $P/N$ is the signal-to-noise ratio (SNR), often quoted in decibels, $10 \log_{10}(P/N)$: $0$ dB gives $0.5$ bit, $10$ dB about $1.73$ bits, $20$ dB about $3.33$ bits. The coding theorem and its converse hold with a sphere-packing proof: $n$ uses give a received vector of squared length about $n(P + N)$, each codeword's noise ball has radius about $\sqrt{nN}$, and $\big((P+N)/N\big)^{n/2} = 2^{nC}$ balls fit.

3. The derivation in five lines

The Gaussian channel is $Y = X + Z$ with $Z \sim \mathcal{N}(0, N)$ independent of the input. Without a limit on $X$ it would carry infinitely many bits per use — send $x = 0, 1, 2, 3, \ldots$ spaced far apart and the noise never confuses them — so a power constraint $E[X^2] \le P$ is part of the channel. Entropies become differential entropies, and the derivation is short:

Notice what did the work: the maximum-entropy property of the Gaussian. Among all distributions of a given variance, the Gaussian has the largest differential entropy, so a Gaussian input — which makes $Y$ Gaussian too — is exactly the one that reaches the bound. The same fact appears whenever a variance is all that is known.

The sphere-packing picture is exact here. Over $n$ uses the received vector lies in a ball of radius $\sqrt{n(P + N)}$, and each codeword's noise cloud is a ball of radius $\sqrt{nN}$. The number of small balls that fit is the ratio of volumes, $$\frac{\big(n(P + N)\big)^{n/2}}{(nN)^{n/2}} = \left(1 + \frac{P}{N}\right)^{n/2},$$ whose logarithm divided by $n$ is exactly $\tfrac{1}{2} \log_2 (1 + P/N)$.

4. Solving the practice problems

1. $P/N = 4^k - 1$: then $1 + P/N = 4^k$ and $C = \tfrac{1}{2} \log_2 4^k = k$ bits per use — the ranges are chosen so the answer is a whole number.
2. $P/N = 2^k - 1$: $C = \tfrac{1}{2} \log_2 2^k = \tfrac{k}{2}$, a half-integer.
3. From decibels: the question gives both the dB figure and the ratio; use the ratio and read the capacity from the table.
4. Why the power constraint: without it, inputs could be spaced arbitrarily far apart and the capacity would be infinite; the capacity-achieving input is Gaussian.

Common mistakes

• Dropping the $\tfrac{1}{2}$: the factor is there because a real channel use carries half as much as a complex one.
• Forgetting the $1 +$: at $P/N = 1$ the capacity is $\tfrac{1}{2}$, not $0$.
• Using the dB number as the ratio: $10$ dB is a ratio of $10$, but $20$ dB is $100$, not $20$.

5. Capacity at $10$ dB

1. $10$ dB means $P/N = 10$.

   Decibels to a ratio.

2. $C = \tfrac{1}{2} \log_2 11 = \tfrac{1}{2} \cdot 3.459 = 1.730$ bits per use.

6. Why the Gaussian input

1. $I(X; Y) = h(Y) - h(Z)$ and $\text{Var}(Y) = P + N$ for any input with power $P$.

   The noise term is fixed.

2. The maximum-entropy distribution for a given variance is Gaussian, so $h(Y) \le \tfrac{1}{2} \log_2 (2\pi e (P + N))$, with equality when $X$ is Gaussian; subtract $h(Z)$ to get $C$.

7. Capacity at $P/N = 15$

1. $1 + P/N = 16 = 2^4$.

   Adding the $1$ first is what makes the logarithm easy.

2. $\log_2 16 = 4$.

3. $C = \tfrac{1}{2} \cdot 4 = 2$ bits per use.

8. Capacity at $13$ dB

1. $13$ dB is $10^{1.3} \approx 20$, so $P/N = 20$.

   $13 = 10 + 3$: multiply by ten, then double.

2. $1 + 20 = 21$ and $\log_2 21 = 4.392$.

3. $C = 2.196$ bits per use, as the table says.

9. Your turn: $P/N = 7$

1. $1 + P/N = 8$, $\log_2 8 = 3$.

2. Your turn: work this step out. Its working is at the end of the packet.

   $C = 1.5$ bits per use.

10. Guided practice

A Gaussian channel has signal-to-noise ratio $P/N = 3$. What is its capacity in bits per use?

Computed value: _answer

11. Guided practice

A Gaussian channel has $P/N = 7$. What is its capacity in bits per use? Give a fraction if needed.

Computed value: _answer

12. Practice

A Gaussian channel has a signal-to-noise ratio of $0$ dB, that is $P/N = 1$. What is its capacity in bits per use, to three decimal places?

Computed value: _answer

13. Practice

A Gaussian channel has signal-to-noise ratio $P/N = 3$. What is its capacity in bits per use?

Computed value: _answer

14. Somewhere new

Why does the Gaussian channel $Y = X + Z$, $Z \sim \mathcal{N}(0, N)$, need a power constraint $E[X^2] \le P$, and which input achieves its capacity?

• Without it the capacity is infinite; the Gaussian input $\mathcal{N}(0, P)$ is optimal because it maximises the output entropy
• The constraint keeps the noise Gaussian; a uniform input on $[-P, P]$ is optimal
• The constraint makes the capacity zero unless $P > N$

15. Lesson test

Lesson test: one question per skill, one attempt each, no hints. Your answers are checked when you submit.

16. Test question

A Gaussian channel has $P/N = 63$. What is its capacity in bits per use? Give a fraction if needed.

Computed value: _answer

17. What you can do now

You can compute the capacity of a Gaussian link from its power, noise and bandwidth. This closes the channel unit; the last unit builds the codes that approach these limits.

Working for the steps left to you

9. Your turn: $P/N = 7$, step 2