Back to the on-screen lesson ·
Estimating a derivative from samples, the order of the error each formula leaves, and why placing the samples symmetrically is worth a whole order.
Paper packet. Every task here also exists on screen, where it is checked automatically; answers written on paper are not assessed by Nydus. When you are back at a device, enter your answers there.
By the end of this lesson you will be able to apply forward, backward and centred difference formulas, derive one from Taylor expansions, state and use the order of each, and say why exactness on a low-degree polynomial is no evidence that a formula is accurate.
You know the definition of the derivative as the limit of a difference quotient, and you can write a Taylor expansion with its remainder. Numerical differentiation is what happens when the limit is not taken and the remainder is kept.
The truncation error is what a formula leaves behind because the step is not zero. A formula has order $p$ when that error is proportional to $h^{p}$. A formula is one-sided when all its samples lie on the same side of the point and centred when they are placed symmetrically about it.
Estimate $f'(x)$ from samples. The forward difference $\dfrac{f(x+h) - f(x)}{h}$ expands as $f'(x) + \tfrac12 f''(x)h + \cdots$: it is first order, and halving $h$ halves the error. The centred difference $\dfrac{f(x+h) - f(x-h)}{2h}$ expands as $f'(x) + \tfrac16 f'''(x)h^{2} + \cdots$ — the $f''$ terms cancel because the samples are symmetric — so it is second order, and halving $h$ divides the error by four, for the same two function evaluations. The second centred difference $\dfrac{f(x+h) - 2f(x) + f(x-h)}{h^{2}}$ estimates $f''$ with error $\tfrac{1}{12}f^{(4)}h^{2}$. The recipe behind all of them is the same: write every sample as a Taylor series about the point, choose weights that cancel the terms you do not want, divide by the power of $h$ that normalises what is left, and read the order off the leading leftover. Two habits follow. Symmetry is worth an order, free; where it is unavailable — at the end of a table of data — an extra sample buys the same order one-sidedly. And exactness on a test polynomial is not accuracy: a second-order formula is exact on quadratics and cubics by construction, so testing it on one measures nothing at all.
Another way: steps
Another way: example
For $f(x) = x^{3}$ at $x = 2$ with $h = \tfrac12$: forward gives $12 + 3 + 0.25 = 15.25$, centred gives $12 + 0.25 = 12.25$, and the truth is $12$. Errors $3.25$ and $0.25$ — and halving $h$ would take them to about $1.5625$ and $0.0625$: roughly halved, and exactly quartered.
Order is often guessed from how many samples a formula uses. It cannot be: the forward and centred differences use two each and differ by a whole order. What decides it is where the samples sit and how they are weighted, which is why the derivation has to be done rather than remembered. The related trap is testing: a formula run on a polynomial low enough for it to be exact returns zero error at every step and proves nothing.
$f(x+h) = f + f'h + \tfrac12 f''h^{2} + \tfrac16 f'''h^{3} + \cdots$
Expand forwards.
$f(x-h)$ is the same with the odd powers negated, so subtracting leaves $2f'h + \tfrac13 f'''h^{3} + \cdots$
The $f''$ term is gone.
Dividing by $2h$ gives $f' + \tfrac16 f'''h^{2}$: second order, for the same two evaluations.
Symmetry was free.
At the first data point there is nothing to the left, so no centred formula exists there.
Symmetry is unavailable.
$\dfrac{-3f(x) + 4f(x+h) - f(x+2h)}{2h}$ uses three samples on one side.
An extra sample instead.
Its error is $\tfrac13 f'''h^{2}$: second order again, bought with an evaluation rather than with symmetry.
Same order, different price.
$\dfrac{(x+h)^{2} - (x-h)^{2}}{2h} = \dfrac{4xh}{2h} = 2x$.
And $f'(x) = 2x$, so the error is zero at every step size.
Because the error term carries $f'''$, and a quadratic has none — which is why this is a bad test of the formula.
For $f(x) = x^{3}$ at $x = 3$ with step $h = 0.125$, work out the forward, backward and centred difference estimates of $f'(3)$.
| Estimate of the derivative | |
|---|---|
| Forward difference | |
| Backward difference | |
| Centred difference |
A forward difference and a centred difference are both used to estimate $f'(7)$, and then the step is halved. What happens to the two truncation errors?
Four difference formulas are used at $x = 1$. Match each to what it gives and at what order.
| The first derivative, first order | The first derivative, second order, by symmetry | The second derivative, second order | The first derivative, second order, using one side only | |
|---|---|---|---|---|
| The forward difference at $1$ | ||||
| The centred difference at $1$ | ||||
| The second centred difference at $1$ | ||||
| The three-point formula using $1$, $2$ and $3$ |
Put the five steps of deriving a difference formula from $3$ samples into order.
Number the steps in order (write the number in the box):
For $f(x) = x^{3}$ at $x = 4$ with $h = 0.5$, evaluate the second centred difference $\dfrac{f(x+h) - 2f(x) + f(x-h)}{h^{2}}$.
Answer:
Lesson test: one question per skill, one attempt each, no hints. Your answers are checked when you submit.
For $f(x) = x^{3}$ at $x = 1$ with $h = 0.5$, the true derivative is $3$. Give the error of the forward, the backward and the centred estimate, signs included.
Forward fwd, backward bwd, centred ctr
You can compute difference estimates and their errors and state the order of each formula. Say in your own words why two formulas using the same number of samples can have different orders.
8. Your turn: the centred difference of $f(x) = x^{2}$, at any $x$, with any $h$, step 3