Back to the on-screen lesson · Print

Step size, error and stability

The hard threshold an explicit step must stay under, why it has nothing to do with accuracy, and what a stiff problem costs a method that has no choice but to respect it.

Paper packet. Every task here also exists on screen, where it is checked automatically; answers written on paper are not assessed by Nydus. When you are back at a device, enter your answers there.

1. What you will learn

By the end of this lesson you will be able to derive the amplification factor of a method on the test equation, find the largest step size at which an explicit method stays stable, tell an unstable run from a merely inaccurate one by what its output does, say what makes a problem stiff, and explain what an implicit method buys and what it costs.

2. What you already have

You can run Euler and Runge-Kutta steps by hand and you know what the order of a method promises: shrink the step and the error shrinks with it, at a rate the order names. Every sentence of that is about accuracy. This lesson is about a second and entirely separate limit on the step size, which does not care how accurate the formula is and which no increase in order removes.

3. Words this lesson uses

4. Two different reasons a step can be too big

Run Euler's method on $y' = \lambda y$ with $\lambda$ negative — a decaying solution, the mildest problem there is. One step gives

$$y_{n+1} = y_n + h\lambda y_n = (1 + h\lambda)\,y_n,$$

so after $n$ steps the value is $(1 + h\lambda)^{n}y_0$. Repeated multiplication by a fixed number stays bounded exactly when that number is at most one in size, so the method is stable when

$$|1 + h\lambda| < 1, \qquad \text{that is} \qquad h < \frac{2}{|\lambda|}.$$

Past that threshold the computed values grow — alternating in sign and doubling, tripling, exploding — while the true solution decays quietly to zero. This is not the answer being inaccurate. It is the answer being qualitatively, unboundedly wrong, and it appears suddenly at a definite step size rather than degrading.

So there are two bounds on $h$, and they are unrelated:

Implicit methods change the picture entirely. Backward Euler uses the slope at the new point, $y_{n+1} = y_n + h\lambda y_{n+1}$, so

$$y_{n+1} = \frac{1}{1 - h\lambda}\,y_n,$$

and for negative $\lambda$ that factor is below one for every positive $h$. There is no stability limit at all. The price is that each step solves an equation rather than evaluating a formula.

5. What stiffness actually is

Stiffness is not a property of an equation being hard, and it is not about large numbers. It is a mismatch between two timescales inside one problem.

Take a system with components decaying like $e^{-1000t}$ and $e^{-t}$. The fast one is numerically dead after a few thousandths of a second and contributes nothing whatever to the answer at $t = 5$. The slow one is what you actually want, and it is so smooth that accuracy would be satisfied by a step of $0.1$.

But the fast component is still in the equation, so it still sets the stability limit: an explicit method needs $h < 2/1000$. To reach $t = 5$ it must take at least two and a half thousand steps to trace a curve that three points would describe. The work is being done entirely to avoid an explosion, not to gain accuracy — and that is the definition of stiff.

Chemical kinetics, circuits with very different time constants and control systems are stiff as a matter of course, which is why every serious solver library offers an implicit method alongside the explicit ones.

6. Where this goes wrong

Reading an explosion as an accuracy problem. Values doubling and flipping sign are never fixed by a higher-order method; they are fixed by a shorter step. Order and stability are different axes.

Dropping the factor of two. The limit is $2/|\lambda|$, not $1/|\lambda|$. Using the second is safe and doubles the work.

Assuming Runge-Kutta has no limit. It has one — a little more generous than Euler's, about $2.78/|\lambda|$ for the classical method — and it is still a hard threshold. Fourth order buys accuracy, not immunity.

Believing an implicit method is more accurate. Backward Euler is first order, exactly like forward Euler, and on a non-stiff problem it is the worse choice: the same accuracy for the cost of solving an equation every step. It is bought for stability and for nothing else.

Shrinking the step indefinitely. Below some size the accumulated rounding error starts to grow faster than the truncation error falls, and the answer gets worse.

7. The method, step by step, and how to check it

Choosing a step size, and a method, for a problem with decaying components follows a short routine.

1. Find the rates. For $y' = \lambda y$ the rate is $\lambda$. For a linear system, the rates are the eigenvalues of its matrix; for a nonlinear one, the eigenvalues of the Jacobian along the solution.
2. Find the stability limit from the fastest rate: $h < \frac{2}{|\lambda|}$ for forward Euler, a little more for classical Runge-Kutta.
3. Find the accuracy step from the slow part you actually need: how finely must it be resolved?
4. Compare. If the accuracy step is inside the stability limit, use it with an explicit method. If the stability limit is far smaller, the problem is stiff: use an implicit method and the accuracy step.

Why the limit is $\frac{2}{|\lambda|}$. Forward Euler on $y' = \lambda y$ multiplies by $1 + h\lambda$ every step. Repeated multiplication stays bounded only if $|1 + h\lambda| \le 1$, which for negative $\lambda$ means $-1 \le 1 + h\lambda$, that is $h \le \frac{2}{|\lambda|}$. Backward Euler multiplies by $\frac{1}{1 - h\lambda}$, whose size is below $1$ for every positive $h$ when $\lambda < 0$, so it has no limit.

How to check a run. Look at the signs and sizes of successive values. A true decaying solution never alternates in sign; a run that does, with growing size, is unstable, whatever else it looks like. Halve the step: an unstable run changes completely, while a stable but inaccurate one changes a little. For an implicit step, substitute the value you found back into its equation, $y_{n + 1} = y_n + hf(x_{n + 1}, y_{n + 1})$, and confirm it holds; that is the whole check, because the step is defined by that equation.

8. In the world: a stiff chemical reaction

Chemists' reaction networks are the classic stiff problems. In Robertson's much-studied example three substances react with rate constants $0.04$, $10^{4}$ and $3 \times 10^{7}$: the fastest reaction finishes in about $10^{-7}$ seconds, while the slowest is still changing the mixture after days. The fast component vanishes almost at once and contributes nothing visible, yet it sets an explicit method's stability limit at a step of roughly $\frac{2}{10^{7}}$ seconds, a fraction of a microsecond.

Simulating a day of the reaction, $86\,400$ seconds, at that step would take hundreds of billions of steps, each one needed only to stop the dead fast component from exploding. An implicit method, which has no stability limit on a decaying component, takes steps sized to the slow chemistry and finishes in a few hundred. This is not a refinement but the difference between a calculation that runs and one that cannot, and it is why every chemical kinetics package, from combustion modelling to atmospheric chemistry, uses implicit methods.

9. In the world: circuit simulators

SPICE, the program family that engineers use to simulate electronic circuits before building them, solves the circuit's differential equations with implicit methods such as backward Euler and the trapezoidal rule. Circuits are stiff almost by nature: a chip contains resistor-capacitor pairs with time constants of nanoseconds next to others of milliseconds.

Simulating one millisecond of a circuit with a nanosecond time constant, an explicit method would need $h < 2$ nanoseconds and so at least $\frac{10^{-3}}{2 \times 10^{-9}} = 500\,000$ steps, even while nothing interesting was happening on the fast scale. An implicit method follows the slow behaviour with steps of microseconds and shortens them only when a signal actually switches. Each implicit step solves a system of equations, which costs more, but a few thousand expensive steps beat half a million cheap ones.

10. In the world: heat on a fine grid

To compute how heat spreads along a rod, engineers divide it into small segments and write one equation for each segment's temperature, a large linear system. Its fastest rate grows like $\frac{4}{\Delta x^{2}}$ for segments of length $\Delta x$: with $\Delta x = 0.01$ metres that rate is about $40\,000$ per second, and explicit Euler needs $h < \frac{2}{40\,000} = 5 \times 10^{-5}$ seconds. Halving the segments to improve the picture quarters the allowed step. This is stiffness created by the modeller's own refinement, and it is why heat and diffusion codes use implicit time stepping as a matter of course.

11. Instability is not a large error, and a higher order is not a cure

The natural reading of a run that explodes is that the step was too big and the answer is therefore very inaccurate — a difference of degree from a run that is slightly off. It is a difference of kind. Truncation error is bounded and shrinks predictably with $h$; an unstable run has a multiplier larger than one, so the error is multiplied at every step and grows geometrically without limit, and the numbers it prints bear no relation to any solution of anything. The practical consequence is that the two failures have different repairs: an inaccurate answer is improved by a higher-order method at the same step, and an unstable one is not improved by any order at all — a fourth-order method past its own threshold explodes just as completely as a first-order one. The question to ask of a bad numerical answer is therefore not how wrong is it but which of the two things went wrong, and the sign pattern of the output usually says so at a glance.

12. The same equation on each side of the threshold

1. Take $y' = -20y$, $y(0) = 1$. Compute Euler's stability limit first.

   $|1 - 20h| < 1 \quad\Rightarrow\quad 0 < h < \dfrac{2}{20} = 0.1$

   The limit first, before any stepping: each step multiplies by $1 - 20h$.

2. Find the multiplier for $h = 0.05$.

   $1 - 20 \times 0.05 = 1 - 1 = 0$

   Inside the limit.

3. Run the steps with $h = 0.05$.

   $1,\ 0,\ 0,\ 0,\ \ldots$

   Stable and heading the right way, though a poor approximation of $e^{-20t}$: stable does not mean accurate.

4. Find the multiplier for $h = 0.15$.

   $1 - 20 \times 0.15 = 1 - 3 = -2$

   Outside the limit: its size is $2$.

5. Run the steps with $h = 0.15$.

   $1,\ -2,\ 4,\ -8,\ 16,\ \ldots$

   The true solution is below $0.05$ by $t = 0.15$: the failure is sudden and total, not a gradual loss of accuracy.

13. A stiff pair, and what each method costs

1. Components decay like $e^{-1000t}$ and $e^{-t}$, wanted to $t = 10$. Find the step accuracy needs.

   $h = 0.1 \text{ for the slow part} \quad\Rightarrow\quad \dfrac{10}{0.1} = 100 \text{ steps}$

   What the answer actually needs: the fast part is gone almost at once.

2. Find the step explicit Euler's stability allows.

   $h < \dfrac{2}{1000} = 0.002 \quad\Rightarrow\quad \dfrac{10}{0.002} = 5000 \text{ steps}$

   The fastest rate sets the limit, even though it contributes nothing to the answer.

3. Compare the two costs.

   $\dfrac{5000}{100} = 50$

   Fifty times the work, spent on a component that vanished in the first hundredth of a second. That is what stiff means.

4. Find backward Euler's multiplier for the fast component.

   $\left|\dfrac{1}{1 + 1000h}\right| < 1 \text{ for all } h > 0$

   The denominator exceeds $1$ for every positive step, so there is no limit.

5. Count backward Euler's steps.

   $h = 0.1: \quad 100 \text{ steps}, \quad \text{fast factor } \dfrac{1}{101}$

   Each step solves a small linear equation, and it still wins by a wide margin.

14. Backward Euler by hand past the explicit limit

1. Take $y' = -20y$, $y(0) = 1$, $h = 0.15$, the step at which forward Euler exploded. Write one backward step.

   $y_{n + 1} = y_n + 0.15\left(-20y_{n + 1}\right) = y_n - 3y_{n + 1}$

   The slope is taken at the new point, so the unknown is on both sides.

2. Solve the step for $y_{n + 1}$.

   $4y_{n + 1} = y_n \quad\Rightarrow\quad y_{n + 1} = \dfrac{y_n}{4}$

   Add $3y_{n + 1}$ to both sides and divide by $4$.

3. Take two steps.

   $y_1 = \dfrac{1}{4} = 0.25, \qquad y_2 = \dfrac{0.25}{4} = 0.0625$

   Each step divides by $4$.

4. Compare with the exact solution $e^{-20t}$.

   $e^{-3} \approx 0.0498, \qquad e^{-6} \approx 0.0025$

   At $t = 0.15$ and $t = 0.3$.

5. Compare with forward Euler at the same step.

   $\text{forward: } 1 \to -2 \to 4; \qquad \text{backward: } 1 \to 0.25 \to 0.0625$

   Forward Euler explodes; backward Euler decays, as the true solution does.

6. Judge the accuracy honestly.

   $\dfrac{0.0625}{0.0025} = 25$

   Backward Euler is off by a factor of $25$ at $t = 0.3$: stable, not accurate. Its value is that the error stays small in absolute terms and the step can be chosen for the slow part of a stiff problem.

15. Your turn: Euler's method on $y' = -8y$ with $h = 0.3$ — what happens, and what would fix it?

1. Compute the stability limit.

   $h < \dfrac{2}{8} = 0.25 < 0.3$

   Compute the threshold before running anything.

2. Your turn: work this step out. Its working is at the end of the packet.

   Compute the multiplier at $h = 0.3$.

3. Your turn: work this step out. Its working is at the end of the packet.

   Name the fix.

16. Guided practice

A stiff system contains a component decaying like $e^{-400t}$ alongside one that changes over several seconds. Why is an implicit method used?

• Its multiplier is below one for every positive step, so accuracy alone decides the step size
• It is more accurate per step, so fewer steps are needed
• It has no truncation error, so the step size does not matter
• A stiff equation has no formula for its solution, so an explicit method cannot be applied

17. Guided practice

Complete the worked solution: the largest stable step for Euler's method on $y' = -5y$.

1. Write one Euler step and factor out $y_n$.

   $y_{n + 1} = y_n + h(-5y_n) = (1 - 5h)\,y_n$

   Every step multiplies by the same number.

2. Evaluate the multiplier at a small step, $h = 0.1$.

   $1 - 5 \times 0.1 =$ _m

   Below one in size, so a step of $0.1$ shrinks the values as the true solution does.

3. Require the multiplier to be less than one in size, and subtract $1$ from all three parts.

   $-1 < 1 - 5h < 1 \quad\Rightarrow\quad -1 - 1 < -5h < 1 - 1$

   Subtracting the same number from every part keeps the inequalities.

4. Divide all three parts by $-5$, reversing both inequalities.

   $\dfrac{-1 - 1}{-5} > h > 0 \quad\Rightarrow\quad h < \dfrac{1 + 1}{5} =$ _h

   Dividing by a negative number reverses every inequality sign.

18. Guided practice

Euler's method is applied to $y' = -10y$. For which step sizes $h$ does the computed sequence stay bounded?

This task has no paper form; do it on a device.

19. Practice

For $y' = -10y$, one Euler step multiplies the value by $1 - 10h$. Fill in the size of that multiplier for each step size.

20. Practice

What is the largest step size for which Euler's method stays stable on $y' = -5y$?

Answer:

21. Practice

A reactive chemical breaks down at a rate $20$ times the amount present each second, a fast reaction inside a much slower process that is being simulated with a long time step; its amount obeys $y' = -20y$. Apply backward Euler to $y' = -20y$ with $y(0) = 32$ and step size $h = 0.15$. Fill in its value after one step and after two, and what forward Euler gives after two steps at the same step size.

22. Somewhere new

Four runs of a solver on $y' = -2y$ and its relatives came out as described. Match each to what caused it.

23. Lesson test

Lesson test: one question per skill, one attempt each, no hints. Your answers are checked when you submit.

24. Test question

A reactive chemical breaks down at a rate $20$ times the amount present each second, a fast reaction inside a much slower process that is being simulated with a long time step; its amount obeys $y' = -20y$. Apply backward Euler to $y' = -20y$ with $y(0) = 16$ and step size $h = 0.15$. Fill in its value after one step and after two, and what forward Euler gives after two steps at the same step size.

25. What you can do now

You can compute a stability limit, recognise an unstable run from its output, and say when an implicit method is worth its price. Say in your own words why a higher-order method does not cure instability.

Working for the steps left to you

15. Your turn: Euler's method on $y' = -8y$ with $h = 0.3$ — what happens, and what would fix it?, step 2

15. Your turn: Euler's method on $y' = -8y$ with $h = 0.3$ — what happens, and what would fix it?, step 3