Back to the on-screen lesson ·
Equality constraints as a stationarity condition, and the multiplier as the price of the constraint.
Paper packet. Every task here also exists on screen, where it is checked automatically; answers written on paper are not assessed by Nydus. When you are back at a device, enter your answers there.
By the end of this lesson you will be able to set up and solve a problem with equality constraints using the Lagrangian, say why the gradients of objective and constraint must be parallel at an optimum, and read the multiplier as the rate at which the optimal value changes when the constraint is loosened — with its units. You will also be able to say what a constraint qualification is for, and recognise a problem where the conditions fail because a constraint gradient vanishes.
Lagrange multipliers from Calculus III, where they were a technique for constrained extrema. Also lesson 10's unconstrained conditions. Here the technique is re-read as an optimality condition rather than a recipe, and the multiplier — which in Calculus III was a nuisance variable to be eliminated — turns out to be the most useful number in the problem.
Equality constraint: $g(x) = b$. The decision must sit exactly on this surface, not merely inside something.
Lagrangian: $L = f - \lambda(g - b)$, the objective with the constraint absorbed into it.
Lagrange multiplier $\lambda$: the extra unknown the Lagrangian introduces, one per constraint.
Stationarity: $\nabla f = \lambda \nabla g$ — the gradients are parallel, which is what $\nabla L = 0$ says.
Tangent direction: a direction along which the constraint is unchanged to first order. The only directions a feasible point may move in.
Shadow price: the multiplier read as a rate — the change in the optimal value per unit of right-hand side.
Regularity: the condition that $\nabla g \ne 0$ at the point. Where it fails, the method has nothing to say.
Minimise $f(x)$ subject to $g(x) = b$. Form the Lagrangian
$$L(x, \lambda) = f(x) - \lambda\,(g(x) - b).$$
First-order necessary condition. At a constrained local minimum $x^\star$ (with a constraint qualification, below) there is a $\lambda^\star$ with
$$\nabla f(x^\star) = \lambda^\star \nabla g(x^\star), \qquad g(x^\star) = b.$$
The first equation is $\nabla_x L = 0$; the second is $\nabla_\lambda L = 0$. So the conditions are just "the Lagrangian is stationary", and lesson 10's machinery applies unchanged.
Why parallel gradients. The directions that keep you on the surface $g = b$ are exactly those perpendicular to $\nabla g$. If $\nabla f$ had any component along such a direction, moving that way would change $f$ while staying feasible — so the point was not optimal. At an optimum $\nabla f$ has no component along the surface, and a vector perpendicular to everything perpendicular to $\nabla g$ is a multiple of $\nabla g$.
What the multiplier is. Let $v(b)$ be the optimal value as a function of the right-hand side. Then
$$\lambda^\star = \frac{dv}{db}.$$
The multiplier is a price: the rate at which the answer improves per unit of loosened constraint, in units of objective per unit of constraint. This is the single most useful output of a constrained optimization, and it is often more useful than the optimum itself — it says which constraint is worth money to relax.
Constraint qualification. The conditions can fail when $\nabla g(x^\star) = 0$, because then no multiple of it can equal a non-zero $\nabla f$. The standard assumption — the constraint gradients being linearly independent at the point — rules that out. It is a hypothesis, and lesson 9's warning applies: a conclusion quoted without checking it is not established.
Another way: picture
Contour lines of the objective, and one curve for the constraint. Walk along the constraint curve and watch the contours you cross. Where the curve is tangent to a contour, you have stopped crossing them — that is the optimum, and tangency of the curves is parallelism of their gradients.
Another way: steps
To solve a problem with equality constraints:
Minimise $x^2 + y^2$ subject to $x + y = b$. Stationarity gives $2x = \lambda$ and $2y = \lambda$, so $x = y = b/2$, and the optimal value is $v(b) = b^2/2$.
Now $\lambda = 2x = b$, and $v'(b) = b$. The two agree, as the theory says they must.
What that buys in practice: told the optimum is $b^2/2$, a manager learns one number. Told in addition that $\lambda = b$, they learn what one more unit of budget is worth right now — and can compare it against what that unit costs to obtain. Almost every real use of constrained optimization is the second conversation rather than the first.
Treating $\lambda$ as junk. It is the price. Eliminating it as fast as possible, which is what the calculus course encouraged, throws away the most actionable output.
Forgetting the sign convention. Writing $L = f + \lambda(g - b)$ instead of $f - \lambda(g-b)$ flips the sign of every multiplier. Neither is wrong; both being used in one piece of work is.
Skipping the constraint qualification. When $\nabla g = 0$ at the candidate, no multiplier exists and the conditions say nothing — even though the problem may have a perfectly good optimum.
Reading stationarity as sufficiency. The conditions are necessary. They produce candidates, exactly as in lesson 10, and something else — convexity, or a second-order check — has to decide.
It is tempting to read $\lambda$ as an amount — of resource, of constraint, of something — because every other symbol in the problem is one. It is a rate: objective per unit of constraint, and its units are the objective's units divided by the constraint's. A budget constraint in pounds with an objective in hours gives a multiplier in hours per pound. Getting this wrong makes the number meaningless and, worse, plausible: it is a number of about the right size, and it will be compared against costs it has no business being compared against. Writing the units next to the multiplier, every time, is what keeps it honest.
Maximise $xyz$ subject to $2(xy + yz + zx) = S$. Lagrangian: $xyz - \lambda(2(xy+yz+zx) - S)$.
One multiplier, one constraint.
Stationarity in $x$: $yz = 2\lambda(y + z)$; in $y$: $xz = 2\lambda(x+z)$; in $z$: $xy = 2\lambda(x+y)$.
One equation per variable.
Subtracting the first two and factoring gives $z(y - x) = 2\lambda(y-x)$, so either $x = y$ or $z = 2\lambda$. Following the symmetry through gives $x = y = z$: the cube. And $\lambda$ is the rate at which the best volume grows per unit of extra surface area — the answer to "is more material worth buying".
Symmetry solves it; the multiplier prices it.
Minimise $x$ subject to $x^3 = 0$. The constraint forces $x = 0$, so the optimum is $0$ and there is nothing hard about the problem.
The answer is obvious.
The conditions demand $1 = \lambda \cdot 3x^2$. At $x = 0$ the right side is $0$ for every $\lambda$, so no multiplier exists.
The method has nothing to say.
The gradient of the constraint vanished at the optimum, so the surface has no well-defined direction there and the parallelism argument never gets started. The lesson is not that the method is unreliable but that it has a hypothesis, and "no multiplier found" is a report about that hypothesis rather than about the problem.
A hypothesis failed, not the problem.
Lagrangian $2x + 3y - \lambda(xy - 6)$. Stationarity: $2 = \lambda y$ and $3 = \lambda x$.
Two equations from the two variables.
Dividing: $2/3 = y/x$, so $y = 2x/3$. Substituting into $xy = 6$ gives $2x^2/3 = 6$, so $x^2 = 9$ and $x = 3$ (taking the positive root, as required).
Then $y = 2$ and the value is $6 + 6 = 12$. And $\lambda = 3/x = 1$: raising the required product from $6$ to $7$ would cost about $1$ more. Note that the two terms of the objective came out equal at the optimum — a recurring pattern with product constraints, and worth recognising before doing the algebra.
Match each Lagrange-multiplier component to what it does.
| Defines the feasible equality surface | Expresses parallel gradients at a candidate | Prices a small change in the constraint right-hand side | |
|---|---|---|---|
| $g(x)=b$ | |||
| $\nabla f=\lambda\nabla g$ | |||
| $\lambda$ |
Minimise $x^2 + y^2$ subject to $x + y = 18$. Write the Lagrangian $L(x, y, l)$, using the convention $L = f - l\,(g - b)$ with $l$ for the multiplier.
Answer:
For the same problem, $2x = \lambda$ at the optimum and $x = 4/2$. What is $\lambda$?
Answer:
Build the argument that $\nabla f$ must be parallel to $\nabla g$ at a constrained optimum of $f$ subject to $g = 15$.
This task has no paper form; do it on a device.
A constraint has multiplier $5$. Estimate the change in the optimal value for each increase in its right-hand side.
| increase in the right-hand side | estimated change in the optimum | |
|---|---|---|
| One more unit | 1 | |
| $3$ more units | 3 | |
| $8$ more units | 8 |
Minimise $x$ subject to $x^3 = 0$. The Lagrange conditions have no solution. Why?
Lesson test: one question per skill, one attempt each, no hints. Your answers are checked when you submit.
For the same problem, $2x = \lambda$ at the optimum and $x = 5/2$. What is $\lambda$?
Answer:
You can solve an equality-constrained problem with a Lagrangian and say what each multiplier prices. Next: inequalities, where a constraint might not be binding at all.
10. Your turn: minimise $2x + 3y$ subject to $xy = 6$, $x, y > 0$, step 3