# Jensen's Inequality

### Definitions

Function $f:\;[a,b]\rightarrow\mathbb{R}\;$ is said to be convex on interval $[a,b]$ if for any $\lambda\in [0,1]\;$ and $x_1,x_2\in [a,b],$

$f(\lambda x_1+(1-\lambda )x_2)\le \lambda f(x_1)+(1-\lambda )f(x_2).$

In other words, function $f$ is convex if its graph lies below (or coincide with) any chord joining two points on the graph. A convex function whose graph has no linear pieces is called strictly convex.

A function for which the inverse inequality holds is said to be concave (respectively strictly concave).

Occasionally a convex function is said to be "convex down" or "concave up" and similarly terminology applies to the concave functions.

### Statement

Jensen's inequality is one of the most basic problem solving tools; it was published by the Danish mathematician Johann Ludwig Jensen (1859-1925) in 1906. This is an extension of the definition of convexity on a finite number of points:

$f:\;[a,b]\rightarrow\mathbb{R}\;$ be convex on interval $[a,b];$ an integer $n\ge 2;$ $\lambda_i\in [0,1],\;$ $i=1,2,\ldots,n\;$ and $\displaystyle\sum_{i=1}^{n}\lambda_i=1.\;$ Then, for any $x_1,\ldots,x_n\in [a,b],$

$\displaystyle f(\sum_{i=1}^{n}\lambda_ix_i)\le\sum_{i=1}^{n}\lambda_if(x_i).$

The equality holds only in two cases: when all $x_i'\text{s}$ are equal or when $f$ is linear (in which case all chords lie on the graph!) If the function is strictly convex the equality holds only if all $x_i'\text{s}$ are equal.

A reversed inequality holds for concave functions.

### Proof

This proof is by induction for which the definition of concavity serves as the base. Assume the statement is true for $n=m\ge 2:$

$\displaystyle\sum_{i=1}^{m}f(\lambda_ix_i)\le\sum_{i=1}^{m}\lambda_if(x_i)$

and prove it for $n=m+1,$ i.e.

$\displaystyle\sum_{i=1}^{m+1}f(\lambda_ix_i)\le\sum_{i=1}^{m+1}\lambda_if(x_i)$

Observe that, since all $x_i'\text{s}$ lie in the interval $[a,b],\;$ so does the linear combination $\displaystyle\sum_{i=1}^{m+1}\lambda_ix_i.\;$ The combination can be represented differently:

$\displaystyle\sum_{i=1}^{m+1}\lambda_ix_i=\lambda_{m+1}x_{m+1}+\sum_{i=1}^{m}\lambda_ix_i=\lambda_{m+1}x_{m+1}+(1-\lambda_{m+1})\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}x_i$

where clearly $\displaystyle\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}=1\;$ and $\displaystyle\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}x_i\in [a,b],\;$ making it possible to apply both the base of the induction and the induction hypothesis:

\displaystyle\begin{align} f\left(\sum_{i=1}^{m+1}\lambda_ix_i\right)&=f\left(\lambda_{m+1}x_{m+1}+(1-\lambda_{m+1})\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}x_i\right)\\ &\le \lambda_{m+1}f(x_{m+1})+(1-\lambda_{m+1})f\left(\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}x_i\right)\\ &\le \lambda_{m+1}f(x_{m+1})+(1-\lambda_{m+1})\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}f(x_i)\\ &= \lambda_{m+1}f(x_{m+1})+\sum_{i=1}^{m}\lambda_if(x_i)\\ &= \sum_{i=1}^{m+1}\lambda_if(x_i). \end{align}

### Examples

• Arithmetic mean - geometric mean inequality

Function $f(x)=\ln x$ is concave on any interval where it is defined. It follows that, for positive $x_i,$ $i=1,\ldots,n,$

$\displaystyle\ln\left(\frac{1}{n}\sum_{i=1}^{n}x_i\right)\ge \frac{1}{n}\sum_{i=1}^{n}\ln (x_i)=\ln\left(\prod_{i=1}^{n}x_i\right)^{\frac{1}{n}}.$

Since $\ln (x)$ is a monotone function, that is equivalent to

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}x_i\ge \left(\prod_{i=1}^{n}x_i\right)^{\frac{1}{n}}.$

More generally, with arbitrary positive coefficients: if $\displaystyle\sum_{i=1}^{n}\lambda_i=1,$ we have

$\displaystyle\sum_{i=1}^{n}\lambda_ix_i\ge \prod_{i=1}^{n}x_i^{\lambda_i}.$

• Arithmetic mean - quadratic mean inequality

Function $f(x)=x^2$ is convex. It follows that, for positive $x_i,$ $i=1,\ldots,n,$

$\displaystyle\left(\frac{1}{n}\sum_{i=1}^{n}x_i\right)^{2}\le \frac{1}{n}\sum_{i=1}^{n}x_i^{2}.$

Written another the inequality becomes

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}x_i\le \sqrt{\frac{1}{n}\sum_{i=1}^{n}x_i^{2}}.$

• Arithmetic mean - harmonic mean inequality

Function $\displaystyle f(x)=\frac{1}{x}$ is convex for $x\gt 0.$ We thus have

$\displaystyle\left(\frac{1}{n}\sum_{i=1}^{n}x_i\right)^{-1}\le \frac{1}{n}\sum_{i=1}^{n}\frac{1}{x_i}.$

Written differently,

$\displaystyle\frac{1}{n}\sum_{i=1}^{n}x_i\ge \left(\frac{1}{n}\sum_{i=1}^{n}\frac{1}{x_i}\right)^{-1}.$

• Comparison of the means in general

Function $\displaystyle f(x)=x^{s/t}$ is concave if $0\lt s\lt t.$ We thus have

$\displaystyle\left(\frac{1}{n}\sum_{i=1}^{n}x_i\right)^{s/t}\ge \frac{1}{n}\sum_{i=1}^{n}x_i^{s/t}.$

If we set $y_i^t=x_i$ we obtain

$\displaystyle\left(\frac{1}{n}\sum_{i=1}^{n}y_i^t\right)^{\displaystyle\frac{1}{t}}\ge\left(\frac{1}{n}\sum_{i=1}^{n}y_i^{s}\right)^{\displaystyle\frac{1}{s}}.$