Jensen's Inequality

Definitions

Function $f:\;[a,b]\rightarrow\mathbb{R}\;$ is said to be convex on interval $[a,b]$ if for any $\lambda\in [0,1]\;$ and $x_1,x_2\in [a,b],$

$f(\lambda x_1+(1-\lambda )x_2)\le \lambda f(x_1)+(1-\lambda )f(x_2).$

In other words, function $f$ is convex if its graph lies below (or coincide with) any chord joining two points on the graph. A convex function whose graph has no linear pieces is called strictly convex.

convex function

A function for which the inverse inequality holds is said to be concave (respectively strictly concave).

concave function

Occasionally a convex function is said to be "convex down" or "concave up" and similarly terminology applies to the concave functions.

Statement

Jensen's inequality is one of the most basic problem solving tools; it was published by the Danish mathematician Johann Ludwig Jensen (1859-1925) in 1906. This is an extension of the definition of convexity on a finite number of points:

$f:\;[a,b]\rightarrow\mathbb{R}\;$ be convex on interval $[a,b];$ an integer $n\ge 2;$ $\lambda_i\in [0,1],\;$ $i=1,2,\ldots,n\;$ and $\displaystyle\sum_{i=1}^{n}\lambda_i=1.\;$ Then, for any $x_1,\ldots,x_n\in [a,b],$

$\displaystyle f(\sum_{i=1}^{n}\lambda_ix_i)\le\sum_{i=1}^{n}\lambda_if(x_i).$

The equality holds only in two cases: when all $x_i'\text{s}$ are equal or when $f$ is linear (in which case all chords lie on the graph!) If the function is strictly convex the equality holds only if all $x_i'\text{s}$ are equal.

A reversed inequality holds for concave functions.

Proof

This proof is by induction for which the definition of concavity serves as the base. Assume the statement is true for $n=m\ge 2:$

$\displaystyle\sum_{i=1}^{m}f(\lambda_ix_i)\le\sum_{i=1}^{m}\lambda_if(x_i)$

and prove it for $n=m+1,$ i.e.

$\displaystyle\sum_{i=1}^{m+1}f(\lambda_ix_i)\le\sum_{i=1}^{m+1}\lambda_if(x_i)$

Observe that, since all $x_i'\text{s}$ lie in the interval $[a,b],\;$ so does the linear combination $\displaystyle\sum_{i=1}^{m+1}\lambda_ix_i.\;$ The combination can be represented differently:

$\displaystyle\sum_{i=1}^{m+1}\lambda_ix_i=\lambda_{m+1}x_{m+1}+\sum_{i=1}^{m}\lambda_ix_i=\lambda_{m+1}x_{m+1}+(1-\lambda_{m+1})\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}x_i$

where clearly $\displaystyle\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}=1\;$ and $\displaystyle\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}x_i\in [a,b],\;$ making it possible to apply both the base of the induction and the induction hypothesis:

$\displaystyle\begin{align} f\left(\sum_{i=1}^{m+1}\lambda_ix_i\right)&=f\left(\lambda_{m+1}x_{m+1}+(1-\lambda_{m+1})\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}x_i\right)\\ &\le \lambda_{m+1}f(x_{m+1})+(1-\lambda_{m+1})f\left(\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}x_i\right)\\ &\le \lambda_{m+1}f(x_{m+1})+(1-\lambda_{m+1})\sum_{i=1}^{m}\frac{\lambda_i}{1-\lambda_{m+1}}f(x_i)\\ &= \lambda_{m+1}f(x_{m+1})+\sum_{i=1}^{m}\lambda_if(x_i)\\ &= \sum_{i=1}^{m+1}\lambda_if(x_i). \end{align}$

Examples

  • Arithmetic mean - geometric mean inequality

    Function $f(x)=\ln x$ is concave on any interval where it is defined. It follows that, for positive $x_i,$ $i=1,\ldots,n,$

    $\displaystyle\ln\left(\frac{1}{n}\sum_{i=1}^{n}x_i\right)\ge \frac{1}{n}\sum_{i=1}^{n}\ln (x_i)=\ln\left(\prod_{i=1}^{n}x_i\right)^{\frac{1}{n}}.$

    Since $\ln (x)$ is a monotone function, that is equivalent to

    $\displaystyle\frac{1}{n}\sum_{i=1}^{n}x_i\ge \left(\prod_{i=1}^{n}x_i\right)^{\frac{1}{n}}.$

    More generally, with arbitrary positive coefficients: if $\displaystyle\sum_{i=1}^{n}\lambda_i=1,$ we have

    $\displaystyle\sum_{i=1}^{n}\lambda_ix_i\ge \prod_{i=1}^{n}x_i^{\lambda_i}.$

  • Arithmetic mean - quadratic mean inequality

    Function $f(x)=x^2$ is convex. It follows that, for positive $x_i,$ $i=1,\ldots,n,$

    $\displaystyle\left(\frac{1}{n}\sum_{i=1}^{n}x_i\right)^{2}\le \frac{1}{n}\sum_{i=1}^{n}x_i^{2}.$

    Written another the inequality becomes

    $\displaystyle\frac{1}{n}\sum_{i=1}^{n}x_i\le \sqrt{\frac{1}{n}\sum_{i=1}^{n}x_i^{2}}.$

  • Arithmetic mean - harmonic mean inequality

    Function $\displaystyle f(x)=\frac{1}{x}$ is convex for $x\gt 0.$ We thus have

    $\displaystyle\left(\frac{1}{n}\sum_{i=1}^{n}x_i\right)^{-1}\le \frac{1}{n}\sum_{i=1}^{n}\frac{1}{x_i}.$

    Written differently,

    $\displaystyle\frac{1}{n}\sum_{i=1}^{n}x_i\ge \left(\frac{1}{n}\sum_{i=1}^{n}\frac{1}{x_i}\right)^{-1}.$

  • Comparison of the means in general

    Function $\displaystyle f(x)=x^{s/t}$ is concave if $0\lt s\lt t.$ We thus have

    $\displaystyle\left(\frac{1}{n}\sum_{i=1}^{n}x_i\right)^{s/t}\ge \frac{1}{n}\sum_{i=1}^{n}x_i^{s/t}.$

    If we set $y_i^t=x_i$ we obtain

    $\displaystyle\left(\frac{1}{n}\sum_{i=1}^{n}y_i^t\right)^{\displaystyle\frac{1}{t}}\ge\left(\frac{1}{n}\sum_{i=1}^{n}y_i^{s}\right)^{\displaystyle\frac{1}{s}}.$

  • Additional examples

References

  1. C. Alsina, R. B. Nelsen, Charming Proofs, MAA, 2010
  2. G. H. Hardy, J. E. Littlewood, G. Polya, Inequalities, Cambridge University Press (2nd edition) 1988.
  3. Xu Jiagu, Lecture Notes on Mathematical Olympiad Courses, v 8, (For senior section, v 2), World Scientific, 2012

 

|Contact| |Front page| |Contents| |Algebra|

Copyright © 1996-2018 Alexander Bogomolny

71470988