Hölder means inequality
This article is translated from a Chinese article on my Zhihu account. The original article was posted at 2019-10-12 18:56 +0800.
Some stipulations:
- Without special statements, all vectors appearing in this article are $n$-dimensional vectors, $n\in\mathbb N$;
- Iteration variable $k$ always iterates over $\left[0,n\right)\cup\mathbb Z$;
- $\operatorname{sum}\vec\xi:=\sum_k\xi_k$;
- $\operatorname{prod}\vec\xi:=\prod_k\xi_k$;
- If the independent and dependent variables of function $f$ are both scalars, then define $f\!\left(\vec\xi\right):=\left(f\!\left(\xi_0\right),f\!\left(\xi_1\right),\ldots,f\!\left(\xi_n\right)\right)$;
- $\vec\xi^{\vec\eta}:=\prod_k\xi_k^{\eta_k}$;
- $\min\vec\xi:=\min_k\xi_k$;
- $\max\vec\xi:=\max_k\xi_k$;
- $\delta_{\xi,\eta}:=\begin{cases}1,&&\xi=\eta,\\0,&&\xi\ne\eta;\end{cases}$
- By saying $\vec\xi$ is congruent, all components of $\vec\xi$ are equal to each other.
Definition 1. Suppose we have samples $\vec x\in\left(\mathbb R^+\right)^n$, weights $\vec w\in\left\{\vec\xi\in\left(\mathbb R^+\right)^n\,\middle|\,\operatorname{sum}\vec\xi=1\right\}$, and parameter $p\in\left[-\infty,+\infty\right]$. Define the Hölder mean by
\[M_{p,\vec w}\!\left(\vec x\right):=\left(\vec w\cdot\vec x^p\right)^{\frac 1p}.\]Note. The function is indefinite when $p\in\left\{-\infty,0,+\infty\right\}$, but actually there exist limits
\[\lim_{p\to0}M_{p,\vec w}\!\left(\vec x\right)=\vec x^{\vec w},\] \[\lim_{p\to-\infty}M_{p,\vec w}\!\left(\vec x\right)=\min\vec x,\] \[\lim_{p\to+\infty}M_{p,\vec w}\!\left(\vec x\right)=\max\vec x.\]The limits are to be proved as theorems later. We can use them to define the Hölder mean for $p\in\left\{-\infty,0,+\infty\right\}$.
Theorem 1.
\[\lim_{p\to0}M_{p,\vec w}\!\left(\vec x\right)=\vec x^{\vec w}.\]Proof.
\[\begin{aligned} \lim_{p\to0}M_{p,\vec w}\!\left(\vec x\right) &=\lim_{p\to0}\left(\vec w\cdot\vec x^p\right)^{\frac 1p} &\text{(Definition 1)}\\ &=\lim_{p\to0}\exp\frac{\ln\!\left(\vec w\cdot\vec x^p\right)}p\\ &=\exp\lim_{p\to0}\frac{\ln\!\left(\vec w\cdot\vec x^p\right)}p\\ &=\exp\lim_{p\to0}\frac{\vec w\cdot\left(\vec x^p\ln\vec x\right)}{\vec w\cdot\vec x^p} &\text{(L'Hôpital's rule)}\\ &=\exp\!\left(\vec w\cdot\ln\vec x\right)\\ &=\vec x^{\vec w}. &\square \end{aligned}\]Theorem 2.
\[M_{p,\vec w}\!\left(\vec x\right)=M_{-p,\vec w}\!\left(\vec x^{-1}\right)^{-1}.\]Proof.
\[\begin{aligned} M_{p,\vec w}\!\left(\vec x\right) &=\left(\vec w\cdot\vec x^p\right)^{\frac 1p} &\text{(Definition 1)}\\ &=\left(\left(\vec w \cdot\left(\vec x^{-1}\right)^{-p}\right)^{-\frac1p}\right)^{-1}\\ &=M_{-p,\vec w}\!\left(\vec x^{-1}\right)^{-1} &\text{(Definition 1)}\\ &&\square \end{aligned}\]Theorem 3.
\[\lim_{p\to+\infty}M_{p,\vec w}\!\left(\vec x\right)=\max\vec x.\]Proof. Because $\forall k:\frac{x_k}{\max\vec x}\le1$, then $\lim_{p\to+\infty}\left(\frac{\vec x}{\max\vec x}\right)^p=\delta_{\max\vec x},\vec x$.
\[\begin{aligned} \lim_{p\to+\infty}M_{p,\vec w}\!\left(\vec x\right) &=\lim_{p\to+\infty}\left(\vec w\cdot\vec x^p\right)^{\frac 1p} &\text{(Definition 1)}\\ &=\left(\max\vec x\right)\lim_{p\to+\infty}\left(\vec w\cdot\left(\frac{\vec x}{\max\vec x}\right)^p\right)^{\frac 1p}\\ &=\max\vec x\left(\vec w\cdot\lim_{p\to+\infty}\left(\frac x{\max\vec x}\right)^p\right)^{\lim_{p\to+\infty}\frac 1p}\\ &=\left(\max\vec x\right)\left(\vec w\cdot\delta_{\left(\max\vec x\right),\vec x}\right)^0\\ &=\max\vec x. &\square \end{aligned}\]Theorem 4.
\[\lim_{p\to-\infty}M_{p,\vec w}\!\left(\vec x\right)=\min\vec x.\]Proof.
\[\begin{aligned} \lim_{p\to-\infty}M_{p,\vec w}\!\left(\vec x\right) &=\lim_{p\to-\infty}M_{-p,\vec w}\!\left(\vec x^{-1}\right)^{-1} &\text{(Theorem 2)}\\ &=\lim_{p\to+\infty}M_{p,\vec w}\!\left(\vec x^{-1}\right)^{-1}\\ &=\max\left(\vec x^{-1}\right)^{-1} &\text{(Theorem 3)}\\ &=\min\vec x. &\square \end{aligned}\]Theorem 5. If $p>q$, then
\[M_{p,\vec w}\!\left(\vec x\right)\ge M_{q,\vec w}\!\left(\vec x\right),\]he equality holds iff $\vec x$ is congruent.
Proof. Case 1: $p>q>0$.
Let $f:\mathbb R^+\to\mathbb R^+:\xi\mapsto\xi^{\frac pq}$, then it has second derivative
\[\frac{\mathrm d^2f\!\left(\xi\right)}{\mathrm d\xi^2}=\frac pq\left(\frac pq-1\right)\xi^{\frac pq-2}.\]Because $p>q>0$, then $\frac pq\left(\frac pq-1\right)>0$, and then $\frac{\mathrm d^2f}{\mathrm d\xi^2}>0$, i.e. $f$ is convex. Therefore, according to Jensen’s inequality,
\[\vec w\cdot f\!\left(\vec x^q\right)\ge f\!\left(\vec w\cdot\vec x^q\right),\]i.e.
\[\vec w\cdot\vec x^p\ge\left(\vec w\cdot\vec x^q\right)^{\frac pq}.\]Take $\frac1p$th power to both sides of the equation. Without changing the direction of the inequality sign, we have
\[\vec w\cdot\vec x^p\ge\vec w\cdot\vec x^q,\]i.e. (according to Definition 1)
\[M_{p,\vec w}\!\left(\vec x\right)\ge M_{q,\vec w}\!\left(\vec x\right).\]According to the condition for the equality to hold in Jensen’s inequality, the equality holds iff $\vec x$ is congruent.
Case 2: $p>q=0$.
Because the logarithm function is concave, according to Jensen’s inequality,
\[\ln\!\left(\vec w\cdot\vec x^p\right)\ge\vec w\cdot\ln\vec x^p.\]Take exponential on both sides of the equation, and we have
\[\vec w\cdot\vec x^p\ge\vec x^{p\vec w}.\]Take $\frac1p$th power to both sides of the equation. Without changing the direction of the inequality sign, we have
\[\left(\vec w\cdot\vec x^p\right)^{\frac1p}\ge\vec x^{\vec w},\]i.e. (according to Definition 1)
\[M_{p,\vec w}\!\left(\vec x\right)\ge M_{q,\vec w}\!\left(\vec x\right).\]According to the condition for the equality to hold in Jensen’s inequality, the equality holds iff $\vec x$ is congruent.
Case 3: $p=0>q$.
\[\begin{align*} M_{q,\vec w}\!\left(\vec x\right) &=M_{-q,\vec w}\!\left(\vec x^{-1}\right)^{-1} &\text{(Theorem 2)}\\ &\le M_{0,\vec w}\!\left(\vec x^{-1}\right)^{-1} &\text{(Case 2)}\\ &=M_{0,\vec w}\!\left(\vec x\right). &\text{(Theorem 2)} \end{align*}\]The equality holds iff $\vec x$ is congruent (Case 2).
Case 4: $0>p>q$.
Because $-q>-p>0$, we have
\[\begin{align*} M_{q,\vec w}\!\left(\vec x\right) &=M_{-q,\vec w}\!\left(\vec x^{-1}\right)^{-1} &\text{(Theorem 2)}\\ &\le M_{-p,\vec w}\!\left(\vec x^{-1}\right)^{-1} &\text{(Case 1)}\\ &=M_{p,\vec w}\!\left(\vec x\right). &\text{(Theorem 2)} \end{align*}\]The equality holds iff $\vec x$ is congruent (Case 1).
By all 4 cases, the original proposition is proved. $\square$
Corollary (HM-GM-AM-QM inequalities).
\[\min\vec x\le n\left(\sum\vec x^{-1}\right)^{-1} \le\left(\prod\vec x\right)^\frac1n \le\frac{\sum\vec x}n \le\sqrt{\frac{\sum\vec x^2}{n}} \le\max\vec x,\]where the equality holds iff $\vec x$ is congruent.
Proof. Let $\vec w=\left(\frac1n,\dots,\frac1n\right)$. Then according to Theorem 5,
\[M_{-\infty,\vec w}\!\left(\vec x\right) \le M_{-1,\vec w}\!\left(\vec x\right) \le M_{0,\vec w}\!\left(\vec x\right) \le M_{1,\vec w}\!\left(\vec x\right) \le M_{2,\vec w}\!\left(\vec x\right) \le M_{+\infty,\vec w}\!\left(\vec x\right),\]i.e. (according to Definition 1)
\[\min\vec x\le n\left(\sum\vec x^{-1}\right)^{-1} \le\left(\prod\vec x\right)^\frac1n \le\frac{\sum\vec x}n \le\sqrt{\frac{\sum\vec x^2}{n}} \le\max\vec x,\]where the equality holds iff $\vec x$ is congruent (Theorem 5). $\square$