Section 5.2 Bessel’s inequality
The following inequality is one of the bedrock observations of Hilbert spaces. With it, we can begin the move from finite to infinite sums of vectors as appear in orthogonal expansions.
Theorem 5.2.1. Bessel’s inequality.
If \((e_n)_{n \in \mathbb{N}}\) is an orthonormal sequence in an inner product space \(\hilbert\) then, for any \(x \in \hilbert\text{,}\)
\begin{equation*}
\sum_{n=1}^\infty \abs{\ip{x}{e_n}}^2 \leq \norm{x}^2.
\end{equation*}
Proof.
We will show the inequality holds for all finite \(n \in \mathbb{N}\) and then apply limits.
For
\(N \in \mathbb{N}\text{,}\) let
\(y_N\) be the finite series
\(y_N = \sum_{n=1}^N \ip{x}{e_n} e_n\text{.}\) Noting that
\(d = \norm{x - y_N}\text{,}\) we can apply
Theorem 5.1.7 to get
\begin{equation*}
\norm{x - y_N}^2 = \norm{x}^2 - \sum_{n=1}^N \abs{\ip{x}{e_n}}^2.
\end{equation*}
Rearranging gives for all \(N\) that
\begin{equation*}
\sum_{n=1}^N \abs{\ip{x}{e_n}} = \norm{x}^2 - \norm{x - y_N}^2 \leq \norm{x}^2.
\end{equation*}
Thus in the limit as \(N \to \infty\) we have the result.
Our goal now is to show that the Fourier series \(\sum \ip{x}{e_i} e_i\) actually represents the vector \(x\text{,}\) and so we need to fix a notion of what it means for an infinite series of vectors to converge in a normed space. The most obvious idea follows directly from the scalar case - we will say that a series conveges to a vector if the partial sums approach \(x\) in norm.
Definition 5.2.2.
Let \((V, \norm{\cdot})\) be a normed space, and let \(x_n \in V\) for \(n \in \mathbb{N}\) be a sequence. We say that \(\sum x_n\) converges and has sum \(x\text{,}\) and so write \(\sum x_n = x\text{,}\) if
\begin{equation*}
\sum_{n=1}^k x_n \to x \, \, \, \text{ as } k \to \infty.
\end{equation*}
That is, \(\norm{x - \sum_{n=1}^k x_n} \to 0\) as \(k \to \infty\text{.}\)
One of the difficulties of Banach space is the lack of geometry induced by an inner product - it is difficult to tell in general when a series converges in Banach space. In Hilbert space, there is a neat characterization in terms of the coefficients. (You should once again be reminded of \(\ell^2\text{,}\) which should continue to make you suspicious.)
Theorem 5.2.3.
Let \((e_n)_{n\in \mathbb{N}}\) be an orthonormal sequence in a Hilbert space \(\hilbert\) and let \(\la_n \in \C, n \in \mathbb{N}\text{.}\) Then \(\sum_{n=1}^\infty \la_n e_n\) converges in \(\hilbert\) if and only if
\begin{equation*}
\sum_{n=1}^\infty \abs{\la_n}^2 \lt \infty.
\end{equation*}
Proof.
\(\Rightarrow:\) Suppose that \(\sum_{n=1}^\infty \la_n e_n = x\text{.}\) For a given \(k\) and any index \(m\) beyond \(k\text{,}\) orthogonality gives
\begin{equation*}
\sum_{n=1}^m \ip{\la_n e_n}{e_k} = \sum_{n=1}^m \la_n \ip{e_n}{e_k} = \la_k.
\end{equation*}
Now let \(m \to \infty\text{.}\) Continuity of the inner product gives
\begin{equation*}
\ip{x}{e_k} = \lim_{m \to \infty} \la_k =\la_k.
\end{equation*}
Then using the equivalence above in Bessel’s inequality
Theorem 5.2.1 gives
\begin{equation*}
\sum_{n=1}^\infty \abs{\la_n}^2 = \sum_{n=1}^\infty \abs{\ip{x}{e_n}}^2 \leq \norm{x}^2 \lt \infty.
\end{equation*}
\(\Leftarrow:\) Suppose that the coefficient sequence satisfies \(\sum_{n=1}^\infty \abs{\la^2} \lt \infty\text{.}\) Then define a sequence \(x_m\) by
\begin{equation*}
x_m = \sum_{n=1}^m \la_n e_n.
\end{equation*}
For \(m, p \in \mathbb{N}\text{,}\) the Pythagorean theorem gives
\begin{align*}
\norm{x_{m+p} - x_m}^2 \amp= \norm{\sum_{n=m+1}^{m+p} \la_n e_n}^2\\
\amp = \sum_{n=m+1}^{m+p} \abs{\la_n}^2\\
\amp \to 0 \,\,\, \text{ as } m\to\infty.
\end{align*}
Thus, \((x_m)\) is a Cauchy sequence in \(\hilbert\) and therefore converges in \(\hilbert\) by completeness.
