Theorem 9.1.1.
Let \(\hilbert\) be a Hilbert space and suppose that \(K \in \Lop\) is a self-adjoint compact operator. Then \(K\) has an eigenvalue \(\la\) so that \(\la = \pm \norm{K}\text{.}\)
Section 9.1 The spectral theorem for compact operators
Note: work in progress.
Now let us assemble the players.
Theorem 9.1.2.
Let \(\hilbert\) be a Hilbert space and suppose that \(A \in \Lop\) is a self-adjoint operator. Then all of the eigenvalues of \(A\) are real, and eigenvalues associated to distinct eigenvectors are orthogonal.
Theorem 9.1.3.
Let \(\hilbert\) be a Hilbert space and suppose that \(K \in \Lop\) is a compact self-adjoint operator. Then the eigenvalues of \(K\) are a finite set of real numbers, or the eigenvalues of \(K\) are a countable sequence of real numbers tending to \(0\text{.}\)
Lemma 9.1.4.
Let \(\M\) be a closed linear subspace of a Hilbert space \(\hilbert\) and \(\M\) be invariant under \(T \in \Lop\) (that is, \(T\M \subseteq \M\)). Then \(\M^\perp\) is invariant under \(T\ad\text{.}\)
Theorem 9.1.5.
Suppose that \(\M\) is a closed linear subspace \(\M\) of a Hilbert space \(\hilbert\text{.}\) Then \(\M\) is a Hilbert space.
Proof.
The space \(\M\) inherits the inner product space structure from \(\hilbert\text{,}\) so we need only show completeness. Suppose that \((x_n)\) is a Cauchy sequence in \(\M\text{.}\) Then \((x_n)\) is also a Cauchy sequence of vectors in \(\hilbert\text{.}\) As \(\hilbert\) is complete, \(x_n \to x\) for some \(x \in \hilbert\text{.}\) As \(\M\) is closed, \(x \in \M\text{.}\) Thus, Cauchy sequences in \(\M\) converge in \(\M\) and so \(\M\) is complete and hence a Hilbert space.
Just as we can restrict the domain of a function on \(\R\) to smaller subsets (typically some kind of smaller interval), we can restrict operators on Hilbert spaces. To preserve structure, we typically restrict an operator to a subspace.
Definition 9.1.6.
Suppose that \(A \in \Lop\) where \(\hilbert\) is a Hilbert space and that \(\M\) is a linear subspace of \(\hilbert\text{.}\) The restriction of \(A\) to \(\M\), denoted \(A|_\M: \M \to \hilbert\) is defined by
\begin{equation*}
A|_\M v = A v.
\end{equation*}
We are now prepared to prove the spectral theorem for compact self-adjoint operators.
Theorem 9.1.7. Spectral theorem for compact Hermitian operators.
Let \(K\) be a compact, self-adjoint operator on a Hilbert space \(\hilbert\text{.}\) There exists a finite or infinite orthonormal sequence \((v_n)\) of eigenvectors of \(K\) with corresponding real eigenvalues \((\la_n)\) such that for all \(x \in \hilbert\text{,}\)
\begin{equation*}
Kx = \sum_{i} \la_i \ip{x}{v_i} v_i.
\end{equation*}
Proof.
The eigenvalues of \(K\) are real and a finite set or a sequence tending to 0 by Theorem 9.1.3.
We will construct a collection of eigenpairs of \(K\) by induction. By Theorem 9.1.1, we know that \(K\) has an eigenvalue \(\la_1 = \pm \norm{K}\text{,}\) for which we can pick a unit eigenvector \(v_1\text{.}\)
Let \(\hilbert_2 = \spn\{v_1\}^\perp\text{.}\) Since \(\spn\{v_1\}\) is invariant under \(K\text{,}\) as \(K(cv_1) = cKv_1 = c\la v_1\text{,}\) by Lemma 9.1.4, we have that \(\hilbert_2\) is invariant under \(K\ad\) and hence is also invariant under \(K\) by self-adjointness. By Theorem 5.5.3, \(\hilbert_2\) is closed, and so a Hilbert space by Theorem 9.1.5. Now let \(K_2 = K|_{\hilbert_2}\text{.}\) Then \(K_2\) maps \(\hilbert_2\) to \(\hilbert_2\text{.}\) \(K_2\) is a bounded linear compact operator because it is the restriction of a bounded, linear, compact operator. To see that \(K_2\) is self-adjoint, pick \(x \in \hilbert_2\) and note that for all \(y \in \hilbert_2\text{,}\)
\begin{equation*}
\ip{K_2\ad x}{y} = \ip{x}{K_2 y} = \ip{x}{Ky} = \ip{Kx}{y}.
\end{equation*}
Since \(Kx \in \hilbert_2\) and this expression holds for all \(y \in \hilbert_2\text{,}\) by Theorem 2.1.4, we get that \(K_2\ad x = K x\text{.}\) As \(x\) was arbitrary, this holds for all \(x \in \hilbert\text{,}\) and so \(K_2 \ad\) must also be the restriction of \(K\) to \(\hilbert_2\text{.}\) Thus, \(K_2 = K_2\ad\text{.}\)
Since \(K_2\) is a compact self-adjoint operator on the Hilbert space \(\hilbert_2\text{,}\) it must have an eigenvalue \(\la_2 = \pm \norm{K_2}\) and we can chose an associated unit eigenvector \(v_2 \in \hilbert_2\text{.}\) Note that since \(K v_2 = K_2 v_2 = \la_2 v_2\text{,}\) we see that \(\la_2\) is an eigenvalue of \(K\) with associated eigenvector \(v_2\text{.}\) Note further that \(v_1 \perp v_2\) as \(v_2 \in \spn\{v_1\}^\perp\text{.}\)
Now suppose that proceeding this way, we have constructed a set of mutually orthogonal unit eigenvectors \(v_1, \ldots, v_n\) with corresponding eigenvalues \(\la_1, \ldots, \la_n\) with \(\la_j = \pm \norm{K_j}\text{,}\) where \(K_1 = K\) and \(K_j\) is the restriction of \(K\) to \(\spn\{v_1, \ldots, v_{j-1}\}^\perp.\) for \(j = 2, \ldots, n\text{.}\) Letting \(\hilbert_{n+1}\) be the Hilbert space \(\spn\{v_1, \ldots, v_n\}^\perp\) and \(K_{n+1}\) the restriction of \(K\) to \(\hilbert_{n+1}\text{,}\) we can follow the same argument to conclude that \(K_{n+1}\) has an eigenvalue \(\la_{n+1} = \pm \norm{K_{n+1}}\) and an associated unit eigenvector \(v_{n+1}\) which are also an eigenpair for \(K\text{.}\) Continue the process as long as \(\norm{K_n} \neq 0\text{.}\)
If the process terminates at \(K_n = 0\text{,}\) then \(x \in \hilbert\) gives
\begin{equation*}
x - \sum_{j=1}^{n-1} \ip{x}{v_j} v_j \in \hilbert_n
\end{equation*}
(since \(\ip{x - \sum_{j=1}^{n-1} \ip{x}{v_j} v_j}{v_i} = \ip{x}{v_i} - \ip{x}{v_i} = 0\)), and so
\begin{equation*}
K_n(x - \sum_{j=1}^{n-1} \ip{x}{v_j} v_j) = 0.
\end{equation*}
Since \(K_n = K\) on \(\hilbert_n\text{,}\) this means that
\begin{equation*}
K(x - \sum_{j=1}^{n-1} \ip{x}{v_j} v_j) = 0
\end{equation*}
and so
\begin{equation*}
Kx = \sum_{j=1}^{n-1} \ip{x}{v_j} Kv_j = \sum_{j=1}^{n-1} \la_j \ip{x}{v_j}v_j.
\end{equation*}
If on the other hand the process does not terminate as \(\norm{K_n} \neq 0\) for all \(n \in \mathbb{N}\text{,}\) then consider the sequence of vectors \(y_n\) defined by
\begin{equation*}
y_n = x - \sum_{j=1}^{n-1} \ip{x}{v_j} v_j.
\end{equation*}
Then \(y_n \in \hilbert_n = \spn\{v_1, \ldots v_n\}\perp\text{.}\) Then write \(x\) as
\begin{equation*}
x = \underbrace{y_n}_{\in \spn\{v_1, \ldots, v_n\}^\perp} + \underbrace{\sum_{j=1}^{n-1} \ip{x}{v_j} v_j}_{\in \spn\{v_1, \ldots, v_n\}}.
\end{equation*}
Then Theorem 5.1.4 gives that
\begin{equation*}
\norm{x}^2 = \norm{y_n}^2 + \sum_{j=1}^{n-1} \abs{\ip{x}{v_j}}^2
\end{equation*}
which implies that \(\norm{y_n} \leq \norm{x}\text{.}\) Then
\begin{equation*}
\norm{Ky_n} = \norm{K_n y_n} \leq \norm{K_n} \norm{y_n} \leq \abs{\la_n} \norm{x}.
\end{equation*}
Of course, since \(y_n = x - \sum_{j=1}^{n-1} \ip{x}{v_j} v_j\text{,}\) this means that
\begin{equation*}
\norm{Kx - \sum_{j=1}^{n-1}\la_j \ip{x}{v_j}v_j} \leq \abs{\la_n}\norm{x}.
\end{equation*}
Since the set \(\{\la_n\}\) is infinite, by Theorem 9.1.3 it must tend to 0, so that
\begin{equation*}
\sum_{j=1}^{n-1} \la_j \ip{x}{v_j} v_j \to Kx \text{ as } n \to \infty \text{ for all } x \in \hilbert.
\end{equation*}
In other words,
\begin{equation*}
Kx = \sum_{j=1}^\infty \la_j \ip{x}{v_j} v_j.
\end{equation*}
One element of the spectral theorem that we appear to be missing in our case is that in finite dimensions is that the columns of the unitary matrix \(U\) form an orthonormal basis for \(\C^n\text{.}\) Since compact operators can only have countably many eigenvalues, it seems that our only hope of a similar result is restrict our attention to separable Hilbert spaces.
Corollary 9.1.8.
Let \(K\) be a compact self-adjoint operator on a separable Hilbert space \(\hilbert\text{.}\) Then there exists a complete orthonormal sequence \((e_n)\) of eigenvectors of \(K\) and for any \(x \in \hilbert\text{,}\)
\begin{equation*}
Kx = \sum_n \la_n \ip{x}{e_n} e_n.
\end{equation*}
Proof.
By Theorem 9.1.7, there is a finite or countably infinite sequence \((v_n)\) in \(\hilbert\) such that
\begin{equation*}
Kx = \sum_n \la_n \ip{x}{v_n} v_n.
\end{equation*}
As any term corresponding to \(\la_n = 0\) contributes nothing to the sum, we can assume that \(\la_n \neq 0\) for all \(n \in \mathbb{N}\text{.}\) Now let us consider the kernel of \(K\text{.}\) As \(\ker K\) is a closed linear subspace of \(\hilbert\text{,}\) it is also a separable Hilbert space. Let \((w_m)\) be a complete orthonormal sequence in \(\ker K\text{.}\) Each \(w_m\) is an eigenvector for \(K\) corresponding to \(\la = 0\text{.}\) By Theorem 9.1.2, because \(\la_n \neq 0\) by assumption, it must be that \(v_n \perp w_m\) for all \(n, m \in \mathbb{N}\text{.}\) Then \(\{v_n\}\cup\{w_m\}\) is an orthonormal sequence in \(\hilbert\text{.}\) It remains to show that this set is a basis for \(\hilbert\text{.}\)
Let \(x \in \hilbert\text{.}\) Since \(Kx = \sum_n \la_n \ip{x}{v_n} v_n\text{,}\)
\begin{equation*}
K(x - \sum_n \la_n \ip{x}{v_n} v_n) = 0,
\end{equation*}
and so \(x - \sum_n \la_n \ip{x}{v_n} v_n \in \ker K\text{,}\) and so we can expand it in terms of the basis \((w_m)\) to get
\begin{equation*}
x - \sum_n \la_n \ip{x}{v_n} v_n \in \ker K = \sum_m \ip{x}{w_m} w_m
\end{equation*}
which implies that
\begin{equation*}
x = \sum_n \la_n \ip{x}{v_n} v_n \in \ker K + \sum_m \ip{x}{w_m} w_m.
\end{equation*}
That is, \(\{v_n\} \cup \{w_m\}\) is a basis for \(\hilbert\text{.}\) Since the set is countable, we can index it by \(\mathbb{N}\) as \((e_n)\text{.}\)
