Section 3.2 Closed linear subspaces
In this section we’ll note the first major complication that working in infinite dimensions introduces. In particular, we need to be careful about what we mean by the term “closed”. Recall that in finite dimensions, a subset of a vector space is called a subspace if it is closed under arbitrary linear combinations. This is closure under algebra - \(S \subset V\) is a subspace if for all \(x, y \in S\) and \(c_1, c_2 \in \F\text{,}\) we have that \(c_1 x + c_2 y \in S\text{.}\)
On the other hand, in analysis we signify by closed set the idea that any convergent sequence in a set has a limit point in the set. This is topological closure. In finite dimensions, linear subspaces are topologically closed. In infinite dimensions it is easy to demonstrate that this is not true.
Example 3.2.1.
Definition 3.2.2.
Let \(\ell_0\) denote the space of sequences in \(\C^\mathbb{N}\) which have only finitely many non-zero terms. (That is, there is some index \(n\) beyond which the sequence has only \(0\) entries.)
It is straightforward to see that \(\ell_0\) is a linear subspace of \(\ell^2\text{.}\) However, \(\ell_0\) is not a closed set inside \(\ell^2\text{.}\) To see this, we will construct a convergent sequence that converges to a point outside of \(\ell^2\text{.}\) For each \(k \in \mathbb{N}\text{,}\) the sequence
\begin{equation*}
x^k = \left(1, \frac{1}{2}, \ldots, \frac{1}{k}, 0, \ldots\right)
\end{equation*}
is an element of \(\ell_0\text{.}\) Considering each \(x^k\) as a vector in \(\ell^2\text{,}\) the sequence of vectors \((x^k)_{k=1}^\infty\) converges to the limit point
\begin{equation*}
y = \left(\frac{1}{n}\right)_{n=1}^{\infty}
\end{equation*}
in \(\ell^2\) since
\begin{align*}
\norm{x^k - y} \amp = \norm{\left(0, \ldots, 0, \frac{1}{k+1}, \ldots \right)}\\
\amp = \left(\sum_{n = k+1}^\infty \frac{1}{n^2} \right)^{1/2} \to 0.
\end{align*}
That is, \(x^k \to a\) but \(a \notin \ell_0\text{,}\) and so \(\ell_0\) is not closed in \(\ell^2\text{.}\)
When we use the term subspace, we will always mean linear subspace of a vector space. At least for normed spaces, we can “fill in the edges” of a subspace by including the limit points. Why should we want to? One of the basic lessons of real analysis in one variable is that completeness is a property of the real numbers that allows analysis to work. What we seek to do is prepare our spaces so that we can work with similar tools for spaces other than \(\R^n\text{.}\)
Theorem 3.2.3.
The closure of a normed subspace is a subspace.
Proof.
Let \(S\) be a subspace of a normed space \(V\text{,}\) and let \(\cl(S)\) denote its closure. Let \(x,y \in \cl(S)\text{.}\) Then there exist sequences \((x_n), (y_n)\) in \(S\) so that \(x_n \to x\) and \(y_n \to y\text{.}\) As addition is continuous, we get \(x_n + y_n \to x+ y\text{.}\) Because \(S\) is a subspace, \(x_n + y_n\) is in \(S\) for all \(n\text{,}\) and so \(x+y\) must be in \(cl(S)\text{.}\) We conclude that \(S\) is closed under addition. The argument that \(S\) is closed under scalar multiplication is similar. Thus, \(\cl(S)\) is a subspace of \(V\text{.}\)
We will frequently wish to work with the smallest linear subspace containing a set of vectors \(A \subset V\text{.}\) (This is similar to the notion of the span of a set of vectors in finite dimensional spaces.)
Definition 3.2.4.
Let \((V, \norm{\cdot})\) be a normed space and let \(A \subseteq V\text{.}\) The linear span of \(A\), denoted \(\spn A\text{,}\) is the intersection of all subspaces of \(V\) that contain \(A\text{.}\) The closed linear span of \(A\), denoted \(\cl \spn A\text{,}\) is the intersection of all closed linear subspaces of \(V\) that contain \(A\text{.}\)
Checkpoint 3.2.5.
Show that \(\spn A\) is the unique subspace of \(V\) which contains \(A\) and is in turn contained in every subspace of \(V\) that contains \(A\text{.}\) Likewise, \(\cl \spn A\) is the unique closed subspace of \(V\) that contains \(A\) and is in turn contained in every subspace of \(V\) that contains \(A\text{.}\)
While this might seem different from the notion of span that we learn in elementary linear algebra, these two definitions in fact coincide.
Theorem 3.2.6.
Let \(F\) denote the set of finite linear combinations of elements of \(A\text{;}\) that is,
\begin{equation*}
F = \left\{\sum_{i=1}^k c_i a_i : n \in \mathbb{N}, c_i \in \F, a_i \in A\right\}.
\end{equation*}
Then \(F = \spn A\text{.}\)
Proof.
Exercise.We can think of the elements “missing” from \(\spn A\) as the limit points of finite linear combinations of elements in \(A\text{,}\) as the next theorem shows (and which justifies the notation \(\cl \spn A\)).
Theorem 3.2.7.
For any set \(A\) contained in a normed space \(V\text{,}\) \(\cl \spn A\) is the closure of \(\spn A\text{.}\)
Proof.
By
Theorem 3.2.3,
\(\cl (\spn A)\) is a subspace of
\(V\text{.}\) It is closed and contains
\(A\text{,}\) and so by
Definition 3.2.4 it must be that
\(\cl \spn A \subseteq \cl(\spn A)\text{.}\)
On the other hand, \(\cl \spn A\) is closed and contains \(A\) and so it must be that \(\cl(\spn A) \subseteq \cl\spn A\text{.}\)
We should point out again that in some sense, there is one finite dimensional vector space over \(\F\) for each dimension \(n\text{,}\) namely \(\F^n\text{.}\) We can make this more precise by pointing out that we can always induce a Euclidean norm via coordinatization and that the norm on \(V\) is equivalent to a Euclidean norm (by which we mean that they determine the same topology).
Theorem 3.2.8.
Any two norms on a finite dimensional vector space \(V\) are equivalent.
The proof of this theorem will require some important results from real analysis. The theorems will be stated in an appendix.
Proof.
Since \(V\) is finite dimensional, let \(\{e_1, \ldots, e_n\}\) be a basis for \(V\) so that every \(v \in V\) has a representation
\begin{equation*}
v = c_1 e_1 + \ldots c_n e_n.
\end{equation*}
Define a “coordinate Euclidean norm” \(\rho\) on \(V\) by
\begin{equation*}
\rho\left(\sum_{i=1}^n c_i e_i \right) = \left(\sum_{i=1}^n \abs{c_i}^2 \right)^{1/2}.
\end{equation*}
We will show that \(\norm{\cdot}\) and \(\rho\) determine the same open sets, and hence the same topology. (That is, we will show that any open \(\rho\)-ball is contained in a \(\norm{\cdot}\)-ball and vice versa.)
Let \(v \in V = \sum_{i=1}^n c_i e_i\text{.}\) Then
\begin{align*}
\norm{x} \amp= \norm{\sum_{i=1}^n c_i e_i}\\
\amp \leq \sum_{i=1}^n \norm{c_i e_i}\\
\amp = \sum_{i=1}^n \abs{c_i} \norm{e_i}.
\end{align*}
\begin{align*}
\norm{x} \amp \leq \sum_{i=1}^n \abs{c_i} \norm{e_i}\\
\amp \leq \left(\sum_{i=1}^n \abs{c_i}^2 \right)^{1/2} \left(\sum_{i=1}^n \norm{e_i}^2 \right)^{1/2}\\
\amp = M \rho(x),
\end{align*}
where \(M\) is the fixed positive constant
\begin{equation*}
M = \left(\sum_{i=1}^n \norm{e_i}^2 \right)^{1/2}.
\end{equation*}
Then for any \(\eps > 0\text{,}\) the open \(\norm{\cdot}\)-ball of radius \(\eps\) about \(v \in V\) contains the open \(\rho\)-ball of radius \(\frac{\eps}{M}\) about \(v\text{.}\) We conclude that every \(\norm{\cdot}\)-open set in \(V\) is also \(\rho\)-open.
In the other direction, we exploit properties of the real numbers. Define a function \(f:\F^n \to \R\) by
\begin{equation*}
f(c_1, \ldots, c_n) = \norm{\sum_{i=1}^n c_i e_i}.
\end{equation*}
(We can think of this function as taking coordinates back to vectors in
\(V\text{.}\))
\(f\) is a clearly a continuous function with respect to the standard topology on
\(\F^n\text{.}\) The
Heine-Borel theorem 11.1.1 gives that the set
\begin{equation*}
U = \left\{(c_1, \ldots, c_n): \sum_{i=1}^n \abs{c_i}^2 = 1\right\}
\end{equation*}
is compact in
\(\F^n\text{.}\) (This set consists of the coordinates of those points
\(v \in V\) for which
\(\rho(v) = 1\text{.}\)) Thus, by
Theorem 11.1.3,
\(f\) attains its infimum
\(m\) on
\(U\text{.}\) Since the
\(e_i\) are a basis and thus linearly independent,
\(m\) cannot be
\(0\text{,}\) and so
\(m > 0\text{.}\) Thus, we have shown that
\(\norm{v} \geq m > 0\) whenever
\(\rho(v) = 1\text{.}\) By
Definition 3.1.3 (2), we conclude that
\begin{equation*}
\norm{v} \geq m \rho(v) \hspace{.1in} \text{ for all } v \in V.
\end{equation*}
Thus, for any \(\eps > 0\text{,}\) the \(\rho\)-ball of radius \(\eps\) about a point \(v \in V\) contains the \(\norm{\cdot}\)-ball of radius \(m \eps\) about \(v\text{.}\) We conclude that every \(\rho\)-open set is also \(\norm{\cdot}\)-open.
The correspondence between finite dimensional normed spaces and \(\F^n\) extends to the Heine-Borel theorem itself.
Corollary 3.2.9.
Closed and bounded sets in a finite dimensional normed space are compact.
Proof.
Let
\((V, \norm{\cdot})\) be a finite-dimensional normed space. Suppose that
\(K \subseteq V\) is closed and bounded. Then, since
\(\rho(x) \leq \frac{1}{m} \norm{v}\) for all
\(v \in V\text{,}\) \(K\) is also
\(\rho\)-bounded. Because
\(\norm{\cdot}\) and
\(\rho\) are equivalent norms, they determine the same topology, and so
\(K\) is also
\(\rho\)-closed. Thus, in the Euclidean space
\((V, \rho)\text{,}\) \(K\) is closed and bounded and thus compact by the
Heine-Borel theorem 11.1.1.
