Definition 4.3.1.
A subset \(A\) of a real or complex vector space is convex if, for all \(a, b \in A\) and all \(0 \lt t \lt 1\text{,}\) the point \(ta + (1-t)b \in A\text{.}\)
Section 4.3 The closest point property
One of the most useful ideas in linear algebra is projecting onto subspaces - if \(v \in \R^n\) and \(U \subseteq \R^n\) is a subspace, then \(\proj_U v\) is the closest point in \(U\) to \(v\) in the sense that the orthogonal distance from \(v\) to \(U\) is minimized (this is also called least squares approximation).
The same idea holds true in Hilbert space and applies to vectors that are functions. This fact has deep consequences - that functions can be understood through the geometry of vectors is the soul of modern approaches to functional analysis.
We should think of convex sets \(A\) as those that contain the line segments \(\{ta + (1-t)b: t \in [0,1]\}\) between all points \(a\) and \(b\) in \(A\text{.}\)
First, we show that the closest point property holds - that a given vector \(v \in V\) has a unique closest point in any appropriately “flat” set \(A\) in \(V\) (like a linear subspace). (Compare the proof below with the standard approach in finite dimensions, e.g. in S. Axler’s Linear Algebra Done Right (4th ed.), Chapter 6C).
Theorem 4.3.2. Closest point property.
Let \(A\) be a non-empty closed convex set in a Hilbert space \(\hilbert\text{.}\) For any \(x \in \hilbert\text{,}\) there is a unique point of \(A\) which is closer to \(x\) than any other point in \(A\text{.}\) That is, there is a unique point \(y \in A\) such that
\begin{equation*}
\norm{x - y} = \inf_{a \in A} \norm{x - a}.
\end{equation*}
Proof.
The proof will consist of two parts. First, we show that the infimum of distance between \(x\) and \(A\) is attained. (We should recall that the infimum itself exists is a consequence of the completeness of the real numbers, e.g.Theorem 2.6.) Second, we show that this point is unique in \(A\text{.}\)
1
www.math.ucdavis.edu/~hunter/m125b/ch2.pdfLet
\begin{equation*}
M = \inf_{a\in A} \norm{x-a}.
\end{equation*}
We assumed that \(A\) was not empty, and so \(m\) must be some finite number. Since the infimum is a limit point, we can construct a sequence \(\norm{x - y_n}\) tending to \(\norm{x - a}\text{.}\) That is, for each integer \(n\text{,}\) there exists some \(y_n\) so that
\begin{equation*}
\norm{x - y_n}^2 \lt M^2 + \frac{1}{n}.
\end{equation*}
We now move from a sequence in norm to a sequence in \(A\) by showing that \(y_n\) is Cauchy in \(A\) - that is, we need to show that \(\norm{y_n - y_m}\) is small. We first apply Theorem 2.2.8 to the sequences \(x - y_n\) and \(x - y_m\text{.}\) For any \(m,n \in \mathbb{N}\text{,}\) we have
\begin{align*}
\amp\norm{(x - y_n) - (x - y_m)}^2 + \norm{(x - y_n)+(x - y_m)}^2\\
\amp= 2 \norm{x - y_n}^2 + 2\norm{x - y_m}^2\\
\amp\lt 4M^2 + 2\left(\frac{1}{n} + \frac{1}{m}\right).
\end{align*}
The term we are trying to bound is \(\norm{(x - y_n) - (x - y_m)} = \norm{y_n - y_m}\text{.}\) On rearrangement, we get
\begin{align*}
\norm{y_n - y_m}^2 \amp\lt 4M^2 + 2\left(\frac{1}{n} + \frac{1}{n}\right) - \norm{2x - y_n - y_m}^2\\
\amp= 4M^2 + 2\left(\frac{1}{n} + \frac{1}{n}\right) - 4\norm{x - \frac{y_n - y_m}{2}}^2
\end{align*}
Now we will use the convexity of \(A\text{.}\) Since \(y_n, y_m \in A\text{,}\) we know that \(\frac{1}{2} y_n + \frac{1}{2} y_m =\frac{y_n + y_m}{2} \in A\text{,}\) and hence that
\begin{equation*}
M^2 = \inf_{a \in A} \norm{x - a}^2 \leq \norm{x - \frac{y_n + y_n}{2}}^2.
\end{equation*}
Consequently,
\begin{align*}
\norm{y_n - y_m}^2 \amp\leq 4M^2 + 2\left(\frac{1}{n} + \frac{1}{m} - 4M^2\right)\\
\amp = 2\left(\frac{1}{n} + \frac{1}{m}\right).
\end{align*}
It follows that \((y_n)\) is a Cauchy sequence in \(A\text{;}\) for a given \(\eps > 0\) (you should verify). Because \(\hilbert\) is a Hilbert space, Cauchy sequences converge, and so \(y_n \to y \in \hilbert\text{.}\) Since \(A\) is closed, \(y \in A\text{.}\) On the one hand, this means that \(\norm{x - y} \geq M\text{,}\) since \(M\) is an infimum. On the other hand, we know that for all \(n\text{,}\)
\begin{equation*}
\norm{x - y_n}^2 \leq M^2 + \frac{1}{n}.
\end{equation*}
On taking \(n \to \infty\text{,}\) we get \(\norm{x - y} \leq M^2\text{.}\) We conclude that \(\norm{x - y} = M\text{,}\) and so \(y\) is a closest point in \(A\) to \(x\text{.}\)
Now we show that \(y\) is the unique closest point. Suppose that there is some \(w \in A\) with \(\norm{x - w} = M\text{.}\) Convexity gives that \(\frac{y + w}{2} \in A\text{,}\) and so
\begin{equation*}
\norm{x - \frac{y + w}{2}} \geq M.
\end{equation*}
\begin{align*}
\norm{y-w}^2 \amp= 2\norm{x-w}^2 + 2\norm{x-y}^2 - 4\norm{x - \frac{y + w}{2}}^2\\
\amp \leq 2M^2 + 2M^2 - 4M^2 = 0.
\end{align*}
Thus, \(y=w\) which gives uniqueness.
