Skip to main content

Hilbert Spaces - Sequel to Linear Algebra

Ryan Tully-Doyle

Section 7.1 Linear functionals

Let us consider an analogy with linear algebra. The space \(\R^n\) can be acted on by a linear transformation, which can be realized as multiplication by a matrix \(M\) that represents the linear transformation. Such an \(M\) can be any size that is compatible with matix mutiplication. If we choose a transformation that is represented by a vector \(y\) of size \(m \times 1\text{,}\) then we’ve created a function on \(\R^n\) that takes \(\R^n \to \R\text{.}\) The space of all such functions is itself \(n\)-dimensional (since each function arises from a vector in \(\R^n\)). Multiplication of two vectors that results in a scalar output already has a form that we know - namely, we can write any such function in terms of its representing vector \(y\) by
\begin{equation*} y: \R^n \mapsto \R \text{ by } y(v) = \ip{v}{y}. \end{equation*}
This is a simple case of a powerful theorem called the Riesz representation theorem. A function on \(\R^n\) of the form \(\ip{\cdot}{y}\) is called a linear functional. One of the core ideas of functional analysis is that we can often move phenomena and relationships that exist among the elements of a vector space into dual relationships among elements of the space of linear functionals.
While it easy to write down the general linear functional in finite dimensions, in infinite dimensions the problem is significantly harder. One useful restriction is to consider just the family of continuous linear functionals (we are doing analysis, after all!). One reason to study Hilbert spaces is that the inner product geometry allows us to write down then general linear functional (in precisely the same form as the finite dimensional case - \(\ip{\cdot}{y}\) for some vector \(y\)). In Banach spaces, we do not have an inner product and it is quite a bit trickier. This means that in Hilbert spaces, duality works much like it does in finite dimensions. We will now make these notions formal.

Definition 7.1.1.

A linear functional on a vector space \(V\) over a field \(k\) is a mapping \(f:V \to k\) satisfying
\begin{equation*} F(\la x + \mu y) = \la f(x) + \mu f(y) \end{equation*}
for all \(x, y \in V\) and \(\la, \mu \in k\text{.}\)

Example 7.1.2. on \(\C^n\).

Define \(f: \C^n \to \C\) by
\begin{equation*} F(x_1, \ldots, x_n) = c_1 x_1 + \ldots + c_n x_n \end{equation*}
where \(c_1, \ldots, c_n \in \C\text{.}\)

Example 7.1.3. on \(\ell^1\).

Define \(f_c: \ell^1 \to \C\) by
\begin{equation*} F_c((x_n)_{n \in \mathbb{N}}) = \sum_{n=1}^\infty c_n x_n \end{equation*}
where \(c = (c_n)_{n \in \mathbb{N}} \in \ell^\infty\text{.}\) The space \(\ell^\infty\) is the space of bounded sequences. The space \(\ell^1\) is the space of sequences \((x_n)\) so that \(\sum \abs{x_n}\) is finite. Combining these facts gives that \(\sum c_n x_n\) is absolutely convergent, which ensures that \(f_c\) is a well-defined function.

Example 7.1.4. on \(C[0,1]\).

Define \(f:C[0,1]\to \C\) by
\begin{equation*} F(x) = \int_0^1 x(t) \dd \alpha(t) \end{equation*}
for a function \(x \in C[0,1]\text{,}\) where \(\alpha\) is a function of bounded variation on \([0,1]\text{.}\) (This is called a Stieltjes integral, and used widely in probability theory. It generalizes the Riemann integral - one recovers the Riemann integral by choosing \(\alpha(t) = t\text{.}\))

Example 7.1.5. on \(\mathcal H\).

Let \(\mathcal H\) be a Hilbert space and \(y \in \hilbert\text{.}\) Define \(f_y: \hilbert \to \C\) by
\begin{equation*} F_y(x) = \ip{x}{y}. \end{equation*}

Checkpoint 7.1.6.

Show that each of the above is a linear functional.
The examples above are also continuous. There is a convenient theorem for checking this fact.
The set \(\{x \in V: \norm{x}\leq 1\}\) is called the unit ball of \(V\text{.}\) So a functional is continuous if and only if it is bounded on the unit ball.

Proof.

(1) implies (2) is obvious.
Suppose that (2) holds. Then given \(\eps = 1\text{,}\) there exists \(\delta > 0\) so that \(\abs{x} \lt \delta\) implies that \(\abs{F(x)}\lt1\text{.}\) Now we will use the fact that \(F\) is linear to show that \(F\) is bounded. For any \(x \in V\) with \(\norm{x} \leq 1\text{,}\) \(\norm{\frac{\delta x}{2}} \lt \delta\text{,}\) which means that \(F(\frac{\delta x}{2}) \lt 1\text{.}\) Then linearity gives that \(F(x) \lt \frac{2}{\delta}\) on the unit ball of \(V\text{,}\) and so (3) holds.
Now suppose that (3) holds. Denote the finite supremum of \(F\) on the unit ball of \(V\) by \(M\text{.}\) For any distinct vectors \(x, y \in V\text{,}\) the vector \((x - y)/\norm{x - y)}\) is a unit vector. Then by (iii),
\begin{equation*} \abs{F\left( \frac{x-y}{\norm{x-y}}\right)} \leq M. \end{equation*}
Because \(F\) is linear, this means that
\begin{equation*} \abs{F(x) - F(y)} \leq M \norm{x - y} \end{equation*}
and so \(F\) is continuous.

Checkpoint 7.1.8. A discontinous linear functional.

Define a linear functional \(F\) on \(C^1[0,1]\text{,}\) the space of continuously differentiable functions on \([0,1]\text{,}\) by
\begin{equation*} F(x) = x'(1). \end{equation*}
Show that \(F\) is discontinuous with respect to the norm \(\norm{\cdot}_\infty\text{.}\)
Just as in the finite-dimensional case, the set of continuous linear functionals is a space in its own right, as long as the underlying space is at least a normed space. Surprisingly, even if the underlying space is not complete (i.e., not a Banach space), the set of functionals will be.

Proof.

First, it is straightforward to show that \(V\ad\) is a vector space over the same field as \(V\text{.}\) By Theorem 7.1.7, \(\norm{F} \lt \infty\) for all \(F \in V\ad\text{,}\) and one can readily verify that \(\norm{\cdot}\) as defined above is a norm (as in Definition 3.1.3).
We will now show that \(V\ad\) is complete. Let \((F_n)\) denote a Cauchy sequence of linear functionals so that \(\norm{F_n - F_m} \to 0\) as \(n, m \to \infty\text{.}\) Choose \(x \in V\text{.}\) Then the sequence \(\abs{F_n(x) - F_m(x)} \to 0\) as \(n, m \to \infty\text{.}\) As this is a sequence of scalars, it must converge. Denote the limit of this sequence by \(F(x)\text{.}\) Then the map \(F:V \to k\) so pointwise defined is a linear functional (by the properties of limits and the fact that the \(F_n\) are linear functionals). It remains to show that \(F \in V\ad\) and that \(F_n \to F\) with respect to \(\norm{\cdot}\text{.}\)
Let \(\eps > 0\) and choose some \(N\) so that \(n, m \gt N\) implies that \(\norm{F_n - F_m} \lt \eps\) (as \(F_n\) is Cauchy). Then for any \(x \in V\) with \(\norm{x} \lt 1\text{,}\) and for all \(n, m \gt N\text{,}\) we have
\begin{equation*} \abs{F_m(x) - F_n(x)} \lt \eps. \end{equation*}
Taking \(m \to \infty\text{,}\) we get that, for all \(\norm{x}\lt 1\) and \(n > N\text{,}\)
\begin{equation} \abs{F(x) - F_n(x)} \leq \eps.\tag{7.1.1} \end{equation}
To complete the proof, note that since \(F - F_N\) and \(F_N\) are bounded on the closed unit ball of \(V\text{,}\) so must be \(F\text{,}\) which by Theorem 7.1.7 means that \(F\) is continuous and so \(F \in V\ad\text{.}\) On the other hand, (7.1.1) implies that \(\norm{F_n - F} \lt \eps\) for \(n \gt N\text{,}\) and so \(F_n \to F\) in \(V\ad\text{.}\) Thus \(V\ad\) is complete (and hence a Banach space of continuous linear functionals).
The space \(V\ad\) is often referred to as the dual space of \(V\text{.}\)
Analysis is built on a backbone of useful inequalities. Let us take note of a nice inequality that is an immediate consequence of the definition of the dual norm.

Proof.

Suppose that \(v \neq 0\text{.}\) Then \(\frac{v}{\norm{v}}\) is a unit vector, and so
\begin{equation*} \norm{F} \geq \abs{F(\frac{v}{\norm{v}})} = \frac{\abs{F(v)}}{\abs{v}}. \end{equation*}

Example 7.1.11. The dual of \(\ell^1\).

We will describe the dual space of \(\ell^1\text{,}\) the space of absolutely summable sequences. It turns out to be the case that one can identify \((\ell^1)\ad\) with \(\ell^\infty\) by way of a map that is an isomorphism of vector spaces and also an isometry (a mapping that preserves norms). Define a map \(T: \ell^\infty \to (\ell^1)\ad\) by
\begin{equation*} Tc = F_c, \end{equation*}
where \(F_c\) is the functional defined in Example 7.1.3. It is easy to see that \(T\) is linear. To see that \(T\) is surjective, choose an element \(\xi \in (\ell^1)\ad\text{.}\) We need an element \(c \in \ell^\infty\) such that \(Tc = \xi\text{.}\) Let \(e_n\) denote the \(n\)th elementary basis vector in \(\ell^1\) (you can think of this as a vector of 0s with a 1 in the \(n\)th position), and let \(c_n = \xi(e_n)\text{.}\) Now let \(c = (c_n)_{n \in \mathbb{N}}\text{.}\) (Notice that \(F_c(e_n) = c_n\text{.}\)) Since \(e_n\) is a unit vector in \(\ell^1\text{,}\)
\begin{equation*} \abs{c_n} = \abs{\xi(e_n)} \leq \norm{\xi}\abs{e_n} = \norm{\xi}. \end{equation*}
This means that \((c_n)\) is a bounded sequence, and so \(c \in \ell^\infty\) with
\begin{equation} \norm{c}_\infty = \sup_n \abs{c_n} \leq \norm{\xi}.\tag{7.1.2} \end{equation}
Now we need to show that \(F_c = \xi\text{.}\) Choose an \(x\in \ell^1\text{,}\) and denote the \(k\)-th partial expansion of \(x\) by \(x^k = \sum_{n = 1}^k x_n e_n\text{.}\) As a sequence, \(x^k \to x\) in \(\ell^1\) as \(k\to \infty\text{.}\) Then we can evaluate \(F\) and \(\xi\) on the sequence by
\begin{equation*} F_c(x^k) - \xi(x^k) = \sum_{n=1}^{k} c_n x_n - \sum_{n = 1}^k x_n \xi(e_n) = 0. \end{equation*}
In the limit, this shows that \(F_c(x) = \xi(x)\) for arbitrary \(x\text{.}\)
The last part of the proof is to show that distances are preserved under \(T\text{.}\) By (7.1.2), we have \(\norm{c}_\infty \leq \norm{F_c}\text{.}\) On the other hand, for any choice of \(x \in \ell^1\text{,}\)
\begin{equation*} \abs{F_c(x)} = \abs{\sum_{n=1}^\infty c_n x_n} \leq \sup_n \abs{c_n} \sum\abs{x} = \norm{c}_\infty \norm{x}_1, \end{equation*}
and so \(\norm{F_c} \leq \norm{c}_\infty\text{.}\) We conclude that \(\norm{F_c} = \norm{c}\text{,}\) and thus that \(T\) is an isometry.
PrevTopNext