Section 1.1 Motivation
One of the most profound ideas of linear algebra is that any finite dimensional vector space over \(\R\) or \(\C\) is secretly \(\R^n\) or \(\C^n\). This insight allows us to reduce the study of vector spaces and the maps between them to the study of matrices.
The key idea is that every finite dimensional vector space can be represented in coordinates once we choose a basis. We denote the representation of a vector \(v \in V\) with respect to a basis \(\mathcal V\) by \(v_\mathcal{V}\text{.}\) Better yet, that basis can be chosen to be orthonormal by way of the Gram-Schmidt process and the dot product structure of Euclidean space. The coordinatization of \(V\) also gives unique representations of linear maps betwen those spaces.
Theorem 1.1.1.
Let \(V, W\) be finite dimensional vector spaces with bases \(\mathcal V, \mathcal W\text{.}\) Then any linear map \(T: V \to W\) has a unique matrix representation with respect to \(\mathcal V, \mathcal W\) by
\begin{equation*}
T(x) = A x
\end{equation*}
with
\begin{equation*}
A = \bbm T(v_1)_{\mathcal W} \amp \ldots \amp T(v_n)_{\mathcal W} \ebm
\end{equation*}
Typical examples introduced in a linear algebra course include the space of polynomials of degree less than or equal to \(n\text{.}\) At the same time, we usually also get to see a very suggestive example of a useful linear map and the representation of that map in matrix form.
Let \(P_n\) denote the space of polynomials of degree \(\leq n\text{.}\) Consider the map \(D: P_3 \to P_2\) defined by
\begin{equation*}
D(a_0 + a_1 t + a_2 t^2 + a_3 t^3) = a_1 + 2 a_2 t + 3 a_3 t^2.
\end{equation*}
That is, \(D\) is the map that takes the derivative of a polynomial. It isn’t hard to use the standard basis for \(P_n\) to get the matrix representation
\begin{equation*}
D(p) = \bbm 0 \amp 1 \amp 0 \amp 0 \\ 0 \amp 0 \amp 2 \amp 0 \\ 0 \amp 0 \amp 0 \amp 3 \ebm \bbm a_0 \\ a_1 \\ a_2 \\ a_3 \ebm
\end{equation*}
for the action of \(D\) on \(P_3\text{.}\)
This example is a good place to begin asking questions about how far we can push finite dimensional linear algebra. The fact that differentiation of polynomials is represented by a matrix multiplying a vector is wonderful - but what else can we apply it to? Nice functions have power series that converge absolutely, and we like to think of an absolutely convergent power series as sort of an “infinite polynomial”. Our intution might lead us to make a connection with calculus at this point. When we learn to work with power series, we learn that for a convergent power series,
\begin{equation*}
\frac{d}{dx} \sum a_n (x-a)^n = \sum n a_n (x-a)^{n-1}.
\end{equation*}
In analogy with our example about polynomials above, we’re tempted to write, for a function \(f\) defined by a convergent power series, that
\begin{equation*}
D(f) = \underbrace{\bbm 0 \amp 1 \amp 0 \amp 0 \amp \ldots \\ 0 \amp 0 \amp 2 \amp 0 \amp \ldots \\ 0 \amp 0 \amp 0 \amp 3 \amp \ldots \\ \vdots \amp \amp \amp \amp \ddots \ebm}_{A} \bbm a_0 \\ a_1 \\ a_2 \\ a_3 \\ \vdots \ebm = a_1 + 2a_2 (x - a) + 3 a_3 (x-1)^2 + \ldots.
\end{equation*}
This idea is shot through with issues that need to be addressed.
The object \(A\) is some kind of \(\infty \times \infty\) matrix. How does that make sense?
What are the vector spaces that \(D\) is mapping between?
Does the idea of coordinitization still work?
If it does, what exactly is “\(\R^\infty\)” supposed to be?
Do infinite dimensional vector spaces and bases make sense at all?
The answers to these questions are the heart of what is known as the theory of Hilbert spaces, which naturally envelop and extend finite dimensional vector space theory. Hilbert spaces are the key objects used to study functions with various kinds of infinite series representations, which is a vast area of mathematics known as functional analysis. The objects \(A\) are called operators, and are the central object of study in operator theory.
The rest of this introductory chapter will review important parts of linear algebra in finite dimensions that we need to motivate and understand Hilbert spaces.
