Example Problems

Calculation

Suppose we wanted to check whether a set of vectors $\{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_n\}$ forms a basis for a subspace $V$.

For this, we need to check that both of the requirements of a basis are met:

1. $\text{span}\{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_n\} = V$: We know that we can check whether $\{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_n\}$ spans $V$ by checking if there is a pivot in every row of the matrix $\begin{bmatrix}\vec{v}_1 & \vec{v}_2 & \cdots & \vec{v}_n\end{bmatrix}$.

2. $\{\vec{v}_1, \vec{v}_2, \ldots, \vec{v}_n\}$ is linearly independent: We know that we can check for linear independence by checking if there is a pivot in every column of the matrix $\begin{bmatrix}\vec{v}_1 & \vec{v}_2 & \cdots & \vec{v}_n\end{bmatrix}$.

Therefore, all we need to do is to check whether $\begin{bmatrix}\vec{v}_1 & \vec{v}_2 & \cdots & \vec{v}_n\end{bmatrix}$ has a pivot in every row and column.

Change of basis

Quick overview of change of basis:

Sometimes, the problem becomes easier if we use another basis to solve the problem, e.g. an orthonormal basis, where the basis vectors are orthogonal to each other ("ortho") and the length of all basis vectors is $1$ ("normal"). In this case, we have to change the basis.

Note: Change of basis only works if both bases $\mathcal{B}$ and $\mathcal{C}$ span the same subspace. This makes sense intuitively: a change of basis should not add or remove elements from the subspace.

It turns out, though, that a change of basis is actually just a linear transformation, so we can use a matrix to express a change of basis. (In fact, a change of basis is a special case of an identity transformation, where a vector is mapped to itself. We are just changing how the vector is expressed.)

Assume that we wanted to change the basis from $\mathcal{B} = \{\vec{b}_1, \vec{b}_2, \ldots, \vec{b}_n\}$ to $\mathcal{C} = \{\vec{c}_1, \vec{c}_2, \ldots, \vec{c}_n\}$.

First, let's explore what $[\vec{v}]_\mathcal{B} = \begin{bmatrix}v_1 \\ v_2 \\ \vdots \\ v_n\end{bmatrix}$ actually means. As we saw above with the standard basis example, this equation merely means $\vec{v} = v_1\vec{b}_1 + v_2\vec{b}_2 + \cdots + v_n\vec{b}_n$

$\vec{v} = \begin{bmatrix}\vec{b}_1 & \vec{b}_2 & \cdots & \vec{b}_n\end{bmatrix}\begin{bmatrix}v_1 \\ v_2 \\ \vdots \\ v_n\end{bmatrix} = \begin{bmatrix}\vec{b}_1 & \vec{b}_2 & \cdots & \vec{b}_n\end{bmatrix}[\vec{v}]_\mathcal{B}$

We define $\textbf{P}_\mathcal{B} = \begin{bmatrix}\vec{b}_1 & \vec{b}_2 & \cdots & \vec{b}_n\end{bmatrix}$, so $\textbf{P}_\mathcal{B}$ is the matrix with the basis vectors of $\mathcal{B}$.

$\textbf{P}_\mathcal{B}[\vec{v}]_\mathcal{B} = \vec{v}$

Similarly,

$\textbf{P}_\mathcal{C}[\vec{v}]_\mathcal{C} = \vec{v}$

Therefore,

$\textbf{P}_\mathcal{B}[\vec{v}]_\mathcal{B} = \textbf{P}_\mathcal{C}[\vec{v}]_\mathcal{C}$

$[\vec{v}]_\mathcal{C} = \textbf{P}_\mathcal{C}^{-1}\textbf{P}_\mathcal{B}[\vec{v}]_\mathcal{B}$

We define $\textbf{P}_\mathcal{C}^{-1}\textbf{P}_\mathcal{B}$ to be the change of basis matrix from $\mathcal{B}$ to $\mathcal{C}$ denoted by $\textbf{P}_{\mathcal{C} \leftarrow \mathcal{B}}$.

Another way of finding the change of basis matrix is by looking at the basis vectors. Since $[\vec{v}]_\mathcal{B}$ is just a linear combination of the basis vectors of $\mathcal{B}$, if we can find how to express each of the basis vectors of $\mathcal{B}$ in terms of $\mathcal{C}$, $[\vec{v}]_\mathcal{C}$ is just a linear combination of the basis vectors of $\mathcal{B}$ in terms of $\mathcal{C}$ with the same weights.

$[\vec{v}]_\mathcal{C} = v_1[\vec{b}_1]_\mathcal{C} + v_2[\vec{b}_2]_\mathcal{C} + \cdots + v_n[\vec{b}_n]_\mathcal{C}$

To express each basis vector of $\mathcal{B}$ in terms of $\mathcal{C}$, we have to solve the following augmented matrices, one for each basis vector in $\mathcal{B}$:

$\left[\begin{array}{cccc|c}\vec{c}_1 & \vec{c}_2 & \cdots & \vec{c}_n & \vec{b}_1\end{array}\right]$

$\left[\begin{array}{cccc|c}\vec{c}_1 & \vec{c}_2 & \cdots & \vec{c}_n & \vec{b}_2\end{array}\right]$

$\vdots$

$\left[\begin{array}{cccc|c}\vec{c}_1 & \vec{c}_2 & \cdots & \vec{c}_n & \vec{b}_n\end{array}\right]$

To make our lives easier and because we are performing the same row operations every time, we can row-reduce the following augmented matrix

$\left[\begin{array}{cccc|cccc}\vec{c}_1 & \vec{c}_2 & \cdots & \vec{c}_n & \vec{b}_1 & \vec{b}_2 & \cdots & \vec{b}_n\end{array}\right] = \left[\begin{array}{c|c}\textbf{P}_\mathcal{C} & \textbf{P}_\mathcal{B}\end{array}\right] \sim \left[\begin{array}{c|c}\textbf{I} & \textbf{P}_{\mathcal{C} \leftarrow \mathcal{B}}\end{array}\right]$

As a side note, since we wrote all of the basis vectors of $\mathcal{B}$ and $\mathcal{C}$ in terms of the standard basis $\mathcal{S}$, $\textbf{P}_\mathcal{B} = \textbf{P}_{\mathcal{S} \leftarrow \mathcal{B}}$ and $\textbf{P}_\mathcal{C} = \textbf{P}_{\mathcal{S} \leftarrow \mathcal{C}}$.

It is also pretty intuitive that $\textbf{P}_{\mathcal{S} \leftarrow \mathcal{B}}^{-1} = \textbf{P}_{\mathcal{B} \leftarrow \mathcal{S}}$ and $\textbf{P}_{\mathcal{S} \leftarrow \mathcal{C}}^{-1} = \textbf{P}_{\mathcal{C} \leftarrow \mathcal{S}}$.

Therefore,

$[\vec{v}]_\mathcal{C} = \textbf{P}_\mathcal{C}^{-1}\textbf{P}_\mathcal{B}[\vec{v}]_\mathcal{B}$

$[\vec{v}]_\mathcal{C} = \textbf{P}_{\mathcal{S} \leftarrow \mathcal{C}}^{-1}\textbf{P}_{\mathcal{S} \leftarrow \mathcal{B}}[\vec{v}]_\mathcal{B}$

$[\vec{v}]_\mathcal{C} = \textbf{P}_{\mathcal{C} \leftarrow \mathcal{S}}\textbf{P}_{\mathcal{S} \leftarrow \mathcal{B}}[\vec{v}]_\mathcal{B}$

This equation makes sense: we are basically first changing the basis from $\mathcal{B}$ to the standard basis $\mathcal{S}$ and then from $\mathcal{S}$ to $\mathcal{C}$.

$\textbf{P}_{\mathcal{C} \leftarrow \mathcal{B}} = \textbf{P}_{\mathcal{C} \leftarrow \mathcal{S}}\textbf{P}_{\mathcal{S} \leftarrow \mathcal{B}}$

Example 1

Let $\mathcal{B} = \left\{\begin{bmatrix}5 \\ 4\end{bmatrix}, \begin{bmatrix}-1 \\ 0\end{bmatrix}\right\}$ be a basis for $\mathbb{R}^2$ and $[\vec{v}]_\mathcal{B} = \begin{bmatrix}2 \\ -1\end{bmatrix}$. Find $\vec{v}$ in the standard basis.

$[\vec{v}]_\mathcal{S} = \textbf{P}_\mathcal{B}[\vec{v}]_\mathcal{B} = \begin{bmatrix}5 & -1 \\ 4 & 0\end{bmatrix}\begin{bmatrix}2 \\ -1\end{bmatrix} = \begin{bmatrix}11 \\ 8\end{bmatrix}$

Example 2

Let $\mathcal{B} = \left\{\begin{bmatrix}7 \\ -2\end{bmatrix}, \begin{bmatrix}2 \\ -1\end{bmatrix}\right\}$ and $\mathcal{C} = \left\{\begin{bmatrix}4 \\ 1\end{bmatrix}, \begin{bmatrix}5 \\ 2\end{bmatrix}\right\}$. Find $[\vec{v}]_\mathcal{C}$ if $[\vec{v}]_\mathcal{B} = \begin{bmatrix}4 \\ 5\end{bmatrix}$.

Let's first find the change of basis matrix $\textbf{P}_{\mathcal{C} \leftarrow \mathcal{B}}$.

$\begin{bmatrix}4 & 5 & \vdots & 7 & 2 \\ 1 & 2 & \vdots & -2 & -1\end{bmatrix} \sim \begin{bmatrix}4 & 5 & \vdots & 7 & 2 \\ 0 & 3 & \vdots & -15 & -6\end{bmatrix} \sim \begin{bmatrix}4 & 0 & \vdots & 32 & 12 \\ 0 & 1 & \vdots & -5 & -2\end{bmatrix} \sim \begin{bmatrix}1 & 0 & \vdots & 8 & 3 \\ 0 & 1 & \vdots & -5 & -2\end{bmatrix}$

$\textbf{P}_{\mathcal{C} \leftarrow \mathcal{B}} = \begin{bmatrix}8 & 3 \\ -5 & -2\end{bmatrix}$.

We then apply this matrix to $[\vec{v}]_\mathcal{C}$:

$[\vec{v}]_\mathcal{C} = \textbf{P}_{\mathcal{C} \leftarrow \mathcal{B}}[\vec{v}]_\mathcal{B} = \begin{bmatrix}8 & 3 \\ -5 & -2\end{bmatrix}\begin{bmatrix}4 \\ 5\end{bmatrix} = \begin{bmatrix}47 \\ -30\end{bmatrix}$

Bonus: All eigenvectors of a matrix are linearly independent, so we have $n$ eigenvectors for an $n \times n$ matrix, we can use the eigenvectors as a basis for the subspace $\mathbb{R}^n$. In this case, we call the basis an eigenbasis. Try expressing the matrix $\begin{bmatrix}1 & -1 & 2 \\ 0 & 0 & 2 \\ 0 & -1 & 3\end{bmatrix}$ (taken from the chapter Eigen-everything) in terms of its eigenbasis, i.e. express each column in terms of the eigenbasis. What do you notice?