Example Problems

Examples

Linear equations	Non-linear equations
$y = \frac{3}{2}x$ $\frac{5}{4}x - \frac{1}{6}y = 3$ $6\sqrt{2}a - 3.3b = c$ $e^{1.72} \cdot \frac{\sqrt{\pi} - 1.6 + \frac{5}{6}}{44.5 - 5\sqrt{6.6}} \cdot x_1 + \int_{0}^{5} e^{-t^2 + \sqrt{5}t}dt \cdot x_2 - \sum_{i = 0}^{100} \sin(i) \cdot x_3 = 2$ $\begin{bmatrix}1.08 & 6.2 & 2.6\end{bmatrix}\begin{bmatrix}x_1 \\ x_2 \\ x_3\end{bmatrix} = 4$	$y = x^2$ $a \cdot b - 5.3c = 5$ $3x^3 - \frac{5}{2}y^5 - 54xy = \sqrt{z}$ $\int_{-5}^{x} 5\csc(t^2) dt = 2y$

System of linear equations

A system of linear equations consists of multiple equations that are solved simultaneously. The easiest method to solve systems of linear equations is through Gaussian elimination.

Examples

$\left\{\begin{array}{l} \frac{1}{3}x_1+\frac{1}{4}x_2+\frac{1}{2}x_3=\frac{15}{4} \\ \frac{7}{2}x_1+\frac{2}{3}x_2+\frac{1}{3}x_3=\frac{16}{3} \\ \frac{1}{5}x_2+\frac{3}{7}x_3=\frac{34}{5} \\ \end{array}\right.$

$a_1\begin{bmatrix}1 \\ 2 \\ 1\end{bmatrix} + a_2\begin{bmatrix}0 \\ 1 \\ 1\end{bmatrix} + a_3\begin{bmatrix}2 \\ 0 \\ 0\end{bmatrix} = \begin{bmatrix}5 \\ 2 \\ -4\end{bmatrix}$

$\left[\begin{array}{ccc|c}1 & -2 & 1 & 0 \\ 0 & 5 & -2 & -1 \\ 4 & 1 & 3 & 2\end{array}\right]$

Overview for the next topics:

Linear combinations

A linear combination of a set of terms is a sum of each term multiplied by a constant. Since a linear combination is basically a weighted sum of each term, the constant coefficients can be thought of as “weights.”

Examples

1. $5\vec{a} + 7\vec{b}$ is a linear combination of $\vec{a}$ and $\vec{b}$.
2. $3\vec{a} + 0\vec{b} + 1\vec{c} = 3\vec{a} + \vec{c}$ is a linear combination of $\vec{a}, \vec{b}, \vec{c}$. (Note that we can multiply a vector by $0$ as well.)

Span

Sometimes, we want to denote all linear combinations of a set of vectors $\{\vec{a}_1, \vec{a}_2, \ldots, \vec{a}_n\}$. To do this, we define the span of these vectors to be the set of all linear combinations of these vectors:

$\text{span}\{\vec{a}_1, \vec{a}_2, \ldots \vec{a}_n\} = \{c_1\vec{a}_1 + c_2\vec{a}_2 + \cdots + c_n\vec{a}_n \mid c_1, c_2, \ldots, c_n \in \mathbb{R}\}$

Note: The zero vector $\vec{0} \in \text{span}\{\vec{a}_1, \vec{a}_2, \ldots \vec{a}_n\}$ for any $n$ vectors because we can multiply all of these vectors by $0$ to obtain $\vec{0}$.

Examples

$\text{span}\left\{\begin{bmatrix}1 \\ 0 \\ 0\end{bmatrix}, \begin{bmatrix}0 \\ 1 \\ 0\end{bmatrix}\right\}$ is basically the $x$-$y$ plane in $\mathbb{R}^3$ because every vector lying on this plane can be expressed as a sum of each vector multiplied by a constant, e.g. $\begin{bmatrix}5 \\ 3 \\ 0\end{bmatrix} = 5\begin{bmatrix}1 \\ 0 \\ 0\end{bmatrix} + 3\begin{bmatrix}0 \\ 1 \\ 0\end{bmatrix}$.

Note: This plane is not the same as $\mathbb{R}^2$. Each point on the $x$-$y$ plane in $\mathbb{R}^3$ has 3 components while all points in $\mathbb{R}^2$ only have 2. In general, $\mathbb{R}^2 \not\subset \mathbb{R}^3$.

Similarly, $\text{span}\left\{\begin{bmatrix}1 \\ 0\end{bmatrix}, \begin{bmatrix}0 \\ 1\end{bmatrix}\right\} = \mathbb{R}^2$ because every vector in $\mathbb{R}^2$ can be written as a linear combination of $\begin{bmatrix}1 \\ 0\end{bmatrix}$ and $\begin{bmatrix}0 \\ 1\end{bmatrix}$.

Calculation

Suppose we wanted to find out whether a vector $\vec{b}$ is in the span of a set of vectors $\{\vec{a}_1, \vec{a}_2, \ldots, \vec{a}_n\}$.

Basically, we are solving trying to find constants $c_1, c_2, \ldots, c_n$ such that $c_1\vec{a}_1 + c_2\vec{a}_2 + \cdots + c_n\vec{a}_n = \vec{b}$

This looks like a system of linear equations, so we can rewrite this in matrix notation: $\begin{bmatrix}| & | & & | \\ \vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_n\\ | & | & & |\end{bmatrix}\begin{bmatrix}c_1 \\ c_2 \\ \vdots \\ c_n\end{bmatrix} = \vec{b}$

We can then transform the equation into an augmented matrix: $\left[\begin{array}{cccc|c}| & | & & | & | \\ \vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_n & \vec{b} \\ | & | & & | & |\end{array}\right]$

Therefore, solving a system of linear equations is basically finding how we can express the vector $\vec{b}$ as a linear combination of the column vectors $\vec{a}_1, \vec{a}_2, \ldots, \vec{a}_n$. We can conclude:

1. If the system of linear equations is consistent, then $\vec{b}$ is in the span of the vectors $\vec{a}_1, \vec{a}_2, \ldots, \vec{a}_n$, so $\vec{b}$ is a linear combination of the vectors $\vec{a}_1, \vec{a}_2, \ldots, \vec{a}_n$.
2. If the system of linear equations is inconsistent, then $\vec{b}$ is not in the span of the vectors $\vec{a}_1, \vec{a}_2, \ldots, \vec{a}_n$, i.e. $\vec{b}$ is not a linear combination of the vectors $\vec{a}_1, \vec{a}_2, \ldots, \vec{a}_n$.

Example 1

Determine whether $\begin{bmatrix}5 \\ 4 \\ 3\end{bmatrix}$ is in the span of $\begin{bmatrix}1 \\ 0 \\ 1\end{bmatrix}$ and $\begin{bmatrix}1 \\ 2 \\ 0\end{bmatrix}$.

We can just solve the augmented matrix:

$\left[\begin{array}{cc|c}1 & 1 & 5 \\ 0 & 2 & 4 \\ 1 & 0 & 3\end{array}\right] \sim \left[\begin{array}{cc|c}1 & 1 & 5 \\ 0 & 1 & 2 \\ 0 & 1 & 2\end{array}\right] \sim \left[\begin{array}{cc|c}1 & 0 & 3 \\ 0 & 1 & 2 \\ 0 & 0 & 0\end{array}\right]$

Since the system of linear equations is consistent, $\begin{bmatrix}5 \\ 4 \\ 3\end{bmatrix}$ is in the span of $\begin{bmatrix}1 \\ 0 \\ 1\end{bmatrix}$ and $\begin{bmatrix}1 \\ 2 \\ 0\end{bmatrix}$. In fact, $\begin{bmatrix}5 \\ 4 \\ 3\end{bmatrix} = 3\begin{bmatrix}1 \\ 0 \\ 1\end{bmatrix} + 2\begin{bmatrix}1 \\ 2 \\ 0\end{bmatrix}$.

Example 2

Determine whether $\begin{bmatrix}1 \\ 0 \\ 1\end{bmatrix}$, $\begin{bmatrix}1 \\ 2 \\ 0\end{bmatrix}$, and $\begin{bmatrix}5 \\ 4 \\ 3\end{bmatrix}$ span $\mathbb{R}^3$.

Let's row-reduce the matrix:

$\begin{bmatrix}1 & 1 & 5 \\ 0 & 2 & 4 \\ 1 & 0 & 3\end{bmatrix} \sim \begin{bmatrix}1 & 1 & 5 \\ 0 & 2 & 4 \\ 0 & 1 & 2\end{bmatrix} \sim \begin{bmatrix}1 & 1 & 5 \\ 0 & 1 & 2 \\ 0 & 0 & 0\end{bmatrix}$

Observe that there is no pivot in the third row. This means that this system of linear equations is not consistent for all vectors in $\mathbb{R}^3$, so $\begin{bmatrix}1 \\ 0 \\ 1\end{bmatrix}$, $\begin{bmatrix}1 \\ 2 \\ 0\end{bmatrix}$, and $\begin{bmatrix}5 \\ 4 \\ 3\end{bmatrix}$ do not span $\mathbb{R}^3$.

In other words, we could have row-reduced the following augmented matrix:

$\left[\begin{array}{ccc|c}1 & 1 & 5 & b_1 \\ 0 & 2 & 4 & b_2 \\ 1 & 0 & 3 & b_3\end{array}\right] \sim \left[\begin{array}{ccc|c}1 & 1 & 5 & b_1 \\ 0 & 2 & 4 & b_2 \\ 0 & 1 & 2 & b_1 - b_3\end{array}\right] \sim \left[\begin{array}{ccc|c}1 & 1 & 5 & b_1 \\ 0 & 1 & 2 & \frac{b_2}{2} \\ 0 & 0 & 0 & b_2 - 2(b_1 - b_3)\end{array}\right]$

Let's pick a vector $\vec{b} = \begin{bmatrix}b_1 \\ b_2 \\ b_3\end{bmatrix} \in \mathbb{R}^3$, such that $b_2 - 2(b_1 - b_3) \neq 0$, such as $\vec{b} = \begin{bmatrix}1 \\ 0 \\ 0\end{bmatrix}$. After row reduction, you will see that $\begin{bmatrix}1 \\ 0 \\ 0\end{bmatrix} \notin \text{span}\left\{\begin{bmatrix}1 \\ 0 \\ 1\end{bmatrix}, \begin{bmatrix}1 \\ 2 \\ 0\end{bmatrix}, \begin{bmatrix}5 \\ 4 \\ 3\end{bmatrix}\right\}$.

In general, there needs to be pivots in every row for a set of vectors to span $\mathbb{R}^n$.

Linear independence

A set of vectors $\{\vec{a}_1, \vec{a}_2, \ldots, \vec{a}_n\}$ is said to be linearly independent if and only if the only solution to the equation

$c_1\vec{a}_1 + c_2\vec{a}_2 + \cdots + c_n\vec{a}_n = \vec{0}$

is the trivial solution, $c_1 = c_2 = \cdots = c_n = 0$.

In other words, a set of vectors is linearly independent if and only if none of its elements is a linear combination of the other elements. Else, the set is considered linearly dependent.

Note: A set containing the zero vector $\vec{0}$ is automatically linearly dependent because there is always a non-trivial set of scalars (the constant corresponding to the zero vector can be anything).

Calculation

To determine whether a set of vectors is linearly independent, we find solutions to the equation

$c_1\vec{a}_1 + c_2\vec{a}_2 + \cdots + c_n\vec{a}_n = \vec{0}$.

$\begin{bmatrix}| & | & & | \\ \vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_n \\ | & | & & |\end{bmatrix}\begin{bmatrix}c_1 \\ c_2 \\ \vdots \\ c_n\end{bmatrix} = \vec{0}$

$\left[\begin{array}{cccc|c}\vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_n & \vec{0}\end{array}\right]$

Therefore, we just row-reduce the augmented matrix $\left[\begin{array}{cccc|c}\vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_n & \vec{0}\end{array}\right]$. Since every row operation will preserve the $0$ in the last column of the augmented matrix, we usually omit the last column and only row-reduce the matrix $\begin{bmatrix}\vec{a}_1 & \vec{a}_2 & \cdots & \vec{a}_n\end{bmatrix}$.

Since $c_1 = c_2 = \cdots = c_n = 0$ if the set if linearly independent, we should not find any free variables (which could be any value). Therefore, we should find pivots in every column.

Note: For a set of $m$ vectors of size $n$:

• If we have pivots in every row, we know that the vectors span the entire subspace $\mathbb{R}^n$.
• If we have pivots in every column, we know that the vectors are linearly independent.

Note: An $m \times n$ matrix, where $m < n$, has at most $m$ pivots. Therefore, there can never be pivots in each columns (or there would have to be $n$ pivots), so the column vectors of such a matrix are automatically linearly dependent. In other words, if the number of vectors in a set exceeds the number of elements in each vector, then this set is automatically linearly dependent. Furthermore, having a “wide” matrix does not guarantee that the set of vectors span $\mathbb{R}^m$.

Example 1

Determine whether $\begin{bmatrix}1 \\ 0 \\ 1\end{bmatrix}$, $\begin{bmatrix}1 \\ 2 \\ 0\end{bmatrix}$, and $\begin{bmatrix}5 \\ 4 \\ 3\end{bmatrix}$ are linearly independent.

We first row-reduce the matrix:

$\begin{bmatrix}1 & 1 & 5 \\ 0 & 2 & 4 \\ 1 & 0 & 3\end{bmatrix} \sim \begin{bmatrix}1 & 1 & 5 \\ 0 & 1 & 2 \\ 0 & 0 & 0\end{bmatrix}$

We see that there is no pivot in the third column. Therefore, the given vectors are linearly dependent.

Example 2

Determine whether $\begin{bmatrix}1 \\ 0 \\ 1\end{bmatrix}$ and $\begin{bmatrix}1 \\ 2 \\ 0\end{bmatrix}$ are linearly independent.

Let's row-reduce the matrix:

$\begin{bmatrix}1 & 1 \\ 0 & 2 \\ 1 & 0\end{bmatrix} \sim \begin{bmatrix}1 & 1 \\ 0 & 2 \\ 0 & 1\end{bmatrix} \sim \begin{bmatrix}1 & 1 \\ 0 & 1 \\ 0 & 0\end{bmatrix}$

We observe that there is a pivot in every column of the matrix. Therefore, the given vectors are linearly independent.

Extra Resources

Links to Guerrilla Section Problems and solutions

Guerilla section 1 Problems

https://d1b10bmlvqabco.cloudfront.net/attach/j6h2cdky5zk7c6/hz1x03ot4mb2mj/j7f2i2m14rgn/gs1_final.pdf

Guerilla section 1 Solutions

https://d1b10bmlvqabco.cloudfront.net/attach/j6h2cdky5zk7c6/hz1x03ot4mb2mj/j7f6yufr2rvo/gs1_solutions.pdf

Links to Exam Problems

https://tbp.berkeley.edu/courses/ee/16A/

https://hkn.eecs.berkeley.edu/exams/course/ee/16a

Vidoes elaborating guassian elimination

https://www.youtube.com/watch?v=2j5Ic2V7wq4 https://www.youtube.com/watch?v=xCIXkm3-ocQ https://www.youtube.com/watch?v=g0h_s1p0ax0