# Chapter 11: Linear Algebra

Linear Algebra in a huge field of mathematics and many standard operations in linear algebra is done in Maple or other CAS software. The core of Linear Algebra is a matrix, a rectangular grid of numbers.

## Linear Algebra library

The Linear Algebra library has many more commands that can be used. To load it type

with(LinearAlgebra):

and take a look at the help page for it:

?LinearAlgebra

## Vectors

There are two standard types of Vectors, column vectors and row vectors and Maple has both. If you want to enter the vector
$$\left[\begin{array}{c} 4 \newline -2 \newline 3 \end{array} \right]$$

you can type:

v:=Vector([4,-2,3])

and let’s say that you have a vector

w:=Vector([2,0,3])

which will be another column vector. If you want a row vector, like
$$\left[\begin{array}{c} 1 & 2 & 3 \end{array}\right]$$
you can enter this as

Vector[row]([1,2,3])

Check out other ways to create a vector using ?Vector

### Dot and Cross Products

You can take the dot product of two column vectors of the same length with the command DotProduct, which is part of the LinearAlgebra library. For example

DotProduct(v,w)

returns 17.

and to find the cross product, try

CrossProduct(v,w)

which returns the vector
$$\left[\begin{array}{c} -6 \newline -6 \newline 4 \end{array}\right]$$

## Matrices

To enter a matrix in Maple, it is entered row by row. Consider the matrix:
$$A=\left[\begin{array}{cccc} 1 & 2 & 3 & 4 \newline 5 & 6 & 7 & 8 \newline 9 & 10 & 11 & 12 \end{array} \right]$$
you can enter this as

A:=Matrix([[1,2,3,4],[5,6,7,8],[9,10,11,12]])

or you can use the Matrix palette. Enter the number of rows as 3 and columns as 4 and leave the Type as Custom Values and Maple will insert a generic 3 by 4 matrix. You can tab to get to each of the elements of the matrix.

Let’s also define the matrix $B$ to be a 3 by 4 matrix of all ones, which can be done with

B:=Matrix(3,4,fill=1)

and this will look like:
$$\left[\begin{array}{cccc} 1 & 1 & 1 & 1 \newline 1 & 1 & 1 & 1 \newline 1 & 1 & 1 & 1 \end{array} \right]$$

And a 3 by 3 matrix with the numbers 2, $-1$ and 4 on the diagonal,

C:=DiagonalMatrix([2,-1,4],3,3)

and this will look like
$$\left[\begin{array}{cccc} 2 & 0 & 0 \newline 0 & -1 & 0 \newline 0 & 0 & 4 \end{array} \right]$$

We can add the matrices $A$ and $B$ because they are the same size and can using standard notation:

A+B

which will return:
$$\left[\begin{array}{cccc} 2 & 3 & 4 & 5 \newline 6 & 7 & 8 & 9 \newline 10 & 11 & 12 & 13 \end{array} \right]$$

and if we subtract

A-B

Maple returns
$$\left[\begin{array}{cccc} 0 & 1 & 2 & 3 \newline 4 & 5 & 6 & 7 \newline 8 & 9 & 10 & 11 \end{array} \right]$$

If we try to add $A$ and $C$, we get the error:

Error, (in rtable/Sum) invalid input: dimensions do not match: Matrix(1 .. 3, 1 .. 4) cannot be added to Matrix(1 .. 3, 1 .. 3)

this is because the two matrices are not the same size.

### Scalar multiplication

Recall that Scalar multiplication of either a vector or a matrix results in a vector a matrix that is the product of the scalar and each element of the vector or matrix. For example, $3v$ for the vector $v$ above is
$$\left[\begin{array}{c} 12 \newline -6 \newline 9\end{array}\right]$$
and $-2A$ is

$$\left[ \begin {array}{cccc} -2&-4&-6&-8\newline -10&-12&- 14&-16\newline-18&-20&-22&-24\end {array} \right]$$

### Multiplying Matrices

Recall that two multiply 2 matrices, the number of columns of the first matrix must equal the number of rows of the second matrix. For example if we have the matrix

P:=Matrix([[3,2,0],[1,-2,4],[0,2,5]])

If we multiply $P$ and $A$, we use Maple’s dot notation or $P.A$ to get

$$\left[ \begin {array}{cccc} 13&18&23&28\newline 27&30&33& 36\newline 55&62&69&76\end {array} \right]$$

We can also use the dot notation to multiply matrices and vectors. For example, $P.v$ is

$$\left[\begin{array}{c} 8\newline20\newline11\end{array}\right]$$

If we don’t have compatibly-sized matrices, for example, $A$ and $B$, and try $A.B$, we get:

Error, (in LinearAlgebra:-Multiply) first matrix column dimension (4) <> second matrix row dimension (3)

### Exercise

Let

$$A=\left[\begin{array}{cc} 1 & 3 \newline -2 & 4 \newline 5 & 0\end{array}\right]$$
$$B=\left[\begin{array}{ccc} 3 & 2 & 3 \newline 0 & 2 & 0 \newline -1 &4 & -1 \end{array}\right]$$
$$C=\left[\begin{array}{cc} 0 &4 \newline -2 & 1 \end{array}\right]$$
$$\vec{v} = \left[\begin{array}{c} 4 \newline 2 \end{array}\right]$$
$$\vec{u} = \left[\begin{array}{c} 0 \newline 5 \newline -2 \end{array}\right]$$

Find the following or state the operation is not compatible.

1. $AB$
2. $AC$
3. $BC$
4. $A^{T}B$
5. $A\vec{v}$
6. $A\vec{u}$
7. $B\vec{u}$.

## Idenity Matrix

A square matrix with all zeros except 1’s on the diagonal, is called an identity matrix. The identity matrices of size 2 by 2, 3 by 3 and 4 by 4 are

$$\begin{array}{ccc} \left[\begin{array}{cc}1&0\newline0&1\end{array}\right]&\left[\begin{array}{ccc}1&0&0\newline0&1&0\newline0&0&1\end{array}\right]& \left[\begin{array}{cccc}1&0&0&0\newline0&1&0&0\newline0&0&1&0\newline0&0&0&1\end{array}\right]\end{array}$$

An identity matrix has the property that if you mulitply a matrix or a vector by the identity matrix, the matrix or vector is returned. More particularly,
$$\begin{array}{cccc}IA=A & AI=A & I\vec{v}=\vec{v} & \vec{v}I=\vec{v}\end{array}$$

To get an Identity matrix in Maple, use the IdentityMatrix command. For example

IdentityMatrix(3)

returns a 3 by 3 identity matrix.

### Exercise

Using the vector $\vec{v}$ and matrix $A$ above, find

1. $I\vec{v}$
2. $\vec{v}I$
3. $IA$
4. $AI$.

and show that you get the vector or matrix back. Make sure that you use the identity matrix of the correct size.

## Matrix Transpose

Recall that the transpose of a matrix is a new matrix where the rows of the tranpose are the columns of the original matrix. For example,

$$A^{T} = \left[\begin{array}{ccc} 1 & 5 & 9 \newline 2 & 6 & 10 \newline 3 & 7 & 11 \newline 4 & 8 & 12 \end{array}\right]$$

This can be done in Maple using the Tranpose command:

Transpose(A)

returns the matrix above. (Note: Transpose is in the LinearAlgebra package so make sure that it is loaded.)

## Gaussian Elimination

One of the first concepts in linear algebra is how to solve linear systems. For example,
$$\begin{array}{rl} 3x- 2y & = 4 \newline x + 5y & = 7 \end{array}$$

Maple can solve this by using the solve command such as

solve([3x-2y=4,x+5y=7])

which returns ${x=2,y=1}$. In linear algebra, we find the augmented coefficient matrix as
$$\left[\begin{array}{ccc} 3 & -2 & 4 \newline 1 & 5 & 7 \end{array}\right]$$

One method is to use row operations on the matix to reduce the matrix to a more useful form. One of these is called Gaussian Elimination. If we define the matrix above by $A$, then in Maple,

GaussianElimination(A)

returns $$\left[ \begin {array}{ccc} 3&-2&4\newline 0&17/3&17/3\end {array} \right]$$

To find the solution, we can use back substitution

The last row can be written back in equation form as
$$\frac{17}{3} y = \frac{17}{3}$$
or $y=1$. Then writting the first equation:
$$3x-2y=4$$
or substituting in $y=1$
$$\begin{array}{rl}3x-2(1)& =4\newline 3x& = 6 \newline x & = 2\end{array}$$

## Reduced Row Echelon Form

Maple can help us even more. If we type:

ReducedRowEchelonForm(A)

This returns:
$$\left[\begin{array}{ccc}1 & 0 & 2\newline0 &1 &1\end{array}\right]$$
and this can be written as the two equations $x=2$, $y=1$, same as above.

### Exercise

The linear system
$$\begin{array}{rl}-3x+4y-5z & = 8\newline -4x-2y-z & = 0 \newline 5x-4y+3z & = -16\end{array}$$
can be written as an Augmented Coefficient Matrix:
$$\left[\begin{array}{cccc}-3 & 4 & -5 & 8\newline -4&-2&-1&0\newline5&-4&3&-16\end{array}\right]$$

Use ReducedRowEchelonForm to reduce the matrix and then write the solution.

## Determinants

The determinant of a matrix is helpful in many different matrix applications including determining if the matrix is singular and it can be used in solving linear systems using Cramer’s Rule.

The determinant is applied only to square matrices and is denoted ${\rm det}(A)$ or $|A|$. If the matrix is a 2 by 2, then
$$\left|\begin{array}{cc} a & b \newline c & d \end{array} \right|=ad-bc$$

When finding a determinant, often Laplace’s Expansion method or row operations are used to find the value. For example

$$\left|\begin {array}{ccc} 2&-3&1\newline 3&3&0\newline2&3&-3\end {array} \right|=2\left|\begin{array}{cc}3&0\newline3&-3\end{array}\right|-(-3)\left|\begin{array}{cc}3&0\newline2&-3\end{array}\right|+1\left|\begin{array}{cc}3&3\newline2&3\end{array}\right|$$

$$\begin{array}{rl}&=2\left((3)(-3)-(3)(0)\right)+3\left((3)(-3)-2(0)\right)+\left((3)(3)-2(3)\right)\newline &=2(-9)+3(-9)+(9-6)=-42\end{array}$$

And using Maple on the 3 by 3 matrix $A$ above (again this in the LinearAlgebra library)

Determinant(A)

returns $-42$ as well.

## Matrix Inverse

If a matrix, $A$ is square and it’s determiant is not zero, then there exists a matrix called the inverse and denoted with the property $AA^{-1}=I$,

One way to find the matrix inverse of $A$ is write the matrix augmented with the identity matrix and then put it in Reduced Row echelon form.

For example, consider the matrix

$$A=\left[\begin{array}{ccc}-3&4&-5\newline-4&-2&-1\newline5&-4&3\end{array}\right]$$
then we write the augmented matrix where the 3 by 3 identity matrix is append on the right side. We can to this using the command:

Aug:=Matrix([A,IdentityMatrix(3)])

which returns $$\left[\begin{array}{ccc}-3&4&-5&1&0&0\newline-4&-2&-1&0&1&0\newline5&-4&3&0&0&1\end{array}\right]$$

and using the ReducedRowEchelonForm command, we get:

$$\left[ \begin {array}{cccccc} 1&0&0&{\frac {5}{36}}&-\frac{1}{9}&{\frac {7}{ 36}}\newline 0&1&0&-{\frac {7}{72}}&-\frac{2}{9}&-{\frac {17}{72}} \newline 0&0&1&-{\frac {13}{36}}&-\frac{1}{9}&-{\frac {11}{36}} \end {array} \right]$$

and the right half of the matrix is the inverse or
$$A^{-1}=\left[ \begin {array}{ccc} {\frac {5}{36}}&-\frac{1}{9}&{\frac {7}{ 36}}\newline -{\frac {7}{72}}&-\frac{2}{9}&-{\frac {17}{72}} \newline -{\frac {13}{36}}&-\frac{1}{9}&-{\frac {11}{36}} \end {array} \right]$$

We can also get the same result from the command MatrixInverse:

MatrixInverse(A)

returns the same matrix above.

### Exercise

1. Find the inverse of
$$C=\left[ \begin {array}{ccc} -1&-3&0\newline-3&2&-1\newline1&-3&0\end {array} \right]$$
using the augmented matrix with the identity matrix.
2. Find the inverse of $C$ using the command MatrixInverse
3. What happens when you try to find the inverse of $$\left[ \begin {array}{ccc} -1&-3&0\newline-3&2&-1\newline2&-5&1\end{array}\right]?$$

### Using the Matrix Inverse to solve a linear system.

Above we saw the linear system:

The linear system
$$\begin{array}{rl}-3x+4y-5z & = 8\newline -4x-2y-z & = 0 \newline 5x-4y+3z & = -16\end{array}$$

and used the Augumented Matrix to find the solution. Instead, we can write:
$$\begin{array}{rl}A=\left[ \begin {array}{ccc} -3&4&-5\newline-4&-2&-1 \newline5&-4&3\end {array} \right]&\vec{b}=\left[\begin{array}{c}8\newline0\newline-16\end{array}\right]\end{array}$$

and then if we write the vector
$$\vec{x}=\left[\begin{array}{c}x\newline y\newline z\end{array}\right]$$, then the linear system can be written as $A\vec{x}=\vec{b}$. We can solve this in the following way.
$$A\vec{x}=\vec{b}$$
Multiply the equation by $A^{-1}$
$$\begin{array}{rl}A^{-1}A\vec{x}&=A^{-1}\vec{b}\newline \text{since A^{-1}A=I\qquad\qquad}&\newline I\vec{x}&=A^{-1}\vec{b}\newline \text{since I\vec{x}=\vec{x}\qquad\qquad}&\newline \vec{x}&=A^{-1}\vec{b}\end{array}$$

Use this to solve the linear system above.

First, we will find $A^{-1}$ using the MatrixInverse command:

Ainv:=MatrixInverse(A)

which returns
$$\left[ \begin {array}{ccc} {\frac {5}{36}}&-\frac{1}{9}&{\frac {7}{ 36}}\newline -{\frac {7}{72}}&-\frac{2}{9}&-{\frac {17}{72}} \newline -{\frac {13}{36}}&-\frac{1}{9}&-{\frac {11}{36}} \end {array} \right]$$
as we saw above.

Lastly, we multiply $A^{-1}$ and $\vec{b}$

Ainv.b

to get:
$$\left[\begin{array}{c}-2\newline3\newline2\end{array}\right]$$

so the result is $x=-2,y=3,z=2$, the same as we found above.

## Eigenvalues and Eigenvectors

Another important topic in Linear Algebra is that of eigenvalues and eigenvectors. Recall that a scalar (number) $\lambda$ is an eigenvalue of a square matrix $A$ with corresponding eigenvector $\vec{v}$ if this satisfies the equation
$$A\vec{v}=\lambda\vec{v}.$$
Finding $\lambda$ and $\vec{v}$ that satisfy this require two steps:

1. The eigenvalues can be found by solving
$$|A-\lambda I|=0$$where the vertical lines is the determinant of the matrix.

2. For each eigenvalue, you seek for a vector $\vec{v}$ such that
$$(A-I\lambda)\vec{v}=\vec{0}$$

### Example

Find the eigenvalues and eigenvectors of the matrix
$$A=\left[\begin{array}{cc}4&-1\newline-2&3\end{array}\right]$$

Step 1: Solve $|A-\lambda I|=0$

Determinant(A-lambda*IdentityMatrix(2))=0

returns $\lambda^{2}-7\lambda+10=0$. This equation is called the characteristic equation of the matrix $A$. And then

solve(#)

where (#) is the line number of the equation, Maple returns $5,2$. There are two eigenvalues for this. Note: the maximum number of eigenvalues for an $n$ by $n$ matrix is $n$.

Step 2: For each of the eigenvalues, solve $(A-\lambda I)\vec{v}=0$. First, consider $\lambda=5$

A-5*IdentityMatrix(2)

return the matrix: $$\left[\begin{array}{cc}-1&-1\newline-2&-2\end{array}\right]$$.
To solve this we will append the zero vector on the right or

Aug:=Matrix([A-5*IdentityMatrix(2),Vector([0,0])])

returns $$\left[\begin{array}{ccc}-1&-1&0\newline-2&-2&0\end{array}\right]$$
and then

ReducedRowEchelonForm(Aug)

which returns $$\left[\begin{array}{ccc}1&1&0\newline0&0&0\end{array}\right]$$

The first row of the equation, means that $x+y=0$. We seek any nonzero solution to this, if $y=-1$ and $x=1$, then the vector is
$$\vec{v}_1=\left[\begin{array}{c}1\newline-1\end{array}\right]$$.

Next, find the eigenvector with eigenvalue $\lambda=2$.

Aug:=Matrix([A-2*IdentityMatrix(2),Vector([0,0])])

returns $$\left[\begin{array}{ccc} 2&-1&0\newline -2&1&0\end{array} \right]$$

and putting in reduced row echelon form:
$$\left[\begin{array}{ccc} 2&-1&0\newline 0&0&0\end{array} \right]$$

Lastly, we find the eigenvector associated with this. The first row is $2x-y=0$ and we seek a nonzero solution to this. Let $x=1$, then $y=2$ and the vector will be
$$\vec{v}_2=\left[\begin{array}{c}1\newline2\end{array}\right]$$

In summary for the matrix
$$A=\left[\begin{array}{cc}4&-1\newline-2&3\end{array}\right]$$

The eigenvalues are 5 and 2 and the associated eigenvectors are
$$\begin{array}{rl}\vec{v}_1=\left[\begin{array}{c}1\newline-1\end{array}\right]& \vec{v}_2=\left[\begin{array}{c}1\newline2\end{array}\right]\end{array}$$

### Exercise

Use the steps above to find the eigenvalues and eigenvectors of
$$A=\left[\begin{array}{cc}6&2\newline4&4\end{array}\right]$$

### More Built-in commands for Maple

Recall that above, the first step in finding the eigenvalues of the matrix is to write down the characteristic equation of the matrix or $|A-\lambda I|=0$. If we return to the matrix above,
$$A=\left[\begin{array}{cc}4&-1\newline-2&3\end{array}\right]$$
we can use Maple to find the characteristic equation by

CharacteristicPolynomial(A,lambda)=0

which returns as above, $\lambda^{2}-7\lambda+10=0$, which we can solve with the solve command to get $5,2$.

Then the next step was to solve $(A-\lambda I)\vec{v}$ for the given values of $\lambda$. Another name for this is the null space of the matrix. If we enter

NullSpace(A-5*IdentityMatrix(2))

Maple returns
$$\left\{ \left[\begin{array}{c}-1\newline 1\end{array}\right] \right\}$$

so the only vector in the null space is the one above. This is just $-1$ times the eigenvector that we found above. (Note: there is a theorem that says that any nonzero scalar times an eigenvector is also an eigenvector. In other words, eigenvectors are not unique.)

If we type:

NullSpace(A-2*IdentityMatrix(2))

Maple returns
$$\left\{\left[\begin{array}{c}1/2\newline1\end{array}\right]\right\}$$

which is $1/2$ times the eigenvector that we found.

### Exercise

Use the CharacteristicPolynomial and NullSpace commands to find the eigenvalues and eigenvectors of
$$A=\left[\begin{array}{cc}6&2\newline4&4\end{array}\right]$$

that found above.

### Maple can find the eigenvalues and eigenvectors in one shot!!!

Of course, Maple can just find the eigenvalues and eigenvectors as well.

Returning to the matrix:$$A=\left[\begin{array}{cc}4&-1\newline -2&3\end{array}\right]$$
type

Eigenvalues(A)

and one gets:
$$\left[\begin{array}{c}5\newline2\end{array}\right]$$

Note: this is not a vector, but the two eigenvectors are just 5 and 2 as we saw above.

To find the eigenvectors, type

Eigenvectors(A)

and you get:
$$\left[\begin{array}{c}5\newline2\end{array}\right],\left[\begin{array}{cc}\frac{1}{2} & -1\newline1&1\end{array}\right]$$

The first vector is the eigenvalues again and the matrix contain the eigenvectors as column vectors. Note: these are the same eigenvectors that we found using the NullSpace command.