Curso completo de algebra lineal, Apuntes de Álgebra Lineal

¡Descarga Curso completo de algebra lineal y más Apuntes en PDF de Álgebra Lineal solo en Docsity! Math 221 Course Notes Chad Davis Updated May 25, 2021 2 CONTENTS iii 6.3.1 Diagonal Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222 6.3.2 Similar Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224 6.3.3 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227 6.4 An Algorithm for Diaongalization & Examples . . . . . . . . . . . . . . . . . . . . . . . . . . 232 6.5 Systems of First Order Differential Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 iv CONTENTS Foreword Linear algebra is the study of linear transformations of vector spaces. A linear transformation is a func- tion that has sets of vectors as its domain and range. A linear transformation is an abstract mathematical construction. The amazing thing about linear algebra is that, under the right conditions, these abstract objects can always be represented by something concrete: an array of numbers known as a matrix. The focus of this course is matrices and their various properties. Learning about matrices in a tangible sense will allow us to learn about the abstract linear transformations. This is a remarkably wonderful thing. We will learn various ways to manipulate matrices and learn how to perform algebraic operations on them, such as addition and multiplication. We will see how all of these things relate to linear transformations in a meaningful way. This is what makes linear algebra such a beautiful subject. We are able to study and learn about complicated, abstract mathematics via objects that we can manipulate in a completely hands on manner. Chapter 1 Linear Systems and Matrices In this chapter, we introduce matrices via linear systems. You have been solving systems of linear equations since high school. Matrices are a natural way to encode all the relevant data in a linear system and can be manipulated in such a way to gain solutions to such systems in a very efficient manner. Moreover, there are many real world problems that can be modelled using linear systems, so we see immediately that matrices find application in many areas. 1.1 Linear Systems We begin with a high school math problem. Example 1.1.1 Wentworth Music sells two types of guitars: Fender Stratocasters for $350 each and Gibson Les Pauls for $600 each. Last year, they sold 230 guitars and made $100,000 from their total sales. How many of each guitar did they sell? Solution. Let x1 be the total number of Stratocasters sold and let x2 be the total number of Les Pauls sold. Since 230 guitars were sold in total, we have the following equation, x1 + x2 = 230. (1.1) We also know that the total revenue Wentworth music made off of guitars last year is $100,000. Since each Stratocaster is sold for $350 and each Les Paul is sold for $600, we get another equation, 350x1 + 600x2 = 100, 000. (1.2) You know how to solve such an equation from high school. You would have seen a number of different ways to do it. You might have graphed the two lines and found their intersection point, or you may have 1 4 CHAPTER 1. LINEAR SYSTEMS AND MATRICES equations. If you have tried to do this before then you know it can get a little bit tedious. In application, linear systems usually have tens of thousands of variables and equations. 1.1.1 Geometry of Linear Systems The graph of a linear equation in two variables is a line. The graph of a linear equation in three variables is a plane, which is a 3-dimensional analogue of a line. Unfortunately, we can’t visualize anything in more than three dimensions, so we can’t graph a linear equation in more than three variables. However, a linear equation in n-variables is merely the n-dimensional analogue of a line. These are called n-dimensional hyperplanes. This is the reason why linear equations are given their name! Let’s take a closer look at what a linear system in two variables looks like when we graph it. Certainly this will be a finite number of lines, one for each equation in the system. How does the solution to a linear system come into play? To gain some insight, let’s graph the linear system in Example 1.1.1 on a set of coordinate axes. 150 151 152 153 154 76 77 78 79 80 x1 x 2 x1 + x2 = 230 350x1 + 600x2 = 100, 000 The solution to the linear system is exactly the point where these two lines intersect. Since there is only one intersection point, there is only one solution to the linear system. This is only one example of what can happen. Another of what can happen is a linear system of equations in two variables and two equations where the equations represent lines that are parallel and have different x2-intercepts. The graph of such a linear system is given below. 1.1. LINEAR SYSTEMS 5 −10 −5 0 5 10 −20 −10 0 10 20 x1 x 2 −2x1 + x2 = 4 −2x1 + x2 = 0 The lines are parallel, so they never intersect. Therefore, this linear system has no solutions. Another scenario we can find ourselves in is if two lines are lying on top of one another. This happens when the equations representing the lines are multiples of one another. Since these lines intersect in infinitely many points, such a linear system has infinitely many solutions. Pictured below is an example of this. −10 −5 0 5 10 −5 0 5 10 15 x1 x 2 x1 + x2 = 5 −x1 − x2 = −5 In the above, the blue line is sitting on top of the red line so you can’t see it. What if we have more than two equations? In general, a single solution to a linear system in two variables with n equations will be a point at which the n lines in the system intersect simultaneously. It is important to realize that all of the lines have to meet at this point. If only some of the lines intersect in a common point, this is not enough for that point to be a solution to the system. Here are two examples: the first linear system has one solution, the second has none. 6 CHAPTER 1. LINEAR SYSTEMS AND MATRICES −10 −5 0 5 10 −10 −5 0 5 10 x1 x 2 −x1 + 2x2 = 1 −x1 + x2 = 0 x1 + x2 = 2 −10 −5 0 5 10 −10 −5 0 5 10 15 x1 x 2 −x1 + 2x2 = 3 −x1 + x2 = 0 x1 + x2 = 2 Since lines don’t curve, it seems that these are the only three scenarios we could wind up with in terms of solutions to linear systems. This is exactly true. If we had a linear system in 3 variables, the same would also be true as planes don’t curve either. In fact, this generalizes to any linear system in n variables. Fact 1.1.1: Number of Solutions of Linear Systems A linear system in k variables with n equations has either i) One unique solution, ii) Infinitely many solutions, iii) No solutions. 1.2. MATRICES 9 Example 1.1.1. Example 1.2.3 The linear system in Example 1.1.1 and its corresponding augmented matrix are given below x1 + x2 = 230 350x1 + 600x2 = 100, 000 ⇐⇒ [ 1 1 230 350 600 100, 000 ] . Denote the first row by R1. R1 represents the equation x1+x2 = 230. The second row, R2, represents the equation 350x1 + 600x2 = 100, 000. To find a solution to this linear system, the first thing we did was subtract 350 times the first equation from the second. We represent this operation in the matrix by replacing R2 with 350 times R1 subtracted R2. This is denoted by R2 ⇒ R2− 350R1. The resulting linear system and corresponding augmented matrix is R2 ⇒ R2 − 350R1 : x1 + x2 = 230 250x2 = 19, 500 ⇐⇒ [ 1 1 230 0 250 19500 ] . The second row of the new matrix represents the equation 250x2 = 19500. We solve for x2 by dividing by 250. This is reflected in the matrix by replacing R2 with (1/250) times R2. This is written R2 ⇒ (1/250)R2. The resulting linear system and corresponding augmented matrix is R2 ⇒ (1/250)R2 : x1 + x2 = 230 x2 = 78 ⇐⇒ [ 1 1 230 0 1 78 ] . We can now solve for x1 by subtracting the second equation from the first. This is written as R1 ⇒ R1 −R2 and the resulting linear system and corresponding augmented matrix is R1 ⇒ R1 −R2 : x1 = 152 x2 = 78 ⇐⇒ [ 1 0 152 0 1 78 ] . The solution of the new linear system is exactly the solution from that in Example 1.1.1. In Example 1.2.3, we perform two operations on the rows of the augmented matrix to find the solution of the corresponding linear system. One is adding a multiple of one row to another and the other is multiplying a row by a non-zero number. These are two of the elementary row operations; that is one more that can be used as well. Definition 1.2.3: Elementary Row Operations & Row Equivalence Let A be an n× k matrix. The elementary row operations on A are the following. 1) Row Replacement. Add a non-zero multiple of one row to another. If c is a non-zero number, then replacing row i by row i plus c times row j is denoted by Ri ⇒ Ri + cRj . 2) Swapping Rows. Interchange two rows. Interchanging row i with row j is denoted Ri ⇐⇒ Rj . 10 CHAPTER 1. LINEAR SYSTEMS AND MATRICES 3) Scaling Rows. Multiply one row by a non-zero number. If c is a non-zero number, then row i replaced by c times row i is denoted Ri ⇒ cRi. Two matrices that can be transformed into one another using a finite sequence of elementary row operations are called row equivalent. If A and B are row equivalent matrices, we write A ∼ B. I generally refer to elementary row operations simply as “row operations.” Example 1.2.4 The following two matrices are row equivalent A =  2 1 1 4 −3 2 −2 −10 1 −2 3 7  , B =  1 0 0 2 0 1 0 −1 0 0 1 1  . Show this is true by finding the sequence of row operations that transform A into B. Solution. What I’m going to show is my solution to this problem. There are many ways to proceed that will give the same answer. However, the way I’m going to perform the operations is foreshadowing for the algorithm we see in the next section. Start in the top left corner of both matrices. The (1,1)-entry of B is a 1. To transform A into B, we must make the (1,1)-entry of A into 1. This can be done by swapping the first and third rows of A. R1 ⇐⇒ R3 : A ∼  1 −2 3 7 −3 2 −2 −10 2 1 1 4  = A1. We now need zeroes below the 1 in the (1,1)-position. We can make the two entries below the one zeroes by doing the following two row operations. R2 ⇒ R2 + 3R1 : A1 ∼  1 −2 3 7 0 −4 7 11 2 1 1 4  = A2, R3 ⇒ R3 − 2R1 : A2 ∼  1 −2 3 7 0 −4 7 11 0 5 −5 −10  = A3. Now we need a 0 below the -4 in the (2,2)-position of A3. To get a 0 there, we could do the row replacement R3 ⇒ R3 + (5/4)R2. However, the more you do row operations on matrices, you’ll come to learn this whole process is a lot easier if you keep the math as simple as possible, which means you don’t introduce fractions unless it is absolutely necessary. Hence, in lieu of the aforementioned row operation, we swap the second and third row and then divide the second row by 5 in order to get a 1 in the (2,2)-position. 1.2. MATRICES 11 R2 ⇐⇒ R3 : A3 ∼  1 −2 3 7 0 5 −5 −10 0 −4 7 11  = A4, R2 ⇒ (1/5)R2 : A4 ∼  1 −2 3 7 0 1 −1 −2 0 −4 7 11  = A5. We get a 0 in the (3,2)-position by doing a row replacement. R3 ⇒ R3 + 4R2 : A5 ∼  1 −2 3 7 0 1 −1 −2 0 0 3 3  = A6. Now we need a 1 in the (3,3)-position. We get this by dividing the third row by 3. R3 ⇒ (1/3)R3 : A6 ∼  1 −2 3 7 0 1 −1 −2 0 0 1 1  = A7. We can get zeroes in the (2,3) and (1,3) positions by doing two row replacements. R2 ⇒ R2 +R3 : A7 ∼  1 −2 3 7 0 1 0 −1 0 0 1 1  = A8, R1 ⇒ R1 − 3R3 : A8 ∼  1 −2 0 4 0 1 0 −1 0 0 1 1  = A9. Finally, we need a 0 in the (1,2) position. We can get this by doing one final row replacement R1 ⇒ R1 + 2R2 : A9 ∼  1 0 0 2 0 1 0 −1 0 0 1 1  = B. We have now transformed A into B via a finite sequence of elementary row operations. Therefore, A ∼ B. ♦ The matrix B in the previous example is in a special form called reduced row echelon form. The steps followed in this example illustrate a standard algorithm used to transform a given matrix into reduced row echelon form. We explore this algorithm in the next section. 14 CHAPTER 1. LINEAR SYSTEMS AND MATRICES Example 1.3.3 The matrices A1, A2, A5, A6 of Example 1.3.1 are all in echelon form. The pivots of each matrix have been boldfaced and underlined: A1 =  1 3 0 0 3 5 0 0 0  , A2 = [ 1 0 4 3 1 0 1 3 9 10 ] , A5 =  1 1 0 0 0 1 0 0 0 0 0 0 0 0 0 0  , A6 =  2 1 3 0 0 4 0 0 0 0 0 17 0 0 0 0  . The first and second columns of A1, A2, and A5 are pivot columns. The first, second, and fourth columns of A6 are pivot columns. The pivot positions of A1 are the (1, 1) and (2, 2) positions. It is similar for the rest. The following result regarding pivot positions follows from RREF Is Unique and is very important. Fact 1.3.2: Pivots are Invariant Let A be an n× k matrix and let B and C be two echelon forms of A. Then, the pivot positions of B are the same as the pivot positions of C. This fact states that the pivot positions in any echelon form of a matrix A never change, though the values of the entries themselves might. It follows that the pivot columns of a matrix don’t change as we pass between different row equivalent echelon forms. Therefore, we are permitted to refer to the pivot posi- tions/columns of a matrix A whether it is in echelon form or not. Many of the questions in this course can be answered by identifying and counting pivot columns. Example 1.3.4 In Example 1.2.4, B is the RREF of A. The first three columns of B contain a pivot. Therefore, we refer to the first three columns of A as pivot columns even though it is not in echelon form. We now have enough machinery to introduce Gauss-Jordan elimination. This is an algorithm that will take any matrix and put it into RREF. Algorithm 1: Gauss-Jordan Elimination To transform an n× k matrix A into echelon form, perform the following steps: Step 1. Start in the left-most non-zero column and, if necessary, interchange rows so that the top entry is non-zero. This is a pivot position. If desired, scale the first row to make this first entry a 1. 1.3. ECHELON FORMS AND GAUSS-JORDAN ELIMINATION 15 Step 2. Use row replacement to create zeros in each position below the top pivot position. Step 3. Ignore the row that contains the pivot in step 1 and any rows above this row. Apply steps 1 and 2 to the matrix that results from ignoring these rows. Repeat this process until there are no more non-zero columns to apply it to. The resulting matrix will be in echelon form. To put the matrix into RREF, perform the following additional step: Step 4. Begin at the right most pivot. Working upwards, use replacement to create zeros in every entry above each pivot. Use scaling to make sure each pivot is equal to 1. Example 1.3.5 The steps executed in Example 1.2.4 to transform A into its RREF B is exactly the steps outlined in Gauss-Jordan Elimination. Note on Language The process of using Gauss-Jordan elimination to put a matrix into echelon form or RREF is re- ferred to as row reduction. We say “Row reduce A to echelon form” to mean “Use Gauss-Jordan elimination to put A into echelon form”. Gauss-Jordan Elimination looks confusing at first but after doing some examples, you’ll see that it’s pretty straightforward. The hardest part is executing the row operations properly. Example 1.3.6 Use Gauss-Jordan Elimination to row reduce the following matrix A to echelon form and locate the pivot columns; then put the matrix in RREF. A =  0 −3 −6 4 9 −1 −2 −1 3 1 −2 −3 0 3 −1 1 4 5 −9 −7  . Solution. We follow the steps outlined in Gauss-Jordan Elimination. Step 1. Starting in the left-most non-zero column, the first entry is a zero. Therefore, interchange two rows to make it not zero. The leading entry of row 4 is a 1. Since 1s are the easiest pivots to work with, we swap row 1 and row 4. 16 CHAPTER 1. LINEAR SYSTEMS AND MATRICES R1 ⇐⇒ R4 : A ∼  1 4 5 −9 −7 −1 −2 −1 3 1 −2 −3 0 3 −1 0 −3 −6 4 9  = A1. Step 2. There is a 1 in the correct pivot position. We need to make everything below this pivot a 0. Hence, we do two row replacements to make the entries below this 1 zero. R2 ⇒ R2 +R1 R3 ⇒ R3 + 2R1 : A1 ∼  1 4 5 −9 −7 0 2 4 −6 −6 0 5 10 −15 −15 0 −3 −6 4 9  = A2. Note the row operations are executed in order from the top down. Step 3. Ignore the row that contains the pivot position we were just working with. In this case, we ignore the first row. The resulting sub-matrix is B =  0 2 4 −6 −6 0 5 10 −15 −15 0 −3 −6 4 9  . The second column of this sub-matrix is the left-most non-zero column, which corresponds to the (2,2)- position in A2. Now repeat steps 1 and 2 starting at this pivot position. If we divide row 2 by 2, we get a 1 in this position, so we start with this row operation. R2 ⇒ (1/2)R2 : A2 ∼  1 4 5 −9 −7 0 1 2 −3 −3 0 5 10 −15 −15 0 −3 −6 4 9  = A3. Note, this row operation is not necessary, but it does make the math easier. To get zeroes below this new pivot, we do two row replacements. R3 ⇒ R3 − 5R2 R4 ⇒ R3 + 3R1 : A3 ∼  1 4 5 −9 −7 0 1 2 −3 −3 0 0 0 0 0 0 0 0 −5 0  = A4. Now we’re back at step 3. Ignoring the row containing the pivot in B, the new sub-matrix is C = [ 0 0 0 0 0 0 0 0 −5 0 ] . The fourth column is the left-most non-zero column. Applying step one, we interchange the first and second row of C so that the non-zero entry is at the top of the pivot column. These rows correspond to the third and fourth rows of A4. 1.4. SOLVING LINEAR SYSTEMS WITH MATRICES 19 x1 = 2 x2 = −1 x3 = 1 This linear system has solution (x1, x2, x3) = (2,−2, 1). Hence, by Proposition 1.4.1, the solution to the original linear system is (x1, x2, x3) = (2,−2, 1). ♦ Going forward, we don’t cite Proposition 1.4.1. We’ll take it as a fact. Example 1.4.2 Use a matrix to find the solution set to the following linear system, −3x2 − 6x3 + 4x4 = 9 −x1 − 2x2 − x3 + 3x4 = 1 −2x1 − 3x2 + 3x4 = −1 x1 + 4x2 + 5x3 − 9x4 = −7 Solution. The augmented matrix for the linear system is A =  0 −3 −6 4 9 −1 −2 −1 3 1 −2 −3 0 3 −1 1 4 5 −9 −7  . From Example 1.3.6, the RREF of A is A′ =  1 0 −3 0 −5 0 1 2 0 −3 0 0 0 1 0 0 0 0 0 0  . The linear system corresponding to A′ is x1 −3x3 = −5 x2 + 2x3 = −3 x4 = 0 Solving the first two equations in terms of x3 gives x1 = −5 + 3x3, and x2 = −3− 2x3. Therefore, as long as x1 and x2 have the values specified above, we get a solution to the linear system regardless of the value of x3. Letting x3 = s where s ∈ R, the solution to the original linear system is (x1, x2, x3, x4) = (−5 + 3s, −3− 2s, s, 0) for any s ∈ R. Since this is a solution for any real number s, this linear system has infinitely many solutions. ♦ 20 CHAPTER 1. LINEAR SYSTEMS AND MATRICES Exercise Pick numerous values of s and verify that (−5 + 3s, −3− 2s, s, 0) is indeed a solution for the linear system in Example 1.4.2. For a system with infinitely many solutions, a variable that can take on any real value is called a free variable because it is “free” to assume any value we like. Variables that are not free are called basic variables. In general, basic variables are written as functions of the free variables in the solution. In Example 1.4.2, x3 is a free variable because it can take any possible value and we still get a solution. x1 and x2 are basic variables. By Fact 1.1.1, a linear system can have one, zero, or infinitely many solutions. We have now seen examples of solving linear systems using matrices that have exactly one and infinitely many solutions. Let’s look at an example of a linear system that has no solutions and see how this effects the augmented matrix for the system. Example 1.4.3 Use a matrix to find the solution set to the following linear system x1 + x2 = 2 x1 − x2 = 0 x1 − 2x2 = −3 Solution. The augmented matrix for this linear system is A =  1 1 2 1 −1 0 1 −2 −3  . We use Gauss-Jordan Elimination to put this matrix into echelon form. Step 1. The first pivot is in the (1,1)-position, and it is a 1. Therefore, step 1 is over. Step 2. We must get zeroes below the 1 in the (1,1)-position. Therefore, we perform two row replacements on the matrix. R2 ⇒ R2 −R1 R3 ⇒ R3 −R1 : A ∼  1 1 2 0 −2 −2 0 −3 −5  = A1. This concludes step 2. Step 3. The next pivot is in the (2, 2)-position. We make it a 1 by dividing by 2, and then make the entry below a zero by doing a row replacement. 1.4. SOLVING LINEAR SYSTEMS WITH MATRICES 21 R2 ⇒ (−1/2)R2 R3 ⇒ R3 + 3R2 : A1 ∼  1 1 2 0 1 1 0 0 −2  = A2. This ends step 3. A2 is an echelon form of A. Before continuing 1 and going to RREF, let’s write out the linear system this echelon form corresponds to x1 + x2 = 2 x2 = 1 0x1 + 0x2 = −2 The last equation implies −2 = 0 which is impossible. This shows the linear system corresponding to A2 is impossible to solve and, hence, has no solution. Therefore, the original linear system has no solution. ♦ 1.4.2 The Existence/Uniqueness Theorem Any given linear system either has a solution or it does not. We give this duality a name. Definition 1.4.1: Consistency A linear system with at least one solution is called consistent. A linear system is called inconsistent if it has no solution. By Number of Solutions of Linear Systems, a consistent linear system has either exactly one or infinitely many solutions. Thus, we might ask ourselves the following two questions. 1. Given a linear system, is it consistent or inconsistent? 2. If the linear system is consistent, does it have exactly one or infinitely many solutions? The following theorem tells us how to answer the above two questions. Theorem 1.4.1: The Existence/Uniqueness Theorem Let A be an augmented matrix corresponding to a linear system and let B be an echelon form of A. The linear system represented by A is inconsistent if and only if the rightmost column of B is a pivot column. That is, A is inconsistent if and only if B has a row of the form[ 0 . . . 0 ∣∣ b] where b is a non-zero real number. If the linear system is consistent, then there are two possibilities: 1. All of the columns to the left of the vertical bar of B are pivot columns. Then, the linear system has exactly one solution. 24 CHAPTER 1. LINEAR SYSTEMS AND MATRICES pivot columns, but the third one is not. Therefore, the linear system will have infinitely many solutions and the variable x3 corresponding to the non-pivot column is the free variable in the solution. Step 4. We now need to go to RREF. We start at the pivot in the (2,2)-position and do the following R2 ⇒ −(1/25)R2 R1 ⇒ R1 − 4R2 : A2 ∼  1 0 1/25 −7/25 0 1 31/25 58/25 0 0 0 0  = A′. This is the RREF of the matrix. Step 5. Write out the linear system the matrix in RREF from step 4 represents. x1 + ( 1 25 ) x3 = − 7 25 x2 + ( 31 25 ) x3 = 58 25 Since x3 is the free variable, write x1 and x2 in terms of x3. x1 = − ( 1 25 ) x3 − 7 25 , and x2 = − ( 31 25 ) x3 + 58 25 . x3 is a free variable so it can take any real value. Therefore, write x3 = s where s ∈ R. The solution to the original linear system is (x1, x2, x3) = ( − 7 25 − ( 1 25 ) s, 58 25 − ( 31 25 ) s, s ) , s ∈ R. ♦ Note on Notation The notation s ∈ R is important. Whenever there is a solution involving free variable, you must write this because it tells the reader that s can be any real value. If this is not present, the solution means nothing. In the previous example, s ∈ R is any rel number. Therefore, we can pick it so that it clears denominators in the solution. Indeed, let s = 25t, where t ∈ R. Then, the solution to Example 1.4.4 can be written as (x1, x2, x3) = ( − 7 25 − t, 58 25 − 31t, 25t ) , t ∈ R. This is not necessary but sometimes certain authors will do this automatically. 1.4. SOLVING LINEAR SYSTEMS WITH MATRICES 25 Example 1.4.5 Determine whether the following linear system is consistent or not and write down the general solution, x1 + x3 = 1 x1 + x2 + x3 = 2 x1 − x2 + x3 = 1 Solution. We follow the procedure outlined in Solving a Linear System Using a Matrix. Step 1. The augmented matrix for this system is A =  1 0 1 1 1 1 1 2 1 −1 1 1  . Step 2. We do Gauss-Jordan Elimination to put A into echelon form. The first step is R2 ⇒ R2 −R1 R3 ⇒ R3 −R1 : A ∼  1 0 1 1 0 1 0 1 0 −1 0 0  = A1. The next pivot is in the (2,2)-position. We need a zero below this. Therefore, we do R3 ⇒ R3 +R2 : A1 ∼  1 0 1 1 0 1 0 1 0 0 0 1  = A2. This is an echelon form of A. Step 3. The rightmost column of the echelon form from step 2 is a pivot column. Therefore, the linear system is inconsistent by The Existence/Uniqueness Theorem. ♦ Warning! In Example 1.4.5, we have the matrix  1 0 1 1 0 1 0 1 0 −1 0 0  . The bottom row of this matrix does not imply inconsistency. Writing out the equation it represents gives −x2 = 0 which certainly is solvable. Do not be fooled into thinking this implies inconsistency. This is a very common mistake so try to avoid it. 26 CHAPTER 1. LINEAR SYSTEMS AND MATRICES Example 1.4.6 Determine the solution (if one exists) to the following linear system, 2x1 + x2 − 2x3 = 3 x1 − x2 − x3 = 0 x1 + x2 + 3x3 = 12 Solution. We follow the procedure outlined in Solving a Linear System Using a Matrix. Step 1. The augmented matrix for this system is A =  2 1 −2 3 1 −1 −1 0 1 1 3 12  . Step 2. Start Gauss-Jordan Elimination to put A into echelon form. We start in the first column and, even though there is a non-zero entry, in the appropriate pivot position, do a row swap to make the math easier. R1 ⇐⇒ R2 : A ∼  1 −1 −1 0 2 1 −2 3 1 1 3 12  = A1. We now need zeroes below the one at the top of the first column which we by performing row replacement. R2 ⇒ R2 − 2R1 R3 ⇒ R3 −R1 : A1 ∼  1 −1 −1 0 0 3 0 3 0 2 4 12  = A2. The next pivot is the 3 in the (2,2)-position. We make it a 1 and get a zero below it by doing a row scaling and a row replacement. R2 ⇒ (1/3)R2 R3 ⇒ R3 − 2R2 : A2 ∼  1 −1 −1 0 0 1 0 1 0 0 4 10  = A3. This is the echelon form of A. Step 3. The rightmost column of the echelon form of A in step 2 is not a pivot column. Therefore, the linear system is consistent. Furthermore, there are no non-pivot columns to the left of the vertical bar. Thus, the linear system has exactly one solution by The Existence/Uniqueness Theorem. Step 4. We now go to RREF. Start at the pivot in the (3,3)-position, make it a 1, and get zeroes above it. R3 ⇒ (1/4)R3 R1 ⇒ R1 +R3 : A3 ∼  1 −1 0 5/2 0 1 0 1 0 0 1 5/2  = A4. 1.4. SOLVING LINEAR SYSTEMS WITH MATRICES 29 As in Example 1.4.4, we can clear denominators by letting s = 11s′, t = 11t′, and writing (x1, x2, x3, x4) = ( 26 11 − 15s′ + t′, − 1 11 + s′ − 25t′, 11s′, 11t′ ) , s′, t′ ∈ R. ♦ We end the section with an example of a matrix that looks like it has free variables, but doesn’t because it has no solution. Example 1.4.8 Consider the following augmented matrix A =  1 0 2 3 0 2 3 0 0 0 0 1  . There is one non-pivot column to the left of the vertical bar, which typically would indicate the existence of a free variable in the corresponding linear system. However, because the last column of this matrix is a pivot column, the linear system has no solutions by The Existence/Uniqueness Theorem. Hence, it has no free variables. 30 CHAPTER 1. LINEAR SYSTEMS AND MATRICES Chapter 2 Vectors and Matrices In this chapter, we introduce the concept of vectors in Euclidean space. It is at this point where the material becomes more abstract. Keep in mind that all of what is coming can be linked back to solving linear systems. In fact, a lot of the challenge of these problems won’t be in the calculation. It will be in determining what the question is asking and how to interpret the calculation. 2.1 Vector Basics We begin by defining vectors. You have probably encountered vectors and scalars before in one course or another. A vector is merely a way of representing a quantity that has both magnitude and direction. We start with vectors in the Cartesian plane and build up from there. Definition 2.1.1: Displacement Vectors Let A and B be two points in the Cartesian plane R2. The displacement vector from a point A to a point B is an arrow with its tail at A and its head at B. This is denoted by ~v = −−→ AB. It is very important that you write vectors with an arrow over top of them to distinguish them from scalars. Here are a few more bits of terminology. Definition 2.1.2: Vector Terminology Let ~v = −−→ AB be the displacement vector between two points A and B in R2. 1. A is called the initial point of ~v and B is called the terminal point. 2. The length of ~v is the distance between the points A and B and is represented by the length of the arrow. 3. The direction of ~v is the direction the arrow is pointing in. 31 34 CHAPTER 2. VECTORS AND MATRICES x1 x2 ~v ~w Next, starting at the terminal point of ~w, draw a vector parallel to ~v with the same length. x1 x2 ~v ~w The vector attained by joining the initial point of ~v to the far vertex of the parallelogram you just constructed is the sum ~v + ~w. This is called the parallelogram law and is shown below. x1 x2 ~v ~w ~v + ~w Now we know how to add vectors! There are other operations we can perform as well. Scalar multipli- cation is another. 2.1. VECTOR BASICS 35 Definition 2.1.3: Scalars and Scalar Multiplication real number r ∈ R is called a scalar. If ~v is a displacement vector and r is a scalar, the scalar multiple r~v is the displacement vector whose length is equal to |r| times the length of ~v and whose direction is the same as ~v if r > 0 and is opposite of ~v if r < 0. Example 2.1.1 The following are scalar multiples of a vector ~v. x1 x2 ~v 2~v (1/2)~v −~v −4~v Note that −~v is merely ~v pointing in the opposite direction. We call this the negative of ~v. If you think about the arithmetic of numbers, the operation of subtraction is really just adding negative numbers. We define subtraction of vectors in the same way. That is, if ~v and ~w are displacement vectors, the difference of ~v and ~w is ~v − ~w = ~v + (−~w). We can calculate differences using the parallelogram law for addition. An example of this is shown below. 36 CHAPTER 2. VECTORS AND MATRICES x1 x2 ~v ~w −~w ~v − ~w Like most things in mathematics there is a shortcut. Notice that the vector we have drawn as ~v − ~w is exactly parallel to the vector we get if we start at the tip of ~w and join this to the tip of ~v. This is shown on the next page. 2.2. VECTORS IN EUCLIDEAN SPACE 39 called n-dimensional Euclidean space and is denoted by Rn. In set notation, Rn =   v1 v2 ... vn  : v1, . . . , vn ∈ R  . In the above, the ... notation is a placeholder that tells you to repeat the pattern up to the given last value. We use this because, to work in total generality, we can not assign a specific positive integer value to n. Indeed, if you are supposed to prove that a specific property holds for vectors in Rn, then you must work with general vectors written down just like above. Proving the property holds by only considering vectors with a fixed number of components, say 3, only proves that property holds for vectors in R3, and not necessarily in R4,R5, etc. Example 2.2.1 R2 is 2-dimensional real space, which we know as the Cartesian plane. This is the space you work in in Calculus 1 and 2. R3 is three dimensional real space. You’ve worked in R3 if you’ve done Calculus 3. R3 is also what we live in. If n ≥ 4, then we can’t visualize Rn. This is why defining vectors in terms of matrices is so important. In the previous section, we defined a vector as a 2-dimensional arrow with length and direction. We then used our geometric intuition to derive how to add, subtract, and do scalar multiplication. We can do the same thing in R3, though addition and subtraction is a little more complicated to figure out geometrically. But since we can’t visualize Rn for any n ≥ 4, we can’t repeat this same geometric intuition process for higher dimensional spaces. What do length and direction even mean in a space like R4? Defining vectors as matrices allows us to skirt around these problems. We can then define operations on n-dimensional vectors via their matrix representations from a purely algebraic standpoint which is motivated by what we know happens in the 2 and 3 dimensional cases. Definition 2.2.2: Vector Equality, Vector Addition & Scalar Multiplication Let ~u,~v ∈ Rn and write ~u =  u1 u2 ... un  , and ~v =  v1 v2 ... vn  . 1. ~u and ~v are equal, written ~u = ~v, if ui = vi for i = 1, . . . , n. 40 CHAPTER 2. VECTORS AND MATRICES 2. The sum/difference of ~u,~v, written ~u± ~v, is defined as ~u± ~v =  u1 u2 ... un ±  v1 v2 ... vn  =  u1 ± v1 u2 ± v2 ... un ± vn  . 3. For any scalar r ∈ R, and any vector ~w ∈ Rn, scalar multiplication of r and ~w, denoted r ~w, is defined as r ~w = r  w1 w2 ... wn  =  rw1 rw2 ... rwn  . There are a pair of things to note. 1) If you want to add or subtract two vectors, they must have the same dimension. If two vectors have different dimension, then they can not be added or subtracted. Therefore, we can not add/subtract a vector in R2 from a vector in R5. 2) We do not define multiplication of two vectors. We have scalar-vector multiplication, but that is it. This is a little different than what we are used to as we now have two different objects to deal with (scalars and vectors). Note Under the operations of addition and scalar multiplication, the set Rn becomes a mathematical structure called a vector space. We won’t delve into the abstract notion of a vector space too much in this course but, these are studied frequently in lots of areas of mathematics. Example 2.2.2 Let ~u =  2 1 1  and ~v = [ −1 −1 ] . Then ~u± ~v is not defined. 2.2. VECTORS IN EUCLIDEAN SPACE 41 Example 2.2.3 Let ~u =  1 0 2  , ~v =  −2 1 3  , ~w =  −3 8 −1  . Calculate 3~u, 2~v − ~w, 2~u+ 3~v + ~w. Solution. Applying the definitions of vector addition and scalar multiplication yields 3~u =  3 0 6  , 2~v − ~w =  −4 2 6 −  −3 8 −1  =  −4− (−3) 2− 8 6− (−1)  =  −1 −6 7  , 2~u+ 3~v + ~w =  2 0 4 +  −6 3 9 +  −3 8 −1  =  −7 11 12  . ♦ There is a special vector that plays the same role that zero does for numbers. It is aptly called the zero vector. Definition 2.2.3: Zero Vector The zero vector in Rn, denoted ~0, is the vector that contains only zeroes ~0n =  0 0 ... 0  . When the dimension of the zero vector is clear from context, we drop the n and simply write ~0. The following theorem gives some properties of vector addition and scalar multiplication. Many of these are similar to those of real numbers. We need these in order to do algebra on vectors. Theorem 2.2.1: Properties of Vectors Let ~u,~v, ~w ∈ Rn be vectors and let r, s ∈ R be scalars. 1. ~u+ ~v = ~v + ~u (commutativity of addition); 44 CHAPTER 2. VECTORS AND MATRICES Exercise Prove the rest of Properties of Vectors. It should be clear why all of these properties are true for 2-dimensional vectors given the geometric definitions of addition and subtraction we derived in the last section. For example, ~u + ~v = ~v + ~u makes sense because the parallelogram defined by ~u and ~v is the same as the parallelogram defined by ~v and ~u. 2.2.1 Vector Forms of Solutions to Linear Systems Solutions to linear systems can be expressed using vectors. In this section, we explain how to write the solution to a linear system using vectors in a special way called the vector form of the solution. The idea is straightforward. Once we have a solution to a linear system, instead of putting the solution in round brackets as an ordered tuple, put the solution into a vector. If a linear system has only one solution, the difference between the two forms of the solution is negligible. For example, the vector form of the solution to the linear system in Example 1.1.1 is the following,[ x1 x2 ] = [ 152 78 ] . Vector forms of solutions to linear systems with infinitely many solutions are a little bit different. In this case, we split the vector apart using vector addition and scalar multiplication so that each free variable sits inside of its own vector. This process becomes clear after seeing a few examples. Example 2.2.4 The solution to the linear system given in Example 1.4.4 is (x1, x2, x3) = ( − 7 25 − ( 1 25 ) s, 58 25 − ( 31 25 ) s, s ) , s ∈ R. Write down the vector form of the solution. Solution. Writing out the components explicitly, the solution to the linear system is, x1 = − 7 25 − s 25 , x2 = 58 25 − 31 25 s, x3 = s. Put this solution into a vector as follows, x1 x2 x3  =  −(7/25)− (1/25)s 58/25− (31/25)s s  2.2. VECTORS IN EUCLIDEAN SPACE 45 Now pull apart the vector on the left using vector addition and scalar multiplication: −(7/25)− (1/25)s 58/25− (31/25)s s  =  −7/25 58/25 0 + s  −(1/25) −(31/25) 1  , s ∈ R. The expression on the right hand side of the above equation is the vector form of the solution to the linear system. ♦ Example 2.2.5 Express the solutions to the linear systems of Examples 1.4.6 and 1.4.7 in vector form. Solution. The solution to the linear system in Example 1.4.6 was (x1, x2, x3) = (7/2, 1, 5/2). The vector form of the solution is,  x1 x2 x3  =  7/2 1 5/2  . The solution to the linear system in Example 1.4.7 is (x1, x2, x3, x4) = ( 26 11 − ( 15 11 ) s+ ( 1 11 ) t, − 1 11 + ( 1 11 ) s− ( 25 11 ) t, s, t ) , s, t ∈ R. To get the vector form, make x1, x2, x3, and x4 components of a vector and split the vector apart into the fixed part of the solution and one vector for each free variable. Doing so yields the following.  x1 x2 x3 x4  =  26/11− (15/11)s+ (1/11)t −1/11 + (1/11)s− (25/11)t s t  =  26/11 −1/11 0 0 + s  −15/11 1/11 1 0 + t  1/11 −25/11 0 1  , s, t ∈ R. The expression on the far right is the vector form of the solution. ♦ Definition 2.2.4: Fixed Part of Solution The vector that is not multiplied by any free variables in the vector form of a solution to a linear system with infinitely many solutions is called the fixed part of the solution. In general, if the solution to a linear system has m free variables, then there are m + 1 vectors in the vector form of the solution. There is one for each free variable and one for the fixed part. If the vector form only has free variables then there is still m+ 1 vectors in the vector form of the solution, but we usually only write m of them down because the fixed part is the zero vector. If you like, you can write the zero vector in the vector form but it isn’t necessary. 46 CHAPTER 2. VECTORS AND MATRICES Example 2.2.6 Suppose the solution to a linear system is (x1, x2, x3, x4) = (2s+ t, 3s, t, s), s, t ∈ R. The vector form of the solution can be written as x1 x2 x3 x4  =  0 0 0 0 + s  2 3 0 1 + t  1 0 1 0  , s, t ∈ R, or as  x1 x2 x3 x4  = s  2 3 0 1 + t  1 0 1 0  , s, t ∈ R. Either way is acceptable, though most references opt for the latter. Example 2.2.7 Suppose a linear system has has the following solution: (x1, x2, x3, x4, x5, x6, x7, x8) = ( √ 2−14r+(2/3)s−13t, 3−πt, 62−(1/11)r−10t, r, 82−8s, 19r, s, t) r, s, t ∈ R. Write the solution in vector form. Solution. There are three free variables so the vector form of the solution has 4 vectors. The vector form of the solution is x1 x2 x3 x4 x5 x6 x7 x8  =  √ 2 3 62 0 82 0 0 0  + r  −14 0 −1/11 1 0 19 0 0  + s  2/3 0 0 0 −8 0 1 0  + t  −13 −π −10 0 0 0 0 1  , r, s, t ∈ R. ♦ 2.3. LINEAR COMBINATIONS AND VECTOR EQUATIONS 49 Since the vectors are equal, the components of each must be equal. Writing out each equality yields the following linear system. −x1 + 10x2 = −2 2x1 + 7x2 = 1 −x2 = 2 x1 − x2 = 1 ♦ Example 2.3.3 Express the following linear system as a vector equation. 2x1 + 3x2 − x3 = 6 −x1 + x2 = 0 4x1 + x2 = 5 Solve the linear system and show that the solution is also a solution to the corresponding vector equation. Solution. Following the above, the linear system can be expressed as the following vector equation, x1  2 −1 4 + x2  3 1 1 + x3  −1 0 0  =  6 0 5  . (2.1) We use Solving a Linear System Using a Matrix to solve the linear system. From now on, I will skip many of the steps and leave it to the reader to fill in the blanks. The augmented matrix for the linear system is A =  2 3 −1 6 −1 1 0 0 4 1 0 5  . The RREF of A is A ∼  1 0 0 1 0 1 0 1 0 0 1 −1  The solution to the linear system is (x1, x2, x3) = (1, 1,−1). This solution is exactly a solution to the vector equation in Equation (2.1) because, when we make the corresponding substitutions, we get, 1  2 −1 4 + 1  3 1 1 + (−1)  −1 0 0  =  (1)2 + (1)3 + (−1)(−1) (1)(−1) + (1)1 + (−1)(0) (1)4 + (1)1 + (−1)(0)  =  6 0 5  . ♦ Let ~v1, ~v2 . . . , ~vk ∈ Rn. The n × k matrix whose columns are the vectors ~v1, ~v2 . . . , ~vk in that order is denoted as [ ~v1 ~v2 . . . ~vk ] . This is shorthand notation to make things slightly more compact. For example, 50 CHAPTER 2. VECTORS AND MATRICES if ~v1 =  1 2 1  and ~v2 =  2 1 4 , then [ ~v1 ~v2 ] =  1 2 2 1 1 4  . Example 2.3.3 shows the solution to a linear system is also a solution to the vector equation representing the linear system. The converse to this is also true: The solution to a vector equation is a solution to the linear system it represents. The next theorem summarizes this. Theorem 2.3.1 Let ~v1, ~v2, . . . , ~vk,~b ∈ Rn. Consider a vector equation x1~v1 + x2~v2 + . . .+ xk~vk = ~b. (2.2) The solution set to this vector equation is the same as the solution set of the linear system whose augmented matrix is, A = [ ~v1 ~v2 . . . ~vk | ~b ] . In particular, ~b can be written as a linear combination of ~v1, ~v2, . . . , ~vk if and only if the matrix A corresponds to a consistent linear system. Proof. Write ~vi =  v1i ... vni  , i = 1, 2, . . . , k, ~b =  b1 ... bn  , so that, A = [ ~v1 ~v2 . . . ~vk | ~b ] =  v11 v12 . . . v1k b1 v21 v22 . . . v2k b2 ... ... . . . ... ... vn1 vn2 . . . vnk bn  . (c1, c2, . . . , ck) is a solution to the vector equation (2.2) if and only if c1~v1 + c2~v2 + . . .+ ck~vk = ~b. Equating components in this linear combination yields the following series of equations, c1v11 + c2v12 + . . .+ ckv1k = b1 c1v21 + c2v22 + . . .+ ckv2k = b2 ... c1vn1 + c2vn2 + . . .+ ckvnk = bn 2.3. LINEAR COMBINATIONS AND VECTOR EQUATIONS 51 which shows that (c1, c2, . . . , ck) is a solution to the linear system whose augmented matrix is A. Reversing all of these steps shows that any solution to the linear system that has augmented matrix A is also a solution to the vector equation. Therefore, the solution sets are equal. The last statement of the theorem is immediate from the definitions of linear combinations and consistent linear systems. 54 CHAPTER 2. VECTORS AND MATRICES i) By part 1 of Matrix Vector Multiplication Properties, A(~u+ ~v) = A~u+A~v. Moreover, we’re given the values of A~u and A~v: A(~u+ ~v) = A~u+A~v = [ −1 23 ] + [ −5 −4 ] = [ −6 19 ] ♦ ii) By part 2 of Matrix Vector Multiplication Properties, A(5~v) = 5(A~v) = 5 [ −5 −4 ] = [ −25 −20 ] ♦ iii) Combining parts 1 and 2 of Matrix Vector Multiplication Properties: A(5~u− 3~v) = 5A~u− 3A~v = 5 [ −1 23 ] − 3 [ −5 −4 ] = [ −5 115 ] + [ 15 12 ] = [ 10 127 ] ♦ Proof. Write ~u =  u1 u2 ... uk  and ~v =  v1 v2 ... vk . Then, ~u + ~v =  u1 + v1 u2 + v2 ... uk + vk . Let ~a1,~a2, . . . ,~ak denote the columns of A, so A = [ ~a1 ~a2 . . . ~ak ] . Therefore, A(~u+ ~v) = [ ~a1 ~a2 . . . ~ak ]  u1 + v1 u2 + v2 ... uk + vk  = (u1 + v1)~a1 + (u2 + v2)~a2 + . . .+ (uk + vk)~ak by definition of matrix vector multiplication = (u1~a1 + v1~a1) + (u2~a2 + v2~a2) + . . .+ (uk~ak + vk~ak) by part 3 of Properties of Vectors = (u1~a1 + u2~a2 + . . .+ uk~ak) + (v1~a1 + v2~a2 + . . .+ vk~ak) by parts 1 and 4 of Properties of Vectors = A~u+A~v by definition of matrix vector multiplication. Part 2 is left as an exercise. Exercise Prove part 2 of Matrix Vector Multiplication Properties. The linear system in Example 1.1.1 is 2.4. MATRIX EQUATIONS 55 x1 + x2 = 230 350x1 + 600x2 = 100, 000 As a vector equation, x [ 1 350 ] + y [ 1 600 ] = [ 230 100, 000 ] . Let A = [ 1 1 350 600 ] . Define the following vector of one dimensional variables ~x = [ x1 x2 ] and let ~b = [ 230 100, 000 ] . Then, by definition of matrix vector multiplication, we can write the vector equation that represents this linear system as an equation involving matrix vector multiplication. A~x = [ 1 1 350 600 ][ x1 x2 ] = [ 230 100, 000 ] = ~b. This is an example of a matrix equation and it gives us another way of representing linear systems. Definition 2.4.2: Matrix Equations Let A be an n × k matrix and let ~x =  x1 x2 ... xk  be a vector comprised of one-dimensional variables. Let ~b ∈ Rn be a fixed vector. A matrix equation is an equation of the form A~x = ~b. A solution to a matrix equation is a vector ~v =  v1 v2 ... vk  ∈ Rk such that when we make the variable substitution ~x = ~v, the equation A~v = ~b is true. Matrix equations are useful because they provide a very compact way of writing down linear systems. Moreover, every matrix equation can be written as a linear system. Example 2.4.5 Write the following linear system as a matrix equation. x1 + 4x2 − ex3 = 1 (2/3)x1 + 6x3 = √ 2 −x2 + πx3 = 1 56 CHAPTER 2. VECTORS AND MATRICES Solution. First write the linear system as a vector equation. x1  1 2/3 0 + x2  4 0 −1 + x3  −e6 π  =  1√ 2 1  . Now use the definition of matrix vector multiplication to translate this into a matrix equation. 1 4 −e 2/3 0 6 0 −1 π  ︸ ︷︷ ︸ A  x1 x2 x3  ︸ ︷︷ ︸ ~x =  1√ 2 1  ︸ ︷︷ ︸ ~b . ♦ Example 2.4.6 A =  2 1 −6 0 1 2 0 0 1 2 1 1  , ~b =  50 2 50 −1  , ~x =  x1 x2 x3  . Write the matrix equation A~x = ~b as a linear system. Solution. We have A~x = ~b ⇐⇒  2 1 −6 0 1 2 0 0 1 2 1 1   x1 x2 x3  =  50 2 50 −1  . Write this as a vector equation using the matrix vector product. x1  2 0 0 2 + x2  1 1 0 1 + x3  −6 2 1 1  =  50 2 50 −1  . This represents the following linear system. 2x1 + x2 − 6x3 = 50 x2 + 2x3 = 2 x3 = 50 2x1 + x2 + x3 = −1 ♦ Note Given a linear system written as a matrix equation A~x = ~b, the matrix A is always the coefficient matrix for the corresponding 2.5. SPANNING SETS 59 2.5 Spanning Sets In this section, we introduce spanning sets. This is where the material starts to get a little more abstract so I will reiterate: what we do in this section is not anything different than what we were doing before. All we introduce is new language and new ways to state the same questions. The methods to solve these questions remains unchanged. We begin with a pair of examples. Example 2.5.1 Consider the 3-dimensional vectors ~v1 =  −1 4 3  , ~v2 =  4 9 −3  , and ~b =  2 1 3  . Can ~b be expressed as a linear combination of ~v1 and ~v2? If it can, write one down. Solution. By the Equivalence of Solutions, asking if ~b is a linear combination of ~v1 and ~v2 is the same as asking if the linear system whose augmented matrix is [ ~v1 ~v2 | ~b ] has a solution. We know how to solve this! Use Solving a Linear System Using a Matrix! Start by forming the augmented matrix. A =  −1 4 2 4 9 1 3 −3 3  . Put A into echelon form by using Gauss-Jordan Elimination. −1 4 2 0 1 1 0 0 −16  . The rightmost column of this echelon form is a pivot column. Therefore, the corresponding linear system has no solution by The Existence/Uniqueness Theorem. This shows ~b can not be written as a linear combi- nation of ~v1 and ~v2. ♦ Note Example 2.5.1 could be reworded as follows: “Let ~v1 =  −1 4 3  , ~v2 =  4 9 −3  ,~b =  2 1 3  . Let A = [ ~v1 ~v2 ] =  −1 4 4 9 3 −3  . Does A~x = ~b have 60 CHAPTER 2. VECTORS AND MATRICES a solution?” Example 2.5.2 Redo Example 2.5.1 with ~b =  9 14 −9  . Solution. For this example, the augmented matrix we work with is, A =  −1 4 9 4 9 14 3 −3 −9  . The RREF of this matrix is A ∼  1 0 −1 0 1 2 0 0 0  . The rightmost column of the RREF of A is not a pivot column so the corresponding linear system is consistent. Therefore, ~b is a linear combination of ~v1 and ~v2. We now determine how to write ~b as a linear combination of ~v1 and ~v2. This means we must find scalars c1, c2 ∈ R such that c1~v1 + c2~v2 = ~b. If this equation holds, then (c1, c2) is a solution to the vector equation x1~v1 + x2~v2 = ~b, and we know the solution to this vector equation is the same as the solution to the corresponding linear system. Reading from the RREF of A, we see that the solution to this linear system is (x1, x2) = (−1, 2). Therefore, we write ~b as a linear combination of ~v1 and ~v2 as follows. −~v1 + 2~v2 = ~b. Moreover, all of the columns to the left of the bar of the RREF are pivot columns. Therefore, there is only one solution to the corresponding linear system. This means that the above is the only way we can write ~b as a linear combination of ~v1 and ~v2. ♦ Determining which vectors can be written as linear combinations of others and how to do it is one of the main themes of this course. This motivates the following definition. 2.5. SPANNING SETS 61 Definition 2.5.1: Spanning Sets Let ~v1, ~v2, . . . , ~vk ∈ Rn. The set of all linear combinations of the vectors ~v1, ~v2, . . . , ~vk is called the span of ~v1, ~v2, . . . , ~vk and is denoted by span {~v1, ~v2, . . . , ~vk} . In set notation, span {~v1, ~v2, . . . , ~vk} = {c1~v1 + c2~v2 + . . .+ ck~vk : c1, c2, . . . , ck ∈ R} . If ~v1, ~v2, . . . , ~vk ∈ Rn are such that, span {~v1, ~v2, . . . , ~vk} = Rn, then we say the set {~v1, ~v2, . . . , ~vk} spans Rn. This means every vector in Rn is a linear combination of the vectors ~v1, ~v2, . . . , ~vk. Warning! It is very important to remember that the span of a set of vectors is itself a set and not a single vector. This is a common mistake. This probably looks complicated. Remember: these definitions are only fancy language that allows us to ask the same question in a different way, and nothing more. For example, we could have reworded Example 2.5.1 as follows. “Determine if the vector~b =  2 1 3  is in the span of the two vectors ~v1 =  −1 4 3  and ~v2 =  4 9 −3  .” Using more compact notation, we could also reword Example 2.5.1 as the following. “Let ~v1 =  −1 4 3  , ~v2 =  4 9 −3  , and ~b =  2 1 3  . Is ~b ∈ span {~v1, ~v2}?” Perhaps this seems like overkill but we use all of this language throughout the course. Hence, it is im- portant to get familiar with all of these different phrasings and know exactly what they mean. 64 CHAPTER 2. VECTORS AND MATRICES “Let A =  2 4 4 3 0 3 1 2 2 . Do the columns of A span R3?” Solution. Let ~b =  b1 b2 b3  be an arbitrary vector in R3. The solution to A~x = ~b is the same as the solution to the linear system whose augmented matrix is B =  2 4 4 b1 3 0 3 b2 1 2 2 b3  . Use Gauss-Jordan Elimination to put B into echelon form. R1 ⇐⇒ R3 R2 ⇒ R2 − 3R1 R3 ⇒ R3 − 2R1 : B ∼  1 2 2 b3 0 −6 −3 b2 − 3b3 0 0 0 b1 − 2b3  . By The Existence/Uniqueness Theorem, A~x = ~b is consistent if and only if the rightmost column of this matrix is not a pivot column. This happens exactly when b1 − 2b3 = 0, so that b1 = 2b3, then the rightmost column is not a pivot column. However, vectors in R3 do not necessarily satisfy this condition. There are many vectors in R3 whose first component is not twice the third, for example  1 1 1 . For all such vectors, the rightmost column is a pivot column and, consequently, the matrix equation doesn’t have a solution. Therefore, the answer is no, A~x = ~b does not have a solution for all ~b ∈ R3. ♦ Let’s look a little closer at the echelon form matrix in Example 2.5.3: 1 2 2 b3 0 −6 −3 b2 − 3b3 0 0 0 b1 − 2b3  . The only thing preventing A~x = ~b from being solvable for any value of ~b is the bottom row of zeroes to the left of the bar. If the third row contains a pivot to the left of the bar, then we can find a solution to A~x = ~b for any value of ~b ∈ R3. Moreover, the converse also holds. That is, if A~x = ~b has a solution, then A has a pivot in each row. This is summarized in the following important theorem (next page). 2.5. SPANNING SETS 65 Theorem 2.5.1: The Span Theorem Let ~v1, ~v2, . . . , ~vk ∈ Rn and let A = [ ~v1 ~v2 . . . ~vk ] be the n × k matrix whose columns are ~v1, ~v2, . . . , ~vk. The following are equivalent (this means that if one of the statements is true, then they are all true, and if one of the equations is false then they are all false): 1. For every ~b ∈ Rn, the equation A~x = ~b has a solution; 2. The columns of A span Rn. This means every vector ~b ∈ Rn is a linear combination of the columns of A. Equivalently, {~v1, ~v2, . . . , ~vk} spans Rn. 3. Any echelon form of A has a pivot in every row. This is a most excellent result. It gives an easy test to see if a set of vectors in Rn spans Rn. Simply make the vectors the columns of a matrix, row reduce the matrix to echelon form, and count pivots! We do some examples before we give a proof. Example 2.5.4 Determine if any of the following sets of vectors span R4: i) S1 =   −2 1 0 1  ,  −2 −1 1 1  ,  3 1 0 3   ii) S2 =   −2 1 0 1  ,  0 4 0 1  ,  1 3 0 1  ,  −2 1 −1 1   iii) S3 =   −2 1 0 1  ,  1 0 0 1  ,  −2 0 2 1  ,  −3 1 2 3   Solution. All three questions can be answered by forming the matrix whose columns are the vectors in each set, row reducing to echlon form, and counting pivot rows. i) We have A1 =  −2 −2 3 1 −1 1 0 1 0 1 1 3  ∼  −2 −2 3 0 −2 5/2 0 0 5/4 0 0 0  . The bottom row contains no pivots. Therefore, by The Span Theorem, the vectors in S1 do not span 66 CHAPTER 2. VECTORS AND MATRICES R4. ♦ ii) We have A2 =  −2 0 1 −2 1 4 3 1 0 0 0 −1 1 1 1 1  ∼  −2 0 1 −2 0 4 7/2 0 0 0 5/8 0 0 0 0 −1  . There is a pivot in every row. Therefore, by The Span Theorem, the vectors in S2 do span R4. ♦ iii) We have A3 =  −2 1 −2 −3 1 0 0 1 0 0 2 2 1 1 1 3  ∼  −2 1 −2 −3 0 1/2 −1 −1/2 0 0 2 2 0 0 0 0  . The bottom row does not have a pivot. Therefore, by The Span Theorem, the vectors in S3 do not span R4. ♦ Proof. To prove a chain of equivalences like this, first prove that 1 implies 2, then that 2 implies 3, and finally that 3 implies 1. The Equivalence of Solutions already shows 1 implies 2. Therefore, we only need to prove that 2 implies 3 and that 3 implies 1. 2 =⇒ 3: We prove this using contraposition. This means we assume the negation of 3 and show that it implies the negation of 2. To this end, suppose B does not have a pivot in every row. We must show there is a vector in Rn that is not a linear combination of the columns of A. Since B doesn’t have a pivot in every row, the bottom row of B contains only zeroes by definition of echelon form. Augment B with the vector ~u =  0 0 ... 0 1  ∈ Rn where there are n− 1 zeroes above the 1. Denote this matrix by B′ = [ B | ~u ] . Since A ∼ B, there is a sequence of elementary row operations that transform A into B. If we apply the opposite of each of these row operations in the reverse order to B, then we transform B into A. Applying this procedure to B′, we get a matrix of the form A′ = [ A | ~w ] where ~w is some vector in Rn. Since B′ is an echelon form of A′, and B′ contains a row of the form [ 0 . . . 0 | 1 ], The Existence/Uniqueness Theorem implies the linear system with augmented matrix [ A | ~w ] is inconsistent. By Equivalence of Solutions, ~w is not a linear combination of ~a1,~a2, . . . ,~ak. 2.6. LINEAR INDEPENDENCE 69 2.6 Linear Independence In this section, we introduce linear independence. This is a very subtle yet very important concept and, like spanning, it is a concept that is used in many different areas of mathematics in a variety of contexts. 2.6.1 The Homogeneous Equation One way to define linear independence is in terms of solutions to a special matrix equation called the ho- mogeneous equation. Definition 2.6.1: The Homogeneous Equation Let A be an n× k matrix. The homogeneous equation is the matrix equation A~x = ~0. The homogeneous equation is always consistent. Certainly, if A = [ ~a1 ~a2 . . . ~an ] and we substitute ~x = ~0, then A~0 = 0 · ~a1︸ ︷︷ ︸ =~0 + 0 · ~a2︸ ︷︷ ︸ =~0 + . . .+ 0 · ~an︸ ︷︷ ︸ =~0 = ~0 +~0 + . . .+~0 = ~0. This shows ~0 is always a solution to the homogeneous equation, which means that the homogeneous equation is always consistent. Therefore, there are only two possibilities for solution sets of the homogeneous equation: 1. The homogeneous equation has exactly one solution ~x = ~0. This solution is called the trivial solution. 2. The homogeneous equation has infinitely many solutions. A non-zero solution is referred to as a non-trivial solution. Note that in this case, the solution to the matrix equation will contain a free variable. To solve A~x = ~b, we form the augmented matrix [ A | ~b ] and follow Solving a Linear System Using a Matrix. For the homogeneous equation, ~b = ~0, so we form the augmented matrix [ A | ~0 ] . The elementary row operations never change the last column because it is full of zeroes. Therefore, when we solve homoge- neous equations, we don’t usually augment A with ~0 and instead, we apply Solving a Linear System Using a Matrix to the coefficient matrix A itself. If you really want to, you can still augment with the column of zeroes, but going forward I will not. Example 2.6.1 Determine if the following homogeneous linear system has a non-trivial solution. 3x1 + 10x2 + 8x3 = 0 4x1 + 4x3 = 0 x1 + 2x2 + 2x3 = 0 Write the solution to the linear system in vector form. 70 CHAPTER 2. VECTORS AND MATRICES Solution. The coefficient matrix A for the linear system is A =  3 10 8 4 0 4 1 2 2  . The RREF of this matrix is A ∼  1 0 1 0 1 1/2 0 0 0  . Since column 3 is a non-pivot column, x3 is a free variable by The Existence/Uniqueness Theorem. Thus, the linear system has infinitely many solutions and, hence, a non-trivial solution. To get the vector form, we look at the RREF of the matrix. The RREF represents the following linear system. x1 + x3 = 0 x2 + (1/2)x3 = 0. Rearranging, we get x1 = −x3 x2 = −(1/2)x3. Letting x3 = s ∈ R, the vector form of the solution is as follows. x1 x2 x3  =  −s −(1/2)s s  = s  −1 −1/2 1  , s ∈ R. ♦ Example 2.6.2 Let A =  2 0 1 1 −1 2 3 −1 0 10 1 5 1 1 2 1 . Determine if A~x = ~0 has a non-trivial solution. Write the solution to A~x = ~0 in vector form. Solution. The RREF of A is A ∼  1 0 0 0 0 1 0 0 0 0 1 0 0 0 0 1  . Every column in the RREF of A is a pivot column. Therefore, A~x = ~0 has exactly one solution, the trivial solution. Thhis shows A~x = ~0 has no non-trivial solution. The vector form of the solution is 2.6. LINEAR INDEPENDENCE 71  x1 x2 x3 x4  =  0 0 0 0  = ~0. ♦ Let A be n× k and ~b ∈ Rn. Suppose A~x = ~b is consistent. The number of solutions to A~x = ~b is related to the number of solutions to A~x = ~0. In fact, all you need is one particular solution to A~x = ~b and the whole solution to A~x = ~0 in order to get every solution to A~x = ~b. Theorem 2.6.1 Let A be an n× k matrix and let ~b ∈ Rn be a fixed vector. Suppose A~x = ~b is consistent. Let ~vp be a fixed solution to A~x = ~b. Then, every solution of A~x = ~b can be written in the form ~w = ~vp + ~vh where ~vh is a solution to the homogeneous equation A~x = ~0. Proof. Since A~x = ~b is consistent, it has at least one solution. Denote this solution by ~vp. Let ~w be any solution to A~x = ~b (this could be ~vp itself). Define ~vh = ~w − ~vp. Then, A~vh = A(~w − ~vp) = A~w −A~vp = ~b−~b = ~0. Therefore, ~vh is a solution to the homogeneous equation A~x = ~0. Finally, ~w = ~w − ~vp + ~vp = ~vh + ~vp which completes the proof. Side Note A similar theorem holds for solving first order partial differential equations, and the proof in that setting is the same as it is here! Cool! The solutions ~vp and ~vh of Theorem 2.6.1 are called particular and homogeneous solutions respec- tively. This idea of writing all solutions to a system as a sum of a fixed particular solution and a solution the corresponding homogeneous system appears in lots of areas of mathematics. It is a useful technique because, in general, solving a homogeneous system tends to be easier than solving a non-homogeneous one. 2.6.2 Linear Independence of Vectors The main question of interest in regards to the homogeneous equation is whether or not the solution to it is unique or not. This is how we define linear independence. 74 CHAPTER 2. VECTORS AND MATRICES Therefore, the vector form of the solution to A~x = ~0 is  x1 x2 x3  = s  −1 1 1  , s ∈ R. Picking any non-zero value of s yields a non-trivial solution to the homogeneous equation. Taking s = 1 say, we have A  −1 1 1  = ~0 =⇒ −~v1 + ~v2 + ~v3 = ~0 The equation, −~v1 + ~v2 + ~v3 = ~0 is a linear dependence relationship for {~v1, ~v2, ~v3} . We could pick any non-zero value of s we want to get a linear dependence relationship. For example, if s = 2, −2~v1 + 2~v2 + 2~v3 = ~0 is another linear dependence relationship. ♦ Note There exist infinitely many linear dependence relationships for a set of linearly dependent vectors. The Span Theorem provides an easy way of checking if the columns of an n×k matrix A span Rn: count pivot rows in the echelon form of a matrix. There is a similar way to check if the columns of a matrix are linearly independent. Theorem 2.6.2: The Linear Independence Theorem Let ~v1, ~v2, . . . , ~vk ∈ Rn and let A = [ ~v1 ~v2 . . . ~vk ] be the n × k matrix whose columns are ~v1, ~v2, . . . , ~vk. The following are equivalent. 1. The homogeneous equation A~x = ~0 has only the trivial solution; 2. The columns of A are linearly independent; that is, {~v1, ~v2, . . . , ~vk} is a linearly dependent set. 3. Any echelon form of A has a pivot in every column. 2.6. LINEAR INDEPENDENCE 75 Example 2.6.6 Determine if the following set of vectors is linearly independent,~v1 =  −1 2 0 1  , ~v2 =  7 −2 1 0  , ~v3 =  −10 −2 −10 1   Solution. Form the matrix A =  −1 7 −10 2 −2 −2 0 1 −10 1 0 1 . An echelon form of A is, A ∼  −1 7 −10 0 12 −22 0 0 −49/6 0 0 0  . Each column in this matrix contains a pivot. Therefore, the vectors {~v1, ~v2, ~v3} are linearly independent by The Linear Independence Theorem. ♦ Proof. It is clear from the definition of linear independence that 1 implies 2. 2 =⇒ 3: Suppose the columns of A are linearly independent. By definition, the matrix equation A~x = ~0 has only the trivial solution. Therefore, the solution to the homogeneous equation does not contain a free vari- able. Thus, The Existence/Uniqueness Theorem implies every echelon form of A has a pivot in every column. 3 =⇒ 1: Suppose B has a pivot in every column. Form the augmented matrix A′ = [ A | ~0 ] . An echelon form of A′ is B′ = [ B | ~0 ] . Since B has a pivot in every column, every column to the left of the bar of B′ is a pivot column. Therefore, the matrix equation A~x = ~0 has only the trivial solution by The Exis- tence/Uniqueness Theorem. 2.6.3 Special Cases of Linear Independence Corollary 2.5.1 provides a quick way of determining whether a set of vectors in Rn spans Rn or not. There are similar results for linear independence that follow as corollaries to The Linear Independence Theorem. In this section, we explore some of them. Corollary 2.6.1 A single vector {~v} ⊆ Rn is always linearly independent unless ~v = ~0. 76 CHAPTER 2. VECTORS AND MATRICES Proof. Suppose ~v 6= ~0 and consider the vector equation x1~v = ~0 =⇒  x1v1 x1v2 ... x1vn  =  0 0 ... 0  . Since ~v 6= ~0, at least one of v1, v2, . . . , vn is not zero. Thus, the only way all the components of the vector on the left can be zero is if x1 = 0. This shows the vector equation x1~v = ~0 has only the trivial solution so {~v} is linearly independent. On the other hand, if ~v = ~0, it is clear that the vector equation x1~v = ~0 has infinitely many solutions so that { ~0 } is linear dependent. Corollary 2.6.2 A set of two non-zero vectors {~v1, ~v2} ⊆ Rn is linearly dependent if and only if ~v1 is a scalar multiple of ~v2. If one of ~v1 or ~v2 are ~0 then the set is linearly dependent. Example 2.6.7 The set of vectors, {[ −1 2 ] , [ 2 −4 ]} is linearly dependent because the vectors are scalar multiples of one another. Proof. Define the matrix A = [ ~v1 ~v2 ] and let B be an echelon form of A. If one of ~v1 or ~v2 is the zero vector, then it is clear that the corresponding column of B will also be zero, and hence B will not have a pivot in every column. Therefore, if one of the vectors is the zero vector, then The Linear Independence Theorem implies that ~v1 and ~v2 are linearly dependent. Now assume both ~v1 and ~v2 are non-zero and that ~v1 and ~v2 are linearly dependent. Then, the matrix equation A~x = ~0 has a non-trivial solution. Since A is n × 2, the solution is a vector in R2, call it ~w =[ w1 w2 ] 6= [ 0 0 ] . Then, A~w = ~0. Rewriting this as a vector equation, we get w1~v1 + w2~v2 = ~0 so that, upon rearranging, we get w1~v1 = −w2~v2. If w1 = 0, then w2~v2 = ~0. 2.6. LINEAR INDEPENDENCE 79 Conversely, suppose that S is linearly dependent. If ~v1 = ~0, then S is linearly dependent by Corollary 2.6.3. Hence, suppose ~v1 6= ~0. Since S is linearly dependent, there exist scalars d1, d2, . . . , dk ∈ R, not all zero, such that d1~v1 + d2~v2 + . . .+ dk~vk = ~0. Let j be the largest subscript for which dj 6= 0. Then j 6= 1 lest we have di = 0 for all i = 2, . . . , n. This would imply d1~v1 = ~0 =⇒ ~v1 = ~0 which runs contrary to assumption. Therefore, j > 1 and, we have d1~v1 + d2~v2 + . . .+ dj~vj = ~0 =⇒ ~vj = −(d1/dj)~v1 − (d2/dj)~v2 − . . .− (dj−1/dj)~vj−1 which completes the proof. This theorem tells us that if we have a set of vectors {~v1, ~v2, . . . , ~vk} in Rn, and one of the vectors is a linear combination of the others, then the set is necessarily linearly dependent and vice versa. Warning! Theorem 2.6.3 does not guarantee that all of the vectors in the set are linear combinations of the others. It only guarantees that at least one is. The following provides some examples of using everything we’ve seen in this section so far. Example 2.6.9 Determine whether the following sets of vectors are linearly independent or linearly dependent. i) S1 = {[ 1 2 ] , [ 3 1 ]} ⊆ R2 ii) S2 =   2 3 1  ,  1 0 0  ,  0 0 0   ⊆ R3 iii) S3 =   3 2 1 1/2  ,  2 2 2 1  ,  1/2 π 2 7  ,  5 12 324/5 1  ,  0 1 0 0   ⊆ R4 iv) S4 =   2 4 1  ,  1 2 1/2  ,  5 2 1   ⊆ R3 80 CHAPTER 2. VECTORS AND MATRICES Solution. S1 is a set of two vectors and they are not scalar multiples of one another. Thus, by Corollary 2.6.2, S1 is linearly independent. S2 contains the zero vector, so is linearly dependent by Corollary 2.6.3. There are more vectors in S3 then there are components in each vector. Therefore, S3 is linearly dependent by Corollary 2.6.4. The first two vectors in S4 are scalar multiples of one another, hence one is a linear combination of the other two. Therefore, Theorem 2.6.3 implies S4 is linearly dependent. ♦ We end by giving an example of a proof involving linear independence. Example 2.6.10 Let {~v1, ~v2, ~v3} ⊆ Rn be linearly dependent. Show that {~v1, ~v2, ~v3, ~v4} is also linearly dependent for any ~v4 ∈ Rn. Solution. Let A = [ ~v1 ~v2 ~v3 ~v4 ]. Since the set {~v1, ~v2, ~v3} is linearly dependent, by definition, there exist scalars c1, c2, c3 not all zero such that, c1~v1 + c2~v2 + c3~v3 = ~0. Now add 0 · ~v4 to both sides. Since 0 · ~v4 = ~0, this doesn’t change the right hand side so we get, c1~v1 + c2~v2 + c3~v3 + 0 · ~v4 = ~0 =⇒ A  c1 c2 c3 0  = ~0. This shows that ~x =  c1 c2 c3 0  is a non-trivial solution to the homogeneous equation A~x = ~0, which means {~v1, ~v2, ~v3, ~v4} is a linearly dependent set. ♦ 2.7. LINEAR TRANSFORMATIONS 81 2.7 Linear Transformations A function f whose domain and codomain is R is a rule that assigns one number x to another number y. We write f(x) = y. These are the types of functions you study in first year calculus. Functions can be generalized to vectors. In this section, we study such functions. 2.7.1 Transformations We start with an example. Example 2.7.1 Define the matrix A = [ 1 3 5 2 4 6 ] . For any vector ~v =  v1 v2 v3  ∈ R3, the matrix vector product is defined as A~v = v1 [ 1 2 ] + v2 [ 3 4 ] + v3 [ 5 6 ] = [ v1 + 3v2 + 5v3 2v1 + 4v2 + 6v3 ] . Some specific examples: [ 1 3 5 2 4 6 ] 1 0 0  = [ 1 · 1 + 3 · 0 + 5 · 0 2 · 1 + 4 · 0 + 6 · 0 ] = [ 1 2 ] , [ 1 3 5 2 4 6 ] 4 1 2  = [ 4 · 1 + 3 · 1 + 5 · 2 2 · 4 + 4 · 1 + 6 · 2 ] = [ 17 24 ] . This shows that multiplication by A assigns a vector in R2 to a vector in R3, hence it is like a function with domain R3 and codomain R2! The previous example shows that we can think of matrix multiplication as some sort of function that takes vectors as input and spits vectors out. Such functions are called transformations. Definition 2.7.1: Transformations A transformation F from Rk to Rn is a rule that assigns to each vector ~v ∈ Rk a vector in Rn. This vector is denoted F (~v). Rk is called the domain of F , and Rn is called the codomain of F . For notation, we write F : Rk → Rn, which is read as “F is a transformation from Rk to Rn”. A transformation is merely a special type of function. It takes vectors from its domain as input and 84 CHAPTER 2. VECTORS AND MATRICES ii) FA(~u) = [ 1 3 5 2 4 6 ] 3 2 1  = [ 1 · 3 + 3 · 2 + 5 · 1 2 · 3 + 4 · 2 + 6 · 1 ] = [ 14 20 ] . ♦ iii) The questions asks if for a vector ~v ∈ R3 such that FA(~v) = ~b. Since FA(~v) = A~v, this means we need to find a vector ~v ∈ R3 such that A~v = ~b; i.e. we need to find a solution to the matrix equation A~x = ~b. We know how to do this! Augment A with ~b and row reduce! [ A | ~b ] = [ 1 3 5 7 2 4 6 8 ] ∼ [ 1 0 −1 −2 0 1 2 3 ] . The vector form of the solution is evident from the RREF: ~x =  x1 x2 x3  =  −2 3 0 + s  1 −2 1  , s ∈ R. Therefore, any vector ~v of the above form is a solution to A~x = ~b; hence satisfies FA(~v) = ~b. A specific vector is found by picking any fixed value of s. If we take s = 0, then, FA   −2 3 0   = [ 7 8 ] . ♦ iv) Yes because there are infinitely many solutions to the matrix equation A~x = ~b and each such solution yields a vector ~v such that FA(~v) = ~b. ♦ v) If ~y = [ 0 1 ] is in the range of FA, then by definition there is a vector ~w ∈ R3 such that FA(~w) = ~y. Such a vector is necessarily a solution to the matrix equation A~x = ~y. Therefore, we can answer the question by determining whether or not this matrix equation has a solution. We know how to do this! Form the augmented matrix and row reduce! [ A | ~y ] = [ 1 3 5 0 2 4 6 1 ] ∼ [ 1 0 −1 3/2 0 1 2 −1/2 ] . The vector form of the solution to A~x = ~y is, ~x = ~x =  x1 x2 x3  =  3/2 −1/2 0 + s  1 −2 1  , s ∈ R. Since A~x = ~y has a solution, we conclude that yes, ~y is in the range of FA. For example, letting s = 1, a specific vector ~p such that FA(~p) = ~y is ~p =  5/2 −5/2 1 . ♦ 2.7. LINEAR TRANSFORMATIONS 85 Parts iii) and v) of Example 2.7.4 ask whether or not a specific vector is in the range of a matrix trans- formation. The question is answered by finding the solution to a matrix equation. Moreover, we know this is equivalent to asking whether or not a given vector is in the span of some others. Therefore, we can now phrase questions that we’ve seen in the last two sections in terms of transformations. For example, part iii) of Example 2.7.4 can be stated in terms of spanning sets as follows. Restatement “Let ~v1 = [ 1 2 ] , ~v2 = [ 3 4 ] , ~v3 = [ 5 6 ] . Is ~b = [ 7 8 ] in span {~v1, ~v2, ~v3}?” 2.7.2 Linear Transformations There are many types of transformations. Those of specific interest are the linear transformations. Definition 2.7.4: Linear Transformations Let F : Rk → Rn be a transformation. F is called a linear transformation if the following two conditions hold: 1. F (~u+ ~v) = F (~u) + F (~v) for all ~u,~v ∈ Rk; 2. F (r ~w) = r(F (~w)) for all ~w ∈ Rk and all scalars r ∈ R. Note on Language If F is a linear transformation, we usually refer to F only as “linear.” I will do this a lot. There are transformations that are not linear, as the following example shows. Example 2.7.5 Let F : R3 → R3 be a transformation defined by F (~v) = F   v1 v2 v3   =  v1 1 v3  . Show that F is not linear. Solution. To show F is not linear, it suffices to find examples of vectors that violate one of the two properties 86 CHAPTER 2. VECTORS AND MATRICES in the definition of linear transformations. To this end, let ~v1 =  1 0 0  , and ~v2 =  0 0 1  . Then, ~v1 + ~v2 =  1 0 1  , so that F (~v1 + ~v2) =  1 1 1  . On the other hand, F (~v1) =  1 1 0  , and F (~v2) =  0 1 1  , so that F (~v1) + F (~v2) =  1 2 1  . This shows F (~v1 + ~v2) 6= F (~v1) + F (~v2) which means F is not a linear transformation. ♦ The answer to the previous example violates the first condition in the definition of linearity to show F is not a linear transformation. It is perfectly valid to violate the second condition to show a transformation is not linear as well. Example 2.7.6 Let F : R3 → R be a transformation defined by F (~v) = F   v1 v2 v3   = v21 . Show that F is not linear. Solution. This time, we find a counterexample to the second condition in the definition of linear transfor- mations. To this end, let ~v =  1 0 0  . Then, F (2~v) = F   2 0 0   = 22 = 4, 2.7. LINEAR TRANSFORMATIONS 89 Theorem 2.7.3 Let F : Rk → Rn be a linear transformation. Let ~v1, ~v2, . . . , ~vm ∈ Rk and let c1, c2, . . . , cm be scalars. Then, F (c1~v1 + c2~v2 + . . .+ cm~vm) = c1F (~v1) + c2F (~v2) + . . .+ cmF (~vm). Proof. We only prove this for 2 vectors. I leave the proof of the general case to the reader. Let ~u,~v ∈ Rk and r, s ∈ R. Then, r~u, s~v, r~u+ s~v ∈ Rk and, since F is assumed linear, the first condition in the definition implies F (r~u+ s~v) = F (r~u) + F (s~v), and the second condition implies, F (r~u) + F (s~v) = rF (~u) + sF (~v). Combining the equalities yields F (r~u+ s~v) = rF (~u) + sF (~v), which completes the proof. 2.7.3 The Geometry of Linear Transformations In this section, we look at what linear transformations do to vectors geometrically in R2. Example 2.7.9 Let [ 1 0 0 −1 ] and define the linear transformation FA : R2 → R2 by FA(~v) = A~v. Describe how FA acts on vectors in R2 geometrically. Solution. Let ~v = [ v1 v2 ] ∈ R2. Then, FA(~v) = A~v = [ 1 0 0 −1 ][ v1 v2 ] = [ v1 −v2 ] . If we consider the vector [ v1 v2 ] as a point in R2, [ v1 −v2 ] is the reflection of [ v1 v2 ] over the x-axis. This means that FA is reflecting the given vector ~v over the x-axis. Therefore, FA acts on vectors in R2 by reflecting them over the x-axis. The following picture shows examples of FA applied to some vectors in R2. 90 CHAPTER 2. VECTORS AND MATRICES x1 x2 ~v FA(~v) ~w FA(~w) We can also apply FA to a set of vectors. Here is FA applied to the square S with vertices at (±1, 1) and (±1, 3). In this case, the vectors are interpreted as points in R2. x1 x2 S FA(S) Exercise Reflections over the y-axis and reflections over the line x = y are matrix transformations as well. See if you can figure out the matrix that defines. 2.7. LINEAR TRANSFORMATIONS 91 Example 2.7.10 Let A = 1√ 2 [ 1 −1 1 1 ] and define the linear transformation FA : R2 → R2 by FA(~v) = A~v. Describe how FA acts on vectors in R2 geometrically. Solution. This one is a little trickier. To get an idea of what FA is doing, apply FA to ~v = [ 1 0 ] a few times and plot them. FA(~v) = 1√ 2 [ 1 −1 1 1 ][ 1 0 ] = 1√ 2 [ 1 1 ] , FA(FA(~v)) = ( 1√ 2 [ 1 −1 1 1 ])( 1√ 2 [ 1 1 ]) = 1 2 [ 0 2 ] = [ 0 1 ] , FA(FA(FA(~v))) = 1√ 2 [ 1 −1 1 1 ][ 0 1 ] = 1√ 2 [ −1 1 ] Plotting these yields the following. x1 x2 ~v FA(~v) FA(FA~v)) FA(FA(FA(~w))) We see that the linear transformation is rotating ~v counter-clockwise about the origin through an angle of 45 degrees. In fact, any counter-clockwise rotation of a vector about the origin through an angle ϕ, is a matrix transformation. The matrix that defines the rotation is called a rotation matrix, denoted Rϕ, and is given by, Rϕ = [ cos(ϕ) − sin(ϕ) sin(ϕ) cos(ϕ) ] . ♦