Vector Spaces II: Finite Dimensional Linear Algebra 1

Transcription

1 John Nachbar September 2, 2014 Vector Spaces II: Finite Dimensional Linear Algebra 1 1 Definitions and Basic Theorems. For basic properties and notation for R N, see the notes Vector Spaces I. Definition 1. X R N, X, is a vector space iff the following conditions hold. 1. For any x, ˆx X, x + ˆx X. (X is closed under vector addition.) 2. For any x X and any α R, αx X. (X is closed under scalar multiplication.) Since 0 R, if X is a vector space then, by condition (2) of the definition, X contains 0x = (0,..., 0). That is, any vector space must contain the origin. Example 1. R N is itself a vector space. Example 2. The set of points in R 2 given by the graph of y = ax + b, a, b 0 does not contain the origin and is not a vector space. On the other hand, the graph of y = ax is a vector space. Definition 2. Let X R N be a vector space and suppose B X is also a vector space. Then B is a vector subspace of X. Example 3. If X R N is a vector space then it is a vector subspace of R N. Example 4. R 1 is a vector subspace of R 2. But the set [ 1, 1] is not a vector subspace because it is not closed under either vector addition or scalar multiplication (for example, = 2 [ 1, 1]). Geometrically, a vector space in R N looks like a line, plane, or higher dimensional analog thereof, through the origin. A key feature of a vector space X R N is that X can be characterized by listing only a few of its vectors. The characterization is not unique, except in the trivial case X = {0}. These characterizing vectors are said to span the space. Definition 3. Let S = {s 1,..., s r } be a set of r vectors in R N. The span of S is { } span(s) = x R N : λ R r such that x = λ i s i. 1 cbna. This work is licensed under the Creative Commons Attribution-NonCommercial- ShareAlike 4.0 License. i=1 1

3 If S is not dependent then it is independent. Note that if S is independent and s i S then s i 0. Theorem 3. Let S = {s 1,..., s r } be a subset of R N. S is independent iff there is no s i S such that s i span(s \ {s i }). Proof. The argument is by contraposition. Possibly relabeling the elements of S, suppose that s 1 span(s\{s 1 }). Then there is a ρ R r 1 such that s 1 = r i=2 ρ is i. Hence 0 = s 1 ρ i s i, which immediately implies that S is not independent. Again, the argument is by contraposition. Suppose S is dependent. Then there is a λ R r such that 0 = r i=1 λ is i but λ 0. Possibly relabeling elements, suppose λ 1 0. Then s 1 = That is, s 1 span({s 2,..., s r }). i=2 i=2 ( λ ) i s i. λ 1 By Theorem 2, an immediate corollary of Theorem 3 is the following. Theorem 4. Let S = {s 1,..., s r } be a subset of R N. S is independent iff there is no s i S such that span(s) = span(s \ {s i }). By Theorem 4, a set S is independent iff it contains no vector that is redundant in the sense that it could be deleted from S without altering the span. This implies that the search for a minimal spanning set should focus on independent sets. In particular, I ask, for a given vector space X, for what independent sets S is span(s) = X? Such sets are called bases for X. Definition 5. Let X be a vector space. S X is a basis for X iff S spans X and S is independent. Remark 1. A basis is also sometimes called a Hamel basis. Suppose S = (s 1,..., s r ) is a basis for X. Then, since S spans X, for any x X there is an λ R r such that x = λ i s i. i=1 λ i is the ith coordinate. For any basis, the coordinate representation is unique (for that basis). If also x = r i=1 ρ is i, so that 0 = x x = r i=1 (λ i ρ i )s i, then the independence of S implies λ i ρ i = 0, or λ i = ρ i for every i. 3

4 When X = R N the standard basis consists of the unit vectors e n = (0,..., 0, 1, 0,..., 0), where the 1 appears in the Nth place. This is indeed a basis for R N. e n spans R N, since, for any x R N, x = N n=1 x ne n. (This is exactly what one means when one writes x = (x 1,..., x N ).) Likewise, the e n are independent, since 0 = N n=1 λ ne n immediately implies (by the definition of e n ), 0 = λ n for all N. In the standard basis, the Nth coordinate is, of course, just x n. It is important to remember that the standard basis for R N is not the only basis for R N. Indeed, except for the trivial case X = {0}, every vector space has an infinite number of bases. Thus, for example, {(2, 1), (1, 2)} forms a basis for R 2. Theorem 8 below provides a tool for checking this assertion. Consider the point (11, 1) (written in the standard basis). In the basis {(2, 1), (1, 2)}, the coordinates get rewritten as (7, 3). This exercise may seem artificial, but finding coordinates in a new, non-standard, basis is effectively what one does when one solves systems of simultaneous linear equations. Definition 6. Let X be a vector space. dim(x), the dimension of X, is r iff there is an independent set of r vectors in X, but no independent set of r + 1 vectors in X. If dim(x) = r then one expects that any basis for X contains r vectors. I confirm this by first noting that if S spans X and S has t vectors then dim(x) cannot be more than t. (It could be strictly less if S were dependent, in which case some of the t vectors would be redundant.) Let S is the number of elements (vectors) in S. Theorem 5. Let X be a vector space. If S spans X then dim(x) S. Proof. I argue by contraposition. Consider any set of vectors S = (s 1,..., s t ). If dim(x) > S then there is an independent set Z = (z 1,..., z r ) X with Z = r > t = S. I show that S does not span X, and that in particular some z i is not in the span of S. Consider first z 1. If z 1 is not in the span of S, I am done. If z 1 is in the span of S then there is a λ 1 R t such that z 1 = t λ 1 i s i. i=1 Since z 1 0 (since Z is assumed independent), λ 1 i 0 for some i. Possibly relabeling the elements of S, suppose that in fact λ Then s 1 = 1 λ 1 1 z 1 + t i=2 4 ( ) λ1 i λ 1 s i. 1

5 It follows that s 1 span(t 1 ) where T 1 = {z 1, s 2,..., s t }). Therefore span(t 1 ) = span(s). Explicitly, since s 1 span(t 1 ), while s 2,... s t T 1, it follows that span(s) span(t 1 ). Conversely, since z 1 span(s), span(t 1 ) span(s). Next consider z 2. Again, if z 2 span(s) = span(t 1 ), I am done. Otherwise, if z 2 span(t 1 ) then there is a λ 2 R t such that z 2 = λ 2 1z 1 + t λ 2 i s i. Since z 2 0 and since Z is assumed independent (hence, in particular, z 2 λ 2 1 z1 ), λ 2 i 0 for some i > 1. Possibly relabeling the elements of S, suppose that in fact λ As above, I conclude that T 2 = {z 1, z 2, s 3,..., s t } has the same span as T 1 and hence the same span as S. Proceeding in this way for i = 1, 2,..., t, I either uncover a z i not in the span of S or I find that the set T t = {z 1,..., z t } Z has the same span as S. Since r = Z > t, there is a z t+1 Z. Since Z is independent, z t+1 is not in the span of T t (by Theorem 3), hence z t+1 is not in the span of S. Thus S does not span X. i=2 Theorem 6. Let X be a vector space. If S is a basis for X then dim(x) = S. Proof: Suppose S is a basis for X. Since S is independent, dim(x) S. On the other hand, by Theorem 5, since S spans X, dim(x) S. Hence dim(x) = S. The following is then an immediate corollary. Theorem 7. dim(r N ) = N. Theorem 8. Let X be a vector space. Suppose that dim(x) = r. 1. If S X and S = r then span(s) = X iff S is independent. 2. X has a basis, and every basis has r vectors. 3. If S X is independent then there is a Ŝ X such that S Ŝ and Ŝ is a basis for X. Proof. 1. I argue by contraposition. Suppose that S is dependent. Then, by Theorem 4, I can remove an element of S without changing the span. Since dim(x) = r, the contrapositive of Theorem 5 implies that S does not span X. Suppose that S is independent. Consider any x X. Since dim(x) = r, {s 1,..., s r, x} is dependent. Then there is an λ R r+1, λ 0, such that 0 = r i=1 λ is i + λ r+1 x. Since S is independent, λ r+1 0. Therefore x = r i=1 ( λ i/λ r+1 )s i, hence x span(s). 5

6 2. Since dim(x) = r there is an independent set S X with S = r. By (1), S spans X, hence S is a basis for X. That every basis has r vectors follows from Theorem If S = r simply set Ŝ = S. Otherwise, S = t < r. Since S is independent, I know from (1) that S does not span X. Therefore choose any x 1 X \ span(s). Form the set T 1 = {s 1,..., s t, x 1 }. To see that this set is independent, consider any point (λ 1,..., λ t, ρ) R t+1 such that 0 = t λ i s i + ρx 1. i=1 If ρ 0 then x 1 span(s). Since I have assumed x 1 span(s), ρ = 0. Since S is independent, this implies λ i = 0. Hence, T 1 is independent. Continuing in this way, I construct T r t, which is independent, has r vectors, and hence, by (1), is a basis for X. 2 Linear Maps. Definition 7. Let X and Y be vector spaces. L : X Y is a linear map iff both of the following conditions hold. 1. For any x, ˆx X, L(x + ˆx) = L(x) + L(ˆx). 2. For any x X, α R, L(αx) = αl(x). Remark 2. Map is just another word for function. The word map is used most frequently when the target space is something other than R. Choosing α = 0 in part (2) of the definition implies that, for any linear map, L(0) = 0. For example, suppose X = Y = R. Then L(x) = ax is a linear map for any a R. However, B(x) = ax + b, b 0, is not a linear map. Rather, it is called an affine map. Definition 8. Let X and Y be vector spaces. The kernel or null space of a linear map L : X Y, denoted K(L), is the the zero set of L. That is, K(L) = L 1 (0) = {x X : L(x) = 0}. The kernel plays a central role in much of the analysis to follow. Theorem 9. Let X and Y be vector spaces. Let L : X Y be linear. Then L(X) is a vector subspace of Y and K(L) is a vector subspace of X. 6

8 an M N matrix Define A = [ a 1... a N] = Ax = a a 1N..... a M1... a MN N n=1 a 1nx n. N n=1 a Mnx n.. Hence L(x) = Ax. I say that L is represented by the matrix A. Any linear map has a matrix representation, and conversely any matrix represents a linear map. Theorem 12. Let L : R N R M be linear. Then L is continuous Proof. Since L is linear, there is a matrix A such that, for any x R N, L(x) = Ax. The claim then follows from the notes on Continuity. Theorem 12 does not generalize to arbitrary metric vector spaces. Linear functions are not, for example, continuous in R with a pointwise convergence metric. Definition 9. Let L : R N R M be a linear map. The transpose of L is L : R M R N, given by L (x) = A x, where the columns of A are the rows of A: A = If A is M n then A is N m. a a M a 1N... a MN. 4 Fundamental Theorem of Linear Algebra Dimension counting arguments play a central role in applications of linear algebra. The canonical example, discussed in Section 5.4, is the analysis of systems of simultaneous linear equations. The central fact used in this application and many others is the following result, sometimes called the Fundamental Theorem of Linear Algebra. Theorem 13 (Fundamental Theorem of Linear Algebra). Let X and Y be vector spaces. If L : X Y is linear then dim(k(l)) + dim(l(x)) = dim(x). 8

10 If P and Q are orthogonal complements, I refer to P and Q as a decomposition of X. Similarly, if x = p+q with p P and q Q, I refer to p and q as a decomposition of x. Example 6. For X = R 2, the horizontal and vertical axes are orthogonal complements. So are the spaces P spanned by (1, 1) and Q spanned by (1, 1). The main result of this section is the following theorem. Theorem 15. Let X and Y be vector spaces and let L : X Y be linear. 1. L (Y ) and K(L) are orthogonal complements. Likewise, L(X) and K(L ) are orthogonal complements. 2. dim(l (Y )) = dim(l(x)). Moreover, L maps the set L (Y ) 1-1 onto the set L(X). 3. For any y L(X), there is an x y L (Y ) such that L 1 (y) = K(L) + {x y }. 5.2 An application to matrices. Let A be any M N matrix and let L : R N R M be the linear map represented by A. L(R N ) is the vector subspace of R M spanned by the columns of A. Accordingly, it is referred to as the column space of A. Similarly, L (R M ) is the space spanned by the rows of A and is, accordingly, referred to as the row space of A. The column rank of A is the number of linearly independent columns of A, which equals dim(l(r N )). The row rank of A is the number of linearly independent rows of A, which equals dim(l (R M )). It follows from (2) of Theorem 15 that for any matrix A, the row rank equals the column rank, which I henceforth refer to simply as the rank of A, written rank(a). By way of illustration, let X = Y = R 2. Suppose [ ] 0 2 A = 0 1 and let L be the corresponding linear map. Note that [ ] A 0 0 =. 2 1 The column space, L(R 2 ), is one dimensional (hence rank(a) = 1) and is spanned by (2, 1). The row space, L (R 2 ), is likewise one dimensional and is spanned by (0, 2). One can verify that the kernel K(L) is one dimensional and is spanned by (1, 0). Finally, one can verify that K(L ) is one dimensional and is spanned by ( 1, 2). The various spaces are illustrated in Figure 1. If the linear map L : R N R N is invertible then let A 1 be the matrix representation of L 1. Linear algebra textbooks provide effective procedures for computing 10

11 L (R 2 ) K(L ) L(R 2 ) 0 K(L) 0 Figure 1: An illustration of Theorem 15. A 1 explicitly. For the moment, I wish only to note that a general function L is invertible if and only if it is 1-1 and onto. By Theorem 14, if L is a linear function then it suffices to check that L is onto, which means dim(l(r N )) = N. Thus A is invertible if and only if the rank of A is N, a condition for invertibility familiar from elementary linear algebra. 5.3 The interpretation of Theorem 15. Theorem 15 implies that the behavior of a linear map L : X Y is characterized by, first, the decomposition of X into the orthogonal complements K(L) and L (Y ) and, second, by the behavior of L on L (Y ). More specifically, for any x k K(L), consider L (Y ) + {x k }, which is a copy of L (Y ) shifted parallel to L (Y ) by the vector x k. See Figure 2 for an illustration using the example of Section 5.2. For any x L (Y ), L(x + x k ) = L(x). Thus, X can be chopped up into parallel copies of L (Y ), and the behavior of L on each copy is effectively the same as it is on L (Y ). In particular, L maps each such copy 1-1 onto L(X) (by (2) of Theorem 15). One can also view X as being chopped up into the preimage sets of L, that is, sets of the form L 1 (y) for each y L(X). Of course, L 1 (0) = K(L). Part (3) of Theorem 15 states that every preimage set of L is simply a copy of K(L), translated by some vector in L (Y ). This is illustrated in Figure 3. Translates of vector spaces are called linear manifolds. Thus L 1 (y) is a linear manifold for each y L(X). One can unambiguously define the dimension of L 1 (y) to be the same as the dimension of K(L), namely dim(x) dim(l(x)). A linear manifold in R N of dimension N 1 is called a plane. For N 4, one 11

13 often sees the word hyperplane rather than plane, but word plane is still correct. In applications, planes often arise as the preimage of a linear map. Specifically, suppose that L : R N R is linear and onto. Then for any y R, L 1 (y) has dimension equal to dim(k(l)) = dim(r N ) dim(l(r N )) = N 1, and is thus a plane. Note that L is represented by a 1 N matrix of the form A = [a 1... a N ] The transpose of A is A = a 1. a N, which can be viewed as a vector in R N, call it v. Thus A = v and the canonical plane, namely K(L), is the set of vectors x R N that are orthogonal to v. 5.4 Application: simultaneous equations, kernels, and planes. A set of M linear equations in N unknowns can be written in the form Ax = y, where A is M N and the ith row of A gives the coefficients for the ith equation. Note that x R N while y R M. This sort of problem arises frequently in economic applications. Geometrically, Ax = y if y lies in the column space of A, in which case x is the coordinate representation for y in terms of the columns of A. One is interested in knowing whether any solutions x exist, and if so, how many. Is there any solution to Ax = y? The answer is yes if and only if y is in the column space of A. A sufficient condition for y to be in the column space of A is rank(a) = M, a necessary condition for which is that N M (at least as many unknowns as equations). If M > N (more equations than unknowns) then Ax = y is said to be overdetermined. If M > rank(a) then Ax = y typically has no solutions; the notion of typically can be formalized but I do not do so here. How many solutions are there to Ax = y? The set of solutions is L 1 (y). Assume that this set is not empty. Then L 1 (y) is a linear manifold of dimension equal to the dimension of the kernel. By the Fundamental Theorem of Linear Algebra, this dimension is N dim(l(r N )) = N rank(a). The solution is unique iff this dimension is zero if and only if L is 1-1 if and only if rank(a) = N, a necessary condition for which is that M N (at least as many equations as unknowns). If N > M (more unknowns than equations) then Ax = y is said to be underdetermined. In summary, if M > rank(a) then typically there is no solution to Ax = y, but if a solution exists then it is unique. If N > rank(a) then there are a continuum of solutions; indeed L 1 (y) is a linear maniforld of positive dimension. If N = M = rank(a) then there is a unique solution. 13

14 The preceding analysis was in terms of the columns of A. One can also analyze Ax = y in terms of the rows of A. Suppose M = 1 and that A has full rank, namely 1. Then for any y R, L 1 (y) is an N 1 dimensional linear manifold, that is, a plane. If M = 2 and A has rank 2 then for any y R 2, L 1 (y) is the intersection of two planes, one corresponding to the first equation (i.e. the first row of A) and the other corresponding to the second equation. From Theorem 13 and Theorem 15, this intersection must be of dimension N 2. Thus, having two equations rather than one drops the dimension of L 1 (y) by one. By way of example, suppose N = 3, so that each plane is two-dimensional. Then their intersection is one-dimensional (a line). The only thing that might possibly go wrong is if the two planes are parallel, in which case there is no intersection at all unless the planes exactly coincide. But the planes cannot be parallel if the two rows of A are independent, as must be the case if the rank is two. This is intuitive and it is confirmed by Theorem 13 and Theorem 15. And similarly if M = 3. L 1 (y) is now the intersection of 3 planes, and Theorem 13 and Theorem 15 tell us that if the rank of A is three then the dimension of this intersection is N 3. Again, adding another equation drops the dimension of L 1 (y) by 1. And so on for M = 4, 5,... until M = N, at which point Theorem 13 and Theorem 15 imply that if the rank of A is N then L 1 (y) has dimension zero, meaning that the intersection is a singleton. If I try to add one more equation, so that M = N + 1 > N, then it is no longer possible for A to have rank M. In a sense that can be formalized, although I do not do so, the intersection of M > N planes in R N is typically empty. This sort of dimension counting applies more generally. In general, given two linear manifolds in R N of dimensions c and d, if c + d N then a typical intersection (and I won t formalize typical, although a formalization is possible) has dimension c + d N. This follows from the fact that any linear manifold can be described as the preimage of a linear map. In particular, one can easily show that for any linear manifold in R N, if the manifold has dimension c then there is a linear map L that maps from R N onto R N c such that the linear manifold is a preimage of L. (Note that the preimage of such a map has dimension N (N c) = c, as desired.) The manifold can, in turn, be viewed as the intersection of N c planes in R N, generated by the N c rows in the matrix representation of L. The intersection of two manifolds of dimension c and d is thus equivalent to the intersection of (N c) + (N d) rows, which has dimension N (2N c d) = c + d N, provided all rows are independent. For example, the intersection of two planes (each of dimension N 1) has dimension (N 1) + (N 1) N = N 2. If c + d < N then typically the intersection of the manifolds is empty. 5.5 Proofs. I establish Theorem 15 by means of a series of lemmas. The first of these establishes the orthogonal half of part (1) of Theorem

15 Lemma 1. Let X and Y be vector spaces and let L : X Y be linear. Then L (Y ) and K(L) are orthogonal. Similarly, L(X) and K(L ) are orthogonal. Proof. Let A represent L, hence A represents L. Consider any x K(L). Then Ax = 0. This means that x is orthogonal to each row of A, which means that x is orthogonal to each column of A. Since L (Y ) is spanned by the columns of A, Ax = 0 implies that x is orthogonal to every element in L (Y ), as was to be shown. Similarly for L(X) and K(L ). To show that K(L) and L (Y ) are in fact orthogonal complements I need to show that the union of the bases for K(L) and L (Y ) span all of X. As a first step in doing so, I establish the following general fact about orthogonal spaces. Lemma 2. Let P and Q be vector subspaces of a vector space X. If P and Q are orthogonal then the union of any basis for P and any basis for Q is independent. Proof. Let V = {v 1,..., v r } be any basis for P and let W = {w 1,..., w t } be any basis for Q. Suppose that 0 = t λ i v i + ρ j w j. i=1 j=1 Let p = r i=1 λ iv i P and let q = t j=1 ρ jw j Q. Then p + q = 0, hence 0 = (p + q) (p + q) = p p + 2p q + q q = p p + q q (since, by assumption, p and q are orthogonal). Since p p 0, with equality only if p = 0, and similarly for q q, it follows that p = q = 0. But this implies, since V and W are bases, that λ i = 0 for all i and ρ j = 0 for all j, as was to be shown. The following is then immediate. Lemma 3. Let P and Q be orthogonal vector subspaces of a vector space X. Then dim(p ) + dim(q) dim(x). To prove part (1) of Theorem 15, it thus remains to show that dim(k(l)) + dim(l (R M )) = dim(x). As a step toward establishing this, I first establish the first half of part (2) of Theorem 15. Lemma 4. Let X and Y be vector spaces and let L : X Y be linear. dim(l(x)) = dim(l (Y )). Proof. From Theorem 13 Then dim(x) = dim(l(x)) + dim(k(l)) (1) dim(y ) = dim(l (Y )) + dim(k(l )). (2) 15

16 From Theorem 3 dim(x) dim(l (Y )) + dim(k(l)) (3) dim(y ) dim(l(x)) + dim(k(l )). (4) Equation (1) and inequality (3) together imply dim(l(x)) dim(l (Y )). Equation (3) and inequality (4) together imply dim(l (Y )) dim(l(x)). Combining, these imply as was to be shown dim(l(x)) = dim(l (Y )), Proof of part (1) of Theorem 15. Orthogonality was established in Lemma 1. In view of Lemma 2, to show that K(L) and L (Y ) are orthogonal complements, it suffices to show that But by Theorem 13, dim(k(l)) + dim(l (Y )) = dim(x). dim(k(l)) + dim(l(x)) = dim(x) and by Lemma 4, dim(l (Y )) = dim(l(x)), and so the claim follows. The proof that K(L ) and L(X) are orthogonal complements is analogous. Proof of part (2) of Theorem 15. The first half of part (2) was established in Lemma 4. Define ˆL : L (Y ) L(X) by ˆL(x) = L(x) for any x L (Y ). I must show that ˆL is 1-1 and onto. By Theorem 14, it suffices to show that ˆL is 1-1. Consider any x, x L (Y ). Suppose that L(x) = L(x ). Then ˆL(x x ) = 0. Since L (Y ) is a vector space, x x L (Y ). Since 0 = ˆL(x x ) = L(x x ), x x K(L). But L (Y ) and K(L) are orthogonal, hence L (Y ) K(L) = {0}. (The latter follows from Lemma 1. More directly, suppose P and Q are orthogonal and let p P Q. Then p p = 0, which implies p = 0.) Therefore, x x = 0, or x = x. Since x and x were arbitrary, this establishes that ˆL is 1-1 and hence also onto. Proof of part (3) of Theorem 15. Consider any y L(X). Since L maps L (Y ) 1-1 onto L(X), there is a unique element of L (Y ), call it x y, such that L(x y ) = y. Consider any x L 1 (y). I must show that there is an x k K(L) such that x = x y +x k. Since K(L) and L (Y ) are orthogonal complements, there is an x L (Y ) and a x K(L) such that x = x + x. But y = L(x) = L(x + x) = L(x ) + L( x) = L(x ), since L( x) = 0. It follows that x = x y, and hence x k = x. 16

NOTES ON LINEAR TRANSFORMATIONS Definition 1. Let V and W be vector spaces. A function T : V W is a linear transformation from V to W if the following two properties hold. i T v + v = T v + T v for all

Math 1512 Fall 2010 Notes on least squares approximation Given n data points (x 1, y 1 ),..., (x n, y n ), we would like to find the line L, with an equation of the form y = mx + b, which is the best fit

John Nachbar Washington University December 16, 2016 Convexity in R N Main Notes 1 1 Introduction. These notes establish basic versions of the Supporting Hyperplane Theorem (Theorem 5) and the Separating

Chapter 2 Geometry of Linear Programming The intent of this chapter is to provide a geometric interpretation of linear programming problems. To conceive fundamental concepts and validity of different algorithms

MATH 304 Linear Algebra Lecture 18: Rank and nullity of a matrix. Nullspace Let A = (a ij ) be an m n matrix. Definition. The nullspace of the matrix A, denoted N(A), is the set of all n-dimensional column

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

Methods for Finding Bases Bases for the subspaces of a matrix Row-reduction methods can be used to find bases. Let us now look at an example illustrating how to obtain bases for the row space, null space,

WEEK 8 Summary of week 8 (Lectures 22, 23 and 24) This week we completed our discussion of Chapter 5 of [VST] Recall that if V and W are inner product spaces then a linear map T : V W is called an isometry

MATH 37 Linear Transformations from Rn to Rm Dr. Neal, WKU Let T : R n R m be a function which maps vectors from R n to R m. Then T is called a linear transformation if the following two properties are

160 CHAPTER 4. VECTOR SPACES 4. Rank and Nullity In this section, we look at relationships between the row space, column space, null space of a matrix and its transpose. We will derive fundamental results

MATH 304 Linear Algebra Lecture 16: Basis and dimension. Basis Definition. Let V be a vector space. A linearly independent spanning set for V is called a basis. Equivalently, a subset S V is a basis for

Mathematics Course 111: Algebra I Part IV: Vector Spaces D. R. Wilkins Academic Year 1996-7 9 Vector Spaces A vector space over some field K is an algebraic structure consisting of a set V on which are

MATH022 Linear Algebra Brief lecture notes 48 Similarity and Diagonalization Similar Matrices Let A and B be n n matrices. We say that A is similar to B if there is an invertible n n matrix P such that

Sec 4. Vector Spaces and Subspaces Motivation Let S be the set of all solutions to the differential equation y + y =. Let T be the set of all 2 3 matrices with real entries. These two sets share many common

Matrix Representations of Linear Transformations and Changes of Coordinates 01 Subspaces and Bases 011 Definitions A subspace V of R n is a subset of R n that contains the zero element and is closed under

Math 5 Exam # Practice Problem Solutions For each of the following statements, say whether it is true or false If the statement is true, prove it If false, give a counterexample (a) If A is a matrix such

Topic 1: Matrices and Systems of Linear Equations Let us start with a review of some linear algebra concepts we have already learned, such as matrices, determinants, etc Also, we shall review the method

Orthogonal Complements and Projections Recall that two vectors in are perpendicular or orthogonal provided that their dot product vanishes That is, if and only if Example 1 The vectors in are orthogonal

Math 571 Inner Product Spaces 1. Preliminaries An inner product space is a vector space V along with a function, called an inner product which associates each pair of vectors u, v with a scalar u, v, and

Solutions to Math 5 First Exam January 29, 25. ( points) (a) Complete the following sentence: A set of vectors {v,..., v k } is defined to be linearly dependent if (2 points) there exist c,... c k R, not

4 Exterior algebra 4.1 Lines and 2-vectors The time has come now to develop some new linear algebra in order to handle the space of lines in a projective space P (V ). In the projective plane we have seen

NULL SPACE, COLUMN SPACE, ROW SPACE Null Space, Column Space, Row Space In applications of linear algebra, subspaces of R n typically arise in one of two situations: ) as the set of solutions of a linear

LINEAR ALGEBRA MATH 27.6 SPRING 23 (COHEN) LECTURE NOTES Sets and Set Notation. Definition (Naive Definition of a Set). A set is any collection of objects, called the elements of that set. We will most

MATRIX ALGEBRA AND SYSTEMS OF EQUATIONS 1. SYSTEMS OF EQUATIONS AND MATRICES 1.1. Representation of a linear system. The general system of m equations in n unknowns can be written a 11 x 1 + a 12 x 2 +

Solving Simultaneous Linear Equations 3.4 Introduction Equations often arise in which there is more than one unknown quantity. When this is the case there will usually be more than one equation involved.

4.5 Linear Dependence and Linear Independence 267 32. {v 1, v 2 }, where v 1, v 2 are collinear vectors in R 3. 33. Prove that if S and S are subsets of a vector space V such that S is a subset of S, then

Images and Kernels in Linear Algebra By Kristi Hoshibata Mathematics 232 In mathematics, there are many different fields of study, including calculus, geometry, algebra and others. Mathematics has been

Linear Algebra Notes Chapter 19 KERNEL AND IMAGE OF A MATRIX Take an n m matrix a 11 a 12 a 1m a 21 a 22 a 2m a n1 a n2 a nm and think of it as a function A : R m R n The kernel of A is defined as Note

1. LINEAR EQUATIONS A linear equation in n unknowns x 1, x 2,, x n is an equation of the form a 1 x 1 + a 2 x 2 + + a n x n = b, where a 1, a 2,..., a n, b are given real numbers. For example, with x and

MATH 23: SYSTEMS OF LINEAR EQUATIONS Systems of Linear Equations In the plane R 2 the general form of the equation of a line is ax + by = c and that the general equation of a plane in R 3 will be we call

Linear Codes Linear Codes In the V[n,q] setting, an important class of codes are the linear codes, these codes are the ones whose code words form a sub-vector space of V[n,q]. If the subspace of V[n,q]

Weak topologies David Lecomte May 3, 006 1 Preliminaries from general topology In this section, we are given a set X, a collection of topological spaces (Y i ) i I and a collection of maps (f i ) i I such

MAT067 University of California, Davis Winter 2007 Linear Maps Isaiah Lankham, Bruno Nachtergaele, Anne Schilling (February 5, 2007) As we have discussed in the lecture on What is Linear Algebra? one of

Lecture Note on Linear Algebra 15. Dimension and Rank Wei-Shi Zheng, wszheng@ieee.org, 211 November 1, 211 1 What Do You Learn from This Note We still observe the unit vectors we have introduced in Chapter

CHAPTER 2 American mathematician Paul Halmos (1916 2006), who in 1942 published the first modern linear algebra book. The title of Halmos s book was the same as the title of this chapter. Finite-Dimensional

LECTURE 5 Solving Systems of Linear Equations Recall that we introduced the notion of matrices as a way of standardizing the expression of systems of linear equations In today s lecture I shall show how

Orthogonal Projection Given any nonzero vector v, it is possible to decompose an arbitrary vector u into a component that points in the direction of v and one that points in a direction orthogonal to v

Problems for Advanced Linear Algebra Fall 2012 Class will be structured around students presenting complete solutions to the problems in this handout. Please only agree to come to the board when you are

806 Problem Set 4 Solution Due Wednesday, March 2009 at 4 pm in 2-06 Total: 75 points Problem : A is an m n matrix of rank r Suppose there are right-hand-sides b for which A x = b has no solution (a) What

The Calculus of Functions of Several Variables Section. Introduction to R n Calculus is the study of functional relationships and how related quantities change with each other. In your first exposure to

Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F) denotes the collection of all m n matrices over F Matrices will usually be denoted

Section 3 Sequences and Limits Definition A sequence of real numbers is an infinite ordered list a, a 2, a 3, a 4,... where, for each n N, a n is a real number. We call a n the n-th term of the sequence.

FUNCTIONAL ANALYSIS LECTURE NOTES: QUOTIENT SPACES CHRISTOPHER HEIL 1. Cosets and the Quotient Space Any vector space is an abelian group under the operation of vector addition. So, if you are have studied

Chapter Three Functions 3.1 INTRODUCTION In this section, we study what is undoubtedly the most fundamental type of relation used in mathematics. Definition 3.1: Given sets X and Y, a function from X to

Linear Algebra Notes for Marsden and Tromba Vector Calculus n-dimensional Euclidean Space and Matrices Definition of n space As was learned in Math b, a point in Euclidean three space can be thought of

Chapter 1 What is Linear Programming? An optimization problem usually has three essential ingredients: a variable vector x consisting of a set of unknowns to be determined, an objective function of x to

1 Introduction to Matrices In this section, important definitions and results from matrix algebra that are useful in regression analysis are introduced. While all statements below regarding the columns

Math 113 Homework 3 October 16, 2013 This homework is due Thursday October 17th at the start of class. Remember to write clearly, and justify your solutions. Please make sure to put your name on the first

Chapter 1 LINEAR EQUATIONS 1.1 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,..., a n, b are given

4.2. Linear Combinations and Linear Independence If we know that a subspace contains the vectors v 1 = 2 3 and v 2 = 1 1, it must contain other 1 2 vectors as well. For instance, the subspace also contains

Math 525 Chapter 1 Stuff If A and B are sets, then A B = {(x,y) x A, y B} denotes the product set. If S A B, then S is called a relation from A to B or a relation between A and B. If B = A, S A A is called

2 Matrix Algebra 2.3 CHARACTERIZATIONS OF INVERTIBLE MATRICES Theorem 8: Let A be a square matrix. Then the following statements are equivalent. That is, for a given A, the statements are either all true

Math 5. Selected Solutions for Week 2 Section. (Page 2). Let u = and A = 5 2 6. Is u in the plane in R spanned by the columns of A? (See the figure omitted].) Why or why not? First of all, the plane in

1 VECTOR SPACES AND SUBSPACES What is a vector? Many are familiar with the concept of a vector as: Something which has magnitude and direction. an ordered pair or triple. a description for quantities such

Solving simultaneous linear equations 3.4 Introduction Equations often arise in which there is more than one unknown quantity. When this is the case there will usually be more than one equation involved.

. INNER PRODUCT SPACES.. Definition So far we have studied abstract vector spaces. These are a generalisation of the geometric spaces R and R. But these have more structure than just that of a vector space.

Math 20 Chapter 5 Eigenvalues and Eigenvectors Eigenvalues and Eigenvectors. Definition: A scalar λ is called an eigenvalue of the n n matrix A is there is a nontrivial solution x of Ax = λx. Such an x

Vector Spaces 4.4 and Independence October 18 Goals Discuss two important basic concepts: Define linear combination of vectors. Define Span(S) of a set S of vectors. Define linear Independence of a set

Continued Fractions and the Euclidean Algorithm Lecture notes prepared for MATH 326, Spring 997 Department of Mathematics and Statistics University at Albany William F Hammond Table of Contents Introduction