0 Replies - 9525 Views - Last Post: 18 May 2013 - 09:11 PMRate Topic:

Linear Algebra Primer: Part 1- Vector Spaces

Posted 18 May 2013 - 09:11 PM

Linear Algebra Primer: Part 1- Vector Spaces

This tutorial is the first in a series designed to introduce Linear Algebra, a field of mathematics that is highly applicable to a number of areas related to programming and computer science. Some of these areas include scientific and numerical computing, graph theory, economics, number theory, dynamical systems, graphics programming, and more.

In this tutorial, the concepts of Vector Spaces and Subspaces will be introduced. I apologize that this tutorial will focus more on the abstracts than on applications, but I will try to keep future tutorials more focused on applications of interest than abstraction.

Vector Spaces
Let's start out with Vector Spaces. A Vector Space is a set of a Vectors. When dealing with other areas of math, the foci are sets of numbers like the integers, the rationals, or the reals. Vector Spaces are analogous to these in Linear Algebra. Vector Spaces are defined over a given Field, which is an algebraic construct. Generally speaking, Vector Spaces are defined over the Real or Complex number systems.

Just as there are operations defined on other number systems like addition and multiplication, Vector Spaces have operations as well. Addition is defined on two vectors, and is closed. That means for any two vectors a and b in the Vector Space, the resultant vector of a + b is also in the Vector Space. Multiplication is defined on a vector in the Vector Space, and a scalar constant from the Field. This operation is also closed, so ka is also in the Vector Space, where k is the scalar and a is in the Vector Space. Looking at it more abstractly, the integers don't form a Field, though this is better addressed in the context of Abstract Algebra.

Now let's look at some of the other properties of Vector Spaces. For the sake of brevity, let's call the Vector Space in question V.

Multiplicative Identity: There is an element x in the Field, such that for any vector a in V, xa = a.

Additive Identity: This property states that every vector a in V has an additive inverse ~a, which is also in V. Thus, a + ~a = 0, where 0 is the 0-vector. By closure under addition, this implies that 0 is contained in all vector spaces.

A lot of these properties should sound familiar from high school algebra, so that familiarity will be useful in Linear Algebra. Now let's examine a couple examples of vector spaces.

Two-dimensional real space (R2) is probably the most familiar. This is the standard (x,y) coordinate system. Let's look at some of the Vector Space properties. It's easy to see that it is closed under addition. Consider two vectors a, b from R2. Let a = (x, y), b = (c, d). Thus, a + b = (x + c, y + d), which falls in the Vector Space of all points in R2. Similarly, all vectors have additive inverses. If a vector is in the first-quadrant, its additive inverse is in the third quadrant. Similarly, vectors in the second quadrant have additive inverses in the fourth quadrant. The other properties are easy to verify, and will be left for you to verify.

The Polynomials of degree n (denoted Pn(F), where F is the Field) also constitute a vector space. Since the coefficients of like terms are added, two polynomials of degree n will add to another polynomial of degree n, thus it is closed under vector addition. Under the distributive law, take a in Pn(F), and scalars x and y: a(x + y) = ax + ay, as the coefficients are just adjusted. For example, 3x(3 + 2) = 9x + 6x = 15x, while 3x(5) = 15x. Similarly, 3(4x + 6x) = 12x + 18x = 3(10x) = 30x.

Hopefully this provides more intuition for vector spaces. To show that a vector set is not a vector space, it is only necessary to find a vector space property that does not hold.

Subspaces
A Subspace is a subset of a Vector Space, and is itself a Vector Space. As such, there are only three properties that need to hold to conclude that a given vector set is a subspace of the Vector Space V. The properties include closure under addition, closure under scalar multiplication, and the 0 vector being included in the subspace.

As is implied by the term "subset," Subspaces often times don't contain all the vectors in the parent vector space. Consider the Vector Space R3, which is three-dimensional real space (the standard (x, y, z) coordinate system). A one-dimensional subspace of R3 might be a line containing vectors k*(1, 1, 1), where k is a real-valued scalar. Similarly, the plane (x, y, 0), for any real-valued x, y coordinates, is a two-dimensional subspace of R3. Note that even though the subspaces have a smaller dimension than the vector space (dimension will be more formally introduced in a later tutorial), the vectors in the subspace still belong to the vector space. In other words, even though the plane (x, y, 0) seems to be the standard (x, y) coordinate system, it is not true that R2 is a subspace of R3.

Conclusion
The next 1-2 tutorials in the sequence will introduce the concepts of Linear (In)dependence, Span, Basis, and Dimension, where the direct relations to computer science topics are more clear. In the meantime, I hope that you found this tutorial helpful. Vector Spaces are abstract, and these concepts require practice to really get a good grasp on them. However, Vector Spaces are the foundation for future tutorials, so it is necessary to start with them.