Section SS Spanning Sets

In this section we will describe a compact way to indicate the elements of an
infinite set of vectors, making use of linear combinations. This will give us a
convenient way to describe the elements of a set of solutions to a linear
system, or the elements of the null space of a matrix, or many other sets of
vectors.

Subsection SSV: Span of a Set of Vectors

In Example VFSAL we saw the solution set of a homogeneous system
described as all possible linear combinations of two particular vectors. This
happens to be a useful way to construct or describe infinite sets of vectors, so we
encapsulate this idea in a definition.

Definition SSCVSpan of a Set of Column Vectors Given a set of vectors S = \{{u}_{1},\kern 1.95872pt {u}_{2},\kern 1.95872pt {u}_{3},\kern 1.95872pt \mathop{\mathop{…}},\kern 1.95872pt {u}_{p}\},
their span, \left \langle S\right \rangle ,
is the set of all possible linear combinations of
{u}_{1},\kern 1.95872pt {u}_{2},\kern 1.95872pt {u}_{3},\kern 1.95872pt \mathop{\mathop{…}},\kern 1.95872pt {u}_{p}.
Symbolically,

The span is just a set of vectors, though in all but one situation it is an infinite
set. (Just when is it not infinite?) So we start with a finite collection of vectors
S
(t of
them to be precise), and use this finite set to describe an infinite set of vectors,
\left \langle S\right \rangle . Confusing the
finite set S with
the infinite set \left \langle S\right \rangle
is one of the most pervasive problems in understanding introductory linear
algebra. We will see this construction repeatedly, so let’s work through
some examples to get comfortable with it. The most obvious question
about a set is if a particular item of the correct type is in the set, or
not.

Example ABSA basic span Consider the set of 5 vectors, S,
from {ℂ}^{4}

and consider the infinite set of vectors
\left \langle S\right \rangle
formed from all possible linear combinations of the elements of
S.
Here are four vectors we definitely know are elements of
\left \langle S\right \rangle , since
we will construct them in accordance with Definition SSCV,

The purpose of a set is to collect objects with some common property, and to
exclude objects without that property. So the most fundamental question about a
set is if a given object is an element of the set or not. Let’s learn more about
\left \langle S\right \rangle
by investigating which vectors are elements of the set, and which are
not.

At this point, we see that the system is consistent
(Theorem RCLS), so we know there is a solution for the five scalars
{α}_{1},\kern 1.95872pt {α}_{2},\kern 1.95872pt {α}_{3},\kern 1.95872pt {α}_{4},\kern 1.95872pt {α}_{5}. This is enough evidence
for us to say that u ∈\left \langle S\right \rangle .
If we wished further evidence, we could compute an actual solution, say

At this point, we see that the system is inconsistent by
Theorem RCLS, so we know there is not a solution for the five scalars
{α}_{1},\kern 1.95872pt {α}_{2},\kern 1.95872pt {α}_{3},\kern 1.95872pt {α}_{4},\kern 1.95872pt {α}_{5}. This is enough evidence
for us to say that v∉\left \langle S\right \rangle .
End of story. ⊠

and consider the infinite set \left \langle S\right \rangle .
The vectors of S
could have been chosen to be anything, but for reasons that will become clear
later, we have chosen the three columns of the coefficient matrix in Archetype A.
First, as an example, note that

is in \left \langle S\right \rangle , since it is a
linear combination of {u}_{1},\kern 1.95872pt {u}_{2},\kern 1.95872pt {u}_{3}. We
write this succinctly as v ∈\left \langle S\right \rangle .
There is nothing magical about the scalars
{α}_{1} = 5,\kern 1.95872pt {α}_{2} = −3,\kern 1.95872pt {α}_{3} = 7, they could
have been chosen to be anything. So repeat this part of the example yourself, using different
values of {α}_{1},\kern 1.95872pt {α}_{2},\kern 1.95872pt {α}_{3}.
What happens if you choose all three scalars to be zero?

So we know how to quickly construct sample elements of the set
\left \langle S\right \rangle . A slightly
different question arises when you are handed a vector of the correct size and asked if it is
an element of \left \langle S\right \rangle .
For example, is w = \left [\array{
1
\cr
8
\cr
5 } \right ]
in \left \langle S\right \rangle ? More
succinctly, w ∈\left \langle S\right \rangle ?

To answer this question, we will look for scalars
{α}_{1},\kern 1.95872pt {α}_{2},\kern 1.95872pt {α}_{3} so
that

This system has infinitely many solutions (there’s a free variable in
{x}_{3}), but
all we need is one solution vector. The solution,

\eqalignno{
{α}_{1} & = 2 &{α}_{2} & = 3 &{α}_{3} & = 1 & & & & & &
}

tells us that

(2){u}_{1} + (3){u}_{2} + (1){u}_{3} = w

so we are convinced that w
really is in \left \langle S\right \rangle .
Notice that there are an infinite number of ways to answer this question
affirmatively. We could choose a different solution, this time choosing the free
variable to be zero,

\eqalignno{
{α}_{1} & = 3 &{α}_{2} & = 2 &{α}_{3} & = 0 & & & & & &
}

shows us that

(3){u}_{1} + (2){u}_{2} + (0){u}_{3} = w

Verifying the arithmetic in this second solution maybe makes it seem obvious that
w is in this
span? And of course, we now realize that there are an infinite number of ways to realize
w as element
of \left \langle S\right \rangle .
Let’s ask the same type of question again, but this time with
y = \left [\array{
2
\cr
4
\cr
3 } \right ], i.e. is
y ∈\left \langle S\right \rangle ?

So we’ll look for scalars {α}_{1},\kern 1.95872pt {α}_{2},\kern 1.95872pt {α}_{3}
so that

This system is inconsistent (there’s a leading 1 in the
last column, Theorem RCLS), so there are no scalars
{α}_{1},\kern 1.95872pt {α}_{2},\kern 1.95872pt {α}_{3} that will create a
linear combination of {u}_{1},\kern 1.95872pt {u}_{2},\kern 1.95872pt {u}_{3}
that equals y.
More precisely, y∉\left \langle S\right \rangle .

There are three things to observe in this example. (1) It is easy to construct vectors in
\left \langle S\right \rangle . (2) It is possible that
some vectors are in \left \langle S\right \rangle
(e.g. w), while others are
not (e.g. y). (3) Deciding
if a given vector is in \left \langle S\right \rangle
leads to solving a linear system of equations and asking if the system is
consistent.

With a computer program in hand to solve systems of linear equations, could
you create a program to decide if a vector was, or wasn’t, in the span of a given
set of vectors? Is this art or science?

This example was built on vectors from the columns of the
coefficient matrix of Archetype A. Study the determination that
v ∈\left \langle S\right \rangle and
see if you can connect it with some of the other properties of Archetype A.
⊠

is in \left \langle R\right \rangle , since it is a
linear combination of {v}_{1},\kern 1.95872pt {v}_{2},\kern 1.95872pt {v}_{3}.
In other words, x ∈\left \langle R\right \rangle . Try
some different values of {α}_{1},\kern 1.95872pt {α}_{2},\kern 1.95872pt {α}_{3}
yourself, and see what vectors you can create as elements of
\left \langle R\right \rangle .

so we are convinced that x
really is in \left \langle R\right \rangle .
Notice that in this case we again have only one way to answer the question
affirmatively since the solution is again unique.

We could continue to test other vectors for membership in
\left \langle R\right \rangle ,
but there is no point. A question about membership in
\left \langle R\right \rangle
inevitably leads to a system of three equations in the three variables
{α}_{1},\kern 1.95872pt {α}_{2},\kern 1.95872pt {α}_{3}
with a coefficient matrix whose columns are the vectors
{v}_{1},\kern 1.95872pt {v}_{2},\kern 1.95872pt {v}_{3}.
This particular coefficient matrix is nonsingular, so by Theorem NMUS,
it is guaranteed to have a solution. (This solution is unique, but that’s
not critical here.) So no matter which vector we might have chosen for
z,
we would have been certain to discover that it was an element of
\left \langle R\right \rangle . Stated differently, every
vector of size 3 is in \left \langle R\right \rangle ,
or \left \langle R\right \rangle = {ℂ}^{3}.

Compare this example with Example SCAA, and see if you can connect
z
with some aspects of the write-up for Archetype B.
⊠

Subsection SSNS: Spanning Sets of Null Spaces

We saw in Example VFSAL that when a system of equations is homogeneous the
solution set can be expressed in the form described by Theorem VFSLS where the
vector c is
the zero vector. We can essentially ignore this vector, so that the remainder
of the typical expression for a solution looks like an arbitrary linear
combination, where the scalars are the free variables and the vectors are
{u}_{1},\kern 1.95872pt {u}_{2},\kern 1.95872pt {u}_{3},\kern 1.95872pt \mathop{\mathop{…}},\kern 1.95872pt {u}_{n−r}.
Which sounds a lot like a span. This is the substance of the next theorem.

Theorem SSNSSpanning Sets for Null Spaces Suppose that A
is an m × n
matrix, and B
is a row-equivalent matrix in reduced row-echelon form with
r nonzero rows. Let
D = \{{d}_{1},\kern 1.95872pt {d}_{2},\kern 1.95872pt {d}_{3},\kern 1.95872pt \mathop{\mathop{…}},\kern 1.95872pt {d}_{r}\} be the column indices where
B has leading 1’s (pivot columns)
and F = \{{f}_{1},\kern 1.95872pt {f}_{2},\kern 1.95872pt {f}_{3},\kern 1.95872pt \mathop{\mathop{…}},\kern 1.95872pt {f}_{n−r}\} be the set of column
indices where B does not have
leading 1’s. Construct the n − r
vectors {z}_{j},
1 ≤ j ≤ n − r of
size n
as

Proof Consider the homogeneous system with
A as a coefficient matrix,
ℒS\kern -1.95872pt \left (A,\kern 1.95872pt 0\right ). Its set of solutions,
S, is by Definition NSM,
the null space of A,
N\kern -1.95872pt \left (A\right ). Let
{B}^{′} denote
the result of row-reducing the augmented matrix of this homogeneous system. Since
the system is homogeneous, the final column of the augmented matrix will be all zeros,
and after any number of row operations (Definition RO), the column will still be all
zeros. So {B}^{′}
has a final column that is totally zeros.

Now apply Theorem VFSLS to {B}^{′},
after noting that our homogeneous system must be consistent (Theorem HSC). The
vector c
has zeros for each entry that corresponds to an index in
F. For entries that
correspond to an index in D,
the value is −{\left [{B}^{′}\right ]}_{
k,n+1}, but
for {B}^{′} any entry in the
final column (index n + 1)
is zero. So c = 0.
The vectors {z}_{j},
1 ≤ j ≤ n − r are identical
to the vectors {u}_{j},
1 ≤ j ≤ n − r
described in Theorem VFSLS. Putting it all together and applying Definition SSCV
in the final step,

Example SSNSSpanning set of a null space Find a set of vectors, S, so that
the null space of the matrix A
below is the span of S,
that is, \left \langle S\right \rangle = N\kern -1.95872pt \left (A\right ).

The null space of A
is the set of all solutions to the homogeneous system
ℒS\kern -1.95872pt \left (A,\kern 1.95872pt 0\right ). If we find
the vector form of the solutions to this homogenous system (Theorem VFSLS) then the
vectors {u}_{j},
1 ≤ j ≤ n − r
in the linear combination are exactly the vectors
{z}_{j},
1 ≤ j ≤ n − r
described in Theorem SSNS. So we can mimic Example VFSAL to arrive at these
vectors (rather than being a slave to the formulas in the statement of the
theorem).

With D = \left \{1,\kern 1.95872pt 2,\kern 1.95872pt 4\right \} and
F = \left \{3,\kern 1.95872pt 5\right \} we recognize
that {x}_{3} and
{x}_{5} are free
variables and we can express each nonzero row as an expression for the dependent
variables {x}_{1},
{x}_{2},
{x}_{4} (respectively) in
the free variables {x}_{3}
and {x}_{5}.
With this we can write the vector form of a solution vector as

We will mechanically follow the prescription of Theorem SSNS. Here we go, in
two big steps.

First, the indices of the non-pivot columns have indices
F = \left \{3,\kern 1.95872pt 5,\kern 1.95872pt 6\right \}, so we will
construct the n − r = 6 − 3 = 3
vectors with a pattern of zeros and ones corresponding to the indices in
F. This
is the realization of the first two lines of the three-case definition of the vectors
{z}_{j},
1 ≤ j ≤ n − r.

Each of these vectors arises due to the presence of a column that is not a
pivot column. The remaining entries of each vector are the entries
of the corresponding non-pivot column, negated, and distributed
into the empty slots in order (these slots have indices in the set
D and correspond
to pivot columns). This is the realization of the third line of the three-case definition of
the vectors {z}_{j},
1 ≤ j ≤ n − r.

We know that the null space of A
is the solution set of the homogeneous system
ℒS\kern -1.95872pt \left (A,\kern 1.95872pt 0\right ), but nowhere
in this application of Theorem SSNS have we found occasion to reference the variables
or equations of this system. These details are all buried in the proof of Theorem SSNS.
⊠

More advanced computational devices will compute the null space of
a matrix.See: Computation NS.MMA . Here’s an example that will
simultaneously exercise the span construction and Theorem SSNS, while also
pointing the way to the next section.

and consider the infinite set W = \left \langle T\right \rangle .
The vectors of T
have been chosen as the four columns of the coefficient matrix in Archetype D.
Check that the vector

{
z}_{2} = \left [\array{
2
\cr
3
\cr
0
\cr
1 } \right ]

is a solution to the homogeneous system
ℒS\kern -1.95872pt \left (D,\kern 1.95872pt 0\right ) (it is the
vector {z}_{2}
provided by the description of the null space of the coefficient matrix
D from
Theorem SSNS). Applying Theorem SLSLC, we can write the linear
combination,

2{w}_{1} + 3{w}_{2} + 0{w}_{3} + 1{w}_{4} = 0

which we can solve for {w}_{4},

{w}_{4} = (−2){w}_{1} + (−3){w}_{2}.

This equation says that whenever we encounter the vector
{w}_{4},
we can replace it with a specific linear combination of the vectors
{w}_{1} and
{w}_{2}. So using
{w}_{4} in the set
T, along
with {w}_{1}
and {w}_{2},
is excessive. An example of what we mean here can be illustrated by the
computation,

So what began as a linear combination of the vectors
{w}_{1},\kern 1.95872pt {w}_{2},\kern 1.95872pt {w}_{3},\kern 1.95872pt {w}_{4}
has been reduced to a linear combination of the vectors
{w}_{1},\kern 1.95872pt {w}_{2},\kern 1.95872pt {w}_{3}. A
careful proof using our definition of set equality (Definition SE) would
now allow us to conclude that this reduction is possible for any vector in
W, so

So the span of our set of vectors, W,
has not changed, but we have described it by the span of a set of three
vectors, rather than four. Furthermore, we can achieve yet another, similar,
reduction.

Check that the vector

{
z}_{1} = \left [\array{
−3
\cr
−1
\cr
1
\cr
0 } \right ]

is a solution to the homogeneous system
ℒS\kern -1.95872pt \left (D,\kern 1.95872pt 0\right ) (it is the
vector {z}_{1}
provided by the description of the null space of the coefficient matrix
D from
Theorem SSNS). Applying Theorem SLSLC, we can write the linear
combination,

(−3){w}_{1} + (−1){w}_{2} + 1{w}_{3} = 0

which we can solve for {w}_{3},

{w}_{3} = 3{w}_{1} + 1{w}_{2}.

This equation says that whenever we encounter the vector
{w}_{3},
we can replace it with a specific linear combination of the vectors
{w}_{1} and
{w}_{2}. So, as before, the
vector {w}_{3} is not needed
in the description of W,
provided we have {w}_{1}
and {w}_{2}
available. In particular, a careful proof (such as is done in Example RSC5) would
show that

So W
began life as the span of a set of four vectors, and we have
now shown (utilizing solutions to a homogeneous system) that
W can
also be described as the span of a set of just two vectors. Convince yourself that
we cannot go any further. In other words, it is not possible to dismiss either
{w}_{1} or
{w}_{2} in a
similar fashion and winnow the set down to just one vector.

What was it about the original set of four vectors that allowed us to
declare certain vectors as surplus? And just which vectors were we able to
dismiss? And why did we have to stop once we had two vectors remaining?
The answers to these questions motivate “linear independence,” our next
section and next definition, and so are worth considering carefully now.
⊠

It is possible to have your computational device crank out the vector form of
the solution set to a linear system of equations.See: Computation VFSS.MMA
.

C23Archetype K and Archetype L are defined as matrices. Use Theorem SSNS directly
to find a set S
so that \left \langle S\right \rangle
is the null space of the matrix. Do not make any reference to the associated
homogeneous system of equations in your solution. Contributed by Robert BeezerSolution [358]

M20 In Example SCAD we began with the four columns of the coefficient
matrix of Archetype D, and used these columns in a span construction. Then we
methodically argued that we could remove the last column, then the third
column, and create the same set by just doing a span construction with the first
two columns. We claimed we could not go any further, and had removed as many
vectors as possible. Provide a convincing argument for why a third vector cannot
be removed. Contributed by Robert Beezer

M21 In the spirit of Example SCAD, begin with the four columns of the coefficient
matrix of Archetype C, and use these columns in a span construction to build the set
S. Argue
that S
can be expressed as the span of just three of the columns of the coefficient matrix
(saying exactly which three) and in the spirit of Exercise SS.M20 argue that no one
of these three vectors can be removed and still have a span construction create
S.
Contributed by Robert BeezerSolution [373]

T22 Suppose that S is
a set of vectors from {ℂ}^{m},
α ∈ {ℂ}^{}, and
x ∈\left \langle S\right \rangle . Prove
that αx ∈\left \langle S\right \rangle .
Contributed by Robert Beezer

Subsection SOL: Solutions

C22 Contributed by Robert BeezerStatement [352]The vector form of the solutions obtained in this manner will involve precisely the
vectors described in Theorem SSNS as providing the null space of the coefficient
matrix of the system as a span. These vectors occur in each archetype in a
description of the null space. Studying Example VFSAL may be of some
help.

With a leading 1 in the last column of this matrix (Theorem RCLS) we can
see that the system of equations has no solution, so there are no values for
{α}_{1} and
{α}_{2} that will allow us
to conclude that y
is in W.
So y∉W.

We want to know if there are values for the scalars that make the
vector equation true since that is the definition of membership in
\left \langle R\right \rangle . By
Theorem SLSLC any such values will also be solutions to the linear system
represented by the augmented matrix,

We want to know if there are values for the scalars that make the
vector equation true since that is the definition of membership in
\left \langle R\right \rangle . By
Theorem SLSLC any such values will also be solutions to the linear system
represented by the augmented matrix,

With a leading 1 in the last column, the system is inconsistent (Theorem RCLS), so there
are no scalars {a}_{1},\kern 1.95872pt {a}_{2},\kern 1.95872pt {a}_{3}
that will create a linear combination of the vectors in
R that
equal z.
So z∉R.

We want to know if there are values for the scalars that make the
vector equation true since that is the definition of membership in
\left \langle S\right \rangle . By
Theorem SLSLC any such values will also be solutions to the linear system
represented by the augmented matrix,

From this we see that the system of equations is consistent (Theorem RCLS), and
has a infinitely many solutions. Any solution will provide a linear combination of the
vectors in R
that equals y.
So y ∈ S,
for example,

We want to know if there are values for the scalars that make the
vector equation true since that is the definition of membership in
\left \langle S\right \rangle . By
Theorem SLSLC any such values will also be solutions to the linear system
represented by the augmented matrix,

With a leading 1 in the last column, the system is inconsistent (Theorem RCLS), so there
are no scalars {a}_{1},\kern 1.95872pt {a}_{2},\kern 1.95872pt {a}_{3},\kern 1.95872pt {a}_{4}
that will create a linear combination of the vectors in
S that
equal w.
So w∉\left \langle S\right \rangle .

{x}_{3}
and {x}_{4}
would be the free variables in the homogeneous system
ℒS\kern -1.95872pt \left (A,\kern 1.95872pt 0\right ) and Theorem SSNS
provides the set S = \left \{{z}_{1},\kern 1.95872pt {z}_{2}\right \}
where

(b) Simply employ the components of the vector
z as the variables in the
homogeneous system ℒS\kern -1.95872pt \left (A,\kern 1.95872pt 0\right ).
The three equations of this system evaluate as follows,

A solution is {α}_{1} = 1
and {α}_{2} = 2.
(Notice too that this solution is unique!)

C60 Contributed by Robert BeezerStatement [355]Theorem SSNS says that if we find the vector form of the solutions to the homogeneous
system ℒS\kern -1.95872pt \left (A,\kern 1.95872pt 0\right ),
then the fixed vectors (one per free variable) will have the desired property. Row-reduce
A,
viewing it as the augmented matrix of a homogeneous system with an invisible
columns of zeros as the last column,

M21 Contributed by Robert BeezerStatement [356]If the columns of the coefficient matrix from Archetype C are named
{u}_{1},\kern 1.95872pt {u}_{2},\kern 1.95872pt {u}_{3},\kern 1.95872pt {u}_{4} then
we can discover the equation

(−2){u}_{1} + (−3){u}_{2} + {u}_{3} + {u}_{4} = 0

by building a homogeneous system of equations and viewing a solution to the
system as scalars in a linear combination via Theorem SLSLC. This particular
vector equation can be rearranged to read

{u}_{4} = (2){u}_{1} + (3){u}_{2} + (−1){u}_{3}

This can be interpreted to mean that
{u}_{4} is unnecessary
in \left \langle \left \{{u}_{1},\kern 1.95872pt {u}_{2},\kern 1.95872pt {u}_{3},\kern 1.95872pt {u}_{4}\right \}\right \rangle , so
that

If we try to repeat this process and find a linear combination of
{u}_{1},\kern 1.95872pt {u}_{2},\kern 1.95872pt {u}_{3} that
equals the zero vector, we will fail. The required homogeneous system of
equations (via Theorem SLSLC) has only a trivial solution, which will not
provide the kind of equation we need to remove one of the three remaining
vectors.

This is an expression for y
as a linear combination of {v}_{1}
and {v}_{2}, earning
y membership
in X. Since
X is a subset of
Y , and vice versa,
we see that X = Y ,
as desired.

T20 Contributed by Robert BeezerStatement [356]No matter what the elements of the set
S are,
we can choose the scalars in a linear combination to all be zero. Suppose that
S = \left \{{v}_{1},\kern 1.95872pt {v}_{2},\kern 1.95872pt {v}_{3},\kern 1.95872pt \mathop{\mathop{…}},\kern 1.95872pt {v}_{p}\right \}.
Then compute