Linear Algebra Dot Product & Matrix Multiplication

Previously we discussed what matrices are and what some operations are that you can do with them. Now we’ll discuss two more operations called the dot product and matrix multiplication. If you’ve taken a physics course, you’ve definitely been introduced to a vector, and perhaps even the dot product of vectors. A matrix dot product is similar to a vector dot product, and a way to keep a clear head about this is to think of a matrix as rows of vectors. The other operation discussed is one that can often be confused with other operations; be sure you fully understand the process of matrix multiplication before moving on.

Hello Professor! although i think i understand the idea behind them for reassurance could you please address the difference between a vector, a matrix and an array?

3 answers

Last reply by: Professor HovasapianSun Sep 9, 2012 8:29 AM

Post by Tomer Eigeson September 8, 2012

When you explained the definition of a dot product you used the standard form of the vector instead of just the magnitude... in other words you said that a vector can be equal to a scalar

1 answer

Last reply by: Josh WinfieldMon Jan 27, 2014 10:27 PM

Post by Real Schiranon March 1, 2012

Very nice explanation !!!!

1 answer

Last reply by: Professor HovasapianSun Jul 15, 2012 9:11 PM

Post by Damion Wrighton February 6, 2012

Hi, can I just check something with you? Toward the end you gave an example matrix AB where you worked out the first number of the second row to be 8, I keep getting 10. Am I making a mistake over and over again? I am incredibly new to mathematics so take all of my own findings with a mountain of salt :P P.s. example was at about 37:05. Thanks in advance!

Dot Product & Matrix Multiplication

Lecture Slides are screen-captured images of important points in the lecture. Students can download and print out these lecture slide images to do practice problems as well as take notes while watching the lecture.

Transcription: Dot Product & Matrix Multiplication

Welcome back to linear algebra, so we have talked about linear systems, we have talked about matrix addition, we have talked about scalar multiplication, things like transpose, diagonal matrices.0000

Now we are going to talk about dot product and matrix multiplication, so matrix multiplication is not numerical multiplication, yes it does involve not just standard multiplying of numbers, but it's handled differently.0011

And one of the things that you are going to notice about this is that matrix multiplication does not commute.0028

In other words, I know that 5 times 6 = 6 times 5, I can do the multiplication in any order and it ends up being 30.0035

However if I take the matrix A and multiply by a matrix B, it's actually not the same as the matrix B multiplied by A.0041

It might be, but there is no guarantee that it will be, and in fact most of the time it won't be, so that's the one thing that's actually different about matrices and then numbers.0049

Whenever we see a normally a lowercase letters a, b, c, d, x we will often use x with an arrow on top, that means it's a vector, and a vector is just a list of numbers.0116

A1, A2 all the way to AN in this particular case we are talking about an N vector, which means it has N entries, so 5 vector would have 5 entries.0131

An example might be, let's say the vector V might be 1, 3, 7, 6, that's all this means, this is the vector in these the components of that vector.0140

It's composed of (1, 3, 7, 6), it's a four vector, because it has four entries in it, that's all this notation means, this is just a generalized version of it.0153

Okay, so let A, the vector A = A1 to An, let the vector B = B1 through BN, now we defines something called the dot product as the following, A.B.0163

The product of two vectors is equal to A1 times B1 + A2 times B2 + ... +AnBn and I am going to write this in σ notation.0180

σ notation, I'll explain in just a minute, if you guys haven't seen it, I am sure you have, but you just, I know that you don't deal with it all too often.0196

Okay, so if the vector A is composed of A1 through AN, B is the list, B1 through BN, the dot product A.B = the product of the corresponding entries added together.0205

When I add these together, I end up with a number, so the dot product of two vectors gives me a scalar; it gives me a number, so I just add them all up.0222

This σ notation is the capital Greek letter S, and stands for sum, and it says take the sum of the Ith entry of A, the Ith entry of B.0233

Multiply them together and add them, so A1B1, 1I = 1, and then go to the next one, I = 2 + A2B2 abd then go to I = 3 + A3B3.0246

This is just a short hand notation for this, we won't deal with σ notation all that much, what end our definitions, whatever we do I'll usually write this explicitly.0260

I just want you to be aware that in your book, you'll probably see this; you'll definitely see it in the future.0270

That's all this means, it's a short hand notation for a very long sum, so don't let the symbolism intimidate you, scare you, confuse you, anything like that, it's very simple.0275

Okay, let's just do an example of a dot product and everything should make sense, so example; we will let...0285

... Vector A = (1, 2, -3 and 4), so this is a four vector, and will let B = (-2, 3, 2, 1), notice I wrote one of them in row form, one of them in column form.0298

This is also a four vector because we have four entries, I wrote it this way because in a minute when we talk about matrix multiplication, it's going to make sense, it will make sense why it is that I wrote it this way, but just for now understand that there is no real difference between these two.0319

I could have written this as a column, I could have written this as a row, it's just a question of corresponding entries.0335

But I did like this because in a minute when we do matrix multiplication, symbolically, its going to help make sense when you move your fingers across a row and down a column, just sort of keep things straight, because matrix multiplication, there is lot, a of lot of arithmetic involved.0340

Okay, so our dot product A.B here, A.B, we just go back to our definition, it says take corresponding entries and just multiply it together, that's you got to do,0356

I take A1 times B1, so which is 1 times -2, which is -2 + 2 times 3, which is 6 + -3 times 2, which is -6...0370

let's take a look at the definition again, A is a matrix, A IJ, it is M by P, B is a matrix, B IJ is P by N.0526

when I multiply those two matrices the, essentially what happens is that the column of the first matrix, the one on the left cancels the column accounts with the row of the matrix on the right an what you end up with is a matrix which is M by N.0536

And that matrix is such that the IJth entry = Ith of A dot end with the Jth column of B, that's why this P and this P have to be the same.0554

In order to multiply two matrices, let's write this one out specifically, okay.0573

... The number of rows of the second and that's what this says M by P, P by N, the number of columns of the first has to equal the number of rows.0611

That's the only way that matrix multiplication is defined and what we mean when we say is defined, means if they are not the same, you can't do the multiplication.0623

That's what defined means, it's the only way you can do it if that's the case, okay.0632

let's see what we have got, so for example if I have a 2 by 3 matrix and I want to multiply it by a 3 by 2 matrix, yes I can do that because the number of columns of the first one is equal to the number of columns of the second one, and essentially they go away.0639

What I am left with is the final matrix which is 2 by 2, this is kind of interesting.0661

Now notice if I reverse them and if I did a 3 by 2 matrix, and if I multiply that by a 3 by 2 matrix, I am sorry 2 by 3...0666

... Now, it is defined, number of columns of the first equals the number of rows of the second, so now I end up with a 3 by 3 matrix, okay.0686

4 times 1, is 4, so we get 15 + 4,., which is 19, 19 - 2 is 16, so this entry is 6.0983

The product so, AB = 4 - 2, 6, 16, 2 by 3 matrix multiplied by a 3 by 2 matrix gives us a 2 by 2 matrix, and we get that by this row this column, this row this column, and then this row this column, this row this column.0996

That's all you are doing, rows and columns, now you know why I arranged it, remember a little bit back when we did dot product, I arranged it, the first one horizontally and the other one vertically.1021

This is the reason why, because when we multiply, we are doing this times that, this times that, this times that, we can move one this way, one this way, it seems sort of, it's a way to keep things separate, as one hand, one finger moves across a row.1031

The other finger should move down a column as used to going this way or this way., okay.1047

Let's let A =(1, 2, -1, 3) and we will let B = (12, 1, 0, 1) in this case, because this is 2 by 2, and because this is 2 by 2, both AB and BA, they are both defined.1203

I can do the multiplication, well let's do the multiplication and see if AB = BA.1224

There are two ways that you can, there are certain demonstrate non-commutivity, is if the dimensions don't match when you switch them or if it's defined, multiplication is defined and doable this way and that way.1229

Then you might end up with different matrices, again proving that it doesn't commute, alright.1246

Let's see what we have got, when we do AB, okay we end with the following, we end up with (2, 3, -2, 2) and when we do BA, we said it is defined.1252

AB and BA are not the same, AB is not equal to BA, matrix multiplication does not commute.1271

Okay, so now let's talk about matrices and linear systems, so we introduced linear systems in our first lesson, we talked about matrices in our second, and we have just introduced matrix multiplication.1283

Now let's combine them together to see if we can take a matrix and represent it as a linear system, or a linear system and represent it in matrix form.1297

And we will let X with the little line, the vector be our vector, let's call it X1, X2, X3, this is the vector formulation, this is the component form, it's just a 3 vector.1332

Okay, so this is a, we can do this in red, this is a 3 by 3 matrix, and this is a 3 by 1, right, so if I multiply this matrix by that vector X, well it's just a 3 by 3 times a 3 by 1.1346

Well those are the same, so I end up with a 3 by 1, it is defined and it's going to equal some vector b, which is going to be a 3 by 1 vector, just something with 3 entries in it.1366

And let's let B therefore equal, we will call it B1, B2, B3, so again we have a matrix.1383

We have this 3 vector, I can multiply them because matrix multiplication is defined, their answer is going to be a 3 vector, so we will call that 3 vector B, and will call it's components B1, B2, B3.1394

Okay, well let's actually do the multiplication here, so A1 X, I am sorry, AX.1407

When i do this multiplication, this row, this column, this row, this column, this row, this column.1415

Here is what I get, A11 times X1 + A12 times X2 + A13 times X 3, that's what I get, that's the multiplication.1423

A11 X1 + A12 X2 + A13 X3, and then I do this second row, that column again, I get A21 X1 + A22 X2 + A2 X3.1435

And then I will do the third row, which is A33 X1 + A32 X2 + A33 X3, that's going to be my matrix.1455

That's my actual matrix multiplication; well I know that equals this B, so I write B1, B2, and B3.1469

Well, this thing = this thing, this thing = equals this thing, this thing = this thing, that's what this says, this is just a 3 by 1 in its most expanded form.1481

That's the A times the X, this thing is the B, that are equals, and so now I am just going to set corresponding entries equal to each other, this whole thing is equal to that.1491

...X1, X2, X3, X1, X2, X3, X1, X2, X3, these A11 's A2, all of these are coefficients and these are the actual solutions.1561

You can actually write a linear system as a matrix, so it looks like A11, A12, A13, this is the coefficient, the matrix of coefficients for the linear system.1581

A21, A22, A23, A31, A32, A33, and then you multiply it by the variables, which are X1, X2, X3, and it equals B1, B2, B3.1602

We can take a linear system and represent it in matrix form; we take the matrix of coefficients, so this is the coefficient matrix.1626

M by N in this case is 3 by 3m, but it can be anything, this is the matrix of variables, it's the variable matrix, and it's always going to be some N vector.1640

And this is just the you might call the solution matrix, it's not really the solution matrix, the solution matrix is once you find X1, X2, X3, those are going to be your solutions, so you know what let's not even give this a name, let's just say this happens to be the, whatever it is.1657

It's the B that makes up linear system on the right side of the equality, okay now given this; we can actually form as it turns out.1678

X is going to be our variables, our variables happen to be X, Y and Z, that's going to be X, Y, and Z, and B, let me go back to red, B vector is going to be (5, 7, 3).1881

That's it, you can represent, now we will do the augmented matrix, which means take the coefficient matrix and add this to it, so we end up with A augment with B, symbolized like that.1903

It is equal to (-2, 0, 1, 5) and I'll go ahead and do a solid line, because I like solid lines.1918

(2, 3, -4, 3, 2, 2), you have your coefficient matrix and you have your matrix that represents the linear system, that was originally given to you like that.1928

Now, let's see, now let's go the other way, let's say we have a matrix, (2, -1, 3, 4, 3, 0, 2, 5) let's say you are given this particular matrix, this particular matrix actually can represent a linear system.1946

We could take a linear system, represent it in matrix form, which we just did, we can take a matrix and represent it as a linear system, if we need to.1977

This ends up being, so let's say that this is the augmented matrix, so that means this is (1, 2, 3), that means we have 3 variables, that's what the column represent are the variables, and these are the equations.1986

Excuse me,, there you go, okay now let's talk about what we did today, recap our lesson.2030

We talked about the dot product of two vectors and a vector is just an N by 1 matrix, either as a column or row, it doesn't really matter.2042

What you do is you multiply the corresponding entries in the two vectors and you add up the total.2051

The dot product gives you a single number, a scalar, it's also called the scalar product, so dot product, scalar product, as you go on in mathematics you will actually refer to it as a scalar product not necessarily an dot product.2056

After that we talked about matrix multiplication where we actually invoke the dot product, so with matrix multiplication you can only multiply two matrices if the number of columns in the first matches the number of rows in the second.2070

Matrix multiplication does not commute, in other words A times B does not equal B times A in general.2084

The IJth entry in the product is the dot product of the Ith row of the first and the Jth column of the next.2095

Okay, now matrix representations of linear systems, any linear systems of equations can be represented as an augmented matrix, you take the matrix of coefficients and you add the column of solutions.2107

Any matrix with more than one column can represent a linear system of equations, that last column is going to be your solutions, that's the augment.2122

Okay, so let's do one more example here, so we will let A = (3, 5, 2, 4, 9, 2, excuse me.2134

And B = (1, 0, 1, 6), oh she knows, (2, 1, 3, 7) so here we have a 3 by 2 matrix and here we have a 2 by 4 matrix.2153

Yes, the 2's on the inside are the same, they end p cancelling and it's going to end up giving up a 3 by 4 matrix, so we are left with 2 outside.2172

We are going to be looking for a matrix which is 3 by 4, this is kind of interesting if you think about it, 3 by 2, 2by 4, now you get a 3 by 4, you get something that's bigger than both in some sense, okay.2182

AB equal to, well we take the first row and first column, 3 times 1 + 5 times 2, 3 times 1 is 3, 5 times 2 is 10, you end up with 13, first row, second column.2195

Well you take the first row, dot product of the second column, 3 times 0 is 0, 5 times 1 is 5, so you end you with 5., then you keep going.2214

Notice 8 times B is defined if I did B times A, well B times is equal to a 2 by 4 times the 3 by 2.2243

This 4 and this 3 aren't equal, BA is not even defined, we can't even do the multiplication, leave alone and find out whether it equals or not, which in general it doesn't, so in this case it's not even defined.2256

It only works when A and B are such that A is on the left of B, B is on the right of A, and they will often say that, we will often say in linear algebra, multiply by this on the left, multiply by this on the right.2267

We don't do that with numbers, we just say multiply the numbers, okay now let's let the variable that's the X, the vector = X, Y and Z and let's let the vector Z = (4, 2, 9).2279

That's all we have done AB, A times B, it is defined, we can find the multiplication.2451

We can take given X and given Z, this is a two vector, this is a three vector, we can take AXC represented as a linear system.2459

We express it this way, we do the matrix multiplication, we said corresponding things equal to each other, and we have actually converted this to a linear system.2469

This and this are equivalent, we can take this linear system and express it completely just as a matrix, an augmented matrix by adding the solutions as the augment on the right, and we end up with that.2477

Okay, thank you for joining us today for linear algebra, and our discussion of dot products and matrix multiplication on linear systems.2493

Thank you for joining us at educator.com, we will see you next time, bye, bye.2499

Related Books

The book introduces concepts early in a familiar, concrete real number setting, develops them gradually, and returns to them again and again throughout the text so that when discussed in the abstract, these concepts are more accessible.

This book emphasizes linear transformations as a unifying theme. It combines a user-friendly presentation with straightforward, lucid language to clarify and organize the techniques and applications of linear algebra. Exercises and examples make up the heart of the text, with abstract exposition kept to a minimum. Exercise sets are broad and varied and reflect the author’s creativity and passion for this course. This revision reflects careful review and appropriate edits throughout, while preserving the order of topics of the previous edition.