3
Basic concepts Vector in R n is an ordered set of n real numbers. e.g. v = (1,6,3,4) is in R 4 A column vector: A row vector: m-by-n matrix is an object in R mxn with m rows and n columns, each entry filled with a (typically) real number:

4
Basic concepts Vector norms: A norm of a vector ||x|| is informally a measure of the “length” of the vector. –Common norms: L 1, L 2 (Euclidean) –L infinity

10
+ Span of a vector space If all vectors in a vector space may be expressed as linear combinations of a set of vectors v 1,…,v k, then v 1,…,v k spans the space. The cardinality of this set is the dimension of the vector space. A basis is a maximal set of linearly independent vectors and a minimal set of spanning vectors of a vector space (0,0,1) (0,1,0) (1,0,0) e.g.

11
+ Rank of a Matrix rank(A) (the rank of a m-by-n matrix A) is The maximal number of linearly independent columns =The maximal number of linearly independent rows =The dimension of col(A) =The dimension of row(A) If A is n by m, then rank(A)<= min(m,n) If n=rank(A), then A has full row rank If m=rank(A), then A has full column rank

15
+ Singular Value Decomposition (SVD) Any matrix A can be decomposed as A=UDV T, where D is a diagonal matrix, with d=rank(A) non-zero elements The fist d rows of U are orthogonal basis for col(A) The fist d rows of V are orthogonal basis for row(A) Applications of the SVD Matrix Pseudoinverse Low-rank matrix approximation

16
+ Eigen Value Decomposition Any symmetric matrix A can be decomposed as A=UDU T, where D is diagonal, with d=rank(A) non-zero elements The fist d rows of U are orthogonal basis for col(A)=row(A) Re-interpreting Ab First stretch b along the direction of u 1 by d 1 times Then further stretch it along the direction of u 2 by d 2 times

17
+ Low-rank Matrix Inversion In many applications (e.g. linear regression, Gaussian model) we need to calculate the inverse of covariance matrix X T X (each row of n-by-m matrix X is a data sample) If the number of features is huge (e.g. each sample is an image, #sample n<
{
"@context": "http://schema.org",
"@type": "ImageObject",
"contentUrl": "http://images.slideplayer.com/14/4275515/slides/slide_17.jpg",
"name": "+ Low-rank Matrix Inversion In many applications (e.g.",
"description": "linear regression, Gaussian model) we need to calculate the inverse of covariance matrix X T X (each row of n-by-m matrix X is a data sample) If the number of features is huge (e.g. each sample is an image, #sample n<

21
+ MATrix LABoratory Mostly used for mathematical libraries Very easy to do matrix manipulation in Matlab If this is your first time using Matlab Strongly suggest you go through the “Getting Started” part of Matlab help Many useful basic syntax

22
+ Installing Matlab Matlab licenses are expensive; but “free” for you! Available for installation by contacting help+@cs.cmu.edu  SCS students only help+@cs.cmu.edu Available by download at my.cmu.edumy.cmu.edu Windows XP SP3+ MacOS X 10.5.5+ ~4GB!