Linear algebra

Linear algebra is an important branch of mathematics. The numpy.linalg package contains linear algebra functions. With this module, you can invert matrices, calculate eigenvalues, solve linear equations, and determine determinants, among other things.

Time for action – inverting matrices

The inverse of a matrix A in linear algebra is the matrix A-1, which when multiplied with the original matrix, is equal to the identity matrix I. This can be written, as A* A-1 = I.

The inv function in the numpy.linalg package can do this for us. Let's invert an example matrix. To invert matrices, perform the following steps:

We will create the example matrix with the mat.

A = np.mat("0 1 2;1 0 3;4 -3 8")
print "A\n", A

The A matrix is printed as follows:

A
[[ 0 1 2]
[ 1 0 3]
[ 4 -3 8]]

Now, we can see the inv function in action, using which we will invert the matrix.

inverse = np.linalg.inv(A)
print "inverse of A\n", inverse

The inverse matrix is shown as follows:

inverse of A
[[-4.5 7. -1.5]
[-2. 4. -1. ]
[ 1.5 -2. 0.5]]

If the matrix is singular or not square, a LinAlgError exception is raised. If you want, you can check the result manually. This is left as an exercise for the reader.

Let's check what we get when we multiply the original matrix with the result of the inv function:

print "Check\n", A * inverse

The result is the identity matrix, as expected.

Check[[ 1. 0. 0.][ 0. 1. 0.][ 0. 0. 1.]]

What just happened?

We calculated the inverse of a matrix with the inv function of the numpy.linalg package. We checked, with matrix multiplication, whether this is indeed the inverse matrix.

Solving linear systems

A matrix transforms a vector into another vector in a linear way. This transformation mathematically corresponds to a system of linear equations. The numpy.linalg function, solve, solves systems of linear equations of the form Ax = b; here A is a matrix, b can be 1D or 2D array, and x is an unknown variable. We will see the dot function in action. This function returns the dot product of two floating-point arrays.

Time for action – solving a linear system

Let's solve an example of a linear system. To solve a linear system, perform the following steps:

Finding eigenvalues and eigenvectors

Eigenvalues are scalar solutions to the equation Ax = ax, where A is a two-dimensional matrix and x is a one-dimensional vector. Eigenvectors are vectors corresponding to eigenvalues. The eigvals function in the numpy.linalg package calculates eigenvalues. The eig function returns a tuple containing eigenvalues and eigenvectors.

Time for action – determining eigenvalues and eigenvectors

Let's calculate the eigenvalues of a matrix. Perform the following steps to do so:

Create a matrix as follows:

A = np.mat("3 -2;1 0")print "A\n", A

The matrix we created looks like the following:

A[[ 3 -2][ 1 0]]

Calculate eigenvalues by calling the eig function.

print "Eigenvalues", np.linalg.eigvals(A)

The eigenvalues of the matrix are as follows:

Eigenvalues [ 2. 1.]

Determine eigenvalues and eigenvectors with the eig function. This function returns a tuple, where the first element contains eigenvalues and the second element contains corresponding Eigenvectors, arranged column-wise.

Singular value decomposition

Singular value decomposition is a type of factorization that decomposes a matrix into a product of three matrices. The singular value decomposition is a generalization of the previously discussed eigenvalue decomposition. The svd function in the numpy.linalg package can perform this decomposition. This function returns three matrices – U, Sigma, and V – such that U and V are orthogonal and Sigma contains the singular values of the input matrix.

The asterisk denotes the Hermitian conjugate or the conjugate transpose.

Time for action – decomposing a matrix

It's time to decompose a matrix with the singular value decomposition. In order to decompose a matrix, perform the following steps:

We do not actually have the middle matrix—we only have the diagonal values. The other values are all 0. We can form the middle matrix with the diag function. Multiply the three matrices. This is shown, as follows:

print "Product\n", U * np.diag(Sigma) * V

The product of the three matrices looks like the following:

Product[[ 4. 11. 14.][ 8. 7. -2.]]

What just happened?

We decomposed a matrix and checked the result by matrix multiplication. We used the svd function from the NumPy linalg module.

Pseudoinverse

The Moore-Penrose pseudoinverse of a matrix can be computed with the pinv function of the numpy.linalg module (visit http://en.wikipedia.org/wiki/Moore%E2%80%93Penrose_pseudoinverse). The pseudoinverse is calculated using the singular value decomposition. The inv function only accepts square matrices; the pinv function does not have this restriction.

Time for action – computing the pseudo inverse of a matrix

Let's compute the pseudo inverse of a matrix. Perform the following steps to do so:

First, create a matrix as follows:

A = np.mat("4 11 14;8 7 -2")print "A\n", A

The matrix we created looks like the following:

A[[ 4 11 14][ 8 7 -2]]

Calculate the pseudoinverse matrix with the pinv function, as follows:

Determinants

The determinant is a value associated with a square matrix. It is used throughout mathematics; for more details please visit http://en.wikipedia.org/wiki/Determinant. For an n x n real value matrix the determinant corresponds to the scaling an n-dimensional volume undergoes when transformed by the matrix. The positive sign of the determinant means the volume preserves its orientation ("clockwise" or "anticlockwise"), while a negative sign means reversed orientation. The numpy.linalg module has a det function that returns the determinant of a matrix.

Time for action – calculating the determinant of a matrix

To calculate the determinant of a matrix, perform the following steps:

Create the matrix as follows:

A = np.mat("3 4;5 6")print "A\n", A

The matrix we created is shown as follows:

A[[ 3. 4.][ 5. 6.]]

Compute the determinant with the det function.

print "Determinant", np.linalg.det(A)

The determinant is shown as follows:

Determinant -2.0

What just happened?

We calculated the determinant of a matrix with the det function from the numpy.linalg module.

Fast Fourier transform

The fast Fourier transform (FFT) is an efficient algorithm to calculate the discrete Fourier transform (DFT). FFT improves on more naïve algorithms and is of order O(NlogN). DFT has applications in signal processing, image processing, solving partial differential equations, and more. NumPy has a module called fft that offers fast Fourier transform functionality. A lot of the functions in this module are paired; this means that, for many functions, there is a function that does the inverse operation. For instance, the fft and ifft functions form such a pair.

Time for action – calculating the Fourier transform

First, we will create a signal to transform. In order to calculate the Fourier transform, perform the following steps:

Create a cosine wave with 30 points, as follows:

x = np.linspace(0, 2 * np.pi, 30)wave = np.cos(x)

Transform the cosine wave with the fft function.

transformed = np.fft.fft(wave)

Apply the inverse transform with the ifft function. It should approximately return the original signal.

print np.all(np.abs(np.fft.ifft(transformed) - wave) < 10 ** -9)

The result is shown as follows:

True

Plot the transformed signal with Matplotlib.

plot(transformed)show()

The resulting screenshot shows the fast Fourier transform:

What just happened?

We applied the fft function to a cosine wave. After applying the ifft function we got our signal back.

Random numbers

Random numbers are used in Monte Carlo methods, stochastic calculus, and more. Real random numbers are hard to generate, so in practice we use pseudo random numbers. Pseudo random numbers are random enough for most intents and purposes, except for some very special cases. The functions related to random numbers can be found in the NumPy random module. The core random number generator is based on the Mersenne Twister algorithm. Random numbers can be generated from discrete or continuous distributions. The distribution functions have an optional size parameter, which tells NumPy how many numbers to generate. You can specify either an integer or a tuple as size. This will result in an array filled with random numbers of appropriate shape. Discrete distributions include the geometric, hypergeometric, and binomial distributions.

Time for action – gambling with the binomial

The binomial distribution models the number of successes in an integer number of independent trials of an experiment, where the probability of success in each experiment is a fixed number.

Imagine a 17th-century gambling house where you can bet on flipping of pieces of eight. Nine coins are flipped. If less than five are heads, then you lose one piece of eight, otherwise you win one. Let's simulate this, starting with 1000 coins in our possession. We will use the binomial function from the random module for that purpose.

In order to understand the binomial function, go through the following steps:

Initialize an array, which represents the cash balance, to zeros. Call the binomial function with a size of 10000. This represents 10,000 coin flips in our casino.

Hypergeometric distribution

The hypergeometric distribution models a jar with two types of objects in it. The model tells us how many objects of one type we can get if we take a specified number of items out of the jar without replacing them. The NumPy random module has a hypergeometric function that simulates this situation.

Time for action – simulating a game show

Imagine a game show where every time the contestants answer a question correctly, they get to pull three balls from a jar and then put them back. Now there is a catch, there is one ball in there that is bad. Every time it is pulled out, the contestants lose six points. If however, they manage to get out three of the 25 normal balls, they get one point. So, what is going to happen if we have 100 questions in total? In order to get a solution for this, go through the following steps:

Initialize the outcome of the game with the hypergeometric function. The first parameter of this function is the number of ways to make a good selection, the second parameter is the number of ways to make a bad selection, and the third parameter is the number of items sampled.

The fit of the histogram and theoretical pdf is excellent, as you can see in the following screenshot:

What just happened?

We visualized the lognormal distribution using the lognormal function from the random NumPy module. We did this by drawing the curve of the theoretical probability density function and a histogram of randomly generated values

Alerts & Offers

Series & Level

We understand your time is important. Uniquely amongst the major publishers, we seek to develop and publish the broadest range of learning and information products on each technology. Every Packt product delivers a specific learning pathway, broadly defined by the Series type. This structured approach enables you to select the pathway which best suits your knowledge level, learning style and task objectives.

Learning

As a new user, these step-by-step tutorial guides will give you all the practical skills necessary to become competent and efficient.

Beginner's Guide

Friendly, informal tutorials that provide a practical introduction using examples, activities, and challenges.

Essentials

Fast paced, concentrated introductions showing the quickest way to put the tool to work in the real world.

Cookbook

A collection of practical self-contained recipes that all users of the technology will find useful for building more powerful and reliable systems.

Blueprints

Guides you through the most common types of project you'll encounter, giving you end-to-end guidance on how to build your specific solution quickly and reliably.

Mastering

Take your skills to the next level with advanced tutorials that will give you confidence to master the tool's most powerful features.

Starting

Accessible to readers adopting the topic, these titles get you into the tool or technology so that you can become an effective user.

Progressing

Building on core skills you already have, these titles share solutions and expertise so you become a highly productive power user.