Pages

Monday, March 16, 2009

Complex Step Jacobian-Free Newton-Krylov

Newton's method are really useful for solving non-linear systems. Each iteration of the Newton's method requires solving a linear equation:where the Jacobian is the matrix of partial derivativesand then applying the update to the solution vector:For small systems it is tractable to analytically calculate the Jacobian and invert it directly. This rapidly becomes intractable for realistic engineering problems because the number of unknowns can be on the order of 1e6 (if you have a good workstation) to 1e9 (if you have a big cluster). Rather than explicitly creating a 1e6-squared matrix and inverting it at every iteration, an iterative method that only requires the action of the matrix on a vector can be used (such as conjugate gradient or GMRES). Then we can approximate this matrix vector product bywhich is a good improvement in storage over the matrix approach, but we can improve it still further by using a complex stepThis means we only need one system evaluation to approximate the Jacobian, and much more accurate estimates can be made. As shown in a previous post it is second order accurate, and not as limited by subtractive cancellation as the standard difference approach is.