3
Background rejtrix.h C++ library (free on authors web site) capably solves ill-conditioned linear systems with constraints, including auto-regularization based on the Discrete Picard Condition. But complete apps were needed for some users. We choose Excel-compatible *.csv files format. Result was an easy-to-use suite of solvers for various weightings, manual vs auto, etc. But… Users wanted more information about the problem. So more diagnostic outputs were added. These Solve* programs are free on authors web site.

5
Each solve* program uses about the same format for output. – Output varies a little with the input data and the solution method chosen – Some analyses are not possible in some circumstances – For example, some only make sense if the equations are over-determined. – Also some may only make sense if the user provides error estimates.

14
First the results of a Monte Carlo error analysis… To do this, we generate a number of random variations using the users error estimates, then discard the high and low 2.5% of each variable. We report the extremes of the remaining values. Note that the effect of ALL CONSTRAINTS is implicitly included.

15
An Analysis of Consistency in the Matrix and RHS Vector Standard least squares minimizes the norm of the residual vector. This residual vector can be viewed as the set of changes required to the RHS to make the system of equations consistent. Total least squares can be viewed as computing the smallest set of changes to both A and b that are required to make the system consistent. So we compute the TLS solution and display to the user the largest of the inconsistencies the TLS suggests.

16
An Example of Consistency Analysis Note: this data is from a different problem than most of the above slides, for better illustration. Note: The constraints (equality and inequality) do not play a role in this analysis.

17
A Different Analysis of the Matrix… (this applies only to overdetermined systems) An old way of estimating the relative variance of the solution elements (ignoring right-hand-side issues) was to look at the diagonal elements of (A T A) -1. We can use the diagonal of (A T A) -1 or some function of it as a heuristic measure of model quality in a single number. (We use the square root of the average variance). Then, we can re-measure this heuristic with an equation deleted. If the heuristic doesnt increase, the equation may be irrelevant. If the heuristic increases greatly, the equation is crucial. We call the increase in the heuristic the importance of the equation.

18
Here is an Example of the Importance Analysis (Equation 6 may be almost useless) Note: this data is from a different problem than most of the above slides. Question: when might the Importance measure be negative? That is, when would the variances be much smaller when an equation is removed??

19
One Common Answer: When the system of equations is square or under-determined, and the approximate rank is one less than the row count. Then the only near-zero singular value may disappear when a particular row, or any of several rows, is deleted.

20
Linear Dependence Analysis (Examination of the U Matrix can reveal clear dependencies) Note: this data is from a different problem than most of the above slides, for better illustration. Note: The constraints (equality and inequality) do not play a role in this analysis. Note: The programs will report several such sets of dependences, if present. In this case, the two rows are evidently the same (or differ by simply a scale factor.)

22
Correlation of Solution with Rows of Matrix… Ideally the solution should be a linear combination of the rows of the matrix. It may be worthwhile to know how each row correlates to the solution. We depict this relation as an angle, using v · w = |v| |w| cos(angle between v and w).

23
Correlation of RHS with Columns of Matrix… Equally, the right hand side data should be a linear combination (given by the solution) of the columns of the matrix. So we depict that relationship by angles between the columns and the RHS. And there may be other such angle relationships depicted also.

24
Now We Give Information About the Unregularized and Unconstrained Solution by SVD (aka the Pseudo-Inverse Solution) in the next few displays…

29
Now, the final solution after all regularization and constraints can be multiplied by V T to show the Modification to the Picard Condition Vector… Compare this vector to the D.P.C.V. two slides back to see how the regularized and constrained solution behaves versus the pseudo-inverse solution. The latter elements behave better here.

30
Finally, Some Measures of Quality of the SVD Orthonormality error refers to the elements of the matrix U T U-I, for example. This matrix should be zero, within round-off.