User menu

User login

You are here

Primary tabs

least squares

The general problem to be solved by the least squares method is this: given some direct measurements yyy of random variables, and knowing a set of equationsfff which have to be satisfied by these measurements (possibly involving unknown parametersxxx), find the set of xxx which comes closest to satisfying

f⁢(x,y)=0fxy0f(x,y)=0

where “closest” is defined by a Δ⁢ynormal-Δy\Delta y such that

f⁢(x,y+Δ⁢y)=0⁢ and ⁢Δ⁢y2⁢ is minimized fxynormal-Δy0 and normal-Δsuperscripty2 is minimized f(x,y+\Delta y)=0\text{ and }\Delta y^{2}\text{ is minimized }

The assumption has been made here that the elements of yyy are statistically uncorrelated and have equal variance. For this case, the above solution results in the most efficent estimators for xxx, Δ⁢ynormal-Δy\Delta y. If the yyy are correlated, correlations and variances are defined by a covariance matrixCCC, and the above minimum condition becomes

Least squares solutions can be more or less simple, depending on the constraint equations fff. If there is exactly one equation for each measurement, and the functionsfff are linear in the elements of yyy and xxx, the solution is discussed under linear regression. For other linear models, see Linear Least Squares. Least squares methods applied to few parameters can lend themselves to very efficient algorithms (e.g. in real-time image processing), as they reduce to simple matrix operations.

Algorithms avoiding the explicit calculation of d⁢f/d⁢xdfdxdf/dx and d⁢f/d⁢ydfdydf/dy have also been investigated, e.g. [1]; for a discussion, see [2]. Where convergence (or control over convergence) is problematic, use of a general package for minimization may be indicated.