5.2 Fitting a straight  5. Least-Squares Methods of  5. Least-Squares Methods of

5.1 General discussion and formulas

Consider the case where a set of measurements $\{y_i\}$ are made under conditions characterized by variables^5.1 $\{{\bf x}_i\}$. The measurement y may be thought of as the dependent variable, functionally related to the independent variables $\{x\}$. Suppose each measurement y_i is characterized by some measure of uncertainty, for example an estimate of the expected standard deviation $\sigma_i$ about the correct value. Then the preceding section showed that the method of maximum likelihood, applied to a case with Gaussian-distributed uncertainties, leads to the criterion that the best-fit values for parameters a=$\{a_k\}$ in a functional representation of the measurements, $y=f({\bf x},{\bf a})$, is the one that minimizes the $\chi^2$ function

$$\chi^2 = \sum_i {{(y_i-f({\bf x}_i,{\bf a}))^2}\over{\sigma_i^2}}.$$ (5.1)

 

It is important, and often forgotten, that minimizing the squares of the deviations is justified by the expectation that the deviations will be Gaussian distributed. In experimental data, especially in meteorology, the relative occurrence of large deviations often is high compared to a Gaussian distribution, and in such cases it may be appropriate to use other fitting techniques. Although least-squares fits are common even in applications where the justification does not hold, it would be preferable to avoid this practice or at least to recognize the logical inconsistency involved when reporting such fits.

A necessary condition for the least-squares solution is

$${{\partial\chi^2}\over{\partial a_j}} = 0,\ \ {\rm all}\ j .$$ (5.2)

 

The information matrix H obtained from the maximum-likelihood solution using (4.22) can also be obtained from the derivatives of the $\chi^2$function:

$$H_{jk} = -{{\partial^2W}\over{\partial a_j\partial a_k}} ={{1}\over{2}} {{\partial^2\chi^2}\over{\partial a_j\partial a_k}}$$ (5.3)

 

so the variances and covariances are given by

V_aj,ak = H_jk^-1 .  (5.4)

 

   
 5.2 Fitting a straight line...  5. Least-Squares Methods  5. Least-Squares Methods