5. Least-Squares Methods of  4. The Method of  4.3 Application to experimental

4.4 Relationship to the method of least squares

Consider a set of measurements $\{y\}$ at points $\{x\}$, with varying measurement precision $\{\sigma\}$. Find the parameters $\{a\}$ in the function f(x;a) that are the best match to the observations. The situation might be as shown in Fig. 4.4:
 

Figure 4.4: Example of measurements of a variable y as a function of the variable x, for which each measurement is characterized by a measurement uncertainty designated by error bars above and below the data points representing deviations of one standard deviation. The line shown is an example of the desired representation of the measurements by a function, in this case a straight line, that provides a "best" fit to the observations.

Assume that the measurements $\{y\}$ are distributed according to a Gaussian probability distribution function (3.2). Then the likelihood function is

$${\cal L} = \prod_{i=1}^N {{1}\over{\sqrt{2\pi}\sigma_i}}\exp\left\{-{{(y_i-f(x_i,\{a\}))^2}\over{2\sigma_i^2}}\right\}$$ (4.76)

 

and

$$W = \ln{\cal L} = -{{1}\over{2}}\sum_i{{(y_i-f(x_i,\{a\}))^2}\over{\sigma_i^2}} + C$$ (4.77)

 

where C is a constant not dependent on the parameters $\{a\}$. The maximum-likelihood solution is then equivalent to the solution that gives the minimum value for the sum

$$\chi^2=\sum_i {{(y_i-f(x_i,\{a\}))^2}\over{\sigma_i^2}} \ .$$ (4.78)

 

This provides one justification for the least-squares method of the next section.

SOURCES AND FURTHER READING

Bevington, P. R., 1969: Data Reduction and Error Analysis for the Physical Sciences. McGraw-Hill, New York, 336 pp.

Brownlee, K. A., 1965: Statistical Theory and Methodology in Science and Engineering. John Wiley and Sons, New York, 590 pp.

Orear, J., 1958: Notes on statistics for physicists, UCRL-8417, University of California Radiation Laboratory, Berkeley, CA.

   
 5. Least-Squares Methods  4. The Method of Maximum Likelihood  4.3 Application to experimental design