5.3 Fitting functions linear  5. Least-Squares Methods of  5.1 General discussion and

5.2 Fitting a straight line to observations

If the measurements all have the same uncertainty $\sigma_i=\sigma$, if only uncertainties in y (and not in ${\bfx}$) are important, and if the functional representation of the measurements is f(x;y₀,b)=y₀+bx, then the best-fit values of y₀ and b are those that minimize

$$\chi^2 = \sum_i {{(y_i-y_0-bx_i)^2}\over{\sigma^2}} .$$ (5.5)

 

Then

$${{\partial\chi^2}\over{\partial y_0}} = -{{2}\over{\sigma^2}}\sum_i (y_i-y_0-bx_i) = 0$$ (5.6)

 

or

$$\sum_i y_0 = N y_0 = \sum_i y_i - b\sum_i x_i$$ (5.7)

 

where N is the number of observations. Then

$$y_0 = \overline{y} - b\overline{x} .$$ (5.8)

 

Also,

$${{\partial\chi^2}\over{\partial b}} = - {{2}\over{\sigma^2}}\sum_i(y_i-y_0-bx_i) x_i = 0$$ (5.9)

 

or

$$y_0 = {{1}\over{\overline{x}}} (\overline{yx} - b\overline{x^2}).$$ (5.10)

 

  The solution to the simultaneous equations (5.8) and (5.10) is

$$b = {{\overline{xy}-\overline{x}\thinspace\overline{y}}\over{\overline{x^2}- \overline{x}^2}}$$ (5.11)

 

$$y_0 = {{\overline{y}\overline{x^2} -\overline{x}\thinspace\overline{xy}}\over{\overline{x^2}-\overline{x}^2}} .$$ (5.12)

 

If new variables $x^\prime=x-\overline{x}$ and $y^\prime=y-\overline{y}$ are used, $\overline{x^\prime}=\overline{y^\prime}$=0, and the preceding formulas simplify to

$$b^\prime = {{\overline{x^\primey^\prime}}\over{\overline{{x^\prime}^2}}}$$ (5.13)

 

$$y_0^\prime = 0 .$$ (5.14)

 

When transformed back to the original variables, the fitted form $y^\prime=b^\prime x^\prime$ gives

$$y = (\overline{y}-b^\prime\overline{x}) + b^\prime x$$ (5.15)

 

or

$$y_0 = \overline{y} - b^\prime\overline{x}$$ (5.16)

 

and

$$b^\prime = b .$$ (5.17)

 

Despite the apparent simplification in (5.13) and (5.14), these forms are not usually used because two passes through the data are needed to first remove the mean and then calculate the needed sums. Instead, it is usually simplest to calculate the average values $\overline{x}$, $\overline{y}$, $\overline{xy}$, and $\overline{x^2}$ in one pass through the data, calculate b from (5.11) and then calculate y₀ from (5.12).

The uncertainty in the fit parameters can be determined by using (5.3):

$$H_{11} = {{1}\over{2}}{{\partial\chi^2}\over{{y_0}^2}} = - {......{\sigma^2}} \sum_i (y_i-y_0-bx_i)\bigr) ={{N}\over{\sigma^2}}$$ (5.18)

 

$$H_{22}={{1}\over{2}}{{\partial^2\chi^2}\over{\partial b^2}} ......_i(y_i-y_0-bx_i)x_i\bigr) ={{N\overline{x^2}}\over{\sigma^2}}$$ (5.19)

 

$$H_{12}=H_{21}={{1}\over{2}}{{\partial^2\chi^2}\over{\partial......}\sum_i(y_i-y_0-bx_i)\bigr) ={{N\overline{x}}\over{\sigma^2}}$$ (5.20)

 

where the indices (1,2) represent respectively y₀ and b.

Note that the variance in y₀, V _y0,y0, is not equal to 1/H₁₁. The error matrix is the inverse of H, and in this case that is not obtained by taking the inverse of each element. Instead, the full matrix H must be calculated and inverted:

$${\bf H} =\pmatrix{{{N}\over{\sigma^2}}&{{N\overline{x}}\over......ne{x}}\over{\sigma^2}}&{{N\overline{x^2}}\over{\sigma^2}}\cr}$$ (5.21)

 

$${\bf H}^{-1} = {{\sigma^2}\over{N(\overline{x^2}-\overline{x......matrix{\overline{x^2}&-\overline{x}\cr-\overline{x}&1\cr} ~.$$ (5.22)

 

Then

$$\sigma_{y_0} = \sigma\Bigl({{\overline{x^2}}\over{N(\overline{x^2}-\overline{x}^2)}}\Bigr)^{1/2}$$ (5.23)

 

$$\sigma_b = \sigma \Bigl({{1}\over{N(\overline{x^2}-\overline{x}^2)}}\Bigr)^{1/2}$$ (5.24)

 

$$V_{y_0b} = -{{\sigma^2\overline{x}}\over{N(\overline{x^2}-\overline{x}^2)}} .$$ (5.25)

 

If $\overline{x}$=0, the covariance between y₀ and b is zero, and the parameters become uncorrelated. This is an advantage to working with the transformed variables $x^\prime$ and $y^\prime$which have the means removed.

If the uncertainty $\sigma$ is unknown, it may be estimated from the measurements:

$$s^2 = {{1}\over{N-2}} \sum_i(y_i-y_0-bx_i)^2 \ .$$ (5.26)

 

The factor 1/(N-2) arises because two degrees of freedom have been lost in this two-parameter fit to the data.

The extension to the case where the uncertainties $\{\sigma_i\}$ vary is straightforward. It will be presented as part of the more general discussion of fitting to linear parameters.

   
 5.3 Fitting functions linear ... 5. Least-Squares Methods ...  5.1 General discussion