9.4 Interpolation and extrapolation  9. Numerical Methods  9.2 The Taylor series

9.3 Finite-difference derivatives

Finite-difference formulas make it possible to use arithmetic operations to determine derivatives. They are based on the Taylor series truncated at various orders in the expansion. The first-order expansion, of course, just gives the form

$$f^\prime(x)={{f(x+\delta)-f(x)}\over{\delta}} + {\cal O}(\delta)$$ (9.3)

 

or

$$f^\prime(x)={{f(x)-f(x-\delta)}\over{\delta}} + {\cal O}(\delta)$$ (9.4)

 

where ${\cal O}(\delta)$ indicates that the error in this approximation is of order $\delta$. If an index is used to indicate consecutive values at intervals $\delta$, so that $f_{i+1}=f(x+\delta)$ if f_i=f(x), then

$$f_i^\prime(x)={{(f_{i+1}-f_i)}\over{\delta}}$$ (9.5)

 

is called the first forward difference and

$$f_i^\prime(x)={{(f_{i}-f_{i-1})}\over{\delta}}$$ (9.6)

 

is the first backward difference formula for the derivative.

Similar expressions can be found for higher-order derivatives. For example, the second derivative is

$$f''_i = {{f'_{i+1}-f'_i}\over{\delta}} + {\cal O}(\delta)$$ (9.7)

  

$$~~~~ ={{{{f_{i+2}-f_{i+1}}\over{\delta}}-{{f_{i+1}-f_i}\over{\delta}}}\over{\delta}} + {\cal O}(\delta)$$ (9.8)

  

$$~~~~ = {{f_{i+2}-2f_{i+1}+f_i}\over{\delta^2}} + {\cal O}(\delta)\ \ .$$ (9.9)

 

The corresponding backward-difference formula is

$$f''_i = {{f_{i-2}-2f_{i-1}+f_i}\over{\delta^2}} + {\cal O}(\delta)\ \ .$$ (9.10)

  

The preceding formulas were obtained by taking only the first term in the Taylor series containing the desired derivative. More accurate formulas can be obtained by retaining more terms. For example,

$$f'(x)={{f(x+\delta)-f(x)}\over{\delta}}-{{\delta}\over{2}}f''(x)+\cdots$$ (9.11)

  

$$~~~~={{f(x+\delta)-f(x)}\over{\delta}}-{{\delta}\over{2}}\le......)}\over{\delta^2}}+{\cal O}(\delta)\right]+{\cal O}(\delta^2)$$ (9.12)

 

or

$$f'_i = {{-f_{i+2}+4f_{i+1}-3f_i}\over{2\delta}} +{\cal O}(\delta^2) \ \ .$$ (9.13)

 

Another improvement is to use central differences:

$$f_{i+1}=f_i+\delta f'_i+{{\delta^2}\over{2!}}f''_i+\cdots$$ (9.14)

  

$$f_{i-1}-f_i-\delta f'_i+{{\delta^2}\over{2!}}f''_i+\cdots$$ (9.15)

  

$$f'_i={{f_{i+1}-f_{i-1}}\over{2\delta}}+{\cal O}(\delta^2) \ \ .$$ (9.16)

 

The centered-difference formula is more accurate than the forward or backward-difference formulas, but still involves only two terms. Table 9 .1 lists some finite-difference formulas for derivatives.
 

 
  Table 9.1: Finite-difference derivatives

The entries in this table are the coefficients of the corresponding function value f_i+j in the expression for the derivative. For example, the second-order centered-difference expression for f''' is $(-f_{i-2}+2f_{i-1}-2f_{i+1}+f_{i+2})/(2\delta^3)$ where $\delta$ is the interval between consecutive values f_i.
 

&#&     #     height2pt&&&& &derivative&& f_i-3&&f_i-2&&f_i-1&&f_i&&f_i+1&&f_i+2&&f_i+3&&f_i+4& height2pt&&&&&&&&&&&&&&&&&&  height2pt&&&&&&&&&&&&&&&&&& &centered&&&&&&&&&&&&&&&&& &(O($\delta^2$))&&&&&&&&&&&&&&&&& height2pt&&&&&&&&&&&&&&&&&&  height2pt&&&&&&&&&&&&&&&&&& & $\delta f'_i$&& &&&&-1&&0&&1&&&&&&& & $\delta^2f''_i$&&&&&&1&&-2&&1&&&&&&& & $2\delta^3f'''_i$&&&&-1&&2&&0&&-2&&1&&&&& height2pt&&&&&&&&&&&&&&&&&&  height2pt&&&&&&&&&&&&&&&&&& &centered&&&&&&&&&&&&&&&&& &(O($\delta^4$))&&&&&&&&&&&&&&&&& height2pt&&&&&&&&&&&&&&&&&&  height2pt&&&&&&&&&&&&&&&&&& & $12\delta f'_i$&&&&1&&-8&&0&&8&&-1&&&&& & $12\delta^2f''_i$&&&&-1&&16&&-30&&16&&-1&&&&& & $8\delta^3f'''_i$&&1&&-8&&13&&0&&-13&&8&&-1&&& height2pt&&&&&&&&&&&&&&&&&&  height2pt&&&&&&&&&&&&&&&&&& &forward&&&&&&&&&&&&&&&&& &(O($\delta$))&&&&&&&&&&&&&&&&& height2pt&&&&&&&&&&&&&&&&&&  height2pt&&&&&&&&&&&&&&&&&& & $\delta f'_i$&&&&&&&&-1&&1&&&&&&& & $\delta^2f'_i$&&&&&&&&1&&-2&&1&&&&& & $\delta^3f'''_i$&&&&&&&&-1&&3&&-3&&1&&& height2pt&&&&&&&&&&&&&&&&&&  height2pt&&&&&&&&&&&&&&&&&& &forward&&&&&&&&&&&&&&&&& &(O($\delta^2$))&&&&&&&&&&&&&&&&& height2pt&&&&&&&&&&&&&&&&&&  height2pt&&&&&&&&&&&&&&&&&& & $2\delta f'_i$&&&&&&&&-3&&4&&-1&&&&& & $\delta^2f''_i$&&&&&&&&2&&-5&&4&&-1&&& & $2\delta^3f'''_i$&&&&&&&&-5&&18&&-24&&14&&-3& height2pt&&&&&&&&&&&&&&&&&&  height2pt&&&&&&&&&&&&&&&&&&  Finite-difference formulas are often used to evaluate derivatives of functions when analytical expressions are not available. For example, the procedures for nonlinear least-squares fitting often need derivatives of the chisquare function with respect to the parameters, and these are sometimes easier to evaluate numerically than analytically. Because the error in the derivatives scales with the size of $\delta$, the step used for evaluating such derivatives should be small, but not so small that rounding errors are introduced by the digital representation of the values in the computer.

   
 9.4 Interpolation and extrapolation  9. Numerical Methods  9.2 The Taylor series