2.5 Propagation of uncertainty  2. Measurement Uncertainty  2.3 More terminology

2.4 The ASME measurement-uncertainty formulation

The conventions advocated here are adapted from Abernethy and Ringheiser (1985), and are the basis for standards recently adopted by many engineering societies. There are five central features of this methodology:

1.

Measurement uncertainties are classified into two categories, those contributing to random uncertainty (or precision) and those contributing to systematic uncertainty (or bias). This classification of errors depends on the measurement process; in a particular process, random errors can be reduced by repeating the process and averaging results while the systematic errors remain constant.

2.

The recommended uncertainty interval to be reported and used in analyses of precision is the 95% confidence interval, which corresponds to approximately two standard deviations in a Gaussian error distribution. For small samples (where the number of degrees of freedom is less than 30) the Student-t statistic should be used. That is, the deviation should be reported that has 5% probability of occurring in a Student-t distribution with the given number of degrees of freedom. (This will always be larger than the deviation in a Gaussian distribution corresponding to 95% probability.)  As far as possible, estimates of bias should also represent 95% coverage of expected variations.^2.4

3.

To obtain the combined precision resulting from the net effects of many uncorrelated sources, the sample standard deviations are combined in quadrature,^2.5 and the number of degrees of freedom is estimated from the Welch-Satterthwaite equation, given later; cf. (2.14).

4.

Bias limits are also combined in quadrature. If some limits are asymmetrical, the positive and negative limits are combined separately to obtain separate upper and lower bias limits.

5.

An overall characterization of uncertainty that includes effects of random and systematic uncertainties can be combined in either of two ways, but the two components should also be reported separately. The two possibilities are to add the bias and (95% confidence) random uncertainties linearly or in quadrature. The latter leads to consistency with the convention that all quoted limits provide a best estimate of 95% coverage, but either is acceptable under the conventions adopted by engineering societies. A complete uncertainty report must also include an estimate of the number of degrees of freedom associated with the result.

It is also a useful convention that error sources are tabulated with associated estimates of precision, degrees of freedom, and bias. Such tabulations make it possible to isolate major sources of error, to consider the validity of other investigators' estimates of error sources, and to repeat the analyses for a new case when only one of the contributions has changed. Also, tabulated uncertainty reports should separate the effects introduced by calibration, data collection, and data analysis.

An important aspect of this methodology is that the degrees of freedom associated with cited estimates of precision should be calculated and quoted. This becomes important when the number of degrees of freedom in the result is small, so that error limits and propagated errors have non-Gaussian character. Even if it is assumed that the individual measurements are distributed according to a Gaussian error distribution, the true standard deviation for an average of n samples, $\sigma_n$, is not known and must be estimated from the observations. The test statistic $t=(\overline{x}-\xi)/S_n$ (where $\overline{x}$ is the average of n measurements, $\xi$ is the true value of x, and S_n is the estimated standard deviation of the average $\overline{x}$ about $\xi$, determined from $S_n=[\sum_{i=1}^n (x_i\overline{x})^2/(n(n-1))]^{1\over 2}$) will not be Gaussian distributed. The appropriate distribution for such averages is the Student-t distribution. The difference between the Gaussian and Student-t distributions is generally insignificant when the number of degrees of freedom^2.6 exceeds about thirty, but for small sample sizes the differences can be quite important. For this reason, when n<30, the confidence limits used should be taken from the Student-t distribution rather than from the normal distribution. Figure 2.2 shows the relationship between the 95% confidence limits and the t statistic for the Student-t distribution.

 

Figure 2.2: Confidence limits for the Student's t distribution.

If the final number of degrees of freedom is larger than thirty, the range to select for the precision error limit in the result is easily determined by use of twice the estimated standard deviation in the result, the approximate 95% confidence limit. Otherwise, it is necessary to know the effective number of degrees of freedom in the final result, as shown in Fig. . The Welch-Satterthwaite formula provides an estimate:

$$n_r = {{[\sum_i S_{Y,i}^2]^2}\over {\sum_i {S_{Y,i}^4}/n_i}}$$ (2.14)

 

where n_r is the number of degrees of freedom in the final result, S_Y,i is the standard deviation in Y that would result from error source i alone, and n_i is the number of degrees of freedom in that error source.

An uncertainty analysis presented in this format should include these components:

1.

A description of the measurement system and a discussion of the limits within which the analysis is valid. For example, the uncertainty in measurements of wind for a research aircraft might be specified for straight-and-level flight within three hours of takeoff, perhaps within some altitude range.

2.

A tabulation and classification of the elemental error sources. An example is shown in Table 2.1. Each elemental error source should be listed with its associated bias limit (B_i), precision ( t_(95)iS_i), and number of degrees of freedom (n_i), where t_(95)i is the 95% confidence limit in the t statistic for n_i degrees of freedom (and is about 2 for $n\ge30$). It is convenient to tabulate the effect of the error source on the measurement, so that tables contain $S_i (\partialY/\partial x_i)$ and $B_i (\partial Y/\partial x_i)$, where S_i and B_i are the estimated standard deviation and bias limit in x_i. This simplifies error propagation to the final result, although special treatment is still needed in cases where the error contributions are correlated. The error sources should be separated into groups contributing to calibration, data acquisition, and data processing.

3.

A discussion of each elemental error source and a description of the basis for the error estimates. These discussions should reflect the evidence for the tabulated values.

4.

A calculation of the resulting net estimates of overall precision and bias, and calculation of a comprehensive limit that combines systematic and random components of the uncertainty. Although not part of the standard, it should be a goal to use estimates that provide 95% coverage wherever possible.

5.

A summary of the results and the uncertainty limitations of the measurement. It is usually helpful here to highlight the main sources of error and possible actions that could improve the measurements.

Tables 2.1 and 2.2 show examples of this procedure applied to pressure and temperature measured by a research aircraft (King Air N312D) operated by the National Center for Atmospheric Research. Further explanations of the origins of these and other elemental uncertainty estimates can be found in NCAR Technical Notes such as Brown (19XX), Brown (19XX), and Cooper (19XX).
 
  Table 2.1: Elemental Uncertainties Affecting Static Pressure: Table 2.1a. Calibration Uncertainties, wing-mounted 1201 sensor  

item	elemental uncertainty	B [mb]	t95S [mb]	n
1	operation of dead-weight standard	0.10	--$^{\dag }$	--
2	calibration of dead-weight standard	--	--
3	repeatability of 1501 transfer standard	0.10	--
4	height uncertainty in 1501 calibration	--	--
5	50% of dynamic inaccuracy of 1501 transfer standard	0.05	0.05	>30
6	stability of 1501 transfer standard	0.10	--
7	resolution of 1501 transfer standard	0.02	--
8	leaks in the lines during calibration	--	--
9	height uncertainty during calibration	0.04	--
10	curve fit inaccuracy	0.10
	1201 transducer (pW):
11	airborne data system digitization	0.03	0.12	>30
12	1201 static inaccuracy, hysteresis	0.21	--
13	1201 static inaccuracy, repeatability	--	0.14	>30
14	1201 static inaccuracy, voltage variations	--	0.08	>30
	1501 transducer (pF):
15	repeatability of the transfer standard	0.10	--
16	50% of dynamic inaccuracy of 1501 sensor	0.05	0.05	>30
	overall calibration uncertainty, 1201:	0.30	0.19	>30
	overall calibration uncertainty, 1501:	0.24	0.07	>30

$^{\dag }$Dashes indicate negligible contribution relative to other uncertainties.

 

  Table 2.1b: Data Acquisition Uncertainties, 1201 Sensor (p_W)  

item	elemental uncertainty	B [mb]	t95S [mb]	n
17	calibration uncertainty (Table 1a)	0.36	--
18	dependence on temperature	1.0	--
19	dependence on temperature change	0.20	0.20	>30
19	"   "  , 1000 ft/min height change	3.0	--
20	1201 dynamic accuracy, vibration	--	0.2	>30
21	1201 dynamic accuracy, noise	--	0.04	>30
22	line leaks	--	--
23	time lag	--	--
24	airborne data system digitization	0.03	0.12	>30
25	aerodynamic effects	0.20	0.14	>30
26	static defect (Appendix A)	--	--
27	truncation during data processing	--	--

Not applicable in level flight

  Table 2.1c: Data Acquisition Uncertainties, 1501 Sensor (p_F)  

item	elemental uncertainty	B [mb]	t95S [mb]	n
28	calibration uncertainty from Table 1a	0.25	--
29	digitization by the digital transducer	--	0.02	>30
30	static error	0.10	0.10	>30
31	1501 dynamic accuracy, acceleration and vibration	--	0.20	>30
32	1501 dynamic accuracy, voltage variations	--	--
33	long-term stability	0.03	--
34	response time	0.05	--
34	response time, 2000 ft/min descent	0.10	--
35	line leaks	--	--
36	static defect (Appendix A)	0.60	2.0	>30
37	truncation during data processing	--	--
38	effects of airspeed and attack angle	--	--

Not applicable in level flight

  Table 2.1d: SUMMARY OF MEASUREMENT UNCERTAINTY  

item	instrument	B [mb]	t95S [mb]	URSS* [mb]
	1201 Sensor (PW):	1.10 mb	0.31 mb	1.14 mb
	1201 Sensor, rapid climb/descent:	3.2	0.31	3.2 mb
	1501 Sensor (PF):	0.67 mb	2.0 mb	2.12 mb

^*U_RSS = (B² + (t₉₅S) $^2)^{{1}\over{2}}$, where B is the bias limit and (t₉₅S) is the 95% confidence limit for random error.
 
  Table 2.2: Elemental Uncertainties Affecting Temperature: Table 2.2a. Calibration Uncertainties, Rosemount 102E2A1 sensor  

item	elemental uncertainty	B [C]	t95S [C]	n
	Calibration of 102 sensor at NCAR (Cal. step 1):
1	bath uniformity	0.02	<0.02	>30
2	self-heating of the sensor	--	0.01	--
3	stability and repeatability of standard	0.01	0.01	>30
4	accuracy of the Wheatstone bridge	0.05	--
5	lead resistance during calibration	<0.01	--
	Calibration of working standard (Cal. step 2):
6	repeatability of standard	--	<0.01	>30
7	Stability of the standard from cal. to use	<0.05	--
8	self-heating (calibrated out)	--	--
	Calibration of factory standard (Cal. step 3):
9	reported resolution	0.001	--
10	calibration assumed to introduce negl. error	--	--
	Calibration of resistance measurement (step 4):
11	resistance box accuracy	0.05	0.01	>30
12	lead resistance differences	0.01	0.01	>30
13	airborne data system characteristics	--	0.01	>30
14	environmental effects (temperature, etc.)	--	--
15	quadratic representation	--	--
	Calibration of resistance box (steps 5,6):
16	assumed to indroduce negl. error	--	--
	Net uncertainty, calibration:	0.091	0.03	>30

$^{\dag }$Dashes indicate negligible contribution relative to other uncertainties.

 
  Table 2.2b: Data Acquisition Uncertainties, 102E2A1 Sensor  

item	elemental uncertainty	B [C]	t95S [C]	n
	Sensor characteristics:
17	Calibration (from Table 2a)	0.10
18	Self-heating	--	--
19	Long-term stability	0.10	--
20	Effects of conduction from housing	0.05	0.05	>30
21	radiative effects	<0.001	<0.001	>30
22	airflow direction (maximum, extreme AOA)	0.05	0.05	>30
23	stresses on sensor	0.05	0.05	>30
	Data system characteristics:
24	random error and drift, 510BH	0.05	0.05	5
25	temperature sensitivity, 510BH	<0.04	--
26	airborne data system	--	0.01	>30
27	Round-off, machine precision	--	--
28	Recovery factor used	0.10	0.10	30?
29	Mach number used	0.03	0.05	>30
30	Variation in Cp with humidity	--	--

 
  Table 2.2c: Summary  

		B [C]	t95S [C]	URSS
	Rosemount Temperature	0.21	0.15	0.26 C

 

   
Next: 2.5 Propagation of uncertainty Up: 2. Measurement Uncertainty Previous: 2.3 More terminology