Solution to measurement uncertainty challenge
The winner of the measurement uncertainty challenge (pub-
Solution to measurement uncertainty challenge
Juris Meija 0
Option A 0
0 Measurement Science and Standards, National Research Council Canada , 1200 Montreal Road M-12, Ottawa, ON K1A 0R6 , Canada
The analysis of sterling silver has produced information regarding the silver content in this alloy . Although we are interested only in the mass fraction of silver in the alloy, let us first collect all that is known about this material: 1. The alloy is produced from pure silver and pure copper. Consequently, no other elements are expected in the final material. 2. The mass of each pin is 1 g. 3. Determination of silver from five pins gave the following results for the mass of silver in each pin: m(Ag) = 0.844 g, 0.888 g, 0.825 g, 0.907 g, 0.882 g, with standard deviation s(m(Ag)) = 0.034 g. 4. Determination of copper from five pins gave the following results for the mass of copper in each pin: m(Cu) = 0.060 g, 0.096 g, 0.067 g, 0.075 g, 0.070 g, with standard deviation s(m(Cu)) = 0.014 g.
This model provides rather a straightforward estimate of the
mass fraction of silver: w(Ag) = m(Ag)/(1 g). The mean of the
five observations is wA(Ag) = 0.869 g/g. The 95% confidence
interval of wA(Ag) can be obtained, assuming a normal
distribution of the measurement errors. Since the actual (true)
magnitude of the measurement errors is unknown, we use the
quantiles of the t distribution with four degrees of freedom to
obtain the expanded uncertainty of wA(Ag):
wCðAgÞ ¼ mðAgmÞðþAgmÞðCuÞ :
U ðwAðAgÞÞ ¼ tp¼1−0:205;v¼4
¼ 0:042 g=g:
sðmpðAffi5ffi gÞÞ ¼ 2:78 0p:03ffi5ffi4
siðmðAgÞÞ ¼ sðmðAgÞÞ
Alternatively, the doubt about the standard deviation s(m(Ag))
can be incorporated with use of the parametric bootstrap
resampling method, where we draw samples from a normal distribution
with a known mean (0.869 g/g) and unknown standard deviation.
Here, the standard deviation itself is modeled as a random variable
that follows a scaled and shifted chi-squared distribution to be
consistent with the observed standard deviation:
We proceed with random sampling (of N = 5 samples) from
the normal distribution with the mean 0.869 g/g and standard
uncertainty si as given in Eq. 2. The means of such samples
provide 95% confidence interval U(wA(Ag)) = 0.042 g/g, in
agreement with the result shown in Eq. 1.
Calculations for this model follow the same route as in option
A, and provide wB(Ag) = 0.926 g/g with expanded uncertainty
U(wB(Ag)) = 0.017 g/g with use of either Eq. 1 or Eq. 2.
To some, the most intuitive choice for data reduction of this
dataset is to take into account both silver and copper
determinations while disregarding the fact that the pins each weigh
1 g. The corresponding measurement model is
Because the same variable appears in the equation, one can
rewrite Eq. 3 to avoid the dependency problem between the
numerator and the denominator:
wCðAgÞ ¼ 1=ð1 þ mðCuÞ=mðAgÞÞ:
From Eq. 4, we obtain wC(Ag) = 0.922 g/g. Modeling the
average results for both m(Ag) and m(Cu) as random variables
with a t distribution, m(Ag) ~ t(μ = 0.869, s = 0.015, v = 4) and
m(Cu) ~ t(μ = 0.074, s = 0.006, v = 4), we obtain expanded
uncertainty U(wC(Ag)) = 0.012 g/g. The parametric bootstrap
method involving Eq. 2 to account for the uncertainty in the
standard deviation of both copper and silver measurements
provides an expanded uncertainty U(wC(Ag)) = 0.017 g/g.
Peter mistakenly thought that both silver and copper
measurements were made from the same five pins. He calculated the
correlation between these data (0.578) and propagated the
uncertainty by taking this into account (The NIST Uncertainty Machine
at http://uncertainty.nist.gov is useful to do such calculations).
Option D combines all the available information while
assuming the homogeneity of the pins. This can be achieved, for
example, by our taking the average of the results wA and wB.
For this purpose, we can choose the weighted average:
u−2ðwAÞ⋅wAðAgÞ þ u−2ðwBÞ⋅wBðAgÞ
u−2ðwAÞ þ u−2ðwBÞ
u2ðwDðAgÞÞ ¼ u−2ðwAÞ þ u−2ðwBÞ :
Using the values wA(Ag) = 0.869(15) g/g and wB(Ag) =
0.926(6) g/g, we obtain wD(Ag) = 0.918(6) g/g. The expanded
uncertainty of wD(Ag) can be obtained by multiplication of the
standard uncertainty and the corresponding t value. The
Welch–Satterthwaite equation provides the approximate
degree of freedom for the weighted average, v = 5.2, which
corresponds to a 95% critical t value of 2.6 and expanded
uncertainty U(wD(Ag)) = 2.6 × 0.006 = 0.016 g/g. The weighted
average is not without its shortcomings. Most notably, as
explained earlier, the estimates of variance can be inaccurate
when they are derived from a small number of replicates.
Option D assumes that all pins have identical mass fraction of
silver, which might not be the case. A method that does not
make such an assumption is nonparametric bootstrap
resampling of the data. For this purpose, we can evaluate the
uncertainty of the silver mass fraction from bootstrap resampling
(with replacement) from the available set of ten results: five
values of m(Ag)/(1 g) and five values of (1 – m(Cu))/(1 g),
which treats both silver and copper data on an equal footing.
The average is calculated for each bootstrap sample and, after
a sufficient number of such samples, the following summary
statistics is obtained: wE(Ag) = 0.898 g/g with expanded
uncertainty U(wE(Ag)) = 0.023 g/g.
In summary, different choices on how to interpret the
measurement results will lead to different uncertainty statements,
] as shown in Table 2. The approaches outlined here are
by no means the only choices that can be made, and many
readers might choose other ways to interpret this
seemingly simple dataset. Indeed, as is noted in the Guide to
the Expression of Uncertainty in Measurement, “the
evaluation of measurement uncertainty is neither a routine
task nor a purely mathematical one” [
measurement uncertainty is somewhat of a personal statement; it
is your uncertainty, not the uncertainty.
1. Meija J . Measurement uncertainty challenge . Anal Bioanal Chem . 2017 ; 409 : 2497 .
2. Possolo A , Pintar A L . Plurality of Type A evaluations of uncertainty . Metrologia 2017 ; 54 : 617 .
3. Joint Committee for Guides in Metrology. Evaluation of measurement data - Guide to the expression of uncertainty in measurement , JCGM 100 : 2008 . Joint Committee for Guides in Metrology; 2008