Skip to main content
Chemistry LibreTexts


Short-term or level-1 standard deviations from J repetitions     

An analysis of the check standard data is the basis for quantifying random errors in the measurement process -- particularly time-dependent errors. Given that we have a database of check standard measurements as described in data collection where


Y(kj)(K=1,...,K, j=1,...,J

represents the jth repetition on the kth day, the mean for the kth day is

YBAR(k.) = (1/J)*SUM[Y(kj)]  where the summation is for j=1 to J


and the short-term (level-1) standard deviation with v = J - 1 degrees of freedom is



s(k) = SQRT((1(J-1)*SUM[j=1 to J][(Y(kj) - YBAR(k.))**2]).

Drawback of short-term standard deviations      An individual short-term standard deviation will not be a reliable estimate of precision if the degrees of freedom is less than ten, but the individual estimates can be pooled over the K days to obtain a more reliable estimate. The pooled level-1 standard deviation estimate with v = K(J - 1) degrees of freedom is




s(1) = SQRT((1/K)*SUM[k=1 to K][s(k)**2]).


This standard deviation can be interpreted as quantifying the basic precision of the instrumentation used in the measurement process.

Process (level-2) standard deviation    

 The level-2 standard deviation of the check standard is appropriate for representing the process variability. It is computed with v = K - 1 degrees of freedom as:


s(chkstd) = s(2) = SQRT((1/(K-1))*SUM[k=1 to K][(YBAR(k.) - YBAR(..))**2])



YBAR(..) = (1/K)*SUM[k=1 to K][YBAR(k)]

is the grand mean of the KJ check standard measurements.

Use in quality control     

The check standard data and standard deviations that are described in this section are used for controlling two aspects of a measurement process:

  1. Control of short-term variability
  2. Control of bias and long-term variability

Case study: Resistivity check standard      For an example, see the case study for resistivity where several check standards were measured J = 6 times per day over several days.