Method Validation: Performance Enhancing Drugs

Purpose:  To allow the students to recognize the key components in method validation by investigating the role of blanks, calibrations, standardizations and uncertainties in certifying measurement data.

Learning Outcomes:

By the end of the assignment students will be able to:

• Create and interpret a calibration curve.
• Explain the differences between types of blanks.
• Develop a validation process.

For the following section consider the analysis of urine samples of an athlete, where a GC/MS analysis is being performed to determine if synthetic testosterone has been used by the athlete.  In addition consider the consequences of a test result that indicates the use of synthetic steroids; an athlete caught using steroids may be banned from the sport for several years.  Consequently, you need to make sure that the data is correct.

1. Is it viable/advisable to perform analyses of urine samples for steroids by turning on the GC/MS and making measurements of the samples without any other measurements? Explain why or why not.
2. What sorts of measurements should be made with the GC/MS before a urine sample can be tested for steroids?
3. Describe the types of blanks you should test prior to undertaking the analysis of steroids in urine by GC/MS?
4. What is the difference between a reagent blank and a calibration blank? (Alternately, define the calibration blank and the reagent blank).
5. In the analysis of steroids in urine, an m/z value of 432 is typically monitored to identify a GC peak as being a steroid that has been derivatized with MTSFA. What might give rise to a signal with the proper GC migration time, and proper m/z value when a blank urine sample is analyzed?
6. What are the criteria upon which you will be satisfied with the calibration of the GC/MS, and the analytical results that it yields?
7. Devise a protocol (sequence of runs to be performed the day of the analysis) for testing the GC/MS to ensure that, when a urine sample is analyzed, the results are accurate?

In order to prevent false positives, which are arguably more detrimental than false negatives in anti-doping analysis, WADA imposes specific guidelines on what constitutes a measurement that exceeds a Threshold Limit.  The Threshold Limit for urinary testosterone has been established by WADA to be 200 ng/mL.  Exceeding this limit triggers the need for the sample to be analyzed by isotope ratio mass spectrometry to confirm the presence of exogenous testosterone.  Similarly, if the ratio of testosterone to epitestosterone exceeds 4:1, the isotope ratio test is triggered.

8. When a testosterone standard is measured multiple times by the same GC/MS the quantified amount of testosterone can vary with each measurement. What might be the cause of these variations?
9. If a urine sample was split and tested for testosterone on different GC/MS instruments in the same lab, should the results be identical on each instrument? Explain your reasoning.
10. In a lab where anti-doping analyses are conducted there are a number of trained technicians who may perform the actual analyses. Is it a problem if an analysis does not yield identical results when different technicians analyze the same sample? Explain your reasoning.
11. Explain what you think would be an acceptable level of variation in the replicate measurements of testosterone in a urine sample by a GC/MS used in anti-doping analysis?
12. WADA states that the maximum allowable concentration of testosterone in urine samples is 200 ng/mL. If a single measurement was made of an athlete’s urine sample, and it was found to have a concentration of 217 ng/mL testosterone, is the athlete in violation of the anti-doping rules? What additional information might you need to better make your decision?
13. For the analysis of testosterone in urine samples, is it appropriate to calibrate the GC/MS with a standard testosterone solutions prepared in pure water?
14. What concentration of testosterone standards should you prepare to develop a calibration curve to test for urinary testosterone levels in an athlete? How should these solutions be prepared?
15. What is an appropriate number of data points to have on a linear calibration curve for the analysis of testosterone in urine samples?
16. What would be an acceptable correlation coefficient for a linear calibration graph for the analysis of testosterone in urine samples?
17. Is it important to make replicate measurements of each calibration point when preparing for the analysis of an athlete’s urinary testosterone levels?
18. Your lab has purchased a new instrument, a capillary electrophoresis mass spectrometer (CE/MS). You have found a published method that appears to be appropriate for the analysis of testosterone in urine (http://rdcu.be/tq5p) and you want to use the instrument to test samples from an upcoming competition. However, before you can use the instrument you need to be certain that the analyses will be accurate.  List what tests you will need to perform in order to validate the use of the new instrument, and briefly describe the purpose of each test in terms of its applicability to the validation.

As athletes live and compete in countries around the world, WADA has accredited individual laboratories in numerous countries.  Furthermore, to deal with all of the samples that must be processed, each lab has a large number of technicians who perform the analyses.  Consequently, methods of analysis must be validated, such that the results obtained by one technician in one lab are equivalent to those obtained by another technician in a different lab.  For the analysis of testosterone WADA requires an absolute combined uncertainty in the measurements of testosterone to be no more than 20 ng/mL (10% relative error).  The combined uncertainty (u_c) is obtained from the intermediate precision standard deviation (s_w) and the root mean square of the bias (RMS_bias) shown in Equation 1 below, both values are expressed as percentages.  To further protect against false positive results, the u_c value is further modified by a multiplication factor of 2, to obtain the expanded uncertainty (U_95%).  The value of k is set to 2 by WADA to obtain a result that is approximates 95% coverage for a two tailed distribution (Equation 2).  The decision limit (DL) is the value above which a sample is considered to have been doped.  This value is established using the combined uncertainty (U_95%) and the established threshold (T) for the compound.  As the threshold for testosterone is 200 ng/mL, and the maximum absolute combined uncertainty is 10%, or 20 ng/mL, the decision limit is set at a maximum of 240 ng/mL.  However, laboratories that can achieve expanded uncertainty levels (U_95%) that are less than the maximum allowed (10%) will have lower decision limits (Equation 3).

\[\begin{align}
&\textrm{Equation 1:}\hspace{30px} u_c = \sqrt{s_w^2 + RMS_{bias}^2}\nonumber\\
&\textrm{Equation 2:}\hspace{30px} U_{95\%} = k \times u_c\:\: (k=2)\nonumber\\
&\textrm{Equation 3:}\hspace{30px} DL = T + U_{95\%}\nonumber
\end{align}\nonumber\]

Definitions:

RMS_bias - The root mean square of the bias is obtained from multiple (n) recovery experiments.  The recovery measurements determine the difference between the mean measured value for a reference standard and the documented values for the reference materials.  The RMS_bias is taken from the root mean square of the differences between the obtained mean values and the reference values.

Intermediate precision – WADA guidelines stipulate that the intermediate precision can be estimated from the standard deviation in the results obtained by multiple measurements of standard solutions (at least 10 different solutions) made to be close to the appropriate threshold value.  The analyses are performed in the same laboratory on different instruments (appropriate for the analysis), over different days, and performed by different operators.  More details on intermediate precision can be found at http://www.astm.org/SNEWS/ND_2010/datapoints_nd10.html

19. The repeatability of a measurement is a crucial part in the validation of any method, as the standard deviation in these measurements is a measure of the intra-laboratory repeatability of the measurement, for which specific guidelines may be established. How should a lab that is measuring urinary testosterone levels go about determining the repeatability in their analysis? What number of measurements should be made?  What concentration of testosterone should be measured?  What matrix should the testosterone by in?  Should these measurements be done on different GC/MS instruments?  In relative terms, when should the sample be prepared and all the measurements be made?
20. As there are WADA-accredited laboratories around the world for testing samples from athletes in various countries and at international competitions, would you expect the same urine sample, analyzed for testosterone, to yield the same results in different accredited labs? If you expect the values to be different, what magnitude of difference would you expect to see? Would this difference be more, less or the same as the differences seen in the repeatability tests?
21. Given that a single laboratory accredited to do urine analysis for testosterone might have multiple technicians who can perform the analyses, would you expect the results obtained by different individuals, in the same lab, using the same equipment, to get the same values and the same standard deviation when measuring the same standard testosterone solution? Do you expect the standard deviation in the measurements to be greater or less than that seen in repeatability measurements? How would they compare to reproducibility measurements?
22. In the tables below is the data collected for the tests of two athletes (A & B). Urine samples were taken from each athlete at two separate competitions, one in May one in August, occurring in different countries. Due to the different locations of the competitions the samples were analyzed by two different labs (Lab 1 & Lab 2). The data presented below is the raw data obtained from the GC-MS analysis, presenting the quantified peak area values for the testosterone peak from each sample.

    The day of any anti-doping analysis each lab conducts a calibration of the GC/MS instrument to be used.  Below is the calibration data (peak area from testosterone) from the two labs on the day of their respective analyses:

Lab 1

Standards (ng/mL)	50	100	150	200	300	400	Reagent Blank	Negative Urine
Trial #1	42,285	78,746	85,141	132,000	195,241	250,003	172	2478
Trial #2	38,467	76,737	93,270	129,261	194,143	237,201	187	2443
Trial #3	40,398	75,595	91,059	129,921	191,443	249,001	170	2423
Trial #4	39,824	74,480	94,348	124,641	188,258	243,823	192	2423
Trial #5	36,606	73,823	90,268	129,175	185,186	237,731	193	2535

Lab 2

Standards (ng/mL)	50	100	150	200	300	400	Reagent Blank	Negative Urine
Trial #1	42,285	78,746	85,141	132,000	195,241	250,003	172	2478
Trial #2	38,467	76,737	93,270	129,261	194,143	237,201	187	2443
Trial #3	40,398	75,595	91,059	129,921	191,443	249,001	170	2423
Trial #4	39,824	74,480	94,348	124,641	188,258	243,823	192	2423
Trial #5	36,606	73,823	90,268	129,175	185,186	237,731	193	2535

As part of the WADA certification each of the labs measured a standard sample of testosterone in a urine matrix.  In each trial, performed on different days the solution was prepared from the stock to be the listed concentration.  The analyst for each trial varied by the day and time that the control was conducted, and the analysis was performed on any one of the many calibrated GC/MS instruments in the lab.  The data from these controls is listed in the table below.

	Control Lab 1 (195 ng/mL)	Control Lab 2 (204 ng/mL)
Trial #1	125,541	114,197
Trial #2	126,969	115,438
Trial #3	126,125	115,043
Trial #4	125,281	115,156
Trial #5	127,099	114,423
Trial #6	124,957	114,197
Trial #7	126,969	115,551
Trial #8	127,618	115,043
Trial #9	125,346	114,084
Trial #10	126,774	114,197

Periodically each lab performs analyses of known standards of testosterone in order to evaluate the precision with which each lab can perform an analysis, for which the “true” analytical result is known.  Listed below are the most recent analyses for each lab for simulated urine samples with testosterone concentrations near the threshold level.

Testosterone conc.	Lab 1 (ng/mL)	Lab 2 (ng/mL)
185 ng/mL	184	187
198 ng/mL	199	199
164 ng/mL	160	163
214 ng/mL	210	216
208 ng/mL	207	210
174 ng/mL	173	170
180 ng/mL	177	181
225 ng/mL	227	222
201 ng/mL	198	201
178 ng/mL	180	182
219 ng/mL	221	216

Tabulated below are the results of the analyses of the athlete urine samples conducted at each of the labs.

Samples	Athlete A (Lab 1)	Athlete B (Lab 1)	Athlete A (Lab 2)	Athlete B (Lab 2)
Trial #1	126,902	110,651	127,854	160,528
Trial #2	120,581	104,976	130,695	146,322
Trial #3	122,000	107,814	139,219	163,369
Trial #4	131,930	112,069	140,639	169,051
Trial #5	128,938	113,488	136,378	159,107

Based on the data collected in the tables above:

• Generate a calibration curve for each of the labs from their data. Evaluate the calibration curve and comment on whether or not the calibration for each lab is reliable, and if it should be used for anti-doping analyses.
• For each lab determine the expanded uncertainty levels (U_95%). Quantify what the respective decision limits will be for a testosterone analysis in each lab.  Comment on the capability of each lab to identify a doping athlete.
• From the above information determine if either athlete was doping during either of the events.

Contributors and Attributions

• Chris Harrison, San Diego State University ([e-mail])
• Sourced from the Analytical Sciences Digital Library