Application of Receiver-Operator Analysis to Diagnostic Tests of Iron Deficiency in Man

916 KIM ET AL. 12. Salzman EW 1977 Platelets. prostaglandins. and cyclic nucleotides. In: Gaetano G. Garattini S (eds) Platelets: a multidisciplinary approach. Raven Press, New York, p 227 13. Simons TJB I974 Resealed ghosts used to study the effect of intracellular calcium ions on the potassium permeability of human red cell membranes. J Physiol 246:52P 14. Skaer RJ. Peters PD. Emmines J P 1970 Platelet dense bodies: a quantitative microprobe analysis. J Cell Sci 20:441 15. Sneddon JM 1973 Blood platelets as a model for monoamine-containing neurons. In: Kerkut GA. Phillis J W (eds) Progress in Neurobiology, vol 1. Pergamon Press. Oxford. p 15 1 16. Steiner M. Tateishi T 1974 Distribution and transport of calcium in human platelets. BiochLm Biophys Acta 367:232 17. Taylor PM. Heptinstall S 1980 The abilities of human blood platelets to bind extracellular calcium and to be aggregated by adenosine diphosphate are related. Br J Haematol 46: 1 15 18. White JG 1970 Origin and function of platelet dense bodies. Ser Haematol 3:17 003 1-3998/84/ 1809-09 16$02.00/0 PEDIATRIC RESEARCH Copyright O 1984 International Pediatric Research Foundation. Inc. Vol. 18, No. 9, 1984 Printed in U.S.A. Application of Receiver-Operator Analysis to Diagnostic Tests of Iron Deficiency in Man INSUN KIM, ERNEST0 POLLITT, RUDOLPH L. LEIBEL, FERNANDO E. VITERI, AND EDMUNDO ALVAREZ School of Public Health, The University of Texas Health Science Center at Houston, Houston, Texas 77225 [I.K., E.P.], Laboratory of Human Behavior and Metabolism, Rockefeller University, New York, New York, 10021 [R.L.L.], Pan American Health Organization, Washington, D. C., 20037IF.E. V.],and Institute of Nutrition of Central America and Panama, Guatemala City, Guatemala [E.A.] Summary The objective of the present report is to demonstrate the use of receiver-operator characteristics (ROC) analysis in the selection of diagnostic tests for iron deficiency in a specific population. Conventional ROC curves were prepared with true positive fraction (TPF) and false positive fraction (FPF) determined by the application of different cut-off points for four indicators of iron status. ROC plots were then transformed into normal deviate scales. The advantages of Gaussian transformation of TPF and FPF when underlying decision functions are normally distributed are: (i) the ROC curve is a straight line; and (ii) the separation between the two distributions and shape of these distributions can be simply quantitated as intercepts and slopes. In the present study, pretreatment hemoglobin concentration was the most robust diagnostic indicator of iron deficiency as operationally defined by a response of hemoglobin to iron treatment. Free erythrocyte protoporphyrin was a more sensitive and specific predictor than either serum ferritin or transferin saturation when a stringent o~erationaldefinition of iron deficiencv was used. These findings illustrate the utility of ROC analysis in discriminating between diagnostic indicators having different degrees of accuracy. Abbreviations FN, false negative FP, false positive Hb, hemoglobin N, normal individuals D, diseased cases ROC, receiver-operator characteristics TPF, true positive fraction Received March 30. 1983: accepted February 8. 1984. Address reprint requests to: Ernesto Pollitt. Ph.D.. The University of Texas, School of Public Health. PO Box 20186. Houston. TX 77225. This study was partially supported by National Institutes of Health Grant ROIHD 12843. FPF, false ~ositivefraction FEP, free erythrocyte protoporphyrin SF, serum ferritin TS, transferrin saturation A wide range of laboratory tests is currently used to assess systemic iron status in man. However, normal biological variability, measurement error, and confounding factors such as intercurrent infection may adversely affect the diagnostic efficiency of these tests. Some of these problems are minimized when iron status is operationally defined by the degree of hematologic response to iron administration. In assessing an individual's iron status, a significant rise in circulating hemoglobin mass in response to iron treatment provides reliable evidence of antecedent iron deficiency. Hemoglobin response can also be used to monitor the diagnostic efficiency of other tests of systemic iron status. The evaluation of a test's diagnostic efficiency requires assessment of its discriminative capacity in circumstances where the frequency and nature of its diagnostic errors can be unequivocally determined. A test's accuracy (ratio of correct decisions to total number of subjects tested) is of limited usefulness as a general index of diagnostic performance because it is strongly affected by disease prevalence (8). If a test is to be used to discriminate iron-replete from irondepleted subjects, some definitive diagnostic criterion is needed to allow evaluation of that test. In k'igure 1, the performance of a hypothetical diagnostic test is examined. Diseased subjects, whose test result places them at the right of the cut-off point, a, will be FNs; normal individuals whose result is to the left of a will be FPs. The number of FPs can be reduced or eliminated by moving the cut-off a toward 6,to the lower end ofthe distribution for N. However. as a result of eliminating FPs, the FN fraction will be increased. Likewise. the number of FNs can be eliminated by moving the cut-off u to c, the upper end of the distribution for D. The cut-off point can be positioned so as to maximize a test's diagnostic performance in a given clinical or epidemiologic context (8) 917 DIAGNOSTIC PERFORMANCE OF IRON INDICATORS Fig. 1. Distribution of diseased (D) and nondiseased (N) subjects in a population. The probability of positive and negative tests is equal to the area under a distribution to either side of the cut-off point. When a cut-off point for a test is set at a, the TPF is equal to the area of D distribution and the FPF is equal to the area of N distribution to the left of a given cut-off point, a. a, b, and c, various cut-off points for a given discriminating test. One important consideration in the performance of tests to screen populations for the presence of disease is the relationship between sensitivity, specificity, and the prevalence of the disease being screened. When the prevalence of a disease is very low (e.g., 1 or 2%). even a highly sensitive and specific test will generate a large number of FPs. A small decrease in test specificity in this context will substantially increase the number of FPs. Unless offset by a large gain in sensitivity, the proportion of FPs will increase or. at best, remain unchanged. The effects on test performance of shifts in cut-off point can be usefully analyzed by examining the ROC of a particular indicator. ROC analysis was developed as part of signal detection theory (7) and has been applied extensively to experimental studies of cognition (10). The ROC curve is a continuous plot of TPF vers1r.s FPF, both (...truncated)