Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK)

Lisa M. McShane 0 1 2 Douglas G. Altman 0 1 2 Willi Sauerbrei 0 1 2 Sheila E. Taube 0 1 2 Massimo Gion 0 1 2 Gary M. Clark for the Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics 0 1 2 0 Affiliations of authors: Biometric Research Branch (LMM) and Cancer Diagnosis Program (SET), National Cancer Institute , Bethesda , MD; Medi- cal Statistics Group, Cancer Research UK, Center for Statistics in Medicine, Wolfson College , Oxford , UK ( DGA); Institut fuer Medizinische Biometrie und Medizinische Informatik, Universitaetsklinikum Freiburg, Germany (WS); Centro Regionale Indicatori Biochimici di Tumore, Ospedale Civile , Venezia , Italy ( MG); OSI Pharmaceuticals, Inc. , Boulder, CO (GMC). metric Research Branch, DCTD, Rm. 8126, Executive Plaza North, MSC 7434, 6130 Executive Blvd., Bethesda, MD 20892-7434 ( 1 Journal of the National Cancer Institute , Vol. 97, No. 16, August 17, 2005 2 We are grateful to the U.S. National Cancer Institute and the European Organi- zation for Research and Treatment of Cancer for their support of the NCI-EORTC International Meetings on Cancer Diagnostics, from which the idea for these guidelines originated. We thank the U.K. National Translational Cancer Research Network for financial support provided to D. G. Altman. Members of the Statistics Subcommittee of the NCI-EORTC Working Group on Cancer Diagnostics are: Douglas G. Altman , DSc (Co-chair) , Medical Statistics Group, Cancer Research UK, Centre for Statistics in Medicine, Wolfson College , Oxford OX2 6UD , UK; Lisa M. McShane, PhD (Co-chair), Biometric Research Branch, US National Cancer Institute , Bethesda, MD 20892; Gary M. Clark, PhD, OSI Pharmaceuticals , Inc. , Boulder, CO 80301; Jose Costa, MD , Yale Cancer Center , New Haven, CT 06510-3202; Angelo Di Leo, MD, PhD , Department of Oncology, Hospital of Prato , 59100 Prato , Italy; Massimo Gion, MD , Centro Regionale Indica- tori Biochimici di Tumore, Ospedale Civile , 30122 Venezia , Italy; Robert J. Mayer, MD , Dana-Farber Cancer Institute , Boston, MA 02115; Willi Sauerbrei, PhD , Insti- tut fuer Medizinische Biometrie und Medizinische Informatik, Universitaetsklinikum Freiburg , 79104 Freiburg, Germany; and Sheila E. Taube, PhD , Cancer Diagnosis Program, US National Cancer Institute , Bethesda, MD 20892. Manuscript received December 28, 2004; revised June 14, 2005; accepted July 21, 2005 Despite years of research and hundreds of reports on tumor markers in oncology, the number of markers that have emerged as clinically useful is pitifully small. Often, initially reported studies of a marker show great promise, but subsequent studies on the same or related markers yield inconsistent conclusions or stand in direct contradiction to the promising results. It is imperative that we attempt to understand the reasons that multiple studies of the same marker lead to differing conclusions. A variety of methodologic problems have been cited to explain these discrepancies. Unfortunately, many tumor marker studies have not been reported in a rigorous fashion, and published articles often lack sufficient information to allow adequate assessment of the quality of the study or the generalizability of study results. The development of guidelines for the reporting of tumor marker studies was a major recommendation of the National Cancer InstituteEuropean Organisation for Research and Treatment of Cancer (NCI-EORTC) First International Meeting on Cancer Diagnostics in 2000. As for the successful CONSORT initiative for randomized trials and for the STARD statement for diagnostic studies, we suggest guidelines to provide relevant information about the study design, preplanned hypotheses, patient and specimen characteristics, assay methods, and statistical analysis methods. In addition, the guidelines suggest helpful presentations of data and important elements to include in discussions. The goal of these guidelines is to encourage transparent and complete reporting so that the relevant information will be available to others to help them to judge the usefulness of the data and understand the context in which the conclusions apply. [J Natl Cancer Inst 2005;97:1180-4] Despite years of research and hundreds of reports on tumor markers in oncology, the number of markers that have emerged as clinically useful is pitifully small (1-3). Often, initially reported studies of a marker show great promise, but subsequent studies on the same or related markers yield inconsistent conclusions or stand in direct contradiction to the promising results. It is imperative that we attempt to understand the reasons that multiple studies of the same marker lead to differing conclusions. A variety of problems have been cited to explain these discrepancies, such as general methodologic differences, poor study design, assays that are not standardized or lack reproducibility, and inappropriate or misleading statistical analyses that are often based on sample sizes too small to draw meaningful conclusions (4-11). For example, in retrospective studies, patient populations are often biased toward patients with available tumor specimens. - Specimen availability may be related to tumor size and patient outcome (12), and the quantity, quality, and preservation method of the specimen may affect feasibility of conducting certain assays. There can also be biases or large variability inherent in the assay results, depending on the particular assay methods used (1317). Statistical problems are commonplace. These problems include underpowered studies or overly optimistic reporting of effect sizes and significance levels due to multiple testing, subset analyses, and cutpoint optimization (18). Unfortunately, many tumor marker studies have not been reported in a rigorous fashion, and published articles often lack sufficient information to allow adequate assessment of the quality of the study or the generalizability of study results. Such reporting deficiencies are increasingly being highlighted by systematic reviews of the published literature on particular markers or cancers (1925). The development of guidelines for the reporting of tumor marker studies was a major recommendation of the National Cancer InstituteEuropean Organisation for Research and Treatment of Cancer (NCI-EORTC) First International Meeting on Cancer Diagnostics (From Discovery to Clinical Practice: Diagnostic Innovation, Implementation, and Evaluation) that was convened in Nyborg, Denmark, in July 2000. The purpose of the meeting was to discuss issues, accomplishments, and barriers in the field of cancer diagnostics. Poor study design and analysis, assay variability, and inadequate reporting of studies were identified as some of the major barriers to progress in this field. One of the working groups formed at the Nyborg meeting was charged with addressing statistical issues of poor design and analysis and with reporting of tumor marker prognostic studies. The guidelines that we pres (...truncated)