Privacy and Accountability in Black-Box Medicine

Michigan Telecommunications and Technology Law Review Volume 23 | Issue 1 2016 Privacy and Accountability in Black-Box Medicine Roger Allan Ford University of New Hampshire School of Law W. Nicholson Price II University of Michigan Law School, Follow this and additional works at: http://repository.law.umich.edu/mttlr Part of the Health Law and Policy Commons, Privacy Law Commons, and the Science and Technology Law Commons Recommended Citation Roger A. Ford & W. Nicholson Price II, Privacy and Accountability in Black-Box Medicine, 23 MICH. TELECOMM. & TECH. L. REV. 1 (2016). This Article is brought to you for free and open access by the Journals at University of Michigan Law School Scholarship Repository. It has been accepted for inclusion in Michigan Telecommunications and Technology Law Review by an authorized editor of University of Michigan Law School Scholarship Repository. For more information, please contact . PRIVACY AND ACCOUNTABILITY IN BLACK-BOX MEDICINE Roger Allan Ford† and W. Nicholson Price II‡ Cite as: Roger Allan Ford & W. Nicholson Price II, Privacy and Accountability in Black-Box Medicine, 23 MICH. TELECOM. & TECH. L. REV. 1 (2016). This manuscript may be accessed online at repository.law.umich.edu. Black-box medicine—the use of big data and sophisticated machinelearning techniques for health-care applications—could be the future of personalized medicine. Black-box medicine promises to make it easier to diagnose rare diseases and conditions, identify the most promising treatments, and allocate scarce resources among different patients. But to succeed, it must overcome two separate, but related, problems: patient privacy and algorithmic accountability. Privacy is a problem because researchers need access to huge amounts of patient health information to generate useful medical predictions. And accountability is a problem because black-box algorithms must be verified by outsiders to ensure they are accurate and unbiased, but this means giving outsiders access to this health information. This article examines the tension between the twin goals of privacy and accountability and develops a framework for balancing that tension. It proposes three pillars for an effective system of privacy-preserving accountability: substantive limitations on the collection, use, and disclosure of patient information; independent gatekeepers regulating information sharing between those developing and verifying black-box algorithms; and information-security requirements to prevent uninten† Associate Professor of Law, University of New Hampshire School of Law; Faculty Fellow, Franklin Pierce Center for Intellectual Property. ‡ Assistant Professor of Law, University of Michigan Law School; Affiliated Faculty, Petrie-Flom Center for Health Law Policy, Biotechnology, and Bioethics, Harvard Law School. For helpful comments and conversations, we are indebted to Andrew Selbst, Anna Slomovic, Bob Gellman, Christo Wilson, Christopher Millard, Deborah Hurley, Dissent Doe, Felix Wu, Frank Pasquale, Hank Greely, Jacob Sherkow, Janine Hiller, Jay Kesan, Jennifer Berk, Margot Kaminski, Mark Lemley, Maya Bernstein, Melissa Goldstein, and Rebecca Eisenberg, and to participants at the Yale Information Society Project’s Conference on Unlocking the Black Box, the Stanford Center for Law and the Biosciences Workshop, the thirteenth Works in Progress in Intellectual Property (WIPIP) Colloquium at the University of Washington, and the ninth Privacy Law Scholars Conference at George Washington University. Cassandra Simmons provided excellent research assistance. Copyright © 2016 Roger Allan Ford and W. Nicholson Price II. After June 2017, this article is available for reuse under the Creative Commons Attribution 4.0 International license, http://creativecommons.org/licenses/by/4.0/. 1 2 Michigan Telecommunications and Technology Law Review [Vol. 23:1 tional disclosures of patient information. The article examines and draws on a similar debate in the field of clinical trials, where disclosing information from past trials can lead to new treatments but also threatens patient privacy. INTRODUCTION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . I. BLACK-BOX MEDICINE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. The Promise of Black-Box Medicine . . . . . . . . . . . . . . . . . . B. The Genesis of Black-Box Medicine . . . . . . . . . . . . . . . . . . . II. THE ACCOUNTABILITY CHALLENGE . . . . . . . . . . . . . . . . . . . . . . . . A. The Need for Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . . B. Verification by Clinical Trials . . . . . . . . . . . . . . . . . . . . . . . . C. Computational Verification . . . . . . . . . . . . . . . . . . . . . . . . . . . III. THE PRIVACY CHALLENGE . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . A. Health Information and Patient Privacy . . . . . . . . . . . . . . . B. The Privacy Challenge of Black-Box Medicine . . . . . . . . . C. Privacy Harms from Black-Box Medicine . . . . . . . . . . . . . IV. RECONCILING PRIVACY AND ACCOUNTABILITY . . . . . . . . . . . . . A. Patient Privacy Versus Algorithmic Accountability . . . . . B. Three Pillars for Privacy-Preserving Accountability . . . . C. Case Study: Data and the Clinical-Trial Debate . . . . . . . CONCLUSION . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 4 5 7 12 12 15 18 21 21 24 26 29 29 31 39 42 INTRODUCTION Medicine is an unpredictable science. A treatment that provides a miraculous recovery for one patient may do nothing for the next. A new chemotherapy drug may extend patient lives by two years on average, but that average consists of some patients who live many years longer and some patients whose lives are not extended at all, or even are shortened. And with new drugs costing more and more money, personalizing medicine is increasingly important, so that doctors can predict disease risk and choose treatments tailored to individual patients. Medicine’s unpredictability has a simple cause. The human body is one of the most complex systems in existence, with endless genetic variations, biological pathways, protein expression patterns, metabolite concentrations, and exercise patterns (to name just a few of the dozens of variables) affecting each person differently. And only a few of these variables are wellunderstood by scientists. When a drug doesn’t work or a patient develops a rare disease, the reason could be some genetic variation or metabolite concentration or environmental difference—or several of these variables acting together in ways doctors will likely never understand. Black-box medicine—the use of big data and sophisticated machinelearning techniques in opaque medical applications—could be the answer.1 1. See, e.g., W. Nicholson Price II, Black-Box Medicine, 28 HARV. J.L. & TECH. 419 (2015) (defining black-bo (...truncated)