Using contextual design to identify potential innovations for problem based learning.

Using Contextual Design to Identify Potential Innovations for Problem Based Learning Michael Blechner MD1 , Valerie Monaco, PhD, MHCI2,3, Isabella Knox MD, EdM4 and Rebecca S. Crowley, MD, MS3,5 1 Hartford Hospital, Hartford CT; 2 Benedum Oncology Informatics Center, University of Pittsburgh, Pittsburgh PA; 3 Center for Biomedical Informatics, University of Pittsburgh, Pittsburgh PA; 4 University of Connecticut School of Medicine, Farmington CT; and 5 Center for Pathology Informatics, Pittsburgh PA ABSTRA CT We report on the use of Contextual Design (CD) to develop models of the information management, resource integration, and collaborative processes of medical students in problem-based learning groups. CD is a modified ethnographic technique designed to provide a detailed understanding of the user’s needs. Although the technique has been used in nonhealthcare related fields, there is limited published data on the application of CD within healthcare settings. In this pilot study, we evaluated the feasibility of the CD methodology for this domain, developed an initial set of CD models, and formulated a series of design ideas based on the data. The study helps to clarify the effectiveness and feasibility of CD as well as the limitations for using this method in health-related domains. INTRODUCTION Contextual Design (CD)1 is a highly structured method for collecting, interpreting, and aggregating qualitative data about work processes. The methodology is designed specifically for the purpose of creating software that addresses user’s needs. Since its original description by Beyer and Holtzblatt, CD has mainly been used by large software corporations. It has only rarely been applied towards the development of medical information systems 2,3 . In this pilot study, we apply CD to the study of educational work processes in Problem-Based Learning (PBL) - a ubiquitous medical educational setting with a unique set of information needs and limited software designed to support this process. RESEARCH GOALS This pilot study was performed to (1) determine the feasibility, advantages and disadvantages of adapting CD for use in a medical and educational domain, (2) provide an initial model of information seeking and information flow for students engaged in PBL and (3) generate an initial set of design ideas for a collaborative learning environment for PBL. BACKGROUND Problem-based learning. PBL was originally developed at McMaster University by Barrows and colleagues 4-6 , and describes a form of case-based learning in which domain knowledge and problem solving skills are acquired as students work through a specific clinical problem. In PBL, students work in small groups with the guidance of a faculty preceptor. The students are first presented with a clinical problem that they do not possess adequate knowledge to solve. The group relies on their current knowledge to discuss the case and identify learning issues. The group then adjourns to pursue independent study on the selected topics and later reconvenes to discuss their research and apply this new knowledge to solving the case. Students engaged in PBL are typically active information-seekers, utilizing multiple types of sources (printed material, on-line books and websites) and institutional resources (libraries, e-mail) while they integrate information for a specific purpose. Although the PBL method is now incorporated in some form in a majority of medical school curricula, there have been few attempts to design software that supports this collaborative learning process7,8 . Contextual Design. Knowledge of the end-user is a prerequisite to designing successful software. Often, needs-assessment is limited to demographic data and end-user opinions. However, detailed data about actual work processes can provide critical information for developing relevant and useable information systems. CD incorporates traditional ethnographic approaches into a multi-part process that supports data-driven design. Unlike traditional ethnography, extensive training in CD is not required for proficiency. In contrast to the extensive narrative of ethnography, the intermediate deliverables of CD are a set of highly formalized paper models of the work process that can be understood at a glance. The initial aspects of CD involve three steps: data collection, modeling and consolidation. Data collection encompasses observation, interviewing and collection of artifacts. Contextual Inquiry (CI) interviews take place while anticipated users are performing their work. The process consists of watching the individual perform their work and periodically interrupting to discuss and clarify some aspect of work just performed. Typically, prewritten questions are not used. In order to keep the amount of data collected manageable and ensure the data is pertinent, the CD team chooses a focus prior to initiating the interviews. Similar to the focus in ethnography, the CI focus is a small set of themes that determines what aspects of the end-user’s work the interviews will emphasize. Ideally, 10-20 CI AMIA 2003 Symposium Proceedings − Page 91 interviews are performed, with individuals who represent different roles within the work environment. Modeling is the first step in the data analysis process. During a given interpretation session, the CD team works together to develop a set of five paper models as they work through notes, transcripts and/or videotape from a single CI interview. These paper models are formalized diagrams that depict a global picture of the work process. The five models are the flow, sequence, cultural, artifact and physical models. The flow model (see figure 2) documents the communication and coordination involved in the work. Individuals and well-defined groups are represented by circles. Information sources and sinks are drawn as rectangles. Labeled arrows between individuals and groups show the directionality and content of information flow. Physical objects passed between parties are also represented and annotated with their purpose. The sequence model depicts the steps used to achieve the individual’s work. Sequences within the model are annotated with the intent of the sequence as well as the trigger that initiated the sequence. The sequence model reveals the individual’s strategy and intent. The artifact model consists of annotated photocopies or drawings of objects used in the work process, and could include paper documents, or screen captures of software used. The model reveals the structure, usage and intent of an object manipulated in the sequence model or passed in the flow model. The cultural model provides a representation of the cultural constraints placed on the worker. Individuals, groups or entire organizations are depicted as overlapping circles connected by appropriately labeled arrows representing the influence of one on the other. Constraints may be due to policy, personal values, organizational culture or other influences. (...truncated)