Just ask Siri? A pilot study comparing smartphone digital assistants and laptop Google searches for smoking cessation advice

PLOS ONE, Nov 2019

Objective To compare voice-activated internet searches by smartphone (two digital assistants) with laptop ones for information and advice related to smoking cessation. Design Responses to 80 questions on a range of topics related to smoking cessation (including the FAQ from a NHS website), compared for quality. Setting Smartphone and internet searches as performed in New Zealand. Main outcome measures Ranked responses to the questions. Results Google laptop internet searches came first (or first equal) for best quality smoking cessation advice for 83% (66/80) of the responses. Voiced questions to Google Assistant (“OK Google”) came first/first equal 76% of the time vs Siri (Apple) at 28%. Google and Google Assistant were statistically significantly better than Siri searches (odds ratio 12.4 and 8.5 respectively, p<0.0001 in each comparison). When asked FAQs from the National Health Service website, or to find information the Centers for Disease Control has made videos on, the best search results used expert sources 59% (31/52) of the time, “some expertise” (eg, Wikipedia) 18% of the time, but also magazines and other low quality sources 19% of the time. Using all three methods failed to find relevant information 8% (6/80) of the time, with Siri having the most failed responses (53% of the time). Conclusion Google internet searches and Google Assistant were found to be significantly superior to the Siri digital assistant for smoking cessation information. While expert content was returned over half the time, there is still substantial room for improvement in how these software systems deliver smoking cessation advice.

A PDF file should load here. If you do not see its contents the file may be temporarily unavailable at the journal website or you do not have a PDF plug-in installed and enabled in your browser.

Alternatively, you can download the file locally and open with any standalone PDF reader:

https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0194811&type=printable

Just ask Siri? A pilot study comparing smartphone digital assistants and laptop Google searches for smoking cessation advice

March Just ask Siri? A pilot study comparing smartphone digital assistants and laptop Google searches for smoking cessation advice Matt Boyd 0 1 Nick Wilson 0 0 Editor: Albert Lee, The Chinese University of Hong Kong , HONG KONG 1 Adapt Research Ltd , Reefton , New Zealand , 2 Department of Public Health, Univeristy of Otago , Wellington , New Zealand ☯ These authors contributed equally to this work; * matt@adaptresearchwriting; com Main outcome measures Ranked responses to the questions. Google laptop internet searches came first (or first equal) for best quality smoking cessation advice for 83% (66/80) of the responses. Voiced questions to Google Assistant (ªOK Googleº) came first/first equal 76% of the time vs Siri (Apple) at 28%. Google and Google Assistant were statistically significantly better than Siri searches (odds ratio 12.4 and 8.5 respectively, p<0.0001 in each comparison). When asked FAQs from the National Health Service website, or to find information the Centers for Disease Control has made videos on, the best search results used expert sources 59% (31/52) of the time, ªsome expertiseº (eg, Wikipedia) 18% of the time, but also magazines and other low quality sources 19% of the time. Using all three methods failed to find relevant information 8% (6/80) of the time, with Siri having the most failed responses (53% of the time). a1111111111 a1111111111 a1111111111 a1111111111 a1111111111 OPEN ACCESS Data Availability Statement: All raw data is provided in the table in the supplementary file. An Excel file with all the results is available from the authors on request. The data contained in this paper and the Supporting Information file constitutes the minimal underlying dataset. Funding: The study was self-funded by the authors and no funder had any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the 'author contributions' section. Objective Design Setting Results To compare voice-activated internet searches by smartphone (two digital assistants) with laptop ones for information and advice related to smoking cessation. Responses to 80 questions on a range of topics related to smoking cessation (including the FAQ from a NHS website), compared for quality. Competing interests: All authors have completed the Unified Competing Interest form (available on request from the corresponding author) and declare: no support from any organisation for the submitted work; no financial relationships with any organisations that might have an interest in the submitted work in the previous three years; no other relationships or activities that could appear to have influenced the submitted work. MB is the owner and sole employee of Adapt Research Ltd, this does not alter our adherence to PLOS ONE policies on sharing data and materials. Conclusion Google internet searches and Google Assistant were found to be significantly superior to the Siri digital assistant for smoking cessation information. While expert content was returned over half the time, there is still substantial room for improvement in how these software systems deliver smoking cessation advice. Introduction The internet is widely used for obtaining health-related information and advice. For example, in the United Kingdom, 41% of internet users report going online to find information for health-related issues, with about half of these (22% of all users) having done so in the previous week [1]. But many people are also wary of the information they find online and value trusted sources [2]. Improving search engine functionality offers a potential solution. For example, Google is cooperating with Mayo Clinic physicians to curate and check health data that is added to the database it uses for instant search results [3]. Similarly, National Health Service (NHS) England is working with Microsoft and Google to increase the visibility of NHS content online [4]. With increasing smartphone use there is also a particular case for studying health informa tion obtainable with digital assistants on smartphones. Present literature on digital assistant use is very limited [5±7]. and there appears to be no published research on the use of these tools in providing information or advice on smoking cessation. Therefore we aimed to assess the current situation using the digital assistants Siri and Google Assistant (GA) and to compare these with internet searches. Methods Selection of digital assistants Siri (Apple) and GA (Google) were selected because they were in common use as personal digital assistants at the time of the Pilot study in October 2017 [5, 6]. Selection of questions The first set of questions (n = 35) were adapted from the most detailed ªfrequently asked questions (FAQ)º we could identify: that of the UK National Health Service (NHS) smokefree website [8]. The specific questions are listed in S1 Appendix (...truncated)


This is a preview of a remote PDF: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0194811&type=printable

Matt Boyd, Nick Wilson. Just ask Siri? A pilot study comparing smartphone digital assistants and laptop Google searches for smoking cessation advice, PLOS ONE, 2018, Volume 13, Issue 3, DOI: 10.1371/journal.pone.0194811