Just ask Siri? A pilot study comparing smartphone digital assistants and laptop Google searches for smoking cessation advice
March
Just ask Siri? A pilot study comparing smartphone digital assistants and laptop Google searches for smoking cessation advice
Matt Boyd 0 1
Nick Wilson 0
0 Editor: Albert Lee, The Chinese University of Hong Kong , HONG KONG
1 Adapt Research Ltd , Reefton , New Zealand , 2 Department of Public Health, Univeristy of Otago , Wellington , New Zealand
☯ These authors contributed equally to this work; * matt@adaptresearchwriting; com
Main outcome measures
Ranked responses to the questions.
Google laptop internet searches came first (or first equal) for best quality smoking cessation
advice for 83% (66/80) of the responses. Voiced questions to Google Assistant (ªOK
Googleº) came first/first equal 76% of the time vs Siri (Apple) at 28%. Google and Google
Assistant were statistically significantly better than Siri searches (odds ratio 12.4 and 8.5
respectively, p<0.0001 in each comparison). When asked FAQs from the National Health
Service website, or to find information the Centers for Disease Control has made videos on,
the best search results used expert sources 59% (31/52) of the time, ªsome expertiseº (eg,
Wikipedia) 18% of the time, but also magazines and other low quality sources 19% of the
time. Using all three methods failed to find relevant information 8% (6/80) of the time, with
Siri having the most failed responses (53% of the time).
a1111111111
a1111111111
a1111111111
a1111111111
a1111111111
OPEN ACCESS
Data Availability Statement: All raw data is
provided in the table in the supplementary file. An
Excel file with all the results is available from the
authors on request. The data contained in this
paper and the Supporting Information file
constitutes the minimal underlying dataset.
Funding: The study was self-funded by the authors
and no funder had any additional role in the study
design, data collection and analysis, decision to
publish, or preparation of the manuscript. The
specific roles of these authors are articulated in the
'author contributions' section.
Objective
Design
Setting
Results
To compare voice-activated internet searches by smartphone (two digital assistants) with
laptop ones for information and advice related to smoking cessation.
Responses to 80 questions on a range of topics related to smoking cessation (including the
FAQ from a NHS website), compared for quality.
Competing interests: All authors have completed
the Unified Competing Interest form (available on
request from the corresponding author) and
declare: no support from any organisation for the
submitted work; no financial relationships with any
organisations that might have an interest in the
submitted work in the previous three years; no
other relationships or activities that could appear to
have influenced the submitted work. MB is the
owner and sole employee of Adapt Research Ltd,
this does not alter our adherence to PLOS ONE
policies on sharing data and materials.
Conclusion
Google internet searches and Google Assistant were found to be significantly superior to
the Siri digital assistant for smoking cessation information. While expert content was
returned over half the time, there is still substantial room for improvement in how these
software systems deliver smoking cessation advice.
Introduction
The internet is widely used for obtaining health-related information and advice. For example,
in the United Kingdom, 41% of internet users report going online to find information for
health-related issues, with about half of these (22% of all users) having done so in the previous
week [1]. But many people are also wary of the information they find online and value trusted
sources [2]. Improving search engine functionality offers a potential solution. For example,
Google is cooperating with Mayo Clinic physicians to curate and check health data that is added to the database it uses for instant search results [3]. Similarly, National Health Service (NHS) England is working with Microsoft and Google to increase the visibility of NHS content online [4].
With increasing smartphone use there is also a particular case for studying health informa
tion obtainable with digital assistants on smartphones. Present literature on digital assistant
use is very limited [5±7]. and there appears to be no published research on the use of these
tools in providing information or advice on smoking cessation. Therefore we aimed to assess
the current situation using the digital assistants Siri and Google Assistant (GA) and to compare
these with internet searches.
Methods
Selection of digital assistants
Siri (Apple) and GA (Google) were selected because they were in common use as personal
digital assistants at the time of the Pilot study in October 2017 [5, 6].
Selection of questions
The first set of questions (n = 35) were adapted from the most detailed ªfrequently asked
questions (FAQ)º we could identify: that of the UK National Health Service (NHS) smokefree
website [8]. The specific questions are listed in S1 Appendix (...truncated)