Social reference managers and their users: A survey of demographics and ideologies
Social reference managers and their users: A survey of demographics and ideologies
Pei-Ying Chen 0 1
Erica Hayes 1
Vincent Larivière 1
Cassidy R. Sugimoto 1
0 Department of Information and Library Science, School of Informatics, Computing, and Engineering, Indiana University , Bloomington , Indiana, United States of America, 2 Carolina State University Libraries, Raleigh, North Carolina, United States of America, 3 EÂcole de biblioth eÂconomie et des sciences de l'information, Universit eÂ de Montr eÂal , MontreÂ al, QueÂ bec , Canada , 4 Observatoire des sciences et des technologies (OST), Centre interuniversitaire de recherche sur la science et la technologie (CIRST), Universit eÂ du QueÂ bec à MontreÂ al , MontreÂ al, Qu eÂbec , Canada , 5 Department of Informatics, School of Informatics, Computing, and Engineering, Indiana University , Bloomington, Indiana , United States of America
1 Editor: Wolfgang Glanzel, KU Leuven , BELGIUM
Altmetric indicators are increasingly present in the research landscape. Among this ecosystem of heterogeneous indicators, social reference managers have been proposed as indicators of broader use of scholarly work. However, little work has been done to understand the data underlying this indicator. The present work uses a large-scale survey to study the users of two prominent social reference managersÐMendeley and Zotero. The survey examines demographic characteristics, usage of the platforms, as well as attitudes towards key issues in scholarly communication, such as open access, peer review, privacy, and the reward system of science. Results show strong differences between platforms: Mendeley users are younger and more gender-balanced; Zotero users are more engaged in social media and more likely to come from the social sciences and humanities. Zotero users are more likely to use the platform's search functions and to organize their libraries, while Mendeley users are more likely to take advantage of some of the discovery and networking featuresÐsuch as browsing papers and groups and connecting with other users. We discuss the implications of using metrics derived from these platforms as impact indicators.
Data Availability Statement: The Mendeley/Zotero
survey data files are publicly available at http://hdl.
handle.net/2022/22043 from IUScholarWorks, an
open access institutional repository service
provided by the Indiana University Libraries for
disseminating and preserving the intellectual
output of Indiana University scholars.
Funding: This work was supported by the Alfred P.
Sloan Foundation Grant #G2014325 to VL and the
Institute of Museum and Library Services Grant #
RE0214002314 to CRS. The funders had no role in
study design, data collection and analysis, decision
to publish, or preparation of the manuscript.
Introduction and background
The scholarly communication system has been transformed over the past two decades by the
digital age, Web 2.0 applications, and open scholarship initiatives. In recent years, social media
platforms have significantly increased the online visibility of scholarly products: with more
than 4.5 million researchers on ResearchGate and 11 million users on Academia.edu, there is
considerable evidence of the increased usage of social media platforms for the creation,
evaluation, and dissemination of scholarship . In parallel, demands by research funders and
administrators for indicators of societal impact [2±4] have generated wide interest among
scholars in evaluating how social media data may be used as a means for measuring scholarly
impact beyond the academy .
Citations have historically served as the dominant measurement for the impact of scholarly
publications . Citation counts, however, serve as indicators of papers' impact on other
scholars, and are not indicative of their impact on the broader public . New sets of metrics,
grouped under the umbrella of ªaltmetricsº, propose new indicators of scholarly impact based
on interactions on social media platforms . Introduced in 2010 by Jason Priem and his
colleagues, ªaltmetricsº claims to ªexpand our view of what impact looks likeº  by identifying
online communities and tracking their online engagement with scholarly content [8,9]. Often
considered as a subset of webometrics due to its reliance on open application program
interface (API) usage data, altmetrics has sought to move beyond the traditional citation
measurement of journal article by tracking online social media activities [
]. These activities
include the appraisal of scholarly articles through the act of mentioning them online; using
these articles to create new scholarly objects; or accessing, curating, and saving scholarly
objects online . There is wide variety in the tools that are used to perform each of these
actions; however, one of the most commonly used online tools to access, save, and curate
documents are social reference managers.
Social reference managers were created to assist researchers in the referencing process
when writing research articles and formatting citationsÐfunctions that ªtraditionalº reference
managers such as Endnote performed and to which ªsocialº functions were added. Social
reference managers allow users to save, bookmark, download, and share research articles with
other scholars that may be of interest or use to them. The two most popular social reference
managers are Zotero and Mendeley. Both platforms provide users with tools to leave
comments, rate papers, cite entries, share work, create subject tags, and join built-in social
networking communities among scholars of varying disciplines . Referred to as academic
social networking services , these social reference management systems provide a means
for scholars to interact and share academic documents with other researchers.
Data from these applications can also be used to reveal readership behavior and create
indicators of documents' online use. Mendeley is particularly useful in this regard as it provides
anonymized usage data for reportedly 2.5 million registered users via an open API .
Founded in August 2008, Mendeley began as a free software, but was purchased by Elsevier for
$76 million in April 2013 . Although negative responses from the research community
followed Elsevier's purchase of Mendeley among open access advocates, Mendeley has argued
that its mission to facilitate data sharing and make scientific research more openly available
has remained much the same since its conception . Mendeley currently provides a desktop
and web platform with 2GB of free web storage space for the management, saving,
bookmarking, and sharing of research papers between scholars on the web. The main features include
automatic extraction of metadata, smart-filtering, tagging, annotating, and full-text search.
Mendeley also offers various bibliographic file formats for exporting scholarly materials,
including EndNote XML, BibTex, and RIS . In addition to reference management,
Mendeley also features discovery of new references via ªLiterature Searchº or ªMendeley Suggestº
and research network via creating or joining groups .
Zotero was released in 2006 by the Center for History and New Media at George Mason
University, which continues to maintain it as an open-source software platform. Zotero
provides 300 MB of free cloud storage for users to store and save research articles. Available since
the latest release as a standalone desktop application for users of all browsers , Zotero
automatically extracts and saves bibliographic references when viewing a book, article, or other
scholarly object via browser extensions . Offering similar features to Mendeley, Zotero
provides users with automatic extraction of metadata and full-text search, which can be
organized by creating collections or assigning tags . Users may also join private or public
closed/open membership groups to collaborate, discuss, and share research online .
Scholarly materials may be exported in common bibliographic data formats, including BibTex,
BibLateX, RefWorks, and RIS . Since Zotero is open source, users contribute new ideas and
2 / 22
improvements to the software on an ongoing basis. The forums on the Zotero website are
frequently updated regarding the most recent changes or improvements to the software with the
latest 5.0 version being released in July 2017 .
The size of the user base for Zotero was estimated at around 620,000 in 2011 . However,
interpretation of this figure is complicated by the fact that Zotero never required users to
register their accounts before downloading the software, making it impossible to know whether
each download is a unique or duplicate user . Furthermore, Zotero does not provide an
API to download data and estimate user accounts from the existing metadata. As a result of
these limitations, no large-scale study of Zotero users has been conducted.
Mendeley API provides users' disciplines and academic status/occupations for each article.
Using these data, previous research investigated the disciplinary and occupational
distributions of Mendeley users and their patterns of information behavior. It showed that the
majority of Mendeley users are doctoral students, followed by master's students and postdocs
[23,24]. Correlations between Mendeley readership counts and citations for the same articles
varies by occupation: correlations are highest when citations are compared with Mendeley
reader counts from research-oriented occupations and lowest when limited to those of
librarians and students . These variations suggest that Mendeley readers reveal important
indicators of use by professionals and students that are not captured by citations. This is reinforced
by user surveys, which demonstrated that Mendeley users primarily use the platform to
support professional and educational activities . The heterogeneity of users in terms of
disciplinary and occupational composition explains the discrepancies observed when examining
the relationship between Mendeley reader counts and other impact indicators .
Mendeley provides full access for document-based analysis, which has led to several
coverage studies, i.e., what proportion of sets of documents are indexed in the database. These
studies have shown that the coverage of articles in Mendeley is generally higher than in other social
platforms, such as CiteULike [27,28] and Twitter , albeit with some variations by research
field and publication venue. For example, rates of coverage tend to be lowest in the humanities
 and highest in biomedicine , with certain subfields, such as bibliometrics , having
high coverage, as well as specific venues, including Public of Library Science (PLoS) ,
Science, Nature , and Journal of the American Society for Information Science and Technology
(JASIST) . The moderate positive correlations (.45-.50) between Mendeley readership
counts and citation counts across disciplines [24,28,30,33,34] suggest that Mendeley
readership counts reflect related but non-identical indicators of scholarly impact. Until recently, no
API allowed for similar coverage analyses of Zotero.
Despite the increasing use of readership counts as indicators of papers' usageÐand their
integration into aggregated altmetric indicators, such as the Altmetric Attention Score
calculated by the company Altmetric, little is known on the processes and individuals that generate
these traces of readership. In order to meaningfully interpret such indicators, we must better
understand the motivations underlying the interactions as well as the characteristics of those
making the interactions. Generalizations about the value of articles are increasingly made on
the basis of these counts. Yet, without a proper understanding of the characteristics of the
population underlying the metrics, it is difficult to understand on whom the impact was made. To
address this, we conducted a survey of Zotero and Mendeley users to understand not only
their demographic characteristics and how they utilize the platforms, but also on their attitudes
towards key issues in scholarly communication, such as open access, peer review, privacy, and
the reward system of science. The results of this study inform our understanding of these
platforms and the degree to which interactions on these platforms can be used to measure broader
3 / 22
This study has been approved by the Institutional Review Board at Indiana University. (IRB
Two online surveys were designed specifically for Mendeley and Zotero users, each composed
of 22 question stems in three parts, including demographic information, user behavior, and
meaning and motivation, with a study and consent information sheet provided in the beginning.
Socio-demographic information collected included gender, age, current place of residence, level
of education, occupation/status, and affiliated disciplines. Questions regarding user behavior
ranged from the length of use; pattern of use over time; platform primarily used; group
membership; to frequency of performing different kinds of tasks including discoverability, adding and
organizing, re-using, and networking. Meaning and motivation questions included rating the
importance of social reference managers in fulfilling different tasks/activities to level of
engagement in reading, citation, curation, and sociability, as well as their ideological positions toward
scholarly communication. Answers to the questions regarding level of importance were marked
on a Likert scale from 1 (not at all important) to 5 (extremely important). Likewise, answers
regarding level of agreement are marked from 1 (strongly disagree) to 5 (strongly agree).
For the purpose of comparison, the wording and the order of the questions for the two
surveys were held constant except when the names of the respective social reference managers
and terminology exclusively used by each were mentioned, e.g., platforms (Mendeley: online/
desktop vs Zotero: standalone/for Firefox) as well as the unit of readership counts (Mendeley
readership vs Zotero library). Moreover, as Zotero was released earlier than Mendeley, Zotero
survey has an extra option of ªmore than seven yearsº in the question about the length of use.
The surveys were hosted on Qualtrics and distributed from mid-April to early June 2016. Due
to different modes of cooperation and levels of concern over privacy, the two surveys were
distributed in different ways. A representative sample of more than 26,000 user emails were
provided by Mendeley to send invitations and reminders with the unique survey link embedded,
as well as to trace the response rate. On the other hand, the anonymous link to the Zotero
survey was disseminated by Zotero through various channels including library-related listservs
(between April 14±18), the Zotero blog (April 13), Twitter (April 18, 22, 29 and May 6, 13, 17),
LinkedIn, Trellis, as well as the Open Scholarship Initiative 2016 Conference (April 19±22). In
total, 1214 responses from Mendeley users and 796 from Zotero users were received, of which
767 and 529 (respectively) were complete cases. The response rate for the Mendeley survey is
4.61%. No response rate could be calculated for Zotero.
While the low response rate and the use of convenient sample may raise some
methodological concern, several studies have pointed out that low response rates are not necessarily
correlated with nonresponse error . A comparison of Mendeley user profiles on occupation and
discipline with previous survey studies [12,25] and studies based on Mendeley readership
statistics gathered through API [23,36] showed that the respondents in our study are well aligned
with the disciplinary mapping of users in the Mendeley directory released in 2013, with an
exception of an underrepresentation of basic science. As to occupation, despite some variations
in the proportion of each category due to research designs and methodological limitations,
previous studies generally agreed, in consistence with ours, that Mendeley is predominantly
used by student, researcher, and professor (see S1 Table). The use of convenience sample for
Zotero users provides an initial, exploratory perspective on Zotero users.
4 / 22
The two separate raw datasets were downloaded from Qualtrics in comma-separated values
(CSV) file format, then read and combined in RStudio, with a new variable ªplatformº to
record the platforms from which the responses were collected. In addition to performing
Chisquare tests of independence on all the nominal/ordinal questions, logistic regression was also
applied to questions regarding motivation to use, level of engagement, and ideological
positions to further examine the effects of other socio-demographic variables. To do this
calculation, 5-point Likert scales were converted into a binary variable by assigning 1 to the original
value 4's (agree/important) and 5's (strongly agree/extremely important) and 0 to the value 1's
(strongly disagree/extremely unimportant) and 2's (disagree/unimportant) while dropping the
value 3's (neither agree/important nor disagree/unimportant). This is a limitation of the study,
as 3 is a legitimate response and dropping these responses lowered the amount of responses
which could be analyzed. However, the decision was made to render the results more
comparable by highlighting stances closer to both ends of the spectrum. The gender variable was
recoded into ªmale,º ªfemale,º and ªotherº with the original ªotherº and ªprefer not to answerº
categories being combined into ªotherº due to the limited cases in the two categories. The full
model includes platform, level of education, affiliated discipline, occupation, gender, and age
as controlled variables to test the respective explanatory power.
Socio-demographic characteristics of participants
Mendeley reported roughly equal rates in terms of gender; whereas Zotero was much more
male-skewed (66% male) (Table 1). The gender difference between the two platforms is
statistically significant (X2 = 44.37, p < .001). Both the majority of Mendeley and Zotero users are in
their mid- to late-thirties, with an average age of 36.28 years for the former and 38.31 years for
the latter, and the difference in age is statistically significant (t = -3.43, p < .001). One plausible
explanation for the age gap is that Zotero was released two years earlier than Mendeley, which
also suggests that users are aging with the platform.
About 90% of the respondents hold a master's degree or above, indicating that they are
more educated than the general public, with Zotero users being marginally more well-educated
(X2 = 11.46, p < .01). In line with their advanced education level is the occupation or status
to which they belong. Students, researchers, and professors are overall the three largest user
groups across two surveys, but the composition of occupations between Mendeley and Zotero
is significantly different (X2 = 69.14, p < .001). In particular, more than 25% of Zotero users
are practitioners, defined as those work in professional fields such as librarianship, nursing, or
medicine that typically require a professional degree or an advanced certificate, in contrast
with less than 10% of their Mendeley counterparts. There is also a significant difference
regarding users' affiliated disciplines (X2 = 164.35, p < .001). While the life sciences, social
sciences, and engineering are the three disciplines that Mendeley users are mostly affiliated
with, Zotero users are mostly affiliated with arts and humanities, followed by social sciences
and life sciences. Also noted is that 16% of Zotero users reported to be affiliated with ªotherº
disciplines, of which 70% were shown to be in library and information science after manual
More than 60% of Mendeley users have been using the product for 1±4 year, and 90% less than
four years. In contrast, Zotero respondents have, on average, a longer history of using the
5 / 22
product, with nearly 55% using it for more than four years (S2 Table). Although the differences
could be largely attributed to the longer history of Zotero in the market, it is worth noting that
21% of Zotero users have been using it for almost the same length of time since the release of
Zotero, indicating either their loyalty to the product, or their early adoption of new technologies.
The patterns of use over time are significantly different between Mendeley and Zotero users (X2 =
38.67, p < .001) (S3 Table). While 70% of Zotero users report either a constantly increased or
relatively constant use over time, 40% of Mendeley users report that their use fluctuates.
Although cooperating with others is one of the features distinguishing Mendeley and
Zotero from other reference managers, nearly half of the Mendeley users and more than one
6 / 22
third of Zotero users join neither public nor private groups (S4 Table). The difference is
statistically significant regarding the group membership between Mendeley and Zotero users (X2 =
35.673, p < .001) despite the fact that both are more likely to join private than public groups.
Also noted is that 21% of Zotero users are part of both private and public groups, compared to
11% of their Mendeley counterparts.
In addition to Mendeley and Zotero, respondents reported the use of diverse tools for
professional purposes. Publication and citation profiles (e.g., Google Scholar, ORCID),
non-academic (e.g., Facebook, LinkedIn) and academic social networking sites (e.g., ResearchGate,
Academia.edu), and to some extent, Wikis (e.g., Wikipedia), have been used extensively by
both Mendeley and Zotero users (Fig 1; S5 Table). However, the two groups differ in the use of
publication (e.g., arXiv, SSRN, institutional repository) as well as data and code repositories
(e.g., GitHub, Figshare, SlideShare), with the percentages of reported use among Zotero users
twice as much as those of Mendeley users. Moreover, Zotero users are much more actively
engaged in blogging and microblogging activities. Finally, compared with tools mentioned
above, social recommending, rating, and reviewing (e.g., F1000Prime, Reddit) are the least
utilized by both groups, probably indicating their relatively short history as a new form of
Overall, Zotero users performed discoverability tasks more often than Mendeley users, with
more than 60% of them, compared to less than 40% of Mendeley users, using the search
function at least weekly. Other discoverability features, however, were much less utilized: 54% of
Mendeley and 36% of Zotero users reported that they never search via tags or groups;
moreover, 53% of Mendeley and 77% of Zotero users reported that they never browse the most read
papers (Fig 2; S6 Table).
Both groups performed adding and organizing tasks quite frequently, but this was more
prominent for Zotero users. Almost two third of Zotero users and 40% of Mendeley users
Fig 1. Tools for professional purposes (i.e., those associated with your vocation rather than for personal use).
7 / 22
Fig 2. How often do you do the following discoverability tasks on Mendeley/Zotero?
uploaded publications at least weekly. However, nearly 70% of Mendeley and 50% of Zotero
users never upload other materials such as datasets, figures, or videos. Other organizational
tasks, such as correcting bibliographic information or annotating were done on a monthly
basis by the majority of Mendeley and Zotero users. The two user groups differed mainly in
the frequency in which they deleted items from their libraries: Zotero users were much more
frequently to do so than Mendeley users (Fig 3; S7 Table).
The re-using task was frequently performed by both Mendeley and Zotero users, with two
third of Mendeley and more than 80% of Zotero users doing this on a monthly or more
Fig 3. How often do you do the following adding and organizing tasks on Mendeley/Zotero?
8 / 22
Fig 4. How often do you do the following re-using task on Mendeley?
frequent basis. Zotero users seem to be more dedicated in using social reference managers to
embed references, as more than half of them did such task at least weekly, compared to less
than 40% of Mendeley users. This is in line with the more professional occupation base of
Zotero (Fig 4; S8 Table).
Compared with the other three functionalities, networking tasks were seldom performed by
both groups, with 50±80% of Mendeley users and 50±65% of Zotero users never performed
tasks such as updating profile information, searching for or browsing users/groups, connecting
with others, and asking or answering questions on message boards or forums (Fig 5; S9 Table).
Overall, Zotero users engage in networking activities relatively more frequently than Mendeley
users except for updating profile information.
Motivation to use
Overall, features of discoverability, adding and organizing, and re-using are deemed important
across two user groups in spite of statistically significant differences (p < .001) with Zotero
users being more likely to stress these features as extremely important (Fig 6; S10 Table).
Nonetheless, the networking feature seems negligible for both user groups with no statistically
significant differences found (X2 = 3.80, p = 0.43): 40% of all the respondents rate this as
unimportant to not at all important and 32% as neither important nor unimportant. While
respondents holding a bachelor's degree are more likely to consider discoverability important than
those with a master's degree and professors consider adding and organizing features important
than students, the overall effects of platform, education level, affiliated discipline, occupation,
Fig 5. How often do you do the following networking tasks on Mendeley/Zotero?
9 / 22
Fig 6. Please rate the importance of the following types of features.
gender, and age are not statistically significant across these features except for networking,
where education and occupation do hold certain explanatory power (p < .001) (S11 Table).
On the one hand, professors, lecturers, and practitioners are more likely than students to
consider networking an important feature. On the other hand, compared to respondents with a
master's degree, high school graduates are more likely to consider networking important
whereas those with a doctorate degree are less likely to think so.
Respondents generally consider social reference managers important at various stages of a
research project from starting a research or assignment, collecting and analyzing data, to
writing and preparing the manuscript for submission (Fig 7; S12 Table). The only exception is at
the stage of post-publication, i.e., disseminating the manuscript: 30% of Mendeley users and
nearly half of Zotero users consider the role of social reference manager either unimportant or
not at all important. Overall, Zotero users are more likely than Mendeley users in rating the
social reference manager as important in initiating and performing the project as well as
preparing the manuscript for submission but less likely at the post-publication stage (p < .001)
Other than platform, the overall effects of education, affiliated discipline as well as
occupation are also found to be statistically significant at the stage of performing the project and
disseminating the manuscript (S13 Table). For example, social reference managers are rated as
less important at both stages for respondents with a doctorate degree than with a master's
degree. Compared to respondents affiliated with social sciences, those affiliated with arts and
humanities and with engineering are more likely to consider social reference managers
indispensable in performing the project, whereas those affiliated with psychology are less likely to
think so. Respondents affiliated with engineering and environmental sciences are also more
likely to recognize their significance in post-publication. Moreover, they are more likely to be
10 / 22
Fig 7. How important is Mendeley/Zotero to you at various stages of a research project?
considered important by researchers, practitioners, and those with other occupations than
students in collecting and analyzing data and by lecturers in disseminating their manuscript.
In addition to research-related activities, Mendeley and Zotero users also reported various
activities the social reference managers contribute to, including thesis/dissertation, courses
teaching and assignments, as well as professional practice. Overall, Mendeley is used mainly
for published research and thesis/dissertation whereas the use of Zotero is more evenly
distributed across activities with higher share of users (Fig 8; S14 Table). In particular, Zotero is more
likely than Mendeley to contribute to published research, course teaching, course assignments,
as well as professional and other practices. The difference between the two groups is not
statistically significant in writing thesis or dissertation (S15 Table).
Other than platform, the overall effects of level of education, affiliated discipline,
occupation, and age are significant on the activities of Mendeley and Zotero users (S15 Table).
Compared to those with a master's degree, respondents with a doctorate degree are more likely to
use social reference managers for their published research but less likely for course
assignments, whereas respondents with a bachelor's degree are in the reverse position. Activities
regarding published research, theses/dissertations, and professional practice also differ by
respondents' affiliated disciplines, most notably computer sciences, life sciences, physical
sciences. Compared to those in social sciences, respondents in the aforementioned disciplines are
more likely to use Mendeley/Zotero for published research but less likely for
theses/dissertations. Also worth noting is that social reference managers are more likely to contribute to
professional activities for respondents affiliated with arts and humanities, life sciences, and other
disciplines but less likely with computer sciences. Not surprisingly, non-students are more
likely than students to use them for published research (professor, lecturer, researcher), course
11 / 22
Fig 8. My Mendeley/Zotero library has contributed to my: (select all that apply).
teaching (professor, lecturer, practitioner), and professional practice, but less likely for theses/
dissertations and course assignment (researcher). Lastly, as people get older, they are more
likely to use social reference managers for course teaching and professional practice but less
likely in theses/dissertations and course assignments.
Level of engagement
Mendeley and Zotero users differ in their level of engagement with social reference managers
in terms of reading, citation, curating, and social behaviors. In reflecting upon the last item
added to the library, the differences are statistically significant in terms of reading (X2 = 17.79,
d.f. = 4, p < .01) and citation (X2 = 29.62, d.f. = 3, p < .001). Not only did Mendeley users
report higher rates of having viewed the title and abstract of the item and having read the full
text either partly or entirely (Fig 9; S16 Table), they also report higher rates of both intending
to cite and have cited the item (Fig 10; S17 Table). In contrast, more than 20% of Zotero users
reported only the intent to read and nearly half were not sure if they will cite the item. Overall,
a fifth of users (from both platforms) reported having read the item in full (identical to the
proportion of users who only viewed the title and abstract of the item). This suggests that adding
an item precedes reading. The patterns are consistent with their general reading and citation
behaviors. 60% and 56% of Mendeley users respectively reported having read and cited most
of the items in their library, whereas only 51% and 30% of Zotero users claimed the same
Fig 9. Consider the last item you remember having added to your library. Which of these statements best characterizes your
level of engagement with it?
12 / 22
Fig 10. Consider the last item you remember having added to your library. Which of these statements best characterize your
Selection of items was fairly similar across platforms. Self-saving appears to be common
across platforms, particularly in Zotero, where more than half of respondents indicated that
they add all of their own publications to their library. Nearly 30% of Mendeley users neither
agreed nor disagreed with this statement, possibly because they do not have publications to
add to the library, leading to the ambiguity of the response. The overall quality of the items
was not a deciding factor for either groupÐeither in adding or deleting items from the library.
Of the two, Mendeley users were slightly more concerned with the quality of items that were
added, while over half of Zotero users disagreed that they only add items of high quality. As
implied by the earlier questions, the social aspects of social reference managers are relatively
unimportant, particularly for Zotero users. Users were unconcerned about saving to the library
as a signaling device to the community and were not concerned about constructing a library
for the good of the community. They also tended not to be interested in networking
capabilities. The main value-add for them was uni-directional: allowing them to keep current with
research rather than researchers in the field (S18 Table).
Compared with Mendeley users, Zotero users are not only less likely to have read and cited
most of the items in their libraries but also have lower intention to cite them as well (S19
Table). Although they curate their libraries more often, quality is less of a concern for them in
adding and deleting items (S20 Table). They also engage with Zotero at a more individual
level: adding items is neither a signal to other scholars nor for the good of the community.
Likewise, they are less likely to consider Zotero a networking tool to keep in contact with other
scholars or to keep current on research (S21 Table).
The overall effect of the level of education is not statistically significant on users' level of
engagement except for adding items as signals to other scholars, where respondents holding a
doctorate degree are less likely than those with a master's degree to do so. By contrast,
occupation is a strong predictor of users' level of engagement in reading, citation, and social
behaviors. Compared to students, non-students are less likely to engage in reading (lecturer,
researcher, practitioner) and citation (researcher, practitioner) and have lower intention to
cite as well (professor, researcher, practitioner, none). Nonetheless, they are more likely to take
sociability into account, such as considering adding items either a signal to other scholars
(lecturer) or for community good (lecturer, practitioner) and acknowledging social reference
managers a tool for networking with other researchers (professor, lecturer, practitioner).
The overall effect of discipline is significant in reading, intent of citing, as well as self-saving to
the library. Compared to respondents in social sciences, people in life sciences and physical
sciences are more likely to have read most of the items in their library, whereas those affiliated with
arts and humanities, engineering, life sciences, other or no discipline are less likely to intend to
13 / 22
cite the items in their library. In addition, people in life sciences are more likely to add all of their
publications to the libraryÐthus taking advantage of the dissemination aspect of the platform.
Gender and age also plays a role in users' level of engagement. On the one hand, females are
more likely than males to have read most of the items in the library and have stronger
intention to read and cite as well. Interestingly, they are less likely not only to add all of their
publications to the library but also to add items with the community in mind. On the other hand,
the older the respondents are, the less likely they cite items in the library, but the more they
only add items of high quality. As they grow older, they are more likely to value the social
networking function of the social reference managers, both in keeping contact with other scholars
and maintaining current in research, though only slightly.
Most Mendeley and Zotero users exhibit a high degree of openness, reflected in their support
for open access, open source software, as well as being early adopters of new technology.
About 85% of Mendeley users and 90% of Zotero users consider themselves advocates of
open access and open-source software. Although fewer consider themselves early adopters
of new technology, this still accounts for nearly 70% of Mendeley and 80% of Zotero users
The ideological cleavage between Mendeley and Zotero users is most clearly seen in their
attitudes toward the current journal-based publishing and peer-review system. While 50% of
Mendeley and 40% of Zotero users have no strong opinions on whether the current
peerreview system is broken, more than 50% of Zotero believe it is whereas only 33% of Mendeley
users hold the same opinion. At the same time, 80% of Mendeley and 73% of Zotero users
acknowledge that journals add credibility to their research. This is largely in line with their
agreement that journals are necessary for scholarly communication, accounting for 82% of
Mendeley and 63% of Zotero users. It should be noted, however, that the percentages of Zotero
users in support of these two statements are consistently lower compared to Mendeley users.
The split between ideological perspectives is manifested even more in their views toward the
publisher's profit margin: 46% of Mendeley users remain in the middle ground, whereas 44%
of Zotero users strongly agree that the publisher's profit margin is too high. More broadly
speaking, those who are critical of this phenomenon account for 72% of Zotero users but only
49% of Mendeley users. Consistent with this result, nearly 60% of Mendeley users agree that
publishers are necessary for scholarly communication, whereas only 37% of Zotero users do
(Fig 11; S23 Table).
Among various indicators, citations are still widely recognized as an appropriate indicator
of scholarly impact, as agreed by 79% of Mendeley and 69% of Zotero users. In contrast,
respondents' attitude toward social media activity and altmetrics are more cautious. The
majority of Mendeley users and 40% of Zotero users doubt the utility of social media activity,
such as tweets and Facebook likes, as indicators of scholarly impact (only a quarter of each
support the idea). When asked whether Mendeley readership/Zotero library counts should be
used as an indicator of scholarly impact, 37% of all respondents had no strong opinion,
while only 38% of Mendeley and 28% of Zotero users were supportive. The idea that the social
reference manager-based counts of a document is a good indicator of its value gained similar
yet slightly higher support: nearly 40% of all respondents held no strong opinions, and those
who support the idea account for 41% of Mendeley users and 31% of Zotero users (Fig 12;
One of the features of social reference managers is the ability to create a profile online,
which is assumed to increase the user's visibility in the academic community. However, more
14 / 22
Fig 11. Indicate your level of agreement with the following statementsÐPeer-review system and scholarly publication.
than 40% of Mendeley and Zotero users respectively make no comment on this statement.
Notably, a reverse direction of opinions is perceived between Mendeley and Zotero users,
where 44% of the Zotero users disagree and 28% of Mendeley users agree that having a social
reference manager profile makes them more visible in their field. While it is tempting to
conclude that Zotero users may be less concerned about online visibility, 53% of Zotero users
consider being visible online critical for their scholar identity, which is 8% higher than the share of
Mendeley users. A plausible interpretation is that while respondents care much about their
online visibility as part of their scholar identity, social reference manager profiles may not be
the prime site for them to construct such an identity and maintain their online visibility. At
Fig 12. Indicate your level of agreement with the following statementsÐIndicator of scholarly impact.
15 / 22
the same time, over 80% of all respondents agree that maintaining their privacy online is very
important to them, and there is no statistically significant difference between platforms. It
seems that while online visibility and privacy are both desired, privacy is still the most
dominant concern for users in the online environment (S25 Table).
S26±S29 Tables provide regression results related with the ideological aspects of the two
platforms. Zotero users in general are strong advocate for open access and early adopters of
technology. They are more critical of the current scholarly communication system centered on
journals and publishers, thinking that the current peer-review system is broken and the profit
margin for publishers too high, and that journals and publishers are not necessary for scholarly
communication. They do, nonetheless, similar to Mendeley user, acknowledge the symbolic
capital of journal. At the same time, they are also more critical of the use of Mendeley
readership/Zotero library counts as a good indicator of scholarly impact, indicating not only their
aversion to the traditional model of scholarly communication relying on journal-based
publication and peer-review system but also their more conservative view toward metrics generated
by social reference managers.
The overall effect of occupation is statistically significant in a broad range of issues related
to scholarly communication. Compared to students, non-students (i.e., professors, lecturers,
and practitioners) are more likely to identify themselves as an early adopter of new technology.
Non-students are also more likely to regard social media activity and Mendeley reader/Zotero
library counts as proper indicators of scholarly impact. In addition, lecturers and researchers
have more negative perceptions of the role of publishers in scholarly communication, but
lecturers are more inclined to contend that having a profile on the social reference managers
increase their visibility in their field.
Disciplinary and educational differences are revealing in how altmetrics are perceived. On
the one hand, compared to their counterparts in social sciences, respondents affiliated with
engineering, physical sciences, and psychology oppose more strongly to the idea of social
media activity being used as an indicator of scholarly impact. Physical scientists do not favor
the idea that social reference manager-based metrics reflect the value of a document either. On
the other hand, those holding a doctorate degree share much of the same negative attitudes
toward altmetrics as compared to those with a master's or professional degree.
Lastly, gender and age also have impacts on ideological positions. Females are less likely
than their male counterparts to be the early adopter of new technologies, but more likely to
agree that journals and publishers are necessary for scholarly communication. The increase in
age tends to have the same impact on evaluating journals and publishers in scholarly
communication more positively. Moreover, the older the respondents are, the more likely they are to
think that having a profile on social reference managers would increase their visibility in their
The portrait of users
Altmetrics are increasingly used as indicators of attention given to scholarly documents. It is
therefore of critical importance to have a better understanding of the audience generating
these metrics. This paper first describes the characteristics of the users of the two platforms
and shows that those differ significantly. This suggests that solely relying on Mendeley-based
indicatorsÐfor example in the Altmetric Attention ScoreÐonly provides a partial portrait of
readership behavior and cannot be generalized to the broader population of social reference
managers. More specifically, Zotero's userbase is male-skewed and slightly older than
Mendeley users. Zotero users are highly loyal in terms of both length and frequency of use. Nearly a
16 / 22
quarter identify as practitioners whereas less than 10% of Mendeley users indicate the same
occupational type. The plurality of Zotero users report education in the arts and humanities
and tend to be more collaborative in their use of the platform than their Mendeley
counterparts. They are also more socially-engaged online: making greater use of social sites (e.g.,
Facebook, Wikipedia, Twitter, Reddit) as well as blogging and repository engagement. Mendeley
users, on the other hand, are more gender-balanced and slightly younger. They are fickle in
their use and more likely to employ other types of reference management systems (e.g.,
EndNote). They cater to the life and social sciences. However, some aspects were consistent across
both groups: most users were students, researchers, or professors and about 90% of users
(across platforms) held a graduate degree or above, which is in line with previous studies of
Mendeley user profiles [24,25,36]. Although this might sound obviousÐsocial reference
managers are specific tools that have little use to those not dealing with scholarly documentsÐthe
finding refutes the claim that social reference managers are indicative of the broader social
impact of scholarship. At best, readership counts can be seen as a broader indicator of
scholarly impactÐi.e., impact on studentsÐalthough the high correlations with citations 
suggest that students' reading patterns align with those of other scholarly practices.
The portrait of use
Altmetrics focus on the interaction that users exhibit with objects online. To fully understand
these interactions, we examined the ways in which both Mendeley and Zotero users interacted
with the platform and, thereby, the scholarly objects. As with their demographic information,
Mendeley and Zotero users differed in the use of the platform. Zotero users were more active
overall, more frequently engaging in tasks such as using search functions, adding to, deleting
from, and organizing their libraries. They were also more likely to upload other types of data,
such as datasets, figures, or videos and to embed references using the cite function or create
bibliographies. Their high activity levels were matched with the comparatively higher
importance they placed on these features. Mendeley users engage in some of discovery and
networking functions relatively more frequentlyÐsuch as browsing papers and groups and updating
profile information. This could be an artifact of the different age and occupational stage of
usersÐwith Mendeley having proportionally more students and on, average, younger users.
Resonating with the stated utility of the platforms, their importance in writing the research
project and contributing to the construction of a thesis are shared by both Mendeley and
Zotero users. Other than that, however, the platform were exploited in rather different ways,
with Zotero not only highly integrated in the research project but also utilized for educational
(e.g., course teaching and course assignment) and professional purposes. These differences are
not trivial: when interpreting the impact of a work through altmetric indicators, the intended
functionÐor group of users it coversÐis foundational, as indicators must be associated to a
corresponding concept. This again reinforces the heterogeneity of altmetrics, even when
focusing exclusively on the subset of readership metrics.
The word impact is often bandied about carelessly among scientometricians. However, at the
risk of sounding pedantic, we return to Merriam-Webster, which defines impact as having a
ªdirect effect uponº or to ªto strike forcefully.º We seek, therefore, to understand the level of
effect that readership counts measures. Our results suggest that these indicators may not be as
ªforcefulº as imagined. For example, only one fifth of users (from both platforms) report that
they have read the last item that they added to their libraries. The majority of Zotero users
state that they ªintend to readº the majority of items currently in their libraries. The users
17 / 22
themselves also do not see the tool as a signaling device: they tend not to consider quality
when adding an item and do not consider the community aspects of saving items. Lecturers
and practitioners are the exceptions, who tend to think about the community aspects of saving
items. This is likely an artifact of the use for pedagogy and practice.
Both sets of users functionally used the platforms throughout the process of preparing a
manuscript; the differences arose in their use of the platforms for post-publication dissemination.
Mendeley users were far more likely to consider the platform as a useful dissemination vehicle
compared to Zotero users. One could consider this activity as the equivalent to self-citingÐby
generating a metric for their own work. However, there were gender differences, with females
more likely to have read and have intentions to cite the items in their libraries than their male
counterparts. This is consistent with the literature on patterns of self-citing, suggesting that these
gendered patterns of self-promotion carry into this online space. These results suggest that caution
should be used when interpreting usage data from social reference managers as impact indicators.
Openness is a pervasive ideology across platformsÐthe large majority of both Mendeley and
Zotero users report high levels of support for open access and open source software, though
Zotero users are statistically more likely to be strong advocates for open access. Platform
differences become apparent when examining the system of peer-review and journal publishing,
with a more pessimistic view on the status quo from Zotero users. More than half of Zotero
users reported that the current peer review system is broken, whereas only a third of Mendeley
users reported the same. Fewer Zotero than Mendeley users thought that journals added
credibility or were necessary for scholarly communication. Publishers received the lowest reviews
from Zotero users in terms of both their capacity and profit margins.
Across both platforms, citations were most likely to be seen as useful indicators of scholarly
impact than social media, generally, or readership counts specifically. However, there were
platform differences in this, with Zotero users rated lower the use of readership metrics as
indicators of scholarly impact and were less likely to consider readership counts indicative of the value
of an item. However, neutral responses were the plurality for both platformÐsuggesting a high
degree of ambiguity regarding the current utility of altmetrics as indicators of scholarly impact.
Controlling for platform, differences in ideology emerge on various demographic
characteristics. Students, perhaps counterintuitively, are less likely to identify as early adopters of
technology or to consider social media activity as an indicator of scholarly impact. Women are less
likely to consider themselves as early adopters. Women are also report more positive views on
journals and publishersÐa view shared with older respondents. Practitioners are more likely to
see readership counts as useful indicators of scholarly impacts. Physical scientists do not view
readership counts as indicating the value of a document and those in engineering, physical
sciences, and psychology are much more cautious about the use of social media activity as an
indicator of scholarly impact than their colleagues in the social sciences. Those with a doctoral
degree are much more likely than those with a master's or professional degree to discount the
value or use of social media indicators and readership metrics as indicators of scholarly impact.
The significant differences in ideology call into question a universal interpretation of the
meanings of altmetrics: if those generating the metrics vary in the value they ascribe to the action,
those generating indicators should be cautious in ascribing a standardized concept to the metric.
Readership metrics are generated by a variety of different types of users and these users bring
competing practices and ideologies to the mix. One of the only unifying characteristics of those
18 / 22
using social reference managers is that they are highly educated. This suggests that Mendeley
readership counts should not be seen as an indicator of social impact, but rather to represent another
dimension of scholarly impact. However, the functional use of the platforms varied significantly
both between the platforms and by various user groups within the platform. Readership counts,
therefore, should not be seen as monolithic, but represent various concepts to different user
communities. These communities also varied considerably in their ideologies toward scholarly
communication and the values they ascribed to metrics. These competing value systems and general
ambiguity towards social media metrics should urge caution around the inclusion of readership
metrics for those in the business of making and using altmetric indicators.
S1 File. Survey instrument.
S1 Table. Comparison of Mendeley user profiles.
S2 Table. How long have you been using Mendeley/Zotero?
S3 Table. Which best describes your use of Mendeley/Zotero over time?
S4 Table. Are you part of any private or public groups on Mendeley/Zotero?
S5 Table. Tools for professional purposes (i.e., those associated with your vocation rather than for personal use).
S6 Table. How often do you do the following discoverability tasks on Mendeley/Zotero?
S7 Table. How often do you do the following adding and organizing tasks on Mendeley/
S8 Table. How often do you do the following re-using task on Mendeley?
S9 Table. How often do you do the following networking tasks on Mendeley/Zotero?
S10 Table. Please rate the importance of the following types of features.
S11 Table. Logistic regression on rating the importance of the following types of features.
S12 Table. How important is Mendeley/Zotero to you at various stages of a research project?
S13 Table. Logistic regression on rating the importance of Mendeley/Zotero at various stages of a research project.
19 / 22
S14 Table. My Mendeley/Zotero library has contributed to my: (select all that apply).
S15 Table. Logistic regression on the Mendeley/Zotero activities.
S16 Table. Consider the last item you remember having added to your library. Which of
these statements best characterizes your level of engagement with it?
S17 Table. Consider the last item you remember having added to your library. Which of
these statements best characterize your citation behavior?
S18 Table. Indicate your level of agreement with the following statements.
S19 Table. Logistic regression on the level of engagementÐReading and citation behavior.
S20 Table. Logistic regression on the level of engagementÐCuration behavior.
S21 Table. Logistic regression on the level of engagementÐSocial behavior.
S22 Table. Indicate your level of agreement with the following statementsÐOpenness.
S23 Table. Indicate your level of agreement with the following statementsÐPeer-review system and scholarly publication.
S24 Table. Indicate your level of agreement with the following statementsÐIndicator of scholarly impact.
S25 Table. Indicate your level of agreement with the following statementsÐOnline privacy, visibility, and scholarly identity.
S26 Table. Logistic regression on ideologyÐOpenness.
S27 Table. Logistic regression on ideologyÐPeer-review system and scholarly publishing.
S28 Table. Logistic regression on ideologyÐProper indicator of scholarly impact.
S29 Table. Logistic regression on ideologyÐOnline privacy, visibility, and scholarly identity.
We thank William Gunn (Mendeley) and Sean Takats (Zotero) for comments on the
questionnaire and assistance with survey distribution.
20 / 22
Conceptualization: Vincent Larivière, Cassidy R. Sugimoto.
Data curation: Pei-Ying Chen.
Formal analysis: Pei-Ying Chen.
Funding acquisition: Vincent Larivière, Cassidy R. Sugimoto.
Investigation: Pei-Ying Chen, Vincent Larivière, Cassidy R. Sugimoto.
Methodology: Pei-Ying Chen, Erica Hayes, Vincent Larivière, Cassidy R. Sugimoto.
Project administration: Cassidy R. Sugimoto.
Supervision: Cassidy R. Sugimoto.
Writing ± original draft: Pei-Ying Chen, Erica Hayes, Cassidy R. Sugimoto.
Writing ± review & editing: Pei-Ying Chen, Vincent Larivière, Cassidy R. Sugimoto.
21 / 22
Van Noorden R. Online collaboration: Scientists and the social network . Nat News . 2014 ; 512 : 126 .
https://doi.org/10.1038/512126a PMID: 25119221 Higher Education Funding Councils Funding Council of England [HEFCE] . Decisions on assessing research impact [Internet]. Bristol , United Kingdom: HEFCE; 2011 Mar. Available: http://www.ref.ac.
uk/2014/pubs/2011-01/ Vuorikari R, Punie Y , Nicholas D , Herman E , Jamali HR , Osimo D , et al. Analysis of emerging reputation and funding mechanisms in the context of Open Science 2.0 . [Internet]. Luxembourg: Publications Office of the European Union; 2015 . Available: http://dx.publications. europa.eu/10 .2791/84669 Wilsdon J, Allen L , Belfiore E , Campbell P , Curry S , Hill S , et al. The metric tide: Report of the independent review of the role of metrics in research assessment and management [Internet] . Bristol, United Kingdom: Higher Education Funding Council for England; 2015 Jul . Available: http://www.hefce.ac.uk/ pubs/rereports/year/2015/metrictide/ Sugimoto CR, Work S , Larivière V , Haustein S. Scholarly use of social media and altmetrics: A review of the literature . J Assoc Inf Sci Technol . 2017 ; 68 : 2037 ± 2062 . https://doi.org/10.1002/asi.23833 Wouters PF. The citation culture [Internet]. University of Amsterdam. 1999 . Available: http://hdl.handle.
net/11245/1 .163066 Priem J. Altmetrics. In: Cronin B , Sugimoto CR , editors. Beyond bibliometrics: Harnessing multidimensional indicators of scholarly impact . 1st ed. Cambridge, MA: The MIT Press; 2014 . Available: http:// arxiv.org/abs/1507.01328 Priem J , Taraborelli D , Groth P , Neylon C. altmetrics: a manifesto . In: altmetrics [Internet]. 26 Oct 2010 .
Available: http://altmetrics.org/manifesto Haustein S, Bowman TD , Costas R . Interpreting ªaltmetricsº: Viewing acts on social media through the lens of citation and social theories . In: Sugimoto CR , editor. Theories of informetrics: A festschrift in honor of Blaise Cronin . Berlin, Germany; Boston, MA: De Gruyter; 2016 . Available: http://arxiv.org/ abs/1502.05701 Almind TC , Ingwersen P . Informetric analyses on the world wide web: Methodological approaches to `webometrics .' J Doc . 1997 ; 53 : 404 ± 426 . https://doi.org/10.1108/ EUM0000000007205 Thelwall M , Vaughan L , BjoÈrneborn L. Webometrics . Annu Rev Inf Sci Technol . 2005 ; 39 : 81 ± 135 .
https://doi.org/10.1002/aris.1440390110 Jeng W , He D , Jiang J. User participation in an academic social networking service: A survey of open group users on Mendeley . J Assoc Inf Sci Technol . 2015 ; 66 : 890 ± 904 . https://doi.org/10.1002/asi.
23225 Bonasio A . Mendeley has 2.5 million users ! In: Mendeley Blog [Internet]. 3 Sep 2013 [cited 26 Aug 2016 ]. Available: https://blog.mendeley.com/ 2013 /09/03/mendeley-has-2 -5-million-users/ Bonasio A. Q&A: Team Mendeley joins Elsevier . In: Mendeley Blog [Internet]. 8 Apr 2013 [cited 7 Jan 2018 ]. Available: https://blog.mendeley.com/ 2013 /04/09/qa-team -mendeley-joins-elsevier/ 32.
Mendeley . Adding your existing documents . In: Mendeley [Internet]. 2017 [cited 7 Jan 2018 ]. Available: https://www.mendeley.com/guides/desktop/02-adding-documents Mendeley . The library view . In: Mendeley [Internet]. 2017 [cited 24 Jan 2018 ]. Available: https://www.
mendeley.com/guides/desktop/01-desktop-interface Stillman D. A unified Zotero experience . In: Zotero Blog [Internet]. 10 Jul 2017 [cited 12 Nov 2017 ].
Available: https://www.zotero.org/blog/a -unified-zotero-experience/ Zotero. Quick start guide . In: Zotero Documentation [Internet]. 21 Jul 2017 [cited 12 Nov 2017 ]. Available: https://www.zotero.org/support/quick_start_guide Zotero. Zotero groups . In: Zotero Documentation [Internet]. 11 Jul 2017 [cited 12 Nov 2017 ]. Available: https://www.zotero.org/support/groups Zotero. Data formats . In: Zotero Documentation [Internet]. 12 Nov 2017 [cited 14 Nov 2017 ]. Available: https://www.zotero.org/support/dev/data_formats Stillman D. Zotero 5.0 . In: Zotero Blog [Internet]. 10 Jul 2017 [cited 12 Nov 2017 ]. Available: https:// www.zotero.org/blog/zotero-5-0/ Takats S. On usage figures . In: The Quintessence of Ham [Internet]. 21 May 2011 [cited 14 Nov 2017 ].
Available: http://quintessenceofham.org/ 2011 /05/21/on-usage-figures/ Haustein S, Larivière V . Mendeley as a source of readership by students and postdocs? Evaluating article usage by academic status . Proceedings of the 35th IATUL Conferences . Aalto University, Helsinki, Finland; 2014 . Available: http://docs.lib.purdue.edu/iatul/2014/altmetrics/2 Mohammadi E, Thelwall M , Haustein S , Larivière V . Who reads research articles? An altmetrics analysis of Mendeley user categories . J Assoc Inf Sci Technol . 2015 ; 66 : 1832 ± 1846 . https://doi.org/10.
1002/asi.23286 Mohammadi E , Thelwall M , Kousha K. Can Mendeley bookmarks reflect readership? A survey of user motivations . J Assoc Inf Sci Technol . 2016 ; 67 : 1198 ± 1209 . https://doi.org/10.1002/asi.23477 Thelwall M. Why do papers have many Mendeley readers but few Scopus-indexed citations and vice versa ? J Librariansh Inf Sci . 2017 ; 49 : 144 ± 151 . https://doi.org/10.1177/0961000615594867 Bar -Ilan J , Haustein S , Peters I , Priem J , Shema H , Terliesner J . Beyond citations: Scholars' visibility on the social Web . ArXiv12055611 Phys . 2012 ; Available: http://arxiv.org/abs/1205.5611 Li X , Thelwall M , Giustini D. Validating online reference managers for scholarly impact measurement .
Scientometrics . 2012 ; 91 : 461 ± 471 . https://doi.org/10.1007/s11192-011-0580 -x Haustein S , Larivière V , Thelwall M , Amyot D , Peters I. Tweets vs. Mendeley readers: How do these two social media metrics differ? ItÐInf Technol . 2014 ; 56 : 207 ± 215 . https://doi.org/10.1515/itit-2014 - 1048 Mohammadi E , Thelwall M. Mendeley readership altmetrics for the social sciences and humanities: Research evaluation and knowledge flows . J Assoc Inf Sci Technol . 2014 ; 65 : 1627 ± 1638 . https://doi.
org/10 .1002/asi.23071 Haustein S , Peters I , Sugimoto CR , Thelwall M , Larivière V . Tweeting biomedicine: An analysis of tweets and citations in the biomedical literature . J Assoc Inf Sci Technol . 2014 ; 65 : 656 ± 669 . https://doi.
org/10 .1002/asi.23101 Haustein S , Peters I , Bar-Ilan J , Priem J , Shema H , Terliesner J . Coverage and adoption of altmetrics sources in the bibliometric community . Scientometrics . 2014 ; 101 : 1145 ± 1163 . https://doi.org/10.1007/ s11192-013-1221-3 Priem J , Piwowar HA , Hemminger BM . Altmetrics in the wild: Using social media to explore scholarly impact . Evanston, IL; 2012 . Available: http://altmetrics.org/altmetrics12/priem/ Bar-Ilan J. Jasist 2001 ± 2010 . Bull Am Soc Inf Sci Technol . 2012 ; 38 : 24 ± 28 . https://doi.org/10.1002/ bult. 2012 .1720380607 Dillman DA , Smyth JD , Christian LM . Internet, phone, mail, and mixed-mode surveys: The tailored design method . 4th ed. Hoboken, NJ: Wiley; 2014 .
Zahedi Z , Costas R , Wouters P . Assessing the impact of publications saved by Mendeley users: Is there any different pattern among users? Proceedings of the IATUL Conferences . 2014 . Available: http://docs.lib.purdue.edu/iatul/2014/altmetrics/4