Citations and science
Int J Clin Pharm
Citations and science
J. W. Foppe van Mil 0 1
James Green 0 1
0 School of Pharmacy, University of Otago , Dunedin , New Zealand
1 van Mil Consultancy , Margrietlaan 1, 9471CT Zuidlaren , The Netherlands
2 J. W. Foppe van Mil
Index, and for which category. Although it has never been
designed as such, the annual reports of the SCI are used as
tool to assign funding to study-groups or institutions [
It also is used in scientific institutions to give a score to
the research quality of researchers or research departments.
This latter use is totally inappropriate and unacceptable, also
because non-English language journals are not included.
Another limitation of the Impact Factor is that disciplines
have different rates of citations and disciplines move faster
or slower. Humanities journals in the Social Sciences
Citation Index (SSCI) have some of the lowest citation rates,
but some of the greatest longevity. However, even within
disciplines, some sub-disciplines can move faster or slower.
As an attempt to moderate this problem, ranking or quartile
within a Subject Category can be used as a crude attempt
to control different citation rates. That is, if a journal is
topranked/in the first quartile of journals in a similar subject
area, then journals in different disciplines, while having
large discrepancies in Impact Factor, can be crudely
compared (for example, first quartiles IF cut-offs for the
following subject categories: Oncology 4.7; Pharmacology and
Pharmacy 3.4; Plant Sciences 2.6; General and Internal
Medicine 2.3; Cultural Studies 1.0).
So how do pharmacy and pharmacy practice journals fare
in measurement of impact? Some (but not all) pharmacy
and pharmacy practice journals are included in the
Journals Citation Report (JCR), most of which are in the subject
category Pharmacology and Pharmacy. Pharmacy practice
related journals such as the International Journal of
Clinical Pharmacy traditionally score low in this group. There
are some reasons for these relative low scores for practice
• Pharmacology papers will be cited by researchers on
pharmacology and pharmacotherapy, as well as the
medical community at large. This boosts the impact factors of
such papers and journals.
• Pharmacy practice is relatively underrepresented within
the journal group, automatically leading to lower citation
frequencies, and thus low impact factors for pharmacy
practice related journals.
• Traditionally, citation scores are calculated over a period
of the past two full years, but the speed at which papers
are cited differs between disciplines. For fast
developing research (such as cancer research) this leads to an
overestimation of impact. A high impact factor, based on
many recent citations, is an indicator of the cutting edge,
rather than long term impact [
]. For pharmacy practice
research, that almost per definition studies practice in a
setting that is difficult to control, such a 2 year impact
factor is not very significant because the time lapses
between research that stands on the shoulders of
previous research will be longer. A 5 year impact factor would
do more justice to the authors and the journals.
In this issue, Minguet et al. [
] discuss a possible new
grouping of the subject category of which our journal also is
a part. This approach, based on MESH coding, could also be
interesting for all other categories within the Index.
According to their calculations, dividing the subject category
‘‘Pharmacology and Pharmacy’ into basic pharmacology,
clinical pharmacology, and pharmacy would have an impact
on the distribution of the journal IFs’ quartiles. This could
have major implications for authors, and also on the ability
to obtain research funding given that several funding
agencies evaluate researchers almost solely based on journals’ IF
and quartile distribution of their publications.
Academic institutions somehow seem to remain stuck
in the old fashioned IFs of the SCI. The question is if
this is going to last much longer. Many researchers and
authors start moving away from the SCI, because Clarivate
Analytics apparently does not want to change, and
correct the misconceptions about their IFs in the scientific
community. In the Excellence in Research for Australia
(ERA) scheme (similar to Britain’s REF/RAE), journal
ranking has been removed entirely, in favour of a
descriptive approach . New, more flexible and responsive,
bibliographic metrics have been developed such as Google
Scholar Metrics which uses 5 years of citations, and has
by the far the broadest scope in terms of citation sources
(including mentions of papers in blogs, newspapers etc.).
Scopus/Elsevier CiteScore is another new alternative
with a better coverage of pharmacy journals, and a 3 year
citation window. The CiteScore platform also includes
several other metrics (Sci-Mago, SNIP) developed by
external parties. The SNIP—Source Normalised Impact
per Paper—is likely more accurate, because, as the name
indicates, it attempts to more carefully account for citation
differences between fields. That is, in a field with a low
overall number of citations, a single citation would count
for much more; and the corollary, in a field with high
citations, a single citation counts for much less. The Elsevier
platform also maintains separate Pharmacy and
Pharmacology subject categories. IJCP is categorised in both, and
fares much better with the SNIP than Elsevier’s CiteScore
(similar to the JCR IF).
Article level metrics have also been proposed as a
solution, in particular de-emphasising the selectivity of work.
The most well-known of these is the Altmetric, which
includes citations, but also news and blog sites, as well
as social media mentions. However, the Altmetric can be
quite influenced by Twitter, and many Twitter mentions
seem to be either very mechanical, or even automated [
None of these alternative metrics is yet well established
as an alternative, but they have the potential to be more
adaptive, accurate and inclusive than the Clarivate
Analytic’s Impact Factor. Clarivate Analytics is a very closed
organisation; their criteria for assigning groupings, or
accepting journals into the Index are not always very clear,
and even the way the IF is calculated is not fully
]. In this case, however, we would be very interested
in seeing a response to the paper of Minguet et al., because
they discuss an issue that is very important for research
level assessment and funding at large, and not only for
practice related research in the field of pharmacy. Without
adaption and change, Clarivate risks being left behind.
8. C r e a g h S . J o u r n a l r a n k i n g s d i t ch e d : t h e ex p e r t s
r e s p o n d 2 0 1 1 . h t t p s : / / t h e c o n v e r s a t i o n . c o m /
9. Robinson-Garcia N, Costas R, Isett K, Melkers J, Hicks D. The
unbearable emptiness of tweeting—about journal articles. PLoS
ONE. 2017;12(8):e0183551. doi:10.1371/journal.pone.0183551.
1. Young NS , Ioannidis JPA , Al-Ubaydli O . Why current publication practices may distort science . PLoS Med . 2008 ; 5 ( 10 ):e201. doi: 10 .1371/journal.pmed. 0050201 .
2. Fraley RC , Vazire S. The N-pact factor: evaluating the quality of empirical journals with respect to sample size and statistical power . PLoS ONE . 2014 ; 9 ( 10 ):e109019. doi: 10 .1371/journal. pone. 0109019 .
3. Christley R . Power and error: increased risk of false positive results in underpowered studies . Open Epidemiol J. 2010 ; 3 ( 1 ): 16 - 19 .
4. Fang FC , Casadevall A , Morrison RP . Retracted science and the retraction index . Infect Immun . 2011 ; 79 ( 10 ): 3855 - 3859 .
5. Brembs B , Button K , Munafò M. Deep impact: unintended consequences of journal rank . Front Hum Neurosci . 2013 ; 7 : 291 .
6. Leydesdorff L , Bornmann L , Comins JA , Milojević S . Citations: Indicators of quality? The impact fallacy . Front Res Metr Anal . 2016 . doi: 10 .3389/frma. 2016 . 00001 .
7. Minguet F , Salgado TM , Santopadre C , Fernandez-Llimos F. Redefining the pharmacology and pharmacy subject category in the journal citation reports using medical subject headings (MeSH) . Int J Clin Pharm . 2017 ;. doi: 10 .1007/ s11096-017-0527-2.
10. Haustein S , Bowman TD , Holmberg K , Tsou A , Sugimoto CR , Larivière V . Tweets as impact indicators: examining the implications of automated “bot” accounts on Twitter . J Assoc Inf Sci Technol . 2016 ; 67 ( 1 ): 232 - 8 .