Implementation fidelity of a clinical medication review intervention: process evaluation
International Journal of Clinical Pharmacy
Implementation fidelity of a clinical medication review intervention: process evaluation
F. Willeboordse 0 1 2 4
F. G. Schellevis 0 1 2 4
M. C. Meulendijk 0 1 2 4
J. G. Hugtenburg 0 1 2 4
P. J. M. Elders 0 1 2 4
0 Department of Public Health and Primary Care, Leiden University Medical Center , Leiden , The Netherlands
1 Department of Information and Computing Sciences, Utrecht University , Utrecht , The Netherlands
2 NIVEL, Netherlands Institute for Health Services Research , Utrecht , The Netherlands
3 F. Willeboordse
4 Department of Clinical Pharmacology & Pharmacy, VU University Medical Center , Amsterdam , The Netherlands
Background Implementation of clinical medication reviews in daily practice is scarcely evaluated. The Opti-Med intervention applied a structured approach with external expert teams (pharmacist and physician) to conduct medication reviews. The intervention was effective with respect to resolving drug related problems, but did not improve quality of life. Objective The objective of this process evaluation was to gain more insight into the implementation fidelity of the intervention. Setting Process evaluation alongside a cluster randomized trial in 22 general practices and 518 patients of 65 years and over. Method A mixed methods design using quantitative and qualitative data and the conceptual framework for implementation fidelity was used. Implementation fidelity is defined as the degree to which the various components of an intervention are delivered as intended. Main outcome measure Implementation fidelity for key components of the Opti-Med intervention. Results Patient selection and preparation of the medication analyses were carried out as planned, although mostly by the Opti-Med researchers instead of practice nurses. Medication analyses by expert teams were performed as planned, as well as patient consultations and patient involvement. 48% of the proposed changes in the medication regime were implemented. Cooperation between expert teams members and the use of an online decision-support medication evaluation facilitated implementation. Barriers for implementation were time constraints in daily practice, software difficulties with patient selection and incompleteness of medical files. The degree of embedding of the intervention was found to influence implementation fidelity. The total time investment for healthcare professionals was 94 min per patient. Conclusion Overall, the implementation fidelity was moderate to high for all key components of the Opti-Med intervention. The absence of its effectiveness with respect to quality of life could not be explained by insufficient implementation fidelity.
Drug-related problems; Implementation barriers; Implementation fidelity; Medication review; Process evaluation
Department of General Practice & Elderly Care Medicine,
Amsterdam Public Health Research Institute, VU University
Medical Center, Amsterdam, The Netherlands
Impact on practice
Performing medication analyses for clinical medication
reviews by external expert teams is feasible.
Cooperation between fixed expert teams, consisting of a
physician and a pharmacist and the use of an online
decision-support medication evaluation facilitates
the implementation of clinical medication reviews.
Time, cost reimbursement, training and a dedicated
practice nurse or coordinator in the GP practice seem to be
necessary for successfully implementing clinical
medication reviews. In addition, software programs for patient
selection, exchange of medical and medication files and
outcomes of medication evaluation are needed.
Implementation fidelity is defined as the degree to which
the various components of an intervention are delivered as
]. Convenience of use and degree of
implementation exert considerable influence on the applicability of a
complex healthcare intervention in daily practice.
Implementation fidelity gives researchers and practitioners a better
understanding of how and why an intervention is effective or
ineffective, and the extent to which health outcomes can be
improved. Implementation fidelity reflects the adherence to
content, frequency, duration and coverage of the
intervention. In addition, there may be moderating factors that
influence the degree of implementation fidelity [
]. As long as
the evaluation of the implementation fidelity has not been
performed, it remains unclear whether ineffectiveness is due
to a poor implementation of the intervention or inadequacies
inherent to the intervention itself.
In this study, the complex intervention of a clinical
medication review (CMR) has been evaluated. A CMR is a
structured, critical examination of the patient’s medicines with
the objective of reaching an agreement with the patient about
treatment, optimising the impact of medicines, minimising
the number of drug related problems (DRPs) and reducing
]. CMRs can improve the appropriateness of drug
prescribing and medication use and are increasingly used
and recommended in primary care [
]. However, in daily
practice the implementation of CMRs is difficult and time
] A recent review highlights the need for
research on intervention development and process
evaluations to improve the understanding of how effective
interventions to prevent potentially inappropriate prescribing can be
sustained and ultimately be translated into improvements in
patient outcomes [
]. Therefore, the Opti-Med randomised
controlled trial (RCT) was recently carried out in a primary
care population to test the effectiveness of CMRs on the
quality of life and DRPs.
The Opti-Med study design and its results have been
published separately [
]. In short, The Opti-Med study was
designed as a cluster RCT in 22 general practices (Fig. 1)
. We studied the effects of CMRs on quality of life
and DRPs in 518 older patients (≥ 65 year). Patients were
selected and invited when they chronically used one or more
prescribed drugs and newly presented themselves to the
general practitioner (GP) with one or more geriatric problems
(immobility, instability, incontinence and impaired
cognition). Patient selection was facilitated by software specifically
developed for the Opti-Med study based on electronic
medical records (EMRs). CMRs were conducted by the expert
teams according to a structured program using the STRIPA
]. Patients in control practices received usual GP care
with no specific attention to their medication use.
The Opti-Med study included three innovative CMR
elements. First, medication analyses were carried out by trained
external expert teams consisting of a pharmacist and a
physician, not being the patient’s own GP and pharmacist.
The second innovative element was a new target group.
We included patients of 65 years and over who chronically
used ≥ 1 prescribed drug and had one or more geriatric
problems, also called geriatric giants (immobility, instability,
incontinence and impaired cognition) instead of
polypharmacy patients, which is the usual target group.
Inappropriate medication use may be associated with a higher risk on
the occurrence and persistence of these geriatric problems.
The nature of this association is complex, as the causes of
these problems are multifactorial; however these geriatric
problems are among the most common adverse drug
The third innovative element was the method of patient
involvement. Patients gave input for the medication analyses
by means of completing a questionnaire and discussed the
results of the analyses during a consultation with their GP.
We hypothesized that these three elements would
facilitate the implementation of CMRs in daily practice and
thereby increase their effectiveness. The results of our
effectiveness study showed that the Opti-Med CMRs indeed
improved appropriate prescribing, i.e. more DRPs were
identified and solved after 6 months of follow-up compared
to usual GP care, but there was no effect on patients’ quality
of life [
]. A process evaluation of the Opti-Med
intervention could clarify whether the limited impact of the
OptiMed intervention was due to a poor implementation or due
to inadequacies inherent to the intervention itself.
Aim of the study
The aim of this process evaluation study is to gain more
insight into the implementation fidelity of the Opti-Med
CMR intervention in daily practice.
This process evaluation was conducted alongside the
Opti-Med RCT. Within the present study, the
implementation fidelity of the Opti-Med intervention was evaluated.
Quantitative data was collected from the start of the study
and qualitative data was collected at the end of the study.
For the evaluation we distinguished five key intervention
right treatment, STOPP screening tool of older person’s prescriptions,
STRIP systematic tool to reduce inappropriate prescribing, STRIPA
systematic tool to reduce inappropriate prescribing assistant.
1Questionnaire by Willeboordse et al. [
A. Patient selection and invitation by GPs and practice
nurses to participate using EMRs through a newly
B. Patient involvement through a patient questionnaire [
C. Preparation of the medication analysis by practice nurses
and Opti-Med researchers;
D. Medication analysis and drafting of a
Pharmacotherapeutic Treatment Plan (PTP) by an expert team. The
expert teams followed accredited online courses for
CMRs and two face-to-face CMR workshops. An
electronic medication evaluation tool, the Systematic Tool
to Reduce Inappropriate Prescribing Assistant (STRIPA)
] was used for the medication analysis;
E. GP consultation with the patient and implementation of
Conceptual framework for implementation
The adapted Conceptual Framework for Implementation
Fidelity was used (Fig. 2) [
]. The framework allows to
evaluate both adherence to the intervention and to assess
moderating factors for adherence to the intervention.
Fig. 2 Adapted conceptual
implementation fidelity for the Opti-Med
process evaluation. The
implementation fidelity is the measurement
of adherence of the categories
content, frequency, duration and
Adherence to the intervention includes the dimensions
content, frequency, duration and coverage.
Moderating factors for adherence to the intervention
include the dimensions participant responsiveness, strategies
to facilitate implementation, quality of delivery and context.
Specific research questions and outcomes per key
intervention component (A–E) for each dimension of the
conceptual framework are presented in Tables 1 and 2. A subjective
rating was used to evaluate the implementation fidelity and
the researchers assigned the ratings for each dimension of
the framework using four categories: very low, low,
moderate, high. ‘Very low’ means that almost none of the
intervention elements were carried out as planned, ‘low’ means that
some elements have been carried out as planned, ‘moderate’
means that the majority of the elements have been carried
out as planned and ‘high’ means that almost all elements
have been carried out as planned.
The following data sources were used to address the specific
Data on selection, inclusion and drop-out of participants,
time planning, performing medication analyses by the expert
teams, and consultations with the GP were recorded by the
researchers alongside the RCT.
Focus group with experts
A focus group was held with seven members (one GP, two
elderly care specialists and four pharmacists) of the four
expert teams to collect data on their experiences with
conducting the medication analyses. The meeting lasted 70 min
and was audio recorded. To facilitate the discussion a topic
list was developed beforehand (online resource 1).
Interviews with the patients’ GPs
From each intervention practice that performed more than
ten consultations, a GP was invited for an semi-structured
interview; all participated. The interviews were held by the
researchers, lasted 15–30 min and were audio-recorded. The
objective of the semi-structured interviews was to discuss
the experiences of the GPs with this method of conducting
i- s )y iss l- P -n g ree n r - .t tss to P
itlittliiitlttirrrrsssseaacaacceaeceeceacdoupnndongdohnuonnonw ,.tititllititiitrfrrrrrrsscecaacaecceaeeaeenpouyoudnbpynuvowHw tiitititltltrrrrrrrrsssseeeeceaceeeaecedxnvdouppoodpuoophubyvm .tiittitltiittitrffrr-sssaeaeeaaeaceeaeononoundoohnnouqnFPPTwm ittlitiltittittrrfrff-ssaecaeeeeaceaecaghhvunduoonnuoonponmmm ,..ttlilttirrrsseaeecaceeeeccagponpbopxhogndommm itilitttitrrfrrsssecaaaeaceaeaaeeednonydhooybuhnopphwm .,tlilltltitrffrrrrrssseeeeaaeeeeaeecaeeounypdnnnpodhghhhngTmm ltiiittiti(rfrrffaaacaecaaeaaaacdndddnoophnoonodERmmmmmM .ittitiitltlrrrrsssecececaeecaeaaeaaadoubyhpnubdnonodnnpywM iittttitffrrrssseceeeeeaeeaaaedunbyxphdonppwmm ttttiittilitirrrrrsssseeeeacaaeaceaaacebhxpyoudynuudnondwm .tliittittfIIrrsseaecaeeaaeeaohpupnohpooonodunnSPTTRCAwm .ltllllttlrfrfrsaeeeeeeaeecaeyknodnghuppxopdnoodogdwAmmmm .,itiiiilirrffrfseceaaceecceececepovodopnnodhnyuqnyuopdxhFwm ,,litiiittrrffrfsaeceaeceaceceeoopvhdpoonoonndnknyopnonowmm .iiililttllrrrrssseacaeeeaeaaeeeeeudpyugdnnpddppxgdSPTAOmm .itttitirffrrsseaaeeeaeeeeacahdngohddduyodnSPPTTTTRAwm tttltiltttfrI-rrrsaeaaeeceeceobuhuyovodnvdbouduuhSPTRAwm irevdopm ttlti:tlltlirrfrffsssseceaeaeeeecdhnouooonpdnndoubpyndoPPGmm ittttltiitiittlfrrssssssaeeeeaeceaaeanduuopphodunonunnounwmm ,.tilttitltrrrrrIfrssacceceaeeeaeeonpnunohoohguoppynohdPPTwm ,ititlttitirrfrfsssseaeaceceaeaeecannohnhopovhophnpybdudndw ,.tittititltitttiirrrIssssaceceaceceecaeaephopnuddnhnuoohdvowm .lijittttittitrsssssceaaceaeeaeeydudnohongnhhhnphhdPwwGAm ,tttiittllittlrfrrrsseaeaeeecaceaaeeenonoopnokngdponnhudwwmm rfrsceeeenp tlttttttifrrrrsssseeeaeeeeaaaaeccoxnuxpoubhgdhnvgogndoPGm iitilitiif(fffssssseaaaeeececceceacbydnnddhonnydnugnhuoCRwM ,,,tjittiittitiil)rfrrrrssseceaeeaeeacceeanobpvypxndxnovnobunggnm .,tittliillttrrfrrssseeeaeaceeaeaeaaeapnovhoydnodnvhdoopuwwHw tiititiitlttirrrr-sssseceaaeaaeceeaeenybuyqngunvnhbynonPwGmm .ittiltiijttittrsseaaeaeeceaeeeeondvdnngokbvyonkonpxhgTNwmm tiittittltittrr’sssceeccceeaeaaeaeuvnpoononphybpnynphPGmm itlttiilttltitrrrsssseaaeeecaaeeeeeadoxppyv–3y2hnodnhnwwmm .,tlliitilttiliilf()sssssssecaeeaeepooduopo112b01945npbgwO%m .ttliltltllrIrfrssssssseceaceceecceeedono3ouondypbopuddduuuw ilttttltrrrrrfsssssseececeeeeeaa4ndpodbduuonpobdyhobpowwGm tittitrssaceennooudm
P P T M G A P
s ru )
a t P
w cu PT
te ( h
x ? t
e d g
t e n
a n t
h n f
w la r
o p d
i A A
- f s r
tsecom .illitititttitiitlrfrrfsssseeeeacaaeeaeednbyhnnoppuqnhpno71w% tiiitlittttllitirfrfsssseeaeeeeeeaeaquhonnndoundnpnodybdbyuwm iitttlitrrfrrrrrrrssssacaeeeeaeceaeeoovdnohhooybyhhmm iitititititillrfffr-ssssseeeeaaaaeeeeedndonhobopnonuqnpPRwD ,,itiitliilttirfrrsssaeccaeaaeacaaecaduonunogddnpbopnPRDmm .littittirrrrssseeaeeaaeaeceaeppdpnno1d8151pdhnonFRCwmM ,tititiitttrrrf)(sscaaceeaceeaeenyophyhonovndndhuwmmmmm itti/ttitlitirrssseeaeaaceeeavdodpbyhpnnodondhduPwG tiiittilitirfrfrfrssssaaeeaaceacaeynpodno472ood275nnpphwmm tttttllitit(rr-frrr)frrseeeeeeeeaaeaaopndouopoubgnopxonvnnddom titltrseacnpono423 , .tlittitttirfrrffrr-eecaeaeeeeeeaaequhnynuno3pdgnohpnooFmm ,ltliiltittirrrrrfrsssssaeaeeeeecaenndondugdpvbondnongnoduonm .,tiittiittittitlrrr-r-sseeeeeeeaevnonno275nhpvnuoonp2821nnvnFw ttlttttrrrrssseeeeaeeeaeeaeecedbyophxnxopponnduooddnnhwmm .,itttlttitlttrrrrrrrssssceeaaeaeeecaaceaeceeeupnpnhoophvhhyRmm ,ltlttitltitirrrfseaeeeeegud2dpob121dhhnooouoTECwDONUmM .,tittltittitttirIrrrsssseeaeeeeeeeeaavnndonnooophp8huonpnyww sP tiitlilitittlirrfr-rrsaeaaeeaeeeacacnohghonnhpnopnvgoononwmmm t,..ilttitttilrr(rssaeecacaaeeacgnoovnon6o92hpnp624od%%mm .)0100< titiittiittlirfrfrfrrsssaeaaeaeeceaeaonoddonongudphonhhgonwmm tp..,,.tttr(rf)scaeepodo3477640gud002=%%m ittititttrf)(sssseeeceeaeedudhohhp742hybnPPPTwwG tititreaenppn1ubom A ,litiitittititi(rfrfrrceaeaeaaaaeenudnngghngpnoop5ndnndnomm t)rssecehpoPPT tliiltittirf)I(sssseaeeeceaaaceaodubybnnuonnndnunoonhdRQwwm .)(sayd15–51033 tilttitiittlttittrffssecaeaeeeaaeadhhnonuophnnho2h1nondhnpw%m iliscnoun tttttii]rr[rrsseeeeeeaaeeeaenu7pxp122dnbnppmmmm
u l d il 9% ic ll li m S po lfo ti p id th RD eh th p h o 0%
O A 1 A A T T 9 A N 1 M 4 M
t? ic G
n d d
ie e n
? ta m a
d p a n
e a e io
from ltec reap lscu
re se rp in
re to to ee
ew iton iton tew
s a a b
on ru ru re
it d d e
ltsau tead tead treh
n m m e
o ti ti w
c s s s
e e y
G eh eh ad e?
y t t ? y ta
n s s s
a a a is an d
m w tw ly m on
w tah A ah aan ow ttia
I ta I – I
c t ic
n n d
re e e
n e n m m
lttisaunoo ifaonodh lttiseecno itlenvonv itrfaaono ilssay
cP tlau itea itea rep an
G a P P P on
.e vE .a .b .c it
w ? i
s s t
e e a
ir lv p
a e e
n s h
n o em f
tio tis th eo
s e t c
e u n n
u q ie e
q t t u
rch iten eap ilfn
se p th teh
ny by sa ? n
m te tw sP m
w le a R w
o p h D o
-u ten ten
rm em fe
o l f
f p i
?d re im ro
e w n e
m s o t
r P it ra
fo R n
r e n
e D v o
ep dn tre tita
r a in n
e s e
w n d
se tio se lem s?n
s n po p io
ly ev ro im tn
an r p e
a te e in rv
n in th se te
tiao sed ree ecn idn
icd po tw fre seo
e ro f
m p ten ?d id ro
y y x e e p
n n e n r
t n e f
am am ?d ah la th o
e p e p
w w ta w s r
o o l o a e ty
A ig ra g
R N H
d c c
,0372PERM tirseeenw llilliraoynm titteaeenph it)rseccapP it)sccae it)scce lirrsceeeuonn trseacdopom2(sseχnowm, ) rrsaeceapodm TPP,raaehpm tirrrseeapoppp
tirehG t)ap% trreee itionv ti)scce eenG rpPG rapPG i“ynO iaaeng tllaeny .0010 endgn lilcapb iceaun
fo (15 yw lte rap tew en en tud r ic p< ea ap ed
tissaebh .tea364 tseeuh irseadb enPG 22–b% teebw teebw -sedM tiffedo iifsegn t,tt(se ireang ANtno lttrooo
p ca t ew e0 4% 1% tip dn re on iff ,y ic
t on iic eb no )s te g 6 3 O i w ti d g t
tsceoum ittir][sseeaeaepnunp9143ndm ltitlliittirsseeceaeeeadnynp140w ttiitittrr)(saeeeeaopnvdn5p8w% titltlrfseceeaeceehonbydudxPG tiiitrfsssaaecceeayonhpoduww ittr(racceeeeaepnb3–3gn4PwG% ilr(reeceea–233bng1ddnu5w%% iltlii(rrsseaeeeacnbnodgdon5w% tllti(rrsaeaeang6–2dnopoddn1% itittlir(raeaeaececng8–ppodnd9% ititlttifrffreeaeeeenhnophoguw ”1 iitttiltitrsceaeaceeapdodpnhodnw ,tttttiirrseeeaaachngohbunppmm p.ilt)ssscaeeeeadduyhnd002=m ittiittrsseeadondpondondohdnw tititrsaacpnpo IT,tltiiittirfrrecaecahoonoonnonm ISTRPA,titir’rsssssaeceynopnpom
O M 2 2 1 4 1 S P P p rs
?n ited ted tebw
lttiscaunoo tissannvw ttiirsacapp itsccea? titirssecac
PG tiea ien rpP rach ?
a p tap ?p G t s
sn itfono lteeced itevnd ll-oouw teeenw tiaepn reonpd
itseo raud tseh iteh fdn seb isen r-sen
u n f a cn ecn no
q a o fo tu re
rscaeh teehm itrono te? itrono -roop iffed ifrfed rsand
irfec saw ropp iiacp ropp tedh treeh treeh edon
ic ta t t s e e sp
e h ah rap tah aw re re re
S W W W W W
II ag ,
D s V re I
it()cennoud itrecnvonom tltiscanouon itfaaenoodh lraen
te P lu e
le1 in .G av .G
ab ey e3 .E a4
T K 4
CMRs. To facilitate the interview a topic list (Electronic
Supplementary Material 3) was developed.
Evaluation of the implementation of the results
of the medication analyses
An evaluation form was used by the GPs to record the
follow-up of the changes in the medication regime as proposed
by the expert team, including the reason(s) why (part of)
these proposals were not implemented. The expert team also
indicated for each proposal whether this was influenced by
the input of the patient via the questionnaire.
Classification and assessment of DRPs
The changes in the medication regime as proposed by the
expert teams were classified by the researchers (FW, JH) into
DRPs using the DOCUMENT DRP classification system
For a random sample of 21 (8%) of all patients a
medication analysis was performed by two different expert teams
to assess reproducibility.
Subsequently, the STOPP and START criteria were
applied to these DRPs to establish their external validity.
STOPP (Screening Tool of Older Person’s Prescriptions) is a
list of medications that are potentially inappropriate for older
people. START (Screening Tool to Alert doctors to Right
Treatment) is a list of medications that should be prescribed
for older people for a number of conditions. The assessment
was carried out by one researcher (HvD) by means of an
iterative process. Eventual difficulties were discussed with
a second researcher (FW) until consensus was reached. A
random sample of 10% of the patients was independently
assessed by a second researcher (FW).
At inclusion, patients completed a questionnaire about their
actual medication use and experienced problems with their
medication. The patients indicated whether they filled out
the questionnaire independently or whether they received
The time investment of the expert teams and the GPs in the
intervention practices for completing the respective elements
of the intervention was calculated by the researchers.
Electronic medical records
Data on gender and age from the GPs’ EMRs was used for
the non-responder analysis.
The intervention patients completed a survey 3 months
after baseline. The survey assessed the preparation and
usefulness of the CMR and satisfaction about the
consultation with the GP.
Survey among GPs in control practices
GPs from the control practices received a short survey
to assess whether CMRs were conducted unintentionally
during the study period for patients of the control group.
Descriptive statistics were used for quantitative data using
SPSS Statistics 23, using t tests for continuous variables
and χ2 statistics for categorized variables.
For qualitative analyses, audio files were transcribed
verbatim. Transcripts of the focus group and interviews
were coded by two independent researchers (respectively
FW and MD, and FW and SY) top-down with a pre-defined
code-list which was formulated based on the topic lists and
knowledge of the intervention. Differences in coding were
discussed until consensus was reached, a few codes were
added retrospectively. Citations and coded transcripts were
arranged to broader themes using Atlas.ti software [
Outcomes per key intervention component for each
dimension of the framework are shown in detail in Table 1 and 2.
Adherence to the intervention
Patient selection was carried out according to the
inclusion criteria. However, for this topic, we deviated from
the study protocol, most practice nurses did not carry out
patient selection and invitation themselves due to
difficulties in using the newly-developed software application and
due to time restraints. The Opti-Med researchers provided
extensive support or carried out the patient completion
Also, the Opti-Med researchers collected most
information (GP EMR data, medication overview from pharmacy
and patient questionnaire) for the medication analyses
instead of the practice nurses, due to time restraints.
ed tih on idd rbo rno eh ro IC th a le n
itiitittlilittifrrrssaaeeceeeecacaznondvnonnudnugomm tttllttilrfrrfseeaeeacaehoppoounonPG tittilllrffrsssseaeeeeaeaenpodnuonondpddPwwwmm itiitttttirrfreeceaaaexhondnonom titilltitittititfrrsseaeeeeeeaanpndonvnhuqouon32n31m ttttitlttiititfrssaeaaaeacehhndphounono1hh4PwG% ittltllrsaceaceonuonh ,tiltitlittlrrrrrfsaeaecaecappuopnddohyonu4on238%% ,i/tiiitittrrrfsseecaeeedhonhdduvov24oughhooow%m titttrfrssseeaaeeeobnhndouqno4bdnoguhdonoow%m itltsacnonuo tititttlitltrfsssseceeeeaceaeaaapnvdphoonhuon8pn8% tlsaaenp lttitttllfrrfssssaceaeeeunohonhguohuuov7uyu2w% iitltiitillitrrsssssacaeceeeaecaaeendoonnugupdygndw ttitlitirrrfssssceaeaacecaeaphdhonpdoonbhoPERGM .iitiliiititlitlitrrffssseacecceaenuoundyuodnuyppnghD ittlitititrrrfrrssssaeaeaaecceeeeannoydnnpnuobwwm .tlititttiltjitrrrseeceaeeeeceaanponhuoypdnndnyhoyTm tlrrrrfssceaeeeehhophm iittitttillttrfrssaeaeeaceezeeaeennondqunphuodhopm lttittittttilirfrrssssaceeeaeeeeaauononhongopvonvpw tilititittrrrsaaeeeeodnnoohndhRCwM itltililittitittrffsssssaacaceeaaeeoknduonnnoghupn itiltitrssscaeaaceeaaeeeadonuuondnnnddoPRDm .illitilitirrfssaaaaceeeeanyyvnnodydunobpOmmm ititsnooph ,itittil-rrrrscaeeeceengunondhughopnooonnovnmm .lltitititIffrrfeceeaaeohodngononohSPTCRAmmm ltititir-sssacaeacaeecanoonvyhpdnbunuonnwmmm ttititilIfsssseeayhonoPTGmm
P G P T P A
g ? io ?
n n t n
ed ito tla ito
,tan ltau snuo revn
u sn c te
o o e n
ab fc th ie
ed llo fro th
m a e v
r c r i
o e a e
f r p c
in tn re re
ts e p p
n ti s ts
ie a tn n
t p e ie
a e ti t
p th a a
e a id id
w ? w d d
ow tio o o o
w w w
H H H H
a e r
tsecoum , illlttiltiirrfrfrsssseceaeaeeeaanonou5nhpodoopxnongnwm tilittitrfrrfssaaaeacaeeaeeeaeceenonydnd2hoppohdbknmm . titiitttittitillrrrffrsssssseeeeaeeavnonunnghononh2honDm ttitttltitrrrrrsssssseeeaeceaeeaeeeeeybdhhpohphhxwwm .llititfrrfIsssssaeeaaaeaaceaaeobouqvonndgkpudSPTRwAw ltiittrssseaaaceeuynnodhoppm ,lilltiittittifrssseeceaeeeaaeecaaeupudyogdgnnhuqndhwwm .ttllttitillrfIrssseeeaeeeaexophhndbokSPTRwwAmmmM .iitlttrrrrfsseeccaeeeeaeeeddugnhuqougdonkdnohduyhTww ttttililttitIrrfrsssaeeacaeaeeaahhddnpbguoppoxSPTRAwm .liiititttttrrrrrIssssaaaecaeeceaahnybnodduuhSPTRAwAm ittittffrrsssaeangodhngopuponPPTw tiitiitrrfrrrrr-ssseeeeaceeeeacedbyhnohnonvdnukgkPwGm .ittilititititfrrrfrseaeeaeceaeeeeenpddvonndngohnvnonmm tiiittitltrrrrrssssaeaeeceeccaccaeaeokbkoodhpvnupdnhPww tittirrrrrrsssssssecaceeeeeacaeeeeaeeenuphnhdhndnddww .liiitllititt-rfrrssaeaeeeaaaeeaaoonhpvnoquobvoddpmWltitttitittrffrrssaeeaaeeabunhooohnubgnoPPPETRGmmM tttitliirrsseaeeeehgnobpovdpoPPTm titilitititirrff-seaeaaeececacndpgndnokkhoounonwwmmmm .iitttiitttrssscaeaceeceeeenddhonnonkundvphgoyudoPGw ttilttttrrfrrsssseeeeaaaeaeceyhyndoonuvhuudbuoPPTwmm iltititittiti-rIrrsssaeceeeceaaobyhphoonnhddnybPTwGmm .iittllttitttiltrfrsssseeceaaeeaeaeeanonovnnnphoyudnoddd iiittitlittitffrfssseaeaaeeaeoddonbononopnnpupPRDwmm ttitliiiiiirrffsssssaecaecaeapdoonohghnyngdodonbPRDmm tp,.lttittititr()ssaeeceeea60ndy64vpnpunp0100<% ittttliilllirrfssaacaceeaaaeacenogodobuopphyouqhoyndwm .iltlliitititlirrffsceeeeaeeaceeecaponhqunodoyohdTwmmm .ittilitirrfffsssssacceaceeeeeeeeeceeponnbddqunnhPAGw lliittirfrrrrsssssseeeeaaeececeeeauundnodnoqudowmmm .,tttitltttttiffrfrsecaecceeaeehpnudonunoohoooonvPGHw titttittllirrrrsssssecacecaaaeeceeanoodnhdopoubdyPGm itlttiitirrrrrrrrrsseeeeccaececeeepopodnyvnoondndngonodow tjseadud
O E T In H Q Im T
S W H
e ? b
p e d
im ru iv
t n ec
o e r
p ic e
p t p
u c s
s r i
to p g
s d te
ie an tra
eg s s
t P e
tr G e
s eh th
re t e
e y e
tah iton ow
ty G t
l e l
a h a
u t u
q y q
e b e
th d t
s te s
a a a
w lu w
w v w
o e o
s d an
P n s
Nineteen percent of all DRPs identified were based on
the input from the patient questionnaire. The majority
of these DRPs were related to medication knowledge or
adherence to medication.
The expert teams carried out medication analyses for
all but one of the 275 participants of the intervention
group and for all 243 control patients. According to the
expert team members, medication analyses were
conducted in a highly structured manner, mainly due to use
of the STRIPA tool. They also mentioned that the method
and high number of medication analyses by fixed couples
improved efficiency and collaboration. The expert team
members and the GPs mentioned the ‘external’ nature of
the team as an additional value, because of the fresh
perspective of such a team allowing an independent
In 90% (247/275) of the patients, GPs discussed the
proposed changes in medication with their patients. 42% of the
patients had their consultation within the planned first month
after inclusion. The method of consultation was deliberately
not specified by the researchers. Most GPs planned double
consultation time and used a few minutes to prepare the
consultations using the PTP.
Figure 3 gives an overview of the frequency, nature of
DRPs and proposed changes in medication as well as their
implementation rate, and reasons for not implementing as
proposed. Nearly 50% of all proposed medication changes
were (partially) implemented (consented implementation).
‘Addition of a drug’ was significantly more often
implemented than ‘cessation of drug’ (46.7 vs. 34.7% (t test,
p = 0.002). The implementation rate of non-pharmacological
recommendations (e.g. laboratory tests) was significantly
higher than proposed changes in medication (69.2 vs. 42.6%
(t test p < 0.001). The most frequent reasons for
non-implementation were: ‘proposed change is based on incomplete
medical or medication files’, ‘prescription originates from a
medical specialist in secondary care’ or ‘the change in
medication has been tried before by patient and/or prescriber’.
The total time spent by all healthcare providers for one
patient was estimated at 94 min. This includes 1 min for
patient selection, 15 min for preparation, 22 min per expert
team member for medication analysis and 34 min for GP
Over half of the patients reported to have prepared
themselves for the consultation with the GP by bringing or
studying their own medication, preparing questions, or bringing
someone to the consultation. Fourteen percent of the patients
who had a consultation with the GP did not recall it. Of the
patients who did recall the consultation, the majority
considered it useful.
Strategies to facilitate implementation
Patient selection was facilitated by software specifically
developed for the Opti-Med study. However, most practice
nurses considered it difficult to use and time consuming.
Collecting information from the GPs’ EMRs and pharmacy
records in preparation of the medication analyses was
useful but time-consuming. The quality of the preparation for
the medication analysis was deemed sufficient by the expert
Training in performing CMRs was deemed useful by the
expert team members. However, they indicated that most
knowledge and skills were acquired when performing the
medication analyses. The use of the STRIPA tool was found
to greatly support and to highly structure the medication
analysis. Some GPs indicated that the form with the PTP
was not very user-friendly; however, after a few
consultations, most GPs became familiar with it. Seventeen percent
of the patients reported to have been assisted in completing
the patient questionnaire.
Quality of delivery
The GPs considered the PTPs drafted by the expert teams
of very good quality.
The mean difference between the number of DRPs per
patient identified by two expert teams was 1.5 (standard
deviation (SD) 1.2) and the mean number of differences in
type of DRPs was 2.4 (SD 1.4).
In total 33.1% of the DRPs identified were related to a
STOPP criterion and 19% to a START criterion (Table 3),
but a considerable part of the identified DRPs could not be
related to a STOPP or START criterion (e.g. practical
medication problems, changes in dosage or evaluation of drug
The majority of the patients indicated that they could ask
(almost) all questions and understood (almost) everything
during the consultation with the GP.
The implementation rate of proposed medication changes
influenced by patient input was significantly higher as
compared to the implementation rate of proposed changes
not influenced by patient input (respectively 60 and 46%,
p < 0.001).
GPs considered the increased attention for polypharmacy,
medication reviews, and the recently published Dutch
multidisciplinary guideline on polypharmacy [
and important for GP care. CMRs were not performed for
patients in the control practices, therefore contamination was
The embedding of the Opti-Med intervention varied
between GP practices. GPs and practice nurses reported
less complaints and questions from patients when a practice
nurse was specifically assigned to the organization of the
intervention. GPs mentioned that personnel changes during
the course of the study was a barrier for the continuity and
implementation of the intervention.
For all key intervention components the implementation
fidelity was moderate to high. Almost all key intervention
components were generally carried out as planned.
However, for the elements patient selection and preparation of
the CMR analyses the researchers were more involved than
intended. Almost half of the proposed changes in medication
were implemented, starting new medications seemed easier
teams. Retrospectively, the researchers identified 1212 drug related
problems with the DOCUMENT tool [
], out of these proposals
Table 3 Prevalence of STOPP-START among intervention patients
per DOCUMENT DRP type
ADR adverse drug reaction, DRP drug related problem, START
screening tool to alert doctors to right treatment, STOPP screening
tool of older person’s prescriptions
DRPs were identified by the expert team at baseline and classified by
the researchers according to the validated DOCUMENT [
classification system to categorize DRPs into 8 categories. Retrospectively,
STOPP and START criteria were assigned to the DRPs
than stopping medications. Patient involvement may also be
considered accomplished as planned, one-fifth of the
proposed medication changes was influenced by patient input.
Training of the expert teams, the use of the STRIPA tool
and the structured PTP forms facilitated implementation
of the intervention. Difficulties with patient selection due
to non user-friendly software and incomplete medical and
medication files used for the medication analyses appeared
factors promoting non-adherence to the intervention. The
reproducibility of the medication analyses between the
expert teams was moderate. There were differences in the
embedding of the intervention between GP practices. A
designated and motivated practice nurse was an important
contextual facilitating factor for adherence to the intervention.
To our knowledge, this is one of the first comprehensive
process evaluations of a CMR intervention study. Other
studies on CMRs did not included or only a limited process
evaluation or a different method of CMR [
comparison with previous studies is therefore difficult, however,
some results can be compared.
The implementation rate of proposed medication changes
of almost 50% is within the range found in other studies [
], higher implementation rates may be found when the
patient’s own pharmacist and GP are involved in the
medication analysis and less non-relevant recommendations may
be formulated. However, GPs did not experience the
irrelevant recommendations as inefficient and time consuming
and reported that this disadvantage often was outweighed by
the advantage of the efficiency, objectivity and expertise of
the external expert team.
The 94 min time spent is acceptable compared to other
studies and estimations in guidelines [
]. Almost a
quarter of the time is spent by the practice nurse instead
of the GP and/or pharmacist, which is less costly.
However the time investment is still considerable, but may
reduce over time. A previous study with Opti-Med data
shows that the expert teams can improve the efficiency
over time [28.]
The moderate reproducibility of the medication analyses
between the expert teams could be partly explained by
variations among experts. In a recent Dutch qualitative study
on case vignettes with polypharmacy and multimorbidity, it
was concluded that GPs varied in medication management
strategies which resulted in differences in proposed
medication changes [
Lessons learned for CMRs in a non‑RCT setting
This process evaluation provides a better insight into the
implementation fidelity of an innovative method for CMRs.
Implementation fidelity was studied alongside a pragmatic
cluster RCT, which does not resemble daily practice. E.g.,
the efforts and time investment of the researchers are
applicable in daily practice.
As the selection of patients and preparation of the CMRs
in this study was mainly performed by researchers there
are still some barriers to overcome before these key
intervention components can be successfully implemented in
daily practice. Time, training and dedication of a practice
assistant or practice nurse in the GP practice for CMRs
The medication analyses being performed by external
expert teams seems feasible, however reimbursement and
organization of expert teams outside the scope of a research
project will be necessary. Currently in The Netherlands GPs
and pharmacists are reimbursed for conducting CMRs. A
dedicated coordinator may be needed to organise the work
of expert teams within e.g. an existing regional collaboration
structure between GPs and/or pharmacists.
Reimbursements for the GPs and reminders by the
researchers for GPs and patients may have increased the
implementation rate of the GP consultations. Of the invited
patients, almost 60% did not reply or indicated that they did
not want to participate. It might be that in daily practice, a
part of this group may need a different approach with
possibly more face-to-face contact to identify the actual
medication intake, DRPs and preferences.
Identified barriers for implementation in daily practice,
such as time restrains and incompleteness of medical files
are commonly known from other pharmaceutical care
studies or evaluation projects [
8, 24, 30
Several limitations may have influenced the evaluation of
the adherence to the intervention and moderating factors
determining the implementation fidelity of the intervention.
First, the researchers who carried out the Opti-Med
intervention were also involved in the process evaluation. We
used a subjective rating to measure implementation
fidelity, an objective rating is impossible in this type of process
Second, as compared to the framework of Hasson the
moderating factors ‘comprehensiveness of the policy
description’ and ‘recruitment’ have not been included in
the present evaluation. Comprehensiveness of the policy
description was not assessed since the number of key
components in the intervention is limited and it was not feasible to
obtain an external assessment of the policy description with
respect to the complex intervention. Recruitment is covered
under the adherence dimension ‘coverage’. Furthermore,
not all dimensions of adherence and of the moderating
factors have been assessed extensively. The assessment of the
quality of delivery of the intervention for GP consultations
and patient involvement was very limited. Video recordings
of consultations might have provided more insight into the
quality of delivery. The duration and topic list of the GP
interview was limited. Finally, results from a patient survey
gave us only limited insight into the patients’ responsiveness
and quality of delivery of the patient involvement, compared
to e.g. qualitative patient interview data.
Overall, the implementation fidelity was moderate to high
for all key intervention components of the CMR
intervention. This means that almost all intervention key
components were delivered as intended. The absence of its
effectiveness with respect to enhancing quality of life cannot be
explained by insufficient implementation fidelity.
Nevertheless, this process evaluation provides insight into how this
method of conducting CMRs can be implemented in daily
practice. Barriers on organizational level must be overcome;
the availability of user-friendly software, easy exchange of
medical and medication data, and coordination and
management of the intervention within a larger collaboration
between GPs and pharmacists are very important for
Acknowledgements We would like to thank all GPs, GP employees
and expert team members who facilitated data collection for this
project. Furthermore, thanks to Melek Dogdu (MD), Sabri Yigit (SY) for
their help with coding respectively the focus group and semi-structured
interviews transcripts. Hanna van Daal (HvD), thank you for your work
with the assessment of all STOPP and START criteria and data entry.
Funding This study was funded by a research grant by the Dutch
Organization for Health Research and Development (ZonMw).
Conflicts of interest All authors declare that they have no conflict of
Ethics approval This study was approved by the Medical Ethics
Committee of the VU University Medical Center (approval reference
2011/408) Informed consent was obtained from all individual
participants included in the study.
1. Carroll C , Patterson M , Wood S , Booth A , Rick J , Balain S. A conceptual framework for implementation fidelity . Implement Sci . 2007 ; 2 : 40 .
2. Hasson H . Systematic evaluation of implementation fidelity of complex interventions in health and social care . Implement Sci . 2010 ; 5 : 67 .
3. Clyne W , Blenkinsopp A , Seal R. A guide to medication review . UK: NHS National Prescribing Centre ; 2008 .
4. Cooper JA , Cadogan CA , Patterson SM , Kerse N , Bradley MC , Ryan C , et al. Interventions to improve the appropriate use of polypharmacy in older people: a cochrane systematic review . BMJ Open . 2015 ; 5 ( 12 ): e009235 .
5. NHS Cumbria Medicines Management Team . Clinical medication review: a practice guide . UK , 2013 .
6. Pharmaceutical Society of Australia. Guidelines for pharmacists providing home medicines review (HMR) services . Australia, 2011 .
7. Nederlands Huisarts Genootschap (NHG). Multidisciplinaire richtlijn Polyfarmacie bij ouderen. (Multidisciplinary guideline Polypharmacy for elderly) . The Netherlands , 2012 .
8. de Bruijne MC , Kemper PF , Bakker L , Heeren MJ , Meijler AP , Dolwel GO , et al. Farmacotherapeutische zorg voor voor kwetsbare ouderen met polyfarmacie (Pharmaceutical care for frail elderly with poypharmacy) . Amsterdam, 2014 .
9. Rosenthal M , Holmes E , Banahan B , III. Making MTM implementable and sustainable in community pharmacy: Is it time for a different game plan? Res Soc Adm Pharm . 2015 .
10. Clyne B , Fitzgerald C , Quinlan A , Hardy C , Galvin R , Fahey T , et al. Interventions to address potentially inappropriate prescribing in community-dwelling older adults: a systematic review of randomized controlled trials . J Am Geriatr Soc . 2016 ; 64 ( 6 ): 1210 - 22 .
11. Willeboordse F , Hugtenburg JG , van DL , Bosmans JE , de Vries OJ , Schellevis FG , et al. Opti-Med: the effectiveness of optimised clinical medication reviews in older people with 'geriatric giants' in general practice; study protocol of a cluster randomised controlled trial . BMC Geriatr . 2014 ; 14 : 116 .
12. Willeboordse F , Schellevis FG , Chau SH , Hugtenburg JG , Elders PJ . The effectiveness of optimised clinical medication reviews for geriatric patients: opti-med a cluster randomised controlled trial . Fam Pract . 2017 ; 34 ( 4 ): 437 - 45 .
13. Meulendijk MC , Spruit MR , Drenth-van Maanen AC , Numans ME , Brinkkemper S , Jansen PA , et al. Computerized decision support improves medication review effectiveness: an experiment evaluating the STRIP assistant's usability . Drugs Aging . 2015 ; 32 ( 6 ): 495 - 503 .
14. Chrischilles E , Rubenstein L , Van Gilder R , Voelker M , Wright K , Wallace R . Risk factors for adverse drug events in older adults with mobility limitations in the community setting . J Am Geriatr Soc . 2007 ; 55 ( 1 ): 29 - 34 .
15. Dros J , Maarsingh OR , Beem L , van der Horst HE , ter Riet G , Schellevis FG , et al. Functional prognosis of dizziness in older adults in primary care: a prospective cohort study . J Am Geriatr Soc . 2012 ; 60 ( 12 ): 2263 - 9 .
16. Larson EB , Kukull WA , Buchner D , Reifler BV . Adverse drug reactions associated with global cognitive impairment in elderly persons . Ann Intern Med . 1987 ; 107 ( 2 ): 169 - 73 .
17. Ruby CM , Hanlon JT , Boudreau RM , Newman AB , Simonsick EM , Shorr RI , et al. The effect of medication use on urinary incontinence in community-dwelling elderly women . J Am Geriatr Soc . 2010 ; 58 ( 9 ): 1715 - 20 .
18. Tinetti ME , Inouye SK , Gill TM , Doucette JT . Shared risk factors for falls, incontinence, and functional dependence. Unifying the approach to geriatric syndromes . JAMA . 1995 ; 273 ( 17 ): 1348 - 53 .
19. Woolcott JC , Richardson KJ , Wiens MO , Patel B , Marin J , Khan KM , et al. Meta-analysis of the impact of 9 medication classes on falls in elderly persons . Arch Intern Med . 2009 ; 169 ( 21 ): 1952 - 60 .
20. Willeboordse F , Grundeken LH , van den Eijkel LP , Schellevis FG , Elders PJ , Hugtenburg JG . Information on actual medication use and drug-related problems in older patients: questionnaire or interview? Int J Clin Pharm . 2016 ; 38 ( 2 ): 380 - 7 .
21. Williams M , Peterson GM , Tenni PC , Bindoff IK , Stafford AC . DOCUMENT: a system for classifying drug-related problems in community pharmacy . Int J Clin Pharm . 2012 ; 34 ( 1 ): 43 - 52 .
22. Atlas . ti version 7 .5.16 Gmbh [Computer software]. Berlin.
23. Allard J , Hebert R , Rioux M , Asselin J , Voyer L . Efficacy of a clinical medication review on the number of potentially inappropriate prescriptions prescribed for community-dwelling elderly people . CMAJ . 2001 ; 164 ( 9 ): 1291 - 6 .
24. Clyne B , Cooper JA , Hughes CM , Fahey T , Smith SM , Team O- Ss . A process evaluation of a cluster randomised trial to reduce potentially inappropriate prescribing in older people in primary care (OPTI-SCRIPT study) . Trials . 2016 ; 17 ( 1 ): 386 .
25. Kwint HF , Bermingham L , Faber A , Gussekloo J , Bouvy ML . The relationship between the extent of collaboration of general practitioners and pharmacists and the implementation of recommendations arising from medication review: a systematic review . Drugs Aging . 2013 ; 30 ( 2 ): 91 - 102 .
26. Chau SH , Jansen AP , van de Ven PM , Hoogland P , Elders PJ , Hugtenburg JG. Clinical medication reviews in elderly patients with polypharmacy: a cross-sectional study on drug-related problems in the Netherlands . Int J Clin Pharm . 2016 ; 38 ( 1 ): 46 - 53 .
27. Mast RM , Schouten GP , van Woerkom M. Niveau van medicatiebeoordeling initiatieven in Nederland kan beter. (Room for improvement in medication review initiatives in the Netherlands) . Pharm Weekbl Sci . 2010 ; 4 ( 11 /12): 189 - 94 .
28. Meulendijk MC , Spruit MR , Willeboordse F , Numans ME , Brinkkemper S , Knol W , et al. Efficiency of clinical decision support systems improves with experience . J Med Syst . 2016 ; 40 ( 4 ): 76 .
29. Sinnige J , Korevaar JC , van LJ , Westert GP , Schellevis FG , Braspenning JC . Medication management strategy for older people with polypharmacy in general practice: a qualitative study on prescribing behaviour in primary care . Br J Gen Pract . 2016 ; 66 ( 649 ): e540 - 51 .
30. Reeve E , Andrews JM , Wiese MD , Hendrix I , Roberts MS , Shakib S. Feasibility of a patient-centered deprescribing process to reduce inappropriate use of proton pump inhibitors . Ann Pharmacother . 2015 ; 49 ( 1 ): 29 - 38 .