- Research article
- Open Access
- Open Peer Review
What’s the uptake? Pragmatic RCTs may be used to estimate uptake, and thereby population impact of interventions, but better reporting of trial recruitment processes is needed
BMC Medical Research Methodology volume 17, Article number: 174 (2017)
Effectiveness of interventions in pragmatic trials may not translate directly into population impact, because of limited uptake by clinicians and/or the public. Uptake of an intervention is influenced by a number of factors.
We propose a method for calculating population impact of clinical interventions that accounts for the intervention uptake. We suggest that population impact may be estimated by multiplying the two key components: (1) the effectiveness of the intervention in pragmatic trials (trial effect); and, (2) its uptake in clinical practice. We argue that participation rates in trials may be a valid proxy for uptake in clinical practice and, in combination with trial effectiveness estimates, be used to rank interventions by their likely population impact. We illustrate the method using the example of four interventions to decrease antibiotic prescription for acute respiratory infections in primary care: delayed prescribing, procalcitonin test, C-Reactive Protein, shared decision making.
In order to estimate uptake of interventions from trial data we need detailed reporting on the recruitment processes used for clinician participation in the trials. In the antibiotic prescribing example, between 75 and 91% of the population would still be prescribed or consume antibiotics because effective interventions were not taken up. Of the four interventions considered, we found that delayed prescribing would have the highest population impact and shared decision making the lowest.
Estimates of uptake and population impact of an intervention may be possible from pragmatic RCTs, provided the recruitment processes for these trials are adequately reported (which currently few of them are). Further validation of this method using empirical data on intervention uptake in the real world would support use of this method to decide on public funding of interventions.
The effectiveness of an intervention in real clinical practice may be estimated in pragmatic trials conducted on patients who represent the full spectrum of the population to which the intervention might be applied, and where the comparator group receives usual care [1, 2]. Pragmatic trials are designed to determine the effects of an intervention under the usual conditions in which it will be applied,  and focus on the choice between options for care rather than biological understanding . They may be contrasted to explanatory trials designed to determine the effects of an intervention under the ideal conditions,  in order to test causal research hypotheses, such as whether the intervention causes a particular biological effect . Tools have been developed to help trialists with design decisions on how pragmatic or explanatory they wish their trial to be, [5, 6] and an extension of the CONSORT statement guides the reporting of pragmatic trials .
Estimates of intervention effectiveness may then be made using an intention to treat analysis of outcomes in the intervention group compared to those in the usual care group (‘trial effectiveness’) . That is, patients are analysed in the group to which they were initially randomised, even if they drop out of the study or change groups . But within trial effectiveness do not translate directly into population impact . One important difference is uptake by clinicians and/or the public . For example, for the long-term follow-up after colorectal cancer treatment, colonoscopy is the most sensitive method. But the less sensitive faecal occult blood testing has greater population impact because patients are willing to complete the four rounds of testing – that is, it has higher uptake . Uptake of an intervention is influenced by a number of factors; for example for clinician uptake, these range from knowledge of the intervention and the skills to implement it, through to emotion regulation and beliefs about the intervention itself .
In this paper, we propose a method for calculating population impact of clinical interventions that accounts for the intervention uptake. We suggest that population impact may be estimated by multiplying the two key components: (1) the effectiveness of the intervention in pragmatic trials; and, (2) its uptake in clinical practice. That is,
population impact = trial effectiveness x rate of uptake.
Obtaining the trial effectiveness is straightforward from either primary pragmatic randomised controlled trials (RCTs) or meta-analyses of pragmatic RCTs. However, obtaining the uptake of an intervention by clinicians is more difficult. The best estimates come from studies measuring this directly (such as surveys of clinicians or patients), but these are rare . An alternative indirect approach is to use data derived from the recruitment process of relevant pragmatic RCTs. Below, we illustrate how this can answer the following question relevant to a public health problem:
“What intervention to minimise antibiotic prescribing for acute respiratory infections in primary care is likely to have the largest effect at a population-level?”
Example using RCTS to estimate population effectiveness
Interventions to minimise antibiotic prescriptions for acute respiratory infections in primary care.
Meta-analyses of pragmatic trials show that 4 interventions reduce antibiotic prescribing and/or consumption for acute respiratory infections in primary care:
Shared decision making
We calculated the population impact of each intervention using a 3-step process:
First, we extracted the meta-analytic estimates of the trial effectiveness of the interventions (two extractors). Where these were not presented in the original meta-analysis, we undertook new analyses in RevMan 5.3  to obtain relative risks (with random effects models) and confidence intervals.
Second, we calculated the rate of uptake. From each systematic review we identified pragmatic RCTs that reported the number of clinicians invited to particpate, and those who did participate. We calculated the uptake by dividing the number of GPs who participated by those invited. Where this was reported for both GP practices, and GPs within participating practices, we calculated uptake by multiplying the two proportions. We excluded RCTs from this analysis that did not have a population based recruitment (e.g. only invited academic GPs attached to a University department). Where there was more than one RCT for estimating the uptake, we estimated the mean uptake rate and confidence intervals using a logistic regression model with random intercepts, using PROC NLMIXED in SAS software, version 9.4 . Where there was only one RCT, we calculated confidence intervals for a single proportion using the binomial distribution.
Finally, we calculated population impact of the intervention by multiplying the trial effectiveness by the uptake.
Table 1 summarises the 4 included systematic reviews that provided estimates of trial effectiveness and the 6 pragmatic RCTS (1 for delayed prescribing , 1 for procalcitonin , 1 for C-reactive protein [CRP] , and 3 for shared decision making [19,20,21]) that provided estimates of uptake. All reviews were on patients with acute respiratory infections, and the RCTs were conducted across several countries including Denmark, Germany and Canada. The RCTs were all primarily of pragmatic type, with high applicability to the actual clinical settings the intervention was intended to be used, and a very good match between the trial usual care arm and the intended primary care population .
Delayed prescribing had the highest trial effectiveness (relative risk reduction [RRR] of 64% in antibiotic consumption) and CRP the lowest (RRR 22% in antibiotics prescribed) (Table 2). New meta-analytic estimates were calculated for procalcitonin (RRR 61%) and delayed prescribing (RRR 64%). There was substantial heterogeneity between studies for both of these interventions (RRR range 42–74% for procalcitonin and 42–79% for delayed prescribing).
In contrast, CRP had the highest uptake rate (28%) and shared decision making the lowest (10%) (Table 2), meaning that between 72 and 90% of clinicians invited to participate did not wish to use these interventions. (We pooled data from 3 RCTS to estimate population uptake for shared decision making, and used data from 1 RCT to estimate uptake for the other 3 interventions).
Delayed prescribing had the highest population impact at 12% and shared decision making the lowest (4%) (Table 2, Fig. 1). For delayed prescribing, this means that if implemented, in 12% of clinical encounters where a person with an acute respiratory infection would otherwise have been prescribed antibiotics, the doctor would use (uptake) the intervention and the patient would not consume antibiotics (Table 2, Fig. 1), in 7% the doctor would use the intervention but the patient would still consume antibiotics, and in 81% the doctor would not use the intervention and the patient would consume antibiotics.
The most notable finding is that the greatest constraint to population impact is the poor uptake of interventions by clinicians rather than the effectiveness of the interventions themselves. Between 72 and 91% of the population would still be prescribed or consume antibiotics because effective interventions have not been taken up, Fig. 1. Of the four potential interventions shown to decrease antibiotic prescribing in RCTs, delayed prescribing is the intervention with the highest population impact, and could prove the best choice for implementation.
Challenges to this method
Better reporting of trial recruitment is needed
The poor reporting of both the uptake by clinicians, and of the methods of recruitment, limit our ability to estimate overall impact and hence inform health policy. In the antibiotic prescribing example, only some trials reported the number of general practices randomised out of those approached, and only a few reported on the number of clinicians who then participated out of the total eligible within each practice. No trials reported on the number of patients randomised out of the total number of eligible patients seen by each of the participating GPs.
Better reporting of both recruitment rates and recruitment methods are crucial to new RCTs. The extended CONSORT checklist  and flow diagram requires reporting of eligibility criteria to show the degree to which they include typical participants and/or typical providers, institutions, communities and settings of care, and the number of participants or units approached to take part in the trial, the number which were eligible, and reasons for non-participation . We recommend that criteria be added to this to include reporting on the methods of recruitment, including the dialogue and incentives, and the resulting recruitment rates. In this way, we can utilise data which are routinely collected in pragmatic trials but not currently publicly available, helping to reduce waste in research . Just as estimates of trial effectiveness may be context-dependent, so too may be the trial estimates of uptake, and results may not be easily translated across different settings. Ideally, estimates of uptake should be based on meta-analysis of multiple trials of the intervention, with exploration of heterogeneity between trials to identify potential facilitators and barriers of intervention uptake. This would allow estimates of population impact under best case and worse case scenarios as well as helping inform implementation strategies. The actual population impact may be lower than in these scenarios, as even when the clinician and patient decide to uptake the intervention, they are generally less adherent to this in real life than in the trial setting (and so the trial effectiveness overestimates the real life effectiveness).
Does uptake in RCTs predict uptake in clinical practice?
The uptake reported in pragmatic RCTs may be either under- or over-estimates: the additional perceived burden of the research element in RCTs may lead us to underestimate uptake; however, the extra support provided in the research setting may result in overestimates of uptake in the real world. These biases may favour one intervention over another. A further issue is that the willingness of clinicians to participate in pragmatic trials where the effectiveness of an intervention is unknown, may be different to the uptake in clinical practice when effectiveness has been established. For instance, interventions that have been shown to be effective may be promoted or even mandated in ways which would affect participation in practice. However, in the early pre-implementation stage where we are suggesting the method may be used to inform funding decisions (i.e. after the completion of the trials, but before a decision to publically fund and/or promote the intervention has been made), these considerations may not be as important.
Validation of our method may be possible for interventions that have previously received public funding. For example, a large prospective cohort study found that 14% of UK based general practitioners used delayed prescribing of antibiotics for adults presenting with a sore throat  – this is not dissimilar to our estimated uptake rate of 19% (95% CI 14–24%). We could not find population based studies on uptake of the other three interventions in clinical populations similar to those in trials. One nationwide cross-sectional study found that 70% of Norwegian GPs used CRP for children aged 5 years or under who presented with a respiratory infection , but use of CRP for adult patients is not known. We estimated that CRP had an uptake rate of 28% (95% CI 20–36%) for use in adults, and although this was the highest uptake out of the 4 interventions we studied, it may still be an underestimate. Prospective evaluation as part of implementation trials could also help us work out if our uptake estimates are reflective of “real-life” uptake rates. If such comparisons support the use of our method for ranking interventions on their likely uptake in clinical practice, then this method might also be used to decide on public funding of interventions at the outset.
For estimating the population impact of interventions, knowing the uptake rate may be as important as the trial effectiveness, but is not usually considered when deciding on public funding. Estimates of uptake may be possible from pragmatic RCTs of the intervention, provided the recruitment processes for these trials are adequately reported (which currently few of them are).
Consolidated Standards of Reporting Trials
General Practitioner, equivalent to Primary Care Physician
Randomised Controlled Trial
Relative risk reduction
Sackett DL. The Principles behind the tactics of performing therapeutic trials. In: Haynes RB, Sackett DL, Guyatt GH, Tugwell P, editors. Clinical epidemiology: how to do clinical practice research. Philadelphia: Lippincott, Williams and Wilkins; 2006. p. 183–8.
Godwin M, Ruhland L, Casson I, MacDonald S, Delva D, Birtwhistle R, Lam M, Seguin R. Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol. 2003;3:28.
Dawson L, Zarin DA, Emanuel EJ, Friedman LM, Chaudhari B, Goodman SN. Considering Usual Medical Care in Clinical Trial Design. PLoS Med. 2009;6:e1000111. https://0-doi-org.brum.beds.ac.uk/10.1371/journal.pmed.1000111.
Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis. 1967;20:637–48.
Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, Tunis S, Bergel E, Harvey I, Magid DJ, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62:464–75.
Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147.
Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:a2390.
Roland M, Torgerson DJ. Understanding controlled trials: what are pragmatic trials? BMJ. 1998;316:285.
Glasgow RE, Lichtenstein E, Marcus AC. Why don't we see more translation of health promotion research to practice? Rethinking the efficacy-to-effectiveness transition. Am J Public Health. 2003;93:1261–7.
Glasziou P, Haynes B. The paths from research to improved health outcomes. Evid Based Med. 2005;10:4–7.
Grobbee EJ, van der Vlugt M, van Vuuren A, Stroobants AK, Mallant-Hent R, Lansdorp-Vogelaar I, Bossuyt P, Kuipers EJ, Dekker E, Spaander M. 176 comparison of colonoscopy. Sigmoidoscopy and Multiple Rounds of FIT-Based Colorectal Cancer Screening: Long-Term Follow-Up Gastroenterol. 2016;150:S45.
Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A. On behalf of the “psychological theory” group. Making psychological theory useful for implementing evidence based practice: a consensus approach. Qual Saf Health Care. 2005;14:26–33.
Little P, Stuart B, Hobbs FDR, Butler CC, Hay AD, Delaney B, Campbell J, Broomfield S, Barratt P, Hood K, et al. Antibiotic prescription strategies for acute sore throat: a prospective observational cohort study. Lancet Infect Dis. 2014;14:213–9.
Review Manager (RevMan) [Computer program]. Version 5.3. Copenhagen: The Nordic Cochrane Centre, The Cochrane Collaboration, 2014.
SAS software, Version 9.4 of the SAS System for Unix. Copyright © 2002–2012 by SAS Institute Inc. SAS and all other SAS Institute Inc. product or service names are registered trademarks or trademarks of SAS Institute Inc., Cary, NC, USA.
Dowell J, Pitkethly M, Bain J, Martin S. A randomised controlled trial of delayed antibiotic prescribing as a strategy for managing uncomplicated respiratory tract infection in primary care. BJGP. 2001;51:200–5.
Briel M, Schuetz P, Mueller B, Young J, Schild U, Nusbaumer C, Periat P, Bucher HC, Christ-Crain M. Procalcitonin-guided antibiotic use vs a standard approach for acute respiratory tract infections in primary care. Arch Intern Med. 2008;168:2000–7.
Diederichsen HZ, Skamling M, Diederichsen A, Grinsted P, Antonsen S, Petersen PH, Munck AP, Kragstrup J. Randomised controlled trial of CRP rapid test as a guide to treatment of respiratory infections in general practice. Scand J Prim Health Care. 2000;18:39–43.
Briel M, Langewitz W, Tschudi P, Young J, Hugenschmidt C, Bucher HC. Communication training and antibiotic use in acute respiratory tract infections. A cluster randomised controlled trial in general practice. Swiss Med Wkly. 2006;136:241–7.
Altiner A, Brockmann S, Sielk M, Wilm S, Wegscheider K, Abholz HH. Reducing antibiotic prescriptions for acute cough by motivating GPs to change their attitudes to communication and empowering patients: a cluster-randomized intervention study. J Antimicrob Chemother. 2007;60:638–44.
Legare F, Labrecque M, Cauchon M, Castel J, Turcotte S, Grimshaw J. Training family physicians in shared decision-making to reduce the overuse of antibiotics in acute respiratory infections: a cluster randomized trial. CMAJ. 2012;184:E726–34.
Rennie D. CONSORT revised - improving the reporting of randomized trials. JAMA. 2001;285:2006–7.
Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S, Michie S, Moher D, Wager E: Reducing waste from incomplete or unusable reports of biomedical research. Lancet. 383:267–276.
Rebnord IK, Hunskaar S, Gjesdal S, Hetlevik Ø. Point-of-care testing with CRP in primary care: a registry-based observational study from Norway. BMC Fam Pract. 2015;16(1):170.
Spurling GK, Del Mar CB, Dooley L, Foxlee R, Farley R: Delayed antibiotics for respiratory infections. Cochrane Database Syst Rev. 2013,4:Cd004417.
Schuetz P, Muller B, Christ-Crain M, Stolz D, Tamm M, Bouadma L, Luyt CE, Wolff M, Chastre J, Tubach F et al: Procalcitonin to initiate or discontinue antibiotics in acute respiratory tract infections. Cochrane Database Syst Rev. 2012,9:Cd007498.
Aabenhus R, Jensen JU, Jorgensen KJ, Hrobjartsson A, Bjerrum L. Biomarkers as point-of-care tests to guide prescription of antibiotics in patients with acute respiratory infections in primary care. Cochrane Database Syst Rev.. 2014,11:Cd010130.
Coxeter P, Del Mar CB, McGregor L, Beller EM, Hoffmann TC. Interventions to facilitate shared decision making to address antibiotic use for acute respiratory infections in primary care. Cochrane Database Syst Rev. 2015:CD010907.
The authors have received funding from the Australian National Health and Medical Research Council (Early Career Fellowship No. 1013390, Australia Fellowship No. 527500 and Centre for Research Excellence No. 1044904). The funders had no role in design and conduct of the study; collection, management, analysis, and interpretation of the data; and preparation, review, or approval of the manuscript.
Availability of data and materials
No additional data available.
Ethics approval and consent to participate
Ethical approval was not required for this paper which only used published data from other studies.
Consent for publication
We have no competing interests to declare.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Bell, K.J.L., McCullough, A., Del Mar, C. et al. What’s the uptake? Pragmatic RCTs may be used to estimate uptake, and thereby population impact of interventions, but better reporting of trial recruitment processes is needed. BMC Med Res Methodol 17, 174 (2017) doi:10.1186/s12874-017-0443-0
- Health policy
- Public health
- Primary health care
- Pragmatic clinical trial
- Drug resistance, microbial