Embedding mentoring to support trial processes and implementation fidelity in a randomised controlled trial of vocational rehabilitation for stroke survivors

Abstract

Background

Little guidance exists regarding how best to upskill and support those delivering complex healthcare interventions to ensure robust trial outcomes and implementation fidelity. Mentoring was provided to occupational therapists (OTs) delivering a complex vocational rehabilitation (VR) intervention to stroke survivors. This study aimed to explore mentors’ roles in supporting OTs with intervention delivery and fidelity, and to describe factors affecting the mentoring process and intervention delivery.

Methods

Quantitative data (duration, mode and total time of mentoring support) was extracted from mentoring records and emails between mentors and OTs, alongside qualitative data on barriers and facilitators to intervention delivery. Semi-structured interviews with mentors (n = 6) and OTs (n = 19) explored experiences and perceptions of intervention training, delivery and the mentoring process. Mean total and monthly time spent mentoring were calculated per trial site. Qualitative data were analysed thematically.

Results

Forty-one OTs across 16 sites were mentored between March 2018 and April 2020. Most mentoring was provided by phone or Microsoft Teams (range: 88.6–100%), with the remainder via email and SMS (Short Message Service) text messages. Mentors suggested strategies to enhance trial recruitment, improved OTs’ understanding of- and adherence to trial processes, intervention delivery and fidelity, and facilitated independent problem-solving. Barriers to mentoring included OT non-attendance at mentoring sessions and mentors struggling to balance mentoring with clinical roles. Facilitators included support from the trial team and mentors having protected time for mentoring.

Conclusions

Mentoring supported mentee OTs in various ways, but it remains unclear to what extent the OTS would have been able to deliver the intervention without mentoring support, or how this might have impacted fidelity. Successful implementation of mentoring alongside new complex interventions may increase the likelihood of intervention effectiveness being observed and sustained in real-life contexts. Further research is needed to investigate how mentors could be selected, upskilled, funded and mentoring provided to maximise impact. The clinical- and cost-effectiveness of mentoring as an implementation strategy and its impact on fidelity also requires testing in a future trial.

Trial registration

ISRCTN, ISRCTN12464275. Registered on 13th March 2018.

Background

Trialists are responsible for ensuring that those delivering trial interventions are adequately prepared and supported to implement them with fidelity to ensure robust trial outcomes [1] and optimal participant outcomes [2]. While guidance describing best practice in trial management and the role of healthcare staff to deliver trial interventions exists [3], evidence on how best to upskill and support staff to deliver trial interventions as opposed to using experts to deliver it is lacking. This is particularly pertinent in the UK, where the funding infrastructure separates research and intervention costs and there is a requirement to use National Health Service (NHS) staff to deliver trial interventions. The premise is worthy, if staff can be trained to deliver trial interventions in real-life contexts, trial findings will be contextually relevant and may rapidly translate into practice. However, there are many obstacles which interfere with intervention delivery [4, 5] that need addressing to maximise trial outcomes.

Multiple strategies may be needed to support staff because barriers to delivering complex interventions with fidelity exist at organisational and practitioner levels [1, 6,7,8], although evidence on the effectiveness of these strategies is limited [9]. In a systematic review, barriers to fidelity in delivery of rehabilitation to people with long-term neurological conditions included lack of availability of training and therapists’ lack of confidence to deliver new interventions, especially where there was a gap between training and initiating intervention delivery [4]. Implementing new interventions in clinical practice or in a trial context may require changes to the staff member’s knowledge, skills, confidence, and attitudes [10, 11], and how they put new learning into practice is complex [9, 12,13,14]. The Cochrane Effective Practice and Organisation of Care (EPOC) Group categorised implementation strategies in a taxonomy [15, 16], including those targeting healthcare staff such as occupational therapists (OTs). Implementation strategies in stroke rehabilitation commonly included educational materials and meetings, outreach visits and support from opinion leaders [17], i.e., trustworthy individuals educating staff on ways to best apply evidence to inform their practice [18].

A novel approach to support intervention delivery was taken in the Facilitating Return to work through Early Specialist Health (FRESH) feasibility trial of vocational rehabilitation (VR) for traumatic brain injury survivors [19]. A training package comprising face-to-face teaching, an intervention manual and monthly mentoring for NHS OTs delivering the intervention was provided. Key components of mentoring included determining whether the OTs were delivering the intervention as intended and providing feedback to encourage implementation fidelity. One unintended benefit was that the mentoring led to early identification of trial-related issues, timely trouble-shooting and local problem-solving [5]. The FRESH mentoring approach was then adapted to provide clinical supervision and promote implementation fidelity among OTs in the UK-based RETurn to work After stroKE (RETAKE) trial [20].

The RETAKE trial aims to determine whether providing early stroke-specialist vocational rehabilitation plus usual NHS rehabilitation is more clinically and cost-effective for supporting post-stroke return to work than usual care (UC) alone [20]. This complex intervention aligns with current NHS outcomes, government policy and clinical guidelines for return to work after stroke [21,22,23,24,25], and is described elsewhere [26]. The RETAKE intervention is individually tailored, commencing within 12 weeks of stroke onset and delivered in participants’ homes or workplaces by NHS OTs for up to 12 months. In the RETAKE trial, two OTs were recruited to deliver the intervention per NHS site. This study was completed as part of the RETAKE trial process evaluation; a mixed-methods approach was taken to describe and explore the mentoring provided to RETAKE OTs.

Methods

The aims were:

• 1) To explore mentors’ roles in supporting OTs with intervention delivery and implementation fidelity.

• 2) To describe factors affecting the mentoring process and intervention delivery.

The qualitative sections of this paper are reported in accordance with the Standards for Reporting Qualitative Research (SRQR) guideline [27]. The methods described henceforth were conducted in alignment with the principles of Good Clinical Practice (GCP) [28], and Declaration of Helsinki (1996) [29]. Further details of ethical approval and participant consent are included in the declarations section.

Training and mentoring in the RETAKE trial

A description of the core elements of the RETAKE mentoring approach can be found in Additional file 1. Prior to intervention delivery, OTs attended a two-day educational meeting where they were provided with evidence on the trial intervention and taught how to deliver it and complete the trial paperwork. Educational materials were provided (e.g., intervention manual, letter and report templates). The OTs’ competency to deliver the RETAKE intervention was assessed via case vignettes immediately following training and 6 months post-randomisation of their first participant. Low competency scores indicated that additional training/mentoring support was required, such as email support or one to one phone calls alongside group mentoring. Following initial intervention training, monthly group mentoring sessions were provided for all OTs via teleconference or Microsoft Teams. Attendees at each session included a mentor and OTs across two trial sites. Following each session, mentors completed an electronic mentoring record (Additional file 2) recording date and duration of the session, OT attendance (including reasons for non-attendance), issues and actions relating to RETAKE OTs, clinical matters, implementation of the intervention, and trial process issues. OTs could contact their mentors via phone, text or email for ad-hoc support outside of sessions; mentors recorded ad-hoc conversations via mentoring records or emails. A one-day refresher training was held 6 months later to reinforce the intervention process, facilitate peer sharing of resources and problem-solve implementation fidelity issues.

Mentors were purposively approached based on national recognition as experts, with substantial experience delivering VR to stroke and/or acquired brain injury patients. All were members of the Royal College of Occupational Therapists (RCOT) Specialist Section in Work or RCOT Specialist Section for Neurological Practice. All but one had been involved in supporting implementation of VR interventions in previous trials for people with traumatic brain injury [5] and stroke [30]. They were approached by the Chief Investigator and their time costed into the grant application. Two mentors held PhDs; they delivered RETAKE intervention training and had prior experience of supporting VR delivery in research. They were embedded within the RETAKE research team and attended trial management group meetings. All mentors received training in the RETAKE mentoring process, potential sources of contamination between trial arms and how to reduce contamination risks, and how to use teleconferencing to deliver mentoring. The mentors’ demographic details are presented in Table 1.

Table 1 Demographic details of mentors and OTs interviewed

Sampling and recruitment

All six mentors were invited to participate in interviews because they had first-hand experience of mentoring to support OTs’ and insight into real-world factors influencing the implementation of the RETAKE intervention. OTs were sampled in two ways, one OT from each site was purposively selected and invited to interview to ensure geographical spread. In addition, some OTs delivering the intervention to participants randomly selected as ‘longitudinal case studies’ were interviewed.

Prior to interviews, mentors and OTs were given study information and opportunity to ask questions. Mentors were geographically dispersed, so consent was obtained verbally at the time of interview. OTs consented in writing at the trial outset, to provide information for the trial duration.

Data collection

Semi-structured interviews

All interviews were semi-structured and audio-recorded. Interviews with mentors were conducted by two research assistants (RAs) via telephone or face-to-face for up to 1 h in February 2020. The purpose was to explore mentors’ thoughts on the mentoring process, including how it may have influenced trial processes and intervention fidelity. One RA interviewed the OTs face-to-face between May 2019 and February 2020 for up to 1 h. Interviews took place once OTs had started seeing participants, to explore their experiences of training, mentoring and intervention delivery. Interview topic guides were informed by Normalisation Process Theory (see Additional files 3 and 4).

Documentary analysis

The number of sessions, emails and mentoring time provided to each OT during the study period was extracted from mentoring records and emails. Duration for individual emails was recorded as 10 min, unless otherwise specified. Emails between OTs and mentors were collated by researchers on an ongoing basis, and relevant, anonymised qualitative data extracted. Data generated between March 2018 and April 2020 enabled data capture across multiple sites from site opening until participant discharge from the RETAKE intervention (12-months post-randomisation). Examination of mentoring records and emails enabled description of factors influencing implementation fidelity and intervention delivery, and exploration of different dimensions of mentoring support during the RETAKE trial.

Data collection and analyses were underpinned by Normalisation Process Theory (NPT) [31]. NPT has been used extensively within implementation research [32], and applied in the RETAKE process evaluation to evaluate how the intervention was implemented, embedded, and sustained. The core constructs of NPT (see Table 2) were applied in the current study because mentors advised OTs on intervention delivery throughout the RETAKE trial, and were thus central to the implementation process. Similarly, OTs’ experiences of training and access to mentoring provided direct insights into the implementation process.

Table 2 Definitions of the core constructs of NPT, adapted from May et al., 2015 [31]

Data analysis

Total mentoring time per OT and per site were summed from March 2018 until April 2020 (Table 3). Time spent per mode (e.g., phone/Microsoft Teams) was calculated, as well as monthly averages per site.

Table 3 Breakdown of mentoring support provided to OTs at RETAKE sites: March 2018–April 2020

Qualitative data from mentoring records, emails, and interviews with mentors and OTs were analysed in three separate thematic analyses [33] using Nvivo™ version 12 [34], adopting a staged approach to ensure trustworthiness. This approach, described in more detail below, was designed to ensure a consistent approach to analysis of each data set. Stages included independent coding, development of coding frameworks and thematic narrative summaries and then synthesis across the qualitative data sets. An inductive approach ensured coding and theme development were data-led. A subsequent deductive approach involved reviewing data against the research questions, NPT constructs and components, and intervention logic model. This combined approach was adopted to direct focus to the research questions, whilst allowing for any important, unexpected issues to be identified.

Analyses of interviews with mentors and OTs

Two RAs independently coded two mentor interview transcripts identifying themes and sub-themes; the same approach was taken with four OT interview transcripts. RAs then compared coding and agreed a provisional coding framework to be applied to the remaining interview transcripts. During analysis new nodes were added if indicated. One RA developed summary memos synthesising findings per theme using the deductive approach described above. Memos were reviewed by the team to clarify and confirm explanations.

Analysis of mentoring records and emails

Two RAs coded data extracted from the mentoring records and emails to identify provisional themes and sub-themes, and then met to review coding and agree themes and sub-themes. Themes and corresponding data were checked independently by a third RA, and disagreements resolved through discussion. Themes and sub-themes were then reviewed by the team.

Researcher characteristics and trustworthiness

RAs were Psychology graduates with qualitative research experience. To increase rigour, various measures were undertaken [35, 36]. To ensure credibility, annotations were made during coding and theme collation stages in analyses, recording RAs’ thoughts on patterns emerging through the data. RAs and senior researchers regularly discussed development of interpretations and plans for action. An RA examined findings across the qualitative and quantitative analyses and across trial sites as part of data triangulation [35] to address the following research questions: 1) How was mentoring delivered?; 2) How did mentors support OTs with intervention delivery?; and 3) What have we learned? Transferability was ensured through description of study boundaries, including: Number of sites contributing to the data; numbers of mentor and OT participants involved; and data collection methods. Confirmability was ensured through multiple RAs and senior research staff being involved in creation of the interview topic guides and in review of themes in qualitative analyses.

Results

Table 1 provides demographic details for mentors (n = 6) and OTs (n = 19) interviewed. Data were extracted from 184 mentoring records and 240 emails between March 2018 and April 2020.

Themes identified across the qualitative analyses related to delivery of mentoring, benefits of mentoring for OTs and for trial implementation, facilitators and barriers to mentoring, and implications for future provision of similar mentoring approaches. Themes, sub-themes and examples of data are presented as follows. Within the text and Table 4 NPT constructs and components have been highlighted, and influences on the mentoring process and RETAKE intervention delivery identified to help explain findings. Within the ‘delivery of the mentoring’ theme,’ reference has also been made to quantitative data from the mentoring records to demonstrate the mode and time spent mentoring each site per month.

Table 4 NPT constructs and components coded per sub-theme

Delivery of the mentoring

Coherence and cognitive participation: preparation of mentors to deliver the mentoring

In addition to training on the RETAKE mentoring process, mentors reported that attending training for OTs at trial outset improved their understanding of trial processes and the RETAKE intervention. This ensured coherence by facilitating shared understanding of the intervention’s aims, objectives and potential benefits. Attending the training also enabled mentors to meet OTs face-to-face and learn about their backgrounds and possible site-related challenges.

“It’s just nice to meet people and find out where they came from and also what the challenges were of working within their Trust.” [Interview with Mentor 1]

All six mentors had extensive experience delivering VR, often with focus on neurological conditions such as stroke or traumatic brain injury. This experience enabled reflection on times they had been in similar situations and use of those experiences to guide OTs.

“And obviously I’ve got experience to draw on as well so that if people hadn’t, you know, I can say, “Well, I had something similar and I found that this worked.” [Interview with Mentor 1]

The combination of mentors’ VR backgrounds and understanding of the trial, RETAKE intervention and OTs’ potential site-related challenges exemplifies cognitive participation because it suggests they were the right people to support delivery of the RETAKE intervention.

Collective action: monthly mentoring sessions and additional support

Two mentors experienced initial difficulties knowing which OTs were speaking when delivering mentoring by teleconference. They developed strategies for overcoming this, such as asking OTs to say each other’s names when speaking, making notes per OT, and planning an agenda.

“ … I’d never done that [teleconference mentoring]. So it was a little daunting for me and I had to sort of devise my own systems of how I recorded it and how it could be – how to recognise people’s voices … ” [Interview with Mentor 2]

The adaptability of mentors and OTs in devising strategies to overcome initial challenges demonstrates collective action, through their work to operationalise the group teleconference format.

Mentoring records and emails demonstrated 41 OTs across 16 trial sites participated in mentoring (Table 3). It was mostly provided by phone or Microsoft Teams (range across sites: 88.6 to 100%), with the remainder via email. Where reported, reasons for non-attendance included OT sickness, annual leave, clinical commitments, technical issues, or redeployment during the Covid-19 pandemic. During this period, 9 OTs across 7 sites left RETAKE to go on maternity leave (n = 2), start new jobs in other Trusts or services (n = 6), or left due to pressures from their usual clinical role (n = 1). As Table 3 indicates, site 2 received the least mentoring with OTs receiving, on average, 43 min per month during the study period. In this site one OT left in August 2019 and was never replaced. The remaining OT was a manager and reportedly didn’t have time to attend mentoring sessions. Three sites received close to the expected 60 min of mentoring per month (sites 9, 10, and 13), and twelve sites received more time (range of averages per month: 73–121 min), which included additional mentoring via phone/Microsoft Teams and email. Known reasons for additional support included larger caseloads (so more participants to be discussed) in sites 1 and 6, OTs being less confident about delivering the intervention (sites 3, 7, 8 and 15), low scores on RETAKE competency assessments (sites 1, 4, 5, 7, 11, 15, and 16), and site-specific issues related to set-up, funding, intervention delivery, and management issues (sites 7, 11, 12, and 14). OT turnover at 7 sites (sites 1, 4, 5, 6, 8, 11, and 15) meant that replacement OTs required more support because they were new to the trial and intervention delivery.

Qualitative analyses of interviews and mentoring records and emails demonstrated that ad-hoc support was provided to OTs via phone, email and/or SMS text message. In contrast to data in Table 4, two mentors interviewed described receiving infrequent queries for additional support, although another reported that some OTs habitually phoned her for advice. This might be partially explained by the fact that mentors arranged additional mentoring sessions for OTs unable to make the planned session, thus increasing the total amount of support provided. Additionally, two mentors left during the study period, due to long-term sickness or moving overseas.

Six of the 19 OTs reported that monthly mentoring sessions plus ad-hoc, additional support provided enough opportunity to engage with their mentor and other OTs.

“I think once a month is enough … But also, [mentor] has said that we can contact her in between. So if we've got something pressing, we can email her or have a conversation with her in between..” [OT 18 from site 10]

The benefits of mentoring for OTs and trial implementation

Coherence, cognitive participation and collective action: strategies to facilitate recruitment to the RETAKE trial

Delays in referrals prevented OTs from delivering early intervention to participants. One mentor reported that recruiters in one site lacked understanding as to why stroke survivors with low level impairments needed support for returning to work, demonstrating lack of understanding of the aims of the trial and intervention (i.e., coherence) among recruiters.

“And they [recruiters] never gave a thought to what happens about work because there was always another service that would deal with that in their minds. So asking them to think differently was quite a challenge.” [Interview with Mentor 2]

Mentoring records and emails indicated that recruitment was sometimes paused because recruiting staff did not have capacity to recruit new participants. In these instances, cognitive participation and collective action were demonstrated because mentors worked with OTs to define strategies to increase and sustain trial recruitment, e.g., OTs prompting recruiters to increase efforts and offering help in recruiting. Following this communication and by involvement of Principal Investigators (PIs), recruitment increased. Other solutions included clarification of eligibility queries and encouraging OTs to obtain consent to follow-up even if in doubt of eligibility.

Coherence: and collective action: support with trial processes

Across mentoring records and mentor interviews a key aspect of the mentors’ role at the trial outset included answering OT’s queries about trial processes, e.g., when and how to complete trial case reporting forms, writing letters to participants and employers. If mentors were uncertain they sought clarification from the trial team. One mentor described reassuring community-based OTs that they could visit participants in acute settings, ensuring participants were seen within trial timelines (initial assessments were due within 2 weeks of randomisation).

“I was like well, you know, it’s been agreed as part of the trial that you can in-reach and you can go and see people in A&E, and it just took a lot of sort of reassuring them (laughs) that they weren’t breaking any rules … .” [Interview with Mentor 2]

Mentors supported OTs in developing coherence by helping them understand their tasks and responsibilities in relation to the trial. According to mentors, the sessions provided a safe place to confidentially discuss caseloads. Mentoring records demonstrated that contamination issues (e.g., usual care therapists learning about and applying the RETAKE intervention) were discussed in sessions and actions planned. For example, seeking advice from the trial team, or advising the OT on direct action.

“We discussed how to avoid contamination and agreed; see the pt alone at the end of the session for voc [vocational rehabilitation] conversation. Educate the other therapists not to copy what has been said/done by the RETAKE OT and just note pt seen by RETAKE OT in standard notes.” [Email from mentor to mentee OT, 14/01/19]

Regular communication between OTs, mentors, and the trial team led to collective action through building of accountability and confidence in each other’s abilities (e.g., OTs avoiding contamination).

Mentoring records/emails also showed that OTs reported they did not always have enough time to complete trial documentation. Cognitive participation was demonstrated when mentors worked with OTs to collectively define practical solutions to speed up and sustain trial documentation .

“[RETAKE OTs] said they were writing letters after seeing pts but also writing continuation sheets... We discussed this and they agreed they could write ‘see letter’ on the continuation sheet … ” [Mentoring Record 21/11/18]

Coherence and collective action: support with applying newly acquired knowledge

Four mentors stated that many OTs needed their support to apply newly acquired knowledge of VR, and both mentors and mentoring records indicated that OTs were sometimes unsure what VR treatment plans would look like. Mentors supported OTs in developing coherence by them on the tasks and responsibilities required when creating VR treatment plans.

“Therapist unclear what voc [vocational] support to offer at this point. Reassured ok still very early stages. Talked through establishing job demands, using job description/break down of job, relating to current activity, how to establish grade plan of activity at home to test out and build up skills.” [Mentoring Record 06/08/18]

Mentors had extensive knowledge of resources to facilitate return to work after stroke. Mentoring records and emails indicated that many resources were provided by mentors to individual OTs, or to all OTs if seen as relevant to all. Collective action was evident with mentors advising OTs on use of these resources and signposting to relevant training to support their use.

Coherence, cognitive participation and collective action: support for delivering the intervention with fidelity

All 19 OTs interviewed demonstrated evidence of coherence, because they believed in the value of the RETAKE intervention. Their most frequently valued components including early intervention, case coordination and opportunity to build relationships between employers and participants. Ten OTs explained that mentoring facilitated intervention delivery through the advice and best practice examples shared. Interviews with mentors and mentoring records and emails indicated that mentors supported OTs in developing coherence by advising OTs on their tasks and responsibilities for delivering core components of the RETAKE intervention. Mentors also facilitated OTs’ adherence to the trial protocol during intervention delivery. For example, by advising OTs to start working with participants early (i.e., within 12 weeks of their stroke) and ensuring OTs did not discharge participants as soon as they had returned to work.

“One participant discharged after 2 weeks back at work, had spoken with manager who was happy with performance. Advised that I felt this was too early for discharge and that she [OT] should be establishing a mechanism for keeping in touch and follow up.” [Mentoring Record 20/11/18]

Mentors also provided guidance on participants’ access to benefits, tailoring interventions according to clinical assessment results, and working with employers.

“I think one of the common themes that comes up is around talking about how they [OTs] manage employers, or how they manage the interactions with employers, or employers who are not really engaging with them.” [Interview with Mentor 5]

OTs frequently lacked confidence in working with employers as it was unchartered territory for many. In addition to managing difficult or non-engaging employers, mentors supported OTs’ progression to cognitive participation by advising them on employer expectations, ensuring clarity in the structure, content and expected outcomes of interactions, and tailoring their communication styles. Four mentors explained that they provided reassurance to OTs, that what they were doing- or had done was appropriate or, “on the right track.” Mentors demonstrated cognitive participation by supporting OTs in believing they were contributing to the RETAKE trial. Further evidence for this came from OTs reporting occasional checks with mentors that they were following the right steps with intervention delivery, and from mentoring records and emails.

“You wanted to just to check you had thought of everything – which you had. Well done.” [Email from mentor to mentee OT, 05/02/2020]

A common issue was that OTs were hindered from starting the RETAKE intervention early, and delivering it at the required intensity, duration, and dose. Across mentor interviews and mentoring records and emails, common reasons included staff shortages and pressures from UC workloads. Three mentors and several mentoring records and emails highlighted unsupportive management issues, leading to increased pressures on OTs to prioritise UC over RETAKE, refusal to let OTs take on any more RETAKE participants, and pressure to drop out of RETAKE altogether. In these instances, coherence and cognitive participation did not seem evident among managers; mentors responded by engaging in collective action to support OTs.

“[RETAKE OT] had been told by their manager that their usual care caseloads takes priority as that was what they were commissioned to do but you rightly reminded them that the trust was being paid to carry out research so the RETAKE pts must be seen as prescribed.” [Email from mentor to mentee OT, 13/02/19]

OTs across two sites reported that where their managers had not been supportive it had been due to a change in management during the RETAKE study period, and the new managers not having had a formal introduction to RETAKE.

“ … with my new manager who didn’t know anything about Retake, she kind of inherited our team, I had to explain it all to her and I am not sure, because she has just got so much to do, they have restructured and she has got loads of teams now … I am not sure if she ever really kind of understood what was going on with it … ” [RETAKE OT 30, Site 15]

Cognitive participation and collective action: facilitation of independent problem-solving and peer support

Cognitive participation was evident because five mentors reported encouraging OTs to independently problem-solve, with mentors providing guidance where necessary. All OTs in the session were encouraged to share advice to facilitate peer support.

“I guess my role is to try and facilitate a discussion amongst the team where they can do some shared learning … it’s not about me giving answers but trying to facilitate a discussion to draw people in … ” [Interview with Mentor 3]

OTs reported their learning was supported by this facilitation of meaningful conversations; seven agreed that mentoring sessions enabled problem-solving, through input from their peers and reassurance from the mentor.

“ … if situations come up there are other people that can sometimes also give advice, you know, I tried this with my patient, or you can advise them on this or whatever...” [OT 36, Site 9]

Frequent liaison with other RETAKE OTs at the same site (i.e., a site partner) was beneficial. Mentoring records showed that OTs within sites (and sometimes across sites) met regularly for peer support and visited participants together when an OT lacked confidence. OTs across 7 of the 11 sites represented in the interviews reported they were able to regularly gain peer support from site partners outside of mentoring sessions. Three of the 19 OTs interviewed reported that regular contact with their site partner provided a safety net and opportunity to talk freely about participants.

“I work really closely with my other counterpart that’s doing the RETAKE. Having somebody else has been really, really valuable because you can’t talk to anyone at work about it. You don’t want to bug people sort of from RETAKE all the time.” [OT 6, Site 3]

In contrast, four OTs with little contact with site partners reported feeling isolated.

“I still feel quite alone in it all, I know [site partner] is there if need be, having said that we have been trying to communicate all day yesterday and failed.” [OT 24, Site 12]

Overall, regular communication between mentors and OTs (including other OTs at the same site or different sites) enabled collective action because it provided opportunity to discuss caseloads and conduct joint visits, thus building accountability and confidence in intervention delivery and supporting local operationalisation of RETAKE intervention delivery.

Facilitators and barriers to the mentoring process

Facilitators to the mentoring process

For mentors, the most important facilitator to the mentoring process was support from the RETAKE trial team, including two mentors who were also members of this team. Four mentors reported receiving support with administration, trial processes and/or mentoring itself (e.g., advice on communication style with mentees). Two mentors also mentioned time availability as a facilitator to the mentoring process, as being retired or working part-time enabled them to provide support when necessary.

“I’ve always tried to keep the mentoring on a Monday because that’s the day I didn’t work … I certainly didn’t find it difficult if OTs contacted me to say, could we have a phone call?” [Interview with Mentor 5]

Barriers to the mentoring process

In contrast, two mentors found it difficult to juggle mentoring with their usual work, leaving them “overwhelmed” as one mentor put it. Another reported having to conduct some mentoring sessions via phone in her car between home visits, and she found it difficult disengaging from one work activity to the next.

“Monday was the only day that worked for all the people in each group as it were … the days where it was a challenge was where I had [home] visits … I would be in the car and I did one or two mentoring sessions on my phone in the car, just park up somewhere, and then went off to my next visit.” [Interview with Mentor 6]

A common barrier to the mentoring process was non-attendance of OTs at mentoring sessions; this was reported by three mentors as well as mentoring records and emails. Mentors’ perceived reasons for non-attendance included OTs having competing priorities in their usual caseloads, OTs needing to cover for colleagues on maternity- or sick leave, a rigid ‘clocking off’ mentality preventing OTs from attending sessions outside of their working hours, and a lack of confidence in participating in sessions and/or delivering the intervention. One mentor put in additional effort to encourage an OT to attend mentoring, as evidence from her competency assessment suggested she might not fully understand her RETAKE OT role.

“I’ve got a new person who hasn’t come into mentoring so I’ve just emailed them now to say, “Can we have a ten minute conversation?” … She thinks she knows what she’s doing but I’ve just marked her paper [written competency assessment] and she doesn’t … ” [Interview with Mentor 4]

Implications for future provision of mentoring

Suggestions for improving mentoring

Suggestions for improving mentoring related to how it was organised, delivered, and evaluated. The most common suggestion from mentors was that it should occasionally occur face-to-face, because it was easier to detect more when communicating this way. Other suggestions included arranging mentoring by site, mentors receiving feedback from OTs, formal evaluation of OTs’ letter-writing, and meetings between mentors to share experiences and discuss OTs’ training needs.

“whether we look at training needs because … as they [OTs] get into the process of the RETAKE, that’s when they’re recognising the skills lack in certain areas and whether we can offer any form of training” [Interview with Mentor 5]

Two of the 19 OTs felt there was insufficient time to discuss caseloads in a 60-min mentoring session; and felt the time could have been structured more efficiently. Four OTs felt mentoring would work better in group format, whilst three others felt it would work better as one-to-one sessions between mentors and OTs.

Future provision of mentoring alongside complex intervention delivery

Among OTs, mentoring was considered essential in supporting complex intervention implementation in a trial context. Fourteen of the 19 OTs interviewed stated that a mentoring system would be needed alongside future roll-out of a similar VR intervention.

“But for me, if a trial is introducing a new way of working with therapists that involves a complex intervention, I would say it’s extremely valuable to have a mentoring system in place.” [OT 5, Site 3]

One mentor expressed concern that if mentoring was not rolled out alongside similar complex interventions in NHS settings, the pressures of new referrals would result in the intervention becoming “watered down.” She stated that adequate mentoring support would be needed to ensure those delivering the intervention did not withdraw too soon and were equipped to handle novel situations and challenges. Half of the mentors explained that how mentoring would be set up and paid for, and who would be suitable to provide mentoring within the contexts of local organisations were key for consideration in future trials.

“the likelihood is that there are going to be local issues that have to be solved. So, either somebody externally has to come in and champion those and try and sort those out, or you have to have internal champions who are willing to think about those implementation issues and look at various different ways that they can be solved from within the organisation.” [Interview with Mentor 1]

Due to the large number of OTs potentially delivering the intervention at any given time, four OTs suggested a future regional hierarchy based on skillset, whereby the more experienced OTs train and support the less experienced OTs. Four OTs also suggested they themselves could act in a mentoring/supervisory role.

Discussion

Little guidance exists regarding how best to upskill and support those delivering complex interventions to ensure implementation fidelity and robust trial outcomes. In the UK, adequate trial-focused training and support for NHS staff may mean trial findings are more contextually relevant and translate more easily into real-world clinical practice. This study examined mentoring provided to OTs delivering a complex VR intervention to stroke survivors in NHS settings. The aims were to explore mentors’ roles in supporting intervention delivery and implementation fidelity; and to explore factors affecting the mentoring process and intervention delivery.

Mentors roles

Mentors’ roles included supporting OTs with coherence, cognitive participation, and collective action through advising them on adherence to core components of the RETAKE intervention, such as commencing intervention early, individual tailoring, communicating with stakeholders, and avoiding discharge of participants immediately following return to work (typically the endpoint of usual NHS rehabilitation). Over half of OTs interviewed felt that mentoring facilitated their intervention delivery through provision of advice and best practice examples, suggesting that they valued mentors’ expertise and saw them as opinion leaders in VR. Similarly, in a Cochrane review [18] health professionals’ compliance with evidence-based practice improved on average by 10.8% when they received input from opinion leaders. Mentors also engaged in cognitive participation by facilitating independent problem-solving among OTs when issues arose, which is important because it encourages new thinking styles, behaviours, and independence among mentees [37]. Both OTs and mentors saw value in mentoring being provided alongside future complex intervention delivery to ensure fidelity, providing further support for use of opinion leaders [4, 38, 39]; and these findings may be replicated in future trials. RETAKE mentoring aligns with the UK government’s vision for clinical health research [40], which includes empowering staff to contribute to NHS clinical research and innovative research designs incorporating virtual processes and technologies (e.g., RETAKE mentoring was delivered via teleconference or Microsoft Teams).

Mentors also supported OTs with coherence and cognitive participation, through advising on trial processes relating to trial documentation completion, contamination, and recruitment. Support with trial documentation completion appears a novel finding in research, noteworthy because complicated trial documentation has been cited a key barrier to clinicians participating in practice-based research [41, 42]. Incomplete or missing trial documentation could have influenced accuracy of findings from RETAKE’s embedded economic and process evaluations. Mentors additionally engaged in collective action by alerting the trial team to potential contamination issues as they arose, reducing risk of minimisation of difference between trial arms [43]. Recruitment in sites was unofficially paused at times due to limited capacity of recruiting staff and was difficult to manage from the trial management perspective. Mentors helped by engaging in cognitive participation, i.e., through defining strategies with OTs to increase and sustain recruitment. Similarly, in a VR study using an identical mentoring model to RETAKE [5], mentoring gave timely insight into trial-related issues, enabled quick responses to support intervention delivery and facilitated implementation fidelity within OTs’ local contexts.

Factors affecting the mentoring process and intervention delivery

Barriers to delivering the RETAKE intervention with fidelity included heavy usual caseloads and staff shortages. Similar issues were reported as barriers to nurses’ delivery of a family violence screening and care model [44]. The authors linked these barriers to key people not being involved to motivate and organise people and collectively define actions needed to sustain intervention delivery (i.e., lack of cognitive participation). Similarly in RETAKE, unsupportive managers pressurised OTs to prioritise UC over RETAKE or even drop out of RETAKE altogether, demonstrating lack of cognitive participation and collective action. These managers may not have developed coherence and lacked understanding of the potential benefits and differences of the RETAKE intervention compared with UC, possibly due to limited research experience. In future trials, trial-related managerial issues should be highlighted to the trial team, so that NHS managers can be educated on research processes and actions needed to sustain local intervention delivery. Importantly, the above findings suggest that RETAKE mentors took on roles akin to PIs and service managers, engaging in cognitive participation and collective action early on, e.g., to support trial processes and advise on interactions with managers to operationalise local RETAKE intervention delivery.

RETAKE mentors were funded through research budgets, but to ensure successful future roll-out alternative methods of resourcing this support would require consideration. Some OTs suggested a regional hierarchy based on skillset, whereby more experienced OTs train and support less experienced OTs. If the experienced OTs were considered opinion leaders their input could facilitate cost-effective intervention delivery, where funding for mentoring could be offset by savings through effective implementation. Such an approach would require justification and testing through implementation research [45].

Another point to consider is how future suitable mentors would be selected and upskilled. RETAKE mentors received training on the mentoring process and benefitted from attending RETAKE intervention training to improve their understanding of training, trial processes, the intervention, and OTs’ backgrounds. It is uncertain how this training would be funded and delivered within or across local contexts. The Royal College of Occupational Therapists (RCOT) trained health and work champions (i.e., opinion leaders) to deliver training to other staff within local organisations, enabling them to deliver brief VR advice to patients wishing to return to work [46]. It is possible professional bodies could train future mentors using similar methods. However, it remains unclear which mentoring model would be most effective alongside wide-scale roll-out of RETAKE or similar complex interventions. In RETAKE mentors with concurrent clinical roles struggled to balance mentoring and clinical workloads. Others with protected time for mentoring experienced no problems, which is an important consideration for future research.

Strengths and limitations

This research had multiple strengths. Firstly, the thematic analysis of qualitative data in this study followed a systematic multi-step approach and included standard measures in order to increase rigour and trustworthiness.

The use of NPT as a sensitising framework for interpreting findings is considered a strength, as it offers insight into the mechanisms that may affect implementation of this intervention in the future.

We acknowledge that the sample of mentoring records and emails which were analysed covered a set time period which, although large, did not include data from later stages of the trial such as issues around closing sites or impact of the COVID-19 pandemic. The sample size of 6 mentors and 19 OTs can be considered small in relation to the size of the RETAKE trial. However, it included all of the mentors and at least one OT from each site and from different time points in the intervention delivery process. These recruitment methods, using purposive sample, together with the methods of analysis and use of NPT suggest that this sample generated data which are likely to be transferable to similar studies and settings [47].

Conclusion

Little guidance exists on how best to support those delivering complex interventions to ensure robust trial outcomes and implementation fidelity. Study findings indicated that RETAKE mentoring enhanced trial recruitment, improved OTs’ understanding of- and adherence to trial processes, supported implementation fidelity and intervention delivery, and aided OTs’ development of problem-solving skills. It remains unclear to what extent RETAKE OTS would have been able to deliver the intervention without mentoring support or how this might have impacted implementation fidelity. Successful implementation of mentoring alongside such interventions may increase the likelihood of intervention effectiveness being observed and sustained in real-life contexts. Further research is needed to investigate the benefits of this approach for other complex interventions, including how mentors could be selected, upskilled, funded and mentoring provided to maximise impact. To ensure those implementing new complex interventions into clinical settings invest in the whole package, the clinical- and cost-effectiveness of mentoring as an implementation strategy and its impact on fidelity require testing in a future trial.