Randomised controlled trials (RCT) are rare within a social care setting and it can be something of a culture shock for people working with children and families when asked to become involved in this sort of research.
We decided to use an RCT to evaluate the NSPCC’s Letting the Future In programme, an intervention for children and young people affected by sexual abuse.
A huge amount of work was required from NSPCC practitioners, their managers and the evaluation department to support researchers from Bristol and Durham universities to make this possible.
Here is what we learned.
The challenges of using RCT
Specific requirements for RCT design can impact on practice, particularly the need for strict adherence to the model of the trial. This can reduce the scope for practitioners to exercise judgment.
One example affected work with carers. As a manager observed: “A specific number (of sessions) had to be decided on as flexibility wasn’t possible through the RCT. We’ll find out through the research whether more sessions were needed.”
Typically, practitioners would expect to use their judgment in such cases and this felt like a constraint on their professional decision making.
And historically, the NSPCC has operated a no waiting list policy: this had to be changed in order to run a waiting list control design.
The policy had been established for safeguarding reasons and it required careful scrutiny before we could proceed.
How to manage randomisation
Running a waiting list RCT while managing case loads is complex. The complexity arises from having enough referrals and spaces to randomise, and taking those who come into the intervention, but also looking at the “back end” to see who is finishing the service, making space for those on the waiting list.
It was essential that our staff working with children and families understood the randomisation process and could communicate it confidently to others: it was their responsibility to explain it to potential service users. Such discussions were new for them and initially they could feel, as one practitioner acknowledged, “tongue-tied, embarrassed and apologetic”.
Working out the exact sequence of the eligibility assessment, research assessment and therapeutic assessment, so they happened in the right order, at the right time, was tricky. The randomisation result can be known immediately but, in practice, it was felt important to allow a gap between explaining the process to families and doing the randomisation. Practitioners can then check the case is safe to go on the waiting list and prepare psychologically to deliver the outcome of the randomisation to families.
The ethics of RCTs
Staff working with children and families were understandably concerned about a child needing a service and being made to wait: it felt counterintuitive, given that their role is to deliver a service.
It was important to explain the research clearly to service users, and to ensure adequate support for those families on the waiting list, without crossing the boundary by providing them with a service.
We set up an independent contact and signposting process for those on the waiting list – but no one used it. Practitioners reported that because families knew they would definitely be getting a service within a clearly defined period of time, they seemed happy to wait.
Lessons learned
Introducing RCT methodology to the NSPCC was a new initiative and staff at all levels needed time to understand and adjust to it.
It helped to identify practitioners who were “champions” for the study, who were seen as understanding what was in the best interests of children and had credibility with their peers when explaining how it would work.
At first, many practical issues were raised that appeared insurmountable, but once our teams were convinced the study was worthwhile, they helped work out how to overcome them.
Staff learned to talk about the RCT to families and professionals. They found the materials provided by the researchers, including suggested scripts and process flowcharts, very helpful.
With this kind of support from researchers and colleagues, and with practice, practitioners felt more confident in discussing the RCT and the rationale for the waiting list. As one noted: “Waiting lists are normal in a service setting, such as health. As I felt more comfortable, parents and children weren’t anxious.”
Ongoing communication was required between the researchers and the NSPCC, and there was a feeling that the collaboration was strong. As one manager observed: “Bristol and Durham [university researchers] were very good at listening. They took on board so many elements of feedback and tried to learn from the feedback and make changes as a result.”
Six simple steps to ensure the success of a randomised controlled trial study
- Plan well ahead with ample preparation time
- Identify a sound strategy and be organised at every stage
- Provide reassurance and support to all those involved
- Prioritise face-to-face communication
- Give information and support materials at every stage
- Create positive feedback loops
Richard Cotmore is acting head of the NSPCC’s evaluation department. He has extensive experience in planning and carrying out evaluations in the context of safeguarding children. Over the last five years he has supported the introduction of RCTs for the evaluation of four NSPCC services. The evaluation of Letting the Future In is the first to report.
Content on this page is produced and controlled by the NSPCC, sponsor of the Guardian Social Care Network practice hub.
