1. Choose your tools wisely
Rachel Margolis: “Be clear what you want to measure so you’re using the right tool. It’s useful to have discussions with colleagues delivering services to pin down what we’re trying to achieve, the changes we’re expecting to see, and exactly what we want to measure so we can say whether the approach has been effective.”
2. Some tools make for easier analysis
Margolis: “Tools that are widely used and validated mean you can be clearer about analysing the data you collect. The four band categorisation of scores from the strengths and difficulties questionnaire (SDQ) means that we can see not only if scores improve between the beginning and end of a programme but also if children move from a higher level of need to a lower level. This is useful for describing changes that are not just statistically significant but also clinically significant.”
Emma Smith: “We were also able to measure the comparative effectiveness of our Dart service by comparing SDQ data from a therapeutic service for children at a domestic abuse refuge.”
3. Help parents and children understand
Smith: “We always say why we’re asking service users to participate in evaluations. Seeking informed consent is at the heart of our ethical guidelines (pdf). We make it very clear what we do with their data, how long it is kept for and the fact what they say is kept confidential unless a child protection concern emerges from something they say. It’s important to us that service users realise they’re in control. They can decide not to take part, and if they do, they can decide to withdraw their data up until the time the study is published. It hasn’t happened so far for us, but the option is always there.”
4. The way in which you invite service users to participate make a difference
Margolis: “When we introduced the Child Abuse Potential Inventory as a tool to measure the impact of two of our services, social workers were understandably anxious. It has 160 questions and was designed in the 1970s so the length of it can be challenging, as can some of the language it uses. Our colleagues delivering the service have worked really hard to figure out how to offer it at the right time, and to explain how and why we’re using it. We’ve shared the inventory scores with some parents and it has been a useful springboard for conversations about what has emerged from it. In this way it can support the work we’re doing with families.”
5. Be prepared
Smith: “Questionnaires can bring up questions about family life, self-reflection or difficult feelings. We have had instances where administering the evaluation tool has uncovered information which hasn’t emerged during the work with a child or parent. Being prepared to respond to a disclosure (pdf) or signposting help where appropriate is really important.”
6. Questionnaires can raise sensitive issues – but this can open doors
Smith: “If we’re trying to measure the impact of reducing abusive behaviour, we have to use some sensitive questions to get to the bottom of what has and hasn’t changed. When we used the parental acceptance and rejection questionnaire to measure for our Dart service evaluation, one mother reported getting upset, but this was because she recognised the negative ways she behaved towards her child – which in some ways could be considered a positive outcome. It shows how tools like this can help shape the approach to supporting a family.”
7. Incorporate evaluation into the work
Mike Williams: “It’s sometimes worth finding measures that you can integrate into work with children. Doing this is likely to increase the volume of measure data you get back. Studies sometimes require practitioners to administer a measure without looking at the findings because this knowledge could influence how they respond to the child. However, this can, understandably, cause frustration amongst practitioners.
“I’ve found the best way to integrate a measure is to look for one that was designed for practice purposes. The outcome rating scale is one such tool which we’re using to evaluate our sexual exploitation service. It uses a simple scale to empower young people to articulate their challenges but also their solutions. In some cases, the measures have prompted first-time disclosure of significant abuse.”
Find out more about tools to measure outcomes for children in families at nspcc.org.uk/evidencehub
Content on this page is produced and controlled by the NSPCC, sponsor of the Guardian Social Care Network practice hub
