Understanding what works
We are constantly building our evidence base to demonstrate the considerable impact of the Citizens Advice service: how we meet a need in society and have value for the individuals and communities we work with, as well as government and wider society.
But we don’t just stop there, as we want all all our direct service delivery interventions to be as effective as possible at helping clients find a way forward.
Our approach to what works
At Citizens Advice we are committed to understanding what works and ensuring organisational resources are responsibly invested into worthwhile services.
This is the right thing to do for our clients and society. It also enables our organisation to embed learning from delivering and measuring effective services:
How do we do it?
Our Impact and Evaluation team are responsible for customer insight research, assessing service effectiveness through evaluating what works, and using our evidence and analysis to understand and demonstrate the impact and value of the Citizens Advice service.
Evaluations are designed to ensure that Citizens Advice is equipped to make evidence-based decisions about the effectiveness of direct services.
Evaluations are primarily delivered in-house by our Impact team, as their understanding of Citizens Advice processes is essential to ensuring that evaluations meet organisational needs. It also enables us to build-up our internal learning of the most appropriate evaluation approaches, best practice and tools.
The impact and evaluation function is independent of those delivering our services, providing additional credibility to evaluation findings.
Where an evaluation requires external validation or requires a specialist approach (e.g. evaluating domestic violence intervention efficacy) we will engage the services of external evaluators.
Each evaluation is custom-designed to measure what matters, but will often focus on looking at understanding advice processes and resources used, as well as the needs of our clients and what outcomes we achieve for them.
- Crucially, as an organisation that does so much for so many, our evaluations need to capture the nuance of what works for which clients and under what condition
We have shared outcomes measures
Citizens Advice is made up of a network of over 300 individual local charities, and we have designed and implemented shared measures to systematically understand the outcomes we achieve for our clients.
We have shared outcome measures with over 800 outcomes codes built into our centralised case system management system, allowing advisers to record what happens to a client as a result of advice. Last year over 400,000 outcomes were captured this way.
These outcomes codes are reviewed annually to make sure that our measures still best-fit the likely issues and outcomes our clients experience.
We support local Citizens Advice to design and deliver outcomes research
As well as supporting local Citizens Advice to adopt shared outcomes measures, the Impact team also produce toolkits on evaluating specific interventions undertaken locally.
This includes theoretical and practical considerations around evaluation and research good practice.
What we’ve learnt: Financial capability
We support thousands of people each year who need help with money problems. That’s why we want to understand the role of people’s financial capability - or as we call it, money skills - so we can provide more holistic, appropriate services and understand what does and doesn’t work to help people find a way forward with their money skills.
Understanding our client’s baseline financial capability
For our national report Understanding money skills [ 4.9 mb] (2017), we surveyed over 1,000 face-to-face local Citizens Advice clients from across England and Wales on their baseline money skills. This research helps us understand the complex nature of our clients’ financial capability so we can provide more holistic, appropriate services that help people find a way forward.
Our approach to measuring financial capability
As part of a year-long project, we developed a framework for evaluating our services’ impact on our clients’ financial capability. In order to do this, we carefully researched and developed a set of measures that are designed to suit our clients and the nature of the services that we deliver, while aligning with the UK Financial Capability Strategy. Learn about how we did this from our paper Measuring Financial Capability [ 2.1 mb] (2017) which includes a foreword from Sharon Collard. The below diagram gives an overview of the progress of this research project.The methodology we are using to develop our financial capability measures
The process of developing financial capability measures that are fit for purpose for different service settings is an ongoing project for us. If your organisation would be interesting in the testing and development of our measures, or would like to use them to help evaluate the impact of your project, please get in touch with firstname.lastname@example.org.
We have completed a number of evaluations in the past aiming to build an understanding of our clients’ financial capability and what our impact has been in this area:
Quids In [ 0.62 mb] (2012) - We found that social housing tenants were £10 a week better off than the comparison group as a result of some targeted financial capability services we delivered to them.
Managing Migration pilot [ 150 kb] (2013) - We assessed how prepared our clients would be to use the new Universal Credit system. As part of this, we looked at clients’ financial capability. Focussing on skills, behaviour and mindset in relation to budgeting, banking and staying informed.
We wanted to take a broader view on how some of our biggest services, such as debt advice, could have an impact on our clients’ financial capability. This led to a number of pilots of different services, such as debt advice incorporating a financial capability element, which we evaluated:
We also have a financial capability team who carry out similar research and evaluation of our interventions along with the impact and evaluation team and other partners.
What works in policy and practice
Understanding what works can extend beyond considering direct service delivery.
By listening closely to the people that come to us, using our real-time data and gathering insight and intelligence from clients and frontline advisers, we spot emerging issues with policies, practices and regulations that are not delivering for society. Our policy research teams have expertise in understanding what works in public policy and private practice, including proposing solutions to existing problems.