Developing evaluation methods to help the NHS provide care more effectively

In an effort to improve the quality of health care and reduce the financial pressure on the NHS, efforts are being made to deliver more care in community settings, with the aim of preventing unnecessary and expensive admissions to hospital. We are developing methods for evaluating how well these interventions perform.

Developments in data linkage mean that it is now possible to provide evidence on the impact of innovations on secondary care utilisation in a timely and less resource-intensive manner.

The Nuffield Trust has developed and applied evaluation methodologies that exploit the large amount of administrative information on individual patients that is available in the NHS and social care.

We have used these methods to evaluate more than 30 community-based service innovations in the NHS over the past five years. Our focus has usually been on whether the service model being implemented and evaluated has had an impact on service use and costs.

In June 2013 we published a report that outlined the key learning from our evaluations – it identifies nine points that should help those designing, implementing and evaluating community-based interventions in future.

We believe the paper will provide useful learning for the new health and social care integration ‘pioneer’ sites that were appointed by the Department of Health by September 2013. Many preventive interventions are available at the NHS’ disposal, ranging from case management for high-risk patients, the use of technology such as telehealth and telecare to monitor patients remotely, and changes to reimbursement systems for care provided by hospitals.

Evaluating novel interventions that use social action

In light of the pressures that statutory care services will be facing in coming years there is keen interest in finding ways to help people who have complex health needs to live independent and comfortable lives.

Voluntary services and local providers of health and social care services are increasingly coming together to develop schemes that deliver practical support and advice to people in their homes, or when they are being discharged home after a stay in hospital.

Quite apart from the emotional and social improvements that these types of service might bring, there is also hope that they could help to reduce the need for very costly care in hospital through reductions in delays in discharge, and in preventing emergency admissions.

The Nuffield Trust are involved in evaluating a number of developing care interventions involving volunteers and voluntary sector staff.

This winter (2014/15), a £2m fund from the Cabinet Office’s Centre for Social Action and partners is being made available to expand up to ten volunteer-led support schemes in England. We will evaluate these services to test whether hospital admissions and care costs did reduce for those on the schemes, and will also look at the impact on individuals’ quality of life. Our concluding report will be published in the second half of 2015.

Evaluation methods: in detail

One particularly robust way of evaluating interventions is by conducting a large randomised controlled trial, where one group receives the new intervention and a control group receives usual care, such as our project that is evaluating the impact of telecare and telehealth.

This project is exploring new, less expensive, ways of evaluating how effectively new preventative interventions in the NHS are working

However, trials like this are not always a feasible option for the NHS or social care as they require a substantial investment of money and time. This project is examining alternative options that make the most of the large amount of administrative information on individual patients that is available in the NHS and social care.

In this programme of work we are applying person-based risk-adjusted evaluation (which allows us to put together a control group for comparative purposes by linking up information stored on patients’ care) to a number of interventions in health and social care, including selected Partnership for Older People Projects. We are also conducting more research into the statistical methods used to select control groups which evaluate the success of interventions.

The key to finding a control group lies in using the large amounts of patient information held throughout the health and social care system, which can now be extracted for each patient and linked together anonymously without compromising their confidentiality. Since we are talking about information on hundreds of thousands of individuals, there is often scope to find controls that were sufficiently similar to those who received the service being evaluated but who did not actually receive it themselves.

This means that we can now match an individual receiving the new service to a control group that were similar in terms of age, sex, recorded health diagnoses and prior hospital use, and who lived in an area with a similar level of deprivation. In the past this would have been very challenging.

This kind of evaluation is economical and can be applied retrospectively, so that pilots can be evaluated even if at first they were not developed with evaluation in mind. The method can be applied in real time too, so it can be used to refine a service in response to emerging evaluation findings. However, controls must be selected carefully because, unlike a randomised controlled trial, matching ensures only that the two groups are similar in ways that can be observed.

This page will pull together developments from our research project to develop evaluation methods further, including published papers and details of events.

Sign-up for our newsletter

About us

The Nuffield Trust is an independent health charity. We aim to improve the quality of health care in the UK by providing evidence-based research and policy analysis and informing and generating debate.