'When you hear hoofbeats, think of horses not zebras' - the old adage is well-known to GPs but what should you do when faced with a zebra, not a horse? Consultant cardiologist Professor Robert Tulloh and GP Dr Louise Tulloh kick off our new series with their advice on how to catch Kawasaki disease in general practice.

In March 2011, RCGP urgent care lead Dr Agnelo Fernandes published the Urgent and Emergency Care Toolkit. He and many colleagues had worked to produce a methodology for developing the effectiveness of patient-facing staff – both clinicians and non-clinicians in urgent and emergency care.

The toolkit enables services to get the best from their staff. As GPs, that means examining delivery of patient care and identifying areas for improvement and is complementary to appraisal and revalidation. This is a summary of how it works in one OOH service in the North West.

The toolkit uses fourteen criteria to assess the performance of clinicians and non-clinicians (Fig. 1). In the Liverpool and Knowsley OOH service auditors review voice recordings from call handlers, observe and interview receptionists and review voice recordings and paper based records from clinicians.

Performance is assessed against each criterion and scored as compliant, partially compliant or non-compliant. Several records per person are audited (four is usually enough). It soon becomes apparent where each person is performing well and where development is needed. Results are fed back to staff and the cycle repeated.

How we used the toolkit

Making it work in an organisation with 60,000 patient contacts a year, over 100 clinicians, 40 call handlers and 25 receptionists requires contributions from a team of part team staff, including a data analyst running a database, a lead clinical auditor, several sessional clinical auditors, a call handler auditor and two receptionist auditors.

But it is not the size of the team that makes the difference. It is the willingness of everyone to take part, to see the process through and pay attention to detail.

As much as it scrutinises the patient facing staff, the auditors must be up to the job. In the OOH service a lot of work goes into benchmarking the auditors. How different auditors' score each criterion is assessed and reported and auditors are encouraged to discuss amongst themselves what they consider to be compliant, partially compliant or non-compliant. Results are fed back to staff.

Individual and benchmark reports are provided. They enable staff to see their strengths and development needs and to compare themselves against others. Where there are criteria that have scored low across the organisation, work is done to highlight the area and training provided.

For example, Safety Netting may be an issue. This is highlighted in broadcasts and through setting out examples of best practice.

On occasion auditors come across examples of practice that give cause for concern. If the example is low level then that staff member is sent a sheet setting out the details of the encounter and asked to reflect on their performance and feedback. This is a Minor Call to Reflect.

More serious concerns are addressed in a similar way but the clinician is invited to attend a session where the episode is reviewed using expert mentoring and a methodology such as Pendleton's rules.Clearly, if an example of malpractice is found then further action must be taken.

Overall the system has worked well although it has not been easy to implement. It has enabled the service to evidence some really good practice and to highlight areas where improvement would enable patients to get better care.