Contributors All authors contributed to the design of the study. DJM, AR and DAB recruited the study participants. DJM analysed all data.
DJM managed the literature review and wrote the manuscript. All authors contributed to the interpretation of the findings
and the critical revision of the manuscript for intellectual content and were involved in the decision to submit the manuscript
for publication. DJM acts as guarantor for the study.

Accepted 20 April 2012

Published Online First 31 May 2012

Abstract

Background Medical revalidation decisions need to be reliable if they are to reassure on the quality and safety of professional practice.
This study tested an innovative method in which general practitioners (GPs) were assessed on their reflection and response
to a set of externally specified feedback.

Setting and participants 60 GPs and 12 GP appraisers in the Tayside region of Scotland, UK.

Methods A feedback dataset was specified as (1) GP-specific data collected by GPs themselves (patient and colleague opinion; open
book self-evaluated knowledge test; complaints) and (2) Externally collected practice-level data provided to GPs (clinical
quality and prescribing safety). GPs' perceptions of whether the feedback covered UK General Medical Council specified attributes
of a ‘good doctor’ were examined using a mapping exercise. GPs' professionalism was examined in terms of appraiser assessment
of GPs' level of insightful practice, defined as: engagement with, insight into and appropriate action on feedback data. The reliability of assessment of insightful practice and subsequent recommendations on GPs' revalidation by face-to-face and anonymous assessors were investigated using Generalisability
G-theory.

Main outcome measures Coverage of General Medical Council attributes by specified feedback and reliability of assessor recommendations on doctors'
suitability for revalidation.

Footnotes

Funding The study was funded by the Chief Scientist Office (CSO) Scottish Government, Royal College of General Practitioners (RCGP),
NHS Education for Scotland (NES) and Scottish Patient Safety Research Network (SPSRN). DM, BG and FS are employed by University
of Dundee. SM is employed by the University of Glasgow; AR is employed by NHS Tayside, and DB by NHS Education for Scotland.
All authors had full access to all the data and agreed responsibility for the decision to submit for publication independently
from any funding source. DM is supported by a Primary Care Research Career Award from the Chief Scientist Office, Scottish
Government.

Competing interests None.

Ethics approval Formal application and submission of the research proposal was made and ethical approval granted for all of the work contained
in this paper by the Tayside Committee on Medical Research Ethics A. Participants gave informed consent before taking part.