This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

The UK general practitioner (GP) appraisal system is deemed to be an inadequate source
of performance evidence to inform a future medical revalidation process. A long-running
voluntary model of external peer review in the west of Scotland provides feedback
by trained peers on the standard of GP colleagues' core appraisal activities and may
'add value' in strengthening the robustness of the current system in support of revalidation.
A significant minority of GPs has participated in the peer feedback model, but a clear
majority has yet to engage with it. We aimed to explore the views of non-participants
to identify barriers to engagement and attitudes to external peer review as a means
to inform the current appraisal system.

Methods

We conducted semi-structured interviews with a sample of west of Scotland GPs who
had yet to participate in the peer review model. A thematic analysis of the interview
transcriptions was conducted using a constant comparative approach.

Results

13 GPs were interviewed of whom nine were males. Four core themes were identified
in relation to the perceived and experienced 'value' placed on the topics discussed
and their relevance to routine clinical practice and professional appraisal: 1. Value
of the appraisal improvement activity. 2. Value of external peer review. 3. Value
of the external peer review model and host organisation and 4. Attitudes to external
peer review.

Conclusions

GPs in this study questioned the 'value' of participation in the external peer review
model and the national appraisal system over the standard of internal feedback received
from immediate work colleagues. There was a limited understanding of the concept,
context and purpose of external peer review and some distrust of the host educational
provider. Future engagement with the model by these GPs is likely to be influenced
by policy to improve the standard of appraisal and contractual related activities,
rather than a self-directed recognition of learning needs.

Background

Since 2003, general practitioners (GPs) in the United Kingdom (UK) have been contractually
obliged to engage in a peer appraisal of their professional practice [1]. This involves an externally nominated peer colleague (appraiser) reviewing specific
aspects of an individual GP's (appraisee) professional practice on an annual basis.
Feedback to aid professional development is provided in a 'non-judgemental' and formative
manner. The appraisal interview remains a confidential interaction between the appraiser
and appraisee, unless issues arise during discussions that give cause for serious
concerns about the well-being of the doctor or patients.

There is an expectation that the combined evidence from five consecutive annual appraisals
would substantially inform a proposed UK system of revalidation for all medical practitioners
[2]. However, a recent White Paper [3] has declared that this is no longer a viable option because the existing appraisal
process is thought to lack the robustness, objectivity and clarity which are necessary
to support a high stakes revalidation system. Additionally, the evidence for the effectiveness
of this form of peer appraisal in improving individual learning and the quality of
patient care remains inconclusive [4].

It is now clear that appraisal will have to move from mere engagement to a demonstration
of both learning for the doctor and impact on patient care. By necessity, therefore,
the process must become more transparent and objective [5]. Peer review and feedback either by the appraiser or through other external 'quality
control' mechanisms is likely to become more prominent in deciding on whether the
evidence supplied for appraisal is of a sufficient standard to inform revalidation.

One suggested method of addressing these criticisms is through the external verification
of specific aspects of a GP's work. In the west of Scotland medical deanery, a peer
feedback model [6,7] has been established for over a decade and has been proposed as one method to 'add
value' to the current Scottish GP appraisal system. The underlying principles of the
model are based on an adaptation of cognitive continuum theory [8], which has been described in related published work 6. GPs in the deanery are currently
able to voluntarily submit evidence of three core appraisal activities (clinical audit,
significant event analysis (SEA) and consultation technique via videotaped consultations)
for external review by trained peer colleagues. A small financial cost is levied for
this service. This type of approach may be of a much wider interest and potential
value beyond the regional deanery. Details of the educational purpose, method and
theoretical origins underpinning this specific peer feedback model have been published
previously [6,7,9].

It is proposed that this peer feedback model may help to resolve the 'appraisal dilemma'
by providing some objective evidence of professional development and performance activities which are necessary
to strengthen the existing model in support of revalidation [6,7]. Furthermore, it would present an opportunity to review whether the activities undertaken
by the GP are of a desired educational standard and, where necessary, allow for informed
feedback to improve the quality of the submitted work.

The empirical evidence underpinning the acceptability, feasibility and educational
impact of the peer feedback model is moderate and growing steadily [9-17]. However, although a significant proportion of GPs has voluntarily participated in
some aspect of the model, a clear majority in the deanery has yet to do so. It is
unclear why many choose to forgo this potentially valuable learning opportunity, while
others are prepared to voluntarily engage. In order to further strengthen engagement
with the peer feedback model, we aimed to explore the views of GPs who had not yet
participated. We were specifically interested to identify potential barriers to engagement
in the per feedback model (beyond the financial cost) and ascertain attitudes to the
concept of external peer review in the current context of appraisal and planned revalidation.

Methods

The research was qualitative in design. Semi-structured, in-depth interviews with
individuals were used to explore understanding, knowledge, attitudes and practicalities
of engaging in and receiving external peer feedback. Initial exploratory interviews
with six GPs were conducted by LP in 2009. The themes emerging from these interviews
formed the basis of the topic guide for the interviews held in 2010. These interview
data were not included in this study as too much time had elapsed and the interviewer
had taken up another research position. An iterative study design [18] was employed in order to allow emerging themes to be explored with greater depth
in subsequent interviews. Adopting this reflexive approach reduces the risk of making
unwarranted assumptions about perspectives on peer feedback in the sample.

Setting, sample and sampling procedure

The research was conducted in the West of Scotland general practice setting, across
five geographical health boards. In the UK GPs have completed vocational training
and are licensed to provide a range of primary health care to individuals and families.
A database review of 3059 GPs working in the region found 1923 (63%) who had not engaged
with the model. The 'non-engagers' group was stratified by three inclusion criteria
(Table 1) to ensure a range of perspectives were captured - gender, type of GP position held,
and postgraduate training practice status. This stratified the population into twelve
groups for sampling. Sample sizes relative to the total group were calculated to result
in a sample of 100 potential study participants. Random selection methods were used
to select individuals from each group. A letter was sent to the sample of 100 explaining
the study purpose, with an invitation to participate in a confidential semi-structured
interview lasting approximately 60 minutes. Of the 29 who replied seven expressed
an interest in participation. A second round of invitations was sent to a further
sample of 100 GPs using the same process, which elicited 24 positive responses with
a further six GPs able to attend interviews. A total of thirteen GPs were interviewed.

Data collection

Interviews were conducted by EC at a venue chosen by each participant. A semi-structured
prompt guide was used to steer discussion on the following areas: understanding of
peer review and feedback; exposure to peer feedback activities; factors affecting
participation in peer feedback (perceptions of barriers and facilitators). Interviews
lasted 45 minutes on average, ranging from 30 to 60 minutes. The interviews were audio-recorded
with permission, transcribed and anonymised. Formation of theory based on the emerging
themes was explored in more depth with subsequent participants until saturation was
achieved. The interviewer reflected on her perspectives of each interview and made
contemporaneous notes in a comments sheet. This was shared and discussed with research
colleagues in order to deepen the understanding of the topic and stimulate further
thinking.

Data analysis

Each interview recording was listened to thoroughly to check for inaccuracies in transcription.
Thematic analysis of the interview transcriptions was conducted by a constant comparative
approach [19]. Sections of meaningful text were identified within each interview independently
by two researchers. The sections of text were coded; codes were then compared with
each other looking for similarities, differences and general patterns to form categories,
which were then compared across interview transcripts to identify themes. The categorisation
changed several times during this process. Particular attention was paid to negative
cases (cases inconsistent with the emerging analysis), which were used to clarify
themes during analysis discussion sessions.

Ethical approval

The study proposal was pre-screened by NHS Lothian MREC 'A' committee for research
ethical approval but was judged to be service evaluation.

Results

Of the thirteen GPs interviewed (nine male, four female), ten were GP principals,
two were sessional doctors and one was a salaried GP. Study participants worked in
a mixture of urban and rural practices across the west of Scotland with a spectrum
of patient list-sizes. Six participants were involved in delivering either under-graduate
or post-graduate training, and five were members of the regional educational programme
which permitted free access to the peer feedback model (Table 2).

Value of the improvement activity

There was disagreement amongst participants as to the inherent professional or clinical
value of all three improvement activities offered for peer review. Although most GPs
could understand why these specific activities could enhance professional practice,
others viewed some or all of them as not having a direct relationship to improving
their day-to-day clinical work and so questioned their importance to them.

"Audit I am sure does improve systems, does raise the standards of personal practice
but maybe not as, not as imminently as looking at a significant event or looking at
yourself doing a consultation. Those are far more in your face, you know far more
personal; they are far more likely to bring around a change in practice or a reflection
about what you are doing" (GP2)

"They are some easily measurable things... they are measurable but I don't know that
that necessarily reflects good practice" (GP1)

For some participants even though they acknowledged that the improvement activities
discussed could enhance aspects of professional practice generally they did not perceive
that it had or would lead to direct changes in their own practices specifically. They
were confident in their own assumptions about the sufficiency of their experiences
and abilities as a means to self-improve and keep up-to-date and did not appear highly
motivated to participate in these specific improvement activities.

"It is highly unlikely at this stage that I am massively going to change my approach
or change my style of operating which has clearly worked reasonably okay up to now" (GP4)

Value of external peer review

It was recognised that GPs can be exposed to many different models or styles of external
peer review often without a full realisation that they are doing so. For some this
was revealed in a limited and possibly conflicting understanding of the context, meaning
and purpose underpinning peer review as an educational construct. For example, several
GPs understood peer review to relate primarily to 'multi-source feedback' for which
they displayed strongly negative feelings. Although this was not wholly accurate it
nevertheless led to a biased perception of what 'peer review' entailed for them.

There was recognition amongst many participants that the possibility of external review
of their professional activities by peers would be more 'challenging' than 'internal'
review from GP partners or associates. For many this was thought to be clearly beneficial
as it would enhance the validity of their work because it was perceived to be more
objective and formalised - particularly if the peer reviewer was not known to the
GP. In addition, it may offer a demonstration to any interested external authorities,
or even their own patients, that their professional performance was of a sufficient
standard.

Conversely, the concept of external review was thought by others to offer little additional
benefit either as a quality assurance tool or as an educational activity. This was
particularly so if study participants' immediate GP work colleagues had recognised
educational expertise and knowledge that may be associated with being a professional
appraiser, specialty training supervisor or holding a medical regulatory position
with a relevant local or national organisation. Additionally, the notion of independent,
external review was also regarded as "more formal review" which in turn was associated with "selection and control" aspects of regulatory medical assessment, rather than as a formative and developmental
educational exercise.

"...the person who would be more detached, could look at the situation more objectively
that is probably a good point also, again they wouldn't be biased by knowing you,
probably more objective and that would probably be the best advantage I think really"
(GP7)

"One of our partners is a CHCP [Community Health Care Partnership] chairman so he is usually ahead of changes in the game, we have got two appraisers...
our Practice Manager is a QOF [Quality & Outcomes Framework] visitor... so we have lots of people who are on the ball learning things you know
from the point of view of our practice" (GP4).

Value of the peer feedback model

The external peer feedback model was thought by many to be constrained by the defined,
structured formats to be adhered to when performing and submitting the improvement
activities under discussion. The act of complying with these formats was thought to
miss the important nuances of daily professional practice that influenced contextual
and performance issues and could only be fully appreciated and understood by those
who worked in and experienced that particular environment.

"...the disadvantage of external feedback would have, it wouldn't it would only have what
it saw on paper whereas internally the people can have a feel for what you are doing
and how you are doing it, so its finer details but it is also sort of the 'wishy-washy'
more difficult to define aspect of what you have done and how you have done it, only
internal people would be able to pick up on." (GP12)

For some GPs their internal working environment was thought to offer numerous opportunities
for 'robust' peer review from a range of clinical and also managerial colleagues.
They therefore perceived very little advantage in engaging with the external peer
feedback model, the preparation for which they thought to be unnecessary, overly formal
and added workload.

"At our meetings in the surgery alone there is probably about thirty or forty folk at
it... so it's a pretty good appraisal in itself... within the practice I think there
is enough people that we get pretty good feedback" (GP10)

"I cannot be bothered with all that extra bits because I don't need it... and well
God ok, that I haven't looked at everything but I just haven't sat down and written
it all bit by bit... but people will argue, 'well if you don't do it properly it is
just sloppy, la la la la', fine, I don't need the formalization anymore thank you
very much" (GP9)

In contrast, those GPs who could foresee a benefit in the peer feedback model did
so because of the perceived advantage that could be gained through the process of
formalising the external review of aspects of their professional performance for regulatory
reasons, rather than the potential educational and developmental impact which may
be accrued.

"I daresay one of the benefits value wise will be strengthening my appraisal and revalidation
folder in terms of ticking all the right boxes there" (GP6)

Attitudes to external peer review (including appraisal)

A lack of professional trust in the true purpose and potential integrity of both the
external peer feedback model and the national appraisal system was clearly evident
for a significant minority of participants. Previous experience of high stakes assessment
processes as part of specialty training led to some expressing negative attitudes
to participation in a formative external feedback system. For others a willful non-participatory
attitude was formed because of a perception that the system is 'disciplinary' in nature,
or is potentially so, or is simply not to be trusted because of the healthcare role
of the organisation which hosts the peer feedback model (the national body responsible
for the education and training of the healthcare workforce in Scotland):

"I ended up having to do a second audit [for summative assessment during GP training]
having failed one on almost a technicality. I think that may have put me off the process...I
didn't really go through a positive feedback process" (GP4)

"Big Brother... I don't see it as helpful at all, I don't see anything that suggests
from anything that I have been involved in to suggest that they are actually trying
to help..." (GP6)

For others, external review was thought of more favourably if it was considered to
be 'true peer review' which was associated with face-to-face feedback. This type of
feedback was considered to be much more valuable than the existing system of providing
a personalised written report summarising reviewers' developmental comments on the
standard of the improvement activity undertaken.

"...it certainly would be useful to know that that somebody was working in General Practice
and not maybe somebody who was you know in a ivory tower as an academic GP you know,
and wasn't you know and didn't have their feet on the ground sort of thing" (GP10)

"...but I think verbal feedback is useful, without verbal feedback (the written report)
I think it would be worse because I think with verbal there is interaction, any misunderstandings
could be rectified then" (GP9)

A range of marketing and resource perceptions about barriers to non-engagement were
also identified such as a lack of awareness of the peer review system; time to participate,
financial cost and equipment constraints; and unfavourable geographical locations
(where potential participants felt isolated from the central co-ordinating office
for peer review). These perceived pragmatic difficulties were raised by, and on behalf
of, sessional and out-of-hours doctors by participants. These particular peripatetic
GP groups were thought to have greater challenges to overcome when arranging to video-tape
consultations or undertake clinical audits before submitting this evidence for peer
review.

"...the difficulty with them [video-taped consultations] is I suppose the timing and the
amount of work required..." (GP3)

"it is not my practice, I am there as an employee, hired locum for that session, and
so obviously before I think before I would be sending it off I would need to clear
that with the Partners and other people involved before I sent it out to an external
source" (GP1)

Current experiences of the national appraisal system also influenced attitudes to
peer review and whether to proactively engage with the separate peer feedback model.
Most participants recognised the existing appraisal system as a form of external peer
review. However, for some, experiences of this system were associated with a perceived
lack of learning and improvement opportunities in relation to the time and resource
invested, while for others this form of external peer review was slightly more challenging
but had a minimally positive impact overall.

"...so far I have had three appraisers, and if that is what an external person offers
thank you - good night...the last guy was slightly better, the first two were a waste
of space... so that has not encouraged outside support" (GP7)

Discussion

This small study represents the first steps in trying to determine why GPs may forgo
potentially important learning opportunities that could strengthen their portfolio
of evidence for professional appraisal. Preparedness by GPs in this study to participate
in the external peer feedback model to enhance appraisal activities involves three
interlinked themes: the value to the practitioner of the improvement activity itself;
the perceived benefits of external review and feedback; and the professional esteem
in which the provider of the peer review is held. The themes represent a hierarchy
whereby ultimate participation depends on acceptance of the value of the first or
subsequent theme. These 'conditions of value' are modified by the participants' attitudes
to the national appraisal system, the educational model of external peer review under
study and the status of the host organisation.

Findings in context

The finding that GPs must first of all value the improvement activity offered as part
of the peer review process is consistent with aspects of adult educational theory
which suggests that learners often undertake activities that they conceive as being
of personal interest and highly relevant to their daily professional practice [20].

The evidence base for the perceived value of the three core appraisal improvement
activities outlined in terms of impact on professional practice and patient care remains
moderate [21-23]. However, two of three activities (SEA and clinical audit) are proposed as key elements
of enhanced GP appraisal and revalidation which implies that regulatory authorities
place a high value on their role as important indicators of professionalism and performance
[24,25]. Some study participants are clearly sceptical about the effectiveness of such activities
and the 'opportunity-cost' of participation, which is of potential concern. However
research on the peer review system has highlighted variation in knowledge, attitudes
and abilities, while also demonstrating the often critical importance of these activities
to making healthcare safer, improving professional performance and enhancing the quality
of services [10-14].

Some participants also appear confused about what constitutes 'peer review' which
may be understandable given the variety of related initiatives described in the literature,
many of which would struggle to truly reflect any mainstream definitions of 'peer
review' [26]. For example, some GPs associated peer review only with 'multi-source feedback',
which involves a range of colleagues not all of who are 'peers'. Paradoxically these
GPs are highly likely to be exposed to a multitude of external peer review activities
such as prescribing reviews, training practice accreditation visits, contractual verification
visits and annual GP appraisal - but did not associate these professional obligations
with the concept of 'peer review'. The importance of understanding the terminology
goes beyond 'semantics' given the professional role of revalidation and appraisal,
and also the advent of the peer review of prescribing patterns and specialist referrals
recently introduced into general medical service contracts [27].

External peer review was seen by some as having the potential to enhance the validity
of their professional practice, which is consistent with findings from those GPs who
have actually taken part in the educational models offered to participants, including
GP appraisers [6,9,10]. Conversely, others did not see the value or thought it limited, which reflects previous
findings for external peer review through the Scottish GP appraisal system [4].

Some participants felt external peer review has little to offer over internal feedback
from immediate work colleagues in strengthening appraisal evidence. This may represent
a 'cognitive dissonance' [28], whereby conflicting ideas about the purpose and value of peer review are held simultaneously.
Equally, for some, where internal review mechanisms are thought to be robust (either
because of work colleagues' special interests, outside appointments or the variety
of feedback options offered) external review may indeed be of limited benefit, although
this will remain speculative without evidence to the contrary. We should acknowledge
that these internal colleagues (as well as GP appraisers), although potentially informed,
are not 'trained' to review and feedback on the standard of clinical audits, significant
event analyses or consultation performance.

We know that many practitioners submit their audits and SEA reports for external peer
review only for confirmation that this submission evidence for appraisal will be judged
by trained colleagues to be of a satisfactory standard or otherwise [9]. It is possible that if one member of the GP team has demonstrated a perceived competence
in a particular area then the value of external review for others may be diminished
if both a 'benchmark' and local guidance are already available within the practice.

It is unsurprising that the host organisation which manages the peer feedback model
is in itself a barrier to engagement. Research suggests that professionals, where
possible, may seek to avoid submissions to, or contacts with, administrative bodies
which have a governance or regulatory remit over them [9,10]. Previous experience of dealing with the organisation may have led to reluctance
to engage voluntarily with it, particularly if that experience was negative (e.g.
refusal of training practice accreditation or a poor GP appraisal encounter). This
distrust by some about potentially exposing elements of their professionalism via
the peer feedback model is interesting as previous research about hosting a hypothetical
reporting system for patient safety incidents found that this organisation was the
option overwhelmingly preferred by GPs compared with other national healthcare bodies
listed [29].

For some participants, the role of GP appraisal is seen as enforcing unnecessary bureaucracy
into an already busy job with little or no educational gain [4]. To try and enhance appraisal with more 'peer judgments' is, for these GPs, even
less appealing. Additionally there may be an underlying resentment to participate
in and provide evidence for appraisal, which may be seen by them as a slight on the
professionalism of GPs individually and as a group [9,10]. This contrasts with previous research which suggests that GPs are acutely aware
of the need for appraisal to evolve in support of revalidation in order to ensure
and maintain the confidence of patients and doctors [30].

A recent literature review suggested that peer review [26] is a common element of performance assessment and is sometimes "...the only mechanism to judge the professional practice of others working in the same
field...because the peer regularly performs similar work and possesses the relevant
expertise to evaluate it". A fundamental irony is that the GP reviewers who are part of the peer feedback
model described are the ones who have undergone the training to gain the 'expertise' necessary to 'evaluate' and 'judge' the elements of 'professional practice' that are under scrutiny from an appraisal (and regulatory) perspective - rather
than the GP appraisers themselves. Study participants were largely aware of this but
were still reluctant to engage with the model because of a combination of professional
matters, pragmatic issues and attitudinal beliefs that all questioned the 'value'
in exposing aspects of their professionalism on an independent, external and verifiable
basis. The corollary is that it is unknowable if these aspects of professionalism
are indeed exposed to some form of internal review by work colleagues or are subject
to self-assessment, which recent evidence suggests is of limited value [31].

Strengths and limitations of the study

Study strengths included interviews being undertaken with GPs at a range of career
stages, working in different practice contexts (training & non-training), holding
different status positions (principals, sessional and salaried doctors) with a mix
of experience and roles outside of general practice, all of which is reflective of
working arrangements in this setting. The interviewer (EC) was a doctor at an early
stage in her career. This was perceived to have been an advantage particularly with
the older GPs who may not view her to be part of the educational establishment. The
interviewer strived to maintain an open attitude to the data; collaboration with a
second researcher at the data analysis stage helped in this respect.

Limitations included the small number of study participants and their voluntary status
which may have skewed the group towards those that already had a prior interest in
the subject. Although we took great care to stratify potential GP recruits depending
on the type of position they held, ultimately of those who were able to attend the
interviews the great majority were GP principals. The interviews allowed in-depth
discussion with individuals; although saturation was reached it is possible a bigger
study would provide further insights than those gained here. The sample size also
limited the ability to judge where most GPs sit along the process i.e. which of the
judgements on value are most critical to most GPs. A reticence to share openly was
perceived amongst younger GPs, who may have more closely identified the interviewer
with the organisation which hosts the peer feedback model and is responsible for delivering
most aspects of education and training in the region. The use of a semi-structured
interview guide may have resulted in topics being missed, however an iterative approach
to the content of the interview guide throughout the period of interviewing and analysis
helped reduce this limitation.

Conclusions

The study identified a range of important barriers and attitudes influencing non-engagement
in the peer feedback model. How engagement with the model is increased from this 'non-engager'
group is highly likely to be influenced by the decisions of local and national policy-makers,
rather than a self-directed recognition of learning need by these GPs or a targeted
'promotional-push' by the host organisation. Future engagement levels will be dependent
on how seriously decision-makers wish to 'add value' to the appraisal process or,
for example, in quality assuring the standard of significant event analyses undertaken
as part of the general medical services contract with health authorities (which contains
a financial inducement for this element). There are, however, potential feasibility
challenges (e.g. peer reviewer training and funding support) to be overcome if the
peer feedback model is to be implemented on a wider scale. At present there appears
to be a lack of potential alternatives other than this approach or the status quo.

Competing interests

PB and JM are educational researchers who contribute to the design, management and
evaluation of the peer review model under study and are also directly employed by
the host organisation.

Authors' contributions

EC and LP helped to design the study, carried out the interviews, and analysed the
data. EC drafted the initial manuscript. JM assisted with data analysis and contributed
to drafting the manuscript. PB conceived of the study, acquired funding, participated
in its design and coordination and helped to draft the manuscript. All authors read
and approved the final manuscript.

Acknowledgements

We offer sincere thanks to the west of Scotland GPs who volunteered to be interviewed
for this study. We also thank NHS Education for Scotland for providing funding for
the study.

References

General Medical Council: A licence to practise and revalidation. London: GMC; 2003.

Secretary of State for Health: Trust, assurance and safety--the regulation of health professionals in the 21st century.
White Paper presented to Parliament by the Secretary of State for Health by Command
of Her Majesty. London: The Stationery Office; 2007.

Coltart I, Cameron N, McKinstry B, Blaney D: What do doctors really think about the relevance and impact of GP appraisal 3 years
on? A survey of Scottish GPs.

Department of Health: Good doctors, safer patients: proposals to strengthen the system to assure and improve
the performance of doctors and to protect the safety of patients. A report by the
Chief Medical Officer. London: Department of Health; 2006.