Abstract

Objectives To explore the extent to which doctor-rating websites are known and used among a sample of respondents from London. To understand
the main predictors of what makes people willing to use doctor-rating websites.

Design A cross-sectional study.

Setting The Borough of Hammersmith and Fulham, London, England.

Participants 200 individuals from the borough.

Main outcome measures The likelihood of being aware of doctor-rating websites and the intention to use doctor-rating websites.

Results The use and awareness of doctor-rating websites are still quite limited. White British subjects, as well as respondents with
higher income are less likely to use doctor-rating websites. Aspects of the doctor–patient relationship also play a key role
in explaining intention to use the websites. The doctor has both a ‘complementary’ and ‘substitute’ role with respect to Internet
information.

Conclusions Online rating websites can play a major role in supporting patients’ informed decisions on which healthcare providers to
seek advice from, thus potentially fostering patients’ choice in healthcare. Subjects who seek and provide feedback on doctor-ranking
websites, though, are unlikely to be representative of the overall patients’ pool. In particular, they tend to over-represent
opinions from non-White British, medium–low-income patients who are not satisfied with their choice of the healthcare treatments
and the level of information provided by their GP. Accounting for differences in the users’ characteristics is important when
interpreting results from doctor-rating sites.

Introduction

In recent years, both the NHS Plan1 and the NHS Improvement Plan,2 set out the changes required for the English NHS to become more patient-focused. Greater patient involvement in the running
of the NHS has gone hand-in-hand with the policymakers’ drive to improve the quality of public healthcare services. The ‘bottom-up’
approach to a more patient-centred NHS has typically focused on three main areas: (1) giving users more choice and personalisation;
(2) making funding respond to users’ choices and (3) engaging users through greater involvement.3 Lord Darzi's 2008 report ‘High Quality Care For All—The Next Stage Review’4 acknowledged that improvements to the NHS should focus on improving the quality of services, and that the best way of achieving
this would be to ensure that services are locally responsive to the needs of the community, for instance, by empowering providers
and patients as decentralised decision-makers in order to foster a culture of continuous quality improvement and innovation.

Websites, such as the NHS Choices and Dr Foster Intelligence, have been developed with the explicit aim of informing patients about the services that the NHS provides and therefore allowing
a better choice of physicians and treatments. In principle, doctor-rating websites can have a profound impact on public involvement
and patients’ choice, as they enable patients to make more informed decisions on where to seek healthcare, and thus to engage
more often in active choices concerning their health. In practice, however, relative little evidence is available on whether,
and to what extent, doctor-rating websites are actually known and actively used in the UK.

A study by the Kings Fund5 explored the information sources used by patients in making decisions about where to receive care. Only 4% of the patients
used the NHS Choices website, with the majority instead drawing information from their own experiences (41%), and advice from
their general practitioner (GP) (36%). Similarly, a national survey on patients’ choice by the Department of Health found
that the NHS Choices website was only used by 5% of respondents.6 These figures are consistent with the evidence from the USA where usage of doctor-rating websites is still quite low.7,8 Moreover, very little is known about the profile of individuals who are more likely to make active use of these sites. Appleby
and Alvarez9 found that women in England desire patient choice more than men (69–56%), suggesting that women may also be more likely to
use sources of information such as rating sites. This is in line with findings from the USA where women and younger adults
are more active ‘online health information seekers’.10

The aim of this study was to contribute to fill these gaps by providing more direct evidence on, first, the extent to which
doctor-ratings websites are known and used among a sample of respondents in a borough of London; and, second, the most significant
predictors of the fact that people are willing to use doctor-rating websites.

Methods

We conducted a self-administered survey to assess the extent and the determinants of (1) the awareness of the existence of
doctor-rating websites; (2) the level of actual usage of those websites; (3) the intention to use doctor-rating websites in
the future.

Questionnaire design

Prior to the data collection, a pilot study was conducted. The aim of the pilot study was to gain an understanding of the
practicalities associated with giving out questionnaires and collecting responses. After listening to feedback from pilot
respondents, and looking at results from the pilot study, several changes were made to make the questionnaire easier to understand.
The changes related to the content, phrasing and ordering of questions.

The content of the final questionnaire was based on findings from the preliminary literature review and was designed in a
number of sections (see online supplementary appendix for full questionnaire). In particular, section A focuses on the awareness
of online rating websites, while section B assesses actual usage of online rating websites. Section C measures the willingness
to use the online rating websites in the future, and explores which aspects of the healthcare providers and which sources
of information are perceived as being important factors in making decisions about where to receive healthcare. Section D assesses
the individual contribution to online rating sites, while section E focuses on aspects of the doctor–patient relationship
and attitudes and dimensions of patient choice. Finally section F controls for internet usage, while section G collects a
broad range of sociodemographic characteristics.

Closed questions were used, worded in a manner easy to understand. A limited number of responses were provided, either with
binary options (eg, yes or no), or with a numerical Likert scale ranging from 1 to 5, with a further option for ‘Not sure’.

A list of variables with a brief description is discussed in the statistical analysis section and is summarised in the online
supplementary table S1 in the appendix.

Ethical approval, informed consent and confidentiality of responses

We completed the checklist for research ethics approval from Imperial College London. As interviews were intended to be conducted
in public places among respondents from the general population, the study involved no risk or harm of any type to respondents,
no link with clinical data was expected to take place, and no incentives were going to be paid to respondents, the study fitted
all the criteria in the first-stage checklist with no further formal application to the Imperial College Research Ethics Committee.

At the beginning of each interview, interviewers showed credentials as research assistants at the University of London, informed
respondents that their answers were anonymous and would remain strictly confidential, and that all responses and data were
going to be used for the purposes of scientific research only. Informed consent by respondents was then given at the beginning
of each interview.

Data collection

The survey was conducted in the field by the researchers involved in the paper. The borough of Hammersmith and Fulham was
chosen for the location of the field survey because it is a transport hub in Central West London, and hosts many offices and
several major business centres. The four interviewers went to different public locations within the borough (underground stations,
high street and residential areas) at different times during the day (early morning, midday and in the evening) and in different
days of the week (including weekends). By covering different times and locations within the borough, we aimed at being able
to approach both working and non-working members of the public. During the surveys in the field, the interviewers approached
every third male and third female that would pass by them.

Sample size calculations were based on the intended objective to look at the correlation coefficient between the likelihood
of using the websites on the one hand, and a typical survey response, on the other. The minimum sample size to test the null
hypothesis of no significant correlation between these two variables was calculated given the most conservative assumption
that the correlation coefficient between the variables in the population was in the region of 0.2 (a ‘low’ effect size, the
variance of one variable accounting for just 4% of the variance of the other). Under the assumptions that all variables are
normally distributed, a bidirectional test (both positive and negative correlation were expected) with 95% significance level
reaches a standard 80% power level at a minimum sample of n=200 subjects.11 We thus targeted a sample size of 200 respondents. The envisaged target was then readily achieved, since only 68 subjects
who were initially approached refused to take part to the survey, with a final response rate of 74%.

Statistical analysis

Besides a correlation analysis, we carried out a multiple regression analysis which aims to explore the determinants of (1)
being either aware or not of doctor-rating websites and (2) the individual intention of using these websites in the future.

The dependent variable in the first case is modelled as a binary variable (Awareness) taking values 1 or 0 for the respondents who reported to be aware or unaware of the websites, respectively. The second dependent
variable is instead modelled as a discrete ordered variable (IntentionToUse) taking values 1, 2 and 3 for subjects reporting to be ‘not likely’, ‘quite likely’ and ‘likely’ to use the websites in the
future, respectively.

The explanatory variables (Xi) include the variables described in the online supplementary table S1, namely: individual sociodemographic characteristics;
a set of variables on the characteristics of the healthcare providers that the respondents consider important for making their
decisions on where to receive healthcare; a set of variables on the sources of information that are important in making decisions
about where to receive healthcare; two dummy variables that capture whether the patient's gender and age are the same with,
or within a comparable range, respectively, than the gender and age of her GP; a set of variables that describe the respondents’
feelings about their relationship with their doctor; a variable indicating the level of participation of the respondents in
their GPs’ decisions; a set of variables on patients’ satisfaction with the level of choice in their healthcare decisions;
a dummy variable controlling for whether the subjects had Internet access at home or at work; a variable on awareness of the
existence of doctor-rating websites; and a variable on whether the subject always asks to see the same GP (see supplementary
table S1 for variables’ details). The choice of the explanatory variables was further informed by the bivariate correlation
analysis reported in the online supplementary table S2 in the appendix.

We used a binary logistic and an ordered logistic model to fit the Awareness and the IntentionToUse discrete variables, respectively, to ensure a reasonable comparability between the empirical results obtained for the two
set of regressions. The two models, in fact, only differ in the number of values that the dependent variables can take, while
the underlying structure of the error terms follows the same standardised logistic distribution. The logistic specification
is particularly appealing because its results can be readily expressed in terms of OR. We have, however, conducted a robustness
check by replicating the multiple regression analysis using the alternative binary and ordered probit specifications. The
two sets of regressions provide consistent estimates and results that are qualitatively fully aligned. Results of the probit
specifications are available, on request, from the authors.

Results

Sample and descriptive statistics

Descriptive statistics of all the dependent and independent variables for the resulting sample of respondents to our survey
are provided in detail in the online supplementary table S1, and here we briefly report their main aspects.

Our sample consisted of 200 subjects. Comparing it with the Census data for the borough the mean age of our sample was slightly
older than that for the borough (39.57 years compared to 35.2 years).12 Our sample, however, was closer to the national mean age of 38.5 years. The range of ages seems to show a positive skew,
with a greater frequency of people aged 40 years and under. This is consistent with the 2001 census data for Hammersmith and
Fulham which showed that the borough contained a larger proportion of young people aged 20–29 (23.8%) than the rest of England
(12.66%).12

Also, the sample had a slightly greater proportion of females than the borough (54.44–52% respectively), and a lower proportion
of ‘White British’ ethnicity (48.79% compared to 58% for the borough).12 This is also significantly lower than figures for England, where White British account for 87% of the population.12 The sample contained 28.99% non-White respondents. This is higher than the 2001 census data for Hammersmith and Fulham which
was 22% and significantly higher than the figures for England, showing non-White ethnic groups accounting for 9% of the total
population.12 Our sample, therefore, allows controlling for high heterogeneity in ethnic background even with a limited sample size.

Regarding working status, 141 individuals were workers (10 of which reported to be currently unemployed), 33 students, 9 officially
unemployed and 6 retired. Eleven respondents did not report their working status. The proportion of subjects who were not
currently working, as given by the sum of the respondents who reported to be unemployed, retired or students, indeed amounts
to 29% of the sample. The majority of actively working respondents reported an income within the £15–35 000 bracket.

Our sample had a high percentage of people with higher-level qualifications: 46.24% of the sample had a university degree
and 27.96% had a postgraduate degree. This is reflective of Hammersmith and Fulham, where 45% of the population have a qualification
of degree level or higher, a figure which is significantly higher than in England, where only 19.8% have a degree or higher
qualification.12

Results on awareness

Only 29 of our respondents were aware of the doctor-rating websites they were asked about and only 6 reported to have used
them.

In the online supplementary table S2 we present the set of bivariate correlations between the fact of being aware of the websites
and each of the variables collected in the survey. As it can be seen, there is a positive correlation between having Internet
access, or being aware of the NHS Choose and Book system, and being aware of the doctor-rating websites. Age exhibits a negative
correlation, whereas the gender concordance with the GP, shows a positive correlation. Positive correlations with the awareness
of doctor-rating websites also hold for respondents who think that those websites are important sources of information, or
who see accessibility and financial performances of hospitals as important factors in making decisions regarding where to
seek healthcare.

In table 1 we present the estimate results of four different specifications of the binary logistic regression for the dependent variable
Awareness with different sets of regressors, which are presented in terms of the OR, together with the standard errors, and levels
of significance.

ORs for the Binary Logit explaining the awareness of doctor-rating websites

Among the demographic factors, age and ethnicity are the only significant variables. Older individuals are less likely to
be aware of the rating websites, which does not constitute a surprise, as they are usually less familiar with the use of internet
in general. Moreover, in most specifications, White British and White non-British respondents appear less likely to be aware
of the websites.

Among the broader sociodemographic factors, only income is sometimes (marginally) significant, pointing to the fact that respondents
with higher reported levels of income tend to be less aware of the websites, while neither education or gender turn out to
be significant predictors of awareness.

Looking at the characteristics of the providers that respondents consider important in making their decisions on where to
receive healthcare, in one specification the reputation of the doctor has a strong positive effect, while both clinical and
financial performance rates of the providers show negative significant effects. Thus, the respondents who consider the reputation
of the doctor important in deciding where to receive care are more likely to be aware of the rating websites, while this is
less often the case for respondents putting a higher weight on financial or clinical performance ratings.

Concerning the sources of information, respondents who consider the hospital statistics important in deciding where to receive
care, turn out to be more likely of being aware of the rating websites, with an effect which is particularly significant and
quite remarkable in terms of OR.

Furthermore, although in one specification the respondents who feel that their GPs spend sufficient time in their consultation
are less likely to be aware of the internet rating websites, neither the statistical significance nor the estimated OR appear
robust across specifications. Although all other variables on doctor–patient relationship were not significant, whenever included
among the regressors, the gender match between the GP and the patient predicts higher awareness of the website ratings, with
a noticeable effect as evident by the reported value of the OR.

As aforementioned, from those that were aware of the existence of doctor-rating websites only six have reported to have used
these websites. In light of this low usage rate, and of the consequent limitations of conducting statistical estimations with
very little variation in the dependent outcomes, we have thus focused the rest of the analysis on the determinants of the
intention to use, rather than actual usage of, doctor-rating websites.

Results on the likelihood to use online rating websites

In the online supplementary table S2 we present the set of bivariate correlations between the intention to use the doctor-rating
websites and each of the variables collected in the survey. As it can be noticed, there is a positive correlation between
having internet access, and being aware of the doctor-rating websites. Both the age and the gender concordance with the GP
show a positive correlation with the intention to use. Positive correlations with the willingness to use doctor-rating websites
also hold for respondents who think that those websites, or hospital statistics, are important sources of information. Also
the fact that respondents believe that online rating is a reliable measure is clearly correlated with the intention to use
them. Finally, positive correlations also hold for respondents who feel that their doctor has time to dedicate to them, or
who see several aspects of healthcare providers—such as reputation, clinical and financial performances, waiting lists, accessibility—as
important factors when making decisions on where to seek healthcare.

In table 2 we present the estimate results of six different specifications of the ordered logistic regression for the dependent variable
IntentionToUse with different sets of regressors, which are presented in terms of the OR, together with the standard errors, and levels
of significance.

ORs for the Ordered Logit explaining the likelihood to use doctor rating websites

Concerning sociodemographic variables, it turns out that White British, as well as respondents who reported income in higher
brackets, said that they were less likely to use doctor-rating websites. Moreover, we do not find any effect of education,
age and gender of the respondents on the likelihood of their intention to use (the results of the specifications including
the age and gender variables are not reported in the table for the sake of space but are available from the authors upon request).

Looking at the characteristics of the healthcare providers that respondents perceived as important while making decisions
on where to receive healthcare, our data suggest that those who consider clinical performance and doctor reputation (in most
specifications) as important factors, are more likely to use doctor-rating websites. These results are consistent with the
nature of the information provided in these websites. Also, and quite intuitively, subjects who consider the familiarity with
their doctor an important factor to decide where to seek healthcare, tend to be less likely to intend to use websites.

Concerning the role of the different sources of information on the decisions of where to seek healthcare, respondents who
see published hospital statistics as important sources of information are more likely to use the rating websites. On the other
hand, those for whom GP advice is an important source of information for decision making are less likely to use doctor-rating
websites.

Also the nature of the doctor–patient relationship seems to play a key role in explaining whether respondents intend to use
online rating websites. First, patients with GPs of the same gender tend to be more likely to use the websites. Second, respondents
for whom the doctor is able to listen to them, and who perceive the nature of the relationship with their GP as friendly,
also tend to be more likely to use the websites. Third, respondents who feel that their doctor explains things clearly are
less likely to use online rating websites. Fourth, it also transpires that the more autonomy patients have in their healthcare
decisions, the more likely they are to be willing to use the rating websites.

Finally, concerning, the interaction between levels of satisfaction with healthcare services within the NHS, and the intention
to use doctor-rating websites, note that those that have reported to be more satisfied with the level of choice of GP, and
with the amount of choice of the hospital to receive outpatient appointments, are more likely to use these websites. On the
other hand, the respondents that are more satisfied with the level of choice of treatments are less likely to use the websites.

Discussion

In this section we briefly discuss our main findings on the determinants of the intention to use doctor-rating websites and
the level of awareness and actual usage of these websites. Results show that sociodemographic characteristics (in particular
income and ethnicity) and the doctor–patient relationship are significant determinants of the intention to use these websites.
Regarding the latter we further show that, from a patient's perspective, the doctor and the Internet can be seen as being
both ‘complementary’ and ‘substitute’ sources of information. Yet, we find that awareness and usage of doctor-rating websites
is low in our sample. In what follows we will discuss these results and relate them to evidence from the literature.

Intention to use

On the sociodemographic variables the results that show that White British and respondents who reported income in higher brackets
said that they were less likely to use doctor-rating websites, are partly in contrast to what found by the previous literature13–15 and can signal that White British subjects and respondents with higher self-reported income may feel less in need of checking
online doctor ratings, perhaps because they may also have private, or employer-paid, health insurance schemes, or because
they are in the position of directly accessing alternative sources of information through their networks of acquaintances.
Another possible explanation may be that White British individuals may trust less information that exists online and they
have more concerns about confidentiality issues as shown in a study among different socioeconomic groups in the US by Brodie
et al.16 As the estimated effect of these variables appear to be robust across all empirical specifications, these findings seem to
suggest that online doctor-rating websites are likely to be particularly attractive to subjects with non-White British ethnicity
and less favoured economic background.

On the other hand, the lack of statistical significance in the ordered logit estimates, seems to suggest that, while age can
be a significant factor in explaining the awareness of Internet for health information, it is not significantly explaining
the intention to use doctor-rating websites once subjects are made aware of their existence. The analogous lack of significance
for the respondents’ gender, on the other hand, does not support the view that women in the UK may be more likely to use patients’
sources of information and rating websites, although they have been found to desire patient choice more than men.9 Both results differ from the findings from the literature on the use of online information. The literature has shown that
sociodemographic characteristics are major determinants of usage of online health information. In particular, women and younger
adults are more active ‘online health information seekers’.10,17–20 Education has also been found to determine the usage of online and offline health information. Cotton and Gupta15 and Diaz et al,13 carried out research into the characteristics of online and offline health information seekers and found that less educated
individuals were less likely to be users of online health information.

Therefore even though, according to our findings, intentions to use do not differ significantly across all sociodemographic
characteristics, actual usage may be greatly determined by access rather than only by intention to use, with the former substantially
differing according to socioeconomic and sociodemographic characteristics. That is, there may exist income-related, education-related
or age-related barriers to actual access that prevent individuals from using doctor-rating sites even though their intentions
to use them are similar.

From the perspective of the doctor–patient relationship, the finding that patients with GPs of the same gender tend to be
more likely to use the websites is of particular interest, and it is consistent with the analogous effect found for the likelihood
of being aware of those websites. Considered together these findings point to the possible explanation that the doctor and
the Internet may sometimes be seen by patients as ‘complementary’, rather than alternative, information channels.14 This interpretation is further confirmed by the finding that respondents for whom the doctor is able to listen to them, and
who perceive the nature of the relationship with their GP as friendly, also tend to be more likely to use the websites.

The doctor–patient gender concordance, in fact, has often been reported in the literature as a factor associated with higher
patient satisfaction with the consultation as well as better outcomes.21 If we interpret the gender match variable as an indication of satisfaction with the consultation, our finding indicates that
the intention to use (as well as being aware of) the doctor-rating websites is not necessarily the result of a poor consultation.
Instead, the Internet and the doctor are likely to be seen as complementary, rather than alternative, information channels.
This could explain why patients that consider hospital statistics as a good source of information are more likely to use these
websites. Indeed, this type of information might not be provided by the doctor in a consultation.

Indeed, a study by Stevenson et al22 shows that although patients use the Internet increasingly more, they show no intention of doing so with the aim of disrupting
the existing balance of roles during the doctor–patient consultation. They all mentioned the Internet as an additional resource
of health and healthcare information. Other evidence suggests that patients with hypertension who search for more information
on the Internet, in addition to that they receive from their doctor, may be more engaged in their treatment, and therefore
more willing to adhere to medication prescribed by them.23

Finally, the positive association between willingness to use doctor-rating websites and levels of satisfaction with the level
of choice of GP, and of outpatient appointments in the hospital, can be considered as reinforcing the above-discussed interpretation
that some dimensions of the doctor–patient relationship may be ‘complementary’ with online information. For instance, patients
who are more satisfied with their GP because they feel the latter is more friendly and empathic may also be more likely to
engage more actively with health and healthcare information more generally. These results, together with the finding that
the respondents who are more satisfied with the level of choice of treatments are less likely to use the websites, suggest
that the choice of doctors and providers may be seen as only instrumental for the choice of treatment, and therefore respondents
that are happy with treatment choice levels are less likely to shop around for different doctors’ opinions.

On the other hand, there may be other dimensions in the patient–doctor relationship which seem to rather point to a ‘substitute’
relationship with information on the Internet. For instance, the fact that respondents who feel that their doctor explains
things clearly and consider the advice given by the GP as being important are less likely to use online rating websites, suggests
that when they are generally more satisfied with the feedback provided by their doctor they are less concerned about finding
about alternative doctors and compare them with their current GP.

This result on a ‘substitute relationship’ is consistent with previous evidence by Diaz et al13 that found that 11% of their respondents said they would rather use the Internet ‘instead of seeing or speaking with their
doctors’, and that 59% of respondents ‘did not discuss information with their doctors’. It also seems in line with the study
by McMullan14 that indicates that patients who become dissatisfied with the information provided to them by the health professionals are
more likely to seek confirmation of the information given as well as additional information on the Internet.

Our findings that online information can be used not only as ‘substitute’ but also, and perhaps mainly, as ‘complementary’
to several dimensions of the doctor–patient relationship do not seem to entail any particular evidence suggesting that online
ratings may put in danger the doctor–patient relationship, an important aspect which has been raised in the literature.24,25 The ‘complementarity’ findings, in particular, seem consistent with the evidence from the USA which shows that the vast majority
of the reviews by patients are generally rather positive.7,8,26 Taken together, this evidence can be seen as providing little support to the related concern that the likeliest to use online
ratings and enter actual comments may be the most disgruntled patients.27

As for the other aspects of the patient–doctor relationship, the finding that the more autonomous patients are in their healthcare
decisions, the more willing they are to use the rating websites is also consistent with previous evidence: a study by McMullan,14 for instance, reports that patients would seek health information before a consultation ‘to manage their own healthcare independently’.
These may be the type of people who are ‘more likely to be health-oriented’ or ‘health conscious’, and therefore be more proactive
in consultations.28

Awareness and actual usage

Only 15% of our sample were aware of the existence of these websites, indicating that the awareness and, consequently, usage
of these online sources is still quite limited in the UK, although significantly higher than what the previous studies have
shown.5

Concerning the low reported rate of active usage of doctor-rating websites, the finding is not too surprising given that the
survey was done among a sample of respondents from the general population: the reason why many more respondents were aware
of the online ratings than did actually use it may simply be because those subjects did not actually need to see a doctor.

These figures are substantially in line with previous evidence brought forward from the literature from the UK.5,6 A slow uptake of online ratings has also been reported in the US. It is indicative that only 6% of Americans were aware of
Hospital Compare, the quality reporting website maintained by the Centres for Medicare and Medicaid Services (CMS).29 Gao et al8 analysed 386 000 national ratings from 2005 to 2010 in the USA and showed that only one of six physicians among those included
in the study had received some rating. Lagu et al7 also reported a low average number of ratings per physician.

The fact that even in the USA, a more market-oriented health system, the use of similar sites is not much higher may suggest
that the slow uptake in the UK cannot be attributed only to the early stage of the ‘choice’ model. Considered together, these
results may pose concerns on the reasons and consequences of the lack of patient awareness and usage of online health-related
information.

Previous studies in the USA have reported a number of reasons behind this slow uptake, including (1) the preference for more
traditional information channels, such as recommendations by family and friends; (2) the lack of time and (3) in many cases
the fact that people do not recognise that the quality of care may vary.25

As for the UK, our study confirms that not only awareness of rating websites is still limited among a sample of respondents
from the general public in London, but awareness and willingness to use per se do not seem a sufficient condition to guarantee
active usage. This poses a double challenge from a clinician and health policy perspective. In fact, on the one hand, the
documented correlation between online ratings and other measures of healthcare quality, including survey-based ratings and
clinical quality indicators,30,31 necessarily requires that patients have already gone through three preliminary hurdles, namely (1) being aware of, (2) having
effective access to and (3) being active users of the doctor-rating websites. If the ultimate goal is indeed the continuous
enhancement of healthcare quality, the effective removal of this triple hurdle is likely to become the next priority to guarantee
the full spread of online rating website.

On the other hand, while appropriate online and offline informational campaigns are likely to overcome the first hurdle, thus
effectively raising patients’ awareness of online ratings as a potential source of information on provider quality, informational
campaigns alone can fail to grant effective access and trigger actual changes in behaviour. Alike in several other health
contexts, in particular, ‘nudging’ behaviour may be difficult as a mere consequence of accessing more information.

If this is the case, other avenues should be explored to increase the active usage of rating websites by patients who are
already aware of them. For instance, the evidence brought forward by the present study confirms the importance of the doctor–patient
relationship as a factor determining individuals’ awareness of and willingness to use online ratings25,32–35 and suggests that tailored behavioural interventions based on the doctor–patient relationship have the potential to help
patients to overcome the last hurdle and actively engage with online ratings.

Limitations of the study

The convenience field survey was considered the most appropriate administration mode to involve a sample of respondents from
the general population. An online survey, in fact, by exclusively reaching the segment of active internet users, would have
failed to address the main goal of the study.

However, while dictated by practical issues, the convenience sampling is a limitation of the study, and tends to over-represent
respondents who are currently not employed, such as unemployed, retired and students. Also the fact that the study was conducted
in only one borough of London limits the possibility to immediately generalise the findings to the broader UK population.

In an attempt to make such limitations of smaller concern to enhance the external validity and generalisability of the analysis,
we have (1) chosen a borough which comprises a mix of both affluent and deprived neighbourhoods from heterogeneous ethnic
backgrounds; (2) conducted surveys in the field at different public locations and at different times of the day and of the
week to approach both working and non-working members of the public and (3) controlled for a wide range of sociodemographic
measures in the statistical analysis.

Conclusions

By collecting a broad range of information on the sociodemographic characteristics of the respondents, their views and perceptions
of the most important aspects of healthcare quality, patient choice and doctor–patient relationship, the study explicitly
explores the usage of doctor-rating websites, the determinants of respondents’ awareness of the doctor ratings websites, and
of their intention to use the sites in the future.

This study brings forward direct evidence suggesting that the awareness and actual usage of doctor-rating websites in the
UK remains particularly low. The main finding suggests that the doctor–patient relationship plays a key role in explaining
intention to use the websites and that the doctor has both a ‘complementary’ and ‘substitute’ role to play with respect to
Internet information.

The existence of both ‘substitute’ and ‘complementary’ effects between the doctor and the Internet as information channels
is not at all conflicting. In fact, they both indicate that the level of concordance achieved during the consultation is likely
to define whether or not individuals will seek for further information channels, such as the Internet.

The findings of our study thus contribute also to the wider debate on the inter-relationships between Internet usage and the
doctor–patient relationship.7,25,26,32–35 The argument, sometimes addressed by the previous literature, that information on the Internet can threaten the trust relationship
and the balance of roles between doctors and patients, seems a concern which is not supported by our evidence. If any, a potential
challenge to the doctor–patient relation can only affect the patients who already feel dissatisfied with the ability of their
doctor to listen to them and provide them with enough information regarding their condition, or with the level of their choice
for healthcare treatments.

The above, however, can hardly be seen as a serious threat by those who advocate a greater choice by patients. On the contrary,
if the latter is indeed a priority in the health policy agenda, online information on healthcare providers should be seen
as a challenging opportunity to enhance patients’ choice in healthcare, and public engagement with health information, especially
for the less favoured segments of the population. Indeed, our findings suggest that subjects of non-White background and with
lower income are more willing to use online ratings.

Finally, our study highlights that subjects who use doctor-rating websites are unlikely to be representative of the overall
patients’ pool. In particular, they tend to over-represent opinions from non-White British, medium-to-low-income patients
who are not satisfied with their choice of healthcare treatments. Accounting for differences in the users’ characteristics
is important when interpreting results from doctor-rating sites and when informing interventions that aim at enhancing the
public engagement with health information on the Internet, and the representativeness of the users who seek and provide feedback
online.

Footnotes

Contributors MM, MMG and CS contributed to the design of the questionnaire, carried the data analysis, drafted the paper, and amended
the paper according to the reviewers suggestions. The remaining authors contributed to the questionnaire design, data collection
and drafting of the paper.

Funding This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors.

Data sharing statement Technical appendix, statistical code and dataset available from the corresponding author at m.miraldo@imperial.ac.uk. Consent
for data sharing was not obtained but the presented data are anonymised and risk of identification is low. All authors had
full access to all the data in the study and take responsibility for the integrity of the data and the accuracy of the data
analysis.