This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Background

Many countries have introduced elements of managed competition in their healthcare
system with the aim to accomplish more efficient and demand-driven health care. Simultaneously,
generating and reporting of comparative healthcare information has become an important
quality-improvement instrument. We examined whether the introduction of managed competition
in the Dutch healthcare system along with public reporting of quality information
was associated with performance improvement in health plans.

Methods

Experiences of consumers with their health plan were measured in four consecutive
years (2005-2008) using the CQI® health plan instrument 'Experiences with Healthcare and Health Insurer'. Data were
available of 13,819 respondents (response = 45%) of 30 health plans in 2005, of 8,266
respondents (response = 39%) of 32 health plans in 2006, of 8,088 respondents (response
= 34%) of 32 health plans in 2007, and of 7,183 respondents (response = 31%) of 32
health plans in 2008. We performed multilevel regression analyses with three levels:
respondent, health plan and year of measurement. Per year and per quality aspect,
we estimated health plan means while adjusting for consumers' age, education and self-reported
health status. We tested for linear and quadratic time effects using chi-squares.

Results

The overall performance of health plans increased significantly from 2005 to 2008
on four quality aspects. For three other aspects, we found that the overall performance
first declined and then increased from 2006 to 2008, but the performance in 2008 was
not better than in 2005. The overall performance of health plans did not improve more
often for quality aspects that were identified as important areas of improvement in
the first year of measurement. On six out of seven aspects, the performance of health
plans that scored below average in 2005 increased more than the performance of health
plans that scored average and/or above average in that year.

Conclusion

We found mixed results concerning the effects of managed competition on the performance
of health plans. To determine whether managed competition in the healthcare system
leads to quality improvement in health plans, it is important to examine whether and
for what reasons health plans initiate improvement efforts.

Background

Nowadays, several western countries have introduced some form of managed competition
in their healthcare system with the aim to accomplish a more efficient and more demand-driven
healthcare [1,2]. For these overall aims to be achieved, the performance of both healthcare providers
and health plans has to be assessed and publicly reported [3]. Consumers/patients need this quality information in order to be able to act as informed
and critical decision-makers on both the healthcare market (choosing between healthcare
providers) and the health insurance market (choosing between health plans). Health
plans are expected to use the quality information on healthcare providers to differentiate
between healthcare providers when contracting health care selectively on the health
purchaser market. Both health plans and healthcare providers should use the information
on their own performance to monitor the quality of their service and/or care and to
initiate quality-improvement projects. Generating and reporting of comparative healthcare
information has therefore become an important quality-improvement instrument in several
countries [4-6].

Fung et al. has shown that public reporting of quality information does indeed stimulate
hospitals to initiate quality-improvement projects and that some consumers use this
information when choosing a hospital [3]. Hibbard et al. showed that publicly reporting quality information stimulated hospitals
to start quality-improvement projects but only for those areas where the quality of
health care was inferior [7,8]. Two years later, the quality of hospital care had improved for these areas and this
improvement was more profound for hospitals that performed worse than expected at
baseline.

The studies of Fung and Hibbard et al. focused on hospital care. In the present study,
we will focus on the question whether the introduction of managed competition within
the healthcare system along with reporting of quality information is associated with
quality improvement in health plans. Concerning health plans, researchers so far have
only investigated whether consumers use quality information when choosing a health
plan. It appeared that consumers do use the information and tend to choose better
performing health plans [3,9]. Empirical studies on the effects of the introduction of managed competition and
the publication of quality information on the performance of health plans is, however,
lacking.

To answer this question, we used data on the performance of health plans in the Netherlands.
In January 2006, the Dutch government introduced more managed competition in their
healthcare system by enacting a new health insurance law. The most important changes
in and characteristics of the Dutch health insurance system are presented in Table
1. A basic obligatory insurance, covering the entire population, was introduced. All
consumers have a free choice between insurance companies during annual open enrolment
periods (November-January). Health plans are obliged to accept every citizen for the
basic package and are no longer allowed to select favourable risks or to differentiate
the premium and conditions according to (proxies for) risk [10]. Health plans can negotiate with the healthcare providers on the price, content and
organisation of the care and do not have to enter into a contract with every provider.
In addition, they can offer collective arrangements to their insured against a reduced
nominal premium. These reforms should make switching health plans easier and are supposed
to lead to a more demand-driven health care system that is cheaper and of higher quality
[11].

Table 1. The Dutch health insurance system after the insurance reform of 1 January 2006

In addition, the performance of health plans is assessed annually in the Netherlands
using the standardized CQI® health plan instrument 'Experiences with Healthcare and Health Insurer'. Consumer
Quality Index (CQ-index or CQI) instruments assess the quality of health care seen
from the consumer's perspective, and measure consumers' experiences instead of inquiring
after their satisfaction [12,13]. The experiences of consumers/patients with their health plan have been measured
in four consecutive years (2005 to 2008) starting the year before the introduction
of the new insurance law [14-17]. In all years, the resulting comparative quality information was published on the
healthcare portal http://www.kiesBeter.nlwebcite (Make better choices), which is an initiative of the Dutch Ministry of Health, Welfare
and Sport. In addition, a press release was published mentioning the quality aspects
that in general need improvement the most. The health plans received a confidential
company report in which their performance was compared with the average across all
health plans.

We examined whether the performance of the health plans improved with the introduction
of managed competition in the Dutch health insurance system and the coinciding publication
of comparative quality information by looking at the changes over the years in consumer
experiences with 2005 as baseline measurement. In line with the studies of Hibbard
et al [7,8,18], we hypothesized that all health plans would improve their performance but that the
changes over the years would be more profound for identified areas of improvement
and for health plans with an inferior performance at the first measurement.

Methods

Available data

Experiences of consumers with the provided health care and the services of their health
plan were measured in four consecutive years (2005-2008) using the standardized CQI® instrument 'Experiences with Healthcare and Health Insurer'. Each year, this questionnaire
was sent to a different random sample of insured. The CQI health-plan instrument consists
of items on health-plan services and received healthcare in the past twelve months.
It contains 54 core items on consumer experiences, four global ratings (family physician,
specialist, healthcare, and health plan), one item on the likelihood to recommend
the health plan to friends and family, and several items on consumer characteristics.
The questionnaire is partly a transformation of the CAHPS 3.0 Adult Commercial Questionnaire
[19]. For this study, we focused on seven quality aspects of the health-plan services
(See Table 2): the global rating of health plan, conduct of employees, health plan information,
access to call centre, getting the needed help from the call centre, reimbursement
of claims and transparency of (co)payment requirements.

Published results

In each year the following statistical analyses were performed [14-17]. The respondents and non-respondents were compared concerning age and sex in order
to determine whether a response bias occurred. Using multilevel linear regression
analyses (consumers' experiences were nested within health plans), means with comparison
intervals (1.39 × standard error; [20]) were calculated per aspect and per health plan while adjusting for consumers' age,
education and self-reported health status. Next, health plans were divided in three
groups by determining whether the comparison interval overlapped with the overall
mean of all health plans. This classification of health plans was published on the
website http://www.kiesBeter.nlwebcite using stars: * = below average (comparison interval lies below overall mean), **
= average (comparison interval overlaps with overall mean) and *** = above average
(comparison interval lies above overall mean).

In addition, the research institute (NIVEL) published a press release each year on
their own website highlighting the most important results of the study. The press
release mentioned among other things on which quality aspects the overall performance
of the health plans was inferior. We expected that health plans would especially try
to improve these aspects. The following areas of improvement were identified in 2005:
transparency of (co)payment requirements, access to call centre and health plan information.
The results of 2005 were published in at least five (national) news papers and the
media also covered the launch of the information on the website http://www.kiesBeter.nlwebcite.

Secondary statistical analyses

We combined the data of the four years and performed another set of multilevel regression
analyses with three levels: respondent, health plan and year of measurement. Per year
and per quality aspect, we estimated the overall mean across health plans while adjusting
for the consumers' age, education and self-reported health status. To determine whether
the performance of the health plans improved over the years, we tested for linear
and quadratic time effects and compared the performance in 2005 with the performance
in 2008 using chi-squares.

In order to determine whether the changes over the years were more profound for quality
aspects that were identified as aspects that needed improvement most and for health
plans with an inferior performance at the first measurement, we focused on the differences
between 2005 and 2008. We choose the longest time frame possible, because health plans
need time to implement and put into effect improvement efforts. First, we examined
whether the performance of health plans increased more often on quality aspects that
were mentioned as important areas of improvement in the 2005-press release than on
the other quality aspects. Then, we investigated whether the differences in performance
between 2005 and 2008 depend upon the performance of health plans in 2005 as published
on the Dutch website http://www.kiesBeter.nlwebcite. Health plans who did not participate in the 2005-study are therefore excluded from
these analyses. For each group of health plans (below-average scoring, average scoring
and above-average scoring health plans), we estimated the mean per aspect for 2005
and 2008 and tested whether the differences were statistically significant using chi-squares.

Results

Respondents

In 2005, 13,819 respondents (response = 45%) of 30 health plans filled out the questionnaire.
The number of respondents per health plan varied from 167 to 1,287 (mean = 461; SD
= 223.80). In 2006, 8,266 respondents (response = 39%) of 32 health plans filled out
the questionnaire, and the number of respondents per health plan varied from 205 to
348 (mean = 258; SD = 38.58). In 2007, the questionnaire was filled out by 8,088 respondents
(response = 34%) of 32 health plans, and the number of respondents per health plan
varied from 154 to 353 (mean = 253; SD = 43.11). In 2008, the questionnaire was filled
out by 7,183 respondents (response = 31%) of 32 health plans, and the number of respondents
per health plan varied from 170 to 313 (mean = 224; SD = 36.73). The large differences
in number of respondents per health plan in 2005 were due to mergers between health
plans during the course of the study.

Table 3 shows the mean age and percentage of male of the respondents and non-respondents.
In all years, the respondents were older than the non-respondents. In 2005, 2006 and
2007, more women than men filled out the questionnaire.

Table 3. Comparison of person characteristics between respondents and non-respondents

Overall changes in performance

Table 4 shows that the performance of the total group of health plans changed significantly
over the years on all quality aspects. The performance on conduct of employees and
transparency of (co)payment requirements improved from 2005 to 2008 leading to a significant
better performance in 2008 than in 2005 (both χ2's > 8.90; p < .01). Concerning access to call centre, getting the needed help from
the call centre and the reimbursement of claims, the performance of health plans first
declined from 2005 to 2006 and than increased. For these three quality aspects, we
found no significant differences in performance between 2005 and 2008 (all χ2's < 1.45). The performance on health plan information increased from 2005 to 2006
and than stabilized and the health plan information was better in 2008 than in 2005
(χ2 = 16.93; p < .001). The general rating of health plans increased from 2005 to 2007
and then decreased somewhat in 2008, but the general rating in 2008 was still significantly
higher than in 2005 (χ2 = 8.59; p < .01). It is important to note that the changes in performance over the
years are in general small.

Table 4. Results of multilevel analyses: estimated mean and standard error per quality aspect
and chi-squares indicating the change over years for all health plans

Comparison of overall changes in performance between quality aspects

Three areas of improvement were mentioned in the 2005-press release, i.e., transparency
of (co)payment requirements, access to call centre and health plan information. Health
plans performed significantly better in 2008 than in 2005 on two of these aspects,
namely health plan information and transparency of (co)payment requirements. The performance
on access to call centre did not differ between 2005 and 2008. Health plans performed
significantly better in 2008 than in 2005 on two out of four aspects that were not
identified as areas of improvement, namely general rating of health plan and conduct
of employees. The overall performance of health plans on the two other aspects, getting
the needed help from the call centre and the reimbursement of claims, did not differ
between 2005 to 2008. In other words, the performance of health plans did not improve
substantially more often for the quality aspects that were identified as important
areas of improvement in the press release.

Comparison of changes in performance between health plans

Table 5 shows the number of health plans that scored below average, average and above average
on each of the quality aspects in 2005. In Table 6, the estimated mean score of each group of health plans (below-average scoring, average
scoring, and above-average scoring health plans) is presented per quality aspect for
2005 and 2008. Also, the chi-squares of possible differences between 2005 and 2008
are given.

Table 5. Number of health plans who scored below average, average and above average in 2005

Table 6. Results of multilevel analyses: estimated mean and standard error per quality aspect
in 2005 and 2008 and chi-squares indicating the time effects for health plans who
scored below average, average or above average in 2005

It appeared that the performance on getting the needed help from the call centre did
not differ significantly between 2005 and 2008 for health plans that scored either
below average, average or above average in 2005. For conduct of employees, health
plan information and transparency of (co)payment requirements, the performance of
both below-average scoring and average scoring health plans increased from 2005 to
2008, while the performance of above-average scoring health plans did not change significantly.
The performance of below-average scoring health plans also improved from 2005 to 2008
on access to call centre and general rating of health plan, while the performance
of average and above-average scoring health plans did not change significantly. For
reimbursement of claims the following picture emerged: health plans that scored below
average in 2005 increased their performance from 2005 to 2008, the performance of
average scoring health plans did not change significantly, whereas the performance
of above-average scoring health plans declined from 2005 to 2008.

Discussion

The aim of the present study was to determine whether the introduction of managed
competition in the Dutch healthcare system and the coinciding publication of comparative
quality information on health plans was associated with performance improvement in
health plans. Experiences of consumers with their health plan were measured in four
consecutive years (2005 to 2008) starting the year before the introduction of a new
health insurance law [14-17]. In all years, the resulting comparative quality information was published on a Dutch
website along with a press release mentioning the quality aspects that in general
needed improvement the most. We expected that the performance of all health plans
would improve over the years, endorsing the expected effects of managed competition.
Moreover, following Hibbard et al., we hypothesized that the improvements over the
years would be more profound for quality aspects that needed improvement most and
for health plans that performed inferior at the first measurement (year 2005) [7,8,18].

When we look at the changes in performance between 2005 and 2008, the expected overall
improvement in performance was found for only four out of seven quality aspects, namely
general rating of health plans, conduct of employees, health plan information and
transparency on (co)payment requirements. For three other aspects (i.e., access to
call centre, getting the needed help from the call centre and the reimbursement of
claims), we found that the overall performance first declined from 2005 and 2006 and
then increased from 2006 to 2008; the performance of health plans was, however, not
significantly better in 2008 than in 2005.

The decline in overall performance from 2005 to 2006 on these three quality aspects
can be explained as followed. In January 2006, the Dutch government enacted the new
insurance law. The new law brought about several changes for Dutch citizens and created
turmoil within the Dutch population. In the beginning of 2006, much more consumers
than usual telephoned their health plan for extra information decreasing the access
of the call centre and making it more difficult for health-plan employees to provide
the needed help. The administrative burden associated with the introduction of the
new health insurance law could explain the decreased performance concerning reimbursement
of claims. However, after the first year of the new health insurance system, the health
plans did not appear to be able to improve the performance of the call centre and
the reimbursement of claims to a level higher than in 2005.

Above-mentioned results also indicate that the overall performance of health plans
did not improve more often for quality aspects that were identified as important areas
of improvement in 2005 (i.e., transparency of (co)payment requirement, access to call
centre and health plan information) than for the other quality aspects. In short,
the introduction of managed competition in the Dutch healthcare system along with
the publication of comparative quality information only had the assumed positive effects
on the overall performance of health plans for a subset of quality aspects.

Next, we examined whether health plans that performed below average in 2005 improved
their performance more often than health plans that did not perform below average
in that year. On most (six out of seven) aspects the performance of below-average
scoring health plans increased more than the performance of average and/or above-average
scoring health plans. In other words, the idea that health plans who scored relatively
low in 2005 would try harder to improve their performance than health plans who scored
relatively high was confirmed [7,8]. It is, however, important to keep in mind that relatively bad-performing health
plans had more possibilities for improving their service. For well-performing health
plans it was probably difficult to improve their performance over the years given
their high point of departure on several quality aspects. For instance, scores on
reimbursement of claims can vary between 1 and 4. Above-average health plans had an
average score of 3.8 in 2005, which is very close to 4, leaving little room for improvement.

An important question is what stimulates health plans to improve their performance.
Three different mechanisms have been proposed to explain why public reporting of quality
information would stimulate healthcare providers to initiate quality improvement projects
[3,8,21]. First, identifying shortcomings may be sufficient to motivate professionals to improve
their performance given their intrinsic motivation to provide service of high quality
(professionalism). Second, comparable to the assumed effects of managed competition, the possible
loss of market share can stimulate efforts to improve quality. Organizations then
have an economical interest to excel in public reports (market forces). Last, it is held that healthcare providers value a good reputation and therefore
do not want to be associated with bad-performing organizations in public reports (reputation protection). The present results can not answer this question conclusively, but the finding that
the overall performance of health plans did not improve more often for general points
of improvement (as mentioned in the press-release) negates the idea that identifying
shortcomings is sufficient to motivate health plans to improve the service they provide.
Relatively bad-performing health plans did show more improvement than relatively good-performing
health plans suggesting that health plans do not want to perform inferior compared
to other health plans. In other words, as in previous studies [3,7,8,18], reputation protection appears to be an factor stimulating health plans to initiate
improvement projects but fear for loosing market share is probably also an important
issue.

In addition, it remains to be seen whether consumers or organisations interesting
in a collective arrangement (for instance, employers or patient organisations) use
the information on Internet to choose between health plans. One way to answer this
question is to establish whether health plans that perform below average indeed loose
market share. Unfortunately, these data were not available. In general, about 3-4%
of the Dutch population switches health insurer each year. This percentage is comparable
to the switching rates in other countries such as Germany (4-5%) and Switzerland (5%)
[10,22,23]. Studies have also revealed that consumers use quality information when choosing
a health plan and that they tend to choose better performing health plans [3,9]. It is, however, unknown whether a switching rate of 3-4% is enough for managed competition
in health care to succeed [24].

Some limitations of the present study have to be noted. For one, a response bias occurred
in all the four questionnaire studies on consumer experiences. Elderly and women responded
more often than younger people and men. Previous studies have revealed that older
people report more positive experiences than younger people; no consistent differences
have been reported for men and women [25,26]. This means that the average performance of the health plans is probably overestimated
in the questionnaire studies. Fortunately, the response bias was present in all the
four years the consumer experiences were measured. This limitation thus probably did
not affect our conclusions concerning the changes in performance over the years.

It is also important to note that the changes we found over the years are small. Although
some differences are statistically significant, we have to ask our selves whether
we can derive policy implications from these changes. In addition, the design of the
present study did not allow us to determine whether the introduction of managed competition
and the publication of comparative quality information were responsible for the observed
changes in performance.

Future research should examine whether and for what reasons health plans initiate
improvement projects. Ideally, experiments should be carried out in which health plans
are randomly assigned to one of several conditions: only a confidential report, only
a public report or a combination of a confidential and public report on their performance.
At the same time, it has to be determined how long it takes for improvement efforts
of health plans to be implemented and to translate into more positive experiences
of consumers. In addition, studies that investigate whether consumers use the comparative
quality information when choosing a health plan and whether relatively bad-performing
health plans indeed loose market share are essential.

Conclusion

The results concerning the effects of managed competition and the publication of comparative
quality information on the performance of Dutch health plans are mixed. The overall
performance of health plans did not improve over the years for all quality aspects
and the improvements were also not more profound for quality aspects that needed improvement
most. Health plans whose performance was below average in the first year (2005), however,
did improve their performance over the years more often than health plans with an
average or superior performance in that year. To determine whether managed competition
in the healthcare system leads to quality improvement in health plans, researchers
should examine whether and for what reasons health plans start improvement efforts.

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

MH participated in the questionnaire studies, drafted the manuscript and contributed
to all other aspects of the study. PS performed the statistical analyses. JR and DD
contributed to the acquisition of the data, drafting the manuscript and critical revision
of this manuscript. All authors read and approved the final manuscript.

Acknowledgements

The health plans financed the data collection in the different studies. We wish to
thank them for their participation and for making the data available for our secondary
analyses.