This report was produced by a joint team INRA and Deloitte & Touche for DG Health & Consumer Protection and represents the views of the contractor or author. These views have not been adopted or in any way approved by the Commission and should not be re-lied upon as a statement of the Commission's or DG Health & Consumer Protection's views. The European Commission does not guarantee the accuracy of the data included in this report, nor does it accept responsibility for any use made thereof.

Background to this assignment 1.1.1 The EU consumer policy In its ‘Consumer Policy Strategy 2002-2006’1, the European Commission states that the de-velopment of a consumer policy at European Union (EU) level is the essential corollary of the progressive establishment of the internal market. Indeed, the free circulation of goods and services requires the adoption of common, or at least convergent, rules to ensure both suffi-cient protection of consumer interests, and the elimination of regulatory obstacles and com-petitive distortions.

The EU consumer policy includes the following areas of concern:

o A number of safety, economic and legal issues relevant to consumers in the internal market place, and consumer information and education. Products and services placed on the internal market should be safe. Consumers should receive adequate and rele-vant information in order to make appropriate choices. Consumers should also be pro-tected from abusive practices.

o A coherent and common environment that will ensure that consumers are confident in shopping across borders throughout the EU. Consumers should have comparable op-portunities to benefit fully from the potential of the internal market in terms of greater choice, lower prices, and the access to and affordability of essential services.

1.1.2 Policy-makers need information In its strategy paper, the Commission underlined the need for consumer policy to be backed by relevant information and data, in order to adjust policies and set the appropriate priori-ties. It stated that a more comprehensive, systematic and continuous effort is needed to de-velop a suitable knowledge base as an essential tool for policy-makers. The development of indicators on consumer satisfaction is mentioned explicitly as one of the key actions needed to expand this knowledge base and to improve the quality of consumer policies.

The Commission, through its Directorate General on Health and Consumer Protection (DG SANCO), has already undertaken preliminary initiatives to gauge the degree of ‘satisfaction’ of consumers. These initiatives include the EUROBAROMETER and focus groups.

Now the time has come to develop a more rigorous policy tool, in the form of a set of con-sumer satisfaction indicators. To achieve this objective, as a first step the Commission launched a call for tenders for the development of a European set of indicators for consumer satisfaction and for the launch of a Pilot survey. The assignment was awarded to a joint team of INRA Europe and Deloitte. This report presents the final results of this assignment2.

1 COM (2002) 208 of 7 May 2002 2 Preliminary research results and a more comprehensive discussion of the components of consumer satisfaction have already been reported to the Commission within the first and second interim reports. Only a synthesis of these elements is reported in this final report. Development of consumer satisfaction indicators & Pilot survey 2

1.2

Scope of the assignment 1.2.1 Main objectives and expected outcomes The first objective of the assignment was to develop a methodology for the construction of consumer satisfaction indicators in the European Union. This methodology should be practi-cal and have a sound scientific basis, reflecting recent research insights into consumer satis-faction and its measurement.

The second objective was to develop and conduct a Pilot survey, based on the proposed methodology. The purpose of this Pilot survey was to test the methodology and its underly-ing modelling, and to propose a preliminary set of indicators.

Both objectives were interlinked. The analysis of the Pilot survey’s outcomes was expected to indicate some adaptations to the methodology developed in the first stage.

The report was also intended to include an appropriate survey framework (i.e. question-naires, population and sampling, survey methods, etc.), a proposal for the statistical meth-ods to be used, and methods for calculating and visualising the consumer satisfaction indica-tors.

1.2.2 Policy-making needs There are many ways of designing indicators. Within this assignment, a key criterion was that the indicators should be designed in such a way that they are helpful in European con-sumer policy-making.

More specifically, the set of consumer satisfaction indicators to be developed was meant to enable the European Commission to: • understand how consumers perceive certain markets, what their main requirements are, and how key service areas meet their expectations • identify priorities for improvements, i.e. the areas where improvements will produce the greatest gain in consumer satisfaction • benchmark performance amongst EU countries within particular sectors • benchmark sectors’ performance within a specific country or at the EU level • set goals for improvement and monitor progress.

Ideally, the indicators should become a reference tool for European policy-makers, allowing them to gauge overall consumer satisfaction levels and to measure the specific elements that determine satisfaction levels in individual areas. The indicators should also provide signals of whether the internal market is functioning properly and whether corrective regulatory or en-forcement measures need to be taken.

Development of consumer satisfaction indicators & Pilot survey 3

1.2.3 Geographical coverage The original specification within the tender (issued for the first time in 2002) was targeted at the former composition of the EU, covering the 15 countries that were then members. The Pilot survey initially foreseen in the proposal of the contractors covered these countries.

During the assignment the Commission and the contractors agreed to modify the original work programme, to ensure that the proposed consumer satisfaction indicators would be ap-plicable to the new EU members (e.g. to avoid ignoring factors that are of particular impor-tance in these countries). This led to some changes in the initial research approach and in the sample of countries used in the Pilot survey.

1.2.4 Sector coverage Some existing tools and surveys in certain countries, such as national customer satisfaction indexes (see next chapter), produce ‘overall’ indicators of consumer satisfaction, relating to the economy as a whole.

This was not the purpose of this assignment. Rather, the Commission was interested in the construction of indicators that covered a relatively small number of sectors in a rigorous and in-depth way. The sectors to be included within the scope of this assignment were:

Extension to other (and possibly all) sectors of the economy could be envisaged at a later stage, but was not part of the assignment. Care has been taken, nevertheless, that such possible extensions at a later stage would not be jeopardised.

In summary, the indicator set should allow meaningful comparisons of how consumers feel (a) across sectors in one Member State; (b) in one sector across Member States; (c) over time.

3 Following the outcomes of the focus groups, it was decided in common agreement with the Commis-sion, that the sector Utilities (gas, water, electricity) was to be represented by one of its components in the Pilot Survey. This was necessary in order to respect the 20 minutes’ duration of the interview. The electricity sector was selected for the pilot survey. However, the model that was developed re-mains applicable to all three sectors. Development of consumer satisfaction indicators & Pilot survey 4

1.2.5 Additional requirements When developing the tools and methodology, the following considerations were also kept in mind:

o The methodology should be powerful as well as practical and easy to use at a reason-able cost. This means that some trade-offs have to be made. • A ‘perfect’ model for measuring consumer satisfaction would take into ac-count all possible factors for a given sector. Such a model may be complete, but it also risks becoming irrelevant from a practical point of view. Indeed, the questionnaire surveys that are required to feed data into a complex model might be so cumbersome and time-consuming that it is no longer feasible to undertake the survey. • A ‘perfect’ model for one sector will be different from that for another sec-tor. This means that comparability will be lost. Such a model could also quickly become outdated, as new changes in the sector would require con-tinuous adaptation. Therefore, a less perfect model may be preferable if it allows greater comparability between sectors and is more robust against changes over time. There is thus no point in merely constructing a methodology for consumer satisfaction measurement that is ‘perfect’, if it would not be economically feasible to undertake the surveys.

o Consumer satisfaction is not something static, but evolves dynamically with the envi-ronment. For instance, satisfaction is strongly related to expectations, and in a period of economic downturn (or in regions that are declining), people tend to lower their ex-pectations. Such considerations should also be built into the survey methodology, to ensure that the results remain reliable and useful over time. The model should allow adaptation at a later stage, if this is necessary.

1.3

Methodology adopted 1.3.1 Overview INRA and Deloitte have undertaken a range of activities and applied several methodologies, in order to complete this assignment.

The key steps in the assignment were: o Clarification of objectives and requirements o Desk research o Organisation of focus group discussions o Model development o Questionnaire design o Conducting a Pilot survey o Statistical analysis of the survey data o Development of final guidelines.

On several occasions, inputs were obtained from experts in the field of customer satisfaction and statistical analyses, to ensure the validity of the proposals and findings. Development of consumer satisfaction indicators & Pilot survey 5

1.3.2 Clarification of objectives and requirements During the early stages of the assignment, the objectives and scope of the assignment were clearly identified.

The following elements were discussed with the Commission: o Purpose of the assignment o Type and nature of the indicators required o Need for comparability with other initiatives, such as ACSI or EPSI o ‘Consumer’ concept (as compared with ‘customer’) o Geographical coverage o Sector coverage o Characteristics of the Pilot survey.

This clarification of requirements mainly took place at the beginning of the assignment. Some elements were discussed following the submission of the interim reports.

1.3.3 Desk research A major desk research effort was undertaken. The focus was on two main areas: o Existing (national) models and tools for consumer satisfaction indicators o Research results in relation to consumer behaviour and satisfaction measurement.

The existing national consumer satisfaction indices and their underlying methodologies and assumptions were analysed in depth4. This analysis allowed us to identify most of the factors that were likely to be related to consumer satisfaction, as well the way these factors have typically been aggregated into so-called ‘latent’ variables such as ‘quality’.

At the same time, a literature survey was conducted on consumer satisfaction and its meas-urement. The state of the art and recent research results were reviewed. This was necessary to fully understand the current methodologies for consumer satisfaction measurement, and to be able to develop a robust and scientifically valid methodology that could incorporate the Commission’s requirements. The full list of documents analysed is included in Annex 1 of this report.

Thanks to the desk research activities, valuable insight was gained about past and current theories on consumer satisfaction measurement and the construction of indicators.

The desk research was mainly conducted in the period January–March 2004; it continued throughout the assignment.

4 A full description of several of these indicators was provided as an annex to the first interim report. Development of consumer satisfaction indicators & Pilot survey 6

1.3.4 Organisation of focus group discussions In the first phase of the assignment, eleven focus group discussions were organised in a number of different cities across Europe, including three in new EU countries5. The focus groups consisted of consumers that were representative of the population by sex, age and socio-cultural background. The focus groups were led by a psychologist or psychologically trained moderator, who made sure that all topics were discussed, that all participants had a chance to express their thoughts and that views were widely exchanged.

The main purpose of these focus groups discussions was to get a clear picture of the most relevant factors for consumer satisfaction/dissatisfaction. This picture served as a comple-ment, and partially a validation, of the results of our desk research.

The specific objectives of the focus groups were: o To develop a good understanding of what ‘consumer satisfaction’ and ‘consumer dis-satisfaction’ really mean for European consumers (in relation to the sectors that were selected for the Pilot survey) o To identify the most relevant drivers of consumer satisfaction, both generally and in each of the sectors concerned o To assist in the development of a model for consumer satisfaction measurement, a model that could be applicable across sectors and countries.

The group dynamic of the focus groups yielded a rich inventory of opinions and attitudes. The focus group outcomes helped to enrich and validate all the key manifest variables that had been identified throughout the desk research6.

The focus group discussions took place in the period March–April 2004.

1.3.5 Model development Based on the results of the desk research and the focus groups, we developed a ‘provisional’ and fairly comprehensive model for consumer satisfaction measurement. The model is pre-sented in the next chapter.

This model incorporates a long list of factors that are related to consumer satisfaction (e.g. reliability, availability of staff, price transparency, respect of confidentiality, trust and so on), including some specific requirements of the Commission concerning its consumer policy (e.g. cross-border purchasing).

These factors, or ‘manifest variables’ 7, were grouped into higher level ‘constructs’ – the so-called ‘latent variables’ (e.g. quality, price, image, market and personal factors), between which cause and effect relationships are assumed to exist (e.g. ‘price’ affects ‘image’). Sur-veys undertaken on the basis of such a model allow the measurement of consumer satisfac-tion and also explain the main factors which contribute to it.

5 In Germany, France, and the UK, two focus group sessions were organised at different locations. One focus group discussion was organised in Italy, Spain, Poland, Hungary and the Czech Republic. 6 Detailed information about the outcomes of the focus group discussions were provided in the first interim report and its annexes. 7 The definition of “manifest” and “latent” variables will be discussed in the next chapter. Development of consumer satisfaction indicators & Pilot survey 7

This model incorporated several characteristics that are also found in national models for consumer satisfaction measurement. However, it also had distinct features. New elements and relationships were added. It was also designed in such a way as to allow a cross-sector coverage, yet can take into account specific characteristics per sector.

The model development was an interactive process, carried out in the period March–May 2004.

The model developed at that stage was a ‘provisional’ one, since it had to be validated and refined following the Pilot survey. A number of modifications were indeed carried out in No-vember 2004, following a detailed data-analysis.

1.3.6 Questionnaire design The provisional model formed the basis for the design of the questionnaire survey. Essentially, the questionnaire started from the lists of manifest and latent variables that had been defined in the model. For each manifest variable, an appropriate survey question was constructed. All questions were to be rated on a scale from 1 to 10. Where needed, ques-tions were adapted to take into account the specificity of the sector concerned.

In addition, survey questions were added in order to screen respondents, identify service suppliers, and to capture the respondents’ profile. Questions in relation to overall satisfaction for the service were also added.

The questionnaire design took place in the period June–July 2004. It was discussed with the Commission and approved before the survey was launched.

1.3.7 Conducting a Pilot survey The main objective of the Pilot survey was to obtain reliable data that could be used to vali-date the model and finalise the methodology.

In agreement with the Commission, a sampling plan for the Pilot survey was established. The original plan, which included 2,400 surveys in 15 countries, was modified into one whereby 3,600 consumers would be surveyed in eight countries: Germany, the UK, Italy, Belgium, Portugal, Sweden, Poland and the Czech Republic. This allowed for a sample size of 450 interviews per country. For each interviewee, questions were asked in relation to two sectors.

This approach greatly increased the statistical validity of the results, while providing a good representativeness of Europe by region (North-South and West-East) and by country size.

The Pilot survey in the eight countries took place in the period August-September 2004. Development of consumer satisfaction indicators & Pilot survey 8

1.3.8 Statistical analysis of the Pilot survey data Based on the results of the Pilot survey, we conducted a detailed statistical analysis. The aim was to define a consumer satisfaction model, ideally common to the 11 sectors covered by the project.

The statistical analysis of the survey data was conducted in September and October 2004.

1.3.9 Development of final guidelines The objective of this final part was to draw guidelines for the implementation of a (continu-ous) Consumer Satisfaction Survey and the production of relevant indicators. The final part of this report - based on the model developed - includes: o Technical specifications: geographical coverage, target group, sample, interview technique, data collection and questionnaire o Analysis and reporting: standard and model-based.

The finalisation of the methodology and the preparation of this final report were carried out in the period September–November 2004.

Development of consumer satisfaction indicators & Pilot survey 9

2. The measurement of consumer satisfaction 2.1

Introduction This chapter discusses the elements and key concepts of the proposed methodology and the underlying model for the construction of consumer satisfaction indicators.

In section 2.2 we introduce and define a number of terms and concepts. This is important in view of the interpretation of the results, since there is no universally agreed definition or in-terpretation of many of the concepts discussed in this report.

Section 2.3 presents the characteristics of a number of existing models and methodologies for customer satisfaction measurement. Since this subject has been extensively discussed and documented in the first interim report, this final report summarises the key findings in this regard.

We then propose in section 2.4 a new model for consumer satisfaction measurement and its components. This model has been the basis for the Pilot survey. It was later slightly modi-fied, in view of the outcomes of this Pilot survey (see chapter 4).

The final section of this chapter discusses the components of the model.

We would like to stress that there are different ways and methods of measuring consumer satisfaction, depending on the context and purpose of the exercise. The existing methodolo-gies differ from each other in one or more aspects and most of them continue to evolve.

The methodology that we propose in this report can therefore not be considered as the ‘de-finitive’ answer to the on-going discussion on consumer satisfaction measurement. We be-lieve, nevertheless, that it is a highly relevant approach that takes into account the specific nature of the scope of the assignment (sectors and countries) and other requirements and expectations of the Commission – which are somewhat different from the assumptions and requirements that underlie other models and approaches.

2.2

Concepts and definitions 2.2.1 Consumers The concepts of ‘consumers’ and ‘customers’ are related. In some languages, hardly any dis-tinction exists between these two terms. Even in English the terms are often used inter-changeably. For instance, the term ‘customer satisfaction’ is widely used, but a closer look would reveal that in most cases a more precise term would be ‘consumer satisfaction’.

Some experts make the following distinction between a consumer and a customer: o The consumer is the one who uses a product or service, o Whereas a customer pays for the product/service, but may not be the consumer.

Simply stated: the consumer is the ‘user’; the customer is the ‘buyer’.

Another dividing line is that organisations (e.g. companies) can be customers, whilst con-sumer as a concept is reserved for individuals. Consumer in this project means “any natural person who buys a product for purposes that do not fall within the sphere of his/her com-mercial or professional activity” (see Art. 2, e), directive (98/6/EC). By this, we clearly mean that we address the end-consumer (B2C) and not any business intermediaries (B2B). Development of consumer satisfaction indicators & Pilot survey 10

These differences may be important when modelling a consumer satisfaction indicator. In-deed, satisfaction requires experience and use of a product or service. Individuals who pay for a product or service but do not use it do not assess the product or service in the same way as consumers.

In this report, we have chosen to use the term ‘consumer’ systematically, unless reference is made to other documents or existing indicators. It should be borne in mind, however, that the model and methodology developed also allow for analysis of the satisfaction of ‘custom-ers’, at least when they are end-users and not any kind of business intermediary.

‘Satisfaction is the consumer’s fulfilment response. It is a judgement that a product or ser-vice feature, or the product of service itself, provided (or is providing) a pleasurable level of consumption-related fulfilment, including levels of under-or-over fulfilment.’

(Oliver, 1997)

In less ‘technical’ terms: satisfaction is the consumer’s assessment of a product or service in terms of the extent to which that product or service has met his/her needs or expectations. Failure to meet needs and expectations is assumed to result in dissatisfaction with the prod-uct or service8.

Depending on the context, the meaning of ‘consumer satisfaction’ may differ:

o Consumer satisfaction may relate to a particular feature or characteristic of a product or service (e.g. the accuracy of the information provided by a bank), or alternatively it may relate to the product/service as a whole.

In general, it is the satisfaction about the product/service as a whole that merits atten-tion, since this satisfaction influences the consumers’ future buying and consuming be-haviour. Yet it is also important to understand the factors that contribute to (dis)satis-faction. Often, dissatisfaction about one particular feature of a service (e.g. the un-friendliness of staff) leads to dissatisfaction about the service as a whole, even if the satisfaction about the other features is high.

o Another conceptual distinction within measures of consumer satisfaction is whether they relate to satisfaction about a single ‘service encounter’ (e.g. your most recent ex-perience with a bank) or rather reflect ‘cumulative satisfaction’ (e.g. in relation to all your experiences with banks). It is possible to be quite satisfied about one particular transaction in the post office, but be dissatisfied about the quality of the postal services overall (or vice-versa).

Most existing national customer satisfaction indexes are based on the measurement of cumulative satisfaction. This implies that different elements are part of the consumer’s assessment, from their pre-purchase intentions up to their loyalty towards a particular provider.

8 We must add a nuance when it comes to ‘commodity’ services such as urban transport. Consumers may have no explicit or specific expectations for such services, making it hard to define satisfaction in relation to prior expectations. Development of consumer satisfaction indicators & Pilot survey 11

o Another potentially important distinction is between consumer satisfaction about a particular supplier, and consumer satisfaction about the sector as a whole. For in-stance, one may be quite satisfied with the offerings and performance level of a par-ticular mobile phone operator, but in general feel uncomfortable with the pricing and marketing strategy as a whole.

Most national customer satisfaction indexes measure the satisfaction level for the sec-tor as a whole. Yet they do this by calculating a weighted average for the satisfaction about individual suppliers.

For the development of the proposed methodology, we have made the following choices in this regard:

o We measure both consumer satisfaction about services overall, as well as about the services’ particular characteristics: quality aspects, pricing issues, image, etc. This dou-ble approach makes it possible to identify the key factors that contribute to increasing or decreasing overall satisfaction.

o We measure the cumulative satisfaction relating to the last 12 months9 – in as far as this makes sense (sometimes there is only one service encounter over that period).

o Survey respondents are asked about their experience with their main provider of ser-vices. This is then aggregated to present an assessment for the sector as a whole. There is, however, no calculation of the satisfaction about the services of a particular supplier.

2.2.3 Manifest and latent variables Measuring consumer satisfaction is, in principle, relatively straightforward: a representative sample of consumers is asked to what extent they are satisfied or dissatisfied about a par-ticular service. In general, such satisfaction is measured on ordinal scales (e.g. not at all sat-isfied – not fully satisfied – more or less satisfied – quite satisfied – very satisfied) or simple numerical scales (e.g. from 0 to 4 or from 1 to 10).

Often it is also useful to know why consumers are (dis)satisfied, i.e. to understand which factors or criteria contribute positively or negatively to the measured level of satisfaction. Un-derstanding what makes a consumer happy or unhappy allows suppliers and policy-makers to implement appropriate action.

Those who are interested in measuring consumer satisfaction are usually also interested in its implications. For instance, will a consumer continue to use the same supplier or will he/she switch to an alternative? Currently, the prediction of consumer behaviour – in particu-lar their loyalty towards the supplier – is what most suppliers are interested in, rather than the satisfaction itself.

Identifying and measuring such factors is, however, not straightforward. Indeed, there are many factors that affect and/or are related to consumer satisfaction. For a typical service, as many as thirty or so of these factors may have to be considered. Moreover, these factors differ between products, services and sectors; and individuals may rank the relative impor-tance of these factors in different ways.

9 Respondents that have not used the service over the last 12 months are excluded from the survey. Development of consumer satisfaction indicators & Pilot survey 12

Whilst these problems can be overcome when focusing on a particular environment (e.g. a specific product or service for a specific target group), some trade-offs have to be made when designing a tool that allows valid comparisons across user groups, countries and sec-tors. This is the case in this assignment and it was also the challenge faced by those in charge of designing the national customer satisfaction indicators that already exist.

One way of approaching this issue is by creating a ‘hierarchy’ of factors:

o First of all, there are the measurable ‘factors’ or ‘criteria’ related to consumer satisfac-tion (e.g. prompt service, adequate opening hours, competitive pricing, etc). These are called the manifest variables. Each manifest variable corresponds to a question in the survey to the consumers. Many of these manifest variables are similar across sec-tors, but some are specific. Even if they are identical, their importance may vary be-tween sectors.

o Secondly, a number of related manifest variables have been aggregated into ‘latent variables’10. A latent variable reflects a relatively complex dimension of consumer satisfaction that cannot be measured directly. Examples of such latent variables are quality, price and image.

This approach is consistent with methodologies used elsewhere. It has several advantages. Firstly, at the level of the latent variables, a model can be constructed that is identical across sectors and countries. Thus, it becomes meaningful to make comparisons between countries and sectors.

Secondly, when using latent variables, the model can remain stable over a considerable pe-riod of time, hence increasing comparability of results. Specific survey questions and mani-fest variables may change over time and across sectors, but the model at the level of latent variables remains unaffected.

Lastly, provided the latent variables have been chosen appropriately and have been labelled in a meaningful way, the use of latent variables makes it possible to explain the key ele-ments affecting consumer satisfaction, without undue reference to a long list of very specific characteristics.

Since some latent variables – in particular quality – consist of quite a considerable number of manifest variables, during the model design it appeared useful to create an intermediate level between manifest and latent variables, called ‘drivers’. Thus, a driver groups a few strongly related manifest variables or survey questions. The ‘drivers’ category was therefore added to facilitate the reading of the list of manifest variables and to categorise them into sub-groups. The model construct and its related methodology are only based on the mani-fest and latent variables.

Please note that, given the absence of consensus on the definitions and terms, the terminol-ogy adopted here – manifest variables/drivers/latent variables – is not necessarily the one used by all the authors. We have opted for a terminology and definitions that are most rele-vant for our approach and which are acceptable to all experts. The following table provides definitions and alternative terms also found in the literature.

10 The aggregation of manifest variables into latent variables was based on assumptions about logical relationships between them, derived from desk research and focus groups. The results of the Pilot survey have to a very large extent confirmed the logic of the proposed groupings. Development of consumer satisfaction indicators & Pilot survey 13

Terminology adopted in this re-port Definitions adopted in this report Terms also found in the litera-ture with an identical or similar meaning Manifest variables A manifest variables corresponds to a question in the survey to the con-sumers Measurement variables Drivers Attributes Characteristics Criteria Drivers A driver groups a few strongly related manifest variables or survey ques-tions

(This intermediate level is not used elsewhere – national models or lit-erature) Latent variables A latent variable reflects a relatively complex dimension of consumer satisfaction that cannot be measured directly Variables Latent factors

From a practical point of view, it is also useful to qualify the latent variables (and the under-lying manifest variables) according to their explanatory power. They can be categorised into three groups:

o ‘Explanatory variables’: these are the factors that directly influence consumer satisfac-tion. In the literature, theses explanatory variables are also referred to as ‘antece-dents’, ‘causes of satisfaction’ or ‘input factors’.

o ‘Resultant variables’: these are the measures of overall consumer satisfaction. In gen-eral, there are only a very few such manifest variables.

o ‘Consequent variables’: these represent the consequences of the consumer (dis)satis-faction, and express concretely the outcomes from the consumer satisfaction/dissat-isfaction. Other terms that can be found are ‘consequences’ or ‘outcomes of satisfac-tion’.

In summary:

Terminology adopted in this report Definitions adopted in this report Terms also found in the lit-erature with an identical or similar meaning Explanatory variables Factors that directly influ-ence the consumer satis-faction Antecedents Causes of satisfaction Input factors Resultant variables Measures of overall con-sumer satisfaction Overall results Outputs Output factors Consequent variables Consequences of the con-sumer (dis)satisfaction Consequences Outcomes of satisfaction

Development of consumer satisfaction indicators & Pilot survey 14

2.2.4 Indicators and indexes Indexes (or ‘indices’) are constructs that try to capture a complex situation into a single fig-ure. Well-known examples are the consumer price index, the inflation rate, the consumer confidence index, the stock market index, etc. Such an index is calculated by aggregating weighted averages of results or prices of a large number of different goods, services, etc. In most cases, an index covers most of the products and services of a given economy, and is presented as a single number or ratio (a value on a scale of measurement).

Most existing national customer satisfaction measurement systems11 produce such an index for consumer satisfaction. The index is often available at several levels, which are then ag-gregated to produce an overall, nationwide and cross-sector index. For instance, the Ameri-can Customer Satisfaction Index (ACSI) (see section 2.3.2.) includes four levels: o A national customer satisfaction index o Indices for seven broad areas of the economy o Indices for 39 economic sectors o Indices for over 200 major companies and government agencies within these sectors.

An ‘indicator’ is also a statistical figure capturing a given situation. The term ‘indicator’ refers to a quantitative measure that provides a simple and reliable basis for assessing a feature (e.g. consumer satisfaction or an aspect of it) and measuring the related performance, proc-esses and outcomes over time. A useful indicator allows comparisons over time, across sec-tors and countries. An indicator could be derived from a single quantitative measure, or rather be based on the measures of several manifest variables. An indicator may even be related to a single customer.

Again, we found no agreed and universally accepted definition of both terms – and their dif-ference – in the literature.

Based on the needs of the Commission, we consider that the term ‘indicators’ is most appro-priate for what is required. Firstly, this assignment covers only a limited number of sectors – so it is not even possible to create an overall index at European or national level. Secondly, the need is not so much to have a single figure, but rather to produce a range of measures that capture different aspects of consumer satisfaction. Lastly, using the term indicators is helpful for distinguishing this methodology from the typical ‘index’-approaches that are more industry-oriented and economic-driven.

11 See next section. Development of consumer satisfaction indicators & Pilot survey 15

2.3

The development of national customer satisfaction indexes12

2.3.1 General concept and origin The last fifteen years have witnessed the emergence of a number of national customer satis-faction ‘indexes’ or ‘barometers’. A customer satisfaction index (CSI) is meant to be a na-tionwide gauge of how adequately companies (and other organisations) satisfy consumers.

There are many similarities between the existing CSIs and their underlying methodologies. The basic structure of the CSI models has been developed over a number of years. The models and methodological developments are based on scientific advances in the under-standing of consumer behaviour, customer satisfaction measurement and product/service quality.

A key feature underlying all approaches is that they are based on a ‘model’. This model con-sists of a number of latent variables (such as ‘quality’ or ‘image’) and the cause and effect relationships between them. Each of these latent variables includes several manifest vari-ables that act as concrete proxies for the latent variable. Consumer satisfaction is the latent variable that is at the centre of the model; it is encased within a system of variables relating to causes and effects. A good model should be capable of predicting a pattern of relation-ships and effects

In 1989 the Swedish Customer Satisfaction Barometer (SCSB) was the first truly national customer satisfaction index for domestically purchased and consumed products and services. Originally, it contained two primary explanatory variables of satisfaction, namely the per-ceptions of a customer’s recent performance experience with a product or service, and the customer expectations regarding that performance.

Since then, other models and methodologies have been developed, taking their inspiration from the SCSB and its successors. In general, the models have become more complex and the number of manifest variables has increased over time.

2.3.2 The American Customer Satisfaction Index and related approaches The American Customer Satisfaction Index (ACSI) was introduced in 1994. Like its Swedish predecessor, it is a uniform and independent measure of household consumption experience. ACSI produces a customer satisfaction index based on measures from seven broad economic areas, 39 industrial sectors and more than 200 companies and public agencies.

The ACSI model is a set of causal equations that link the latent variables of customer expec-tations, perceived quality and perceived value to customer satisfaction. In turn, satisfaction is linked to consequences as defined by customer complaints and customer loyalty. When simplified, this model may be presented as follows.

12 This section only presents some of the key findings about the existing national indexes. More infor-mation about these approaches can be found in the first interim report and its annexes. Development of consumer satisfaction indicators & Pilot survey 16

The ACSI approach includes a number of features that can also be found in other similar na-tional indexes: o It is based on an econometric model with measures of an index of satisfaction and measures of related indices for latent variables that are general enough to be compa-rable across companies, industries and sectors. o Customer satisfaction itself is measured as a latent variable using several manifest variables (questions). o Customer satisfaction is embedded in a system of cause-and-effect relationships. This serves to validate the index. o Finally, a primary objective is to estimate the effect of ACSI on customer loyalty, a construct of universal importance for future business performance.

This model is an evolution of the original Swedish Barometer. It started from the same framework, but became more complex in several regards. It rapidly evolved to include two distinct types of perceived quality – namely the product quality and the service quality, this distinction being used solely for the manufactured products.

The ACSI model became the basis of the frameworks used in customer satisfaction initiatives undertaken in New Zealand, Taiwan, Austria, Norway, Hong Kong and elsewhere.

The Norwegian development is worthwhile mentioning. The Norwegian Customer Satisfac-tion Barometer was created in 1996. It is almost identical to ACSI, except for the introduc-tion of a new manifest variable ‘corporate image’ and its relationships with customer satisfac-tion and customer loyalty. The argument for this addition was that people relate corporate image to organisation-related associations. CustomersatisfactionCustomer

expectations

Loyalty Customercomplaints Perceivedvalue Perceived

quality

ConsumersatisfactionConsumer

expectations

Loyalty Consumercomplaints Perceivedvalue Perceived

quality

Development of consumer satisfaction indicators & Pilot survey 17

2.3.3 The European Consumer Satisfaction Index The ECSI initiative (European Consumer Satisfaction Index) is another variation of the ACSI model. The so-called ‘European Customer Satisfaction Programme’ was launched by a num-ber of private and non-profit European organisations, complemented by some national plat-forms. Three objectives were pursued with this initiative: o Provide companies, public services, consumers, investors, regulators and policy-makers with an annual customer satisfaction index and analysis of companies and public ser-vices in Europe o Provide companies and public services surveyed with the means to analyse the percep-tions of their customers (causes and effects) and to compare them with the percep-tions of customers of other companies and public services at different levels (sector, country, Europe, USA, East Asia) o Introduce the European Consumer Satisfaction Index as a recognised economic macro indicator measuring the performance of the national and European economy.

The experience started in 1997, with a European feasibility study for coordinated national CSIs. The first result was a model for CSI measurement, based on the Swedish and Ameri-can model. The pilot phase study took place in 1999 and covered 11 countries (100,000 in-terviews conducted). The survey conducted in 2003 covered nine countries, five common industries and required 90,000 interviews. In 2003, nine countries were involved in the en-deavour (Denmark, Finland, Greece, Iceland, Ireland, Norway, Portugal, North-West Russia and Sweden) and five industries were covered (retail banking, fixed-line telecoms, mobile phones, insurance and supermarkets).

ECSI has been modelled in a way that is similar to ACSI. Yet there are some differences:

o The split between ‘product quality’ (hardware) and ‘service quality’ (software) has been generalised. Quality is related to the consumer’s quality experience with a service, and refers both to: • its technical components (product quality or hardware): security/safety, informa-tion, reliability, confidentiality, etc. • the quality of the associated functional services (service quality or software), such as opening hours, friendliness of personnel, consumer dialogue, complaint han-dling, etc.

o The latent variable ‘customer loyalty’ has been specified in a different way. It includes likelihood of retention, the likelihood of recommending the company or brand, and the likelihood of an increase in the amount of customers purchasing the product.

o The variable ‘customer complaints’ was not taken into account.

o The variable ‘corporate image’, included in the Norwegian index, has become a latent variable, with effects on customer expectations, satisfaction and loyalty. Development of consumer satisfaction indicators & Pilot survey 18

Graphically, this can be displayed as follows:

It should be noted that, unlike the American and other national approaches, there is quite some variation in the implementation of the measurement system – in terms of the coverage of the model, the characteristics of the population surveyed, the size of the sample, the spe-cific questions included, and so on. This variation is noticeable both between countries and even within countries for different years.

2.4

Development of a provisional model 2.4.1 Conceptual model Based on the requirements of the Commission, the results of the extensive desk research and the organisation of the focus groups, we have gradually developed a fairly comprehen-sive model for the measurement of consumer satisfaction. In this model, satisfaction is treated as an overall evaluation of the consumption experiences (in general with several pro-viders). It is comprehensive because all factors that may be related to consumer satisfaction within the sectors concerned have been incorporated.

The model includes nine latent variables, each of which consists of several manifest vari-ables. Based on the research undertaken, assumptions were made about the main cause-and-effect relationships between the latent variables. CustomersatisfactionImage

Customer

expectations

LoyaltyPerceived

quality

Perceivedvalue Soft features

Hard features

ConsumersatisfactionImage

Consumer

expectations

LoyaltyPerceived

quality

Perceivedvalue Soft features

Hard features

Development of consumer satisfaction indicators & Pilot survey 19

The following graph illustrates the model proposed:

The latent variables on the left-hand side of the model – Market / Image / Expectations / Quality / Pricing (and the underlying manifest variables) – are the explanatory variables that contribute to explaining satisfaction. The latent variables on the right-hand side – Complaint behaviour / Alternatives / Trust – are the consequent variables.

The review of literature suggests that there could potentially be links between all the latent variables listed. However, we have only drawn the paths (cause-effect relationships) which were assumed to be the most probable and relevant ones given the scope of this project. As will be demonstrated later in this report, the results of the Pilot survey have proven that most of these assumptions were correct (see chapter 4).

Two important remarks need to be made at this stage:

o From a strictly scientific point of view, it is not possible to create a unique model for consumer satisfaction measurement that will perfectly fit all sectors –not even at the level of latent variables.

o The model that was created served for the design of the questionnaire for the Pilot survey. It was only a provisional model, since an analysis of the survey responses was needed to confirm the appropriateness of the grouping of manifest variables into latent variables, and the existence of the proposed cause-and- effect relationships. Again this has to a large extent been the case (see chapter 4). Market & personal factorsImage Trust & commitment Alternatives & substitutes Complaints Perceivedquality Perceivedpricing Expectations Overall satisfaction Market & personal factorsImage Trust & commitment Alternatives & substitutes Complaints Perceivedquality Perceivedpricing Expectations Overall satisfaction Development of consumer satisfaction indicators & Pilot survey 20

2.4.2 Overview of the variables for the provisional model The following table lists the latent variables, the drivers13 and the manifest variables of the provisional model. Each manifest variable corresponded with a question in the Pilot survey.

In this provisional model, the number of manifest variables exceeded 50. This high number was achieved because absolute care was taken not to overlook any specific factor that might contribute to or be related to consumer satisfaction.

Type Latent variables Drivers Manifest variables14

Reliability Correct delivery of the service in due time High quality of the technical service Problems occurring with the service Comfort of transport (for transport sectors)

Professional, helpful and friendly staff Maintenance and modernisation of infrastructure Nearby agencies or point of sale Friendliness with the customers Flexible attitude and easy to deal with Customer-minded. Putting the customer first Prompt and adequate reaction to problems and complaints Helpfulness in resolving problems Satisfactory solving of problems Affordability Affordability Special attractive offers Pricing Reasonable/competitive prices Profitability (banking/insurance)

13 Drivers mentioned in italics are drivers with only one manifest variable or survey question. In other words, the driver is identical to the manifest variable. 14 Not all manifest variables/survey questions are applicable to all sectors. The wording and content of the survey questions may also differ somewhat between sectors. Development of consumer satisfaction indicators & Pilot survey 21

Image Overall corporate image Overall reputation. Positive image Belief that one is being fairly treated Unique image Familiar with the supplier and what they do Popularity Technologically advanced and ability to innovate Respecting the environment Confidence Confidence in continuity of service provision Competitive structure Sufficient competition Ease of changing from one supplier to another Differences between suppliers in the market Cross-border opportunities Possibilities for cross-border purchase Accessibility Availability of services everywhere for everyone Knowledge of the services of the supplier Explanatory variables

Market and personal factors Personal factors Care about the services to deal with Tendency to stick with the same suppliers Concern about always getting the best deal Preference to work with a national supplier Belief in the advantages of a liberalised market

Negative appraisals Speaking negatively about the supplier Complaint behaviour Complaints Forwarding complaints to the provider Forwarding complaints to third parties

Alternatives and substi-tutes Move to an alternative or substitute

Abandoning service Turning to another supplier Positive appraisal Recommending to friends or colleagues Positive out-come/stability Continuing to use this type of service Continuing to use the same supplier Consequent variables Trust and commitment Tolerance Sensitivity to price increase or decrease