__PS_-__3O7
POLICY RESE ARCH WORKING PAPER 2307
Surveying Surveys and How to nakefirm-level
surveys more consistent,
Questioning Questions yielding data more relevant to
policy analysis.
Learning from World Bank
Experience
Francesca Recanatini
Scott J. Wallsten
Lixin Colin Xu
The World Bank
Development Research Group
Regulation and Cornpetition Policy U
March 2000
POLICY RESEARCH WORKING PAPER 2307
Summary findings
The World Bank has increasingly focused on firm-level * Consider the costs and benefits of numeric scales
surveys to build the data foundation needed for accurate compared with adjectival scales. Scales in which each
policy analysis in developing and transition economies. point is labeled may be more precise than numeric scales
Recanatini, Wallsten, and Xu take stock of some recent in which only the endpoints are labeled. But responses
Bank surveys and discuss how to improve their results. are very sensitive to the exact adjective chosen and it
Lessons on data issues and hypothesis testing: may be impossible to translate adjectives precisely across
* Use panel data, if possible. languages, making it impossible to compare responses
* Have enough information about productivity to across countries.
estimate a production function. * Recognize that the share of respondents expressing
* Avoid the paradigm of "list the severity of the opinions will be biased upward if the survey does not
obstacle/problem on a scale of 1 to S." Instead, ask for include a middle ("indifferent" or "don't know")
data on specific dimensions of the problem that will shed category and downward if it does include the middle
light on alternative hypotheses and policy category.
recommendations. * When asking degree-of-concern and how-great-an-
* Pick particular disaggregated industries and sample obstacle questions, consider first asking a filter question
those industries in each survey. (such as "Do you believe this regulation is an obstacle or
- Identify the most important policy interventions of not?"). If the answer is yes, then ask how severe the
interest and consider how you will empirically identify obstacle is.
specific changes by picking instruments useful for doing * Be aware of the effects of context. The act of asking
so. questions can affect the answers given on subsequent,
Lessons on questionnaire design: related questions.
* Incorporate only one idea or dimension in each - Think carefully about how to ask sensitive
question. Do not ask, in one question, about the "quality, questions. Consider using a self-administered module for
integrity, and efficiency" of services, for example. sensitive questions. Alternatively, a randomized response
mechanism may be a useful, truth-revealing mechanism.
This paper - a product of Regulation and Competition Policy, Development Research Group - is part of a larger effort
in the group to develop consistent cross-country firm level surveys. Copies of the paper are available free from rhe World
Bank, 1818 H Street NW, Washington, DC 20433. Please contact Paulina Sintim-Aboagye, room MC3-422, telephone 202-
473-7644, fax 202-522-1155, email address psintimaboagye@worldbank.org. Policy Research Working Papers are also
posted on the Web at www.worldbank.org/research/workingpapers. The authors may be contacted at
frecanatini@worldbank.org, wallsten@leland.stanford.edu, or Ixul@worldbank.org. March 2000. (44 pages)
The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about
development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The
papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this
paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the
countries they represent.
Produced by the Policy Research Dissemination Center
SURVEYING SURVEYS AND QUESTIONING QUESTIONS:
LEARNING FROM THE WORLD BANK EXPERIENCE'
Francesca Recanatini, Scott J. Wallsten, and Lixin Colin Xu2
We would like to thank Philip Keefer, Howard Pack, Mary Shirley and seminar participants at the World Bank for
their coniments. Any remaining errors are our own.
2 The World Bank, 1818 H Street, NW Washington DC, 20433; and the Stanford Institute for Economic Policy
Research, 579 Serra Mall at Galvez Street, Stanford, CA, 94305; email: frecanatini@worldbank.org;
wallsten@,leland.stanford.edu; Lxul@worldbank.org.
1
I
I. INTRODUCTION .....................................................2
II. GENERAL DATA ISSUES AND THE BANK'S EXPERIENCE .....................................................3
III. IDEAL DATA VS. EXISTING DATA .....................................................6
A. The Firm .....................................................7
B. Human Capital .....................................................9
C. Technology .................................................... 10
D. Market Structure .....................................................11
E. Transaction Environment .................................................... 14
F. The Role of the State .................................................... 16
G. Beyond the firm: "The micro-foundations of macroeconomics" ........................ 19
IV. QUESTIONING QUESTIONS .................................................... 24
General Issues .................................................... 24
Response scales .................................................... 26
The "Don't Know" Problem .................................................... 28
Filters and Branching .................................................... 29
Order Effects .................................................... 31
Specific and general questions .................................................... 31
Recent and less-recent behavior .................................................... 32
Within-question order effects .................................................... 33
Sensitive Questions and Truth-Revealing Mechanisms ........................................ 33
V. CONCLUSIONS .................................................... 35
I. INTRODUCTION
Industrialized nations make rigorous data-collection the foundation of their policy
analysis. The paucity of data on firms in developing countries, however, makes policy analysis
in those countries more difficult, The World Bank has increasingly focused on firmn-level surveys
to help build a data foundation in developing countries and transition economies. The most
extensive firn surveys implemented by the World Bank include the Regional Program on
Enterprise Development (RPED) survey in 8 African countries, the Industrial Competitiveness
Study (ICS) in East Asia, a series of surveys in transition countries, and many surveys on small
and medium enterprise (SME) issues in South Asia and South America (see Table 1 for a partial
list of World Bank firm-level surveys). This paper takes stock of recent World Bank firm
surveys, discussing what we have learned from them and how we could more consistently and
efficiently gather data for policy analysis.
2
World Bank surveys are generally done in a decentralized manner, making it difficult to
synthesize the lessons that have emerged from these ambitious and often expensive efforts. Still,
as the Bank continues its survey efforts, it is worth investigating what we have learned from its
experiences to date. We first present a brief overview of World Bank surveys organized by
research topic, including technology, incentives, market structure, transaction environment, the
role of the state, and the importance of micro data in understanding macro phenomena. We note
that to analyze the effects of policy intervention it is often important to collect panel data and to
define industries at sufficiently disaggregated levels. Building on our "survey of surveys," we
review the literature on survey design, highlighting the point that questions themselves can bias
responses and provide some guidelines for recognizing the direction of this bias and minimizing
it when possible.
While we address only firm surveys conducted by the World Bank, we believe that this
discussion is also relevant to policy analysts outside the Bank. The World Bank has probably
conducted more surveys in developing countries than any other single institution. As a result of
the large number of surveys and survey topics, the Bank now possesses enormous institutional
knowledge not just from the data collected, but from the survey experiences themselves. The
Bank should share this knowledge with its member countries. This paper represents such an
attempt.3
II. GENERAL DATA ISSUES AND THE BANK'S EXPERIENCE
Rigorous policy analysis requires a great deal of data. Ideally, the data are longitudinal
(i.e., firm level information collected at discrete points over time) and fairly dissagregated (i.e.,
industries are defined as narrowly as possible). Only with panel data can one investigate the
effects of policy interventions and the process of economic growth. Consider, for example, the
' See Blank and Grosh (1999) for a complementary paper on household surveys. They suggest that political will,
appropriate resources and careful planning are key requirements for developing analytical capacity. With these
prerequisites in place, policy-makers must concentrate on four areas to strengthen the analytical capacity of their
country: appropriate training of policy analysts; use of external technical assistance to work closely with local
counterparts; establishment of fully staffed policy analysis units; and direct participation of staff members in the
development and implementation of the research agenda. Our paper builds on Blank and Grosh by assuming the
will and resources are in place, and discusses the best methods for gathering the data.
3
case of new regulations and firn entry and exit. Regulations may affect firm entry and exit in
complicated ways, They may impose entry barriers to new finns and impose exit barriers to
inefficient (or state-owned) firms. Av "snapshot" of firms at a given point in time (a cross-
section) provides limited information on this matter. Only by surveying firms over time and
across countries can we begin to understand how new regulations or other policy interventions
affect firm entry and exit.
Entry and exit also lead to attrition bias in the sample-another reason to obtain panel
data. Properly accounting for both entry and exit of firms is therefore important to understand
industry evolution. Attrition bias may arise when firms disappear (exit) because of mergers,
acquisition, consolidation, or bankruptcy, or when a firm is included in the sample only if it
survives up to the point of interview. Without accounting for attrition, the sole reliance on
surviving firms will inevitably bias (usually upward) performance measurements. The
additional survey rounds necessary for constructing panel data can help account for this bias by
deterrnining which firms have disappeared and why. In addition, panel data can help better
understand firn entry.4 New entrants may differ significantly from surviving firnms. Moreover, a
high death rate of firms is not necessarily worrisome if it is accompanied by a high birth rate. To
deal with such issues, firm surveys should resample some firms annually and strive to add new
entrants.
Surveys should also define industries as precisely as possible and offer detailed
informnation of the industry analyzed. When this type of micro data is collected in panel data
sets, it becomes possible to compare the technological progress of particular industries in
different countries. Such data allow us to better address important unanswered questions, such as
why inter-firm dispersion in productivity is greater in industrialized economies than in
developing countries (as suggested in Tybout, 1998). A variety of reasons (such as a smaller
extent of market and excessive regulation) have been suggested. But there is no consensus
among the existing empirical work largely because every data set aggregates industries
differently, making comparisons nearly impossible (Tybout, 1998).
4 Finding new entrants, however, clearly makes their inclusion easier said than done.
4
Table 1 offers a partial list of some recent surveys done within the Bank. The table
reveals four main types of surveys: (1) Comprehensive surveys (RPED, ICS); (2) Surveys on
transition (Russia: Economic and Civil Society; Corporate Governance in Poland, Hungary, and
Czech Republic; Large Scale Privatization in Mongolia; The Emergence of Private Sector
Manufacturing in Poland, in Hungary; Industrial Enterprises and Adjustment in Russia; Private
Service Firms in Russia-St. Petersburg.); (3) Surveys of the market environment, including
regulation and govemment-industry relations (World Business Environment Survey); and (4)
Topic-specific surveys, such as on training (Enterprise Training in Developing Countries, Tan
and Batra), or on SME problems (CECPS, "Small-and-Medium Industry Impact Evaluation,"
1995; and "Enterprise training in developing countries," Tan and Batra).
How well do World Bank surveys meet the data needs outlined above? We begin to
address that question below. One lesson from the Bank experience is that surveys are extremely
difficult. Thus, while we make the comments below with the ideal data set in mind, we
recognize that in practice surveys may not be able to meet this ideal. Longitudinal surveys are
costly and take too long for many Bank operations. Some data, meanwhile, simply do not exist.
Nonetheless, it is instructive to consider what would have made these surveys more useful, and
use that knowledge when designing future surveys.
1. Data consistency: Although World Bank surveys are usually designed to address a
particular issue, each could be more broadly useful by consistently including a small set of
standard questions on firm performance, such as total output and profitability. The survey
should also collect price data on the firm's main products, inputs, and investment goods. In
addition, the survey becomes more valuable when it contains enough information to estimate
production functions, including value added, number of employees, capital stock, and material
inputs. This information would help to isolate the contribution to productivity changes of
various production, technological, and institutional factors.5 Only a few surveys, such as RPED
and ICS, include consistent productivity measures.
5Consider the Bank's general interest in Small and Medium-sized Enterprises (SMEs). To understand if SMEs are
as efficient as other firms, we must hold constant capital and labor and then compare productivity to that of different
types of firms.
5
Similarly, to investigate financial constraints, we need to know not only whether some
types of firms (by ownership, size, or the owner's gender) had more difficulty obtaining external
finance, but also the consequences of the lack of funds. Estimating production functions would
allow us to learn whether, holding other characteristics constant, financially-constrained firms are
indeed less productive. If such firns are not less productive, then there is less need to worry
about financial constraints.
2. Importance of panel data: Most Bank surveys are cross-sectional, leaving the need
for panel data unfulfilled (again, with some exceptions, notably, RPED and ICS). As a
consequence, we have much less confidence in inferring whether productivity differences reflect
unmeasured firm heterogeneity or the variable of interest.
3. Need to address endogeneity: Surveys can be most effective when they deal
explicitly with the endogeneity of policy interventions. Many researchers find that ex post it is
difficult to demonstrate any causal effects because they lack of good instrumental variables-
those variables uncorrelated with the error term but reasonably good predictors of the
endogenous variables. It will likely be too late to derive instrumental variables at the data
analysis stage. Instead, at the survey design stage researchers should consider the endogenous
policy variables of interest and think about potential instruments to identify those variables.
III. IDEAL DATA VS. EXISTING DATA
In each of the following subsections we give a brief overview of some questions that have
yet to be empirically investigated in the context of developing countries, the data that would
allow hypothesis testing, and an evaluation of whether existing World Bank survey have
addressed these issues. In particular, we discuss research topics such as the firm itself, market
structure, the business environment and its impact on firmn's performance, the role of the state,
and analysis of the macro-economy through microeconomic data.
6
A. The Firm
Corporate governance6
The fast growth of the corporate sector throughout the world makes corporate governance
increasingly important. However, we have an especially poor understanding of corporate finance
in developing countries, largely because of the lack of firm-level data. For instance, what are the
ownership patterns in the developing countries? How are firms financed? Do firms face
financial constraints? How well are creditors and shareholders protected in these countries? The
answers to these illustrative questions can have important policy implications. The recent East
Asian Crisis, for example, has been attributed to the poor protection of minority shareholders
rights. The policy implications of these questions call for the collection of such data.
Bank surveys ask many good questions about corporate governance, although the
coverage varies greatly. Consider the contrast between what theory suggests we should measure
and what Bank surveys actually collect:
1. Theory suggests that competition in the managerial labor markets and reputation concerns
force managers to perform well for future career prospects, making it important to measure
aspects of managerial labor market, such as turnover, performnance criteria, rewards and
punishment. Such questions are hard to find in Bank surveys. Suggested questions: What
determines managerial pay (current profits, stock options, relative performance, regulators'
evaluations, etc)? If profits increase by one percent, what percentage increase does the
managerial wage? Has a manager ever been removed because of poor performance? Can
shareholders replace the CEO when performance is bad? What was the worst performance in
the past x years? What consequences did the then-CEO suffer (income losses, demotion, or
others)?
2. A firn's board of directors can be an important disciplinary force. Theory suggests that
because of the free-riding problem associated with monitoring by shareholders, concentrated
ownership may emerge to internalize the benefits and costs of monitoring. Thus it is
important to uncover the ownership stake of the largest shareholders. Most Bank surveys ask
about ownership type, but ask few questions beyond that. Suggested questions: What is the
6 This subsection draws heavily on Shleifer and Vishny (1996).
7
composition of the board of directors? What percentage of board members represents
management? Major owners (more than 5% shares)? Small shareholders?
3. Theory suggests that debt and equity contracts give shareholders and especially creditors
control rights to intervene when a contract is violated. Bank surveys, however, in general do
not provide information on the financial structure and the rights of stakeholders. Suggested
questions: (1) What is the share of outside equity? Share of inside equity? Share of debt
financing? Debt-equity ratio? What type of institutions or individuals are the main
shareholders (bank, pension funds, a holding company, a foreign investor, or a domestic
investor)? The main creditors? (2) What are the control rights of the equity-holder, of the
debt-holder? Would a firm become bankrupt if it defaults interest payment for debts? Does
the firm post collateral when borrowing from banks? What is the share of tangible assets?
What are the shares of long- and short-term debts in total debts respectively''
Internal Incentives
Internal incentives help align managerial and employee objectives (see Holmstrom and Tirole,
1989, for a survey). There are many ways to motivate workers:
* Promotion, In theory, a large wage differential between workers at different levels improves
performance by spurring employees to work hard in order to advance to the next layer, but
this benefit has to be balanced against the loss of control and of information. The tournament
literature suggests measuring the number of layers within the firm and wage differential
between production workers and managers (Lazear and Rosen, 1981; Nalebuff and Stiglitz,
1983; Rosen, 1986).
* The threat of firing. Presumably, internal incentives are stronger when involuntary turnover
is a real and credible threat. This may be achieved by allowing a certain percentage of
contract workers. A high share of temporary workers, however, may have unintended
consequences: The threat cf firing as a disciplinary option spurs worker incentives, but
temporary workers face reduced incentives to accumulate firm-specific humlan capital. Since
the net effect is unclear, surveys will provide the best source of data to answer this question
and provide policy advice.
8
* Linking a worker's pay to firm performance. Supposedly a close link should increase
employee incentives.
Understand internal incentives may help policy makers provide better advice on private
sector reforms. For instance, suppose surveys reveal little correlation between pay and firm
performance in a particular country. Such a finding may imply that the labor force is not
properly motivated, perhaps due to poor performance measurement or the many layers of
hierarchy. Alternatively, if all employees perceive their jobs as permanent, increasing the share
of contract workers may be a valid option to motivate workers.
Many World Bank surveys do not measure internal incentives at all. Some have a
minimum number of measures, such as the percentage of work being contracted out, employment
benefits, bonuses, payment method (e.g., piece rate or hourly rate), total wages, non-wage costs,
and labor turnover rates. The RPED covers internal incentives most extensively. It asks many
useful questions, including the payment scheme (piece rate, time rate, or based on firms' sales),
the number of people fired, the presence of a union, overtime pay, and detailed wage data by
worker category. An especially innovative feature of the RPED is its survey of a subsample of
workers in the surveyed firms, which asked workers about issues such as bonuses, wages, and
housing allowances.
Despite the strong advances made by the RPED, Bank surveys leave many questions
unexplored. For example, we still know little about pay differences between different levels of
workers, and between managers and production workers. Likewise, in few countries and
industries do we have any sense of the involuntary turnover rate. Without such data we cannot
understand the effects and determinants of firm internal incentives, and whether performance
improves with better internal incentives in different countries.
B. Human Capital
Human capital theory emphasizes the importance of schooling, labor market experience,
seniority, and training in explaining individual productivity (Willis 1986). In the context of
firms we should then expect these human capital variables to positively impact productivity. The
lack of data has thus far prevented us from learning how education and training affect
9
productivity and earning in countries with different macro environment and protection of
property rights.
Another important economic question is how to finance general and specific human
capital investments in firms (Willis 1986, Becker 1962, Oi 1962). General human capital raises a
worker's productivity regardless of whether she stays at one firm or moves to another. Specific
human capital, on the other hand, is lost if she moves to another firm. Efficiency dictates that the
employer should sho-ulder much of the costs for specific training, since the worker could not
benefit from the training if she moves to another firm. Likewise, there may be little reason for
the employer to pay for general training, since the worker could benefit from it even if she moves
to another firm. In this case, the worker may pay for the training since she benefits from it
wherever she works. It also makes sense for the government to subsidize general training since
the economy benefits from more productive workers wherever in the country they work. We
have little empirical evidence, however, about the relationship of productivity to labor market
experience and how training is financed in different countries. The finding would have policy
implications for how governments and firms should approach worker training.
Many Bank surveys cover human capital issues quite well (for instance, RPED and ICS).
Most surveys contain the primary measures, although they tend not to explore the distribution of
workers in term of labor market experience and seniority structure, and quit rates. Such
questions can be important. The turnover rate, for example, can provide an indication of how
vigorous a labor market is in one developing country compared to another, or cornpared to
industrialized nations. Quit rates may indicate the importance of match-specific human capital
and the vitality of a country's labor market (Tybout, 1998). Involuntary turnover rates could
indicate the extent to which workers are disciplined or how firms adjust employnient in response
to changes in economic conditions.
C. Technology
In the long run, technological progress is largely responsible for growth. It is therefore
important to measure it in developing countries. But what determines technological progress and
how can we measure it? These questions are exceedingly difficult to answer, even for large firms
in industrialized countries. A common input necessary for technological progress is research and
10
development (R&D) expenditures. However, few firms in developing countries categorize any
expenditure as R&D, meaning we must look for altemative measures. One determinant is market
structure (discussed in the following section), since it directly impacts technological progress.7
Other aspects of technology can be proxied by equipment vintage, the number of
technical personnel and their wages, and purchased foreign inputs and equipment. Sources and
vintage of equipment proxy for technology under the assumptions that newer equipment
represents a higher level of technology, and that imported equipment is more advanced. In
addition, technological know-how is embodied in technical personnel. A higher percentage of
the work force with technical knowledge-engineers and employees with college degrees, for
example-represents a more advanced state of technology. Finally, the development literature
sometimes considers the relationship between purchased foreign inputs and productivity to be an
indicator of the firm's level of technology (Tybout, 1998).
Few Bank surveys have attempted to measure technology. In addition, most of these data
sets are only cross-sectional, making it difficult to measure technological change. Some surveys
contain a minimum amount of information about technology, including questions on the
percentage of inputs imported from different foreign countries. The RPED and the ICS surveys
serve as nice guides for future firm surveys on technology. These surveys gather data on
licensing fees, the presence and amount of foreign technical assistance, the number of
expatriates, equipment vintage, level of new investment, imported equipment, the number of
scientists and engineers, R&D expenditures, types of R&D, new product or process adoption,
and whether the firm sells any technology.
D. Market Structure
Market structure is one of most important determinant of firm behavior and performance.
It thus is useful to compare the market structure across countries in the same industries to explore
whether competition leads to faster innovation and superior performance. In addition to the
7Scherer and Ross (1990, p. 660) note that "What is needed for rapid technological progress is a subtle blend of
competition and monopoly, with more emphasis on the former than the latter, and with the role of monopolistic
elements diminishing when rich technological opportunities exist."
11
extent of the market, economists have also focused on concentration, the scope and the extent of
the firm (Viscusi, Vernon, And Harrington, Jr., 1990).
Concentration. Two hypotheses about the implication of concentration lead to
dramatically different policy recommendations. The first hypothesis is collusion. Viscusi,
Vernon, and Harrington (1990) note that "the more concentrated an industry is, the less
competitive are firms and thus the higher is the price-cost margin." In this case, it is desirable to
break up highly concentrated industries. The second hypothesis is Demsetz's superior efficiency
hypothesis. That is, superior firms have both higher market shares and larger price-cost margins.
Thus, at the firm level, both profitability and price-cost ratios are positively correlated with
market shares, and the relationship may be present in a weaker form in the industry level.8 In this
case, the relationship is association, but not causation. The policy implication then is not to
break up concentrated industries; the government, after all, does not want to punish firms with
superior technology.
The empirical evidence from industrialized countries so far tended to support Demsetz's
hypothesis. The evidence is that a firm's profit is strongly correlated with its market share, and
the positive association is still observed at the industry level, but in a weaker form (Salinger,
1990; Viscusi, Vernon, And Harrington, Jr., 1990). But what is the evidence in developing
countries? To learn the answers and thus give useful policy advice, we need to collect
information about concentration ratio, profitability, price/cost ratio, entry barriers as
characterized by license fees to entry, sunk costs (the percentage of the value of equipment can
be recouped if the firm is to quit), and the existence of exclusive patents.9
The scope of the firm. The scope of the firm is characterized by its mix of activities
(Viscusi, et al., 1990). Several issues of interest to the Bank and to development more generally
may be related to fimn scope. For example, firms may be more likely to be vertically integrated
in countries with weak contract enforcement. Whether this is true is, again, an empirical
question requiring much data to answer. Few World Bank surveys have paid much attention to
8 Weaker because one has to average profitability and price-costs ratios with those of the competitors, for which the
profitability and price-cost ratios are lower
9 This measurement has some problems. It is hard to measure costs, especially because it is subject to accounting
methods and may be easily manipulated.
12
this matter. To address this question surveys should ask about the firm's lines of business
(defined as precisely as possible), whether the firm is a conglomerate (i.e., engaged in unrelated
activities), the extent of vertical integration (i.e., produces its main inputs or has its own
distribution channels), and whether the firm has any subsidiaries. With these measures,
researchers can link productivity with the scope of the firm to see how firm scope changes under
different institutional environments.
The extent of markets. Theory suggests that the extent of markets may fundamentally
constrain firn development. Some believe that a small market, for instance, may help explain
the so-called "missing middle" phenomenon in developing countries-the existence of some
large firms and many small firms, but very few middle-sized firms, which are common in
industrialized countries. Small markets have also been suggested as a barrier to technological
change. Existing Bank surveys tend to neglect this issue. Many surveys can become more useful
by asking certain questions about the extent of market: Where firms sell their products, the
distance to a major city, and the firm's access to all types of transportation.
Two Bank surveys have made impressive efforts to explore market structure. One, the
RPED, asks two especially useful questions: (1) What characterizes a firm's competitors? (None,
domestic firms, foreign competitor, or imports). (2) How does the firm set its prices? (Market-
price-taker, markup over costs, in line with imports, price discriminate in different markets, price
increases with quality, follow largest competitors, government-set, negotiate-with-the-buyer, or
set by a business association). Although the category of "domestic firms" in the first question is
rather broad, the second question is especially rich.
The second, Russia.- Economic and Civil Society, asks detailed information about market
structure:
* How many Russian enterprises the respondent considers to be direct competitors.
* Whether the respondent's firm's most important products compete with imported products
from other countries listed.
* Whether the firm is classified as a monopoly producer.
* The ownership structure of input suppliers.
* Whether distribution channels offer credit assistance.
13
* How the firm sets its prices (e.g., based on production costs, competitors' prices, the target
profit level, or willingness-to-pay by the consumers) and how it adjusts price for inflation
(e.g., an index based on inflation, own costs, or dollars and exchange rates).
E. Transaction Environment
According to transaction costs economics (Williamson 1989, 1993), incomplete
information and the potential for opportunistic behavior prevent contracts from specifying
outcomes under every uncertain contingency. Firms thus establish various types of relationships
-including using contracts-with suppliers, dealers and customers. Maintaining many types of
relationships entails distinct benefits and costs for firms. These myriad relationships require time
and productive resources, but allow degrees of specialization and different levels of investments.
Researchers therefore must study contractual arrangements that facilitate adaptation and dispute
settlement such as the legal system, arbitration, long-term contract, and vertical integration.
Transaction cost economics suggest that to better understand the impact of these costs on
firm's performance we should have at least the following information about the principal
characteristics of firm's exchanges (Williamson 1989):
(1) The frequency with which they occur,
(2) The degree and type of uncertainty to which they are subject, and
(3) The condition of asset specificity.
When asselt specificity is important, theory suggests that the transacting parties will
arrange transaction structures-for instance, vertical integration, merger, long-term contracts-to
economize transaction costs. Existing surveys pay only limited attention to such transaction
characteristics. Future Bank surveys may become more valuable when this typie of information
is collected.
An important determinant of transaction costs is the business and legal environment in
which the firn operates. This set of considerations is especially important for transition
economies and developing countries, where the legal and business environment is in flux and
may seriously impact the firm's decisions. To investigate these issues, surveys should collect
detailed information on the firns' exchanges. For example, what are the costs of establishing
new trading relationships when old ones are broken? Is the trade between the parties bilateral?
14
Do the parties rely on legal mechanisms to enforce their contracts or on alternative enforcement
devices? If a contract dispute arises, does the contract stipulate an arbitration mechanism? Does
the contract require "hostages" for deterring opportunistic behavior, such as collateral, and down
payments? What devices do the parties involved in the exchange employ to economize on
transaction costs? Are organizational forms and the scope of the firm affected by transaction
costs considerations-for instance, because of contract enforcement problem, are family firms
over-represented? Does the firm become more vertically-integrated to reduce contracting costs?
Bank surveys have collected a significant amount of data on these issues, especially in
recent years. Four major surveys-RPED, ICS, Bosnia, and Survey of Russian Enterprises-all
gather detailed information on the type of relationships firms had and the devices used to
regulate transactions. Together, these surveys yield a rich picture of the enviromnent in which
firms operate. The following extracts from existing bank surveys may serve as nice recipes for
future surveys on transaction environments:
* The nature of contract: Do contract specifications cover price, methods of adjusting price to
rate of inflation, quantity, quality, delivery date, method of payment, warranty, penalty for
late payment, penalties for breach of contract, and the method of resolution if a dispute
arises? (RPED Survey and the survey of Russian Enterprises).
* The contractual relations with suppliers, with subcontractors, and with clients: ownership,
payment method, frequency of purchase, relationship with the firm, whether the other party
comes from the same ethnic group, whether there is a trade discount, whether there is
subcontracting, the amount of credit purchase, interest, guarantee to supplier (none, physical
collateral, third-party checks, witnesses, signed invoices, group guarantees); whether using
written contracts more frequently with new trading partners than old trading partners;
whether using written contracts more frequently with new private enterprises than other new
trading partners? (The survey of Russian Enterprises).
* Informal lending: the amount, maturity, the existence and amount of collateral, and
provisions in case of default? (RPED survey)
The data gathered through these surveys has allowed researchers to better understand the role
of contracts and legal institutions on firm's performance. In particular, Hendley, Murrell, and
Ryterman (1999) focus on the importance of laws and legal institutions for Russian enterprises
15
by asking how laws facilitate transactions. They examine the significance of legal institutions in
the transacting process. Their analysis suggests that firms that invest in constructing contracts
and have better knowledge of the law have better chances to successfully complete a transaction.
To illustrate the potential advances in research on transaction costs, consider the example of
credit access. The World Bank's efforts to collect data on this particular type of costs has
already allowed researchers to explore more closely the theorized link between credit access and
firm performance. Fan, Lee and Schaffer (1996), for example, analyze the impact of the
financial sector reform in Russia on firms in the early years of transition. Their data, collected at
the end of 1994, shows that a partial reformn of the banking system, with many banks still acting
"soft" in their dealings with firms, slowed the enterprise restructuring process amd undermined
financial sector reform. Bank loans are not the firm's only source of financing, especially when
credit is rationed or the credit market is underdeveloped. A recent study on Zimbabwean
manufacturing firms using the RPED data (Fafchamps, 1996) suggests that trade credit is an
important, alternative source of short-term liquidity for many firms.
F. The Role of the State
Even in industrialized countries, the state is generally the most importaint single actor in
the economy. The policies the government implements can both harm and benefit economic
growth. As Stiglitz (1998) notes, "there is a growing consensus that governments can play a
vital role in successful development efforts, but we also recognize that the wrong kind of
government intervention can be highly detrimental." Surveys of firms can therefore be an
important tool in determining which aspects of government involvement are beneficial and which
are harmful.
Regulation
Despite its potential importance in explaining different growth rates across nations, little
is known empirically about regulation in developing countries. For instance, 'what are the
effects of regulation on firm performance, employment, and pricing? What determines the extent
and the types of regulation that the government of a developing country adopts? Do regulations
change over time? How do regulations affect firms' perception on the likelihood of
16
expropriation? Given the profound implications for firm productivity and innovations, further
research should address the impacts and the deterrninants of regulation in developing countries.
To accomplish this research agenda it is important to gather panel data on, for example, pricing
behavior, service coverage and quality, profitability, market structure, rent distribution (i.e., were
employment and wages affected by regulatory changes), and dynamic efficiency (measured by
the rate and direction of innovation and productivity).
World Bank surveys probably have focused more on regulation than any other theme,
although the variation in coverage is vast across surveys. Those that cover regulation thoroughly
(the Bosnia Survey, the World Business Environment Survey, The Emergence of Private Sector:
Hungary, and the RPED) ask questions about: the waiting period for goods to arrive, the level of
government to deal with in regulation, the main problems in dealing with govermnent agencies,
the frequency with which firms are required to meet with government officials, the costs of
facilitators necessary for dealing with the government, the burdens of licensing requirements,
costs of obtaining licenses and permits, various types of taxes, tax treatments for profits
reinvested in company, and incentives for investment in new machinery and equipment.
A general problem with current Bank surveys on regulation, however, is that they focus
on the barriers that regulations impose on business. They tend to ask firms "how severe" certain
regulations are to firrn operations. It is true that burdensome and often unnecessary regulations
are common. However, many regulations are important to the functioning of the economy (e.g.,
environmental regulations that force firms to internalize all costs of their production). In those
cases, the regulation will be an obstacle from the firm's perspective, but efficient from the
perspective of the entire economy. Surveyors need to think carefully about how to uncover the
true costs of regulation, since estimates will be biased upwards by simply asking the firm
whether a regulation is an obstacle.
State-Owned Enterprises (SOEs)
Developing countries claim a higher share of SOEs in their economies than do industrialized
countries (Haggarty and Shirley, 1997). While SOEs resemble private firms in many respects,
surveys of SOEs must ask some different questions. These differences reflect the different
objectives of SOE management and objectives. SOE principals are typically bureaucrats or
17
politicians (whose incentives differ from private shareholders and creditors). In particular, the
following are potentially important for understanding SOEs behavior in developing countries.
* Cash flow rights. What tax rate does the SOE face? Does it adjust from year to year? Does
the state charge different rates for different SOEs? Since the state may use tax rate as a
transfer mechanism, it is useful to see the extent to which the government uses it for this
purpose. How do tax rates of SOEs differ from those of comparable private firms? Does the
government subsidize or bailout SOEs when they are in financial trouble?
* Control rights. Does the government appoint the top managers? Does the government
appoint the middle managers? How were managers selected (bidding, election by workers,
appointment by the government)? How long is manager tenure? What is the scope of
managers' authority (e.g., decisions regarding production, investment, wages, staffing, and
borrowing)? Does the government often interfere in production decisions? Often in wages?
In hiring and firing? In new investment? In acquisition and consolidation? In borrowing?
* Incentives. How does the government align the manager's and government's objectives?
Does it use state plans, performance contracts that spell out goals and incentives, private
managers, who have control rights for a fee, or private parties bidding for concessions to
operate and invest in SOEs and keep the profits (World Bank, 1995)?
To illustrate the usefulness of firm-level data to yield insights about policy questions,
consider studies based on a 1980s panel data set of Chinese SOEs (Li, 1997; Xu, 1998; Groves et
al. 1995a, 1995b). Using these data, researchers have found that decentralizing production
decisions, allowing managers to determine wages, and allowing firms to retain a larger share of
their profits all improved productivity. Most importantly, increased competition, as measured by
lower markup ratios, substantially increased productivity.
18
G. Beyond the firm: "The micro-foundations of macroeconomics"10
The scope of firm-level surveys goes well beyond testing theories of the firm. As many
researchers have suggested, firm-level data has become key to understand the macro economy.
Macroeconomic analysis concentrates mainly on understanding the determinants of output,
unemployment and inflation. Researchers have thus built macro-economic variables from
industry-level data under the assumption of a "representative firm" within each industry. Recent
studies in developed economies, however, demonstrate that seemingly similar firms in the same
industry exhibit different behavior in terms of output, investment, and employment, suggesting
that aggregation may obscure important phenomena. Though these findings do not justify
rejecting the representative firm assumption, they highlight how we can more completely
understand changes in macro variables by analyzing these variables at the firm level.
Firm-level data is also useful for macro-analysis.since it helps test robustness by
complementing aggregate, official, sources of data. Robustness tests are especially relevant for
transition economies, where pre-transition data is often not reliable and the process of collection
of post-transition data is still under revision. Firm-level surveys can help reconcile seemingly
contradictory aggregate results by disentangling contrasting effects and offering a different
perspective on the issue in question.
Next, we provide a brief overview of the most important theoretical macro-economic
issues, which can benefit from this micro approach to macro-analysis, and of the (limited)
empirical testing done using World Bank surveys.
Growth and Investment
An important aspect of macro-economic analysis is the link between growth and
investment. In particular, recent studies suggest that the rate of investment is positively linked to
the rate of growth. To better understand this link, researchers have focused on the determinants
of the investment decision and found that this decision depends on financial constraints, the risks
firms face, and the productivity of investment itself (Jenskin, 1998; Gyimah-Brempong and
Traynor, 1999). This line of research however has used mainly aggregate (country-level) data.
10 This section draws heavily from Haltiwanger (1997) and Campbell (1997).
19
The problem with this approach is that aggregate data fails to capture heterogeneity across firms
and non-linearities in investment decisions recently highlighted by a new streamn of works (Doms
and Dunne, 1994; Caballero, et al, 1995; Cooper, et al, 1995).
The problems of an "aggregate" approach to the study of investment decisions are even
more evident by comparing two recent studies on investment productivity and growth in Africa.
The first - Devarajan, et al (1999) - focuses on investment productivity using cross-country data.
Contrary to the aforementioned theoretical predictions, the authors find that neither public nor
private investment significantly impacted growth and output in Africa over the past three
decades. To reconcile the empirical findings with the theoretical predictions, the authors
integrate cross-country exploration with firn-level data on investment in the mianufacturing
sector in Tanzania. This analysis suggests that the low level of investment can be partially
attributed to the low productivity of capital for African firms. The low level of productivity of
capital then explains the absence of a cross-country correlation between growth performance and
investment.
The second study - Gunning and Mengistae (1999) - on the other hand reaches the
opposite conclusion: the productivity of capital for African firms has increasecl over the past ten
years. In particular, the authors explore the rate and the productivity of investment using firm-
level panel data collected by the RPED in eight African countries. Their detailed data set
contains information on capital stock and market structure and helps document low levels of
investment as well as heterogeneity in firm and investment productivity in the African
manufacturing industries. They conclude that manufacturing firms are profitable and that
investment productivity has improved in the past ten years. 1 These results, in contrast with the
findings of Devarajan et al. (1999), suggest that solving the puzzle of the low level of investment
and the weak link between investment and growth in Africa may require exploring more closely
other determinants of investment, such as the degree of reversibility or risk.
Micro panel data on characteristics of capital stock and firm productivity could improve
our understanding of the link between investment and growth. Unfortunately, this type of data is
not readily available. Bank coverage has improved recently with the implementation of the three
1 l This result is derived by looking directly at the productivity of the firms, and indirectly at the degree of tolerance
of the markets toward inefficient investment.
20
waves of the RPED surveys. But we need more detailed survey data to enhance our
understanding of investment dynamics. In particular, very limited information is available on the
fixed costs of investment, the degree of reversibility or on the perceived risk that firms associate
to a certain investment decision.
Unemployment and the labor market
Recently, researchers have investigated the high rates of job creation and job destruction
experienced by market economies.'2 This continuous reallocation of resources, which occurs
within industries and is somewhat obscured by the aggregate measures of unemployment, is an
important part of economic adjustment and growth. The smoothness of these reallocation
processes impacts economic performance by affecting the evolution of firms and industries and
thus productivity growth. Reallocating jobs and factor inputs from less efficient to more efficient
plants may help improve industry-level productivity in an environment with frictions and
imperfect infornation (Baily, Hulten and Campbell 1992, and Olley and Pakes 1996). The same
theoretical approach, which allows for frictions in reallocating workers and jobs, helps also to
shed light on business cycle behavior of gross worker flows (Mortensen, 1994; Ramey and
Watson, 1997).
World Bank coverage of employment dynamics is limited in part because the literature
itself is limited. The firm-level data necessary to test hypotheses is not yet fully available even
for industrialized economies. Moreover, empirical research on this front is hampered by the lack
of micro time-series data. Some notable exceptions include the East Asia Competitiveness
Survey, the RPED, and the data collected by Haltiwanger and Vodopivec in Poland and Estonia
(1995).
The East Asia survey collects detailed infornation on the workforce, but has limited data
on average tenure. The Labor section of the RPED questionnaire focuses on the structure of the
workforce, but less on the dynamic aspect of employment flows. The RPED collects informnation
on seasonal hires, apprentices, paid and unpaid relatives of the owner, but mainly for the second
or third waves of the survey. In addition, the usefulness of the data is limited by consistency
12 For excellent surveys of this literature, see Davis and Haltiwanger (1998), Mortensen (1986), Pissarides (1990),
and Mortensen and Pissarides (1997).
21
problems across countries in terms of the survey instruments used. The Estonia and Poland data
sets document in great detail the employment dynamics experienced by these countries between
1989 and 1995, using both official government data and information collected through a labor
force survey.
Responses to Crises and Institutional Changes
Following the transition from a planned to a market economy, Eastern European and
fonner Soviet countries experienced a larger-than-expected decline in aggregate activities.
Recent important studies have looked for causes of this output collapse. Ex ante, researchers
agreed that output may initially decline before resuming its growth as the market reallocated
resources from less to more productive uses. Borensztein and Ostry (1995) hypothesize that the
sharp fall was mainly the result of a supply side shock: as prices were liberalized, state owned
enterprises were unable to pay for inputs. Commander and Coricelli (1992), and Borensztein, et
al. (1993) emphasize a demand shock: the increase in prices reduced the domestic demand for
final goods. Berg (1994), Atkeson and Kehoe (1996), and Shimer (1995) argue that structural
adjustments related to the transition itself were the main cause of the initial decline. Finally,
Murphy, et al (1992) and Shleifer and Vishny (1993) focuse on the political economy of the
transition process and emphasized the perverse effects of a partial price liberalization reform and
government corruption.)3
The World Bank coverage of output behavior in transition economies is somewhat
limited and unsystematic across countries, hampering cross-country analysis. I]n addition, the
country-specific analyses are based on firm surveys designed to test specific hypotheses,
preventing researchers from evaluating competing hypotheses even within the same country.
Nonetheless, existing surveys have provided us with great insights into the effects of transition
on output.
13 Estrin, Schaffer and Singh (1993) offer new insights on the structural restructuring process that naturally
accompanies a change in regime for some Eastern European countries by integrating their industry-level analysis
with firm-level data. Their exploration suggests that the process of industrial adjustment across a few Eastern
European countries has been remarkably similar, and that the speed at which reforms are introduced affects
enterprise profitability. Pinto, Belka and Krajewski (1992) complement Estrin et al. (1993)'s results with a study of
the state enterprises in Poland before and after the first round of reforms. Their analysis reveals that firms, helped
22
The transition process did not simply lead to demand and supply shocks. It also led to a
deep institutional restructuring. It is plausible that the transition process, by removing the
existing coordinating mechanisms, led to an increase in search costs, and in turn to a decline in
aggregate activities. Some Bank surveys-for example, Ickes and Ryterman (1994), Qimiao Fan
(1994), De Melo (1 997)-have attempted to measure these changes. The data collected however
suffer of the limitations emphasized above, focusing mainly on few of the aspects of the
transition process.
Another dimension of particular interest to policy-makers is firn responses to macro
shocks. Since this type of analysis has a less defined set of theoretical hypotheses and may be
more specific in scope, it is difficult to evaluate the coverage of the World Bank surveys.
Instead, we discuss the advantages and disadvantages of recent surveys undertaken in response to
macro-economic crises.
Research on transition economies mentioned above is a natural starting point since they
provide some information on how firns react to shocks or changes in the environment in which
they operate. Earle, Estrin and Leshchenko (1996), for example, focus on the privatization
process and its impact on firm performance. Using survey data collected in Russia in 1994, the
authors analyze the effects of different ownership structures on firm behavior. Their findings
suggest that outsider-owned and state-owned firms differ significantly in terms of perforrnance
and but less in terms of restructuring behavior. The latter result may be partly a function of the
timing of the survey (it was conducted only two years into the transition process) and,
consequently, the restructuring process had made only limited progress.'4
Recanatini and Ryterman (1999a and 1999b) study the reaction of firns to the
institutional vacuum left by the removal of government institutions. Their analysis suggests that,
because of the sudden loss of information and the underdeveloped legal system, firms reacted by
by an underdeveloped banking system, initially postponed the inevitable restructuring process. The decline in real
wages and the increase in interest rates appear to be therefore responsible for the collapse in output.
1 The timing of these surveys and their "once for all" approach seriously limit the usefulness of these data sets.
Especially when focusing on macro-economic issues it is important to analyze cross-country time series.
Unfortunately, most of these surveys are not repeated over time, making it difficult to test both the existing
alternative theories and the validity of the survey instruments used.
23
attempting to join efforts with each other. This effort translated into the creation of new,
infornmal networks, which had a positive effect on firms' performance.
Firm-level survey data has also proven useful in our understanding of the recent East
Asian crisis. Detailed information collected on the crisis through surveys administered in 1997
and again in early 1998 offer good examples of how firm-level data can help studying responses
to shocks in a way that complements the more traditional macro/aggregate approach. Dollar and
Hallward-Driemeier (1998) focus on the adjustment process in the Thai industry following the
crisis. Their work highlights the link existing between firms' behavior and (poor) government
policies in the year preceding the crisis. As the crisis developed, firms struggled to cope with the
existing constraints and to adjust to the new environment.
IV. QUESTIONING QUESTIONS
Having discussed the type of data that would be most useful for firm surveys to gather,
we now turn to the subject of questionnaire design itself. Surveys generally ask respondents both
about their attitudes towards various issues and their recollection of past factual events.
Moreover, in-person surveys pose these questions in a social setting, albeit one with a script and
a purpose. Survey designers must realize both that the survey asks the respondent to perform a
series of cognitive tasks and that it is an interaction governed by governed by social norms
(Sudman, Bradburn, and Schwarz 1996). It is therefore important to consider how the questions
themselves can influence responses. This section reviews the literature on questionnaire design.
In particular, it seeks to learn how the survey design itself affects responses to questions, and
therefore conclusions the researcher may draw from those responses.
General Issues
Rea and Parker (1997) note six general problems to consider when preparing questions:
(1) an inappropriate level of wording; (2) ambiguous words and phrases; (3) rnultipurpose
questions (i.e., questions that ask two or more opinions simultaneously); (4) rmanipulative
information; (5) inappropriate emphasis; and (6) emotional words and phrases.
Additional problems include questions that can be answered differently by people with
the same opinion and questions that can be answered identically by people with opposite
24
opinions. Many of these problems are obvious, but some of their manifestations are not. For
example, a simple "yes/no" question can accidentally be manipulative or suggest to the
respondent that there is a correct response by not including all possible responses in the question
itself. A question that simply asks, "Do you do x?" may make the respondent more likely to
answer "yes" since the alternative was not presented. One possibility is to ask instead, "Do you
do x, or not?" (Fowler 1995). An induced response is more of a concern when one answer to a
question is more socially acceptable than another. We discuss issues in asking sensitive
questions below.
Asking multipurpose questions-questions that ask about more than one issue or along
more than one dimension-turns out to be unfortunately simple and common. For example, the
World Business Environment Survey (World Bank 1998) asks respondents to "rate the overall
quality, integrity, and efficiency of services delivered by the following public agencies or
services." Although quality, integrity, and efficiency are probably highly correlated, they are
three separate issues and should be asked separately. Sometimes, however, the problem is not so
obvious. "Agree/disagree" questions, for example, usually ask the respondent to rate how much
she agrees or disagrees with a statement. The survey asks the respondent whether she "strongly
agrees, agrees, disagrees, or strongly disagrees." The problem with this scale is that it combines
an emotional component ("strongly") with the more important cognitive component. Instead, the
survey should ask whether the respondent "completely agrees, generally agrees, generally
disagrees, or completely disagrees" (Fowler 1995).
Careful survey writing can minimize problems of the sort described above. Other
problems are more complicated. Small differences in question wording, for example, can
generate large differences in responses. A particularly well-known example is from a 1941
experiment in which matched samples of respondents were asked one of two questions (Sudman,
Bradburn, and Schwarz 1996). The first group was asked whether they thought the United States
"should allow public speeches against Democracy," while the second group was asked whether
they thought the United States "should forbid speeches against Democracy." Among the first
group, 21 percent favored free speech. Among the second group, 39 percent favored free speech.
Either question may be useful in tracking changes in perceptions, as long as the same question is
asked consistently across people and over time. That is, whichever question is asked, the exact
25
same question must be asked of every respondent. While it seems obvious to ask all respondents
exactly the same question, in practice interviewers often deviate from the survey script. Fowler
and Mangione (1990, p.35) note four studies that tracked interviewer-respondent interactions
found that interviewers deviated from their script on 20 to 40 percent of the questions on the
survey.
It is not possible to ensure that all interviewers follow the script to the letter all the time.
Indeed, turning interviewers into little more than programmed question-readers could have
unintended consequences by preventing them from interacting properly with the respondent.
Nonetheless, survey designers can take steps to minimize this source of error. Specifically, the
survey designer should recognize that the more likely a respondent is to interrupt the question,
the more likely the interviewer is to change the question wording. The survey writer should
design questions that need little additional explanation. Pretests can measure which questions
respondents tend to interrupt for clarification or which questions seem most likely to lead to ad-
libbing by the respondent. The researcher can use the pretest information to change the
questionnaire to minimize opportunities for the interviewer to change the written question.
The issues discussed above are general ones to keep in mind when developing and
implementing a survey. Many more specific issues are inherent to most surveys. Those issues
are discussed in the sections below.
Response scales
Many surveys ask respondents to rate issues on some scale (say, 1 to 5). In World Bank
surveys, such questions typically ask about infrastructure, regulation, or competitors. If phrased
properly, such questions may tell us what a firm views as problematic relative to other issues
presented in the questionnaire. Such methods, however, should be used with caution. First, they
are inherently subjective and respondent- (rather than firm-) specific, making interpretation
difficult. Second, respondents have little incentive to answer truthfully. Indeed, they may have
an incentive to provide answers they perceive as beneficial to themselves. A firm whose
production process generates a great deal of pollution, for example, has an incentive to report that
26
environmental regulations are onerous. Third, respondents may have distinct criteria in judging
the severity of a problem, making it impossible to divine the cause of the response.
Sometimes it may be impractical or impossible to avoid the rating method. Other times
the objective may be explicitly to measure manager opinion, in which case the rating method is
appropriate. Often, however, the goal is to determine, from an economic perspective, how such
issues affect economic growth. Rather than asking whether something is a problem, ask for
objective measures and use that data to determine their economic impacts empirically. For
example, rather than asking managers whether labor costs are a problem, ask for wages and
productivity measures. Empirical analysis can then determine whether labor costs in a particular
industry-country context are, in fact, higher than in other industry-country contexts, or whether
those costs have changed over time or in response to some policy intervention.
When one must rely on the rating method, it is important to consider how to interpret
scales, the costs and benefits of numeric and verbal scales, and the optimal number of responses.
Below we synthesize the recommendations from the survey literature on these issues.
1. The surveyor must remember that ordinal scale measurement is relative. That is, the
scale allows us to measure how people feel about an issue relative to the scale presented, but not
necessarily how they feel in any absolute sense. Consider a question that provides respondents
with the possible answer set A E {good, fair, or poor}. The same question that provides
respondents with possible answer set B E {excellent, good, fair, or poor} will not simply divide
the "good" responses of set A into subcategories. Instead, the meanings of all the possible
answers change in set B relative to set A, so that the meaning of the words "good" or "fair," for
example, are not comparable across sets A and B.
2. Numeric versus verbal scales. Numeric and verbal scales each have advantages and
disadvantages. The main advantage of numeric scales is that they allow greater variation in
responses without descriptive adjectives for each category. The main disadvantage is that the
anchors of the scale may affect responses. For example, a survey asking "how successful have
you been in life" found that on a scale of 0 to 10, 34 percent of respondents answered between 0
and 5, but on a scale of-5 to 5 only 13 percent answered between -5 and 0 (Sudman, Bradburn,
and Schwarz 1996). The main advantage of using descriptive adjectives for each category is that
each category can be relatively well-defined. The main disadvantage of descriptive adjectives is
27
that it is difficult to come up with meaningful and unambiguous adjectives to use on a multi-
point scale. Indeed, "a problem in international research .. . is how to get consistent
measurement of subjective states for different cultural groups. In particular, when scales are
defined adjectivally, it has been found that it is virtually impossible to have exact translations
across languages .... Numerical response scales, with only the ends of the continuum defined,
perhaps with some general discussion of how to use the points between the extremes, have
numerous advantages and constitute a very good way to have people perform some rating tasks"
(Fowler, 1995).
3. Number of categories. The researcher also must decide how many categories of
responses to include. The number of responses often varies by survey and by question within the
survey. Researchers-and econometricians in particular-wish to maximize variation among
respondents as much as possible, which suggests a larger number of potential r.esponses. On the
other hand, Fowler (1995) notes that studies show that more than 10 categories does not increase
variation in responses and that respondents seem to be able to meaningfully differentiate between
five to seven categories, but not more.
The "Don't Know" Problem
Every survey wrestles with the problem of whether to include a "don't know" or
"indifferent" category. Research on the topic is mixed (Gilljam and Granberg 1993). One view
holds that excluding the middle category forces some respondents onto one side of the issue
when they either have no opinion or do not have enough information to hold an informed
opinion. Excluding the middle category thus biases responses towards expressing preferences
even when respondents are indifferent. Another view holds that people who have slight
preferences are inclined to pick the middle category if it exists. Including the middle category
thus biases responses towards indifference even though, when pressed, many respondents who
expressed indifference actually have slight preferences.
Researchers offer differing advice, although they all recognize that the answer is unclear.
Gilljam and Granberg (1993) ask and conclude, "Should public opinion researchers take don't
know for an answer? Our view is that we don't know for sure, but probably should not."
28
Sudman, Seymour, and Bradburn (1982, p.141), however, note that "[w]hile it is impossible to
make any hard and fast rule, our advice would be contrary to general practice: include the middle
category unless there are persuasive reasons not to." Fortunately, the positive/negative response
ratio seems to remain relatively constant regardless of whether the survey offers a middle option
(Sudman, Seymour, and Bradburn 1982).
The important conclusion from the "don't know" debate is that the number of
respondents expressing neutrality will be biased upwards if the survey includes a "don't know"
or "indifferent" category, and that the number of respondents expressing an opinion will be
biased upwards if the survey does not include this option.
Filters and Branching
Filters and branching refer to questions that guide the respondent down a decision tree
that first determines whether the respondent has an opinion and then the intensity of that opinion.
One common problem with surveys is that respondents typically do not give the same responses
to the same questions; that is, there is a great deal of noise in each person's response to any given
question. Krosnick and Berent (1993) have shown that filters and branching increases the
probability that a respondent will answer questions identically when re-surveyed at a later date."5
This empirical finding is consistent with the above recommendation to not ask a single question
along two dimensions. Questions that ask, "How do you feel about policy x?" are, in effect,
asking two questions. First, "do you have an opinion on policy x." Second, "what is the
intensity of that opinion?"
Filters may be especially useful when the nature of the question mandates that responses
can include only one side of the "response distribution." For example, surveys may ask "how
concerned" someone is about an issue, or "how problematic" is a particular regulation to firm
growth. Allowed responses will likely range from "not concerned" (or "no obstacle") to "very
concerned" (or "severe obstacle"). Although it is not obvious, this answer set is biased because it
15The phrasing of the filter question itself will impact the results. The filter must be phrased so that it is unbiased.
For example, if trying to determine whether the respondent is concerned about an issue, ask, "Are you concerned
with x, or do you feel it's not a problem?"
29
presents only half the possible answers. The responses do not allow firms to say that they favor
the issue or regulation in question and convey to the respondent that the interviewer believes the
answer should be negative. An unbiased response scale, therefore, would include as many
positive as negative responses, with "no obstacle" as the middle, indifferent, category. By
presenting "not concerned" or "no obstacle" as an endpoint, the positive responses have. in
effect, been truncated. Often, however, there is no other obvious way to ask the question as
including the positive responses may seem inappropriate. Asking a filter question first can
mitigate the bias that these truncated response distributions would otherwise impose.
Sterngold, Warland, and Herrmann (1994) found that including a filter that asks
respondents whether they were concerned about an issue before asking them how concerned they
were dramatically reduced the percentage of respondents expressing concern. Moreover, the
drop in those expressing concern came not just from those who expressed "a little concern," but
also among those who were "somewhat" and "very" concerned. The authors concluded that
"standard degree-of-concern items may be leading questions that encourage res-pondents to
overstate their concerns about issues."
Filters and branches could be used effectively in many World Bank surveys, especially
when the question itself may bias the respondent's answers. For example, The World Bank often
wishes to measure whether institutional features of a country impede firm growth. Such surveys
often ask firms to rate regulations or potential production bottlenecks from "no obstacle" to
"major obstacle." These questions may suggest to the firm that the interviewer believes that the
issues in question are obstacles. In effect, the question leads the respondent to an answer, and
thus to biased results. An acceptable, although perhaps unlikely, response is also that the
regulations are "very helpful." Presenting these positive responses may make the question seem
awkward. Few firms are likely to report that tax inspectors are "very helpful." Not including the
positive responses, however, is likely to lead to a strong upward bias in the number of firms
reporting that the issue is a problem. World Bank surveys can mitigate this problem by imposing
filters on these questions. First ask the firm the binary question of whether the regulation in
30
question is an obstacle or not and then ask how great an obstacle if the firm replies in the
affirmative."6
Order Effects
It is well-known that the order in which related questions are asked can impact responses.
Order effects probably occur because additional questions about a topic prod the respondent's
memory, making certain recollections easily accessible for future questions. Question order
matters when asking about related specific and general questions and about the timing of past
events. The order in which responses to a question are presented can matter as well, although the
literature does not provide any rules as to how they matter.
Specific and general questions
Asking respondents specific questions influences how they will answer related general
questions. These "context effects" are termed "contrast" and "assimilation" effects. Contrast
effects occur when specific questions cause the respondent to answer a general question in
contrast to the preceding specific (but related) questions. Assimilation effects occur when
specific questions cause the respondent to answer the general question as a kind of summary of
the preceding specific questions.
For example, suppose people are asked first if they have a successful marriage, and then
asked how successful their lives are, in general. A contrast effect would cause a respondent to
rate his success in life relative to his marriage. An assimilation effect would cause a respondent
to include her marriage as one of the factors determining her success in life."7
16 Stemgold, Warland, and Herrrnann (1994) also recognize that including a filter for many questions is "a rather
time-consuming and burdensome approach that may encourage respondents to give answers that allow them to
bypass the follow-up degree-of-concern questions. An alternative is to use filters intermittently during the course of
the survey-especially before the first degree-of-concem response item-to communicate to respondents that [not
concemed] is a legitimate response. This may have a carryover effect on other degree-of-concern items in the
survey that are not preceded by [not concemed filters]. A second altemative is to ask the filter for an entire list of
items, and then to cycle back to those iterns respondents said they were concerned about and ask the follow-up
questions."
]7As another, example, suppose one asked a random sample citizens whether they believe politicians are, in
general, honest. Assume that 20 percent of this sample reported that they do. Next, suppose one asked another
random sample first whether they believed their representative to Congress was honest, and then whether they
believed politicians, in general, were honest. The results of the general question are likely to change, reflecting
31
Surprisingly, there is agreement about when assimilation and contrast effects emerge. As
Schul and Schiff (1993) note, "[r]esearch has shown that the impact of early specific questions
have on the responses to a later general question is influenced by the number of specific
questions and their positioning in the survey. [Wihen a general question is preceded by a single
specific question, responses to the general question show a contrast effect unless the specific and
general questions are separated." Sudman, Bradbum, and Schwarz (1996) note that asking the
specific questions first yields assimilation effects when there are unrelated questions between the
specific and general questions. Even graphically separating the specific and general questions on
a self-administered survey or putting the specific and general questions in different sections of an
interviewer-led survey generated assimilation effects. The contrast and assim.ilation effects are
more intense the more sweeping or ambiguous is the general question.
To sunmnarize, a specific question followed immediately by a general question tends to
generate a contrast effect, while a specific question followed by several unrelated questions or a
section break and then a general question tends to generate an assimilation effect. A general
question asked before specific questions or completely absent specific questions may reflect
contrast or assimilation effects, but the researcher has no way to know which. The survey can
minimize such problems by avoiding ambiguity as much as possible and by asking the
respondent to answer general questions either to reflect his experiences or actions or what he
believes the general condition to be relative to his experiences or actions. That is, ask for a
contrast or assimilation of specific questions posed first.
Recent and less-recent behavior
Often the researcher is interested in learning about recent and typical behavior, which are
not always identical. Surveys frequently ask whether a respondent has done something within
the past x weeks (or months or years) and then within the past x + y weeks. It turns out that
asking about the shorter time period first tends to lead respondents to over-report that activity
(Fowler 1995). This over-reporting may happen because of the respondent's desire to report
contrast or assimilation effects. Assume 75 percent of the second sample believed its representative was honest. If
10 percent of this second sample believed that politicians in general were honest, it could be an example of a
contrast effect, while if 30 percent of this second sample believed that politicians were honest, it could be an
example of an assimilation effect.
32
engaging in that activity even if it falls slightly out of the specified time period. Asking about
the longer time period first could make the respondent more willing to state that she did not
engage in that activity more recently.
Within-question order effects
Order effects occur within questions, as well. That is, the order in which responses to a
question are presented can affect the frequency with which those responses are chosen or rated.
For example, Boardman, et al. (1996) note "a study that asked some respondents to value
preserving seals and then whales, while others were asked to value preserving whales and then
seals. Seal values were considerably lower when the seal question was asked after the whale
question." Unfortunately, empirical findings on within-question effects are ambiguous.
Sudman, Bradbum, and Schwarz (1996) note that "despite the considerable empirical evidence
for the emergence of response order effects, we know relatively little about the conditions that
determine their emergence and direction, and the area is characterized by a large number of
apparently inconsistent findings .... Because of the shortcomings in the available data, it is
currently impossible to draw strong conclusions about the processes that underlie response order
effects in survey measurement." Survey designers may be able to minimize this type of bias by
randomizing across respondents the order the responses are presented.
Sensitive Questions and Truth-Revealing Mechanisms
Often researchers wish to study behavior that society (or the respondent) considers
inappropriate or illegal. For these or other reasons, the respondent may have an incentive to be
untruthful when responding to questions. Social scientists typically worry about this problem in
the context of drug use or sexual behavior (Fowler 1995, Tourangeau and Smith 1996). World
Bank surveys face similar problems when they ask about firm behavior regarding issues such as
corruption, regulation, and taxation. A firm may understandably be reluctant to admit to
engaging in illegal activities or to avoiding regulations. While there is no way to guarantee
truthful responses, research has demonstrated that the mode of data collection and the method of
asking the question can affect whether respondents admit to embarrassing OT illegal behavior.
33
Unfortunately, these suggestions may be of limited usefulness-or will require some creativity to
use-in in-person surveys conducted for the purpose of building a large dataset.
Not surprisingly, respondents are less likely to admit to incriminating behavior in a face-
to-face interview than in a self-administered survey. Tourangeau and Smith (1996) note that the
"literature on sensitive questions demonstrates that the method of collecting the data can affect
the answers that are obtained.... Several of these studies have demonstrated that self-
administration of sensitive questions increases levels of reporting relative to administration of the
same questions by an interviewer. Respondents are apparently reluctant to admit to an
interviewer that they have engaged in illegal or otherwise embarrassing activities. Studies
comparing self-administered questionnaires with conventional paper-and-pencil interviewer
administration have shown that self-administration increases reporting of abortions, alcohol
consumption, and illicit drug use."
Although most World Bank firm surveys are conducted in-person, it may be possible to
combine a self-administered component with the in-person interview. For exarnple, the
interviewer could hand each respondent a package containing additional questions on sensitive
topics and an addressed, starnped envelope that the respondent can use to retuni the survey. The
self-administered fbrm would have to be coded so the self-administered section can be matched
to the in-person section. This combination raises two issues that are not addressed in the
literature. First, would respondents be more likely to be truthful on a self-administered
questionnaire if they know that, despite answering the questions in private, thei.r answers will not
be anonymous? On one hand, they can answer the questions in private and not admit their
behavior verbally.8 On the other hand, if the answers can be linked back to their firm it may not
matter whether the answer is given in written or oral form. This is, however, an open question
and could be tested empirically during the survey pretest phase.
Randomized responses are another way to increase reporting of such behavior (Sudman
and Bradburn 1982, Fowler 1995). As will be clear, this method is especially useful for learning
the percentage of a population engaged in certain activities, but by its nature acids substantial
measurement error, making hypothesis testing difficult. The method involves presenting the
s It is conceivable that speaking out loud may affect truthfulness, especially since the interv.iewer is likely to be a
citizen of that country, and most likely a local resident.
34
respondent with two questions simultaneously: the sensitive or embarrassing question, and a
completely innocuous question. For example, question A might be, "Have you ever bribed a
government official?" Question B might be, "Is your birthday in December?" The respondent
then flips a coin (not revealing the result to the interviewer) and on seeing heads answers
question A and on tails answers question B. The surveyor simply records the response. As a
result, the respondent can give a completely truthful response knowing that the surveyor has no
way of knowing which question she is answering.
Determining the percentage of a sample that has bribed a government official then
becomes simple. Assume that 400 of 1200 respondents answered "yes" to the question. We
would expect 50 percent of the respondents to answer the birthday question, and 1/12 of those to
answer the question affirmatively. That is, 1200(0.5)/12 = 50 of the "yes" responses were to the
birthday question. The expected number of people answering "yes" to the bribery question is
therefore 400 - 50 = 350. Because 600 (i.e., 1200*0.5) people answered the bribery question we
could conclude that 350/600 z 58.3 percent of the sample had bribed a government official. This
method complicates hypothesis testing because of the measurement error it introduces, but World
Bank surveys intent on measuring particular behavior in populations could potentially use this
method to great effect.
V. CONCLUSIONS
This review has attempted to examine existing World Bank firm surveys and the state-of-
the-art in questionnaire design. World Bank surveys have dramatically improved our
understanding of firm behavior in developing countries. In addition, most of the data and
measurement requirements we spell out in this paper are fulfilled in some surveys, such as the
use of panel data, and detailed measurements about various themes. Needless to say, not all
Bank surveys should pay heed to the specific themes and hypotheses this paper mentions; we
recognize that no survey can address all the concerns raised here given time constraints, ad hoc
purposes of specific surveys, and survey costs. We also recognize that our coverage in this
survey of surveys is not comprehensive. Ultimately surveyors must decide how they wish to
address the question at hand, and design questions accordingly. However, if a firm survey aims
35
to become what LSMS has become for household surveys (that is, an excellent source of data
useful for answering a range of questions across countries), it may be worthwhile to take into
account the observations made in this paper about data and measurement issues.
Many questions about collecting firm data remain unresolved: How do we check for data
quality? How do we know whether the numbers come from a false accounting book? What are
the best ways to ask questions? How important is survivor bias and recollection errors? Should
we pay interviewees based on the percentage of missing information (Philipson, 1997)?
Our analysis however offers a few important lessons:
1. Data issues and hypothesis testing:
* If possible, use panel data.
* Have enough information on productivity to estimate a production funct'ion.
3 Avoid the paradigm of "list the severity of the obstacle/problem on a scale of 1 to 5."
Instead, ask for data on specific dimensions of the problem that will shed light on
alternative hypotheses and policy recommendations.
* Pick particular disaggregated industries, and sample those industries in each survey.
- Identify the most important policy interventions of interest, and consider how you will
empirically identify that change by picking useful instruments.
2. Questionnaire Design:
* Include only one idea or dimension per question. Do not ask in one question about, for
example, the "quality, integrity, and efficiency" of services.
* Consider the costs and benefits of numeric versus adjectival scales. Scales in which each
point is labeled may be more precise than numeric scales in which only the endpoints are
labeled. On the other hand, responses are very sensitive to the exact adjective chosen,
and it may be impossible to translate adjectives precisely across languages, making it
impossible to compare responses across countries.
* Recognize that the share of respondents expressing opinions will be biased upwards if the
survey does not include a middle (i.e., "indifferent" or "don't know") category, and
biased downward if the survey does include the middle category.
36
* When asking degree-of-concern or how-great-an-obstacle questions consider first asking
a filter question (e.g., "Do you believe this regulation is an obstacle, or not?") and if the
answer is yes, then asking how severe the obstacle is.
* Be aware of context effects. The act of asking questions can affect the answers given on
subsequent, related questions.
* Think carefully about how to ask sensitive questions. Consider using a self-administered
module for sensitive questions. Alternatively, a randomized response mechanism may be
a useful truth-revealing mechanism.
37
REFERENCES
Alchian, Armen and Harold Demsetz (1988) "Production, Information Costs, and Economic
Organization" in Ownership, Control, and the Firm, vol. 1, Oxford and New York: Blackwell,
1988, 119-43. Previously published, 1972.
Atkeson, A. and P. Kehoe (1996), "Industry Evolution and Transition: A Neoclassical Benchmark",
Department of Economics, University of Pennsylvania, mimeo, December.
Baily, M., C. Hulten and D. Campbell (1992) "Productivity Dynamics in Manufacturing Plants",
Brookings Papers on Economic Activity " Microeconomics, pp. 1 8 7-249.
Becker, Gary S. (1962) "Investment in Human Capital," Journal of Political Economy (supplement), 70:
9-49.
(1983) "A Theory of Competition among Pressure Groups for Political Influence,"
Quarterly Journal of Economics, 98: 371-400.
Berg, A. (1994), "Supply and Demand Factors in the Output Decline in East and Central Europe",
Empirica, Vol.21.
Blank, L. and M. Grosh (1999), "Using Household Surveys to Build Analytic Capacity", The World Bank
Research Observer, Vol.14, No.2, August 1999, pp.209-227.
Boardman, Anthony, David Greenberg, Aidan Vining, and David Weimer (1996) "Contingent Valuation:
Using Surveys to Elicit Information about Costs and Benefits." Chapter 11 in Cost-Benefit
Analysis: Concepts and Practice. Prentice-Hall.
Borensztein, E. And J.Ostry (1995), "Common Causes or Structural Adjustment? Output Decline in
Eastern Europe and Poland", in R.Holzmann et al.(eds.), Output Decline in Eastern Europe,
Kluwer Academic Publishers, Netherlands.
,D. G. Demekas, and J.D. Ostry (1993), "Output Decline in the Aftermath of reforn: the cases
of Bulgaria, Czechoslovakia, and Romania", in Eastern Europe in Transition. From Recession to
Growth?, Mario Blejer, et al. (eds.), World Bank Discussion Paper N. 196.
Caballero, R., E. Engel and J. Haltiwanger (1995) "Plant Level Adjustment and Aggregate Investment
Dynamics", Brookings Papers on Economic Activity, Vol.2, pp.1-39.
Campbell, Jeffrey (1997) Commentary to "Measuring and Analyzing Aggregate Fluctuations: The
Importance of Building from Microeconomic Evidence." Federal Reserve Bank of St. Louis
Economic Review. May/June 1997, pp. 83-85.
Caves, Richard (1989) "International Differences in Industrial Organization," in Handbook of Industrial
Organization, vol. 1, ed. by R. Schmalensee and R.D. Willig, Elsevier Science Publishers.
Comanor, W.S. and Wilson, T.A. (1967) "The effect of Advertising on competition: a survey," Journal
of Economics Literature, 17: 435-476.
38
Commander, S. And F. Coricelli (1993), "Output decline in Hungary and Poland in 1990/1991: Structural
changes and aggregate shocks", in Eastern Europe in Transition: From Recession to Growth?,
Mario Blejer, et.al. (eds.), World Bank Discussion Paper, N.196.
Cooper, R., J. Haltiwanger and L. Power (1995) "Machine Replacement and the Business Cycle: Lumps
and Bumps", NBER Working Paper no. 5260, September 1995.
Deaton, Angus (1997) The Analysis of Household Surveys: A Microeconometric Approach to
Development Policy. Baltimore: The John Hopkins University Press.
Denison, Wdward F. (1962) "Sources of Growth in the United States and the Alternatives Befor Us,"
Supplement Paper 13, New York, Committee for Economic Development.
Devarajan, S., W. Easterly and H. Pack (1999) "Is Investment in Africa Too Low or Too High?", The
World Bank, mimeo, May.
Dollar, D. and M. Hallward-Driemeier (1998) "Crisis, Adjustment, and Reform in Thai Industry",
mimeo, The World Bank, November.
Doms, M. and T. Dunne (1994) "Capital Adjustment Patterns in Manufacturing Plants", unpublished
manuscript, Center for Economic Studies, U.S. Bureau of the Census.
Fowler, Floyd (1995) Improving Survey Questions. Applied Social Research Methods Series. Vol. 38.
(1988) Survey Research Methods. Revised Edition.
and Thomas Mangione (1990) Standardized Survey Interviewing. Minimizing Interviewer-
Related Error. Sage Publications.
Gilljam, Mike, and Donald Granberg (1993) "Should we take Don't Know for an answer?" Public
Opinion Quarterly. Vol 57:348-357.
Groves, Theodore, Yongmiao Hong, John McMillan, and Barry Naughton (1 995a) "Autonomy and
Incentives in Chinese State Enterprises." Quarterly Journal of Economics, 109, pp. 181-209.
Groves, Theodore, Yongmiao Hong, John McMillan, and Barry Naughton (1995b) "China's Evolving
Managerial Labor Market." Journal of Political Economy. 103, pp. 873-92.
Gunning, J. and T. Mengistae (1999) "Determinants of African Manufacturing Investment: The
Microeconomic Evidence", The World Bank, mimeo, May.
Gyimah-Brempong, K. and T.L. Traynor (1999) "Political Instability, Investment and Economic Growth
in Sub-Saharan Africa", Journal of African Economies, Vol.8, pp.52-86.
Haggarty, Luke, and Mary Shirley (1997) "A New Data Base for State-Owned Enterprises," World Bank
Economic Review, 11(3), 491-513.
39
Haltiwanger, John (1997) "Measuring and Analyzing Aggregate Fluctuations: The Importance of
Building from Microeconomic Evidence." Federal Reserve Bank of St. Louis Economic Review.
May/June 1997, pp. 55-78.
Jenkins, C. (1998) "Determninants of Private Investment in Zimbabwe", Journal of African Economies,
Vol.7, pp.34..61.
Krosnick, Jon and Matthew Berent (1993) "Comparisons of Party Identification and Policy Preferences:
The Impact of Survey Question Format." American Journal of Political Science. Vol 37, No 3.
August 1993, pp.941-964.
Lazear, Edward, and Sherwin Rosen (1981) "Rank-order Tournament as Optimum Labor Contracts,"
Journal of Political Economy, 89: 841-864.
Li, Wei (1 997) "The Impact of Economic Reform on the Performance of Chinese State Enterprise: 1980-
1980," Journal of Political Economy, pp 1080-1106.
Lopez-de-Silanes, Florencio. 1997.
Morck, R., A. Shleifer, and R. Vishny (1986) "Ownership structure and corporate performance: an
empirical analysis," NBER: Cambridge, Mass., mimeo.
Mortensen, D. (1994) "The Cyclical Behavior of Job and Worker Flows", Journal of Economic Dynamic
and Control, no. 18, November, pp.1 121-1142.
Noll, Roger (1989) "The politics of regulation," in Handbook of Industrial Organization, vol. 1, ed. by
R. Schmalensee and R.D. Willig, Elsevier Science Publishers.
Nalebuff, B. and J. Stiglitz (1983) "Prizes and Incentives: Towards a General theory of Compensation
and Competition," Bell Journal of Economics, 14: 21-43.
Olley, S. and A. Pakes (1992) "The Dynamics of Productivity in the Telecommunications Equipment
Industry", NBER Working Paper, no.3977, January 1992.
Philipson, Tomas (1997) "Data Markets and the Production of Surveys," Review of Economic Studies, pp.
47-72.
Ramey, G. and J. Watson (1997) "Contractual Fragility, Job Destruction and Business Cycle", Quarterly
Journal of Economics, Vol. 112, no.3, August, pp.873-911.
Rea, Louis and Richard Parker (1997) Designing and Conducting Survey Research: A Comprehensive
Guide.
Recanatini, F. and R. Ryterman (1999a), "Transition and Output Collapse: Measuring the Impact of
Institutional Change", mimeo, The World Bank, June.
and (1999b), "From Disorganization to Self-Organization", rnimeo, The World
Bank, September.
40
Rosen, Sherwin (1986) "Prizes and Incentives in Elimination Tournaments", American Economic Review,
76: 701-715.
Rosen, Sherwin (1992) "Contracts and The Market for Executives," in Lars Werin and Hans Wijkander,
eds., Contract Economics, Oxford, U.K.: Basil Blackwell.
Salinger, Michael A. (1990) "The Concentration-Margin Relationship," Brookings Papers on Economic
Activity, pp. 287-335.
Scherer, Frederic M., and Mark Perlman (1992) Enterpreneurship, Technological Innovation, and
Economic Growth, ed., Ann Arbor: University of Michigan Press.
Scherer, Frederic M., and D. Ross (1990) Industrial Market Structure and Economic Performance, 3rd
ed., Boston: Houghton, Mifflin.
Schul, Yaacov and Miriam Schiff(I 993) "Measuring satisfaction with organizations: Predictions from
information accessibility." Public Opinion Quarterly. Vol 57: 536-551.
Shirley, Mary, and Lixin Colin Xu. (1 998a) "Information, Incentives and Commitment: An Empirical
Analysis of Contracts between Government and State Enterprises." Journal of Law, Economic
and Organization, 14(2), pp. 358-78.
Shirley, Mary, and Lixin Colin Xu (1998b) "The Empirical Effects of Performance Contracts," mimeo,
the World Bank.
Shleifer, Andrei, and Robert W. Vishny (1996) "A Survey of Corporate Governance," Journal of
Finance, 52(2), June, pp. 737-83.
_ and (1993), "Corruption", Quarterly Journal of Economics, Vol. 108, N. 3.
Stemgold, Arthur, Rex Warland, and Robert Herrnann (1994) "Do surveys overstate public concerns?"
Public Opinion Quarterly. Vol 58:255-263.
Stigler, George (1968) The Organization of Industry, Homewood, ILL.: Richard D. Irwin.
Stiglitz, Joseph E. (1998) "Redefining the Role of the State." Speech given on the l Oth anniversary of
the MITI research institute. March 17, 1998.
Stone, Andrew H.W. (1992) "Listening to Firms," Policy Research Working Paper #923, the World
Bank.
Suchman, Lucy and Brigitte Joran (1990) "Interactional Troubles in Face-to-Face Survey Interviews."
Journal of the American Statistical Association. March 1990, Vol. 85, No. 409.
Sudman, Seymour, Norman Bradburn, and Norbert Schwarz (1996) Thinking About Answers. The
Application of Cognitive Processes to Survey Methodology. Jossey-Bass.
Sudman, Seymour and Norman Bradburn (1982) Asking Questions. Jossey-Bass, San Francisco.
41
Tan, Hong W., and Geeta Batra (1995) Enterprise Training In Developing Countries.: Incidence,
Productivity .Effects, and Policy Implications, Private Sector Development Department, The
World Bank.
Tourangeau, Roger and Tom Smith (1996) "Asking sensitive questions. The impact of data collection
mode, question format, and question context." Public Opinion Quarterly. Vol 60: 275-304.
Tracy, Paul E. and James Alan Fox (1981) "The Validity of Randomized Response for Sensitive
Measurements." American Sociological Review. Vol 46. April, pp.1 87-200.
Tybout, James (1998) "Manufacturing Firms in Developing Countries: How Well do they do, and why?"
Policy Research Working Paper, No. 1965, the World Bank.
Viscusi, W. Kip, John M. Vernon, and Joseph E. Harrington, Jr. (1995) Econotnics of Regulation and
Antitrust, Cambridge, Massachusetts: The MIT Press.
Willis, Robert (1986) "Wage Determinants: A Survey and Reinterpretation of Human Capital Eaming
Functions." in Handbook of Labor Economics, ed. by 0. Ashenfelter and R. Layard.
Winston, Clifford (1993) "Economic Deregulation: Days of Reckoning for Microeconomists," Journal of
Economic Literature, 31(September), pp. 1263-1289.
World Bank (1998) World Business Environment Survey.
World Bank (1995) Bureaucrats in Business: The Economics and Politics of Government Ownership, A
World Bank Policy Research Report, New York: Oxford University Press.
Xu, Lixin Colin (1998) "Control, Incentives, and Competition: The Impact of Reform on Chinese State-
Owned Enterprises," mimeo., the World Bank.
Zaller, John and Stanley Feldman (1992) "A simple theory of the survey response: Answering questions
versus revealing preferences." American Journal of Political Science. Vol 36 No 3, August, pp.
579-616.
42
Table 1. WORLD BANK ENTERPRISE SURVEYS - A PARTIAL SUMMARY
Title Country and Year Contact Description
Asian Corporate Indonesia, Korea, M. - Reaction to the crisis:
Recovery Malaysia, Hal]ward- - production, infrastructure, R&D, corporate finance, banking
Philippines, Driemeier - information on access to capital and finance
Thailand - information on subcontractors
1998 - data on output, domestic sales, exports, inputs, capital stocks and
invest., HR, financial structure
Industrial Thailand D. Dollar - production, infrastructure, R&D, corporate finance, banking
competitiveness (about 150 firms) - information on access to capital and finance
study 1997-98 - information on subcontractors
- data on output, domestic sales, exports, inputs, capital stocks and
investment, human resources, financial structure
RPED Many African T. Biggs - production, infrastructure, R&D, Entrepreneurship, banking
Countries - information on access to capital and finance
(about 200 firms in - information on technology
each country) - data on output, domestic sales, exports, inputs, capital stocks and
1993 - 1995 investment, human resources, financial structure
- survey on the workers of a sub-sample of firms
Private service Russia - St. M. de Melo - owner and firm profile, capital, inputs, finance, production (change in
firms Petersburg the past 3 or 9 months)
(86 firms) - membership in association
January 1993; - use of informal means to obtain financing and inputs
October 1993
Enterprise Poland Belka, - production, market structure and competition, labor, finance,
adjustment (200 firms) Estrin, investment and technology, ownership.
End 1993 Schaffer, - methods used to control level of receivable
Singh
The emergence of Former Czech and Leila - general information (entrepreneurs and firm profile, labor, capital,
private sector Slovak Federal Webster product markets), finance, inputs, production (change in the past 3
manufacturing Republic months), govemment policies and regulations,
(121 firms) - membership in association
January 1992
The emergence of Poland Leila - general information (entrepreneurs and firm profile, labor, capital,
private sector (93 firms) Webster product markets), finance, inputs, production (change in the past 3
manufacturing May 1991 months), government policies and regulations,
- membership in association
The emergence of Hungary Leila - general information (entrepreneurs and firm profile, labor, capital,
private sector (106 firms) Webster product markets), finance, inputs, production (change in the past 3
manufacturing 09/91 months), government policies and regulations,
- membership in association
Reform of state- China Eliana - losses in state-owned enterprises, employment
owned enterprises (156 firms) Cardoso - data on subsidies (also implicit) and safety net
03/95 - general information, output, sales, financing
Industrial Russia Qimiao Fan - general information, market structure, costs and profits, employment,
enterprises and (439 firms) finance (quantitative),
adjustment mid-94 - data on federal subsidies and preferential credit
- data on industry association
- output
Corporate Poland, Hungary, Cheryl Gray - ownership, privatization, managers, employees, company boards, day-
governance Czech Rep. to-day operations, market environment, products, capital,
(535 firms) - quantitative data (1990-94) on employment, ownership, costs and
End-1994 inputs, revenues
Enterprise and Poland Cheryl Gray - general information, financial distress & dependency on banks, change
bank (139 firms) in sales (91-94), arrears, conciliation, bankruptcy, liquidation, debt
restructuring repayment
43
Small and Sri Lanka M. Penalver - general information, entrepreneur, sales, profits, product mix, output
medium industry (300 firmns) and market shares, labor, capital, liabilities, debt
impact evaluation 10/1995 - sources of information
Small scale Czech Rep. Jea So - general information, privatization and transfer,
privatization Hungary, Poland - changes in supplier relationships
study (300 firms) - information on preferential credit ancd subsidies
12/92
Private Russia and Ukraine E. Rueda- - general information, financing, change in volume, credit availability
Entrepreneurs (300 Firms) Sabater and cost
04/92 - information on outlets (state channels vs. private channels of
distribution)
Economic and Russia R. - general information, contracts and judicial system, change in
civil society 02/97 Ryterman production, stockholders,
- membership in associations and financial and industrial groups,
associations as source of information, dispute resolution, legal
relations, customer relations, supplie:r relations.
Survey of private Lithuania A. Stone - general information, procurement and sales, entry, operations,
entrepreneurs (200 firms) infrastructure, technology and business services, growth and expansion
10/94
Large scale Mongolia J. Anderson - general information, ownership, shareholders, boards, arrears
privatization in (250 firms) - quantitative data on output, investment, credit, employment, sales,
Mongolia 06/96 inputs.
44
Policy Research Working Paper Series
Contact
Title Author Date for paper
WPS2289 Exchange Rate Overvaiuation and Howard J. Shatz February 2000 L. Tabada
and Trade Protection: Lessons David G. Tarr 36896
from Experience
WPS2290 Decentralization and Corruption: Raymond Fisman February 2000 E. Khine
Evidence across Countries Roberta Gatti 37471
WPS2291 Incentives for Pollution Control: Jerome Foulon February 2000 Y. D'Souza
Regulation andPublic Disclosure Paul Lanoie 31449
Benoit Laplante
WPS2292 Dividing the Spoils: Pensions, Ethan B. Kapstein March 2000 P. Sader
Privatization, and Reform Branko Milanovic 33902
in Russia's Transition
WPS2293 Should Capital Flows be Regulated? Roumeen Islam March 2000 R. Islam
A Look at the Issues and Policies 32628
WPS2294 Reforming the Urban Water System Mary M. Shirley March 2000 P. Sintim-Aboagye
in Santiago, Chile L. Colin Xu 38526
Ana Maria Zuluaga
WPS2295 Resolvirng Bank Failures in Argentina: Augusto de la Torre March 2000 M. De Loayza
Recent Developments and Issues 38902
WPS2296 An Ecological and Historical Yujiro Hayami March 2000 P. Kokila
Perspective on Agricultural 33716
Development in Southeast Asia
WPS2297 Sources of Ethnic Inequality Dominique van de Walle March 200C H. Sladovich
in Vietnam Dileni Gunewardena 37698
WPS2298 Fiscal Deficits, Monetary Reform, Nina Budina March 200C N. Budina
and Inflation Stabilization in Romania Sweder van Wijnbergen 82045
WPS2299 Household Savings in Transition Cevdet Denizer March 2000 A. Cubukcu
Economies Holger C. Wolf 38449
Yvonne Ying
WPS2300 Single Mothers in Russia: Household Michael Lokshin March 200C P. Sader
Strategies for Coping with Poverty Kathleen Mullan Harris 33902
Barry Popkin
WPS2301 Identifying Welfare Effects from Martin Ravallion March 200C P. Sader
Subjective Questions Michael Lokshin 33902
Policy Research Working Paper Series
Contact
Title Author Date for paper
WPS2302 Why Liberalization Alone Has Not Klaus Deininger March 2000 M. Fernandez
Improved Agricultural Productivity Pedro Olinto 33766
in Zambia: The Role of Asset
Ownership and Working Capital
Constraints
WPS2303 Malaria and Growth F. Desmond McCarthy March 2000 H. Sladovich
Holger Wolf 37698
Yi Wu
WPS2304 Disinflation and the Supply Side Pierre-Richard Agenor March 2000 T. Loftus
Lodovico Pizzati 36317
WPS2305 The Impact of Banking Crises on Maria Soledad Martinez March 2000 A. Yaptemco
Money Demand and Price Stability Peria 31823
WPS2306 International Contagion: Implications Roberto Chang March 2000 E. Mekhova
for Policy Giovanni Majnoni 85984