Measuring the Efficacy of the World’s Managers

Over the past seven years, Harvard Business School's Raffaella Sadun and a team of researchers have interviewed managers at some 10,000 organizations in 20 countries. The goal: to determine how and why management practices differ vastly in style and quality not only across nations, but also across various organizations and industries.

by Carmen Nobel

Firms in the United States, Japan, and Germany tend to be managed especially well, while firms in Brazil, China, and India tend to be managed poorly.

Those are among the initial findings of the World Management Survey (WMS), a huge international research initiative to measure differences in organizational management practices all over the world.

The project was borne of a widely perceived gap in economic research. In business academia, there is an optimistic tendency to assume that managers generally make decisions in the best interest of their firms' performance. Reality is more frustrating. The fact is, some companies are managed beautifully, others dismally.

"During the interview she got a marriage proposal. The manager wanted her to marry his son."

"It's very tough to believe that there are such wide differences in management out there," says Raffaella Sadun, an assistant professor at Harvard Business School who coleads the WMS with Nicholas Bloom of Stanford University and John Van Reenen of the London School of Economics. "If you really want to be convincing, it's important to have large-scale statistical evidence," she adds.

To that end, over the past seven years, large teams of affiliated WMS analysts have interviewed managers at some 10,000 organizations in 20 countries, setting out to determine how and why management practices differ vastly in style and quality.

Best practices

The researchers used an evaluation tool designed by a leading consultancy firm that broadly rated management practices in three areas: monitoring—how well managers keep track of what's happening in a firm and make good use of that information; targets—how well organizations set appropriate goals and outcomes, and whether they take action if the two are inconsistent; and incentives—whether organizations promoted and rewarded employees based on performance and tried to keep the best performers from quitting.

To collect the data, the researchers hired teams of MBA students who could interview managers in their respective native languages. The respondents chosen for the survey included primarily middle managers in manufacturing plants, although over time the data collection was extended to other industries, such as retail, schools, and hospitals. The approach was to interview those who were high enough on the corporate totem pole to have a good overview of management practices, but low enough that they were familiar with day-to-day operations. The researchers also chose to target primarily small and medium-sized firms, employing between 100 and 5,000 workers, to maximize the chances that the interview would capture salient features of the whole organization, as opposed to the characteristics of single plants.

Rather than limiting the responses with multiple choice or yes/no questions, the interviewers kept the questions open-ended to get a complete picture of the actual practices adopted in the organization. This method was also useful to build an effective bond between the interviewer and the manager, which sometimes led to answers that were both thoughtful and entertaining. (Responding to a question about staff retention, for instance, one manager said, "I spend most of my time walking around cuddling and encouraging people. My staff tells me that I give great hugs.")

"We had very interesting experiences," Sadun recalls. "Once, a student was interviewing a manager, and during the interview she got a marriage proposal. The manager wanted her to marry his son."

Key findings

The researchers have summarized the initial results of the study in a new working paper, Management Practices across Firms and Countries,—also coauthored by Christos Genakos of the Athens University of Economics and Business—and have posted a comprehensive collection of the survey data on the World Management Survey website. The WMS site also features individual policy reports for manufacturing, education, health-care, and retail organizations, providing a management benchmark for executives in any of those fields.

On average, based on the evaluation tool, the research showed that firms in the United States, Japan, and Germany tend to be the best-managed in the world, while firms in Brazil, China, and India scored poorly.

"You have a lot of companies where the transmission of leadership is based on criteria completely unrelated to your ability to lead."

Of the 20 countries in the survey, the United States received the highest overall management scores in retail, health care, and manufacturing. But US schools scored comparatively lower, coming in fourth behind the United Kingdom, Sweden, and Canada.

In the United States, India, and China, managerial use of incentives are much more common than the use of monitoring and target-setting, especially in the manufacturing field. In Japan, Sweden, and Germany, monitoring and target-setting far exceed the use of incentives.

But the differences surpassed international boundaries. "There is so much dispersion, even within the same industries and same countries," Sadun says, explaining that the researchers managed to discover certain ownership patterns that help to explain the dispersion.

For instance, government-owned organizations tend to receive low management scores across all the sectors and countries in the study. "They are particularly weak at incentives," the paper explains. "Promotion is more likely to be based on tenure (rather than performance), and persistent low-performers are much less likely to be retrained or moved."

Family-owned businesses scored even lower—particularly those run by a firstborn son who inherited the role of CEO. Family-owned businesses that employed a non-related CEO scored much higher. "The finding there is not so much that family ownership per se is associated with lower scores, but rather family ownership when the selection of the CEO is not meritocratic," Sadun says. "Especially in Europe, you have a lot of companies where the transmission of leadership is based on criteria completely unrelated to your ability to lead."

Also receiving low management scores across the board: organizations at which the company founder was also the CEO. "We systematically find that founders have lower levels of scores," Sadun says. "One possible explanation is that founders are great at the start-up phase because they have the vision and the motivation. But as the firm grows, to do the ongoing management on a daily basis requires a different set of skills."

Market competition appears to be good for management. The research showed that a company's overall management score was directly related to how many competitors a firm faces—the higher number of reported competitors, the higher the management score. And multinational firms scored higher than their domestic counterparts across all the industries and countries in the survey.

Education makes a difference, too. The data showed that the greater the percentage of employees with a college degree in manufacturing and retail firms, the greater the management scores. In hospitals, a greater management score correlated with the percentage of managers who also had clinical experience.

Next steps

With the initial data tracked and scored, the researchers are now working with the US Census Bureau to build an even larger set of management data. Last spring they conducted a survey of more than 48,000 US companies, using the questions and the evaluation tool from the initial WMS survey. The Census Bureau already keeps general track of firms' financial performance in the United States. The WMS team plans to match the management survey data against the financial performance data in order to investigate the link between management quality and the bottom line across an even larger number of organizations.

The researchers have also started to work on management experiments, modeled on the randomized control trials adopted in the medical field. This experimental approach is much more costly and labor- intensive than the survey methodology, Sadun says, but it allows them to identify with much greater precision the causal link between management and performance.

"We want to get at the causality between management and performance," she says. "And to do that, you have to go beyond surveys and start running experiments with firms. That's the most challenging but also the most interesting part of where this research is going."

About the author

Carmen Nobel is senior editor of Harvard Business School Working Knowledge.

Comments

Anonymous

I think that managers should have all the facts in front of them before setting any strategy and it should be different ones according to the type of their business.

In your article you started with Monitoring and then goals setting where it should be reversed.

I am working on a revolutionary theory of management called Roots, Trunk and Fruits where it copy's God's way of management.
If you are interested then please let me know.

Anonymous

Interesting!
But what's the value of this research?

jack slavinski

SVP

Great area of research. I have often found that within many organizations, the expectations for managers (at all levels) are not conducive to enabling them to be successful leaders. Basic example: if an organization puts tremendous emphasis on senior leaders personally managing projects without ALSO ensuring they truly lead, coach, mentor, optimize their organizations, then those leaders might not be set up for success. Not intended to be a sweeping statement, but leaders who might be great a "managing a task or a project" might not be so effective at leading an organization through major change.

Gene C Walker

retired (HBS MBA 1960)

For 25 years I advised and sometimes ran small companies that were experiencing difficulties. Some were
owned by groups of investors and others were closely
family controlled. It was generally easier to institute
needed changes in the investor owned companies.
First and second generation controlled companies were
problematic since owners did not want to replace
themselves or allow someone from outside make the
decisions. Selling the company was usually the best
option - if a buyer and realistic price could agreed to.

Anonymous

In my organisation, a state energy player, as with many other local Caribbean nationalised corporations, the political protectionism, the low incidence of meritocratic incentivisation, the subtle but deeply-seated ethnic divisions seem to proliferate managers who naturally are poor. Formal critical thinking training and increased and objective monitoring and leadership training & reward within a projectised environment can help reverse the unfortunate natural trend. We are not "learning organizations" by nature as we may know what to know but implement those actions poorly.

TA

Anonymous

Similar to one of the previous comment, this article is very interesting, but I don't see a tangible benefit of this at this point. Perhaps with further research with metrics like performance, financial results, etc, this could be more comprehensive.

I for one had gone through the article with interest, since I happen to be involved in similar activity for the last four decades or so - at a very modest and limited level. Specifically, the aim has had been propagation and inculcation of of human values in management education and profession. It is in this context that the last paragraph of the article on the need for going beyond surveys and start running experiments with firms has attracted particulary my attention.
Spandan (Heartbeat) approcah is based upon the innate divinity in human beings. With reference to management of organsiations, faith in the basic goodness of others is sought to be the credo of an effective management. A 3D approcah consisting of Diagnosis, Discovery and Development for evolving- what is called as- Functionally Humane Organisation is experimented upon by the given organisation.
Spandan (Heartbeat) philosophy of management is an odyssey and demands going beyond one's own professional and organisational interests and becoming instrumental for the growth of the society and the world.
G.P.Rao, Founder Chairman, Spandan, Foundation for Human Values in Management and Society, India.

Hemant Ghatay

Club Millesime Manager, Sofitel Luxury Hotels

I am from an hospitality industry background in India. I agree that Monitoring is an important aspect from management point of view but your Vision,Mission,Objectives,Goals and then there is monitoring.If we as manager does not know how to lead the teams by inculcating the company values in the team,it is very difficult to judge the efficacy of that firms managers.

K Ravindran

Faculty - Business Studies, Edgewater College

The very basic of excellence in management resides in how emotionally one connect and motivates his people within the organization and outside. It is this principle concern on people lead to interest and close involvement in their job leading to improved quality and greater productivity. Managers, barring a few, in the developing world are yet to be enlightend on the human aspects in anthropology, sociology and psychology, which are fundamental to connect with people. Therefore, in this parts of world, it is still evolving and will take sometime to get embedded in the system. Another area that can be explored in this regard is the culture and value system practiced in those countries. Unless there is competition at hand, managers will lack the spirit of efficency and excellence.

Prof. K Ravindran

Anonymous

I agree with opinion which said human spirit has value to correlate with their performance, hence organization in a broader way. Essentially, faith of the people is the interest of study. However this area is so hard to be quantified and thus managers find difficult to manage. I think this stage is most dynamic elements of all parts to manage.

Paul Nicholas

Director, Soul-Chaplain Consultancy

This is a really interesting and thought provoking item - thank you.
I agree with Professor Ravindran's point that central to effective leadership (and management, unless we are talking about automated management systems) are effective relationships. At the heart of any leadership or management decision is the relationship between at least two people.

Didn't Colin Powell warn that we should fit no stereotypes, chase no management fads, but let the situation determine the approach adopted? This means that flexibility becomes a major determinant of success, and culutres/organizations that engender this in their leaders and managers - particularly with respect to human relationships - will benefit.

Kapil Kumar Sopory

Company Secretary, SMEC(India) Private Limited

Efficacy of world managers varies not only from country to country but also from firm to firm within any country.
Managers are guided (or not guided) by their working environment which includes the systems and procedures laid downand the tenacity(or laxity) with which these are followed and monitored. There is hardly any entity which has not set noteworthy plans of action which, if given adequate attention, would lead to efficacy of desired level. However, though the guidelines,etc. are very well documented, the implementation at the ground level leaves a lot to be desired.
Side by side, role of regulators is also critical. Where laws are not observed in letter and spirit, corrupt practices develop impacting the organisational health in the long run.
In my view, a technically advanced country would generally have an edge over those where outmoded working methods go on and investment on betterment is given a back-seat.
Innovation, Research and Development also lead to improvements. Countries/organisations low on this have naturally to get left behind.
Last but not the least is the attention given to HR from all angles. That manpower is the most important asset is professed but not displayed time and again.

Anonymous

Pity there were no empirical results shared, for example I would be interested in the statistical variations that back up the suggestions that one country was more or less well managed. It would be worth analyzing the data for bias by interviewer. I would suggest that the variation of student interviewer scoring might be more erratic than the normal geographical statistical variation. Of a study of this size I should imagine we could learn just as much about how interviews should be conducted, as of the content of the results.

Too bad no one spell checked the article before it wass released [sic]. Perhaps it was sent by Blackberry.

Also in reference to one of the comments, "Roots Trunk and Fruits", I don't see any evidence of a God managing anything. In the early parts of the Old Testament, I thought he did a particularly bad job, a petty, unjust, racist, unforgiving control-freak. I would be very concerned if my line manager put the fear of god into me and threatened to smite me down. Perhaps we best steer clear of mixing religion and corporate management techniques.

Thanks for your rapid response, I take back "most" of what I said. I neglected to read the associated links you had already provided. I will enthusiastically browse the detail now!

Kaitki Agarwal

Great article. I feel it will be good to add data from high tech industry as well. I think there is a lack of coaching managers in all the industries. Managers do not get enough time to reflect, lead, motivate and coach who are reporting to them and they do not get coaching from upper management. There are many good project managers but not many good people managers. I wonder how fair is the reward system, Politics may come into play when rewarding people. This may be an area for further research.

VCR Kumar

Business Consultant, KamRam Enterprises

managers are mostly damagers, that is what my wife calls them at her workplace. some maintain what is given, very few inspire and lead.

Atul Guglani

Director, Mantex Technologies

Efficacay has to be measured out of the targeted deliverables, value creation and Innovation of the managers and not out of Supervisory rolls of Monitoring, Target setting and Incentives.
It is these coordinates that have created a very mediocre organization, which delivers feed for an average standard organization.

Though interestingly the results of the research are much in line with what we see around us.

However, the research does not highlight , how Monitoring, Target Setting and Incentive practices were generally seen and practised across the countries factoring the cultural , social, government and economic variables and the consequent thruput diffirences. Eg A manager in Asia or Africa would work with very low resources and hence targets and monitoring then becomes ineffective tools of measure as the resources will impede the organization in all its efforts.

I have personally seen this in many a countries like Sri lanka, India, Bangladesh, Indonesia and China, where Target Setting largely fails , becasue of resource contraints and monitoring is only about routine supervision. Incentives do not exist at all.

Second, the attitude of people, arising out of culture and social differences is another major detterent factor, which inhibits the efficacy of managers.

But, the efficacy of a manager was to be measured out of deliverbales like number of goals achieved, new systems designed, product value engineered , system reconfiguration, etc etc, then perhpas, it would show the ingenunity and efficacy of managers across the sphere. Now it measures the standard average performance of managers, which also good to know.

Nevertheless very interesting results.

Anonymous

I see one findamental issue. Judging what is good and bad practice is really how it fits to a predefined understanding of what is good management practices.
If the understanding is American style MBA school, no wonder American companies rate the best. But is it real ? In an Indian or European family owned business managed by the founder, for instance, long term objectives are rated higher than in US style companies who need to satisfy investors confidence on quarterly basis and focus their targets on short term profits which may put long term survival at risk. Obviously, different management styles with different tools are required.