This copy is for your personal, non-commercial use only. To order presentation-ready copies for distribution to your colleagues, clients or customers, click the "Reprints" link at the bottom of any article.

A new book explains why so many women must struggle with themselves to claim success—and suggests how to achieve it on their own terms

Women sometimes set themselves up to fail when it comes to money and confidence. (Illustration: Traci Daberko)

In recent years, banks, brokerage firms and fund companies have pulled out all the stops to make women more confident about money. Prompted by surveys showing women's feelings of helplessness around money management, discomfort with investing and greater risk aversion, the financial industry has sprouted women's initiatives, hosted seminars and published educational brochures by the ream. We joined them, publishing a book that debunked money myths holding many women hostage to insecurity.

All to no avail. Women's self-reported financial literacy gap is not shrinking and may even be growing, according to research by Prudential and Financial Finesse.

For the two of us—one with decades of clinical experience as a psychotherapist and an observer of gender differences, the other with a long track record of communicating financial information—this news was disheartening. For heaven's sake, Janet Yellen is chair of the Federal Reserve. Former French Finance Minister Christine Lagarde heads the IMF, and Angela Merkel runs Europe's most successful economy. Clearly, women don't lack a gene for finance. Why do we as a gender come across as so inept?

Finally we have some answers. ABC News correspondent Claire Shipman and “BBC World News America” anchor Katty Kay recently published “The Confidence Code: The Science and Art of Self-Assurance—What Women Should Know,” which assesses findings and observations to make sense of a persistent “confidence gap” between women and men.

‘Feeling Like A Fraud’

Shipman's and Kay's curiosity was piqued during their interviews with successful women in business. “We would hear repeatedly that they felt like frauds,” Kay told us. “Or that they got where they were because they were just lucky or in the right place at the right time. We never heard this from men.”

This became the first clue that a confidence gap wasn't just an issue for women who struggled in high-school math. “We realized that lack of confidence is holding many women back from getting where they want to go in the work world,” Kay said.

For instance, she and Shipman cited the findings of Carnegie Mellon economics professor Linda Babcock, who determined that men initiate salary negotiations four times as often as women do. When women do negotiate, they ask for 30% less than men do. Another study at Manchester Business School in the U.K. found that male business students thought they deserved to earn $80,000 a year, while women thought they deserved only $60,000. Shipman sums it up bluntly, “Women effectively believe they are 25% less valuable than men do.” (Can it be a coincidence, we wonder, that women earn an average of 23% less than men?)

Confidence Is a Verb

Men often demonstrate confidence in the workplace with high-decibel confrontation, quick decisions, a drive to win and a disregard of risk (or an appetite for it). This kind of behavior “is wholly unappealing and downright foreign to women,” Shipman said. “Most of us just aren't comfortable throwing our weight around in a conference room, interrupting others or blowing our own horn.”

In either women or men, confidence isn't a particular way of acting or the state of feeling good about yourself, said Shipman and Kay. One of the experts they consulted defined it as “purity of action produced by a mind free of doubt.” Another said, “Confidence is the belief you can accomplish the task you have in mind.”

Their eventual conclusion was concisely expressed by Ohio State University psychology professor Richard Petty: “Confidence is the stuff that turns thoughts into action.” Conversely, lack of confidence leads to inaction—no matter how smart or well-prepared one may be. This is the challenge many women need to overcome.

Sabotaging Their Own Competence

Women's low confidence hurts them not just in the workplace, but in other life achievements. The irony is that by hesitating to try something they’re unsure of, they may sabotage their own hard-earned competence.

An example reported by Kay and Shipman: When psychologist Zachary Estes of the University of Milan gave computerized spatial tests to men and women, the women scored measurably worse than the men. “But when he looked at the data,” explained Shipman, “he found that women were leaving some questions blank. When instructed to answer all the questions, women did as well as men.” In other words, she emphasized, “we need to push ourselves to answer all the questions, even when we’re not sure.”

Perplexing to Male Bosses

Most men don't understand the reasons behind women's low confidence, only that it keeps them from achieving their best. According to WNBA coach Mike Thibaut of the Washington Mystics, female basketball players’ tendency to dwell on past or potential failures and mistakes instead of focusing on the task at hand is their biggest psychological impediment to top performance.

Shipman and Kay noted the greater appeal to a boss of a go-getter employee who doesn't let setbacks deter him from popping in with unsolicited bright ideas, compared to another worker who does her job with quiet competence, recoils from criticism and speaks up only when she's sure her proposals are flawless.

Note that this example of confidence doesn't mean a woman has to turn into Attila the Hun. She just needs to obsess less about having to be, and feel, perfect.

Perfection: Enemy of the Good

Women's pursuit of perfection is the most crippling way they sabotage their own confidence. “We set the bar impossibly high, so we inevitably feel inadequate,” Shipman said. “And we avoid taking action until we feel we’re better informed, better prepared and better qualified than anyone else. We watch our male coworkers lean in, while we hold back until we believe we’re perfectly ready.”

We learned this ourselves some years ago when we surveyed financial confidence among different generations of Mount Holyoke College graduates. One of the issues we hoped to resolve was why even educated women were so unsure of their money mastery. One member of an investment club made up of MHC alumnae told us, “Women never claim their competence until they know everything, while men are willing to wing it.”

Even Christine Lagarde admitted to Kay and Shipman that she overprepares for everything. “[German chancellor] Angela Merkel and I have talked about it,” the International Monetary Fund managing director said. “We want to be completely on top of everything and we want to understand it all and we don't want to be fooled by somebody else.”

One of the major conclusions of Kay and Shipman's investigation—and perhaps their most striking finding—is that success correlates more closely with confidence than it does with competence. In other words, even a Ph.D. from the world's best business school won't get you far if you aren't confident.

Granted, women's greater cautiousness can be an advantage in the financial world. Studies have found that female hedge fund managers earned triple the return of their male counterparts over the past five years, while female investors often do better than men because they’re less susceptible to the thrill of frequent trading.

“Women's instinct for caution is valuable,” Shipman agreed. “But being able to make decisions and move on is critical to a manager. Women need to find a balance between our natural caution, and knowing when to make decisions quickly and move on.”

Wired for Overthinking

Women's lack of confidence is due to “a cocktail of lots of different things” besides perfectionism, Kay noted. One of the key ingredients is genetics-influenced brain chemistry. What she and Shipman call “a messy stew of hormones and neural activity” tends to slow down decision making in the female brain. In particular, while testosterone—the primary male hormone—promotes competitiveness, risk taking and demonstrations of power, women's larger supply of estrogen fosters social skills, bonding and avoidance of risk and conflict. Simply put, women are genetically programmed not to take chances that might create dissension or put them in danger.

A further complication is women's greater proportion of brain “white matter,” essential for integrating information. Men have more “gray matter,” useful for tackling isolated problems. Although we both have the same primitive fear centers (the two amygdalae), men's brains tend to activate the one associated with taking action while women rely more on the one involving thought processes. In essence, when a man is leaping forward to fight off a saber-toothed tiger, a woman may still be evaluating the options.

One other key difference reported by Shipman and Kay is that women's brains produce 52% less serotonin than men's do. The relative shortage of this hormone, crucial in calming the amygdalae, leads women to be more anxious and worry-prone, and (we speculate) consequently to seek support from therapies and medications that promote serotonin uptake.

‘A Critical Workplace Skill’

Along with the discovery that lack of confidence is partly genetic, Shipman and Kay were glad to find that it can be fixed. “There's a lot we can do about it, even late in life, to change our confidence level,” Shipman told us. “We think that the cornerstone for women is learning to risk and fail, and jettisoning our perfectionism.”

“The Confidence Code” gives practical advice about how women can build their confidence using tools from psychologists and neurologists. “A lot of it comes down to taking action,” said Kay. “Doing the thing that's a little bit hard for you; going outside your comfort zone”—in other words, “practicing the nonhabitual” as Mellan has advocated for years. (For more tips, see sidebar above.)

Kay continued, “One other thing we can do is to talk about this and recognize the value of developing confidence. One male law partner said to us that confidence is so important, it should be a formal part of the review process.”

“We’ve come to the conclusion that confidence is essential for women,” said Shipman. “We need to start thinking about it as a critical workplace skill.”