Organisations are filtering out top female talent at all levels, says Institute of Leadership and Management. Photograph: Johannes Eisele/AFP/Getty Images

Banking's "macho culture" is the biggest barrier to women reaching the top, according to a report published arguing that quotas for women executives are not the solution for Britain's unbalanced boardrooms.

The Institute of Leadership and Management warned that organisations are "filtering out" top female talent at all levels.

Charles Elvin, its chief executive, said enforced quotas to increase the number of women executives are "patronising" and unpopular with both sexes. Instead he called for changes to attitudes and management processes, more flexible working, and a genuine meritocracy.

The ILM's survey of 800 men and women found that nearly half of women working in banking believe they face career barriers because of their gender, and more than one-third of men agree. Nearly three-quarters of women, and half of men, blamed the attitudes of senior male managers. The greater number of men in senior roles, organisational culture, lack of flexible working opportunities and shortage of female role models were seen as the other main barriers to women's career progression.

The ILM survey is published a day before the Cranfield School of Management releases its annual Female FTSE report, on the number of female executives on the boards of Britain's top 100 companies. It will show whether the City will hit the government's goal for 25% of boardroom directors to be female by 2015.

Research from the recruitment firm Korn/Ferry Whitehead Mann found that female representation on the management board, which sits just below the main company board, actually fell by 2.2% last year, despite an increase at the most senior level from 12.5% to 15%. It said 17 FTSE 100 companies have no female executives on the management board, including Associated British Foods, WM Morrison, Shell, Tate & Lyle and Schroders.

Rather than a single "glass ceiling", the ILM sees a series of barriers that filter out female talent at each stage of the management hierarchy, mainly due to the attitudes and behaviour of senior managers.

Elvin spoke of a form of brain drain. "It is in danger of being really quite a dreadful waste if you are losing very, very competent people all the way through the process, several steps before the board," he said. "It is bad for the organisation, bad for the country and bad for society."

Overall, 87% of women felt that some form of positive action was needed, compared to two-thirds of men. Flexible working was identified as the number one solution by 68% of women, compared to 42% of men. Just 19% of women and 10% of men felt quotas should be introduced to increase the number of women in senior roles.

Elvin said the government should lead the way by training the managers in government-funded organisations to "manage flexibly" across both genders – focused on "achieving objectives rather than sitting in your seat" – and to make sure employees are hired and retained on the basis of merit. Flexible working should apply to both men and women. "It is completely inappropriate for people to look at a person of child-bearing age in a different way," he added. Women tended to be less confident, partly due to a lack of female role models, and needed more support, he added.