Last year, Richard Laermer decided to let his employees work from home on a regular basis. “We hire adults, they shouldn’t be tied to the office five days a week,” said Laermer, who owns a New York-based public relations firm. “I always assumed that you can get your work done anywhere, as long as you actually get it done.”

Turns out, he was wrong.

Employees took advantage of the perk, Laermer said. One was unavailable for hours at a time. Another wouldn’t communicate with co-workers all day, which Laermer found suspicious. The last straw, he said, was when someone refused to come in for a meeting because she had plans to go to the Hamptons. “That was the most unbelievably nervy thing I’d heard in years,” he said.

Ten months in, he scrapped the benefit and now requires all of his employees to come into the office every day.

While telecommuting, the umbrella term for any work occurring outside the traditional office, has ballooned over the last 20 years, some offices are rethinking overly broad policies. Flexible work remains popular at many organizations, but most companies want workers at work at least some—if not most—of the time. More than 60 percent of organizations surveyed by the Society of Human Resource Management this year said they allow some type of telecommuting, up from 20 percent in 1996. But telecommuting comes in many flavors, and 77 percent of organizations don’t let people work from home on a full-time basis. Most employers allow ad-hoc remote work for the person who needs to stay home for the plumber or wait for a package.

Technology such as chat programs and collaboration software made remote work feasible for many white collar workers in the last couple of decades. Employees love flexibility, often rating it high on benefits surveys. Parents in particular say it’s “extremely important,” a 2013 Pew survey found. Researchers have argued that unconventional work hours could even help close the pay gap.

In a bid to attract and retain employees—and cut down on real estate costs—companies permitted more remote work, and employees took advantage.

At the same time, work has also become more team-based. Only 38 percent of companies are “functionally” organized today with workers grouped together by job type, a 2016 Deloitte survey found. Most comprise collaborative groups that shift depending on the work. Deloitte found that one California organization was made up of over 30,000 constantly shifting teams. “I think that’s why we’re seeing remote work come back in,” said Erica Volini, a U.S. Human Capital Leader at Deloitte. “In order to work in teams, you need a higher level of collaboration.”

Some organizations found the most lenient work-from-home policies kept workers too isolated for that kind of work. These companies “took it to the extreme on virtual work,” added Volini.

International Business Machines Corp. is one such company. Earlier this year the tech giant told 2,000 U.S. workers they could no longer work from home and about the same number of employees that they had to commute into offices more often. Facing 20 consecutive quarters of falling revenue, IBM hopes that bringing people back together will lead to faster, more productive, and more creative workers. (A 2012 report from IBM found (PDF) that companies with flexible work policies reported improvements in productivity and cost savings.)

“IBM’s strategy is about adopting the best work method for the work being done,” said an IBM spokesperson. “For example, small, multi-disciplinary teams of engineers, coders, project managers and designers work in close proximity, often directly with clients or end-users, continually generating and refining ideas.”

One of the challenges with ending remote work is keeping employees happy. “It’s going to require organizations to think about how to still provide flexibility for their workforce,” said Deloitte’s Volini. Companies removing the perk risk employee backlash and attrition. IBM, for instance, has been careful not to eliminate all flexible work arrangements. The company still offers ad-hoc work-from-home arrangements to accommodate appointments and child-care needs.

Having everyone in the office has had “quite a positive impact” on business, said Laermer. Meetings are more productive, and employee morale has improved, he said. Laermer claims that employees “didn’t mind” losing their freedom. “Youngish people need structure,” he said, adding that they have small apartments and might not have a good place to work, anyway. He still offers flex time and lets workers leave at 3:00 p.m. on Fridays.

“I think people have to be trusted,” Laermer said. “But the working-from-home thing has to be on a per-person basis, and it can’t be very often. It just doesn’t work.”

(Corrects the number of IBM employees affected by the policy change in the tenth paragraph.)

Disclaimer: The opinions, beliefs and viewpoints expressed by the various author(s), publisher(s) and forum participant(s) on this web site do not necessarily reflect the opinions, beliefs and viewpoints of the @SmitaNairJain or official policies of the #SmitaNairJain

TEDx Talksr | Break The Shackles To Climb The Ladder | Smita Nair Jain | TEDxIIMRaipur | “Women empowerment for me is the power and freedom to choose my own opportunities and enhancing my capabilities to do what I really want to do.” She is one of the new generation management gurus who throws new light on corporate behavior and particularly its global aspect. She is also one of the best keynote speakers of all times. This talk was given at a TEDx event using the TED conference format but independently organized by a local community. Learn more at http://ted.com/tedx