A posh Manhattan building has ordered all its employees to only “speak English” when at work – drawing the ire of workers who in some cases only speak Spanish.

London Terrace Gardens, located at 415-435 W. 23rd St. in Chelsea, issued a memo to its employees, ordering them to only “speak English in all public areas as well as on the [walkie-talkie] radio.”

Workers said yesterday that most of the building’s staff is Hispanic, and nearly all are upset.

“I think it’s a very bad policy,” said doorman George Guzman. “I can’t work without Spanish. How do we talk to delivery boys who only speak Spanish. I have to speak Spanish.”

The union representing the workers, SEIU Local 32BJ, said the policy is a violation of federal civil-rights laws.

But the owner of the pre-war luxury building insisted that the employees have missed the point of the Jan. 11 memo.

“It has been the longstanding policy of the building that employees speak English while in work-related areas, when in contact with residents and when using building-issued walkie-talkies,” Rose Associates said in a statement.

“The policy in no way precludes employees from speaking languages other than English when engaging in private conversations.”