An Orange County judge Friday removed the law firm representing Bruce Williams in lawsuits among members of the Champ Williams family.

Circuit Judge Lon Cornelius went along with a request by the lawyer for Bruce's relatives to disqualify Leon Handley's firm.

Handley represented Bruce and David King represents the other members of the Williams family of Orlando airport restaurateurs in four bitter lawsuits. King contended that during negotiations over a buyout of Bruce's stock in the family business in 1983, Handley improperly threatened that Bruce would tell federal authorities and The Orlando Sentinel that his relatives had committed crimes.

That makes Handley a key witness and disqualifies him, under ethical rules and appeal-court decisions, from also acting as a lawyer in the case, King argued. The judge agreed.

''You're kidding,'' Handley said when informed of the decision. ''To say it's a surprise is an understatement. I've only been practicing law 34 years, and in all that time I've never been fired by the other side.''

Handley, 57, said the decision whether to appeal is up to Bruce, who said he expects to do so.

Handley said he has no hard feelings toward King, who worked for his firm from 1968 to 1971. ''He's a splendid lawyer and he knows I'm not an extortion artist. For the life of me I don't know why he wants me out of the case.

''I'm not so vain as to think that a lawsuit is any different with me as counsel than without me. The truth will out.''

Bruce sued his relatives in September 1983, alleging that they engaged in a three-year campaign of harassment and slander against him. They countersued last month, accusing him of extortion.

Last year the other family members sued Bruce to recover $125,000 he took on his way out of the business. He countersued, contending that they looted the corporation.

King denied that the extortion allegation was made to engineer Handley's removal.