In that ruling, Lester wrote that Zimmerman was "flouting" and "manipulating" the legal system.

In his order, Lester expressed concerned that Zimmerman might be hatching a plan to flee the country to avoid prosecution, something Zimmerman's defense strongly denies.

An appeals court ruled 2-1 in Zimmerman's favor with the dissenting judge saying Lester had not "crossed the line" and should not be removed.

The two judges in the majority admitted in their opinion that this was "a close call," but given the allegations Zimmerman presented in total, it did meet the threshold of legal sufficiency to have Lester step down.

Zimmerman's attorney, Mark O'Mara, released a statement after the decision.

"We appreciate the appellate court reviewing our petition, and we believe it is the correct decision," read the statement.

Lester had originally ruled that he would not step down from the case.

Zimmerman is charged in the shooting death of Trayvon Martin on Feb. 26.