But parting ways with Charney, who is the top shareholder at his company, won't be easy, said Eric Beder, analyst at Brean Capital.

"It's an awkward situation to get rid of your CEO when he owns more than a quarter of the company," Beder told Business Insider. "The new CEO will have to sort out that situation."

Charney will still have considerable say in company decisions like hiring executives and board members until he sells his shares, Beder said.

When reached by phone, Charney declined to comment.

While the company hasn't commented on the allegations that led to Charney's departure, analysts don't believe it had to do with his performance.

Earlier this year, the company faced being delisted from the Nasdaq. Despite financial woes, Charney had begun to turn around the business, and fundamentals were solid, Beder said.

"If this were business-related, it would have happened sooner," Beder said.

It's also unlikely that Charney is leaving the company over money. From the beginning, he has insisted on making clothes in the United States, foregoing the profits he could have made by taking his business overseas.

Charney has faced repeated sexual harassment allegations, but the vast majority were found to be bogus.

Beder expects the details of Charney's firing to be made public in the coming weeks.

"I would be shocked if this doesn't go to some court, meaning we'll find out what happened," Beder said.

The company culture at American Apparel, which uses sexy marketing to sell high-quality basics made in Los Angeles, will probably remain the same, according to Beder.

"The company still has a big cult following despite everything its gone through, and it understands customers," Beder said. "The future is pretty bright."