The new leader represents a "kiss of death wrapped in a smile" for the retailer, which will likely go private soon, Brian Sozzi, chief equities strategist at Belus Capital Advisors told us.

He said that Ullman, who is known for his aggressive promotions, will set the company up for poor profits after it burned through available cash on its turnaround strategy, Sozzi said.

Especially expensive were Johnson's shop-in-shops, which showcased certain brands in boutique-style formats in the stores.

"JCPenney has already plowed tons of shareholders funds into shop-in-shops, so an environment of Ullman’s 1980s promotional strategy sets the stage for poor returns on that investment right out of the gate," Sozzi said.

The timing of Johnson's departure is "telling," Sozzi said.

"You're already worried about what's going on there, you have new strategies that are in motion, you're still remodeling stores, and employee morale is even worse than ever," Sozzi said. "If they intended to stay public, it would make more sense to at least wait for Johnson to finish out parts of his strategy."

It's possible that JCPenney is going to work things out "behind the scenes."

Firms such as Apollo Global Management LLC and Leonard Green & Partners LP are considering a potential buyout of JCPenney, reported Richard Collings at The Deal.

The company will likely hire a permanent CEO after the holiday season, Sozzi said.