Authors of the report dispute that, saying documents they collected show that in 2011, the biggest proportion of Alianza's production was for JCPenney, ordered through a middle-man supplier.

Wal-Mart also said its business with Alianza was conducted through a middle-man supplier.

"Our relationships are with suppliers, and we paid them in full for all merchandise produced in that factory when it was in operation," said Wal-Mart spokesman Kevin Gardner. " As the world's largest retailer, we strive to positively influence global supply chain practices by raising our own standards and improving working conditions in the countries from which we source, and we expect our suppliers to adhere to our standards for suppliers and local wage laws. If we learn of a violation, we take appropriate action."

A spokesperson for Macy's said that a "relatively small amount of private good brands were manufactured for Macy's" at the factory "earlier this decade." Macy's became concerned about previous allegations against the company made by the Institute for Global Labour and Human Rights in 2010, but said the allegations were not confirmed at the time. The company said its supplier has not used to Alianza factory for production since October 2012.

Attempts to reach the owner of the now-shuttered Alianza factory were unsuccessful.

Guatemalan investigator Gabriel Zelada told ABC News that factory workers want to recover payment for their past work, and they want factory owners to pay into the social security system and pensions.

Labor rights groups attempted to share their findings with a number of major retail brands whose clothes were made in the factory. PVH, the clothing company that produces garments with the Tommy Hilfiger and Calvin Klein label, was one of only a handful that responded to them – donating $100,000 to the workers who lost their jobs when the factory closed. Those workers were each paid $125 from the fund this month, according to the report.

"The 59 other labels should follow the lead of PVH. Each should contribute $100,000 so that the Alianza workers can be paid at least the back wages and benefits they are owed and desperately need," the report says.