OAKLAND (CBS SF/AP) — In a surprise move, a judge on Friday rejected a plea agreement for Ghost Ship warehouse master tenant Derick Almena and creative director Max Harris for the fire at the warehouse in Oakland’s Fruitvale district in December 2016 that killed 36 people.

As he handed down his decision, Judge James Cramer said Almena didn’t accept “full responsibility and remorse” for the fatal blaze which occurred during an unlicensed concert at the dilapidated Oakland warehouse known as the “Ghost Ship.”

The plea deal had called for Almena to be sentenced to nine years in prison and 28-year-old Max Harris to six years. The judge said he found Harris to be sincere but because the plea bargain was for both Harris and Almena, both pleas were rejected.

In court on Friday, Harris apologized to the families of the victims for his actions but told them he didn’t expect forgiveness.

“I know nothing I can say will come close. I’m sorry,” he said. “You’re in my prayers and will be for the rest of my life.”

The men could have been released from prison after serving just half their sentences, which is typical for California inmates who keep a good disciplinary record. Relatives of victims who died in the blaze had slammed the proposed sentences as too lenient.

Cyrus Hoda, the brother of victim Sarah Hoda, said Almena and Harris were “culture vultures” trying to become San Francisco Bay Area arts players by luring people to a dangerous place to live and party.

Authorities have alleged Almena rented the warehouse and illegally converted it into an entertainment venue and residences before the fire. Harris was accused of helping him collect rent and schedule concerts.

Prosecutors said the two men had turned the warehouse into a residential “death trap” by cluttering it with highly flammable knick-knacks, blocking the building’s few exits and failing to make adequate safety precautions before inviting the public inside.

Almena’s attorney, Tony Serra, said the judge’s decision Friday came as a complete surprise as Cramer had approved the plea deal last month.

On Thursday, Paul Matiasic, an attorney reading a statement by a father of one of the victim’s, asked the judge rhetorically if he had ever lost a child.

“Yes, I have,” Cramer said with a sad smile. “Just so you know.”

Many people in the courtroom gasped. The judge did not elaborate.

Almena’s wife and three children also lived in the warehouse but were staying in a nearby hotel the night of the fire. His wife, Micah Allison, and one of their daughters sat quietly in court alongside the grieving families on Thursday and both addressed the judge Friday.

Almena told the victims’ families that he should have died in the fire and that he was “guilty for believing we were safe.”

“Forgive me if you can,” he said. “If I could give you my children’s lives I would.”

Almena and Harris could face life in prison if convicted at a trial. They already have spent a year behind bars.

Serra said he believed the case had “weaknesses” and hopes prosecutors will consider another plea offer. A trial date has not been set.

Investigators from the U.S. Bureau of Alcohol, Tobacco, Firearms and Explosives said they were unable to determine a cause of the blaze.

Victims’ relatives allege in lawsuits that the Oakland Fire Department failed to inspect the warehouse annually as required and that inspectors would have discovered the illegal conversions.

Alex Katz, a spokesman for the city attorney, declined to comment.

The lawsuits also claim Pacific Gas & Electric Co. failed to properly monitor, inspect and repair electrical equipment that provided power to the warehouse.

PG&E said in a statement that it cooperated with the investigation and that a review of its records found no electrical problems at the building in the 10 years before the fire.

Warehouse owner Chor Ng, who has not been charged, did not return a phone message seeking comment on Thursday.