Platinum-selling rapper Ja Rule left an upstate New York prison Thursday morning after serving most of his two-year sentence for illegal gun possession and headed straight into federal custody in a tax case.

US Marshals escorted the 36-year-old musician out of Mid-State Correctional Facility at 9:30 a.m. He was being held at the Oneida County Jail in central New York as he awaited word from the Federal Bureau of Prisons about where he’ll serve time in the tax case.

The rapper, who had been in protective custody because of his celebrity, has some time remaining on a 28-month sentence for tax evasion that ran concurrently.

Ja Rule, whose real name is Jeffrey Atkins, may have less than six months left and may be eligible for a halfway house, defense attorney Stacey Richman said. Back taxes are one of the main reasons he wants to get back to work, she said.