CAMDEN, N.J. (CBS) — The on-again, off-again New Jersey state senate candidacy of former Olympian Carl Lewis is off again. He’s been kicked off the balllot for a second time by a federal judge in Camden.

Judge Noel Hillman cited the same reason he used to remove Lewis from this past June’s Democratic primary ballot: that the Willingboro, NJ native did not comply with the state’s four-year residency requirement — one that applies, the judge said, “equally to the famous and the obscure, to the overachiever and the pedestrian, to the athletically gifted and the passive observer.”

Attorneys for Lewis, a nine-time Olympic gold medalist, have maintained that their client has owned property within the state’s 8th Senatorial District for years. They persuaded the Third Circuit Court of Appeals to put Lewis back on the primary ballot (which he won without opposition) and said they will return there in an effort to get this decision overturned.

“The decision was not unexpected,” said Lewis attorney William Tambussi. “We figured all along that we’d have to get through the District Court to the Third Circuit in order to get ultimate relief.”

If Lewis gets back on the fall ballot he will face incumbent state senator Dawn Addiego in November in the Republican-leaning district.

The 50-year-old Olympic icon grew up in New Jersey but attended college in Texas and later lived in California. He has owned homes in New Jersey since 2005, but also pays taxes and utilities on homes in California. He’s been a volunteer track coach in his native Willingboro since 2007, but voted in California as recently as 2009.

He switched his voter registration to New Jersey just before declaring his candidacy in April.

Lt. Gov. Kim Guadagno, a Republican, had previously said Lewis was ineligible to run.

Guadagno last month declined to certify his name for the November ballot. The federal judge today affirmed that decision, just ahead of the deadline for gathering candidate information for the printing of the fall election ballots.