The governor of Mississippi has expressed disappointment in a judge's ruling that temporarily blocked the state from closing its last remaining abortion clinic, as the legal battle to keep it open continued.

The ruling, on Sunday, gave a last-minute reprieve to Jackson Women's Health Organization by halting a new law designed to limit abortions which was scheduled to come into effect on Monday.

It was the first round of a legal fight which will determine whether the state will effectively become the first in the union to effectively ban abortion. A new state law, HB1390, passed by governor Phil Bryant, requires doctors who perform the procedure to have admitting privileges at a local hospital.

Both parties are required to submit briefs in the case and a hearing has been set for next week. The temporary restraining order is in effect until the hearing, on 11 July. Judge Daniel Jordan, of the US district of Mississippi, will then rule whether there is sufficient evidence that HB1390 is unconstitutional. If he decides that is the case, and orders a preliminary injunction against it, he would then need to decide if he can permanently block the law.

The lawsuit, brought about by the clinic represented by the Center for Reproductive Rights, argued that supporters of the new law have publicly stated the broader intention to limit abortions. It cites lieutenant governor Tate Reeves's website, which states that the law "not only protects the health of the mother but should close the only abortion clinic in Mississippi".

State lawmakers argued that their concern was for the health of women, saying the law was designed to make sure they were properly cared for in the event of complications during abortions.

The judge's initial ruling, on Sunday, states: "Though the debate over abortion continues, there exists legal precedent the court must follow."

The judge goes on to cite the 14th amendment which allows a woman the right to chose to have an abortion without "undue burden" from the state. An "undue burden" is one which places a substantial obstacle in the path of a woman seeking an abortion.

He states: "In this case, plaintiffs have offered evidence – including quotes from significant legislative and executive officers – that the act's purpose is to eliminate abortions in Mississippi. They likewise submitted evidence that no safety or health concerns motivated its passage. This evidence has not yet been rebutted."

On Monday, governor Phil Bryant's spokesman Mick Bullock issued a statement, which vowed to fight on. "Governor Bryant believes house bill 1390 is an important step in strengthening abortion regulations and protecting the health and safety of women. The federal judge's decision is disappointing, and governor Bryant plans to work with state leaders to ensure this legislation properly takes effect as soon as possible."

A spokeswoman for the Mississippi department of health, who were due to send inspectors into the clinic on Monday, said it would no longer do so.She said that they would follow the law. Liz Sharlot, director of communications, said: "A judge has ordered the state health department's office and the district attorney's office to temporarily restrain from enforcing the bill."

On Sunday, Diane Derzis, owner of the clinic, told the Guardian she was "jubilant" over the ruling, adding that it would allow her team of physicians to carry on providing safe, legal abortions in the state.

She had earlier warned that closing the clinic would have pushed some vulnerable women into the hands of backstreet operators.

Under the new law, physicians in Mississippi would have been barred from carrying out terminations without first being registered as a practitioner with privileges to admit patients to a local hospital.

None of the doctors at the clinic currently meet that requirement, which has been described by pro-choice campaigners as "medically unjustified".

As such, the new requirements had threatened the closure of a Jackson clinic, a centre that has been running for 17 years in the face of threats, protest and opposition from religious conservatives in the state.

Derzis has consistently claimed that the new legislation was designed explicitly with the intention of forcing her clinic out of business and had little to do with the safety of patients. The clinic has an agreement with a local physician who will admit the centre's clients should the need arise.

Doctors at the centre note that the number of women who experience a complication during an abortion that requires treatment at a hospital is extremely low – less than 0.3% according to the sexual health think tank the Guttmacher Institute.

Nancy Northup, president of the Center for Reproductive Rights said: "While we are pleased the court has decided to temporarily block enforcement of this medically unwarranted restriction, this battle is far from over.

"We will continue to fight alongside the Jackson Women's Health Organization to ensure that the women of Mississippi are not relegated to a second class of US citizens, denied the constitutionally-protected rights that other women nationwide are guaranteed."

Mississippi, one of the poorest states in America, also has one of the highest rates of teen pregnancy – more than 60% above the national average.

Figures from the state's health department reveal that 2,297 abortions were performed in the state in 2010, with the vast majority of these terminations carried out by doctors at the Jackson clinic.