He said he believes it's necessary to reject the pleas after the 2nd U.S. Circuit Court of Appeals in Manhattan last week reversed two insider-trading convictions, saying defendants Anthony Chiasson and Todd Newman were too far removed from the inside information to be prosecuted. The appeals court ruling stemmed from a government crackdown on hedge fund insider trading that resulted in more than 80 convictions.

Tetra Images | Getty Images

Repeatedly Thursday, the judge told lawyers he had "strong doubts" that guilty pleas to conspiracy charges by four men are supported by the facts of the case. A fifth man is awaiting trial.

The men, all friends, were charged with conspiring in 2009 to earn hundreds of thousands of dollars by trading on a secret about IBM's planned acquisition of a software company.

The judge said it was possible the government could still build a case against them, and he left open the possibility they could still plead guilty.

Read MoreLevin takes on commodity insiders

Prosecutors asked that the judge not toss out the pleas until the government had a chance to argue against doing so in court papers to be submitted next month.

The government has not said whether it will ask the full 2nd Circuit to hear an appeal of the ruling dropping charges against Chiasson and Newman. Chiasson, of New York, had been sentenced to 6 1/2 years in prison while Newman, of Needham, Massachusetts, received a 4 1/2-year sentence. The ex-portfolio managers were convicted in December 2012.

The judge said that if he vacates the pleas by the four men, he will include them in a trial to begin on Feb. 23.