The U.S. National Security Agency and Department of Justice exceeded their legal authority to conduct surveillance when collecting the telephone records of millions of U.S. residents, several U.S. lawmakers said Wednesday.

Several members of the U.S. House of Representatives Judiciary Committee, both Republicans and Democrats, ripped into representatives of the DOJ and the U.S. intelligence community for their collection of U.S. phone call records, saying the bulk collection violates Patriot Act restrictions that limit surveillance to information relevant to an antiterrorism investigation.

Representative John Conyers Jr., a Michigan Democrat, called on the agencies to stop the data collection program.

"We never, at any point in this debate, have approved the type of unchecked, sweeping surveillance of United States citizens employed by our government," he said during a hearing on the NSA. "If the government cannot provide a clear, public explanation for how its program is consistent with the statute, it must stop collecting this information immediately."

Other committee members said they will look for ways to amend the Patriot Act to stop the NSA's collection of U.S. telephone records. Even without more immediate changes in the law, Congress isn't likely to reauthorize the business-records collection provision in the Patriot Act when it expires in late 2015, unless the NSA scales back its surveillance, said Representative Jim Sensenbrenner, a Wisconsin Republican and author of the original Patriot Act.

"Unless you realize you've got a problem, that [provision] is not going to be renewed," Sensenbrenner told NSA and DOJ officials. "There are not the votes in the House of Representatives ... and then you're going to lose the business-record access provision of the Patriot Act entirely. It's got to be changed, and you have to change how you operate .... otherwise in a year-and-a-half, you're not going to have it anymore."

The bulk collection of U.S. phone records has made "a mockery" of the Patriot Act's relevancy limits, Sensenbrenner said.

Several lawmakers questioned how the NSA and DOJ could view all U.S. phone records as relevant to a terrorism investigation.

"The problem, obviously, from what we're hearing is that everything in the world is relevant," said Representative Jerrold Nadler, a New York Democrat. "You're disregarding the statute entirely."

The bulk collection of U.S. phone records is necessary for later searches, said James Cole, deputy attorney general at the DOJ. The phone records and a related Internet communications surveillance program have helped U.S. authorities in dozens of terrorism cases, officials said.

"If you're looking for the needle in the haystack, you have to have the entire haystack to look through, but we're not allowed to look through that haystack willy-nilly," Cole said.

DOJ and NSA officials defended the phone records collection program, saying the bulk collection of phone records is allowed in the Patriot Act. The collection does not violate the U.S. Constitution's Fourth Amendment, protecting U.S. residents against unreasonable searches and seizures, because, in 1979, the U.S. Supreme Court ruled that telephone records aren't private information that require a court-ordered warrant, Cole said.

The Foreign Intelligence Surveillance Court and Congress both have significant oversight of the NSA surveillance programs, Cole said.

The phone records are protected because analysts never access the vast majority of them, said John Inglis, the NSA's deputy director. In 2012, NSA analysts ran queries on fewer than 300 telephone numbers, officials have said, and analysts need to show to agency officials that those queries are relevant to a terrorism investigation before accessing the numbers, he said.

During the hearing, lawmakers raised few concerns about the Prism program, in which the NSA collects the content of email and other Internet communications sent by people not believed to be U.S. citizens. Former NSA contractor Edward Snowden revealed both the phone records collection and the Prism program in news stories published in early June.

"Our primary responsibility at the National Security Agency ... is to defend the nation," Inglis said. "These programs are a core part of those efforts. We use them to protect Americans and our allies and partners worldwide."

While officials defended the surveillance court's review of the collection requests, several lawmakers suggested the court is a rubber stamp. Since the court was established in 1979, U.S. agencies have made nearly 34,000 surveillance requests to the court, and 490 of those were amended at the court's request, said Representative Hakeem Jeffries, a New York Democrat. In that time, the court rejected just 11 requests, he said.

Those statistics don't capture significant negotiations in many cases between the court and the NSA and DOJ before the judges grant the orders, said Robert Litt, general counsel for the U.S. Office of the Director of National Intelligence.

Several lawmakers called on the agencies to release more information about the surveillance programs and better explain to the public why the programs are necessary.

"We try very hard to keep in mind both the protection of national security and the privacy and constitutional rights of Americans," Litt said. "We think we've struck that balance in the right place, but if the people in the Congress determine we've struck that balance in the wrong place, that's a discussion we need to have."

Grant Gross covers technology and telecom policy in the U.S. government for The IDG News Service. Follow Grant on Twitter at GrantGross. Grant's e-mail address is grant_gross@idg.com.