WASHINGTON (AP) — Federal agents trying to patrol the U.S.-Mexico border say they're hampered by laws that keep them from driving vehicles on huge swaths of land because it falls under U.S. environmental protection, leaving it to wildlife — and illegal immigrants and smugglers who can walk through the territory undisturbed.

A growing number of lawmakers are saying such restrictions have turned wilderness areas into highways for criminals. In recent weeks, three congressional panels, including two in the GOP-controlled House and one in the Democratic-controlled Senate, have moved to give the Border Patrol unfettered access to all federally managed lands within 100 miles of the border with Mexico.

Two of the panels expanded the legislation's reach to include the border with Canada.

The votes signal a brewing battle in Congress that will determine whether border agents can disregard environmental protections as they do their job.

Dozens of environmental laws were waived for the building of the border fence, and activists say this is just another conservative attempt to find an excuse to do away with environmental protections.

But agents who have worked along the border say the laws crimp their power to secure the border.

Zach Taylor, a retired Border Patrol agent who lives about nine miles from the Arizona-Mexico border, said smugglers soon learn the areas that agents are least likely to frequent.

"The (smuggling) route stays on public lands from the border to Maricopa County," Taylor said, referring to the state's most populous county. "The smugglers have free rein. It has become a lawless area."

Environmental groups said lawmakers lining up to support the legislation have routinely opposed the Endangered Species Act, the Safe Drinking Water Act and dozens of other laws, and they accused the lawmakers of using illegal immigration as the latest excuse to gut protections.

"For every problem that's out there in society, there's some extremists in Congress who say the solution is, 'Well, let's roll back the environmental laws, let's open up the public lands,'" said Paul Spitler, spokesman for the Wilderness Society. "It doesn't comport to reality, but it fits their mindset that it's simply the environmental regulations that are holding back America."

Nearly 40 percent of the land on the U.S.-Mexico border and about a quarter of the land on the U.S.-Canadian border is public land, including Big Bend National Park in Texas and Glacier National Park in Montana. Driving is prohibited on those parts of the land that are designated wilderness areas.

Wildlife officials say vehicle use can be particularly hazardous in the desert. Water gathers in the tire tracks instead of in natural pools and evaporates more quickly, leading to less vegetation and less available food. Some areas, such as Big Bend and the desert farther west, are deadly to traverse in certain months and immigrants and smugglers avoid them.

The wilderness areas also have other restrictions on development. Border patrol agents, for example, must get permission from other federal agencies before maintaining roads and installing surveillance equipment. Federal auditors found it can take months to get that permission.

"What the Border Patrol says they really need down there is not necessarily more manpower or money," said Rep. Rob Bishop, R-Utah, whose bill easing the restrictions passed the House Natural Resources Committee along party lines. "They need more east-west access on those public lands."

Sen. John McCain, R-Ariz., sponsored an amendment that requires the Agriculture and Interior departments to give Border Patrol personnel immediate access to federal lands on the southern border for security activities, including for routine motorized patrols. The amendment passed a Senate committee with the support of five Democrats and eight Republicans.

McCain told colleagues that up to 100 people sit on mountaintops near the border serving as lookouts for smugglers, suggesting that improved law enforcement access on those mountains would deter the lookouts.

"What he says is absolutely true," said Democratic Sen. Mary Landrieu of Louisiana, who visited Arizona with McCain. "For the life of me, I can't understand the hesitancy on the part of Interior or Agriculture to provide access to border security guards."

Rep. Ben Quayle, R-Ariz., sponsored a similar amendment that extends the law to the Canadian border as well, and it passed by a voice vote, which is usually reserved for noncontroversial legislation.

During a House subcommittee hearing in April, Ron Vitiello, deputy chief of the U.S. Customs and Border Patrol, said he had "no complaints" about environmental laws.

But George McCubbin, president of the union that represents about 17,000 Border Patrol agents and support staff, likened current policy to telling city police officers they can't patrol a particular neighborhood.

"If they want to get serious about this problem on the border, they can't be restricting areas we go in," said McCubbin, who works in Casa Grande, Ariz. "Don't let us there and you have nothing but the bad element going through that area."

The Government Accountability Office, Congress' investigative arm, reported that supervisors at 17 of 26 Border Patrol stations along the Mexican border said access to federal lands had been limited because of environmental restrictions. Yet, the vast majority of the agents in charge also said that they were generally able to adjust their patrols without sacrificing effectiveness.

Democratic lawmakers and environmental groups cite the GAO's findings in arguing against giving the Border Patrol authority to operate as it sees fit on federal lands.

"The record is clear. The problem this bill claims to be solving does not exist," said Rep. Raul Grijalva, D-Ariz. "So, if this is not about border security, what is it about? It's about undermining fundamental environmental protections for millions of Americans."

Bishop said federal agents would be better stewards of sensitive lands than illegal immigrants and smugglers.

"What is so ironic is that the environmental degradation is not being done by the Border Patrol," Bishop said. "It's being done by the illegals who are coming across."