Attorney General Eric Holder speaks during a news conference at the Justice Department in Washington, Monday, Oct. 4, 2010. (AP)

The Justice Department is sending a small pack of election observers to Arizona as Hispanic groups sound the alarm over an anti-illegal immigration group's mass e-mail seeking to recruit Election Day volunteers to help block illegal immigrants from voting.

Hispanic voting rights groups say the e-mail is just an attempt to intimidate minority voters. But election fraud monitors say that there are hundreds of examples of duplicate registrations, wrong information and past unregistered voters getting ballots.

Voter fraud allegations have emerged just days before the midterm in many crucial battleground states across the country, including Nevada, Pennsylvania and Florida.

The Justice Department announced Friday it would deploy more than 400 federal observers to 30 jurisdictions in 18 states ahead of Tuesday's election. But Arizona officials say the department had already committed to sending observers to their state.

Justice Department officials had notified Maricopa County that they would send a "handful" of election observers, even before the e-mail was sent, Matthew Benson, a spokesman for Arizona's secretary of state, told FoxNews.com.

More On This...

"We don't know why (the observers are) coming," he said. "But we do not believe it's in response to the e-mails making the rounds."

But William Gheen, executive director of Americans for Legal Immigration, said he knows why they're coming.

"They're sending them out because the Obama administration is doing everything it can to make sure as many illegal aliens vote in 2010 although that is a violation of federal law," he told FoxNews.com.

Benson noted that the Justice Department sends election observers to at least one Arizona county every election. In 2006, the department sent around 30 election observers to Maricopa County, he said.

Benson said the plan had nothing to do with the firestorm elicited by Ban Amnesty Now, or B.A.N., the group that sent an e-mail entitled "6 Days to STOP Illegals from Stealing the Election!"

The e-mail said "open-border criminals" in the unions, the White House and Hispanic groups are "determined to undermine" the election.

The e-mail originally included the signature of Maricopa County Sheriff Joe Arpaio, an immigration enforcement proponent and B.A.N. honorary co-chairman, but Arpaio's name was later removed after Sean McCaffrey, the president of B.A.N., said he didn't mean to send out the e-mail attributed to the sheriff.

"We are very concerned when you have Sheriff Joe Arpaio putting out this type of call to the community with the fear and concern that illegals are going to take over the election," said Ben Monterroso, president of the group. "It's not that we're not concerned that the elections are secure, but the way that it's been written – anyone who has the appearance of a foreign person, i.e. being a Latino – then that can create intimidation and violate their rights to vote."

McCaffrey said Friday that, despite the erroneous attribution to Arpaio, he is looking for volunteers to provide information to his group about voter fraud, "especially stories concerning activities involving illegal aliens."

"We hope that by forwarding interested volunteers to state elections officials we may contribute to smooth, fair and legal elections in all 50 U.S. states," he wrote in a blog on the group's website. "Furthermore, we hope that those who seem so shocked and outraged that Americans are concerned about stopping voter fraud will own up to their actions and stop committing vote fraud."

McCaffrey told FoxNews.com in an interview that nearly 500 people have volunteered so far -- about 100 of them in Arizona.

Part of the concern for voter fraud monitors is a result of a ruling this week by a federal appeals court that stripped the state of its ability to request proof of citizenship when residents register to vote. State officials say the ruling could increase the likelihood of voter fraud but not in next week's midterms because the deadline for registration passed before the decision.

At Arizona polls, voters must show one piece of identification that includes their name, address and photograph, such as a driver's license. They also can provide two forms of ID with their names and address, such as a utility bill or bank statement. Voters can also provide one form a acceptable photo ID with another form of non-photo ID that includes their name and address, such as a passport and a utility bill.

Arpaio declined to comment, but McCaffrey told FoxNews.com that he apologized to the sheriff Thursday for using his name without his permission. Arpaio, he said, stepped down as honorary co-chairman.

"He'll continue to support us but he didn't want an official title with the group while he was being investigated by everyone with a badge at the White House," he said, alluding to the Justice Department's investigation into whether Arpaio's office was discriminating against Hispanics with its crackdown on illegal immigrants.

Monterroso said he's happy the Justice Department is sending poll watchers, even if it's not in response to his group's letter.

"I think it's the right thing to do," he said. "It's better to be safe than sorry. If they have people out there, we want to make sure people vote without fear of intimidation or harassment by anyone else."