(CNN) — A group of former top counternarcotics officials are taking on Attorney General Eric Holder over the Justice Department’s decision to soften its stance on marijuana legalization.

Nine former administrators of the Drug Enforcement Administration wrote Holder to criticize the department’s decision not to challenge laws in Colorado and Washington state, which are allowing legal recreational use of marijuana after voter-approved ballot measures. More than two dozen other states and the District of Columbia allow medicinal use of marijuana.

The Justice Department last month issued new guidelines to prosecutors intended to focus on more serious trafficking offenses, largely avoiding a conflict with state laws. The federal government still classifies marijuana as one of the most dangerous substances, and it remains illegal under federal law.

Buoyed by the Justice Department’s move and by public polls showing increasing voter support, pro-marijuana activists are now looking to the ballot box to legalize marijuana in more states.

The nine former DEA chiefs, who span the years of the U.S. Drug War dating to the 1970s, aren’t moved by changing public opinion on the issue. In the letter dated Monday, they say they were “shocked and dismayed” by Holder’s move.

“You took an oath of office, as each of us did, to support and defend the Constitution of the United States,” the former DEA officials say. “Great harm will be done in Colorado and Washington and throughout the country with the legalization of marijuana.”

The Justice Department didn’t immediately respond to a request for comment.