But Holder said "we must be suspicious of legal classifications based solely on sexual orientation" and that upholding equal rights is an important principle.

Democratic attorneys general in five states -- Pennsylvania, Virginia, California, Illinois and Nevada -- have declined to defend same-sex marriage bans against lawsuits filed by gay couples.

Pennsylvania Attorney General Kathleen Kane announced last June her decision not to defend Pennsylvania's law banning same-sex marriage, two weeks after the United States Supreme Court struck down the federal ban.

"Denying equality is the very definition of discrimination," she said.

Copyright 2014 WFMZ. The Associated Press contributed to this report. All rights reserved. This material may not be published, broadcast, rewritten or redistributed.