U.S. Attorney General Eric Holder thanks Justice Department employees as he delivers his parting remarks at the Robert F. Kennedy building April 24, 2015 in Washington. (Photo by Chip Somodevilla/Getty Images)

Former Attorney General Eric Holder praised the work of Sen. Rand Paul, R-Ky., and the Koch brothers to promote criminal justice reform, saying he was optimistic that legislation could make it through the current Republican-majority Congress.

In an interview with the National Law Journal posted Sunday, Holder, who has returned to private practice, said Washington was experiencing a rare "moment in time where the Left and the Right seem to agree" that there are serious problems with current federal sentencing rules. He attributed this in part to Paul's and the Koch's efforts to lead the charge on the right.

"I had a meeting in my conference room where we had representatives from the ACLU, the founder of the Tea Party, the Koch brothers, and it was like, wow. [I thought,] people need to see who's here. Pretty conservative Republicans, and me. So I'm satisfied with what we did [during my time in the administration], but the job is not done. And I would hope in this Congress, not the next one, but in this Congress, that legislation will pass," Holder said.

Holder also said a lunch he had with Paul in February was "breathtaking" given "the amount of agreement that we had on this subject."

Paul, a Republican with strong libertarian leanings who is a 2016 presidential contender, has repeatedly urged others in the GOP to take up the issue of sentencing reform, saying that the GOP needs to be "the party of the entire Bill of Rights," not just the First and Second Amendments.

He has co-sponsored bipartisan legislation called the Smart Sentencing Act, which would halve most federal drug-related mandatory minimum sentences, among other reforms. The bill passed the Senate Judiciary Committee last year, but under then-Senate Majority Leader Harry Reid, D-Nev., it never received a floor vote.

Paul has also introduced legislation called the Justice Safety Valve Act, which would allow federal judges to ignore mandatory minimum sentences if they believe doing so is in the interest of justice.

Holder and Paul have clashed in the past on other issues. In 2013, Paul led a 13-hour filibuster against a White House nominee to be CIA director in order to oppose President Obama's drone strike policy. The filibuster was prompted by Senate testimony from Holder in which he refused to completely rule out a drone strike on a U.S. citizen on U.S. soil. Paul ended the filibuster after Holder issued a statement that a drone strike could not be used in that scenario against a noncombatant citizen.

Paul also launched a class-action suit last year against Obama administration officials over National Security Agency surveillance programs.

Holder said that despite all of this they were able to put their differences aside on the criminal justice issue. "As he left he said, 'But I'm suing you,' " Holder said, describing a conversation with Paul. "I said, 'OK, fine.' I don't know what he's suing me for, but he was suing me for something else."