The U.S. Congress should remove obstacles it has imposed to closing the Guantanamo Bay prison, the White House said on Thursday, even as the House of Representatives’ top Republican voiced support for keeping prisoners at the military facility in Cuba.

Earlier on Thursday, House Speaker Paul Ryan said prisoners at the facility should remain there even as the White House has said it could consider executive action.

White House spokesman Josh Earnest told reporters at a daily briefing that Ryan’s comments run counter to the views of many other Republicans in Congress as well as the Obama administration, but gave not new timing about a forthcoming report on the prison where the United States holds terrorism suspects from around the world.