TOKYO—The U.S. and Japan tried to put the best face on their failure to reach a trade deal during President Barack Obama's Tokyo visit, saying major progress was made during nearly round-the-clock talks.

But it was Japan's often-blunt finance minister, Taro Aso, who gave voice to the frustration many felt here.