WASHINGTON—In reaching a historic agreement to restrain Iran’s nuclear program, President Barack Obama effectively shredded the foreign-policy playbook that had guided the U.S. for three decades.

Mr. Obama has come to an understanding with a country that has been perhaps Washington’s most troublesome adversary since 1979—so estranged that American officials wouldn’t even meet with their Iranian counterparts just a few years ago. Now Mr. Obama will seek to capitalize on his personal investment in this transformation by pivoting...