ISTANBUL — A high-stakes power struggle between Iran’s moderate president and his hard-line opponents in the judiciary appeared to escalate with the arrest of the president’s brother and the conviction of an American student for espionage this weekend — rulings that seemed timed to embarrass the Iranian leader at home and abroad.

President Hassan Rouhani, who was reelected in a landslide in May, has challenged the conservative establishment by pledging reforms in Iran and advocating diplomacy and openness toward the rest of the world. His recent criticisms of the hard-line judiciary and powerful security forces have prompted public rebukes from the supreme leader, Ayatollah Ali Khamenei, who wields ultimate authority in Iran.

The tensions come as Iran and the United States spar over the terms of a nuclear deal struck with world powers to limit Iran’s nuclear weapons program. On Monday, the White House grudgingly certified to Congress that Iran is in compliance with the deal, which was negotiated by the Obama administration and lifts major sanctions. The Trump administration has taken a much harsher stance on Iran, threatening to abandon the deal, and the Treasury Department on Tuesday announced new sanctions primarily targeting Iran’s ballistic missile program.

But the moves by Iran’s judiciary — including the sentencing of a Princeton graduate student, Xiyue Wang, to 10 years in prison for spying — also undermine Rouhani’s attempts to build better relations with the West, which more-reactionary Iranian institutions such as the judiciary oppose. And they suggest an effort by ruling clerics to pressure the president to back down from confrontation on the domestic front, particularly ahead of the official inauguration of his second term next month, when Rouhani will pick his new cabinet.

More broadly, however, the actions by the judiciary and Khamenei paint a picture of a hard-line establishment hitting back at an outspoken and popular president who has promised to curb some of the regime’s worst excesses.

Rouhani’s pro-reform agenda “poses a major threat to their worldview and political agenda,” Nader Hashemi, director of the Center for Middle East Studies at the University of Denver, said of the hard-liners.

In recent weeks as well as during the May presidential campaign, Rouhani rapped the judiciary for what he said were arbitrary arrests and a history of atrocities. He also criticized the economic role of the elite Revolutionary Guard Corps, Iran’s most powerful security institution, at the expense of the country’s private sector.

Those admonishments led Khamenei to publicly defend the judiciary.

“The judiciary should be a pioneer in establishing public rights within the society . . . and confront anyone who violates laws,” Khamenei said in a speech this month, according to the Center for Human Rights in Iran, an independent nonprofit based in New York.

Rouhani, addressing a gathering of judicial officials the previous day, had called on jurists to limit the practice of summoning people for interrogation without due cause.

Last month, Khamenei dressed down the president in front of the country’s most senior politicians, warning Rouhani against suffering a fate similar to that of Iran’s first post-revolution president, who served from 1980 to 1981. Abol Hassan Bani-Sadr was impeached after facing off against the powerful clergy and was forced to flee to France.

“Using the institutions of the state that they control — primarily the judiciary — they are sending a message to Rouhani and his supporters that they are in control of the political system,” Hashemi said. “And that they will oppose his attempts to engage with the Western world and promote more freedoms at home.”

The arrest and conviction of Wang, a 37-year-old scholar at Princeton, appeared to target Rouhani’s wider foreign policy and engagement with the West. Although Wang was detained in August 2016, the timing of the verdict is suspect, analysts say.

“Why did they keep it a secret as long as they did? Timing is important,” said Alex Vatanka, an Iran expert at the Middle East Institute in Washington.

Wang, who colleagues say traveled to Iran to research the Persian Empire’s Qajar dynasty for his thesis, was accused of attempting to create a digital archive for the State Department and Western academic institutions.

“Wang’s sentencing by the Iranian judiciary is yet another indicator that the hardest of Iran’s hard-liners are the ones who set the direction for Iranian domestic and foreign policy,” said Behnam Ben Taleblu, an Iran analyst at the Foundation for the Defense of Democracies in Washington.

But the arrest of Rouhani’s brother, Hossein Fereydoun, this weekend appeared to be a more immediate and direct attack on Rouhani. Fereydoun is a close adviser of the president and was a key player in nuclear negotiations. He came under attack by conservatives this year for alleged financial impropriety, although the formal charges are still unclear.

Corruption and graft are widespread in Iran, but the probes “are often politically motivated phenomenon,” said Taleblu, adding that they “have more to do with political score-settling than reforming business practices.”

“Elements of the Iranian judiciary and hard-line establishment have been looking at taking down Fereydoun for quite some time,” he said.

According to Suzanne Maloney, a senior fellow and expert on Iran at the Brookings Institution, targeting Rouhani’s brother “is a very convenient way to cause pain to the family without necessarily provoking a crisis of office.”

“The general message that the rest of the system is trying to send to Rouhani is not to get too far ahead of himself,” she said, “to not allow his decisive election victory to give him illusions of greater autonomy and authority than his position actually has.”

Whether Rouhani will bow to the pressure remains to be seen. During his first term, the president deferred to the supreme leader and failed to push through more-serious reforms.

The relationship between Rouhani and Khamenei in the coming years “will be tense,” Hashemi said. “There has been an ongoing public feud between both figures, but ultimately power lies with the supreme leader.”

“If I had to bet, my bet would be for Rouhani to reluctantly submit to the limits established by the supreme leader,” he said. “All ­second-term Iranian presidents had to do this.”