U.S. President Donald Trump and Secretary of State Rex Tillerson at a cabinet meeting in the White House, November 20, 2017. Trump says he is not planning to fire Tillerson. Kevin Lamarque

Central Intelligence Agency Director Mike Pompeo, shown here testifying before the U.S. Senate Select Committee on Intelligence on May 11, 2017, has been named as a favorite to replace Rex Tillerson as secretary of state. Eric Thayer

Donald Trump may be denying it, but there seems no doubt that Secretary of State Rex Tillerson is a marked man.

Last month the speculation was on how soon United Nations Ambassador Nikki Haley would replace Tillerson. Last week, a fresh burst of reports signaled the end for the secretary – who has not denied he once called Trump a “moron” – and named CIA director Mike Pompeo as a favorite to succeed him.

Trump was quick to tweet out a response: “The media has been speculating that I fired Rex Tillerson or that he would be leaving soon - FAKE NEWS! He’s not leaving and while we disagree on certain subjects, (I call the final shots) we work well together and America is highly respected again!” Trump may be sincere, or he may be buying time to avoid confrontational confirmation hearings for a new candidate at a fraught time for an administration mired in the Russia scandal.

But whether Tillerson leaves now or later, it’s important to look at what lies ahead for the State Department.

It is unclear whether Pompeo can succeed where Tillerson failed. The climate for political appointees in Washington today feels more like that of a lame-duck presidency; many of the old standbys are not interested in signing up for what may turn out to be short-run jobs. That leaves a small pool for Pompeo to fill the State Department’s many unfilled posts. Pompeo's tenure at the Central Intelligence Agency was brief enough that he is unlikely to bring over many loyalists, and most at CIA headquarters in Langley see working for State as a kind of step down anyway (many at the Agency view themselves as jocks to State’s nerds.)

There is also the culture issue. Pompeo’s hard-line stances at the CIA rubbed many officials the wrong way, leaving them wondering if the boss could navigate the nuances that drive good decision making. How poorly that will play out at the State Department, with its culture of discussion and deliberation, its love of what-ifs and maybes, is easy to imagine.

And there's the record: Pompeo caught Trump's eye in part for his tough stance on Iran. Inside the State Department, the Iran nuclear deal is seen as one of the institution's modern-day signature accomplishments. Pompeo is a conservative, and State has always been the most "liberal," as in committed to the global system of trade and democracy part of modern administrations. Tillerson, weakly, but in line with State-think, pushed for some sort of talks with North Korea and supported the Iran deal. Pompeo opposes both.

But whether Tillerson is replaced by Pompeo, who brings specific issues with him, or by someone else, the basics of a transition remain the same. State is a hybrid institution, half foreign and half domestic. The unique interplay between the civil service (non-diplomats, including support staff stationed in Washington DC) and the Foreign Service (who have primary responsibility for policy in Washington and who staff the embassies and consulates abroad) complicates secretary of state transitions. Engaging both workforces, with their different vested interests, can be tough. And unlike the military, where chains of command and internal procedures are written on checklists, State has a more fluid structure. It is a vertically-oriented bureaucracy, with layers below the boss’ office waiting for bits of policy to fall so as to inform them of what their own opinions are.

A huge part of Tillerson's failure was in missing that last point. The traditional way of engaging the bureaucracy is for a new secretary to fill key positions with political appointees, who will shape the rank and file below them. Tillerson left too many slots vacant too long, and now finds himself without allies inside Foggy Bottom. Meanwhile, left on their own, his diplomats found ways to make trouble, including disclosing once-sacrosanct internal “dissent memos” on matters such as the administration’s approach to child soldiers and Trump’s executive orders banning travelers from listed Muslim-majority countries.

Alongside building their version of the organization, it is incumbent on new secretaries to aim the State Department at some broad goal. State is an agency without primary agency. Under one administration it focuses on arms control; under another, State tries to rebuild Iraq and Afghanistan. More recently, the emphasis has been on "soft power:" programs to empower women, the use of social media, promoting democracy, and the growth of LGBTQ rights globally. Tillerson never articulated much of a goal beyond some unfocused thoughts on structural reform that will never again see daylight. Though it is fashionable to label Tillerson as the worst secretary of state of modern times, in reality Tillerson will be remembered as perhaps the most pointless of secretaries.

Based on my conversations with former State Department colleagues (I served 24 years as a diplomat) Tillerson's successor will encounter a mood inside the State Department reminiscent of a rescue dog kennel: over there are the mutts who feel abused, wary of any new human. Off to the side are the ones who have given up; still on the books to log the service needed to get their generous pensions but with little to offer a new boss. The majority will be open and waiting to see what happens. But watch out for a few who feel newly empowered, the ones who think they helped drive a bad secretary out of office. They may still bite.

At the end of the day, the mismatch is not really about who is secretary of state, but who is president. A lot of the anger directed at Tillerson was using him as a stand-in for Trump. The primary driver of foreign policy remains the White House, and the White House appears to have little love for its diplomats. Trump agreed with dramatically cutting the State Department budget, and many believe the number of unfilled positions is a Trump-driven effort to cripple the institution. Trump himself stated on foreign policy no one but him matters and seemed to dismiss diplomacy as part of a solution in North Korea. Trump seems to favor the military and military men as some of his closest advisors (Pompeo graduated from West Point.) One commentator described the State Department as “an institution that the president doesn’t value, respect, or understand.”

Tillerson, an establishment Republican, had within him a bit of divergent thinking from Trump on issues like Iran and North Korea. Pompeo is an old school hawk and a Trump loyalist. If the president's intention is indeed to dismantle the State Department, it is hard to imagine a person better suited to the task than a partisan guy like Pompeo.

The irony will be if in a few months from now some of the people who wanted Tillerson out start wondering if they had not been better off under his leadership.