After leaving West Point the younger Hilsman was immediately posted to the South-East Asian theatre of World War II and joined the Merrill's Marauders long-range penetration jungle warfare unit, fighting the Japanese during the Burma Campaign.[4] There he found morale to be poor due to typhus outbreaks and unhappiness with the generals leading the unit.[9] He participated in infantry operations during the battle for Myitkyina in May 1944 and suffered multiple stomach wounds from a Japanese machine gun while on a reconnaissance patrol.[2][4][5]

After recovering in army field hospitals, Hilsman joined the Office of Strategic Services.[5] By now a lieutenant,[12] he at first served as a liaison officer to the British Army in Burma.[5] He then volunteered to be put in command of a guerrilla warfare battalion, organized and supplied by OSS Detachment 101, of some three hundred local partisans, mercenaries, and irregulars of varying ethnicities, operating behind the lines of the Japanese in Burma.[4][5] There he developed an interest in guerrilla tactics and found them personally preferable to being part of infantry assaults.[5][9] By early 1945 Hilsman was considered, as Detachment 101 commander William R. Peers later stated, to be one of a number of the guerillas' "good ... junior officers, every one outstanding and experienced."[12] Hilsman's group made hit-and-run attacks on Japanese forces and kept a Japanese regiment ten times its size occupied far from the front lines,[5] all the while staging their own battle with ever-present leeches and other insects and various diseases.[9] In one particular engagement in May 1945, Hilsman led mixed company of Kachins, Burmese, and Karens in staging successful raids in the area between Lawksawk and Taunggyi, culminating in a carefully orchestrated ambush that caused a hundred casualties among the Japanese at no cost to the guerillas.[12] Hilsman wanted to deploy his unit further south into the Inle Lake area but was constrained by orders to help hold the road between Taunggyi and Kengtung.[12]

Soon after the Japanese surrender in 1945, Hilsman was part of an OSS group that staged a parachute mission into Manchuria to liberate American prisoners held in a Japanese POW camp near Mukden.[4] There he found his father, who became one of the first prisoners to be freed.[4] His father asked as they hugged, "What took you so long?"[13] At some point, Hilsman was promoted to captain.[14] (Decades later, Hilsman related his wartime experiences in his 1990 memoir American Guerrilla: My War Behind Japanese Lines.[9])

Returning from the war, Hilsman served in the OSS as assistant chief of Far East intelligence operations during 1945–46, and then once the Central Intelligence Agency had been created, served in it in the role of special assistant to executive officer during 1946–47[3] (he belonged to the Central Intelligence Group during the interim period between the two organizations).

During staffing of the incoming Kennedy administration, Under Secretary of State-nominee Chester Bowles aggressively sought people from the ranks of academia and the press who would be committed to the ideals of the New Frontier.[14] As part of this, Hilsman was selected to be the Director of the Bureau of Intelligence and Research for the U.S. Department of State,[14] assuming the position in February 1961. There his duty was to analyze foreign events and trends as part of the department's long-range planning.[2] Hilsman soon became a key planner within the administration's foreign policy circles.[4] Like many of the "New Frontiersmen", he had fought with distinction as a junior officer in World War II,[14] and Hilsman was particularly effective at talking to members of the U.S. Congress because that military background and war record appealed to hard-liners while his academic history and intellectual leanings appealed to those more of that bent.[5]

A Hilsman memorandum in November 1962 tried to account for the deployments of Soviet intermediate-range ballistic missiles in Cuba

Due to his background in guerrilla warfare, during 1961, Hilsman, together with Walt Rostow, pushed for the U.S. armed forces and the State Department to emphasize counterguerrilla training.[14] Hilsman was involved for more than two months in the U.S. responses to Soviet actions during the Cuban Missile Crisis of 1962, including developing informal communications with Soviet officials and the briefing of congressional leaders.[6][7][13][14] He was also involved in the State Department's analysis of the Sino-Soviet split and the possible conditions for future warming in Sino-American relations.[7]

Hilsman became one of the main architects of U.S. policy in Vietnam during the early 1960s and, in January 1962, he presented the plan "A Strategic Concept for South Vietnam".[2] It stated that the war was primarily a political struggle, and proposed policies that emphasized that the Vietnamese in rural areas were the key to victory.[2] It also recommended that the Army of the Republic of Vietnam start using guerrilla tactics.[2] Out of the report came Kennedy's approval of U.S. participation in the Strategic Hamlet Program, the relocation of rural peasants into villages consolidated and reshaped to create a defensible, networked perimeter, with the goal of removing population from contact and influence with the Viet Cong. Implementation of the program by the South Vietnamese government became problematic, however, and Hilsman himself later stated that their execution of it constituted a "total misunderstanding of what the [Strategic Hamlet] program should try to do."[19]

During 1962, reports from American journalists in South Vietnam about the progress of the conflict of the Viet Cong, and the characteristics of the South Vietnamese government under President Ngô Đình Diệm differed from the picture the U.S. military was portraying.[14] President Kennedy became alarmed, and in December 1962, Hilsman, together with Michael Forrestal of the National Security Council staff, were sent by Kennedy on a fact-finding mission to South Vietnam.[20] The resultant Hilsman–Forrestal Report was delivered to President Kennedy on January 25, 1963.[20] It described weaknesses in the South Vietnamese government; the corruption of Diệm and his brother Ngô Đình Nhu and their cohorts; and the increasing isolation of, and lack of support for, the Diệm regime from the South Vietnamese people.[20] Overall, however, the report came to some optimistic conclusions:[20] "Our overall judgment, in sum, is that we are probably winning, but certainly more slowly than we had hoped. At the rate it is now going the war will last longer than we would like, cost more in terms of both lives and money than we anticipated ..."[21] It thus contributed to the escalation of U.S. involvement in Vietnam and to growing doubts in U.S. government circles about the usefulness of the Diệm regime.[20]

Hilsman (far right) at the White House in April 1963 during a presentation of gifts with President Kennedy and Deputy Prime Minister of Malaya Tun Abdul Razak

In March 1963, the White House announced that Hilsman would become Assistant Secretary of State for Far Eastern Affairs, replacing Averell Harriman, who was promoted to an undersecretary position.[22] Hilsman had risen quickly in the government bureaucracy, partly because Kennedy liked his willingness to challenge the military.[6] A New York Times profile that year described Hilsman as "a restless, bouncy, aggressive but deeply reflective man".[4][7] Hilsman assumed the new position in May 1963. That same month, the Buddhist crisis began in South Vietnam, which featured a series of repressive acts by the South Vietnamese government and a campaign of civil resistance led mainly by Buddhist monks. Doubts grew further about Diệm, and within the administration, Hilsman became the most outspoken proponent of a coup against that government.[23]

On August 24, 1963, in the wake of government raids against Buddhist pagodas across the country, Hilsman, along with Forrestal and Harriman, drafted and sent Cable 243, an important message from the State Department to Henry Cabot Lodge Jr., the U.S. ambassador to South Vietnam. It declared that Washington would no longer tolerate Nhu remaining in a position of power and ordered Lodge to pressure Diệm to remove his brother, and that if Diệm refused, the Americans would explore the possibility for alternative leadership in South Vietnam. The cable had the overall effect of giving tacit U.S. approval for a coup against the Diệm regime.[2] Hilsman was the point man for the cable – some contemporaries referred to it as the "Roger Hilsman cable" – as it was approved and sent while many higher-ranking officials were out of town, with each of those officials who were called to approve it doing so because he thought some other official had approved it.[23] The events surrounding the sending of the cable led to Kennedy's becoming quite upset over the disorganization within his government.[24] They have also long been critiqued as at best an example of a bizarrely poor decision-making process[23] and at worst a case where a small group of secondary, anti-Diệm figures was able to circumvent normal procedures with a consequent harmful effect on the situation in Vietnam.[25]

On November 1, the 1963 South Vietnamese coup came; although conducted by South Vietnamese generals, they had been encouraged by the U.S., which thus shared responsibility.[26] U.S. decision-makers did not want the coup to involve assassination of the current leaders,[23][26] but by the next day, the arrest and assassination of Ngô Đình Diệm and his brother had taken place. The coup set off a period of political instability in South Vietnam that opened the door to a greater U.S. involvement.[13]

Hilsman was one of the academics and intellectuals in the Kennedy administration whom author David Halberstam later grouped together in his book as The Best and the Brightest, for the erroneous foreign policy they crafted and the disastrous consequences of those policies in Vietnam. And Hilsman's role has been variously interpreted. Mark Moyar's 2006 book Triumph Forsaken: The Vietnam War, 1954–1965 paints Hilsman as one of the key Americans who short-sightedly and arrogantly pushed out Diệm when, Moyar says, the struggle against the Communists was being won.[27]Guenter Lewy portrays Hilsman as being "farsighted and correct" in his 1964-going-on perspective, while scholar Howard Jones views the coup against Diệm that Hilsman acted in favor of as "a tragically misguided move".[23]

Following the assassination of John F. Kennedy on November 22, 1963, Hilsman stayed in his position under the new president, Lyndon B. Johnson. But Johnson sought a narrower range of opinion on foreign policy matters than Kennedy had and Hilsman, along with a number of other formerly influential State Department figures, was now not being listened to.[25] Furthermore, by this time, in the words of Halberstam, "[Hilsman] had probably made more enemies than anyone else in the upper levels of government."[28] Secretary of Defense Robert McNamara and the Joint Chiefs of Staff disliked Hilsman for his constant questioning of military estimates and forthrightness, Secretary of State Dean Rusk had been angered by Hilsman's tendency to go circumvent proper channels and by the friction Hilsman caused with the military, and as vice president, Johnson had not liked Hilsman's brashness or his policies.[28] Kennedy as Hilsman's protector was gone, and Johnson determined that he wanted Hilsman out.[28]

At the same time, Hilsman disagreed with Johnson's approach to the Vietnam War, viewing the new president as primarily seeking a military solution there rather than a political one.[29] Not liking anyone to quit outright, the president offered the position of U.S. Ambassador to the Philippines, but Hilsman declined.[28][30] And while Hilsman would later say that he had initiated the resignation, Secretary of State Rusk later presented a different picture: "I fired him".[31]

In any case, on February 25, 1964, the White House announced that Hilsman had resigned; the statement was front-page news in The New York Times with Hilsman claiming he had no policy quarrels with the current administration.[1] As his tenure ended, Hilsman argued in favor of continued perseverance in the conflict using a pacification-based counter-insurgency strategy,[32] but against increased military action against North Vietnam, saying that until the counter-insurgency efforts had demonstrated improvement in the South, action against the North would have no effect on the Communists.[26] His stance lost out within the administration to those who advocated the virtues of air power.[26] Hilsman's last day in office was March 15, 1964. He was replaced at the Bureau of Far Eastern Affairs by William Bundy.

In his resignation letter, Hilsman had said that he considered university teaching his "basic profession".[1]
Hilsman became a professor at Columbia University in 1964,[18] joining the Department of Public Law and Government within its School of International Affairs.[7] The course he gave on foreign policy decision-making became known for the anecdotes he told about the famous figures in the Kennedy administration and for the political theory he introduced in explanation.[33][34] Indeed, Hilsman became known as one of the expansive "Kennedy network",[35] and his office at Columbia was adorned with Kennedy-era momentos.[36]

Professor Hilsman (second from right) at a conference at the United States Military Academy at West Point in December 1969, with syndicated columnist Joseph Kraft, U.S. Representative from Ohio Robert A. Taft, Jr., and Colonel Amos A. Jordan, Jr.

Hilsman was one of the institute's most prolific book authors.[17] Of particular note was his 1967 work To Move a Nation: The Politics of Foreign Policy in the Administration of John F. Kennedy, which combined a theoretical political science approach with a personal memoir.[29] It was the first book by a U.S. maker of policy to dissent on the course of the Vietnam War.[30]The New York Times Book Review called it a "highly informative study of the internal and external forces that shaped much of American foreign policy" and said that "Hilsman makes many wise and perceptive comments on the politics of policy-making."[29]To Move a Nation became a National Book Award finalist[40] and has been viewed as influential.[6] His 1971 volume, Politics of Policy Making in Defense and Foreign Affairs: Conceptual Models and Bureaucratic Politics, was used as the textbook for his class[34] and went through three editions.

Hilsman continued to speak publicly, in print and on television, regarding what he thought should be done in Vietnam, such as in August 1964, when he warned against over-militarizing the conflict,[6] and in mid-1967, when he said the war was not politically "winnable" and that the U.S. should scale down its military involvement and stop the ongoing bombing campaign against the North.[41] He consistently maintained that had Kennedy lived, he would not have escalated the war the way Johnson did.[6] Hilsman was an ardent supporter of Robert F. Kennedy and his 1968 presidential campaign, serving as one of the experts advising the younger brother.[42] He was part of a large "brain trust" of advisers to Kennedy during the crucial Democratic California primary in June 1968;[43] that eventual campaign victory ended with another assassination.

Hilsman retired from Columbia in 1990 upon reaching the then-mandatory retirement age of 70.[34] Reflecting upon his life, he said, "I've been doing the same thing in the military, on Capital Hill, and at Columbia. The content is the same. ... Of all my careers, I think university teaching is the most satisfying."[34] He and his course, "The Politics of Policy Making", were not directly replaced.[34] The variety of careers Hilsman had once led U.S. Senator Claiborne Pell to compare him to Lawrence of Arabia.[6]