Discussions regarding German rearmament took place as early as 1946, but the British government opposed anything more than an armed and mobile police force, while the French government did not wish to see Germans rearmed in any way. Serious negotiations over the creation of a German military began in 1950, however, when the Korean War stretched Western military resources thin. Once again, French resistance was the major obstacle.

The Paris Accords of May 1955 overcame these obstacles by creating a Federal Republic of Germany (FRG, West Germany) army, the Bundeswehr, that was firmly under civilian and Allied control. In stark contrast to previous periods in German history, the Bundeswehr was under intense oversight by parliamentary committees and a public wary of military institutions. Soldiers would be subject to civil law in all matters that were not strictly military. All Bundeswehr units were designated to the North Atlantic Treaty Organization (NATO) for action, with the German government retaining peacetime control. Weapons and military matériel were imported from the United States, Britain, and France on a large scale until the late 1960s, when domestic military production developed.

Under the 1955 agreement, the United States and Great Britain committed themselves to maintaining a troop presence in Germany. NATO's current European partners would provide naval and air forces to complement British and American commitments. In return, West Germany would provide twelve mixed divisions (approximately 340,000 men) for the common defense of Europe by 1959, with a final strength of 500,000. Each infantry division would have three combat commands with three motorized infantry battalions and an armor battalion each; antiaircraft, engineer, communications, and reconnaissance battalions; a company of aircraft; a military police company; and combat support from three light artillery battalions and one medium artillery battalion. Tank divisions contained two armored and three mechanized infantry battalions in each combat command. Two of the divisions would be further specialized for airborne and mountain operations.

A civilian screening process was created to recruit officers and noncommissioned officers (NCOs) from World War II veterans. NATO would be responsible for all war planning and for the direction of all operations. No German general staff was permitted, on grounds that it might revive German militarism. The Bundeswehr was allowed an operations staff but on the condition that officers rotated through such duty periodically. Planners believed that half of the necessary manpower for the divisions would be volunteers, while a draft would provide the remaining numbers.

The first West German military volunteers reported for duty in November 1955, and the Law for Compulsory Service passed the West German parliament on 7 July 1956. It required all men between the ages of eighteen and forty-five to serve for twelve months, with provisions allowing conscientious objectors to fulfill their obligation through alternative means, usually administered at the state and local levels. Further exemptions, thought to include some 10 percent of eligible males, covered the sons of deceased and wounded veterans, economic hardship cases, clergy, and those deemed unfit for service. Men who became officers or NCOs would remain in the reserves, known as the Territorial Army, until age sixty. The term of service was increased to eighteen months in 1962 at the height of the Berlin Crisis (1958–1963). It was then reduced to fifteen months in 1972, as population growth began to provide more than adequate numbers of draftees.

The West German government experienced some difficulty in providing the promised troops. Part of the reason was monetary. Between 1956 and 1989, the share of defense-related expenditures in the West German federal budget never sank below 12 percent. It reached its zenith in the early 1960s when it was one-third of the overall budget, yet most of the money was going to purchase matériel. On average, throughout the Cold War the West German government annually spent about 20 percent of its total budget on defense.

Beyond that, rearmament was never popular in West Germany. In October 1956, West German Defense Minister Franz Josef Strauss announced that manpower targets had been reduced from 96,000 to 80,000 for 1956 and from 240,000 to between 175,000 and 200,000 for 1957 largely because volunteers were not coming forward in the numbers anticipated. Only eleven of the twelve German divisions originally planned for 1959 were filled out and under NATO control by 1963. It took until the mid-1970s before West Germany's armed forces reached the final benchmark of 500,000 troops envisaged in 1955. In 1975, the Bundeswehr contained 345,000 soldiers, 110,000 air force personnel, and 39,000 seamen. At any given time, only 48 percent of the German armed forces were volunteers, far short of the 55 percent target that the West German governments maintained. Recruitment of long-term (twenty-one-month) volunteers and of NCOs in particular continuously fell far short of expectations, although the federal government offered numerous incentives such as vocational education programs and career guarantees.

Popular opposition to the Bundeswehr, although always strong, increased notably in early 1957 when the United States indicated that it would arm its European allies with nuclear weapons as part of a shift in NATO strategy. Before 1957, NATO planned a forward defense along the German frontier. Chancellor Konrad Adenauer had pushed for and won the integration of German units all along the line rather than having the Bundeswehr assigned a particular sector. He had also readily accepted a ban on atomic, biological, and chemical weapons for German units in order to make rearmament more palatable to the public and to the Social Democratic opposition. NATO's new strategy of nuclear deterrence, officially adopted on 21 March 1957, thus set off a maelstrom in German politics that continued into the 1960s.

Adenauer renounced German construction of nuclear weapons and declared that Germany would not accept national control over such weapons. NATO responded by introducing the two-key system, whereby German units possessed nuclear capabilities but the nuclear warheads and launchers remained under Allied control. After winning by-elections in Nordrhein-Westfalen in July 1958, Adenauer's government announced that it was prepared to equip the Bundeswehr with atomic weapons. German units received Matador rockets and nuclear-capable artillery pieces so that they could fight either a conventional or a tactical nuclear war. Bundeswehr units were also divided into either the six armored infantry or four purely armored divisions to facilitate mobility and, supposedly, increase defense against nuclear attack.

Even before this restructuring was complete, however, NATO, with some prompting from West Germany and led by U.S. President John F. Kennedy's administration, moved to a strategy of flexible response. This deemphasized the role of nuclear weapons where German forces were concerned by creating a multilateral force that deployed nuclear-armed Polaris submarines. Two of the German armored infantry divisions created in 1957 were reorganized as straight infantry divisions, and the Bundeswehr's deployed division strength was increased by about 10,000 men to compensate for its reduced nuclear role. The question of arming West Germany with nuclear weapons was shelved for good when the coalition government led by Willy Brandt and the Social Democratic Party (SPD) signed the Nuclear Non-Proliferation Treaty (NPT) in November 1969.

In the late 1970s, however, NATO plans to station intermediate-range nuclear missiles in Germany renewed the debate over atomic weapons. German Chancellor Helmut Schmidt, who had earlier been one of the principal advocates of flexible response in Germany, declared in early 1978 that West Germany would accept nuclear weapons only if other NATO countries did as well. U.S. President Jimmy Carter eventually led a NATO climbdown on the issue, agreeing to a two-track policy that tied deployment in Germany to Soviet deployments in Eastern Europe. In the 1980s his successor, President Ronald Reagan, worked closely with German Chancellor Helmut Kohl to manage the deployment of Pershing missiles in Germany. Nuclear weapons always remained under NATO and U.S. control, and the German and American governments continued to cooperate in reducing the number of nuclear weapons stationed in West Germany.

Since the collapse of the Soviet Union and the end of the Cold War, the Bundeswehr has changed immensely. Despite the integration of the former East German Army since 1990, overall troop strength has been roughly cut in half. Although during the Cold War West German troops were never deployed on foreign soil except for training exercises and joint NATO maneuvers, the Bundeswehr has undertaken several foreign peacekeeping missions since 1991, most notably in the Balkans. German armed forces now serve as part of different international missions on several continents, most notably in the Balkans and in Afghanistan.