While in high school and college, Cohen was a basketball player and was named to the Maine all-state high school and college basketball team, and at Bowdoin was inducted into the New England All-Star Hall of Fame. Cohen attended law school at the Boston University School of Law, graduating with a Bachelor of Laws degree cum laude in 1965.

During his first term in Congress, Cohen became deeply involved in the Watergate investigation. As a member of the House Judiciary Committee, he was one of the first Republicans to break with his party, and voted for the impeachment of President Richard Nixon. During this time, Time magazine named him one of "America's 200 Future Leaders".

After three terms in the House, Cohen was elected to the U.S. Senate in 1978, defeating incumbent William Hathaway in his first bid for reelection. Cohen was reelected in 1984 and 1990, serving a total of 18 years in the Senate (1979–1997). In 1990, he defeated Democrat Neil Rolde. Cohen developed a reputation as a moderate Republican with liberal views on social issues, and has been described as "a career-long maverick with a reputation for fashioning compromise out of discord."[6][7]

In 1994 Cohen investigated the federal government's process for acquiring information technology, and his report, Computer Chaos: Billions Wasted Buying Federal Computer Systems, generated much discussion. He chose not to run for another Senate term in 1996; Susan Collins, who had worked for Cohen, was elected to succeed him. (Former Maine senator, Olympia Snowe, had also worked for Cohen while he was in the House of Representatives.)[8]

On December 5, 1996, President Clinton announced his selection of Cohen as secretary of defense. Cohen, a Republican about to retire from the United States Senate, was the "right person," Clinton said, to build on the achievements of William Perry, "to secure the bipartisan support America's armed forces must have and clearly deserve." In responding to his nomination, Cohen said that during his congressional career he had supported a nonpartisan national security policy and commended the president for appointing a Republican to his cabinet.

During his confirmation hearings, Cohen said he thought on occasion he might differ with Clinton on specific national security issues. He implicitly criticized the Clinton administration for lacking a clear strategy for leaving Bosnia and stated that he thought U.S. troops should definitely be out by mid-1998. He also asserted that he would resist further budget cuts, retain the two regional conflicts strategy, and support spending increases for advanced weapons, even if it necessitated further cuts in military personnel. Cohen questioned whether savings from base closings and acquisition reform could provide enough money for procurement of new weapons and equipment that the Joint Chiefs of Staff thought necessary in the next few years. He supported the expansion of NATO and looked on the proliferation of weapons of mass destruction as the most serious problem the United States faced.

One of Cohen's first major duties was to present to Congress the Fiscal Year 1998 Defense budget, which had been prepared under Secretary Perry. Cohen requested a budget of $250.7 billion, which represented 3 percent of the nation's estimated gross domestic product for FY 1998. Cohen stressed three top budget priorities: people (recruiting and retaining skilled people through regular military pay raises, new construction or modernization of barracks, and programs for child care, family support, morale, welfare, and recreation), readiness (support for force readiness, training, exercises, maintenance, supplies, and other essential needs), and modernization (development and upgrading of weapon and supporting systems to guarantee the combat superiority of U.S. forces). This meant increasing the funds available for procurement of new systems, with the target set at $60 billion by FY2001.

When he presented the FY1998 budget, Cohen noted that he would involve himself with the Quadrennial Defense Review (QDR), which would focus on the challenges to U.S. security and the nation's military needs over the next decade or more. When the QDR became public in May 1997, it did not fundamentally alter the budget, structure, and doctrine of the military. Many defense experts thought it gave insufficient attention to new forms of warfare, such as terrorist attacks, electronic sabotage, and the use of chemical and biological agents. Cohen stated that the Pentagon would retain the "two regional wars" scenario adopted after the end of the Cold War. He decided to scale back purchases of jet fighters, including the Air Force's F-22 Raptor and the Navy's F/A-18E/F Super Hornet, as well as Navy surface ships. The review included cutting another 61,700 active duty service members—15,000 in the Army, 26,900 in the Air Force, 18,000 in the Navy, and 1,800 in the Marine Corps, as well as 54,000 reserve forces, mainly in the Army National Guard, and some 80,000 civilians department-wide. Cohen also recommended two more rounds of base closings in 1999 and 2001. The Pentagon hoped to save $15 billion annually over the next few years to make possible the purchase of new equipment and weapon systems without a substantial budget increase above the current level of $250 billion.

As he settled into office, Cohen faced the question of the expansion of the North Atlantic Treaty Organization, which he supported, and its relationship to Russia. At a summit meeting between President Clinton and Russian PresidentBoris Yeltsin in Helsinki, Finland, in March 1997, Yeltsin acknowledged the inevitability of broader NATO membership. Two months later he agreed, after negotiations with NATO officials, to sign an accord providing for a new permanent council, to include Russia, the NATO secretary general, and a representative of the other NATO nations, to function as a forum in which Russia could air a wide range of security issues that concerned that country. Formal signing of this agreement would pave the way for a July 1997 invitation from NATO to several nations, probably including Poland, Hungary, and the Czech Republic, to join the organization.

The proposed U.S. missile defense system received attention at the Helsinki summit, where Clinton and Yeltsin agreed to an interpretation of the 1972 Anti-Ballistic Missile Treaty allowing the United States to proceed with a limited missile defense system currently under development. Specifically, Clinton and Yeltsin agreed to distinguish between a national missile defense system, aimed against strategic weapons, not allowed by the ABMT, and a theater missile defense system to guard against shorter range missile attacks. Some critics thought that any agreement of this kind would place undesirable limits on the development of both theater and strategic missile defenses. The Helsinki meeting also saw progress in arms control negotiations between the United States and Russia, a matter high on Cohen's agenda. Yeltsin and Clinton agreed on the need for early Russian ratification of the Second Strategic Arms Reduction Treaty (START II) and negotiation of START III to make further significant cuts in the strategic nuclear arsenals of both nations.

Cohen (left) and Japanese Prime Minister Yoshiro Mori pose for photographers prior to their meeting at the Kantei building in Tokyo, Japan, on September 22, 2000.

The continuation, at least until mid-1998, of the existing peacekeeping mission involving U.S. forces in Bosnia and the possibility that other such missions would arise worried Cohen, who earlier had expressed reservations about such operations. Humanitarian efforts that did not involve peacekeeping, such as in Rwanda in the recent past, also seemed likely. Other persistent national security problems, including tension with Iraq in the Persian Gulf area, Libya in North Africa, and North Korea in East Asia, could flare up again, as could the Arab–Israeli conflict.

In preparing future budgets, the challenge would be to find the right mix between money for operation and maintenance accounts on the one hand and modernization procurement funds on the other, while facing the prospect of a flat DoD budget of about $250 billion annually for the next decade or so. A relatively new problem that could affect the DoD budget was vertical integration in the defense industry. It occurred on a large scale in the 1990s as mergers of major defense contractors created a few huge dominant companies, particularly in the aerospace industry. They were called vertical because they incorporated most of the elements of the production process, including parts and subcomponents. Cohen and other Pentagon leaders began to worry that vertical integration could reduce competition and in the long run increase the costs of what the Department of Defense had to buy.

Finally, Cohen had to address social issues that engaged the widest public interest. These issues included the status and treatment of lesbians and gays in the military, the role of women in combat as well as in other jobs in the services, racism, and sexual harassment.

On January 5, 2006, he participated in a meeting at the White House of former Secretaries of Defense and State to discuss United States foreign policy with Bush administration officials.

Cohen has written several books, including mysteries, poetry, and (with George Mitchell) an analysis of the Iran-contra affair. He is a Chairman Emeritus of the US-Taiwan Business Council. The Washington Post ran an article entitled "From Public Life to Private Business" about Cohen's abrupt transition to the business of Washington lobbying within "weeks of leaving office."[11] It discussed the affairs of the Cohen Group in greater detail and while alleging no specific impropriety, took a generally negative view of the former Senator and Secretary of Defense.

On August 21, 2006, Cohen's novel, Dragon Fire, was released. The plot revolves around a secretary of defense who contends with a potential nuclear threat from a foreign country. In December 2006 he released a memoir with his wife, author Janet Langhart, entitled Love in Black and White. It is a memoir about race, religion, and the love Langhart and Cohen share over similar life circumstances and backgrounds.[12] On August 22, 2006, Cohen appeared on The Daily Show to promote his novel.[13]

On August 25, 2006, Cohen was interviewed by Brian Kilmeade on Fox & Friends First, primarily to promote his new novel, but towards the end of the broadcast he said: "I think there should be a commitment to universal service. I think that only a few people are really committed to this war against terrorism.... We ought to have a real call to national service to commit ourselves to some form of public service...to put us on a war footing mentality."

On January 3, 2007, Cohen appeared on CNN to support John Shalikashvili's op-ed in support of ending the policy known as 'Don't ask, don't tell' saying, "The vast majority of service members are personally comfortable working and interacting with gays and lesbians, and there is only so long that Congress can ignore the evidence".[14]

Cohen serves as an Advisory Board member for the Partnership for a Secure America and is also a member of the ReFormers Caucus of Issue One.[19] In addition, he serves as a board member of the U.S.-China Business Council, having served as vice chairman between 2011-2013, and the U.S.-India Business Council.[20]

Cohen filed for divorce from his first wife, Diana Dunn, on February 15, 1987.

On February 14, 1996, Cohen and Janet Langhart[22] were married. Janet had actually refused multiple proposals due to concerns about potential reactions in Senate reelection campaigns of an interracial marriage. Langhart is a former model, Boston television personality, and BET correspondent. Janet Langhart was known as the "First Lady of the Pentagon" during Cohen's tenure as Secretary.[23]

On the afternoon of June 10, 2009, Cohen was present at the U.S. Holocaust Museum, waiting for his wife Janet Langhart, for the world premiere of her one-act play, Anne and Emmett. The play imagines a conversation between Anne Frank and Emmett Till.[26] While Cohen waited, an elderly man with a long gun attacked the facility, fatally shooting a security guard before being wounded himself by the other guards. Cohen and Langhart were not injured, and appeared on CNN that afternoon to tell what they had seen and respond to the shooter's racist beliefs. The man was identified as James W. von Brunn, 88, of Annapolis, a longtime "hard-core" supremacist whose Internet writings contain extensive criticism against Jews and African Americans.[27] Langhart's play had been promoted in The Washington Post the week before, and was being presented in honor of the eightieth anniversary of Anne Frank's birth.[28]