On February 18, 1991, Congressman Silvio Conte died one month after taking office for his 17th term. That June, Olver, who had been sworn in for his 10th term in the state Senate a few months earlier, in January 1991, narrowly defeated RepublicanSteven Pierce in a special election for the seat, becoming the first Democrat to win the seat since it changed from being the 13th district in 1893.[2][4] Olver resigned from the state Senate on June 15 and was sworn in June 18.[4][5]

Olver won election to a full term in 1992. In 1994, the year which Republican took control of the House for the first time in 40 years, Olver ran unopposed.[6] In 1996, he defeated State Senator and future Lieutenant and Acting GovernorJane Swift by a 53% to 47% margin.[7] After 1996, Olver continually won re-election with high margins above 60%.[8]

On October 26, 2011 Olver announced he would not seek re-election in 2012, clearing the way for any potential Democratic primaries due to Massachusetts losing a congressional seat after the 2010 United States Census.[9] His district had been eliminated in redistricting. The bulk of his former territory had been merged with the 2nd district of fellow Democrat Richard Neal, while his home in Amherst was drawn into the 2nd District (the old 3rd District) of fellow Democrat Jim McGovern.

Olver has been a staunch opponent of the United States involvement in Iraq, and opposed a United States military presence in Iraq since the 2002 authorization for the use of force in Iraq.[12] Olver has advocated for the removal of U.S. troops from Iraq, and has consistently voted against proposals to increase funding for military operations and increased deployment.[13] Olver stated that he believes that the United States should seek out a political solution in cooperation with the states neighboring Iraq, such as Iran and Syria, rather than pursuing a strategy based primarily on military means in order to create a stable and democratic Iraq.[12]

Olver has been critical of the United States lack of involvement regarding the genocide in Darfur. Olver was one of five members of Congress arrested April 28, 2006 after protesting the genocide outside the Sudanese Embassy.[14] Olver was arrested again in front of the Sudanese Embassy during a protest in 2012 along with other members of Congress and actor George Clooney.[15]

Olver co-sponsored H.R. 676 which would introduce a universal health insurance program with single-payer financing. In addition to supporting different pieces of healthcare reform legislation in the House, Olver has also strongly supported allowing federal funds related to health care programs to fund abortion operations.[17] In addition to supporting the inclusion of coverage for abortions in healthcare, Olver has consistently supported pro-choice legislation in the House and has received favorable ratings from pro-choice interest groups such as the NARAL Pro-Choice America, which gave Olver a rating of 100.[17][18]

Olver supports efforts to reduce the number of illegal immigrants entering the United States, but also believes in immigration reform so that qualified immigrants could more easily be granted entrance.[19] Olver also believes that illegal immigrants currently residing in the United States need to be provided with a pathway to citizenship, and that these immigrants should not have to return to their countries of origin before obtaining citizenship.[20]

Olver was married to Rose Richardson Olver, a Professor of Psychology and Sexuality, Women's and Gender Studies at Amherst College, from 1959 until her death in 2014, and they had one daughter named Martha.[21] He has lived in Amherst, Massachusetts, since 1963.[22]

On October 28, 2010 Olver was awarded the Soldier On Award presented by the veterans group, Soldier On. This award, created for Soldier On by sculptor Andrew DeVries, recognizes individuals whose leadership and actions have advanced the goal of ending veteran homelessness.[24]