Already admitted to the bar in 1919, before completing law school (later calling himself "a simple country lawyer"), Ervin entered politics straight out of Harvard.[1] Even before he had received his degree, Democrats in Burke County, North Carolina had nominated him in absentia for the North Carolina House of Representatives, to which he was elected in 1922, 1924, and 1930. In 1927, in his role as attorney for Burke County, NC, Ervin served as the legal advisor to the local sheriff during the hunt for Broadus Miller, a black man believed to have murdered a teenaged white girl. The county officials invoked the outlaw provision of the North Carolina constitution which permitted any citizen to kill a declared outlaw without formal charges being brought. Miller was shot down while being pursued and his body displayed in the local courthouse square. Ervin was also elected and served as a state judge in the late 1930s and early 1940s.

In 1956, Senator Ervin helped organize resistance to the 1954 Brown v. Board of EducationSupreme Court decision calling for desegregation of schools by drafting The Southern Manifesto; this influential document encouraged defiance of desegregation and was signed by all but a few of the Southern members of Congress.[7] (In his autobiography, Preserving the Constitution, Ervin said he later changed his mind on the Brown decision, stating that the decision, to the extent it eliminated mandatory segregation, was correct, but that forced integration, required under later decisions, was improper.)

Defenders of Ervin argue that his opposition to most civil rightslegislation was based on his commitment to the preservation of the Constitution in its pristine formulation, and a general belief that the Equal Protection Clause of the Constitution only applied to white men. He repeatedly stated the Constitution encapsulated civil, human and equal rights for all those he considered worthy. There is little if any evidence that he engaged in the racial demagoguery of many of his Southern colleagues. Some historians consider Ervin's position to be one of "cognitive dissonance" because he opposed federal legislation to combat race-based discrimination, but did not do so in harsh terms. While he once maintained that Americans were entitled to "their prejudices as well as their allergies", he did not seem to be motivated by prejudice himself, but more by his suspicion of federal power. Ervin said he disliked what the Warren Court "has done to the Constitution".[8]

On March 30, 1965, Ervin announced that he would offer a substitute to the Johnson administration's voting rights bill. Ervin referred to the administration's bill as cockeyed and unconstitutional and that his version would provide for federal registers being appointed in areas certified to having finds of racial discrimination as defined under the Fifteenth Amendment to the United States Constitution. Ervin said he would seek approval of the Senate Judiciary Committee and that he would carry the fight to the Senate floor in the event that the committee rejected his legislation.[9]

Ervin was also a staunch opponent of the Immigration and Nationality Act of 1965 which abolished nationality quotas beginning in 1968. He felt that the principle of tying allowed numbers of immigrants from a given country to the number of people who had ancestral origins in that country and lived in the United States should be retained.[10]

In November 1970, Ervin was one of three Senators (all from Southern states, the others being James Eastland and Strom Thurmond), to vote against an occupational safety bill that would establish a federal supervision to oversee working conditions.[11]

When the Senate voted on the Equal Rights Amendment (ERA) in 1971, Ervin proposed an amendment that would exempt women from the draft; Ervin's amendment to the ERA overwhelmingly failed.[12] However, he was a staunch opponent of the ERA and after it passed the Senate Ervin used his influence to dissuade the North Carolina General Assembly from ratifying it, maintaining that it was the "height of folly to command legislative bodies to ignore sex in making laws".[13]

Ervin gained lasting fame through his stewardship of the Senate Select Committee to Investigate Campaign Practices, also known as the Senate Watergate Committee, from the 1972 presidential election. Senate Majority Leader Mike Mansfield chose Ervin because it was unlikely Ervin was going to run for re-election in 1974 (and in fact did not),[15] had no aspirations beyond his office, had deep knowledge of the law and the Constitution, and because he was considered to be an even-keeled, conservative, independent-minded Democrat. President Nixon thought at first that Ervin might potentially be supportive, but that turned out to not be the case.

Following the 1972 elections, Ervin proposed five bills to limit the power of the presidency. Two restricted the President's ability to use funds for reasons other than their appropriated purpose, one allowed more congressional oversight of appointed officials, one banned the President from pocket vetoing legislation when Congress was not in session, and the final bill required the President to inform Congress of any executive agreements made with foreign governments.[16]

Ervin resigned in December 1974, just before his term ended. After retirement, Ervin practiced law, wrote several books, and appeared in various commercials for products. As a lawyer, he served as a co-counsel with Womble Carlyle Sandridge & Rice PLLC on several high-profile cases, including a successful appeal in Joyner v. Duncan.
[17] In 1973, Ervin was recorded on CBS Records on the LP record, Senator Sam at Home, which featured tracks of Ervin speaking his mind and telling anecdotes, separated by tracks of him singing popular songs. One of those songs, "Bridge Over Troubled Water" was released as a single,[18] and subsequently appeared on the 1991 compilation album Golden Throats 2.

In a 1964 essay called "The Naked Society", Vance Packard criticized advertisers' unfettered use of private information to create marketing schemes. He compared a recent Great Society initiative by then-president Lyndon B. Johnson, the National Data Bank, to the use of information by advertisers and argued for increased data privacy measures to ensure that information did not find its way into the wrong hands. The essay led inspired Ervin to fight Johnson's flagrant disregard for consumer privacy. He criticized Johnson's invasive domestic agenda and saw the unfiltered database of consumers' information as a sign of presidential abuse of power. Ervin warned that the "The computer never forgets".[24]

In an interview[26] on William F. Buckley's Firing Line program, Ervin suggested that people in public life need to have more "backbone", and Buckley playfully suggested Gordon Liddy as a model to which Ervin responded, "Well, Gordon Liddy has a little too much backbone. I'll have to admit that I have a sort of sneaking admiration for a fellow like Gordon Liddy that does have an excess of backbone. His backbone exceeds his intelligence, really."

Ervin was a staunch opponent of the polygraph calling the tests "20th century witchcraft":

Probably no instrument in modern time so lends itself to threats to constitutional guarantees of individual freedom as the polygraph or so-called lie detector. The threat of its use or the intimidation inherent in its use restricts free expression and communication of ideas, intrudes on an individual's subconscious thought, makes him fear to speak his thoughts freely, or compels him to speak against his will. To my mind, the entire purpose of these machines is to invade a man's mind and find what lurks in the innermost part of his mental consciousness for reasons which have nothing to do with his ability to perform a job. If the right of privacy means anything at all and if it is a right to be cherished in our society, it means that people should be entitled to have thoughts, hopes, desires, and dreams that are beyond the reach of a bureaucrat, an employer, or an electronic technician. This is something which enthusiasts for these machines do not seem to understand. They do not understand and they do not appreciate how important privacy is to each American, and as long as that lesson is not understood, we all will find our right to privacy constricted if not abrogated entirely. I propose this legislation to ban the use of the polygraph for employment purposes in the hopes that Congress will pause for a moment, step back, and take a long look at the issues involved in the unrestrained use of the polygraph. Legislation is necessary to bring some order and control to the practice ...[27]

He also famously said of religion and government:

Political freedom cannot exist in any land where religion controls the state, and religious freedom cannot exist in any land where the state controls religion.[28]