Arnsparger was born in Paris, Kentucky in 1926. He attended Paris High School, and became connected with the school's longtime football and basketball coach, Blanton Collier. The relationship would have a major impact on his future career.

In 1954, Arnsparger re-connected with Collier, who had been hired as head football coach at the University of Kentucky in Lexington, Kentucky. Arnsparger remained at Kentucky for the next eight years until Collier was fired on January 2, 1962. During the 1959 season, he was joined on the coaching staff by a young coach who had served at the University of Virginia the previous year. That coach was Don Shula, with the two coaches forging a strong bond that would tie them for much of the next quarter century.

When Shula left to become head coach with the Miami Dolphins after the end of the 1969 NFL season, he brought along Arnsparger, who was promoted to defensive coordinator. In just two seasons, the formerly moribund team had reached the Super Bowl, with Arnsparger fashioning what became known as the "No-Name Defense." World championships in each of the next two seasons, including an undefeated season during 1972, made Arnsparger a prime candidate for a head coaching position.

Just two days after his dismissal from the Giants, Arnsparger was rehired by Shula and was restored to his previous position as Miami's defensive coordinator while adding the title of assistant head coach.[2] In the team's first game under his leadership, the Dolphins won a 10–3 defensive battle with the New England Patriots, who had averaged thirty points per game entering the contest.

Miami finished the 1976 NFL season with a 6–8 mark, then narrowly missed a playoff berth the following season. During the next two seasons, the Dolphins reached the postseason, but dropped their first playoff game. During the strike-shortened 1982 NFL season, Miami reached Super Bowl XVII, but dropped a 27–17 decision to the Washington Redskins. Bill Arnsparger again had created an elite defensive unit, known as the Killer B's (so named because of the number of surnames beginning with "B" on the Dolphins defense).

On December 2, 1983, Arnsparger was hired as head coach at Louisiana State University, but finished his season with the Dolphins. As the Tigers' head coach, Arnsparger led LSU to two Sugar Bowl berths in three seasons, in 1984 and 1986. Both times, their opponent was Nebraska. The 1986 squad was the school's first Southeastern Conference champion since 1970. Shortly after the final regular season game in 1986, Arnsparger announced he was resigning to become the athletic director at the University of Florida.

At Florida, Arnsparger was tasked with cleaning up an athletic department which had been roiled by NCAA violations and subsequent punishments. The Florida Gators football team has been found in violation of many NCAA rules in the early 1980s and was still suffering under significant sanctions and probation when Arnsparger arrived, a situation which weakened the financial strength of the entire athletic department.

More problems surfaced in 1989. Head football coach Galen Hall was accused of committing minor rules violations, which became a major issue with the NCAA because of the just-completed probation. Hall was forced to resign during the season in October 1989, and after an NCAA investigation, the football program was placed back on probation the following year.[3] At about the same time, questions about the Florida Gators men's basketball program also surfaced. Allegations that the coaching staff had paid star players and allowed sports agents to do the same led to the forced early retirement of long-time head basketball coach Norm Sloan and his entire coaching staff less than a month before the start of the 1989-90 season, less than three weeks after the head football coach had resigned.[4] Sloan subsequently claimed that the allegations were false and that Arnsparger's zealous attempts to clean up Florida's athletic department turned into an unfair "witch hunt".[5] Former Tennessee head coach Don DeVoe was quickly hired as interim coach.

Despite having both of the university's marquee sports placed on NCAA probation in 1990, Arnsparger was able to quickly rebuild Florida's athletic program by establishing better compliance procedures and by hiring new coaches who would quickly bring success while following NCAA guidelines. In December 1989, Arnsparger hired Steve Spurrier, then the Duke University head coach and the Gators' Heisman Trophy-winning quarterback in 1966. Over his twelve years at Florida, Spurrier became the school's all-time wins leader and brought the school its first six conference titles and the 1996 national championship. After a disastrous 1989-90 season under DeVoe, Arnsparger hired Lon Kruger as Florida's next full-time basketball coach. Kruger led the Gators to their first Final Four appearance in 1994 and set the stage for later and greater success under Billy Donovan.

During his time at Florida, Arnsparger was thought by some athletic department staff and boosters to be "domineering" and that, keeping with his background in coaching, he was "inflexible... sticking by his game plan at all costs.".[6] In hindsight, observers give him credit for "helping to right the ship during a very difficult time at the University of Florida" and setting the program up for unprecedented success after his tenure.[7]

On January 13, 1992, Arnsparger resigned to become the defensive coordinator of the San Diego Chargers. During his three seasons with the Chargers, the team's defense showed marked improvement, culminating with a berth in Super Bowl XXIX. Just days after the team's Super Bowl appearance, Arnsparger announced his retirement, citing the prostate cancer surgery he had undergone the year before.