Koharski began his officiating career in the World Hockey Association in 1975, as a 19-year-old linesman. (He is the final player or official from the WHA to be involved in professional hockey in an on-ice capacity.) In 1976, he was signed by the American Hockey League while working with a local dairy farm, for which he was milking cows. He began his NHL career 1977, as a linesman, before moving to the referee role in 1980.

Koharski is perhaps best known for an incident that occurred following a 1988 playoff game between the Boston Bruins and the New Jersey Devils. New Jersey coach Jim Schoenfeld was infuriated with Koharski’s performance and waited for him in the hallway near the officials’ dressing room. Koharski fell, but believed Schoenfeld had pushed him. Koharski yelled several times after Schoenfeld "You're done", implying that he would be suspended over the incident, and said he hoped the incident was on tape. Schoenfeld responded by yelling in Koharski's direction, "Good, because you fell, you fat pig! Have another doughnut! Have another doughnut!" (A videotape of this incident recorded by WABC-TV has become part of sports television lore ever since, usually when discussing coaches losing their tempers.)

The NHL responded by suspending Schoenfeld, but the Devils took legal action and were granted a court order staying the suspension. The National Hockey League Officials' Association stood with Koharski; its members said they would not take the ice in the next game between the Devils and the Bruins if Schoenfeld was allowed to coach. This resulted in the NHL being forced to use low-level replacement officials for the game, with the two linesmen wearing Devils-coloured long pants (at the time, green pants with red-and-white stripes), along with white helmets and yellow practice jerseys over their street clothes.