Drafted by the Montreal Expos in after playing in the Cape Cod League, he quickly came to the majors with them by . His years with the Expos were uneven, with a reasonable performance followed by a solid and a mediocre .

That year, the San Francisco Giants traded Mark Leiter, then the organization's most prominent starting pitcher, to the Expos for Reuter and Tim Scott. Scott was a disappointment, posting an 8.24 ERA with the Giants, but Rueter blossomed into one of the Giants' most dependable starters and was with the team for nine seasons. For many fans, Rueter's defining moment as a Giant was his gutsy bullpen performance in Game 2 of the NLDS, where he relieved starter Shawn Estes after Estes sprained his ankle on a horrible baserunning play and proved far more effective in relief than Estes did starting.
, the year of the Giants last World Series appearance, was statisticly Rueter's best year. He went 14-8 with a 3.23 ERA. Rueter was the winning pitcher in Game 4 of the 2002 World Series; he went 6 innings surrendering 3 earned runs. Rueter also pitched shut out ball in relief of Liván Hernández in Game 7 of the 2002 World Series, but the Giants failed to score enough runs to come back.

However, he began to struggle in with a 9-12 record and a 4.73 ERA. In , after posting a 2-7 record and 5.95 ERA the Giants designated him for assignment. His nine year tenure in San Francisco ended with some controversy. Rueter complained about having to pitch out of the bullpen and only pitching 3 times in his last 41 days as a Giant.

Throughout his career, Rueter was primarily a control and changeup pitcher. His fastball rarely hit 90 mph. He threw changeups, fastballs, sinkers, curveballs, cut fastballs, and sliders. Some credited the effects of the QuesTec umpiring system to his decline, as Rueter's success came mostly from being able to "paint the corners" of the strike zone and the system, which encouraged umpires to call a tighter strike zone, effectively taking that ability away from him. Rueter was never a strikeoutpitcher; he only struck out more than a hundred batters twice in his career.

In 2000, Rueter was the first pitcher to start a major league game at Pacific Bell Park in San Francisco.

Rueter's trademarks were his fast-paced pitching style and his large ears. Rueter resides in Nashville, Illinois, with his wife and two daughters and his home is famous for its "Shed", a large recreational facility filled with games and sports memorabilia. Rueter also resided at the Shed during the off-seasons of his playing career. When the Giants made trips to St. Louis during the baseball season, Rueter invited the team to relax at his Shed.

On March 6, , Rueter announced his retirement from the game after 13 seasons. He retired as the winningest left-handed pitcher in San Francisco Giants history, with 105 of his 130 career wins in a Giants uniform. Rueter is the 20th winningest pitcher in Giants franchise history. He is the 3rd winningest pitcher in San Francisco Giants history. He made the third most career starts in San Francisco Giants history. Only Juan Marichal and Gaylord Perry had more career starts and wins. The Giants honored Rueter's career during pregame cermonies on "Kirk Rueter Day" at SBC Park on August 19, 2006, by giving Rueter a lifesize bobblehead of his likeness and giving him and his family a trip to Hawaii.