Gantner was a 12th-round draft pick in the 1974 amateur draft. Gantner spent two years in the minor leagues before being called up to the majors. He was best known for his consistent play, which culminated in a respectable .274 career batting average after 17 seasons. Gantner, or "Gumby" as he was affectionately known because of the way he turned double plays, was the second baseman for the Brewers, who won the American League pennant in 1982 but fell in seven games to the St. Louis Cardinals in the World Series. Gantner scored the go-ahead and ultimate winning run in the bottom of the seventh inning of the fifth and decisive game of the ALCS, scoring behind Charlie Moore on Cecil Cooper's RBI single. He comprised a stellar infield throughout much of the 1980s beside Hall of Famers Robin Yount (SS/OF), Paul Molitor (3B), and former Brewer great Cecil Cooper (1B).

Gantner rarely hit home runs. On September 3, 1991, he hit his first home run since 1987, ending a streak of 1,762 at-bats without one. Gantner's last career home run came on August 14, 1992, in the 13th inning, against Boston Red Sox reliever Jeff Reardon. Gantner hit the first pitch into the right field bleachers in Milwaukee County Stadium, giving the Brewers an 8-7 victory. Brewers announcer Bob Uecker had the call with, "Here's the pitch to Gantner. He hits one to right and deep. Get up. Get up. Get outta here - gone! A dinger for Klinger! And this game is over. Woah, Jimmy Gantner!"

Gantner pitched an inning of relief against the Kansas City Royals on August 29, 1979, allowing two hits. Despite the game being a blowout, he was almost ejected for arguing balls and strikes with the umpire.

Gantner coached several years after he retired as a player. He currently runs Hale Park Automotive Services in Hales Corners, Wisconsin. He also is employed by the Brewers' organization. Jim also is partial owner to a bar in Eden, Wisconsin called "Scuds Buds."