His voice is perhaps best known from the Schoolhouse Rock! cartoons of the 1970s, such as "Conjunction Junction" and "I'm Just a Bill". Later, he parodied his own performance in "I'm Just a Bill" in an episode of The Simpsons called "The Day the Violence Died", where he is just an "amendment to be." "I'm Just a Bill" was also parodied in the Family Guy episode "Mr. Griffin Goes to Washington". Here, the bill on Capitol Hill begins singing the opening lyrics, when a janitor comes up and stabs him with a stick, then throws him away. He also parodied his voice in the Family Guy episode "Running Mates" where Peter made changes to the sex education curriculum with a spin-off of the "Conjunction Junction" skit, "Vagina Junction". Sheldon returned to the Schoolhouse Rock! series for a 2002 episode titled "I'm Gonna Send Your Vote to College", explaining the electoral college process, and distributed on the series' DVD collection that same year. Sheldon sang and played trumpet for the new segment.

Sheldon voiced "Louie The Lightning Bug" in a series of animated musical public service announcements aimed at children from the 1980s, promoting safety with electricity.[1] In 2001, the "Louie the Lightning Bug" videos were updated with new voice-overs by Sheldon and new music tracks produced by Mark Harrelson, with updated musical arrangements by Ray Reach.

He made an appearance in the 1994 film Radioland Murders as the ill-fated trumpet player Ruffles Reedy, who becomes a victim of the gruesome goings-on during a 1939 radio show. In 2004, Jack performed live at the end of ALF's Hit Talk Show. He appeared in one episode of Johnny Bravo as the Sensitive Man. He sang a few songs in the episode similar to the Schoolhouse Rock! style. He has also appeared on The Ellen DeGeneres Show.