Born in Starkville, Mississippi, Bell joined the St. Louis Stars of the Negro National League as a pitcher in 1922. By 1924, he had become their starting center fielder, and was known as an adept batter and fielder, and the "fastest man in the league". After leading the Stars to league titles in 1929, 1930, and 1931, he moved to the Detroit Wolves of the East-West League when the Negro National League disbanded. Detroit soon folded, leaving Bell to bounce to the Kansas City Monarchs and the Mexican winter leagues until finding a home with the Pittsburgh Crawfords in the reorganized NNL. In Pittsburgh, he played alongside Ted Page and Jimmie Crutchfield to form what is considered by many to have been the best outfield in the Negro Leagues. Bell left the Crawfords in 1938 to return to Mexico, coming back to baseball in the United States in 1942 to play for the Homestead Grays, who won Negro League titles in 1942, 1943, and 1944 with his help. He last played for the semi-pro Detroit Senators in 1946. He coached for the Monarchs in the late 1940s, managing their barnstorming "B" team, scouting for the club, signing prospects, and teaching the ins and outs of the game to future major-league baseball greats Ernie Banks, Jackie Robinson, and Elston Howard, among others.

Because of the opposition the Negro Leagues faced, and because of the lack of reliable press coverage of many of their games, no statistics can be given for Bell with any accuracy. What is undeniable is that Bell was considered to be one of the greats of his time by all the men he played with (including Satchel Paige and Josh Gibson). He is recorded as having rounded the bases in 13.3 seconds, and claimed he could make it in 12 under good conditions. As Paige himself noted in his autobiography, Maybe I'll Pitch Forever, "If Cool Papa had known about colleges or if colleges had known about Cool Papa, Jesse Owens would have looked like he was walking."

Paige also liked to tell a tall tale referencing one hotel at which he and Bell stayed, in which there was a short delay between flipping the light switch off and the lights actually going off, sufficient for Bell to jump into bed in the interim. Leaving out the explanatory details, Paige liked to say that Bell was "so fast he can turn off the light and be in bed before the room gets dark!"

Many tales exist of "Cool Papa". For example, one claims that Bell scored from second base on a sacrifice fly. Another states that he went from first to third on a bunt, while another states that he stole two bases on a single pitch. There are many other, possibly exaggerated anecdotes about Bell: He was once clocked running a full trip around the bases in 11 seconds. Perhaps the most unlikely was that he was once called out for being hit by his own batted ball while trying to slide into second base.

"Cool Papa" Bell died in his home on Dickson Street in St. Louis, Missouri at age 87. In his honor, the city renamed Dickson Street as "James 'Cool Papa' Bell Avenue". He has also been inducted into the St. Louis Walk of Fame.

Also named for him is Cool Papa Bell Drive, the road leading into the Mississippi Sports Hall of Fame and Museum in Jackson, of which he is a member. The Hall and Drive are adjacent to Smith-Wills Stadium, longtime home of the Jackson Generals of the Texas League, now home to the Jackson Senators of the independent Texas-Louisiana League.

In 1999, Cool Papa Bell was ranked 66th on The Sporting News' list of the 100 Greatest Baseball Players, one of five players so honored who played all or most of his career in the Negro Leagues, and was nominated as a finalist for the Major League Baseball All-Century Team.