Banks was selected as the thirteenth pick in the first round of the 2003 NBA draft by the Memphis Grizzlies; however, he was then traded to the Boston Celtics along with Kendrick Perkins. He was traded to the Lakers in 2004 for Gary Payton, but returned to Boston after the trade was amended following Payton's refusal to take a physical. In Boston, Banks was known for his good defense and ball handling; Allen Iverson, a division rival point guard, named Banks as the toughest defender he faced in his career.[2] Coach Doc Rivers had high hopes for rookie Delonte West, who also played point guard.

In 2007, the Suns spent training camp working Banks into a shooting-guard role when they were not seeking a trade, a tough proposition with Banks in the second year of a five-year, $21.3 million contract.