NEW YORK (AP) - LeBron James understands it will take time, but he wants Donald Sterling out of the NBA.

And he said Sunday that NBA players believe nobody in Sterling's family should be able to own the Los Angeles Clippers if he's gone.

Sterling has been banned for life for making racist comments and Commissioner Adam Silver has urged owners to force Sterling to sell the franchise. But Silver said no decisions had been made about the rest of Sterling's family.

Shelly Sterling, Sterling's estranged wife, said she intends to keep her 50 percent of the franchise.

"As players, we want what's right and we don't feel like no one in his family should be able to own the team," James said after the Miami Heat practiced for Game 4 of the Eastern Conference semifinals against Brooklyn.

But Shelly Sterling told ABC News' Barbara Walters that she doesn't believe she should be punished for what her husband said.

"I will fight that decision," she said in the interview. "To be honest with you, I'm wondering if a wife of one of the owners, and there's 30 owners, did something like that, said those racial slurs, would they oust the husband? Or would they leave the husband in?"

James was one of the first and strongest voices to speak out after a recording of Donald Sterling's remarks to V. Stiviano were posted on TMZ's website last month, saying the comments were unacceptable and that there was no place in the league for Sterling.

The league is trying to act quickly to remove him. Dick Parsons has been installed as the Clippers' interim CEO, and the owners' advisory/finance committee has held conference calls each of the last two weeks to discuss that process and timeline for a forced sale. That would require a three-fourths vote of owners.

But Sterling, who bought the team in 1981 and is the NBA's longest-tenured owner, could choose to fight those attempts by the league.

"At the end of the day, this is going to be a long litigation when it comes to that," James said. "This guy who's owned the team since the '80s is not going to just give the team up in a day. So we understand it's going to be long, but we want what's right."