LOS ANGELES, California (CNN) -- Michael Jackson's dermatologist did not rule out that he may be the biological father of Jackson's children, and Dr. Arnold Klein denied that he ever gave Jackson dangerous drugs.

When asked if he were the father of Jackson's oldest children, Klein said "not to the best of my knowledge."

Klein, in an interview Wednesday on ABC's "Good Morning America," denied that he was on the list of doctors being investigated by the Los Angeles Police Department.

When he saw that someone gave Jackson a dangerous drug, he was the one "who limited everything, who stopped everything," Klein said.

Jackson danced around Klein's Beverly Hills office just three days before his death and was "not in terrible pain," Klein said.

Debbie Rowe, who was briefly married to Jackson and gave birth to his two oldest children, worked for 23 years in Klein's office, he said. The doctor refused to say whether he thought Rowe should get custody of them instead of Jackson's mother, Katherine Jackson. Photos: Stars come out for memorial »

"I can't make those answers, because 'should have' and 'will' are two different things," he said.

Klein was scheduled to talk to CNN's Larry King about Michael Jackson on Wednesday night.

"Not to the best of my knowledge," Klein said. "All I can tell you is, best of my knowledge, I am not the father of these children. But I am telling you, if push comes to shove, I can't say anything about."

Klein said he "can't answer it in any other way, because, you know what, I don't want to feed any of this insanity that is going around."

Katherine Jackson was given temporary guardianship of the children by a judge several days after her son's death. Rowe was considering whether she will seek custody or visitation of the two born to her, her lawyer said last week.

The Los Angeles County coroner is waiting for toxicology test results -- not due for at least another week -- before determining what killed Jackson last month. The death certificate listed the cause of death as "deferred." Watch the latest details in the Jackson probe »

Sources said Tuesday that when Jackson collapsed, his arms were riddled with marks, and their veins had collapsed, both characteristics found in intravenous drug users. The revelations add to the growing speculation that prescription drugs played a part in Jackson's death.