"As soon as that baby's lips touch that girl's bosom, forget it," he said. "Every single investment idea. Every desire to understand what's going to make this go up or go down is going to beoverwhelmed by the most beautiful experience, which a man will never share about a mode of connection between that mother and baby."

But women who want to get married and not have children could consider a career as a hedge fund manager, Tudor Jones said.

Tudor Jones, who graduated from the University of Virginia in 1976, has four children. He said men with children can be good money managers, but divorce could hurt their ability to focus. Tudor Jones says he redeems his money from male managers as soon as he learns that they are going through a divorce. "You can automatically subtract 10% to 20% from any manager when he is going through a divorce."

Tudor Jones runs Tudor Investments, a hedge fund with roughly $13 billion under management. He's never been divorced and has been married to his wife, Sonia, for 25 years.

In the video clip, Tudor Jones was responding to a question about why only "rich, middle aged, white men" were sitting on a panel about "views from investment masters." Two other hedge fund legends, Julian Robertson of Tiger Management, and John Griffin of Blue Ridge Capital, didn't weigh in, but Griffin commended Tudor Jones for his candor.

While women who want children shouldn't consider trading, Tudor Jones said they can consider careers in management.

Tudor Jones concluded his remarks by paraphrasing his mother who said that if women ran the world, there would be no wars.

"That would be fantastic. I believe her 100%," he said.

After the controversy that his remarks have created, Tudor Jones apologized and backtracked a little bit Friday.

"As I've told my three daughters, all of whom I've at one time encouraged to go into macro trading, any man or woman can do anything to which they set their heart and mind," he said in a written statement.

He added that his "off the cuff remarks" were about global traders, who are on call all the time.

Tudor Investments employs 370 people. A spokesperson for the hedge fund declined to comment when asked how many women work for the firm and what roles women there hold.