(CNN) -- When Piers Morgan pointed out that "Vicious" Victor Ortiz will be "facing a guy that has beaten just about everyone that's come in front of him," the boxing champ told the CNN host, "Fear doesn't even exist in my dictionary anymore. It was one of those things where my upbringing alone got rid of all that."

Morgan was referring to Ortiz's upcoming September 17 bout with "Pretty Boy" Floyd Mayweather. Ortiz and his mentor, boxing great Oscar de la Hoya, are guests on Wednesday's "Piers Morgan Tonight."

Abandoned by his mother at the age of 8, Ortiz was left to essentially raise his younger brother when his father walked out of the family's Kansas home four years later.

"It was my brother and I just roaming around like two strays, you know," said Ortiz.

Ortiz took care of his brother, who is two years younger, day in and day out and even put him through college.

"He and I had each other's back usually," said Ortiz. "Like, you know, like two brothers would."

Interestingly enough, once Ortiz became famous, his parents did not seek him out.

"I actually went out of my way to find both of my parents," said Ortiz, who told Morgan that he traveled to Kansas and "asked around everywhere," in order to track down his dad.

When he finally found the elder Ortiz, the World Boxing Council Welterweight Champion of the World said that his father was still "very intimidating," and that he resisted the urge to ask the man why he'd abandoned his family.

"I was just like, 'Hey, Dad, you know what? I did your job and my brother and I grew up."

Morgan asked Ortiz, 24, if he feels that his opponent holds a psychological advantage over him. Mayweather has won 10 world titles and is undefeated as a professional boxer. Mayweather, however, hasn't stepped in the ring in over a year, while Ortiz has fought five fights in that time.

Oscar De La Hoya on Victor Ortiz

Champion boxer grew up without parents

De la Hoya won a fight without fighting

"I'm not really sure how the balance lies," answered Ortiz. "But that's one thing that I'm willing to put aside, because I want the best pound for pound, Floyd Mayweather to arrive, September the 17th, across the ring from me."

Morgan asked 10-time world champion and Olympic medalist Oscar de la Hoya, who was defeated by Mayweather in 2007, what kind of man Mayweather is to fight.

"He is a great fighter," said de la Hoya. "He is the best pound for pound fighter in the world today. He is great for the sport. He has done great things for boxing. And, you know, on September 17th, Victor is going to face the best. And, you know, there's no excuses whatsoever.

"But when I fought him, I mean you can say that I was over the hill. I was an old 35 years old when I faced him. He beat me, yes, hands down. There's no doubt about that. I congratulate him."

De la Hoya alluded to the possibility of Mayweather, who at 34 is 10 years Ortiz's senior, passing the torch on to Ortiz, who de la Hoya believes "will be one of the greats."

Does de la Hoya believe he'd beat Ortiz if the pair got into the ring today?

"Let me put it this way," de la Hoya told Morgan. "If I was at my prime and -- and Victor is at his prime -- it would be one hell of a fight, that's for sure. Maybe a draw."