The welterweight bout between Floyd Mayweather Jr. and Manny Pacquiao—which is expected to break boxing revenue records—will be held at the resort and casino company's MGM Grand in Las Vegas. But Kelly noted that MGM shares have moved only slightly higher ahead of the event, limiting their upside.

"I would just stay away completely from the casino space right now," he said.