Josh Brown of Fusion Analytics said that the rally has pushed the "perma-bears" to stake out narrow bullish positions in certain stocks or sectors while not fully buying into the bull case.

"But let's keep in mind, this market has fought all the way up with one hand tied behind its back. We have not had the participation of about a third of the S&P when we're talking about those cyclical sectors," he said. "So, if they're going to play now and the defensives are going to take a back seat, that could represent the next leg higher."

Karen Finerman of Metropolitan Capital Advisors said that she preferred staying long.

Finerman said that with the volatility index coming in, it was worth buying protection, which "helps me sleep at night."

But a bit of worry remained.

"Especially with oil ticking up as it has, that does make me a little nervous for the retail space," she said.

Trader disclosure: On May 3, 2013, the following stocks and commodities mentioned or intended to be mentioned on CNBC's "Fast Money" were owned by the "Fast Money" traders: Steve Grasso is long AAPL; Steve Grasso is long ASTM; Steve Grasso is long BA; Steve Grasso is long BAC; Steve Grasso is long GDX; Steve Grasso is long GOOG; Steve Grasso is long LF; Steve Grasso is long LNG; Steve Grasso is long MHY; Steve Grasso is long MJNA; Steve Grasso is long PXD; Steve Grasso is long NVIV; Steve Grasso is long PFE; Steve Grasso is long S; Steve Grasso is funds long AAPL; Steve Grasso is funds long MSFT; Steve Grasso is funds long HPQ; Steve Grasso is funds long NEM; Steve Grasso is funds long ZNGA; Josh Brown is long AAPL; Josh Brown is long XLU; Josh Brown is long GOOG; Josh Brown is long F; Josh Brown is long VAW; Brian Kelly is long BK CORP; Brian Kelly is long TLT TREASURIES; Brian Kelly is long JJG; Brian Kelly is short EWG; Brian Kelly is short FXE; Brian Kelly is short FXY; Brian Kelly is short EURO; Karen Finerman is long AAPL; Karen Finerman is long BAC; Karen Finerman is long C; Karen Finerman is long JPM; Karen Finerman is long TGT; Karen Finerman is long GOOG; Karen Finerman is long GM; Karen Finerman is long M; Karen Finerman is long FINL; Karen Finerman is long FL; Karen Finerman is long FS; Karen Finerman is long MAY PUTS.