Howard Schultz, Starbucks' (SBUX) executive chairman, has been fortunate enough to never have had to deal with an activist investor given the stellar return of the coffee king's stock. And that's something Schultz is totally fine with.

"But when I see Carl Icahn, I look the other way," Schultz said on Wednesday at a conference held by The Economic Club of New York.

Billionaire investor Icahn gained notoriety in the 1980s and 1990s when he launched activist campaigns against giant corporations like Apple (AAPL) , Mylan (MYL) and Family Dollar (FDO) .

Schultz said "there are bad actors" [companies] who need to be pushed to evolve, but that most company executives, touting himself, "are the good guys." He said activist investors aren't always in the right, denouncing Pershing Square Capital Management's Bill Ackman's controversial campaign against J.C. Penney (JCP) , under which the billionaire investor forced himself onto the board of directors in 2011 and overhauled the retailer bottom up.