I have a lot of affiliate links on my site and was told I should set all my affiliate links to rel="nofollow" so that Google doesn't devalue it but it is changing my seo color to yellow and giving me a warning that: "This page has 5 outbound link(s), all nofollowed."

A nofollow link will not receive any credit by Google in websites ranking, so I don't understand why you say that nofollow attribute doesn't devalue it.
Declaring as "no follow" legitimate links doesn't make sense, even less if those links are affiliate, so I think isn't a good thing in general, but it depends on what you want.
If you want that Google doesn't consider those links as good links, then keep no follow attribute, otherwise remove it.

No, from the Google point of view isn't incorrect, as always.
I hadn't realized that they're tons of affiliate links to Amazon, and probably Google see them, if not as spam, as excessive advertising, and Google doesn't like sites with too many ads, especially in the header, so tells you to put nofollow.
The nofollow attribute was invented by Google to contrast spam, after all (and it has been useless, as we see every day in our blogs).
The SEO plugin, as I think, but I could be wrong, doesn't see the difference between a link or another, and simply tells you not to put the nofollow attribute because it doesn't help the page ranking, stricly the purpose of the plugin.
Google has interpretation limits, SEO plugin also have (e.g.: I have all my photogallery pages signed orange or red, 'cause they are missing words and other stuff... but they're are photo pages!)
As always, SEO is a controversial matter, and depends from the different point of view. It's complicated.
I'm not a SEO expert, it's just my view of this case, and I think you must balance between the two.
So, if they are too much (links or nofollow), decrease it, if they are important or if you can't do in other way, don't care about SEO plugin check color.