Well when we got the tank it hadn't been cleaned in awhile we did take out a lot of algae but left some for the shrimp to eat. However they don't seem to be eating all thy much. Do I suggest to clean it all out?

Shrimp don't eat much algae, if any. What I would do is keep the water clean. get on a weekly partial water change regimen and feed reasonably for a while. The algae will slowly disappear.
You can feed as little as two or three times a week with no ill effects.