It says that is already installed. That's even after unchecking, uninstalling it and clearing the caches. The other weird thing is that when I go to remove it again, it doesn't show up in the uninstall module list. But if I go to the complete list of modules, it's still there.

Not sure how to proceed now and would appreciate guidance. Been trying to upgrade since the new version came out. Thanks for the help in advance.

Could you check access rights of 'sites/all/modules/cleantalk' folder?
Just disable module and delete 'sites/all/modules/cleantalk' folder manually.
Drupal must delete it itself but sometimes it doesn't.