It could be that you need more RAM and/or have bad RAM. Like ez061111 stated, you really need more than 4GB these days. You should also consider doing a clean install of Mountain Lion. I mean, if that wasn't the method of your upgrade. You might also try repairing your disk permissions and clearing your caches. There's still a chance you have some crud from Lion. Stuff that was slowing you down in the first place.

Honestly, you really can't run Mountain Lion with 2GB of RAM. I know it meets the minimum system requirements, but it's not going to run very well with a couple apps open. Even basic apps like mail and your browser. I would even skip over 4GB and go towards 8GB with a 2011 Mac mini.