I think my host found the issue. Just FYI for anyone else with the same issue:

I have checked this further and noticed that OS installed in your vps is centOS 4.9. We have noticed similar issue in centOS 4 servers. It seems like PHP 5.3 and MySQL 5.5.10 requires centOS 5. Details can be found in following url.