Now you have a clean foundation to build on, but how to install apps without Google Play? That's actually quite easy - most opensource apps can be had at the F-Droid store.
To install proprietary apps, I use Raccoon on my desktop to download the apk files from Google Play. Once I move them to the phone they can be installed by tapping on them in the file manager.