When you add the email back, be sure "contacts" is toggled on. If you don't want that email account, then you need to make those contacts iCloud contacts. Then you can delete the email. Check to be sure you have "contacts" toggled on for iCloud.