When I removed the vinyl tile in my kitchen, I left the mastic behind. I installed stone tile directly on top of the mastic, no problem. For hardwood, you could almost certainly install a thin layer of plywood over the mastic, and install the hardwood above that. I suspect removing the mastic will prove to be more trouble than it is worth.