Possible solutions:
Add logic from wp_delete_category() above each call for wp_delete_term and pass the args directly to wp_delete_term or conditionally check for taxonomy type and pass to wp_delete_category (a wrap for wp_delete_term) to pick up needed logic.

The attached diff takes approach two and passes to wp_delete_category if 'category' == $taxonomy