I assume you mean the path specified by $dump_path. Anything below that will be removed just fine with what you have (unless permissions are blocking you). Of course, you'd only want to remove that path if there was nothing contained, otherwise there'd be no reason to perform the check in your script above. So, you could just add the following to the end of your script.

No. Those folders should already be removed by what you have if they meet the criteria. If you add the -whatif parameter to your Remove-Item command, you should see lines like the following for any folders that should be removed.

In this video we show how to create an email address policy in Exchange 2013. We show this process by using the Exchange Admin Center.
Log into Exchange Admin Center.: First we need to log into the Exchange Admin Center.
Navigate to the Mail Flow…

In this video we show how to create a mailbox database in Exchange 2013. We show this process by using the Exchange Admin Center.
Log into Exchange Admin Center.: First we need to log into the Exchange Admin Center.
Navigate to the Servers >> Data…