Who's using the folder? You are. Whenever Environment.CurrentDirectory is set in an application, it prevents anything else from deleting the folder. Note that the Environment.CurrentDirectory can also be set through a variety of other means, including the FolderBrowserDialog, OpenFileDialog and SaveFileDialog.

What's the solution? Simple. Set Environment.CurrentDirectory to something else. @"C:\" works just fine.