Magento: How to delete / remove extension from MagentoConnect?

Suppose, you have added a new extension to MagentoConnect. You may have situation to delete that extension later on. However, there is no delete option in Manage Extension section in MagentoCommerce (My Account -> Extensions and Languages -> Manage Extensions). You can Edit your extensions yourself but you cannot delete them.