Different Sub Cat List on Left Sidebar in each Product Cat Page

I have created product category pages and I want that the subcategories of the specific product category page would be displayed as list in the left sidebar. What Im trying to say is; for instance

My Product category hierarcy:

Men
-Suits
-Shoes
-Bags

Women
-Dresses
-Shoes
-Bags

I have created the product cat pages of these categories and I want that Dresses,Shoes and Bags only would be displayed as a list in Women product cat page and Suits, Shoes,Bags should be displayed as a list category in Men pro cat page. Any idea? Please help! 🙁