In my experience, 3-4 weeks is usually sufficient. This is influenced by several factors including pain, wound healing, the extent of surgery that you had, etc. Having said that, when most men return to excercise, they tend to have a litle more swelling which ulitmately goes away.