There have been instances reported wherein the Dragonborn was able to keep Balgruuf's Greatsword when sent to Cidhna Mine as part of "The Forsworn Conspiracy"/"No One Escapes Cidhna Mine" questline. Whether or not this sword is kept when entering Cidhna Mine or other prisons through normal arrests has not been reported.

The way to keep the sword when entering prison is to favorite the item. It will then appear not in the Dragonborn's inventory, but in their favorites list. The sword can still be equipped and used when accessed in this way.

​Another way to keep Balgruuf's Greatsword is when you hand it in to Proventus Avenicci, quickly exit the dialog with him and check his inventory, he should be holding the sword in his inventory and shouldn't have it equipped. Take it from him to keep the item. Note: It is impossible to remove the "Stolen" tag from this item.