I only every really watch HS ball.. but I watch a lot of it. Have seen hundreds of games, and I never actually recall a ref keeping his whistle in his pocket. Seemed weird as heck to me, but I have been corrected. Turns out it is not weird at all.

Having said that it is possible others have done it and been smooth about it, hitting the whistle then deftly slipping it in the pocket as you jog to reporting area. That is not this ref's mechanic though-- he's probably 70 plus, so moved a bit slow to start with. When he stopped play, he would then go through a painfully slow process of putting the whistle away and only then would start to move towards the reporting area.