The Case of the Crusty Horns!

I always slightly scrub and rinse the horns/pump parts after I pump... then I boil them about once a week. I just noticed the other day, the very inside of the pump (near the back part, where it angles), it's turned yellowy and looks like dried up milk. I wonder if the gunk would be bad for the baby?

How are you guys keeping your parts nice and clean? I'm thinking of grabbing a Q-tip and just wiping it down... it's kinda hard to reach though