Just to make sure:
Go to full deflection, then 'ease' the stick back toward center, and see if it stays deflected some in that direction.
(Don't let the stick bang back to center. Ease it there.)
Then full deflection the other way, and ease back to center again, and see if it stays deflected some in that direction as well.
If it's really coming all the way back to center after each of those, then that's an odd one.
Possibly a dirty/bad/loose pot, either in the servo, or in your gimbal. Or some floating bit of debris, perhaps.
It's also 'possible' though unlikely, that there's some sort of thermal drift happening.
Honestly, I'd just fit a different servo first, and go from there.

Afterthoughts:
Have you given the control horn in the elevator a good wiggle, to make sure it's still secure?
Broken hinge(s)?
Loose/cracked wood in the elevator, or the joiner between the halves?
Solar flares, sunspots, zombie invasion? I dunno. Just brainstorming

Edited to add:

Quote:

Originally Posted by KE Spin

I had a similar problem and thought it was the servo but found that he control horn had actually come loose and was moving in the slot.

I see someone else had the same thought while I was posting. Definitely seen the 'loose horn' thing before