I didn't cut the caffeine out completely just drank coffee or full fat tea up until midday. The theory being I was caffeine free by sleep time in the evening.
Redbush tea works for me as a tea substitute as I can have it with or without milk, though it taste is similar to "real" tea.

I did the same a year back, I'm a heavy tea drinker and found the Tetley decaf tea to be pretty much indistinguishable from the full bore one. Needs a bit longer brewing time though.

Ginger tea is also great, particularly in the morning (Burt and Tangs I like - but you can make a great cup with grated root ginger in hot water with a dash of honey if you want it sweet) - and Jasmine tea (but I think this may be caffeinated?).

Best thing about it is I really look forward to my one or two nice coffees a week. And, when I drink coffee on bike ride i end up flying for a couple of hours, where as before it did nothing except make me feel less crap.