You need to sanitize it before use, yes, but if it is clean, then you don't necessarily need to sanitize it until the next time you use it. Store it clean and dry.

HOWEVER <---- BIG HOWEVER
Never never never use dish soap or soap of any kind on your brewing equipment. Soap is the bane of beer. It will ruin your head retention. Anything that touches soap will need to be rinsed one zillion times.

When you're done with a carboy, rinse with water, then scrub remaining crud with your carboy brush. Rinse more. Then fill it with water to soak. Then after a while, scrub it again. Never use soap. Water and elbow grease. MAke it immaculately clean, dry it and store it. Then be sure to sanitize it next time before using.

I followed some cleaning instructs in the joy of brewing book of 2oz bleach to 5g cool water for 24h after use. It chewed away at the stuff on the sides and "looks clean". When I was going to get ready to use it next (5 days or so) I was going to hot water rinse and then use the no-rinse sanitizer I got. Does this sound good or do I really need to scrub even if it appears clean?

I soak in bleach often too. It seems to help lift the crud off the glass, as you describe.

If you have the carboy visually clean, with no specks of anything on it, then you're good. Usually it takes scrubbing to achieve that state, but hey, as long as it's absolutely spotless. Store it dry like that.

Your plan for use next time sounds perfect. Rinse, then sanitize with your sanitizer. Drain the sanitizer immediately before use or take a sanitized plug and airlock and stick it in the carboy after you drain the sanitizer. Remember to slosh the sanitizer around on the neck and mouth of the carboy too.

fill the carboy up with water and a scoop or two of this stuff. Similar to PBW and oxyclean. Leave it sit for a day or two and you won't have to hardly scrub it. Rinse the hell out of it and let it dry.