If you aren't seeing Hulu Plus show up on your Apple TV yet, you simply need to restart it. Turning it off and back on wasn't enough for me, I had to actually go in and restart it from the settings menu. Here's how.

On your Apple TV, go to Settings > General.

Restart is hidden all the way at the bottom of the General Settings screen.
Jason Cipriani/CNET

Scroll down to the bottom of the list. The very last option will be Restart. Select it.

Your Apple TV will then restart, taking about 30 seconds or so to complete. Once it comes back on, you should then have the Hulu Plus icon next to the Netflix icon.

If you're new to Hulu Plus, you'll get a one-week free trial of the service, and after which you will be billed $7.99 per month to keep the service active.