Well, you can read all the numbers about fov and angles etc. like in this thread, but our new forum-member skelter asked for a real life view of the differences between a fisheye, namely the Tokina 10-17mm fisheye-zoom and another ultra-wide zoom like the Sigma 10-20mm.
As I don't have the Tokina 10-17mm handy I took my trusty Nikon 10.5mm fisheye out on this great day and made 2 shot from the same point of view with both lenses to compare. And I also threw in a defished version of the fisheye-pic for good measure. These defished versions were made in CaptureNX, but a word of warning here: I don't think (but cannot test) that CaptureNX will defish other than Nikkor lenses. So when defishing a Tokina 10-17mm e.g. you have to rely on third-party software.

Wonderful, thank you very much, this is really the test I needed !
Anyway, now I am more oriented to the Sigma, than to a fisheye (nikon or tokina).
If you have other chances with different subjects, please feel free to post them !

1. Saves me certain post-processing steps. This is a moderate benefit, since I always do some post processing regardless.

2. As Thomas points out, the image quality is usually higher in the straight wide angles.

However, grabbing a wider field of view onto one image can be an advantage is specific situation where you are unable to take two pictures and stitch later - typically when there is fast-moving action going on.

From an artistic point of view, fish-eye lenses save you a post-processing step if you want to have the distortion they offer.

The poll-results are as one might expect: there are more people who shoot wide angle without distortion.