In short: "web application projects" may be formed of multiple Visual Studio projects and are also to be used for websites designed around the MVC pattern. Whereas a "Web site" project is just that, one "web" project and all files must be contained inside (you may however link to other assemblies); it is better suited for old-school ASP.Net websites.

That article I linked to above gives a lot more information, check it out!

Websites aren't necesarily just one project. We have multiple ASP.NET Websites that haven't been converted to Web Applications yet, and they all share some common assemblies.
–
Davy8Aug 20 '11 at 19:08

I edited my answer, what I meant was that you have one "web" project but that does not mean it cannot depend on other project references, or other assemblies.
–
JalaynAug 20 '11 at 19:39

There is a HUGE list of reasons to use one over the other, but usually when I'm making a decision I base it on a few parameters. For instance, do you want your code files to be out in the open (well, still behind server security) or would you rather they are compiled into code libraries?