I am wondering how companies such as Alexa Internet determine a given site's (not my own) overall traffic and traffic for each unique page. I would appreciate a technical response - if you were to design this feature (i am sure it is complicated but hypothetically...) how would you go about it?

There are a lot of guesses after a Google search: google.com/… though it probably revolves around collecting statistics from Google and crawling sites such as Twitter, Facebook, etc to generate a popularity index.
–
user195488Dec 29 '09 at 16:53

Installing the Alexa toolbar modifies the browser user-agent, so you can estimate the % of visitors to your site who are contributing data to Alexa by scanning your server logs for requests with the appropriate user-agent strings.