Make sure to follow guidelines and set up a root account with a decent password. See the links above for how to set up a LAMP server if you need it.
Create a user bugzilla (change it if you want something else)

Bugzilla uses a script checksetup.pl to check if everything is set up correcly and if things changed to set them up for you. By manually installing the perl modules you can skip this.
Check if all modules are installed

cd /var/www/bugzilla/
sudo ./checksetup.pl --check-modules

If not installed...

sudo perl -MCPAN -e install

localconfig holds the configurations and needs to be set up. So...

sudo gedit localconfig

and change $db_name to the database name, #db_user to the user and $db_password to the password you used during setup of MySQL.

This didn't work for me until I changed your AllowOveride Limit to AllowOveride All, just a little note to all like me :) Other than that, you wrote a great guide....
–
Luke San AntonioJul 20 '12 at 23:58

I have followed your advices on Quantal Ubuntu. sudo ./checksetup.pl reported no errors. But when I open thehttp//<servername>/bugzilla3 I see listing of content of the /var/www/bugzilla dir, not the bugzilla itself. Can you help me? (BTW, access to any file in /var/www/bugzilla is denied if accessed by the Firefox; maybe this is a problem).
–
Adam RyczkowskiFeb 1 '13 at 13:31

All dependencies are installed automatically, everything is configured automatically. Automatic install should work in most cases, even if Apache or Mysql are already installed and configured (unless some non-trivial changes in their configuration have been made)

Novice Ubuntu users are misled by the detailed build-it-from-tarballs instructions.

I want to point a great guide that I found on a step by step install.
–
SaarikoSep 20 '11 at 10:35

You should've specified that you need Bugzilla 4. Installing bugzilla 3 is a one-step process, even in the guide you linked to.
–
SergeySep 20 '11 at 11:04

I tried it with bugzilla 3 and it did not only not install all perl modules for me it did also not adjust apache2 for me. The above setup worked for me (but I started with a working apache2 (and in a VM ( ;) )) so perl and php set up to work).
–
RinzwindSep 20 '11 at 12:12