6) Install phpMyAdmin. When you get to the prompt where it ask if you want allow “dbconfig-common” to handle the installation and configuration of database just enter the letter “Y” and retype your password from step 2.

sudo apt-get install phpmyadmin

7) Reboot Apache one last time to complete the LAMP configuration

/etc/init.d/apache2 restart

Your LAMP server should now be up an running. You can also access phpMyAdmin by going to YourDomainName/phpmyadmin. Enjoy!

Update:

Although this tutorial is intended for Ubuntu 13.04 there seems to be some changes to how you install PHP my admin in new version. All the above steps still works the same except the phpmyadmin fix below.

If phpmyadmin isn’t working after finishing the above steps
Create a link in /var/www like this:

sudo ln -s /usr/share/phpmyadmin /var/www/

…And since 14.04 you may want to use /var/www/html/ instead of /var/www/

If thats not working for you, you need to include phpmyadmin inside apache configuration.