Ok I figured out what the problem was. Apparently this is Maverick related bug. I have checked and confirmed it with couple of guys on #ubuntu-server who were running Lucid. All the options in /etc/libvirt/qemu.conf are commented out by default.

I had to uncomment these two out:

# The user ID for QEMU processes run by the system instance
user = "root"

# The group ID for QEMU processes run by the system instance
group = "root"