My host is a freshly installed Ubuntu 2013.04, wireless network access worked out-of-the-box so I don't have any strange network configuration.

In VirtualBox 4.2.10, with default (NAT) settings, I installed CentOS 6.4 minimal.
Immediately after install, the first thing I did was ping 173.194.38.98 (google) and I a told connect: Network is unreachable.

I tried running /etc/init.d/network start as root, no joy.
I downloaded a VM image and tried it: exact same problem.

When I installed Ubuntu and Windows VMs, they are able to access the Internet without any problem.
What's wrong with this one?

Issues with udev

See my answer to this question: Is VMWare causing my Linux host to lose network connectivity?. In general when your network shows up with eth1 instead of eth0 bells should be going off (at least in CentOS 6.x). It usually means that the ethernet NIC is under the control of NetworkMangaer or udev is messing up the detection of your NIC as eth0.

I usually start by looking at the rules.d directory of udev:

# /etc/udev/rules.d/70-persistent-net.rules
# This file was automatically generated by the /lib/udev/write_net_rules
# program, run by the persistent-net-generator.rules rules file.
#
# You can modify it, as long as you keep each rule on a single
# line, and change only the value of the NAME= key.
# net device () (custom name provided by external tool)
SUBSYSTEM=="net", ACTION=="add", DRIVERS=="?*", ATTR{address}=="54:52:00:ff:ff:dd", ATTR{type}=="1", KERNEL=="eth*", NAME="eth0"

You want to make sure that only one entry exists in this file. If there are multiple, then you'll need to manually clean up this file, so that there is only a single entry and that it's pointing to eth0.

Issues with NetworkManager

The other thing you'll need to take a look at is how is NetworkManager setup. Again check out this other Unix & Linux quesiton: CentOS no network interface after installation in VirtualBox. I provided an answer on that question as well which shows you the steps for manually setting up a wired network using NetworkManager.

Incidentally, NetworkManager is now the default setup for networking when doing installations of CentOS 6.x!