To understand the ways in which a quantum computer is different from a
classical computer you must first understand the rudiments of the
classical computer. The most fundamental building block of a
classical computer is the bit. A bit is capable of storing one piece
of information, it can have a value of either 0 or 1. Any amount of
information can be encoded into a list of bits. In a classical
computer a bit is typically stored in a silicone chip, or on a metal
hard drive platter, or on a magnetic tape. About 1010 atoms were
used to represent one bit of information in 1999. The smallest
conceivable storage for a bit involves a single elementary particle of
some sort. For example, for any particle with a spin-1/2
characteristic (such as a proton, neutron, or electron), it's spin-1/2
characteristic on measurement will be either +1/2 or -1/2. We can
thus encode a bit using a single particle by mapping 1 to be spin
+1/2 and 0 to be -1/2, assuming we can measure and manipulate the
spin of such a particle. If we were to try to use this spin-1/2
particle as a classical bit, one that is always in the 0 or 1 state,
we would fail. We would be trying to apply classical physics on a
scale where it simply is not applicable. This single spin-1/2
particle will instead act in a quantum manner. (Williams, Clearwater)

This spin-1/2 particle which behaves in a quantum manner could be the
fundamental building block of a Quantum computer. We could call it a
qubit, to denote that it is analogous in some ways to a bit in a
classical computer. Just as a memory register in a classical computer
is an array of bits, a quantum memory register is composed of several
spin-1/2 particles, or qubits. There is no particular need for the
spin-1/2 particle, equally well we could use a Hydrogen atom, and
designate its electron being measured in the ground state to be the 0
state, and it being in the first excited state to be the 1 state.
There are a multitude of possible physical qubit representations that
could work. For simplicity I will discus only the spin-1/2 particle
from here on, but analogous arguments could be made for many things.