The sense of achievement and closure for theoretical
physics that came with the brilliant success of the classical field
theory of electromagnetism was short lived. The new technology invented
out of the mathematical unification of electricity with magnetism produced
copious data about the nature of matter and light that snapped all of
the mathematical threads that physicists had just succeeded in tying
down.
And after this new data was unraveled and understood
and explained using mathematics, the unified worldview of classical
theoretical physics became split into two very different views of the
universe -- the particle view and the geometric view.

Particles and waves

The first sign of trouble was when J.J. Thomson discovered
the electron in 1897. Experimentalists
began to see data that suggested a model of the atom with negatively
charged particles orbiting around a positively charged core. But according
to Maxwell's equations, such a system should be physically unstable.
Classical field theory was unable to explain
or describe the emerging data on atomic structure.
Another big mystery that came out of Maxwell's equations
was the thermal behavior of light. Hot objects, like a hot coal, glow
by emitting light and that light is observed to consist of a distribution
of waves of different frequencies. But physicists who tried to explain
the observed distribution of frequencies using light waves as described
by Maxwell's equations met with continued failure.
Then as the new 20th century was beginning, a young
German physicist, in an "act of despair"
over the gaps in the understanding of thermal radiation, made a guess
called the Quantum Hypothesis, which
explained the observed thermal spectrum of light as coming from a collection
of identical discrete quanta of energy. His formula worked, but he didn't
know why.
This was the beginning of the idea known as particle-wave
duality, and the field of quantum mechanics.
Einstein used Planck's idea to explain the newly-observed
photoelectric effect. Einstein proposed
that light was emitted or absorbed by an excited electron in discrete
quanta called photons whose energy was
proportional to the frequency of the light according to the relation

,

where h is a number called
Planck's constant, determined by measurement
to be 6.6 x 10-34 joule seconds.
If a light wave could behave like a particle, then
could a particle behave like a wave of some kind? In 1923, French aristocrat
Louis de Broglie put forward the idea that an electron traveling with
some momentum p could act like a continuous wave with wavelength l
according to the relation

When
the dust was settled, the new quantum theory described a given physical
system not in terms of the path of a particle or the strength of a field,
but as the probability amplitude for
a given system to be in a given quantum state. This probability amplitude
is the square of a function called the wave
functionY(x,t), which
is a solution to the Schrodinger equation

Solutions
to Schödinger equation for more then one identical particle have
an interesting symmetry. For example, let's consider a two particle
system and exchange the two particles. The wave function will obey the
relation

In the plus case, the two particles are what we call bosons.
Two bosons can occupy the same quantum
state at the same time.
In
the minus case, the two particles are what we call fermions.
Two fermions cannot occupy the same
quantum state at the same time. This effect is called Pauli
repulsion, and Pauli repulsion explains the structure of the
periodic table of elements and the stability of atoms, and hence of
all matter.

Relativity and geometry

The
radical new idea of the quantum physics of atoms and light marked one
direction of departure from the comforting sureness of 19th century
classical field theory. The other big surprise of the 20th century came
with the astounding observation in an experiment by Michelson and Morley
that the speed of light was independent of the motion of the observer.
Now
normally one would think that is a person were capable of throwing a
javelin at 5 miles per hour while standing still, that same person,
when running across the ground at 10 miles per hour, would be capable
of making the javelin travel across the ground at a speed of 15 miles
per hour.
But
according to the data from the Michelson-Morley experiment, if one uses
a laser instead of a javelin, then whether the person is sanding still
or running 60 miles per hour or in a rocket traveling near the speed
of light -- the light from the laser still travels the same speed!
This
was an astounding result! How could it be explained using physics? Einstein
came up with a powerful, simple theory, called the Special Theory of
Relativity. Einstein used the geometric notion of a metric.
The most familiar metric is just the Pythagorean
Rule, which in three space dimensions in differential form looks
like

This formula has the special property that it is invariant
under rotations. In other words, the length of a straight line
does not change when you rotate the line in space. In the Special Theory
of Relativity the idea of a metric is extended
to include time, with a very crucial minus sign:

Like the space metric, the spacetime is invariant under rotations in
space. But now there is a new twist -- the spacetime metric is also
invariant under a kind of rotation of space and time called a Lorentz
transformation, and this transformation tells us how different
observers who are moving with some constant velocity relative to one
another see the world.
And
under a Lorentz transformation, the speed of light always stays the
same, which is consistent with the shocking Michelson-Morley experiment.
Einstein's
next target of revision was Newton's Universal Law of Gravitation. In
Newton's formula the gravitational force F12 between two
planets of masses m1 and m2 as depending on the
inverse square of the distance r12 between the planets

GN is called Newton's constant
and is measured to be 6.7x10-8 cm3 /(gm sec2).
Newton's
Law was extremely successful at explaining the observed motions of the
planets around the Sun, and of the moon around the Earth, and easily
extendible through the techniques of classical field theory to continuous
systems.
However, there was no hint in Newton's theory as to how a gravitational
field would change in time, especially
not in a manner that was consistent with
the new understanding in Special Relativity that nothing
can travel faster than the speed of light.
Einstein
took a very bold step, and reached out to some radical new mathematics
called non-Euclidean geometry, where
the Pythagorean rule is generalized to include metrics with coefficients
that depend on the spacetime coordinates in the form

where repeated indices imply a sum over all space and time directions
in the chosen coordinate system. Einstein extended the idea of Lorentz
invariance to general coordinate invariance,
proposing that the values of physical observables should be independent
of a choice of coordinate system used to chart points in spacetime.
He called this new theory the General Theory
of Relativity.
In
Einstein's new theory, spacetime can have curvature,
like the surface of a beach ball has curvature, compared to the flat
top of a table, which doesn't. The curvature is a function of the metric
gab and its first and second derivatives. In the Einstein
equation

the spacetime curvature (represented by Rmn
and R) is determined by the total energy and momentum Tmn
of the "stuff" in the spacetime like the planets, stars, radiation,
interstellar dust and gas, black holes, etc.
The Einstein equation is not strictly a departure from classical field
theory, and the Einstein equation can be derived as the solution to
Euler-Lagrange equations that represent the stationary point, or extremum,
of the action

Two views of the world

Using
quantum mechanics, the typical questions that can be answered concern
the types of quantum states and allowed transitions in a system that
features one or more particles that has some type of potential energy
represented by the potential V(x). A typical method of working is to
take some given V(x) and use the Schrödinger equation find the
wave function, the energies of the quantum states of the system, and
the allowed transitions between those states.
In
general relativity, things are very different. One performs calculations
that compute the evolution and structure of an entire universe at a
time. A typical way of working is to propose some particular collection
of energy and matter in the universe,to provide the Tmn.
Given a particular Tmn,
the Einstein equation turns into a system of second order nonlinear
differential equations whose solutions give us the metric of spacetime,
gmn, which holds all the
information about the structure and evolution of a universe with that
given Tmn.
Given
the difference in the fundamental questions and methodologies used in
quantum mechanics and in general relativity, it seems hardy surprising
that uniting quantum physics with gravity, for a theory of quantum
gravity, would prove to be a very tough challenge.