5 Answers
5

A probabilist can tell you that getting 100 heads in a row is no less likely than any other outcome when tossing a fair coin 100 times. A statistician will suspect the coin is biased.

Statistics is a discipline that relies heavily on mathematics, but is not within mathematics.

For example, consider the Behrens–Fisher problem: What should one infer about the difference between the means of two normally distributed populations, which may have different variances, when one observes a random sample from each?

Bartlett criticized Fisher's "fiducial" solution to this problem on the grounds that Fisher's fiducial intervals are not confidence intervals, i.e. they don't have constant coverage rates. That is certainly a mathematical fact. But Fisher disputed the idea that they ought to have constant coverage rates. That's essentially a philosophical position. Suppose you had prior probability distributions on the means and variances of the two populations, and then asked what's the conditional distribution of the difference between the two means, given the observed samples? That's just a math problem, and the posterior probability intervals that you get don't have constant coverage rates, so under some circumstances it clearly makes sense not to have constant coverage rates. Just which of several math problems should be used to model the Behrens–Fisher problem? That's more akin to a philosophical question than to a math problem. It's certainly not itself a mathematical question. But in a sense, it is the Behrens–Fisher problem.

Only some very poor statistician would not know that getting 100 heads in a row is no less likely than any other outcome when tossing a fair coin 100 times. Only some very poor probabilist would not suspect the coin is biased.
–
DidJan 25 '13 at 17:40

1

@did : Certainly that's true. But I was addressing the question of what the difference is.
–
Michael HardyJan 26 '13 at 6:19

@omeid : Yes. I'm using the terms synonymously. I'm saying the statistician will suspect the coin is not fair. And of course so will anybody else, since some simple things in statistics are obvious even to someone who's never thought about the subject.
–
Michael HardyAug 14 '13 at 2:48

Do not look at math as some 'equations' or 'formulas'. Math is an infinite-sized construction set, sort of like LEGO, but much, much more exciting. It is a way of combining beautiful logical structures to derive new beautiful logical structures.

Statistics can be seen as an inverse of probability theory, to some extent at least. If, say, you are given some database, you can estimate parameters of the distribution that this data set follows. Probability theory concerns itself with random variables and their distributions. So you assume that some phenomena (e.g. the number of infected species in the population) follows some distribution, and from this you can derive some property of this phenomena (e.g. mean time until extinction).

If time is also a factor, you should look at random/stochastic processes too.

A probabilist, a statistician and a mathematician sit in a cafe across from a parked sedan. They see five people get into the car. After a minute passes, six people exit the car. The probabilist says that the event is non measurable since the sigma algebra of the car accounts only for 5 seats. The statistician says that six people walking out qualifies as a significant statistic against the null hypothesis that there were no people inside the car to begin with. The mathematician says that if another person were to enter the car now, then there would be exactly zero people in the car.

Probability is a branch of mathematics. Period. (Nowadays, that is... in Poisson's day, it was a branch of Physics....but that is what makes this question so interesting...)

So what is mathematics. Mathematics is the branch of logic which deals with sets, and hence, with any or all of what we usually call `mathematical objects', e.g., numbers, transformations, equivalences of categories, points, generic points, etc.

There is a famous quotation, which André Weil repeats, «Qui dit mathematiques, dit preuves.»
You are not doing mathematics unless you define your terms and state your axioms and come up with a proof of your assertion. (Well, formulating a conjecture counts, at least if you have a lot of numerical evidence for it, i.e., have proved many special cases.)

Now what is statistics. Statistics can be part science or part math or both. Mathematical statistics is a branch of mathematics which is founded on probability theory. Its practitioners do proofs just like any probabilist or other mathematician. (As stated by the great Boris Levit.) But Statistics can also be the analysis of data, in which case it is a branch of science which merely uses mathematical results like any other branch of science does, but which requires scientific intuition and experimental results to compensate for lack of precise definitions, lack of proofs, etc. For this reason, many statisticians have the intellectual traits which scientists have and mathematicians almost always lack these days. Sir Ronald Fisher had mathematical ability but preferred to hide it, because he had so much more....

Last, as to the curious way the field of probability has evolved. In its beginnings, it was really thought of as a branch of Physics or Natural Science. (See Leo Corry on HIlbert's Sixth Problem, http://www.icm2006.org/proceedings/Vol_III/contents/ICM_Vol_3_82.pdf.) When Poincaré began his career, he occupied chairs of physics, of probability, and of rational mechanics, at different points. He did not consider himself only a mathematician (and, didn't he discover the Theory of Relativity?) but this was already beginning to seem a little strange to other physicists. Still, this was the last gasp of the tradition that The Calculus of Probabilities was part of Physics, just like Mechanics. It was very hard for people to think of it as part of mathematics since for millenia, mathematics had been defined as that which dealt with numbers or points. Only after both numbers and points had been finally unified, via the axiomatic method and the use of set theory, could some mathematicians begin to think of the possiblity of a purely Mathematical Probability which would be axiomatised and founded on logic plus set theory: Frechet and Wiener worked this out and Kolmogoroff offered the definitive formulation. But even now we then have inherited a curious question, focussed on by Littlewood in his math club talk, The Dilemma of Probability, (see http://books.google.ca/books?id=MjVgeT7Laf8C&pg=PA72&lpg=PA72&dq=Littlewood+dilemma+of+probability+%22inverted+commas%22&source=bl&ots=ou2am-HEHZ&sig=baPTMGuK9VYXOQ4CFi8OkG4gSh8&hl=es&sa=X&ei=qbGOUqqPIafP2wXF54GwCg&ved=0CC8Q6AEwAQ#v=onepage&q=Littlewood%20dilemma%20of%20probability%20%22inverted%20commas%22&f=false), published in his Mathematician's Miscellany, that the meaning of «physical probability» is not explained by the mathematical theory of probability, nor by its axioms, and remains somewhat mysterious in spite of the fact that the mathematical theory of probability works very well when applied to the real world applications.