It really depends on the distribution of $x_i$'s. For example, if they are sub-Gaussian random variables you can get bounds that decay exponentially in $n$ using Chernoff's inequality. The bound you have here is a direct application of Markov's inequality.
–
S.B.Oct 1 '12 at 17:56

But if nothing is known about the distribution of $x_i$? I think that this bound cannot be sharpened but I can't come up with an example.
–
grozhdOct 1 '12 at 17:59

The problem is that defining the "sharpest bound" is a little vague. Assuming that by sharpest you mean fastest decay rate in terms of $n$, you want a bound that holds uniformly for all distributions (given a fixed mean and variance). I don't know whether or not your claim holds in that sense, but I think Central Limit Theorem may contradict the claim.
–
S.B.Oct 1 '12 at 18:10

@S.B. : "Sharp" in this context would mean the slowest possible decay rate of the variances, and you need just one distribution in which the slowest rate is achieved. In other words, you would want the variance to be exactly equal to $V/(n\varepsilon^2)$. That would prove that there is no tighter bound. There's nothing vague about that.
–
Michael HardyOct 1 '12 at 18:21

One way to prove the inequality is to say $\Pr(|\bar X - M|>\varepsilon)$ $=\Pr((\bar X - M)^2>\varepsilon^2)$ and apply Markov's inequality, which says that if $\Pr(Y\ge 0)=1$ then $\Pr(Y>y)\le \mathbb{E}(Y)/y$. Markov's inequality itself is sharp in that there is a distribution for which equality holds, namely any distribution supported on a set of the form $\{0,y\}$ where $y>0$. However, I think S.B. may be right when he says the central limit theorem may show the inequality is not sharp.
–
Michael HardyOct 1 '12 at 18:31

The deduction after So is definitely not kosher (and has many counterexamples) since it assumes that $a$ and $b$ are $\pm\varepsilon\sqrt{n}$ while the CLT is valid when $a$ and $b$ do not depend on $n$.
–
DidOct 3 '12 at 6:44

I think I was hoping "$\cong$" might get me past that, but I see your point. We need to get subtler than this.
–
Michael HardyOct 3 '12 at 15:42