There are many reasons why a well-designed test might fail. For example, seasonal effects might be ignored, the dataset might be too small or the marketplace might change during the test.
However, a very common error in test design is not accounting for volatility – fluctuations in performance due to unpredictable events in the marketplace...Discuss