I think the theory isn't that the galaxy is large enough for anything that could have happened to have happened, it is that the universe is large enough. The galaxy is a pretty darn big place, but it pales in comparison to the entire universe. If someone somewhere in the universe invented a warp drive, it could still take them so long to get here that we would never know about it.

And even then, the theory only works if there really is only X possible things to happen and the universe is larger than X. I'm inclined to believe that unless the universe is infinitely large, there are things that are possible that haven't happened.