With R version 2.15.1 (2012-06-22) on Windows, I computed the p=1.e-20 quantile of the geometric distribution with parameter prob=0.1.
> qgeom(1.e-20,0.1)
[1] -1
But this is not possible, since X=0,1,2,...
May be the 1.e-7 in the code :
http://svn.r-project.org/R/trunk/src/nmath/qgeom.c
may be producing this error.