Kernel Conjugate Gradient

We propose a novel variant of conjugate gradient based on the
Reproducing Kernel Hilbert Space (RKHS) inner product. An analysis of
the algorithm suggests it enjoys better performance properties than
standard iterative methods when applied to learning kernel machines.
Experimental results for both classification and regression bear out the
theoretical implications. We further address the dominant cost of the
algorithm by reducing the complexity of RKHS function evaluations and
inner products through the use of space-partitioning tree
data-structures.