This submodule contains functions that approximate the feature mappings that
correspond to certain kernels, as they are used for example in support vector
machines (see Support Vector Machines).
The following feature functions perform non-linear transformations of the
input, which can serve as a basis for linear classification or other
algorithms.

The advantage of using approximate explicit feature maps compared to the
kernel trick,
which makes use of feature maps implicitly, is that explicit mappings
can be better suited for online learning and can significantly reduce the cost
of learning with very large datasets.
Standard kernelized SVMs do not scale well to large datasets, but using an
approximate kernel map it is possible to use much more efficient linear SVMs.
In particular, the combination of kernel map approximations with
SGDClassifier can make non-linear learning on large datasets possible.

Since there has not been much empirical work using approximate embeddings, it
is advisable to compare results against exact kernel methods when possible.

The Nystroem method, as implemented in Nystroem is a general method
for low-rank approximations of kernels. It achieves this by essentially subsampling
the data on which the kernel is evaluated.
By default Nystroem uses the rbf kernel, but it can use any
kernel function or a precomputed kernel matrix.
The number of samples used - which is also the dimensionality of the features computed -
is given by the parameter n_components.

The RBFSampler constructs an approximate mapping for the radial basis
function kernel, also known as Random Kitchen Sinks[RR2007]. This
transformation can be used to explicitly model a kernel map, prior to applying
a linear algorithm, for example a linear SVM:

The mapping relies on a Monte Carlo approximation to the
kernel values. The fit function performs the Monte Carlo sampling, whereas
the transform method performs the mapping of the data. Because of the
inherent randomness of the process, results may vary between different calls to
the fit function.

The fit function takes two arguments:
n_components, which is the target dimensionality of the feature transform,
and gamma, the parameter of the RBF-kernel. A higher n_components will
result in a better approximation of the kernel and will yield results more
similar to those produced by a kernel SVM. Note that “fitting” the feature
function does not actually depend on the data given to the fit function.
Only the dimensionality of the data is used.
Details on the method can be found in [RR2007].

For a given value of n_componentsRBFSampler is often less accurate
as Nystroem. RBFSampler is cheaper to compute, though, making
use of larger feature spaces more efficient.

The additive chi squared kernel is a kernel on histograms, often used in computer vision.

The additive chi squared kernel as used here is given by

This is not exactly the same as sklearn.metrics.additive_chi2_kernel.
The authors of [VZ2010] prefer the version above as it is always positive
definite.
Since the kernel is additive, it is possible to treat all components
separately for embedding. This makes it possible to sample
the Fourier transform in regular intervals, instead of approximating
using Monte Carlo sampling.

The class AdditiveChi2Sampler implements this component wise
deterministic sampling. Each component is sampled times, yielding
dimensions per input dimension (the multiple of two stems
from the real and complex part of the Fourier transform).
In the literature, is usually chosen to be 1 or 2, transforming
the dataset to size n_samples*5*n_features (in the case of ).

The approximate feature map provided by AdditiveChi2Sampler can be combined
with the approximate feature map provided by RBFSampler to yield an approximate
feature map for the exponentiated chi squared kernel.
See the [VZ2010] for details and [VVZ2010] for combination with the RBFSampler.

It has properties that are similar to the exponentiated chi squared kernel
often used in computer vision, but allows for a simple Monte Carlo
approximation of the feature map.

The usage of the SkewedChi2Sampler is the same as the usage described
above for the RBFSampler. The only difference is in the free
parameter, that is called .
For a motivation for this mapping and the mathematical details see [LS2010].

Kernel methods like support vector machines or kernelized
PCA rely on a property of reproducing kernel Hilbert spaces.
For any positive definite kernel function (a so called Mercer kernel),
it is guaranteed that there exists a mapping
into a Hilbert space , such that

Where denotes the inner product in the
Hilbert space.

If an algorithm, such as a linear support vector machine or PCA,
relies only on the scalar product of data points , one may use
the value of , which corresponds to applying the algorithm
to the mapped data points .
The advantage of using is that the mapping never has
to be calculated explicitly, allowing for arbitrary large
features (even infinite).

One drawback of kernel methods is, that it might be necessary
to store many kernel values during optimization.
If a kernelized classifier is applied to new data ,
needs to be computed to make predictions,
possibly for many different in the training set.

The classes in this submodule allow to approximate the embedding
, thereby working explicitly with the representations
, which obviates the need to apply the kernel
or store training examples.