2.4.3 Common kernel functions

The kernel function needs to be a scalar product in some feature space. A sufficient condition is that the kernel matrix is positive semidefinite (Schölkopf et al., 1998b; Schölkopf and Smola, 2002, p. 44). Some common kernel functions that fulfill this condition are the polynomial kernel,

$$\bf x \bf y \bf x^{T}_{} \bf y$$ (2.33)

with a constant integer d, the Gaussian kernel,

$$\bf x \bf y \left(\vphantom{-\frac{\Vert{\bf x}-{\bf y}\Vert^2}{2\sigma^2}}\right. {\frac{{\Vert{\bf x}-{\bf y}\Vert^2}}{{2\sigma^2}}} \left.\vphantom{-\frac{\Vert{\bf x}-{\bf y}\Vert^2}{2\sigma^2}}\right)$$ (2.34)

with a constant $\sigma$ > 0, and the inverse multiquadric kernel,

$$\bf x \bf y {\frac{{1}}{{\sqrt{\Vert{\bf x}-{\bf y}\Vert^2+c}}}}$$ (2.35)

with a constant c > 0 (Schölkopf and Smola, 2002, p. 54). The last two functions result in a kernel matrix with full rank (Micchelli, 1986). That is, all eigenvectors are linearly independent. Thus, the dimensionality of the feature space is not restricted (it is infinite).