2.4.2 Centering in feature space

So far, we have assumed that {$\varphi$($\bf x_{i}^{}$)} has zero mean, which is usually not fulfilled. Therefore, the formalism needs to be adjusted (Schölkopf et al., 1998b). The following set of points will be centered:

$$\tilde{\boldsymbol} \varphi \bf x_{i}^{} \varphi \bf x_{i}^{} {\frac{{1}}{{n}}} \sum_{{r=1}}^{n} \varphi \bf x_{r}^{}$$ (2.31)

The above analysis holds if the covariance matrix is computed from $\tilde{\boldsymbol}$$\varphi$($\bf x_{i}^{}$). Thus, the kernel matrix K_ij = $\varphi$($\bf x_{i}^{}$)^T$\varphi$($\bf x_{j}^{}$) needs to be replaced by $\tilde{K}_{{ij}}^{}$ = $\tilde{\boldsymbol}$$\varphi$($\bf x_{i}^{}$)^T$\tilde{\boldsymbol}$$\varphi$($\bf x_{j}^{}$). Using (2.31), $\tilde{K}$ can be written as,

$$\tilde{K}_{{ij}}^{} \varphi \bf x_{i}^{} \varphi \bf x_{j}^{} {\frac{{1}}{{n}}} \sum_{{r=1}}^{n} \varphi \bf x_{i}^{} \varphi \bf x_{r}^{} {\frac{{1}}{{n}}} \sum_{{r=1}}^{n} \varphi \bf x_{r}^{} \varphi \bf x_{j}^{}$$ =

$${\frac{{1}}{{n^2}}} \sum_{{r,s=1}}^{n} \varphi \bf x_{r}^{} \varphi \bf x_{{ s}}^{}$$ +

$${\frac{{1}}{{n}}} \sum_{{r=1}}^{n} {\frac{{1}}{{n}}} \sum_{{r=1}}^{n} {\frac{{1}}{{n^2}}} \sum_{{r,s=1}}^{n}$$ = (2.32)

Therefore, we can evaluate the kernel matrix for the centered data using the known matrix $\bf K$. For the remainder of this thesis, I denote with $\alpha$ the eigenvectors of $\tilde{{\bf K}}$ instead of $\bf K$, and they are normalized according to (2.29) using the eigenvalues of $\tilde{{\bf K}}$. The principal components are $\tilde{{\bf w}}$ = $\sum_{{i=1}}^{n}$$\alpha_{i}^{}$$\tilde{\boldsymbol}$$\varphi$($\bf x_{i}^{}$).