5.3.2.4 Recall on synthetic data

To test recall, the sine-wave distribution was chosen, which included noise. Thus, this task could also demonstrate the ability to generalize. 15 principal components were extracted (explaining 84.8% of the total variance). The output follows the shape of the sine wave, and it does not get distorted by the outliers (figure 5.6). Here, the right balance of the number of principal components was important. With too many (q=40) extracted components, also the noise was included in the potential field.

The mixture of local PCA could also restore the input-output relationship despite the noise (figure 5.7). Here, all noise points were assigned to one big ellipse (in the center of the image). This ellipse did not disturb the recall because the algorithm punishes large ellipsoidal volumes (section 4.2).

Figure 5.6: Recall in the kernel PCA model trained on a sine wave surrounded by noise (training patterns can be seen in Figure 5.7). The black points show input-output relations. The input is on the x-axis. The right image is a magnification of the region marked with a square in the left image.

Figure 5.7: Recall in the mixture of PCA model (10 PCAs, with two principal components each). The axes' lengths of the ellipses equal the square root of the eigenvalues. Gray points are the training data. The black lines show input-output relations. The input is on the x-axis.