2.2.3 Deterministic annealing

Deterministic annealing extends soft-clustering to an annealing process (Rose et al., 1990; Rose, 1998). For each temperature value, the algorithm iterates between the calculation of all P(j|$\bf x_{i}^{}$) and the update of the code-book vectors (in batch mode), until convergence is reached. The annealing starts with a high temperature (low $\beta$). Here, all code-book vectors converge to the center of the pattern distribution (independent of their initial positions). Below a critical temperature the vectors start to split. Further decreasing the temperature (increasing $\beta$) leads to more splittings until all code-book vectors are separate. The annealing can therefore avoid (if it is sufficiently slow) the convergence to local minima of (2.12). Deterministic annealing is originally formulated as batch method, but also an on-line version exists (Qiu et al., 1994).