The generalized Hebbian algorithm (GHA), also known in the literature as Sanger's rule, is a linear feedforward neural network for unsupervised learning with applications primarily in principal components analysis. First defined in 1989,^[1] it is similar to Oja's rule in its formulation and stability, except it can be applied to networks with multiple outputs. The name originates because of the similarity between the algorithm and a hypothesis made by Donald Hebb^[2] about the way in which synaptic strengths in the brain are modified in response to experience, i.e., that changes are proportional to the correlation between the firing of pre- and post-synaptic neurons.^[3]

The GHA combines Oja's rule with the Gram-Schmidt process to produce a learning rule of the form

${\displaystyle \,\Delta w_{ij}~=~\eta \left(y_{i}x_{j}-y_{i}\sum _{k=1}^{i}w_{kj}y_{k}\right)}$,^[4]

where w_ij defines the synaptic weight or connection strength between the jth input and ith output neurons, x and y are the input and output vectors, respectively, and η is the learning rate parameter.

The GHA is used in applications where a self-organizing map is necessary, or where a feature or principal components analysis can be used. Examples of such cases include artificial intelligence and speech and image processing.

Its importance comes from the fact that learning is a single-layer process—that is, a synaptic weight changes only depending on the response of the inputs and outputs of that layer, thus avoiding the multi-layer dependence associated with the backpropagation algorithm. It also has a simple and predictable trade-off between learning speed and accuracy of convergence as set by the learning rate parameter η.^[5]

• Hebbian learning
• Factor analysis
• Contrastive Hebbian learning
• Oja's rule