A memory can be called up by part of its content
You recognise a familiar face even when a scarf hides its lower half. A computer file usually asks for an address or a name; recognition starts from part of the thing itself. John Hopfield’s 1982 paper offered a small mathematical system with this kind of content-addressable memory. Give it enough of a stored pattern and its changing internal state can recover the rest.
Picture a square of lamps, each either lit or dark. A pattern might be a letter made from the lit lamps. Every lamp receives signals from the others. The system has no single lamp in charge of recognising the letter. Recognition appears through the way the lamps collectively change.
Store the picture in the connections
To remember a pattern, choose connections that favour its arrangement. If two lamps have the same state in a stored picture, their connection can encourage agreement. If their states differ, it can encourage opposite states. Repeating this contribution over several pictures puts information about all of them into the same connections. This familiar prescription is called Hebbian storage.
Now change some lamps in one picture and let the network run. Select one lamp, add the weighted signals arriving from the others, and set it to the state favoured by that total. Then select another lamp using the updated arrangement. Repeated local decisions can repair the whole picture because each correct portion supplies clues to the damaged portion.
An energy function gives every update a direction
A useful way to follow this motion assigns an energy to every complete arrangement of lamps. A pair contributes less energy when it satisfies the preference in its connection. With symmetric connections, no self-connections and one unit updated at a time, an update that changes a unit lowers this energy. A tie can leave the unit unchanged.
This provides a direction without an instruction sheet for each damaged image. The network goes downhill until no single lamp can improve the arrangement. A stable arrangement is called an attractor: nearby starting patterns can arrive there too. In the landscape picture, a stored memory is a valley and the repair process descends into it.
The energy is a mathematical function of the model’s states and connections. Its numerical decrease should not be read as a measurement of the electricity used by a computer or the metabolism of a brain. The construction tells us why the specified update rule settles.
Sources: [2]
A lower energy can still be the wrong memory
A landscape can have several valleys. A badly damaged cue may fall into a different stored memory, or into an unwanted mixture created by the shared connections. Similar memories compete particularly strongly. Reaching a stable state is therefore different from reaching the intended picture.
A good test records both the energy and the mismatch with the target. If energy falls while the mismatch remains large, the update rule may be working exactly as designed: it has found a different attractor. Adding memories also changes the connections, so the set of recoverable cues changes. How many pictures can be recovered depends on their similarity, the number of units and the storage rule.
The collective idea survived changes in the units
In 1984, Hopfield studied units whose outputs vary continuously instead of switching only between two levels. With the appropriate symmetric structure, an energy function still organised the network’s motion. That linked the computational picture more closely to circuits with amplifiers, resistors and capacitors.
The 2024 Nobel Prize recognised Hopfield and Geoffrey Hinton for foundations of learning with artificial neural networks. The memorable physical idea is already visible in the lamp board: useful computation can emerge when many simple components settle together. Reconstructing a picture becomes a collective relaxation process.