D-X2 · A damaged picture finds a memory

Hopfield Associative Memory

How can a network recover a whole picture from a damaged cue? The stored pictures set connections that favor familiar combinations of cells. Starting from a damaged cue, each cell follows the influence of the others and the network energy falls. The final pattern can recover a memory or settle in a competing local minimum.

Scratch over a stored picture and let a network repair it one cell at a time. Add competing memories or scramble most of the cue to see when recall finds a different resting state.

Error against selected picture $0$

Network energy $0$

Cells changed in this sweep $0$

Recall state $0$

Try a different condition

Damaged cells

22

\text{percent}

Stored pictures

2

P

Picture to recall

Drag across the picture to damage it. Release to replay its recall.

Model and sampling

The network has one hundred and forty-four binary units with symmetric Hebbian weights, zero self-coupling and zero thresholds. Units update asynchronously in a shuffled order; a zero local field leaves a unit unchanged. Each successful update lowers the network energy. The three hand-drawn patterns are correlated, so a stored pattern need not remain an exact fixed point when all three are stored. The energy has other local minima, including inverted and mixed patterns; convergence alone does not guarantee the intended memory. There is no learning during recall, no thermal noise and no fitted model of a biological brain. Slowed updates make individual changes visible. The energy curve is a model diagnostic, not an electrical energy measurement.

Energy falls during this recall

Pictures stored in the weights

A recorded asynchronous recall is replayed. Cyan: computed energy; amber: current frame.

Scientific sources

Start with the essentials

Focus question: How can a network recover a whole picture from a damaged cue?
One-sentence intuition: Symmetric connections let asynchronous updates descend an energy function toward a stable pattern.

Core mathematical model

Pictures set the connections

w_{ij}=\frac{1}{N}\sum_{\mu=1}^{P}\xi_i^{\mu}\xi_j^{\mu},\qquad w_{ii}=0

A stored picture assigns one binary value to each cell. A pair that often agrees gets a positive connection; a pair that often differs gets a negative one. The count of cells sets the normalization.

A cell follows its local field

s_i\leftarrow\operatorname{sgn}\!\left(\sum_j w_{ij}s_j\right)

One unit updates at a time, in a shuffled order. It follows the weighted influence of the other units. A zero sum leaves it unchanged.

Energy decreases when a unit changes

E=-\frac12\sum_{i,j}w_{ij}s_i s_j,\qquad\Delta E=-(s_i^{\mathrm{new}}-s_i^{\mathrm{old}})\sum_j w_{ij}s_j\leq0

Symmetric weights and zero self-coupling make each allowed change lower the energy. The units eventually reach a state that no single update can improve.

Common difficulties

A minimum can be the wrong memory

Typical misconceptionOnce the energy stops falling, the intended picture must have been recovered.

Better mental modelThe energy can have many local minima. Correlated stored pictures can shift or merge them, and every picture has an inverted partner in this zero-threshold model.

Run the experiment

01
Watch the repair
Choose Repair a scratched heart and follow the amber marker on the energy curve.
What to observe: The damaged cells change as the energy falls. The recorded recall repeats so you can follow the same repair.
02
Give it a different clue
Choose Recall lightning, then drag over part of the picture.
What to observe: Release the pointer to start a new recall from your drawing. The connections stay fixed during recall.
03
Let memories compete
Choose Competing pictures and inspect the final error.
What to observe: The correlated patterns can pull the network into a resting state that differs from the selected picture.
04
Try the inverted picture
Choose An inverted memory.
What to observe: The network can settle on the inverted picture, which has the same energy as its original.

Core mathematical model

Pictures set the connections

A cell follows its local field

Energy decreases when a unit changes

Common difficulties

A minimum can be the wrong memory

Run the experiment

Watch the repair

Give it a different clue

Let memories compete

Try the inverted picture